bash：「sed-replace」関数を作成しようとし、任意の文字入力に対して堅牢（拡張、置換など）

Question

引数として任意の入力を使用して「sed-replace」関数を作成しようとしていますが、うまく機能しません。まず、入力ファイル（簡略化されたファイル）を表示して、問題を説明します。

$ cat /tmp/makefileTest
#(TEST CASE 1) bla bla line 1, relatively simple:
CFLAGS += -Wunused # this is a comment

#(TEST CASE 2) bla bla line 4, uses some expansion
cp $(OBJ_DIR)/$(EXE_NAME) /tmp/somewhere

#(TEST CASE 3) bla bla line 7, here is a complicated line ending in weird characters:
cd $(SOME_UTIL_BIN); ./somecommand $(BUILD_DIRECTORY_PATH)/$(OBJ_DIR)/\$\^

だから私はこの入力ファイルにいくつかのカスタムのものを適用したい（毎回「git pull」する）、つまり私はクリーンなコピーをチェックアウトするプルスクリプトを持っていて、そして必要な修正を行うべきスクリプトを持っている。最新バージョンの。以下のメソッドは、上記のテストケース1とテストケース2で使用されますが、問題は、多くの手動作業が伴うため、「面倒なメソッド」と呼んでいます。入力行を受け取り、変更すると、sed-functionが必要な置換を実行するはずです。

$ cat /tmp/testFunctionTedious.sh 
#!/usr/bin/env bash

# The old, tedious method, first defining input-file with test-cases:
inFile='/tmp/makefileTest'    

#    ----==== TEST-CASE 1 FROM THE INPUT FILE ====----
charsFoundFromGrep=$(grep -in 'CFLAGS += -Wunused # this is a comment' "$inFile" | wc -c)
if [ "$charsFoundFromGrep" = "0" ]; then
    echo "Custom makefile modification (CFLAGS += -Wunused # this is a comment) NOT found, doing nothing!"
Elif [ "$charsFoundFromGrep" = "41" ]; then
    echo "Custom makefile modification (CFLAGS += -Wunused # this is a comment) found and will be applied..."
    sed -i 's/CFLAGS += -Wunused # this is a comment/CFLAGS += -Wall # here I changed something/g' "$inFile"
else
    echo "ERROR: Unhandled custom makefile modification (CFLAGS += -Wunused # this is a comment), please fix..."
    exit 1
fi

#    ----==== TEST-CASE 2 FROM THE INPUT FILE ====----
# Notice below that I need to escape $(OBJ_DIR) and $(EXE_NAME), not to
#  mention the two forward slashes in the "sed"-line, it's definately not just "plug-and-play":
charsFoundFromGrep=$(grep -in 'cp $(OBJ_DIR)/$(EXE_NAME)' "$inFile" | wc -c)
if [ "$charsFoundFromGrep" = "0" ]; then
    echo "Custom makefile modification (cp \$(OBJ_DIR)/\$(EXE_NAME)) NOT found, doing nothing!"
Elif [ "$charsFoundFromGrep" = "43" ]; then
    echo "Custom makefile modification (cp \$(OBJ_DIR)/\$(EXE_NAME)) found and will be applied..."
    sed -i 's/cp \$(OBJ_DIR)\/\$(EXE_NAME)/cp \$(OBJ_DIR)\/\$(EXE_NAME_NEW)/g' "$inFile"
else
    echo "ERROR: Unhandled custom makefile modification (cp $(OBJ_DIR)/$(EXE_NAME)), please fix..."
    exit 1
fi

私はより良い/よりスマートな方法を作ることを学び、bash変数の展開/置換と特殊文字の処理について学びます。物事をより効率的にするために、次のスクリプトを作成しようとしましたが、ここで物事が複雑になりすぎます。

$ cat /tmp/testFunction.sh 
#!/usr/bin/env bash

# The method I struggle with and ask for help with, first defining input-file with test-cases
inFile='/tmp/makefileTest'

# *** Defining a sedReplace-function below ***
#   First arg: Search (input) string
#   Second arg: Replacement (output) string
#   Third arg: Expected number of characters using 'grep -in "$1" "$inFile" | wc -c)',
#      this is just to ensure the line I'm going to run sed on didn't change, otherwise
#      output and error involving the input message (hence the string comparison that
#      relates argument 3 with grep from argument 1 (the input string).
sedReplace(){
    # sed -i 's/$1/$2/g' "$inFile"
    charsFoundFromGrep=$(grep -in "$1" "$inFile" | wc -c)
    if [ "$3" == "$charsFoundFromGrep" ]; then
        # Getting the line below right is REALLY difficult for me!
        execLine="sed -i 's/$1/$2/g' \"$inFile\""
        # Printing the line, so I can see it before executing the line:
        echo "$execLine"
        # Executing the line if ok (disabled as it doesn't work at the moment):
        #$($execLine)
    else
        echo "ERROR: Unhandled custom makefile modification (expected: $1)), please fix..."
        exit 1
    fi
}

# And below the function is used (1st arg is input, 2nd arg is sed-
#   output and 3rd arg is grep comparison Word count):

#    ----==== TEST-CASE 1 FROM THE INPUT FILE ====----
sedReplace 'CFLAGS += -Wunused # this is a comment' 'CFLAGS += -Wall # here I changed something' 41

#    ----==== TEST-CASE 2 FROM THE INPUT FILE ====----
#sedReplace 'cp $(OBJ_DIR)/$(EXE_NAME)' 'cp $(OBJ_DIR)/$(EXE_NAME_NEW)' 43

#    ----==== TEST-CASE 3 FROM THE INPUT FILE ====----
# Once the above 2 cases work, here's the last test-case to try the sedReplace function on (the hardest, I imagine):
# And here grep don't work, due to the special characters
#sedReplace 'cd $(SOME_UTIL_BIN); ./somecommand $(BUILD_DIRECTORY_PATH)/$(OBJ_DIR)/\$\^' 'cd $(SOME_UTIL_BIN); ./someOTHERcommand $(BUILD_DIRECTORY_SOMETHING_ELSE)/$(OBJ_DIR)/\$\^'

最後のスクリプトが機能しないことが簡単にわかります。私はグーグルを試みました、そして同様の問題について多くを試みました、それを見つけることができません。 sed関数を終了する方法がわかりません。それが私が助けを求めていることです。資格のある関心のある人は、ここに示すようにスクリプトと入力ファイルを正確に実行できるはずです。誰かが問題を解決できるかどうか楽しみにしています。

Answer 1

これはスクリプトの修正バージョンで、最初のテストケースでのみ機能します。

#!/usr/bin/env bash

inFile='/tmp/makefileTest'

sedReplace(){
    charsFoundFromGrep="$(grep -in "$1" "$inFile" | wc -c)"

    if [ "$3" == "$charsFoundFromGrep" ]; then
        # 1. The single quotes inside double quotes are threat as regular characters
        # 2. During the assignment, the variables $1, $2 and $inFile will be expanded
        # 3. The variable $execLine will have the following value:
        #    sed -i 's/CFLAGS += -Wunused # this is a comment/CFLAGS += -Wall # here I changed something/g' '/tmp/makefileTest'
        execLine="sed -i 's/$1/$2/g' '$inFile'"

        # We need 'eval' to convert the variable to a command in this case,
        # because the value of the variable contains spaces, quotes, slashes, etc.
        eval "$execLine"
    else
        echo "ERROR: Unhandled custom makefile modification (expected: $1)), please fix..."
        exit 1
    fi
}

sedReplace 'CFLAGS += -Wunused # this is a comment' 'CFLAGS += -Wall # here I changed something' '41'

上記の例ではコマンドevalを使用していますが、最近、その使用法、賛否両論を this answer の最後の部分と関連するコメント内で破棄しています。可能であればevalの使用を避けることをお勧めします。そのため、次の提案をします。

#!/usr/bin/env bash

sedReplace(){
    # 1. Note we do not need to store the output of the command substitution $()
    #    into a variable in order to use it within a test condition.
    # 2. Here is used the bash's double square brackets test [[, so
    #    we do not need to quote the variable before the condition.
    #    If the statement after the condition is not quoted the (expanded) value
    #    will be threat as regexp. Currently it is treated as string.
    if [[ $3 == "$(grep -in "$1" "$inFile" | wc -c)" ]]
    then
        # 1. Note the double quotes here.
        # 2. The sed's /g flag is removed, because, IMO, we don't need it in this case at all.
        sed -i "s/$1/$2/" "$inFile"

    else
        echo "ERROR: Unhandled custom makefile modification (expected: $1)), please fix..."
        exit 1
    fi
}

# here are used double quotes in case the value comes from a variable in the further versions
inFile="/tmp/makefileTest"

sedReplace 'CFLAGS += -Wunused # this is a comment' 'CFLAGS += -Wall # here I changed something' '41'

上記の例は、最初のテストケースでのみ機能します。残りのテストケースでは、固定文字列（ references ）としてパターンを脅かすためにgrep -Fを使用する必要があります。また、sedで使用する前に、検索された文字列/パターン内の一部の文字を置き換える必要があります（おそらくよりエレガントな解決策がありますが、見つかりませんでした）。 3番目に必要なことは、sedの区切り文字を/から、文字列内で使用されていない任意の文字に変更することです。以下の例では:を使用しています。

さらに、入力ファイルの名前も位置パラメータとして含め、読みやすい順序で位置変数をローカル変数に割り当てます。

これが最終的な解決策です（実際の変更を行うには、-iのコメントを外してください）。

#!/usr/bin/env bash
sedReplace() {
    local the_searched_string="$1" the_replacement="$2" the_lenght="$3" the_file="$4"

    if [[ $the_lenght == "$(grep -inF "$the_searched_string" "$the_file" | wc -c)" ]]
    then
        the_searched_string="$(sed -r 's/(\^|\$|\\)/./g' <<< "$the_searched_string")" # replace some special characters within the searched string by any char '.'
        sed "s:$the_searched_string:$the_replacement:" "$the_file" #-i
    else
        echo "ERROR: Unhandled custom makefile modification (expected: ${the_searched_string})..."
        exit 1
    fi
}

inFile="/tmp/makefileTest" 

# Test all cases:
echo -e '\n\n# --- Test case 1 -----'
the_string='CFLAGS += -Wunused # this is a comment'
sedReplace "$the_string" \
           'CFLAGS += -Wall # something is changed' \
           "$(wc -c <<< '2:'"$the_string")" \
           "$inFile"
echo -e '\n\n# --- Test case 2 -----'
the_string='cp $(OBJ_DIR)/$(EXE_NAME) /tmp/somewhere'
sedReplace "$the_string" \
           "${the_string} # something is changed" \
           "$(wc -c <<< '5:'"$the_string")" \
            "$inFile"
echo -e '\n\n# --- Test case 3 -----'
the_string='cd $(SOME_UTIL_BIN); ./somecommand $(BUILD_DIRECTORY_PATH)/$(OBJ_DIR)/\$\^'
sedReplace "$the_string" \
           "${the_string} # something is changed" \
           "$(wc -c <<< '8:'"$the_string")" \
           "$inFile"

おそらく、必要に応じて、sha256sumの代わりにwc -c（または他のチェックサムツール）を使用して、より厳密な行チェックを行うことができます。

#!/usr/bin/env bash
sedReplace() {
    local the_searched_string="$1" the_replacement="$2" the_lenght="$3" the_file="$4"

    if [[ $the_lenght == "$(grep -inF "$the_searched_string" "$the_file" | sha256sum)" ]]
    then
        the_searched_string="$(sed -r 's/(\^|\$|\\)/./g' <<< "$the_searched_string")" # replace some special characters within the searched string by any char '.'
        sed "s:$the_searched_string:$the_replacement:" "$the_file" #-i
    else
        echo "ERROR: Unhandled custom makefile modification (expected: ${the_searched_string})..."
        exit 1
    fi
}

inFile="/tmp/makefileTest"

# Test all cases:
echo -e '\n\n# --- Test case 1 -----'
the_string='CFLAGS += -Wunused # this is a comment'; the_line='2'
sedReplace "$the_string" \
           'CFLAGS += -Wall # something is changed' \
           "$(sha256sum <<< "${the_line}:${the_string}")" \
           "$inFile"
echo -e '\n\n# --- Test case 2 -----'
the_string='cp $(OBJ_DIR)/$(EXE_NAME) /tmp/somewhere'; the_line='5'
sedReplace "$the_string" \
           "${the_string} # something is changed" \
           "$(sha256sum <<< "${the_line}:${the_string}")" \
            "$inFile"
echo -e '\n\n# --- Test case 3 -----'
the_string='cd $(SOME_UTIL_BIN); ./somecommand $(BUILD_DIRECTORY_PATH)/$(OBJ_DIR)/\$\^'; the_line='8'
sedReplace "$the_string" \
           "${the_string} # something is changed" \
           "$(sha256sum <<< "${the_line}:${the_string}")" \
           "$inFile"

更新：検索された文字列は本当に複雑なので、デリミタを動的に計算する方法の例を次に示します（4番目のテストケースに注意してください）。

#!/usr/bin/env bash
sedReplace() {
    local the_searched_string="$1" the_replacement="$2" the_lenght="$3" the_file="$4" d="$5"

    if [[ $the_lenght == "$(grep -inF "$the_searched_string" "$the_file" | wc -c)" ]]
    then
        the_searched_string="$(sed -r 's/(\^|\$|\\)/./g' <<< "$the_searched_string")" # replace some special characters within the searched string by any char '.'
        the_expression="s${d}${the_searched_string}${d}${the_replacement}${d}"
        #echo "$the_expression"
        sed "$the_expression" "$the_file" #-i
    else
        echo "ERROR: Unhandled custom makefile modification (expected: ${the_searched_string})..."
        exit 1
    fi
}

get_delimiter() {
    unset delimiter

    for d in '/' ':' '#' '_' '|' '@'
    do
        if ! grep -qoF "$d" <<< "$the_string"
        then
            delimiter="$d"
            break
        fi
    done

    if [[ -z $delimiter ]]
    then
        echo 'There is not appropriate delimiter for the string:'
        echo "$the_string"
        exit 1
    fi
}

inFile="/tmp/makefileTest"

# Test all cases:
echo -e '\n\n# --- Test case 1 -----'
the_string='CFLAGS += -Wunused # this is a comment'
get_delimiter
sedReplace "$the_string" \
           'CFLAGS += -Wall # something is changed' \
           "$(wc -c <<< '2:'"$the_string")" \
           "$inFile" "$delimiter"

echo -e '\n\n# --- Test case 2 -----'
the_string='cp $(OBJ_DIR)/$(EXE_NAME) /tmp/somewhere'
get_delimiter
sedReplace "$the_string" \
           "${the_string} # something is changed" \
           "$(wc -c <<< '5:'"$the_string")" \
            "$inFile" "$delimiter"

echo -e '\n\n# --- Test case 3 -----'
the_string='cd $(SOME_UTIL_BIN); ./somecommand $(BUILD_DIRECTORY_PATH)/$(OBJ_DIR)/\$\^'
get_delimiter
sedReplace "$the_string" \
           "${the_string} # something is changed" \
           "$(wc -c <<< '8:'"$the_string")" \
           "$inFile" "$delimiter"

echo -e '\n\n# --- Test case 4 -----'
the_string='/:#_|@'
get_delimiter
sedReplace "$the_string" \
           "${the_string} # something is changed" \
           "$(wc -c <<< '8:'"$the_string")" \
           "$inFile" "$delimiter"

上記の別のバージョンは次のとおりです。

#!/usr/bin/env bash
sedReplace() {
    local d the_searched_string="$1" the_replacement="$2" the_lenght="$3" the_file="$4"
    # the content of this function could be placed here, thus we will have only one function
    get_delimiter "$the_searched_string"

    if [[ $the_lenght == "$(grep -inF "$the_searched_string" "$the_file" | wc -c)" ]]
    then
        the_searched_string="$(sed -r 's/(\^|\$|\\)/./g' <<< "$the_searched_string")"
        the_expression="s${d}${the_searched_string}${d}${the_replacement}${d}"
        sed "$the_expression" "$the_file" #-i
    else
        echo "ERROR: Unhandled custom makefile modification (expected: ${the_searched_string})..."
        exit 1
    fi
}

get_delimiter() {
    # define an array of possible delimiters, it could be defined outside the function
    delimiters=('/' ':' '#' '_' '|' '@' '%')

    for delimiter in ${delimiters[@]}
    do
        if ! grep -qoF "$delimiter" <<< "$1"
        then
            d="$delimiter"
            break
        fi
    done

    if [[ -z $d ]]
    then
        echo "ERROR: There is not appropriate delimiter for the string: ${1}"
        exit 1
    fi
}

inFile="/tmp/makefileTest"

# Test all cases:
echo -e '\n\n# --- Test case 1 -----'
the_string='CFLAGS += -Wunused # this is a comment'
sedReplace "$the_string" \
           'CFLAGS += -Wall # something is changed' \
           "$(wc -c <<< '2:'"$the_string")" \
           "$inFile"

echo -e '\n\n# --- Test case 2 -----'
the_string='cp $(OBJ_DIR)/$(EXE_NAME) /tmp/somewhere'
sedReplace "$the_string" \
           "${the_string} # something is changed" \
           "$(wc -c <<< '5:'"$the_string")" \
            "$inFile"

echo -e '\n\n# --- Test case 3 -----'
the_string='cd $(SOME_UTIL_BIN); ./somecommand $(BUILD_DIRECTORY_PATH)/$(OBJ_DIR)/\$\^'
sedReplace "$the_string" \
           "${the_string} # something is changed" \
           "$(wc -c <<< '8:'"$the_string")" \
           "$inFile"

echo -e '\n\n# --- Test case 4 -----'
the_string='/:#_|@%'
sedReplace "$the_string" \
           "${the_string} # something is changed" \
           "$(wc -c <<< '8:'"$the_string")" \
           "$inFile"

Answer 2

これは必ずしも質問に直接回答するとは限りませんが、sedを簡略化する他の同様の試みを確認することもできます。そのような例の1つは sd です。