How to search & replace arbitrary literal strings in sed and awk (and perl)

The quotemeta, which implements \Q, absolutely does what you ask for

all ASCII characters not matching /[A-Za-z_0-9]/ will be preceded by a backslash

Since this is presumably in a shell script, the problem really is of how and when shell variables get interpolated and what the Perl program ends up seeing.

The best way is to avoid working out that interpolation mess and instead properly pass those shell variables to the Perl one-liner. This can be done in several ways; see this post for details.

Either pass the shell variables simply as arguments

#!/bin/bash

# define $target

perl -pe"BEGIN { $patt = shift }; s{\Q$patt}{$replacement}g" "$target" file.txt

where the needed arguments are removed from @ARGV and utilized in a BEGIN block, so before the runtime; then file.txt gets processed. There is no need for \E in the regex here.

Or, use the -s switch, which enables command-line switches for the program

# define $target, etc

perl -s -pe"s{\Q$patt}{$replacement}g" -- -patt="$target" file.txt

The -- is needed to mark the start of arguments, and switches must come before filenames.

Finally, you can also export the shell variables, which can then be used in the Perl script via %ENV; but in general I'd rather recommend either of the above two approaches.


A full example

#!/bin/bash
# Last modified: 2019 Jan 06 (22:15)

target="/{"
replacement="&"

echo "Replace $target with $replacement"

perl -wE'
    BEGIN { $p = shift; $r = shift }; 
    $_=q(ah/{yes); s/\Q$p/$r/; say
' "$target" "$replacement"

This prints

Replace /{ with &
ah&yes

where I've used characters mentioned in a comment.

The other way

#!/bin/bash
# Last modified: 2019 Jan 06 (22:05)

target="/{"
replacement="&"

echo "Replace $target with $replacement"

perl -s -wE'$_ = q(ah/{yes); s/\Q$patt/$repl/; say' \
    -- -patt="$target" -repl="$replacement"

where code is broken over lines for readability here (and thus needs the \). Same printout.


Me again!

Here's a simpler way using xxd(1):

t=$( echo -n "$target" | xxd -p | tr -d '\n')
r=$( echo -n "$replacement" | xxd -p | tr -d '\n')
xxd -p file.txt | sed "s/$t/$r/g" | xxd -p -r

... so we're hex-encoding the original text with xxd(1) and doing search-replacement using hex-encoded search strings. Finally we hex-decode the result.

EDIT: I forgot to remove \n from the xxd output (| tr -d '\n') so that patterns can span the 60-column output of xxd. Of course, this relies on GNU sed's ability to operate on very long lines (limited only by memory).

EDIT: this also works on multi-line targets eg

target=$'foo\nbar' replacement=$'bar\nfoo'

Tags:

Bash

Perl

Awk

Sed

Xxd