Delete the first n bytes of files

for file in /foo/*
do
  if [ -f "$file" ]
  then
    dd if="$file" of="$file.truncated" bs=31 skip=1 && mv "$file.truncated" "$file"
  fi
done

or the faster, thanks to Gilles' suggestion:

for file in /foo/*
    do
      if [ -f $file ]
      then
        tail +32c $file > $file.truncated && mv $file.truncated $file
      fi
    done

Note: Posix tail specify "-c +32" instead of "+32c" but Solaris default tail doesn't like it:

   $ /usr/bin/tail -c +32 /tmp/foo > /tmp/foo1
    tail: cannot open input

/usr/xpg4/bin/tail is fine with both syntaxes.

If you want to keep the original file permissions, replace

... && mv "$file.truncated" "$file"

by

... && cat "$file.truncated" "$file" && rm "$file.truncated"

The following commands cut first 31 bytes from $file (using $file~ as a temp. copy):

dd if="$file" of="$file~" bs=1 skip=31
mv "$file~" "$file"

You only need to list or find all files under /foo/ and execute the two above for every $file found.


tail -c +32 outputs its input minus the first 31 bytes. (Yes, the argument is off by one.) To edit a file in place, use sponge in a loop, or if you don't have it and don't want to bother, do its job in the shell:

for x in /foo/*; do tail -c +32 "$x" | sponge "$x"; done
for x in /foo/*; do tail -c +32 "$x" >"$x.new" && mv "$x.new" "$x"; done

If the commands are interrupted for whatever reason (e.g. a power failure), it might be hard to figure out where you left off. Writing the new files to a separate directory would make things easier.

mkdir /foo.tmp
cd /foo
for x in *; do tail -c +42 -- "$x" >"/foo.tmp/$x" && rm -- "$x"; done
mv /foo.tmp/* /foo
rmdir /foo.tmp

If the files are really large (as in, large enough that having two copies of even a single one is a problem), you can use one of the techniques mentioned in this thread.