bash: moving files with spaces

Never, ever use for foo in $(cat bar). This is a classic mistake, commonly known as bash pitfall number 1. You should instead use:

while IFS= read -r file; do mv -- "$file" "new_place/$file"; done < file_list.txt

When you run the for loop, bash will apply wordsplitting to what it reads, meaning that a strange blue cloud will be read as a, strange, blue and cloud:

$ cat files 
a strange blue cloud.txt
$ for file in $(cat files); do echo "$file"; done
a
strange
blue
cloud.txt

Compare to:

$ while IFS= read -r file; do echo "$file"; done < files 
a strange blue cloud.txt

Or even, if you insist on the UUoC:

$ cat files | while IFS= read -r file; do echo "$file"; done
a strange blue cloud.txt

So, the while loop will read over its input and use the read to assign each line to a variable. The IFS= sets the input field separator to NULL^*, and the -r option of read stops it from interpreting backslash escapes (so that \t is treated as slash + t and not as a tab). The -- after the mv means "treat everything after the -- as an argument and not an option", which lets you deal with file names starting with - correctly.

^* _{This isn't necessary here, strictly speaking, the only benefit in this scenario is that keeps read from removing any leading or trailing whitespace, but it is a good habit to get into for when you need to deal with filenames containing newline characters, or in general, when you need to be able to deal with arbitrary file names.}

That unquoted $(cat file_list.txt) in POSIX shells like bash in list context is the split+glob operator (zsh only does the split part as you'd expect).

It splits on characters of $IFS (by default, SPC, TAB and NL) and does glob unless you turn off globbing altogether.

Here, you want to split on newline only and don't want the glob part, so it should be:

IFS='
' # split on newline only
set -o noglob # disable globbing

for file in $(cat file_list.txt); do # split+glob
  mv -- "$file" "new_place/$file"
done

That also has the advantage (over a while read loop) to discard empty lines, preserve a trailing unterminated line, and preserve mv's stdin (needed in case of prompts for instance).

It does have the disadvantage though that the full content of the file has to be stored in memory (several times with shells like bash and zsh).

With some shells (ksh, zsh and to a lesser extent bash), you can optimise it with $(<file_list.txt) instead of $(cat file_list.txt).

To do the equivalent with a while read loop, you'd need:

while IFS= read <&3 -r file || [ -n "$file" ]; do
  {
    [ -n "$file" ] || mv -- "$file" "new_place/$file"
  } 3<&-
done 3< file_list.txt

Or with bash:

readarray -t files < file_list.txt &&
for file in "${files[@]}"
  [ -n "$file" ] || mv -- "$file" "new_place/$file"
done

Or with zsh:

for file in ${(f)"$(<file_list.txt)"}
  mv -- "$file" "new_place/$file"
done

Or with GNU mv and zsh:

mv -t -- new_place ${(f)"$(<file_list.txt)"}

Or with GNU mv and GNU xargs and ksh/zsh/bash:

xargs -rd '\n' -a <(grep . file_list.txt) mv -t -- new_place

More reading about what it means to leave expansions unquoted at Security implications of forgetting to quote a variable in bash/POSIX shells

bash: moving files with spaces

Tags:

Bash

Mv

Files

Related

Recent Posts