What's a good way to filter a text file to remove empty lines?

You can use grep's -v (invert match) mode to do this:

grep -v '^$' old-file.csv > new-file.csv

Note that those need to be different files, because of how shell redirects work. The output file is opened (and emptied) before the input file is read. If you have moreutils (not by default on Mac OS X), you can use sponge to work around this:

grep -v '^$' file.csv | sponge file.csv

But of course, then you have a harder time going back if something goes wrong.

If you "blank lines" actually may contain spaces (it sounds like they do), then you can use this instead:

egrep -v '^[[:space:]]*$' old-file.csv > new-file.csv

That will ignore blank lines as well as lines containing only whitespace. You can of course do the same sponge transformation on it.


The easiest option is just grep .. Here, the dot means "match anything", so if the line is empty, it is not matched. Otherwhise it prints the whole line as is.


To remove empty lines, in place, with ksh93:

sed '/./!d' file 1<>; file

The <>; redirection operator is specific to ksh93 and is the same as the standard <> operator except that ksh truncates the file after the command has terminated.

sed '/./!d' is a convoluted way to write grep ., but unfortunately GNU grep at least complains if its stdout points to the same file as its stdin. You'd say one could write:

grep . file | cat 1<>; file

But unfortunately, there's a bug in ksh93 (at least my version (93u+)), in that the file seems to be truncated to zero length in that case.

grep . file | { cat; } 1<>; file

Seems to work around that bug, but now, it's far more convoluted than the sed command.