File redirection vs dd

Direct access to file

some-command > file
some-command | dd $MAYBE_SOME_OPTIONS_LIKE_BS of=file

In the first case some-command knows it writes to file, so it can retrieve metadata (like mtime or size) for whatever reason. Such metadata have useful meaning for regular files but for a unnamed pipe they are rather meaningless.

some-command may even be interested in opening what its stdout points to for reading, which is rather safe for regular files and obviously not the right thing to do in your second case.

But these scenarios are quite rare, I think.

Seekable vs non-seekable

The behavior of some-command may depend on whether its stdout is seekable or not. It's usually not the case, although it's possible. A regular file is seekable (you can read/write at any location). A pipe is not seekable.

dd itself is a valid example. Consider some-command being dd if=/dev/zero bs=1 count=1 seek=1. Your two examples become:

dd if=/dev/zero bs=1 count=1 seek=1 > file
dd if=/dev/zero bs=1 count=1 seek=1 | dd of=file

The first command will work, it will leave file with two null bytes in it. In the second command the first dd will complain it cannot seek, it will exit; the second dd will get nothing, write nothing and exit without error.

I admit with seek this is not exactly plain "piping the output of some command into a file". But in general some-command may detect if its stdout is seekable and then stream different output without actually seeking. This shell function kinda does it as a proof of concept:

is-seekable() {
  if </dev/null dd bs=1 count=0 seek=1 conv=notrunc 2>/dev/null; then
    echo seekable
  else
    echo non-seekable
  fi
  }

Try these and examine the file after each one:

is-seekable > file
is-seekable | dd of=file

Knowing all the above, there's a scenario in which I would consider your second command: if some-command indeed behaves differently in two cases (like our is-seekable function) and I want it to "think" it's writing to a pipe but I want the output in a file. Then yes. Although I would rather use cat instead of dd (it probably wouldn't matter). E.g. I may want to get non-seekable from is-seekable and write it to a file:

is-seekable | cat > file

But only if it makes any difference. Otherwise it would be (in)famous "useless use of cat". Or in your case "useless use of dd"; or even "harmful use" if you handle the exit status poorly.

Exit status

Now try these two examples:

false > file
false | dd of=file

If you check $? after the second one, you will discover it's 0 (unless dd failed). Your second case discards the exit status of some-command (usually; e.g. in Bash investigate set -o pipefail, which is not portable).

So in the second case some-command may fail for whatever reason and you (or rather the logic of your script) will not know it. This is enough to avoid unnecessary piping to dd (or unnecessary piping at all).

Permissions

Another nuance: some-command > file makes the (sub)shell open the file before some-command is spawned (exec-ed to); dd of=file opens the file on its own behalf.

There are ways to grant more access rights to dd than to any other "regular" command. In other words there are ways to make plain dd behave more like sudo dd without password. You shouldn't do it and hopefully nobody does, yet it's possible.

What you can do is granting access rights on demand: sudo dd. It can be useful when trying to write to a restricted file. Analyze this:

some-command > restricted_file               # doesn't work
sudo some-command > restricted_file          # doesn't work
some-command | sudo dd > restricted_file     # doesn't work
sudo sh -c 'some_command > restricted_file'  # works, but it runs some_command as root
                                             #  you may not want this
some-command | sudo dd of=restricted_file    # works
some-command | sudo tee restricted_file      # works, more common, possibly with
                                             #  > /dev/null to suppress output to tty

I think this is very close to "non-negligible benefit or use case".

Maybe some options

Obviously $MAYBE_SOME_OPTIONS_LIKE_BS can alter dd behavior. I understand that options like conv=swab are out of scope because you want file to receive same data in both cases.

Still there are few options that can make a difference. Examples:

  • status=progress will print the transfer rate and volume statistics on stderr, when processing each input block;

  • oflag=noatime will not update the file's access timestamp;

  • oflag=nolinks will fail if the file has multiple hard links;

  • and more.

The above examples are GNU extensions, not portable. Portable dd is characterized here.

KISS

Finally there's the KISS principle: keep it stupid simple. Your first case is stupid simple.

Tags:

Bash

Pipe

Dd

Recent Posts