Chapter 28 — Redirection and the pipeline as a design pattern

In one sentence

>, >>, <, and | are all the same underlying idea — rewiring which file descriptor points where (Chapter 23) — and understanding that makes every redirection trick predictable instead of memorised.

Why this matters

Getting 2>&1 in the wrong place, or piping something that should have been redirected, is one of the most common real shell-scripting bugs — and it's entirely avoidable once the model clicks.

How it actually works

   command > file        # redirect STDOUT (fd 1) to a file, OVERWRITING it
   command >> file        # same, but APPEND instead of overwrite
   command < file          # redirect STDIN (fd 0) to read FROM a file
   command 2> file         # redirect STDERR (fd 2) to a file
   command > file 2>&1     # redirect stdout to file, THEN point stderr
                          # at "wherever fd 1 currently points" (the
                          # file) -- ORDER MATTERS, this is the classic
                          # gotcha (see below)
   command 2>&1 > file     # THIS DOES SOMETHING DIFFERENT: stderr is
                          # pointed at wherever stdout CURRENTLY goes
                          # (the terminal), THEN stdout is redirected to
                          # the file -- stderr still goes to the
                          # terminal, not the file!
   command1 | command2      # connect command1's STDOUT directly to
                          # command2's STDIN, via a pipe (Chapter 19) --
                          # no file involved at all

Why the order matters, explained properly:

   `2>&1` means "make file descriptor 2 point at wherever file
   descriptor 1 CURRENTLY points, right now, at this moment in the
   command line" -- it's not a persistent link, it's a one-time copy of
   a destination, evaluated left to right.

     > file 2>&1     step 1: fd 1 -> file.  step 2: fd 2 -> (wherever
                     fd 1 now points) -> file. BOTH end up in the file.
     2>&1 > file      step 1: fd 2 -> (wherever fd 1 currently points)
                     -> the terminal (unchanged so far). step 2: fd 1 ->
                     file. fd 2 is STILL pointed at the terminal from
                     step 1 -- it never got updated again.

Real-world example

   The widely-discussed real security debate around "curl | bash"
   (or `wget ... | sh`) style installation instructions -- a genuinely
   common real pattern, historically used by the official install
   instructions of several well-known tools -- is directly about THIS
   chapter's pipe mechanism, examined critically: piping a script
   DIRECTLY from a network download into a shell interpreter, with NO
   step where a human reviews what will actually run. Security
   researchers and engineering blog posts have repeatedly, publicly
   demonstrated real, concrete issues with this pattern: a server can
   detect it's being accessed by `curl` specifically (vs. a browser) and
   serve DIFFERENT content, a connection interrupted partway through can
   leave a shell executing a PARTIAL, syntactically-different script than
   intended, and a compromised download server or a machine-in-the-
   middle (this wiki's security guides cover this exact threat model)
   can inject arbitrary commands with no visible warning to the user at
   all.
   The safer, real-world-recommended alternative pattern -- also just
   this chapter's redirection, applied deliberately -- is: `curl -o
   install.sh <url>`, THEN actually read the file, THEN run it
   (`bash install.sh`) as a separate, reviewable step. Same underlying
   mechanism (Chapter 23's file descriptors and this chapter's
   redirection); very different risk profile, purely because of WHERE
   the human review step sits in the pipeline.

Try it yourself

cd ~/lab/playground

# See the 2>&1 order gotcha for yourself
echo "stdout line" > /dev/null   # (setup, ignore)
{ echo "to stdout"; echo "to stderr" >&2; } > both.log 2>&1
cat both.log                     # BOTH lines are here

{ echo "to stdout"; echo "to stderr" >&2; } 2>&1 > stdout_only.log
cat stdout_only.log              # only "to stdout" -- stderr went to
                                 # your terminal instead, watch for it
                                 # printing above this line when you ran
                                 # the command

# See a pipeline moving data with no file at all
echo "hello world" | tr 'a-z' 'A-Z' | rev

rm -f both.log stdout_only.log

Common mistakes

Writing 2>&1 > file when you meant > file 2>&1. The single most common redirection bug — order genuinely changes the behaviour, as shown above.
Piping when you meant to redirect, or vice versa. | connects two COMMANDS; > connects a command to a FILE. Using > when you have two commands just overwrites a file named after your second "command."
Blindly running curl <url> | bash. Download first, read it, then run it — the extra step costs seconds and closes a real, documented risk.

Check yourself

What does 2>&1 actually do, precisely?
Why does command > file 2>&1 behave differently from command 2>&1 > file?
What's the specific, documented risk with piping a downloaded script directly into a shell, and what's the safer alternative?