CertGrid CertGrid
Hands-on Lab·Red Hat Certified System Administrator

Shell Redirection and Pipelines

One objective, and most of it is muscle memory. The part that is not is the order of `>` and `2>&1`, which produces two different results from what looks like the same instruction - captured here side by side so the difference is visible rather than explained.

Essential Tools Guide 5 of 67 Beginner

Written against the versions above. `&>file` is a bash extension and works everywhere bash is the shell, which on RHEL it is. In a strict POSIX `sh` script, `>file 2>&1` is the portable form and the one worth defaulting to.

Every command on this page runs on RHCSA-A01.
Server NameIP AddressOSRolesCPURAMHDD
RHCSA-A01192.168.0.31RHEL 10.0 (Coughlan)Practice node (graded) - spare /dev/sda2 Core4 GB50 GB + 15 GB

Before you start

  1. Two streams, not one

    Every command has two output channels, and they look identical on a terminal:

    ls: cannot access '/etc/nosuchfile': No such file or directory
/etc/hostname

    Redirect stdout alone and the difference appears:

    ls /etc/hostname /etc/nosuchfile > out.txt
ls: cannot access '/etc/nosuchfile': No such file or directory
--- stdout file:
/etc/hostname

    The error stayed on the terminal. > redirects file descriptor 1 (stdout) only; the error went to descriptor 2 (stderr), which was never touched.

    Send each to its own file and both are captured separately - 2> for stderr, > for stdout. That separation is the whole model: 1 is results, 2 is complaints, and every operator below is a way of routing those two numbers.

    bash Example session 
    mkdir -p ~/redir && cd ~/redir && ls /etc/hostname /etc/nosuchfilels: cannot access '/etc/nosuchfile': No such file or directory/etc/hostname[exit 2]cd ~/redir && ls /etc/hostname /etc/nosuchfile > out.txt; echo "--- stdout file:"; cat out.txtls: cannot access '/etc/nosuchfile': No such file or directory--- stdout file:/etc/hostnamecd ~/redir && ls /etc/hostname /etc/nosuchfile 2> err.txt > out.txt; echo "--- stdout:"; cat out.txt; echo "--- stderr:"; cat err.txt--- stdout:/etc/hostname--- stderr:ls: cannot access '/etc/nosuchfile': No such file or directory

    Expected resultThe error surviving a stdout redirect, then both captured separately.

    Success conditionYou can route results and errors independently.

  2. Overwrite, append, and refusing to clobber

    two

    > truncates first. The one was written and then destroyed by the second command, and there is no warning.

    one
two

    >> appends. On an exam that is the difference between adding a line to /etc/fstab and replacing the file with one line - a mistake that stops the machine booting and costs every other task.

    Bash can be told to refuse:

    bash: b.txt: cannot overwrite existing file
exit=1

    set -o noclobber makes > fail on a file that exists, and >| overrides it for one command. It is not exam material, and it is a habit worth having on any machine you care about.

    bash Example session 
    cd ~/redir && echo one > a.txt && echo two > a.txt && cat a.txttwocd ~/redir && echo one > b.txt && echo two >> b.txt && cat b.txtonetwocd ~/redir && set -o noclobber; echo three > b.txt; echo "exit=$?"; set +o noclobberbash: line 2: b.txt: cannot overwrite existing fileexit=1

    Expected resultTruncation, appending, and a refused overwrite.

    Success conditionYou will not truncate a config file you meant to add a line to.

  3. The ordering trap

    These two look like the same instruction written two ways. They are not.

    >file 2>&1 - everything in the file:

    --- '>file 2>&1' captured:
ls: cannot access '/etc/nosuchfile': No such file or directory
/etc/hostname

    2>&1 >file - look where the error went:

    ls: cannot access '/etc/nosuchfile': No such file or directory
--- '2>&1 >file' captured:
/etc/hostname

    The error printed before the captured: line - it went to the terminal, not the file. Only stdout was captured.

    The reason is that redirections are applied **left to right, and 2>&1 copies where stdout points *at that moment*.** In the second form, stdout still points at the terminal when stderr is pointed at it; >file then moves stdout to the file and leaves stderr behind. It is a copy of a destination, not a lasting link.

    &>file does the obvious thing in one token and cannot be got wrong:

    --- '&>file' captured:
ls: cannot access '/etc/nosuchfile': No such file or directory
/etc/hostname

    Rule of thumb: 2>&1 goes last, or use &>.

    bash Example session 
    cd ~/redir && ls /etc/hostname /etc/nosuchfile > both1.txt 2>&1; echo "--- '>file 2>&1' captured:"; cat both1.txt--- '>file 2>&1' captured:ls: cannot access '/etc/nosuchfile': No such file or directory/etc/hostnamecd ~/redir && ls /etc/hostname /etc/nosuchfile 2>&1 > both2.txt; echo "--- '2>&1 >file' captured:"; cat both2.txtls: cannot access '/etc/nosuchfile': No such file or directory--- '2>&1 >file' captured:/etc/hostnamecd ~/redir && ls /etc/hostname /etc/nosuchfile &> both3.txt; echo "--- '&>file' captured:"; cat both3.txt--- '&>file' captured:ls: cannot access '/etc/nosuchfile': No such file or directory/etc/hostname

    Expected resultThe same command capturing different things depending on operator order.

    Success conditionYou can read 2>&1 and know which way round it must go.

  4. Discarding, and seeing output twice

    /etc/hostname

    2>/dev/null throws errors away. Useful for find / -name ... as a normal user, where permission-denied noise buries the results - and dangerous everywhere else, because it hides the message that explains the failure.

    tee writes to a file *and* passes output on:

    2
--- tee.txt holds:
ls: cannot access '/etc/nosuchfile': No such file or directory
/etc/hostname

    The 2 is wc -l counting what came through the pipe, while the file holds the same two lines. That is the point of tee: a copy without interrupting the pipeline.

    Its real use on an exam is writing a file that needs root when your shell is not root:

    echo 'some line' | sudo tee -a /etc/somefile

    sudo echo 'x' > /etc/f does not work - the redirect is performed by your unprivileged shell before sudo ever runs. sudo tee is the fix, and -a makes it append rather than truncate.

    bash Example session 
    cd ~/redir && ls /etc/hostname /etc/nosuchfile 2>/dev/null/etc/hostname[exit 2]cd ~/redir && ls /etc/hostname /etc/nosuchfile 2>&1 | tee tee.txt | wc -l; echo "--- tee.txt holds:"; cat tee.txt2--- tee.txt holds:ls: cannot access '/etc/nosuchfile': No such file or directory/etc/hostnamecd ~/redir && echo appended | tee -a tee.txt >/dev/null && tail -2 tee.txt/etc/hostnameappended

    Expected resultErrors discarded, then output both counted and saved.

    Success conditionYou can write a root-owned file from an unprivileged shell.

  5. Feeding input in

    adm
bin
chrony
daemon
ftp

    A pipeline: cut splits the field, sort orders it, head truncates. Each program reads stdin and writes stdout, which is what makes them composable.

    < redirects a file *into* a command, and the difference from passing a filename is visible:

    30
--- versus:
30 /etc/passwd

    Same count, different output. With <, wc reads stdin and has no filename to print; given the path it prints it. That matters in scripts that parse the output.

    Two more input forms. <<< passes one string:

    HERE STRINGS FEED STDIN

    and << passes a block until a marker - the here-document, which is how a multi-line file gets written in a script:

    apple
cherry
pear
    bash Example session 
    cut -d: -f1 /etc/passwd | sort | head -5admavahibinchronycleviswc -l < /etc/passwd; echo "--- versus:"; wc -l /etc/passwd41--- versus:41 /etc/passwdtr 'a-z' 'A-Z' <<< "here strings feed stdin"HERE STRINGS FEED STDINsort <<'EOF'pearapplecherryEOFapplecherrypear

    Expected resultA pipeline, and three ways of supplying input.

    Success conditionYou can build a pipeline and feed it from a file, a string or a block.

Troubleshooting

Official sources

← All RHCSA guides