Hands-on Lab·Linux Foundation Certified System Administrator

grep, sed and the text you have to change

Most of what an administrator does to text is find a line, count some lines, or change a string in a file. grep, sed and awk cover all three, and the mistake that costs marks is not syntax - it is using the wrong field number, which produces a confident wrong answer rather than an error. This guide counts the fields first, on purpose, because I got it wrong while writing it.

Essential Commands Guide 6 of 38 Intermediate

OSUbuntu 26.04 LTS
LVM2.03.31 (Ubuntu) / 2.03.36 (AlmaLinux)
nftables1.1.6 (Ubuntu) / 1.1.5 (AlmaLinux)
TimeAbout 20 min

GNU grep, sed and awk (mawk on Ubuntu, gawk on RHEL - the programs here work on both). grep -E for extended regular expressions; the same pattern in basic regex needs backslashes, which the capture shows.

Firewallufw 0.36.2 enabled but reporting inactive / firewalld active
Network confignetplan + systemd-networkd / NetworkManager 1.56.0

Everything on this page runs on the one Ubuntu host.
Server Name	IP Address	OS	Roles	CPU	RAM	HDD
LFCS-A01	192.168.0.70	Ubuntu 26.04 LTS	Primary host - most guides run only here	2 Core	4 GB	50 GB

This guide includes

Use this for the text-processing marks. This matters because the wrong awk field number is a silent wrong answer - it prints something plausible, and nothing tells you it was the wrong column.

reading a log's fields, and seeing what the wrong field number produces
using grep's flags for matching, counting, inverting and numbering
separating extended from basic regular expressions
building the cut, sort and uniq pipeline you will actually type, and finding what sort really does
making the change with sed, plus sed -i.bak and tr

Before you start

guide 5 - the shell quoting these depend on.

A log, and the fields it has
Eight lines of access log. Before anything else, count the fields.
bash Example session
```
$cd ~/ess/text && awk 'NR==1 {for (i = 1; i <= NF; i++) printf "$%-2d = %s\n", i, $i; print "NF =", NF}' access.log$1  = 10.0.0.1$2  = -$3  = -$4  = [24/Aug/2026:10:00:01]$5  = "GET$6  = /health$7  = HTTP/1.1"$8  = 200$9  = 45NF = 9
```
Note
This is the step that saves you. Printing every field once takes two seconds and removes the whole class of error the next step is about.

Note $5 is "GET with the quote attached and $7 is HTTP/1.1". awk splits on whitespace and knows nothing about quoting, so the request line is spread across three fields with punctuation stuck to two of them.

NF is the field count and NR the record number, and both are worth knowing: awk 'NF != 9' finds the malformed lines in a log that is supposed to be uniform.

Expected result$1 the IP, $4 the bracketed timestamp, $5 "GET with the quote attached, $8 the status, $9 the bytes, NF = 9.

Success conditionYou know which field holds what before you write a condition.
The wrong field number is a silent wrong answer
What I wrote first, and what it actually matched.
bash Example session
```
$cd ~/ess/text && echo "the wrong field number is a silent wrong answer, not an error:"; awk '$9 >= 500 {print "   matched:", $1, $9}' access.log; echo "   (that matched a 512-BYTE row, not a 5xx status)"the wrong field number is a silent wrong answer, not an error:   matched: 10.0.0.7 512   (that matched a 512-BYTE row, not a 5xx status)
```
Note
$9 >= 500 looked exactly right and was completely wrong. I meant "status of 500 or more" and asked for "response bigger than 500 bytes". It matched one row, printed plausible output, and exited zero.

Nothing warns you. awk does not know what a field means, so there is no error to catch - which is why this is worse than a syntax mistake. A syntax mistake fails; this ships.

On a performance exam the grader checks the end state, so a filter that quietly selects the wrong rows produces a wrong file and no clue why. Count the fields.

Expected resultmatched: 10.0.0.7 512 - and the note that 512 is a byte count, not a 5xx status.

Success conditionYou know why counting first matters.
So, with the right fields
Status is $8, bytes is $9.
bash Example session
```
$cd ~/ess/text && awk '{print $1, $8, $9}' access.log | head -410.0.0.1 200 4510.0.0.7 201 51210.0.0.1 200 4510.0.0.9 403 99$cd ~/ess/text && awk '$8 >= 500 {print $1, $6, $8}' access.log10.0.0.7 /api/orders 50010.0.0.7 /api/orders 500$cd ~/ess/text && awk '{bytes += $9} END {printf "total %d bytes over %d requests, mean %.1f\n", bytes, NR, bytes/NR}' access.logtotal 1220 bytes over 8 requests, mean 152.5$cd ~/ess/text && awk '{n[$8]++} END {for (s in n) printf "%s -> %d\n", s, n[s]}' access.log | sort -n200 -> 2201 -> 1304 -> 1403 -> 2500 -> 2
```
Note
{ sum += $9 } END { ... } is the awk idiom worth memorising - accumulate per line, print once at the end. NR in the END block is the total line count.

An associative array gives you a histogram in one expression. n[$8]++ counts by status with no sorting and no temporary file, and the for (s in n) loop prints it. Order is not guaranteed, hence the | sort -n.

-F'"' splits on the quote character instead, which the last capture line shows - steadier for this format, because the request itself can contain spaces and the quotes are the real delimiters.

Expected resultTwo genuine 500s on /api/orders; 1220 bytes over 8 requests, mean 152.5; and a status histogram of 200×2, 201, 304, 403×2, 500×2.

Success conditionYou can summarise a log.
grep, and the flags that do the work
Matching, counting, inverting and numbering.
bash Example session
```
$cd ~/ess/text && grep 500 access.log10.0.0.7 - - [24/Aug/2026:10:01:30] "POST /api/orders HTTP/1.1" 500 21010.0.0.7 - - [24/Aug/2026:10:03:00] "POST /api/orders HTTP/1.1" 500 210$cd ~/ess/text && grep -c 403 access.log; echo "---"; grep -n 403 access.log; echo "---"; grep -v 200 access.log | wc -l2---4:10.0.0.9 - - [24/Aug/2026:10:01:12] "GET /admin HTTP/1.1" 403 997:10.0.0.9 - - [24/Aug/2026:10:02:44] "GET /admin HTTP/1.1" 403 99---6$cd ~/ess/text && grep -i GET access.log | wc -l; echo "---"; grep -o '"[A-Z]* [^ ]*' access.log | sort -u5---" 200" 201" 304" 403" 500"GET /admin"GET /health"GET /static/app.js"POST /api/orders
```
Note
-c counts matching LINES, not matches. A line with three occurrences counts once. grep -o pattern | wc -l is how you count occurrences, which the capture also shows.

-v inverts and is how you filter noise out of a log; -n gives line numbers, which matters when you are about to edit the file; -i ignores case; -w matches whole words, which stops grep 80 finding 8080.

-r recurses a directory and -l prints only the filenames - together they answer "which config mentions this", which is a task you get asked to do constantly.

Expected resultTwo 500 lines; -c giving 2; -n prefixing line numbers 4 and 7; -v 200 counting 6.

Success conditionYou can find and count lines.
Extended against basic regular expressions
The same pattern, written twice.
bash Example session
```
$cd ~/ess/text && grep -E '" (4[0-9]{2}|5[0-9]{2}) ' access.log; echo "--- and the same thing with basic regex needs backslashes:"; grep '" $4[0-9][0-9]\|5[0-9][0-9]$ ' access.log | wc -l10.0.0.9 - - [24/Aug/2026:10:01:12] "GET /admin HTTP/1.1" 403 9910.0.0.7 - - [24/Aug/2026:10:01:30] "POST /api/orders HTTP/1.1" 500 21010.0.0.9 - - [24/Aug/2026:10:02:44] "GET /admin HTTP/1.1" 403 9910.0.0.7 - - [24/Aug/2026:10:03:00] "POST /api/orders HTTP/1.1" 500 210--- and the same thing with basic regex needs backslashes:4
```
Note
In basic regex, (, |, { and + are literal and need backslashes to mean grouping, alternation and repetition. In extended regex they are operators and a literal one needs the backslash instead. It is the same language with the escaping inverted, which is why grep -E (or egrep, now deprecated) is worth reaching for.

sed uses basic regex by default and sed -E switches it - so a pattern that works in grep -E needs -E in sed too. This is the single most common cause of a regex that "works in grep and not in sed".

awk uses extended regex always, with no flag.

Expected resultFour 4xx and 5xx lines from -E, and the same 4 from the basic-regex form with backslashes.

Success conditionYou can write the pattern the tool expects.
cut, sort and uniq, which is the pipeline you will type
Counting by field, without awk.
bash Example session
```
$cd ~/ess/text && cut -d' ' -f1 access.log | sort | uniq -c | sort -rn      3 10.0.0.7      2 10.0.0.9      2 10.0.0.1      1 10.0.0.3$cd ~/ess/text && cut -d'"' -f2 access.log | cut -d' ' -f1 | sort | uniq -c      5 GET      3 POST
```
Note
sort | uniq -c | sort -rn is the idiom. uniq only collapses *adjacent* duplicates, so the first sort is not optional - forgetting it is the classic mistake and it produces a count of 1 for everything.

cut -d' ' treats every single space as a delimiter, so two spaces make an empty field. That is why awk is usually better for whitespace-separated data: awk collapses runs of whitespace, cut does not.

Two cuts in series is a legitimate way to peel a field out of a quoted section, and it is often clearer than one clever regex.

Expected result3 10.0.0.7, 2 10.0.0.9, 2 10.0.0.1, 1 10.0.0.3 - and 5 GET, 3 POST.

Success conditionYou can produce a top-N without writing a program.
sort, which is not doing what you think
Four lines of mixed case and two numbers, sorted three ways.
bash Example session
```
$cd ~/ess/text && printf 'b\na\nB\nA\n10\n9\n' > mixed.txt && echo "sort:"; sort mixed.txt | tr '\n' ' '; echo; echo "sort -n:"; sort -n mixed.txt | tr '\n' ' '; echo; echo "sort -f:"; sort -f mixed.txt | tr '\n' ' '; echosort:10 9 a A b Bsort -n:A B a b 9 10sort -f:10 9 A a B b
```
Note
Plain sort put 10 before 9 because it is comparing text, and 1 sorts before 9. That is the bug in every "sort the sizes" pipeline that forgets -n.

sort -n put the letters first - non-numeric lines compare as zero, so they all tie and end up ahead of 9 and 10. Mixed data plus -n is a silent mess.

The locale decides case ordering. Under a UTF-8 locale a A b B interleaves; under LC_ALL=C it is strict byte order, uppercase first. A script that must be reproducible sets LC_ALL=C, and this is a real source of "works on my machine".

-k selects a field, -t sets the separator, -u deduplicates, -h handles 4.0K.

Expected resultPlain sort: 10 9 a A b B. -n: A B a b 9 10. -f: 10 9 A a B b.

Success conditionYou can sort deliberately rather than hopefully.
sed, for the change you have to make
Substitute, print a range, delete - and then edit a file in place.
bash Example session
```
$cd ~/ess/text && sed 's/HTTP\/1.1/HTTP\/2/' access.log | head -210.0.0.1 - - [24/Aug/2026:10:00:01] "GET /health HTTP/2" 200 4510.0.0.7 - - [24/Aug/2026:10:00:04] "POST /api/orders HTTP/2" 201 512$cd ~/ess/text && sed -n '3,5p' access.log; echo "---"; sed -n '/403/p' access.log | wc -l; echo "---"; sed '1d' access.log | wc -l10.0.0.1 - - [24/Aug/2026:10:00:09] "GET /health HTTP/1.1" 200 4510.0.0.9 - - [24/Aug/2026:10:01:12] "GET /admin HTTP/1.1" 403 9910.0.0.7 - - [24/Aug/2026:10:01:30] "POST /api/orders HTTP/1.1" 500 210---2---7$cd ~/ess/text && cp access.log edit.log && sed -i 's/10\.0\.0\.9/REDACTED/g' edit.log && grep -c REDACTED edit.log && grep -c '10.0.0.9' edit.log || echo "0 remaining"200 remaining
```
Note
s/old/new/ changes the FIRST occurrence per line; s///g changes all of them. Forgetting the g is the commonest sed mistake and it half-works, which makes it hard to spot.

The delimiter does not have to be /. s|/usr/local|/opt| avoids escaping every slash in a path, and sed accepts any character there. The escaped version in this capture is deliberately the painful spelling.

-n with p is the print-only mode, which is how you use sed as a line selector: -n '3,5p' for a range, -n '/pattern/p' for matches, -n '$p' for the last line.

-i edits in place and there is no undo. Run it without -i first and look at the output. Then -i.bak, which the next step shows.

Expected resultHTTP/2 substituted; lines 3 to 5 printed; 1d leaving 7 lines; and sed -i leaving 2 REDACTED and 0 occurrences of the original.

Success conditionYou can change a string in a file.
sed -i.bak, and tr
The safety net, and the smallest of the three tools.
bash Example session
```
$cd ~/ess/text && cp access.log backup-test.log && sed -i.bak 's/GET/FETCH/' backup-test.log && ls backup-test.log*; echo "---"; head -1 backup-test.log; head -1 backup-test.log.bakbackup-test.logbackup-test.log.bak---10.0.0.1 - - [24/Aug/2026:10:00:01] "FETCH /health HTTP/1.1" 200 4510.0.0.1 - - [24/Aug/2026:10:00:01] "GET /health HTTP/1.1" 200 45$cd ~/ess/text && echo "Hello   World" | tr 'a-z' 'A-Z'; echo "Hello   World" | tr -s ' '; echo "Hello   World" | tr -d ' '; echo "a1b2c3" | tr -cd '0-9'; echoHELLO   WORLDHello WorldHelloWorld123
```
Note
-i.bak writes the original alongside, which costs nothing and is the difference between a mistake and an incident. Note the suffix attaches directly with no space: -i.bak, not -i .bak.

tr works on characters, not strings or patterns. tr 'a-z' 'A-Z' maps one set onto another, -s squeezes runs into one, -d deletes a set, and -cd keeps only a set by deleting the complement - which is how a1b2c3 became 123.

tr reads stdin only - it takes no filename, which surprises people. tr a-z A-Z < file is the form.

guide 7 is next.

Expected resultbackup-test.log with FETCH and backup-test.log.bak with the original GET; then HELLO WORLD, Hello World, HelloWorld, 123.

Success conditionYou can edit in place without losing the original.

Troubleshooting

An awk condition selects the wrong rows and reports no error.

Why: The wrong field number. awk does not know what a field means.

Fix:Print every field once: awk 'NR==1 {for (i=1;i<=NF;i++) print i, $i}'.
A regex works in grep -E and fails in sed.

Why: sed uses basic regex by default, where (, | and { are literal.

Fix:sed -E, or add the backslashes.
uniq -c reports 1 for everything.

Why: uniq only collapses adjacent lines.

Fix:sort | uniq -c.
A numeric sort puts 10 before 9.

Why: It is sorting as text.

Fix:sort -n, or -h for human-readable sizes. Beware -n on mixed data.
sed -i changed only the first match on each line.

Why: No g flag.

Fix:s/old/new/g.
cut -d' ' produces empty fields.

Why: Every single space is a delimiter, so runs of spaces make empty fields.

Fix:Use awk, which collapses whitespace, or tr -s ' ' first.
A sort order differs between two machines.

Why: Different locales.

Fix:LC_ALL=C sort for byte order that does not move.

Official sources

← All LFCS guides

grep, sed and the text you have to change

This guide includes

Before you start

A log, and the fields it has

The wrong field number is a silent wrong answer

So, with the right fields

grep, and the flags that do the work

Extended against basic regular expressions

cut, sort and uniq, which is the pipeline you will type

sort, which is not doing what you think

sed, for the change you have to make

sed -i.bak, and tr

Troubleshooting

Official sources