YanoSecYou have a list of lines and want to keep only some of them. A pipe connects commands so one can process what the other prints. You can then display the selected lines or save them to a carefully chosen file.
A stream is data passed into or out of a command. Standard input, stdin, supplies incoming data. Standard output, stdout, carries its ordinary result. Standard error, stderr, carries errors separately. An ordinary pipe | connects the left command's stdout to the right command's stdin.
Before you begin
- Recommended first
- Install Ubuntu LTS and open a terminal, Where practice is allowed, find, grep, file, strings, and which
- Estimated time
- 15 min reading, 15 min activity. Setup and optional practice may take longer.
- Where you work
- Use Bash in your owned Ubuntu practice environment as an ordinary user. The supplied text files use UTF-8, a text encoding; where shown, LC_ALL=C requests consistent text ordering. Work only under your own new ~/yanosec-lab directory, never a host share or personal/system file. Refuse an existing exercise destination rather than overwriting it. No account, paid platform, or remote target is required. Optional extensions take about 10–15 minutes and are not part of the time above. A virtual machine is optional. You can also finish the reasoning from the printed evidence.
- Without a lab
- The page supplies the files, source and text records needed for the activity. Without a shell, follow the numbered steps on paper, label each result as predicted, and use the separate evidence supplied by the question. Running commands can wait until your lab is available. Reading works without JavaScript. Answer checking needs JavaScript and a secure context, such as HTTPS; saving a correct result also needs usable browser storage. Missing tools, managed or mobile hosts, or a deferred guest setup never require buying a service. Keep owned lab files for review; do not use broad deletion.
After this lesson you can:
- Trace input and output through a two-command pipeline.
- Save only matching lines to a fresh owned result file.
- Distinguish overwrite, append and the separate error stream.
01Example
printf prints red and blue on separate lines. The pipe passes those lines to grep, which selects lines containing blue. Follow the connection before looking at the predicted display. This trace is an expected example rather than recorded execution.
Command:
printf '%s\n' red blue | grep blue
Flow:
printf stdout -> grep stdin -> grep matching stdout
Predicted display:
blue
The pipeline does not need an intermediate file.02Choose the output destination deliberately
In the worked example, printf produces two lines and grep blue reads those lines through its stdin. grep prints only matching lines to its own stdout. The pipe controls where data flows; the pattern argument blue controls what the filter selects.
After a pipeline, > names a file to receive output and replace its content. >> instead appends at the end. The shell sets up a redirection before the command runs, so > can empty an existing file even if a later operation fails. Use only the stated fresh destinations.
An error is normally written to stderr. An ordinary pipe does not automatically send that error to the next command, and > does not automatically save it with stdout. Seeing a message on screen therefore does not tell you which stream carried it.
A filename argument and standard input are different sources. cat colors.txt reads that file and emits its text; grep blue with no filename reads the incoming stream. These small text cases keep the filenames and line boundaries controlled.
Four newline-terminated source lines become two blue lines; wc -l counts those two newline characters. The model uses no intermediate file.Model. A teaching model drawn by hand. It shows the idea, not real tool output.
Read the full explanation below.
Authored stdout model · no shell is run · no intermediate file is created
Named file → cat (opened directly)
stdout traced
red↵blue↵green↵blue↵Output newline count: 4
Pipe from cat colors.txt stdout → stdin
stdout not traced yet
Use the named stage control to reveal this supplied output.
Output newline count: not traced
Pipe from grep blue stdout → stdin
stdout not traced yet
Use the named stage control to reveal this supplied output.
Output newline count: not traced
A pipe connects one command’s stdout to the next command’s stdin. wc -l counts newline characters in its input; the printed result 2 itself occupies one output line.
cat sends all four supplied newline-terminated lines to stdout.
Follow the explanation
- cat sends all four supplied newline-terminated lines to stdout.
- The first pipe feeds grep stdin; grep retains both blue lines.
- The second pipe feeds wc -l; its predicted terminal stdout is 2.
Every supplied pipeline output
1. cat colors.txt
red
blue
green
blue
Exact output contains 4 newline characters; final row is newline-terminated.
2. grep blue
blue
blue
Exact output contains 2 newline characters; final row is newline-terminated.
3. wc -l
2
Exact output contains 1 newline characters; final row is newline-terminated.
Four source lines → two blue rows with their original identities → predicted count 2. No command or file operation is performed.
Predict first printf '%s\n' red blue | grep blue | wc -l. What number prints?
1. printf sends two lines, grep keeps one, and wc -l counts that one line.
03Use a fresh owned workspace
The paths lesson introduced ~/yanosec-lab: ~ expands to your home directory. Before a file activity, return with cd ~/yanosec-lab. Use pwd to check your location and ls -a to inspect names. Continue only if this is your recognized owned lab. Then create the lesson's new child directory.
If mkdir reports that a destination already exists, preserve that work. Choose an unused name and use it consistently, or follow the paper route. Enter one command at a time and continue only after the preceding step succeeds. Record an unexpected result before doing anything else.
Unless the lesson says otherwise, leave the small activity folder for review and return with cd ~/yanosec-lab when finished. Closing the terminal does not delete files you saved. These activities need no sudo or system-file changes.
04Try filtering and saving
Begin with this separate paper activity: source lines are amber, teal, amber and amber, each on its own line. A text producer sends them into a filter selecting amber, and the filter's stdout goes into a new result file. Predict each stage before reading the supplied trace below.
Numbered paper steps: (1) copy the four source lines; (2) cross out only the nonmatching line; (3) write the retained lines in their original order; (4) put those lines into the new result file; (5) repeat on paper with append rather than replacement. Check that replacement retains this run's three lines, while a second append adds another three.
The colors worksheet below is a separate supplied trace, using blue rather than amber. Keep that source for the separate question cards. The numbered terminal procedure practices the amber input in a new streams directory. On paper, trace the same amber file states without running commands. The later checks supply their own fixed cards.
red
blue
green
blue
Follow a text stream into a destination
Fresh guided input, one value on each line:
amber
teal
amber
amber
Commands for that guided input in a fresh owned streams directory:
printf '%s\n' amber teal amber amber > practice-colors.txt
cat practice-colors.txt | grep amber
cat practice-colors.txt | grep amber > practice-selected.txt
cat practice-selected.txt
Predicted saved result: three amber lines. A second append adds three
more; replacement keeps only the current run. stdout carries ordinary
results, stdin supplies incoming data, and stderr carries errors.
The separate colors.txt source and commands below belong to the
question cards. Trace them independently without copying the activity
result. They use different input and a different matching word.
Worked: printf '%s\n' red blue | grep blue
Predicted stdout: one blue line.
Separate question source: red, blue, green, blue on separate lines.
cat colors.txt | grep blue > selected.txt
Use the separate colors.txt lines to predict selected.txt yourself.
> replaces destination; >> appends; | connects stdout to next stdin.
stderr is a separate error stream unless explicitly redirected.
Question: printf '%s\n' gold blue | grep blue
Which command receives printf's stdout? Enter the command name only.
Download the guided stream trace and separate question source
- From your owned lab root create mkdir streams, then cd streams only after success. Check pwd and ls -a before writing any new file.
- Create the separate guided input with printf '%s\n' amber teal amber amber > practice-colors.txt. Predict four source lines, then inspect cat practice-colors.txt.
- Read the filter first, then run cat practice-colors.txt | grep amber. Explain why three input lines match while the teal line does not.
- Run cat practice-colors.txt | grep amber > practice-selected.txt. Inspect cat practice-selected.txt: expect three amber lines. Point to the pipe receiver and the final file destination separately.
- On paper, follow the same four input lines and record the final three-line file. Explain how a second append would differ from replacement. Return to the lab root and retain the small files. Read the separate colors card for the later checks without substituting your practice values.
05Optional challenge
Allow about ten extra minutes. Repeat the amber activity twice using append into an initially empty result. Explain its six selected lines and compare that with two runs using replacement. Then describe why an input-read error on stderr would not become an ordinary matching input line.
06Fresh cases for the checks
Use these fictional cases for the checks below. They have different inputs from the worked examples. Read them here or make notes; no installation or external service is needed.
Streams card A: independent fictional producerOwned fixture. An original file made for this course. Follow the activity's instructions in your own lab.
The harmless fictional command inspect-owned writes:
stdout: one line, item=oak
stderr: one line, warning=sample
Both lines end with a newline. The command does not read any files.
Paper pipeline: inspect-owned | grep oak > selected.txt
selected.txt is a fresh owned destination. grep succeeds and writes its match.
No other redirections or special shell settings are in use.
Streams card B: optional changed redirectionOwned fixture. An original file made for this course. Follow the activity's instructions in your own lab.
Initial owned result.txt contains one line: start
Both following commands succeed, in this order:
printf '%s\n' north >> result.txt
printf '%s\n' south > result.txt
Changed case: replace only the second command's > with >>.
Run the changed case from the same initial start line, not from the original result.
Each printed word ends with a newline. No other command changes the file.
07Explain output and file state
Optional self-check: explain how a pipeline can save a matching result while leaving a warning on screen. Compare running a producer twice with > and with >>.
Check your explanation: name stdin, stdout and stderr by their roles; trace the ordinary pipe into the next command; identify the final stdout destination; record a file state after every write. Replacement discards earlier contents; append preserves them.
08Common mistakes
If no output appears on screen after >, inspect the named new destination with cat. Redirection changed the destination; silence alone is not failure or success evidence.
If the result includes duplicate selected lines, inspect source duplicates and whether >> was used. Preserve the original input and explain the flow before rewriting anything.
If an error stays on screen, recall that stderr is separate. Record it rather than assuming it became the filter's input or suppressing it to satisfy a prediction.
References
- Ubuntu 24.04 packaged manual: bash(1)
Shell grammar, quoting, builtins, redirection and job control; use only the sections introduced in this lesson. Use the named command and option sections. This manual describes Ubuntu 24.04 packages; your installed version can differ. The lesson supplies the facts needed for its activity.
- Ubuntu 24.04 packaged manual: grep(1)
Use the named command and option sections. This manual describes Ubuntu 24.04 packages; your installed version can differ. The lesson supplies the facts needed for its activity.
- Ubuntu 24.04 packaged manual: printf(1)
Use the named command and option sections. This manual describes Ubuntu 24.04 packages; your installed version can differ. The lesson supplies the facts needed for its activity.