YanoSec
Progress0 of 152 lessons complete

Course / 04 Code / Lesson 07 of 11

Reading a file as text and as bytes

Estimated time: 15 min reading · 15 min activity

A script can read the same file as text or as bytes. That choice changes what the script receives and what a length counts. Choose the mode before interpreting the result: text mode decodes bytes with an encoding, while binary mode leaves the bytes undecoded.

In open, the mode argument tells Python what kind of access you want. Mode r reads text and is the default. Mode rb reads binary data; b is the binary marker. A text read returns a str, and a binary read returns a bytes value. Use encoding="utf-8" for the text path. Do not pass an encoding argument with rb.

The with statement gives the open file a temporary name, called a handle, and closes it when the indented block finishes. handle.read() reads its contents. len asks how many units a value holds: bytes for a bytes value, Unicode code points for a str. Those counts need not match.

Before you begin

Estimated time
15 min reading, 15 min activity. Setup and optional practice may take longer.
Where you work
Read and trace the supplied source and input in your browser. A later optional run uses Python 3.12 and its standard library in a fresh directory under your own ~/yanosec-lab. The ~ means your home directory. Keep paired files together and avoid overwriting existing work. No extra packages or network calls are required.
Without a lab
Follow each source line against the supplied input and predicted trace. You can complete the same reasoning without running Python; the practical run can wait. Earlier recorded Python results are labeled separately from predictions. Reading works without JavaScript. Answer checking needs JavaScript and a secure context; saving additionally needs usable browser storage.

After this lesson you can:

  • Open a file in text mode with an explicit UTF-8 encoding.
  • Open the same file in binary mode and keep the bytes undecoded.
  • Predict when the character count and the byte count differ.

01Example

The worked file label.txt is a supplied byte model: 4f 4b 0a, meaning O, K, and one LF newline. It was not created or opened. Every code point here uses one UTF-8 byte, so the predicted text and binary lengths happen to agree.

label.txt bytes: 4f 4b 0a
Text mode, encoding utf-8, predicted characters: O, K, and a newline
Predicted len(text): 3
Binary mode rb, predicted len(data): 3
The counts match because every character is ASCII and the newline is one byte.
They will not match in the activity file.

02Text mode may rewrite line endings; binary mode does not

Text mode also handles newline conventions. With the default newline setting, it translates common stored line endings to a text newline character while reading. Binary mode keeps the original byte sequence. That distinction matters when a tool needs exact file bytes rather than text lines.

The activity download uses LF, the one-byte newline 0a. Some editors save CRLF, the two bytes 0d 0a. Retyping the word can therefore change the binary size even if the editor shows the same line. Use the supplied pair list as the evidence for this prediction.

Encoding goes in both directions: encode converts text to bytes; decode converts bytes to text using a named rule. A strict UTF-8 decode reports invalid sequences instead of silently discarding them. Ignoring errors would remove evidence and can change both content and length.

Raw-byte and decoded-character counts happen to match for 4f 4b 0a; they measure different data.Model. A teaching model drawn by hand. It shows the idea, not real tool output.

Read the full explanation below.

Playback is optional. Step through at your own pace.

Supplied label.txt bytes

label.txt raw bytes

4fbyte
4bbyte
0abyte

Explicit UTF-8 text

Ocharacter
Kcharacter
LF / newlinecharacter

Counts for this supplied ASCII/LF example

len(data) = 3binary count
len(text) = 3decoded count

4f + 4b + 0a → len(data) = 3Three bytes in rb mode

Binary rb reads exactly 4f 4b 0a: three bytes.

Follow the explanation

  1. Binary rb reads exactly 4f 4b 0a: three bytes.
  2. Explicit UTF-8 decoding aligns O, K and LF with the original bytes.
Read every state and value

1. Binary rb reads exactly 4f 4b 0a: three bytes.

Example
Supplied label.txt bytes
Visible Item
4f · 4b · 0a · len(data) = 3
Current mappings
Three bytes in rb mode
Annotations
None

2. Explicit UTF-8 decoding aligns O, K and LF with the original bytes.

Example
Supplied label.txt bytes
Visible Item
4f · 4b · 0a · O · K · LF / newline · len(data) = 3 · len(text) = 3
Current mappings
UTF-8 4f → O · UTF-8 4b → K · UTF-8 0a → LF · Three decoded characters here
Annotations
Item: len(text) = 3; Text: The counts match for these ASCII/LF bytes, not for every file.
Raw-byte and decoded-character counts happen to match for 4f 4b 0a; they measure different data.

03Compare the two lengths of note.txt

The supplied note.txt contains Café followed by one LF. Its complete byte display is 43 61 66 c3 a9 0a. This is the file read by both with blocks; the script does not make a second file.

Café

Download note.txt, UTF-8, one trailing LF

with open("note.txt", "rb") as handle:
    data = handle.read()

with open("note.txt", encoding="utf-8") as handle:
    text = handle.read()

print(len(data))
print(len(text))

Download the text-and-bytes reader

Predicted print:
6
5
Binary length counts 43 61 66 c3 a9 0a.
Text length counts C, a, f, é, and the newline.
The supplied file bytes are 43 61 66 c3 a9 0a.
  1. Count the supplied hex pairs before inspecting the script. Check: six pairs. Mark c3 a9 as the two UTF-8 bytes for the composed é.
  2. Find the rb block in read-note.py. Its data name receives raw bytes. Find the other block, which supplies encoding="utf-8"; text receives decoded characters.
  3. Follow the two print calls in order. len(data) predicts 6; len(text) predicts 5 because it counts C, a, f, é, and LF. Compare with the printed trace.
  4. Stop after explaining both counts. If you later run the script, save it beside the unchanged note.txt in a fresh lab directory and enter python3 read-note.py. The 2026-10-12 Python 3.12.3 record contains the same two lines, 6 then 5; it does not establish the newline behavior of a retyped file.

04Optional challenge

Spend about ten extra minutes considering a separate broken byte model, 43 ff. The second byte ff cannot begin a valid UTF-8 sequence, so strict text decoding should fail. A previous Python 3.12.3 run on a disposable two-byte file recorded a UnicodeDecodeError at position 1. That position belongs to 43 ff, not to the Café download. Your explanation should keep the decode failure visible rather than using errors="ignore".

05Fresh cases for the checks

Use these fictional cases for the checks below. They have different inputs from the worked examples. Read them here or make notes; no installation or external service is needed.

File case AOwned fixture. An original file made for this course. Follow the activity's instructions in your own lab.

File badge.txt has exactly these bytes: c2 a3 21 0a.
Supplied UTF-8 mapping: c2 a3 is £, 21 is !, 0a is LF.
There are no CR bytes or other bytes.
Case B: independent bytes 51 ff; ff is invalid under a strict UTF-8 decode.

06Explain your prediction

Optional written self-check: Show the bytes, named encoding and predicted text separately. Compare raw and decoded lengths, and preserve decoding failures instead of silently discarding bytes.

07Common mistakes

If the counts for a recreated file differ from 6 and 5, compare its bytes with the supplied pair list. The accent or newline may have changed; do not change the script to force a match.

If a binary read includes 0d 0a while text shows one newline, inspect the editor’s saved line-ending convention. Use the original LF download for the stated prediction.

If encoding is supplied with rb, choose the intended result first. Keep rb for bytes without decoding, or use text mode with the encoding argument.

References