Two files may describe the same request in different orders. A join combines related rows using a shared identifier, a stable label for the same event. Pairing the first row in one file with the first in another can attach a result to the wrong request.
CSV means comma-separated values, a text table with column names in a header. DictReader reads each row into a dictionary addressed by those column names. The supplied Python script checks required columns, records duplicate identifiers and prints matching rows. You will trace its source; running it is unnecessary.
Before you begin
- Recommended first
- The rule for every tool from here on, Notes: scope, command, result, time, Passwords, hashes, and dumps, Access control and IDOR, A one-page sample
- Estimated time
- 18 min reading, 18 min activity. Setup and optional practice may take longer.
- Where you work
- Use only the supplied North Desk fictional cards and source in a desktop or mobile browser. Keep predictions in private notes. No login attempt, recovery tool, framework module, permission change, domain setup or target connection is required. Optional later lab setup has a separate budget.
- Without a lab
- All evidence needed for the activity appears on this page. Copy it into notes if downloads are unavailable. Read the labeled model and predict each result; a paper trace practises the reasoning while tool operation waits for a lab. Reading needs no JavaScript. Answer checking needs JavaScript and a secure context; saving progress also needs usable browser storage.
After this lesson you can:
- Hand-trace a join that follows the shared id column and keeps the first row of a duplicate.
- Name the unmatched request in the activity files.
- Attach one ungraded line from each module to that unmatched id.
01Example
Fresh authored trace, separate from the Q7/Q8/Q9 activity files.
Requests in order: W1 Noor read; W2 Ina list
Evidence in order: W2 allowed; W1 denied
Join by identifier: W1 Noor read denied; W2 Ina list allowed
Pairing by position would attach W2’s result to W1.
These are predictions from the matching rule, not program output.02What the script keeps
Read the source in stages. import csv loads the standard-library reader; with open(..., newline="", encoding="utf-8") opens local text while letting the CSV reader handle newline conventions. DictReader uses the first row as field names; values remain strings. Brackets such as row["request_id"] look up a named field.
load checks headers, builds a lookup dictionary keyed by the identifier and retains the first occurrence of each identifier. Later duplicates go into a separate list rather than overwriting earlier rows. The main loop follows request order, finds matching evidence with get and records unmatched requests when no evidence exists. Missing evidence is not a denial. Evidence without a request is separately reported.
The sample output uses comma-joining for fields whose supplied notes contain no commas. A future comma-bearing value would need proper CSV writing/quoting. Header checks are not comprehensive data validation: empty or malformed values and duplicate conflicts still require review. Recorded sample: the earlier Python3.12.3 run matched the rows beneath the label in predicted-join.txt; its label/prose is not stdout.
Trace a CSV join by stable identifiers, retain missing/duplicate records and preserve their evidence limits.Model. A teaching model drawn by hand. It shows the idea, not real tool output.
Read the full explanation below.
Requests in source order
Evidence in source order
W1 Noor read denied
W2 Ina list allowed
Join by stable identifier: W1 Noor read denied; W2 Ina list allowed.
Follow the explanation
- Join by stable identifier: W1 Noor read denied; W2 Ina list allowed.
- Join by stable identifier: W1 Noor read denied; W2 Ina list allowed. Inspect the W2 association.
- Incorrect positional pairing attaches W2’s allowed result to W1 and swaps the associations.
- Incorrect positional pairing attaches W2’s allowed result to W1 and swaps the associations. Inspect the W2 association.
Read every state and value
1. Join by stable identifier: W1 Noor read denied; W2 Ina list allowed.
- Mode
- identifier
- Selected Request
- W1
- Results
- Request: W1; Evidence: W1; Status: matched; Text: W1 Noor read denied · Request: W2; Evidence: W2; Status: matched; Text: W2 Ina list allowed
2. Join by stable identifier: W1 Noor read denied; W2 Ina list allowed. Inspect the W2 association.
- Mode
- identifier
- Selected Request
- W2
- Results
- Request: W1; Evidence: W1; Status: matched; Text: W1 Noor read denied · Request: W2; Evidence: W2; Status: matched; Text: W2 Ina list allowed
3. Incorrect positional pairing attaches W2’s allowed result to W1 and swaps the associations.
- Mode
- incorrect-position
- Selected Request
- W1
- Results
- Request: W1; Evidence: W2; Status: matched; Text: Incorrect by position: W1 attached to W2 allowed · Request: W2; Evidence: W1; Status: matched; Text: Incorrect by position: W2 attached to W1 denied
4. Incorrect positional pairing attaches W2’s allowed result to W1 and swaps the associations. Inspect the W2 association.
- Mode
- incorrect-position
- Selected Request
- W2
- Results
- Request: W1; Evidence: W2; Status: matched; Text: Incorrect by position: W1 attached to W2 allowed · Request: W2; Evidence: W1; Status: matched; Text: Incorrect by position: W2 attached to W1 denied
03Hand-trace the activity files
Read both CSV files and the source; keep their original rows unchanged. The printed result provides a reference for the same trace.
request_id,account,action
Q7,lea,read
Q8,sam,write
Q9,ida,list
request_id,result,note
Q9,allowed,list right
Q7,allowed,read right
#!/usr/bin/env python3
"""Join request rows to evidence rows by request_id.
The printed rows are in predicted-join.txt.
The owned requests.csv and evidence.csv are the inputs for those rows.
A later run that disagrees with that card should correct the card.
Later use, only on the owned files in this folder:
python3 join-evidence.py requests.csv evidence.csv
Standard library only. No network. No password field.
Duplicate ids are reported. The first row is kept. Later copies do not overwrite.
A request with no evidence row is unmatched. It is not printed as a denial.
""",
import csv
import sys
def load(path, required):
with open(path, newline="", encoding="utf-8") as handle:
reader = csv.DictReader(handle)
if reader.fieldnames is None:
raise SystemExit(f"{path}: missing header")
missing = [name for name in required if name not in reader.fieldnames]
if missing:
raise SystemExit(path + ": missing columns " + ", ".join(missing))
rows = []
seen = {}
duplicates = []
for row in reader:
key = row["request_id"]
if key in seen:
duplicates.append(key)
continue
seen[key] = row
rows.append(row)
return rows, seen, duplicates
def main(argv):
if len(argv) != 3:
raise SystemExit("usage: join-evidence.py requests.csv evidence.csv")
requests, request_map, request_dups = load(
argv[1], ["request_id", "account", "action"]
)
_evidence_rows, evidence_map, evidence_dups = load(
argv[2], ["request_id", "result", "note"]
)
print("joined")
print("request_id,account,action,result,note")
unmatched = []
for row in requests:
key = row["request_id"]
evidence = evidence_map.get(key)
if evidence is None:
unmatched.append(key)
continue
print(
",".join(
[
key,
row["account"],
row["action"],
evidence["result"],
evidence["note"],
]
)
)
print("unmatched_requests")
for key in unmatched:
print(key)
print("evidence_without_request")
for key in evidence_map:
if key not in request_map:
print(key)
print("duplicate_request_ids")
for key in request_dups:
print("requests:" + key)
for key in evidence_dups:
print("evidence:" + key)
if __name__ == "__main__":
main(sys.argv)
Historical sample output: join-evidence.py with Python 3.12.3 against the supplied CSV files, recorded 2026-10-12.
The rows below are that output.
If a later run differs, investigate input/source differences before changing the record.
joined
request_id,account,action,result,note
Q7,lea,read,allowed,read right
Q9,ida,list,allowed,list right
unmatched_requests
Q8
evidence_without_request
duplicate_request_ids
Activity files, as printed
requests: Q7 lea read, then Q8 sam write, then Q9 ida list
evidence: Q9 allowed, then Q7 allowed
Predicted joined rows: Q7, then Q9
Predicted unmatched request: Q8
Predicted evidence without a request: none
Predicted duplicates: none
A position pairing would attach Q9's result to Q7.- Confirm both files have request_id. Note that evidence lists Q9 before Q7, so position is not a reliable relationship.
- Follow request order: Q7 finds evidence, Q8 does not, Q9 finds evidence. Copy each associated result without substituting a missing row with denial.
- Compare your trace with joined Q7/Q9 and unmatched Q8. Check that no duplicates or evidence-only rows are supplied in this case.
- Stop with source/method, predicted rows and unmatched status. A later disagreement from an actual run would require investigating inputs and source, not automatically trusting either record.
04Reporting note and stage evidence
Within the stated activity time, write a stage evidence note with one sourced conclusion from each module. Include the tiny local-comparison budget, framework lead versus permission concern, directory assignment boundary and the unmatched CSV row. Keep predictions, synthetic cards and historical outputs labeled.
Your final report should connect claim → evidence → limit → fitting next check or remedy. Completing this note establishes reading and reasoning practice, not a professional test or a changed directory. The next Reversing stage applies source-reading habits to small program and instruction models.
Stage evidence note
Label: ungraded. Counted inside this lesson's activity estimate.
This note does not certify a penetration test, a directory design, or tool proficiency.
Passwords: budget 3 on the local label card, stop after the third candidate, no service named.
Framework: proposal M-1 was a version lead. This card does not depend on running it.
Privilege: lea has write on run.bat, and NightIndex is recorded as LocalSystem. The file was not changed.
Directory: assigned host vm-desk-3.lab.invalid. Excluded host vm-other-9.lab.invalid. No connection.
Reporting: joined rows are Q7 and Q9. Unmatched request Q8 has no evidence row, so it is not a denial and not a finding.
Rubric: one line from each module, the unmatched id, and a limit. Leave out passwords, tickets, and module commands.
- Use the passwords files to record the three-comparison budget and the absence of a login endpoint.
- Use M-1 and NightIndex to distinguish an unconfirmed version lead from an explicitly modeled risky permission path.
- Use the directory scope to name the assigned target/action and excluded peer without making a connection.
- Use Q8 to explain why unmatched evidence is neither denial nor finding. Stop with a private report and no secret bytes.
05Optional challenge
For about twelve optional minutes, imagine a comma inside one note field. Explain why plain comma-joining would create an extra apparent column and why csv.writer can quote the field correctly. Also list a validation question about empty IDs; do not edit or run the source for this extension.
06Fresh cases for the checks
Use these fictional cases for the checks below. They have different inputs from the worked examples. Read them here or make notes; no installation or external service is needed.
Join card A: independent fictional CSV rowsOwned fixture. An original file made for this course. Follow the activity's instructions in your own lab.
requests, in order:
request_id,account,action
R4,noa,read
R5,kai,write
R6,umi,list
evidence, in order:
request_id,result,note
R6,denied,list absent
R4,allowed,read right
Rule: join by request_id and follow request order. Missing evidence is unmatched, not denial.
Duplicate card B: separate first-row-wins inputOwned fixture. An original file made for this course. Follow the activity's instructions in your own lab.
requests: D8,eri,read
Evidence rows in file order:
D8,allowed,read right
D8,denied,conflicting later row
Rule from the lesson source: keep the first occurrence, report later duplicate identifiers, and do not silently overwrite.
Orphan card C: separate inputsOwned fixture. An original file made for this course. Follow the activity's instructions in your own lab.
Request ids: V2, V3. Evidence ids: V3, V9.
Each id occurs once. Join by identifier.
07Explain the evidence and its limit
Optional self-check: produce an evidence note with joined results, unmatched requests, evidence-only ids and duplicate conflicts kept separate.
Review it: join by the named id, follow request order, keep the first duplicate while reporting the conflict, and never rewrite missing evidence as a denial. These are source predictions, not a program run.
08Common mistakes
If Q7 receives Q9’s result, inspect the identifier lookup rather than the row position.
If Q8 becomes denied, restore unmatched status; no evidence row makes that decision.
If the printed label is called stdout, separate the historical output rows from explanatory text surrounding them.
References
- Python 3.12: csv
DictReader, reader/writer newline handling and CSV quoting.
- NIST ITL Bulletin, December 2008
Assessment background on supported findings, analysis, mitigation and reporting.
- NIST SP 800-115 publication page
Assessment background on supported findings, analysis, mitigation and reporting.
Next is the first lesson of Reversing.