PYTHON / FILE HANDLING
Reading and writing CSV files
Use Python's csv module to write and read delimited files, handle quoted fields and headers, and convert text fields to real types.
What you will learn
- Serialize rows with csv.writer and parse them back with csv.reader
- Always open CSV files with newline='' so quoting and \r\n endings survive
- Use DictReader/DictWriter to address columns by name instead of index
- Convert numeric columns yourself: every field read from CSV arrives as str
Understanding Reading and writing CSV files
A CSV file looks like text you could split on commas, but it is a real format with escaping rules: a field that contains the delimiter, a double quote, or a line break gets wrapped in quotes, and quotes inside it are doubled. That means line.split(',') gives the wrong number of fields the first time someone types a company name like "Acme, Inc.". The csv module exists to encode and decode those rules for you: csv.writer(f) takes sequences and writes properly quoted lines, csv.reader(f) takes a file object that yields lines and gives back a list of strings per record.
The reader and writer never touch the file themselves, they wrap an object you opened, and that is why you should always open CSV files with newline=''. The default excel dialect ends every record with \r\n; if text mode is also translating \n to the platform line ending, you get \r\r\n and every second line in the file looks blank. On the reading side, newline='' stops Python from translating line breaks that live inside quoted fields, so a multi-line address stays one field instead of turning into a broken record.
CSV has no type system. Everything you write is passed through str() and everything you read comes back as str, so '12' + '7' is '127' and sorted() puts '10' before '9'. Convert at the boundary, right where you consume the row, using int(), float(), or a date parser. When the file has a header line, csv.DictReader consumes it as field names and yields dicts, and csv.DictWriter(f, fieldnames=[...]) plus writeheader() does the reverse, which keeps your code working when columns are reordered.
import csv
from pathlib import Path
path = Path("sales.csv")
rows = [
["product", "units", "note"],
["widget", 12, "plain"],
["gizmo", 7, "ships in 2, maybe 3 boxes"],
]
with path.open("w", newline="", encoding="utf-8") as f:
csv.writer(f).writerows(rows)
print(path.read_bytes())
with path.open(newline="", encoding="utf-8") as f:
for record in csv.reader(f):
print(record)
path.unlink()A CSV file is delimited text with its own quoting and line-ending rules, so let the csv module encode and decode records while you handle the types.
Worked examples
Headers with DictReader and DictWriter
Reads records as dicts keyed by the header line, then writes a new file with an explicit header.
import csv, io
text = "name,dept,salary\nAmara,eng,120000\nBo,sales,90000\n"
total = 0
for row in csv.DictReader(io.StringIO(text)):
total += int(row["salary"])
print(row["name"], "->", row["dept"])
print("total:", total)
out = io.StringIO()
writer = csv.DictWriter(out, fieldnames=["name", "dept"], lineterminator="\n")
writer.writeheader()
writer.writerow({"name": "Cleo", "dept": "ops"})
print(repr(out.getvalue()))Example explained
Line 1DictReader eats the first line as field names, so the loop only sees the two data records.
Line 2int(row["salary"]) is required: without it total += '120000' raises TypeError.
Line 3DictWriter needs fieldnames up front because dicts must be flattened in a fixed column order.
Line 4lineterminator="\n" overrides the excel dialect's \r\n, which is why the output ends in a single \n.
Why splitting on commas fails
Compares str.split(',') with csv.reader on a record holding a quoted comma and a quoted newline.
import csv, io
line = 'A101,"Smith, Jane","line one\nline two",42\n'
print("naive:", line.rstrip("\n").split(","))
print("csv :", next(csv.reader(io.StringIO(line, newline=""))))Example explained
Line 1split(",") produces five fields because it cannot tell a quoted comma from a separator.
Line 2The naive result also keeps the quote characters, which would poison any later comparison.
Line 3csv.reader recognises the opening quote and keeps consuming lines until the field closes, so the embedded \n stays inside one field.
Line 4newline="" on the stream is what preserves that \n unchanged instead of translating it.
A semicolon-delimited file with numeric data
Parses a European-style CSV by naming the delimiter and converting each value to float.
import csv, io
text = "city;temp_c\nOslo;-3.5\nLima;21.0\n"
reader = csv.DictReader(io.StringIO(text), delimiter=";")
temps = {r["city"]: float(r["temp_c"]) for r in reader}
print(temps)
print("warmest:", max(temps, key=temps.get))Example explained
Line 1delimiter=";" changes the field separator; without it each line would parse as one long field.
Line 2float(r["temp_c"]) converts at the point of reading, so the dict holds numbers, not text.
Line 3max with key=temps.get compares floats; on the raw strings '21.0' vs '-3.5' it would compare characters instead.
Important notes
csv.writer's default excel dialect ends lines with \r\n; pass lineterminator="\n" if a downstream tool insists on Unix endings.
Spreadsheet exports often begin with a UTF-8 BOM; open them with encoding="utf-8-sig" or your first field name becomes '\ufeffname'.
Common mistakes
Opening the file without newline="": on Windows the writer's \r\n becomes \r\r\n, so the file shows a blank line between every record and re-reading it yields empty rows.
Treating read values as numbers: total += row[2] concatenates strings or raises TypeError, and sorting puts '100' before '9'.
Forgetting that csv.reader hands you the header as an ordinary first row, so int(row[1]) blows up with ValueError: invalid literal for int() with base 10: 'units'.
Try it yourself
Change, predict, then run
Using io.StringIO with the text "name,score\nAda,9\nGrace,7\nLin,10\n", read it with csv.DictReader and print the name with the highest score. Then write the same rows back out with csv.DictWriter sorted by descending score, and print the resulting CSV text.
Open the Python workspaceCheck your understanding
You call open("out.csv", "w") with no newline argument and write three rows with csv.writer. On Windows the file shows a blank line between every row. What happened?
- The writer emits \r\n and text-mode newline translation expanded the \n into \r\n, producing \r\r\n per record
- csv.writer adds an extra line terminator after each writerow call
- The file needed mode "a" instead of "w", and "w" pads records with a separator line
- Windows requires a blank line between CSV records, so this is the expected format
Show answer
The excel dialect's lineterminator is already \r\n; with newline=None the text layer also translates every \n into the platform ending, so each record ends \r\r\n and readers see an empty line. Passing newline="" disables that translation. Option 1 is tempting but wrong: writerow writes exactly one lineterminator, which is why the same code produces a clean file on Linux.