Data Engineering for Beginners: Part 1.2 – Your First Data Source: Working With a CSV

You’ve completed Part 1.1 → This is where QuakeFlow gets its first real data

Time to code!

In the last article, you set up your development environment. You installed Python, VS Code, and Git. You created the QuakeFlow project, wrote your first script, and made your first commit.

Your project currently does nothing useful.

That’s about to change.

Today we’re introducing the first thing every data pipeline needs: data.

How do you actually read a CSV file in Python? Open it with Python’s built-in csv module, treat the first row as the header, and read every row after it as a record — no external libraries required to get started. That’s the whole mechanical answer. The rest of this article is about what happens once you actually try it.

And we’re deliberately starting with the most boring possible data source.

A CSV file.

Why Start With a CSV?

QuakeFlow will eventually pull live earthquake data from the USGS Earthquake API. That’s the plan. But we are not doing that yet.

Here’s the problem with starting with an API on day one. An API introduces HTTP requests, authentication, rate limits, JSON parsing, and network errors, all at the same time as you’re trying to learn how to actually process data once you have it.

That’s too much at once.

A CSV file removes all of that. It’s just a file. It sits on your disk. It doesn’t time out, it doesn’t rate-limit you, and it doesn’t change while you’re working with it.

That stability is exactly what you want while you’re learning how to read data, inspect it, and think about its quality.

Later, when we swap the CSV for the live API, almost everything else you build today keeps working. Only the ingestion step changes. That’s not an accident. It’s the whole point of building QuakeFlow in layers instead of one giant script.

Getting the Data

USGS doesn’t just publish a live API. They also publish plain CSV snapshots of their earthquake catalog, updated on a schedule, at fixed URLs.

We’re going to use one of the smaller ones, the significant earthquakes of the past 30 days:

https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv

Download it and save it inside your project, in a new data/raw/ folder:

QuakeFlow/
  data/
    raw/
      all_month.csv
  src/
  tests/
  README.md
  .gitignore
  requirements.txt

We’re calling the folder raw on purpose. This file is exactly as USGS gave it to us. We have not touched it, cleaned it, or transformed it in any way.

Keeping raw data separate from anything you process later is one of the most basic habits in data engineering. If something goes wrong three steps into your pipeline, you want to be able to go back to the original, untouched file and start again.

The CSV I used for the project is found here:

all_monthDownload

Look at the File Before You Write Any Code

Before writing a single line of Python, open the CSV file directly, either in VS Code or in a plain text editor.

Resist the urge to open it in Excel first. Excel is great for a lot of things, but it silently reformats dates, strips leading zeros, and generally hides exactly the kind of quirks you need to learn to notice as a Data Engineer.

You should see something like this:

time,latitude,longitude,depth,mag,magType,nst,gap,dmin,rms,net,id,updated,place,type,horizontalError,depthError,magError,magNst,status,locationSource,magSource
2026-08-12T03:15:22.140Z, -6.1942,130.4123,112.3,5.6,mb,84,42.1,1.284,0.87,us,us7000abcd,2026-08-12T04:02:11.000Z,"Banda Sea",earthquake,7.9,1.9,0.06,12,reviewed,us,us

Two things should jump out immediately.

The first line is the header row. It names every column. Without it, you’d have no idea that the third value in every row is a longitude.

Every line after that is one record. One row equals one earthquake.

Take a moment and just read through a handful of rows. Don’t write code yet. Ask yourself:

  • Are all the rows the same length?
  • Do any fields look empty?
  • Does every row actually look like an earthquake?

You’ll probably already spot a few rows with missing values in columns like nst, gap, or horizontalError. Keep that in mind. We’ll come back to it.

Reading the CSV With Python

Python has a built-in module for reading CSV files, called, unsurprisingly, csv. We don’t need to install anything for this. It ships with Python.

Create a new file:

src/inspect_data.py

Add the following:

import csv

DATA_PATH = "data/raw/all_month.csv"

with open(DATA_PATH, "r", encoding="utf-8") as f:
    reader = csv.reader(f)
    header = next(reader)
    print("Columns:", header)
    print()

    for i, row in enumerate(reader):
        print(row)
        print()
        if i >= 4:
            break

Run it:

python src/inspect_data.py

You should see the column names printed once, followed by the first five rows of data, each one printed as a Python list.

(.venv) (base) jasper@macbookair QuakeFlow % python src/inspect_data.py

Columns: ['time', 'latitude', 'longitude', 'depth', 'mag', 'magType', 'nst', 'gap', 'dmin', 'rms', 'net', 'id', 'updated', 'place', 'type', 'horizontalError', 'depthError', 'magError', 'magNst', 'status', 'locationSource', 'magSource']

['2026-09-26T14:15:22.597Z', '57.701', '-154.863', '76.1', '1.9', 'ml', '13', '253', '0.4', '0.2', 'ak', 'aka2026tccjzs', '2026-09-26T14:16:29.652Z', '28 km WNW of Karluk, Alaska', 'earthquake', '31.8', '9.1506', '0.1', '6', 'automatic', 'ak', 'ak']

['2026-09-26T14:14:42.350Z', '33.338', '-116.86033333333', '14.17', '0.86', 'ml', '44', '32', '0.01569', '0.17', 'ci', 'ci41339527', '2026-09-26T14:18:15.247Z', '2 km S of Palomar Observatory, CA', 'earthquake', '0.18', '0.44', '0.22862563536276', '23', 'automatic', 'ci', 'ci']

['2026-09-26T14:03:57.770Z', '38.828498840332', '-122.79900360107', '1.9800000190735', '1.02', 'md', '16', '73', '0.01004', '0.01', 'nc', 'nc75442392', '2026-09-26T14:05:33.813Z', '7 km W of Cobb, CA', 'earthquake', '0.3', '0.400000006', '0.31', '17', 'automatic', 'nc', 'nc']

['2026-09-26T13:59:54.810Z', '59.218', '-151.773', '43', '2.1', 'ml', '6', '209', '0.4', '0.4', 'ak', 'aka2026tcbwoq', '2026-09-26T14:01:19.055Z', '15 km SSE of Port Graham, Alaska', 'earthquake', '23.3', '34.6123', '0.1', '4', 'automatic', 'ak', 'ak']

['2026-09-26T13:55:41.690Z', '38.769165039062', '-122.70816802978', '1.710000038147', '1.09', 'md', '12', '112', '0.007018', '0.07', 'nc', 'nc75442382', '2026-09-26T13:57:18.639Z', '2 km WSW of Anderson Springs, CA', 'earthquake', '0.32', '0.439999998', '0.22', '11', 'automatic', 'nc', 'nc']

Let’s slow down on what just happened.

csv.reader(f) doesn’t load the whole file into memory as one big blob. It gives you an iterator. Each time you ask it for the next item, it gives you one row, as a list of strings.

We called next(reader) once, up front, specifically to pull off the header row. If we hadn’t done that, the header would show up as if it were a row of data, which it very much is not.

In case you are wondering about the empty print statements – these were added to add an empty line between the header and the separate rows.

Bonus point if you remember to commit!

From Lists to Dictionaries

Working with rows as plain lists works, but it’s fragile. If you want the magnitude of an earthquake, you have to remember that it’s row[4]. If USGS ever reorders their columns, or you’re comparing two files with different layouts, your code silently breaks.

Python’s csv module has a better tool for this: DictReader.

Update your script:

import csv

DATA_PATH = "data/raw/all_month.csv"

with open(DATA_PATH, "r", encoding="utf-8") as f:
    reader = csv.DictReader(f)

    for i, row in enumerate(reader):
        print(row["time"], row["place"], row["mag"])
        print()
        if i >= 4:
            break

DictReader automatically uses the header row to build a dictionary for every row. Instead of remembering that magnitude is the fifth column, you just ask for row["mag"].

(.venv) (base) jasper@macbookair QuakeFlow % python src/inspect_data.py

2026-09-26T14:15:22.597Z 28 km WNW of Karluk, Alaska 1.9

2026-09-26T14:14:42.350Z 2 km S of Palomar Observatory, CA 0.86

2026-09-26T14:03:57.770Z 7 km W of Cobb, CA 1.02

2026-09-26T13:59:54.810Z 15 km SSE of Port Graham, Alaska 2.1

2026-09-26T13:55:41.690Z 2 km WSW of Anderson Springs, CA 1.09

This is a small change, but it’s an important one. Referring to data by name instead of position is more readable, and far less likely to break when the shape of the data shifts slightly. You’ll see this pattern again and again in real Data Engineering work.

Understanding Data Types

Run this small addition to your script:

print(type(row["mag"]))

You’ll get:

<class 'str'>

A string.

Not a float. Not a number of any kind. A string that happens to contain digits.

This surprises almost everyone the first time they see it. A CSV file has no concept of data types. It’s just text, separated by commas. Every single value you read from a CSV, whether it looks like a number, a date, or a boolean, arrives in Python as a plain string.

That means "5.6" and 5.6 are not the same thing to Python. You cannot do this:

average = row["mag"] / 2

That will raise a TypeError, because you can’t divide a string.

If you want to actually do math with magnitude values, you need to convert them yourself:

magnitude = float(row["mag"])

This step, converting raw text into the correct data type, is one of the most basic jobs a Data Engineer does. It seems trivial with one field in a script. It becomes very much not trivial when you’re processing millions of rows and a handful of them have a typo, a stray space, or a missing value where a number should be.

Which brings us to the next section.

Spotting Data Quality Problems

Try converting every magnitude value in the file:

import csv

DATA_PATH = "data/raw/all_month.csv"

with open(DATA_PATH, "r", encoding="utf-8") as f:
    reader = csv.DictReader(f)

    for i, row in enumerate(reader):

        print(row["time"], row["place"], row["mag"])
        magnitude = float(row["mag"])
        
        print(magnitude)
        print()
        if i >= 4:
            break

Depending on the exact file you downloaded, this may run without any issue. Or it may not.

If any row has an empty string in the mag column, float("") will raise a ValueError, and your script will crash.

This is not a bug in your code. This is real-world data behaving like real-world data.

Rather than letting one bad row take down the entire script, wrap the conversion in a try-catch statement and decide what should happen when it fails:

import csv

DATA_PATH = "data/raw/all_month.csv"

valid_rows = 0
skipped_rows = 0

with open(DATA_PATH, "r", encoding="utf-8") as f:
    reader = csv.DictReader(f)

    for row in reader:
        try:
            magnitude = float(row["mag"])
            valid_rows += 1
        except ValueError:
            skipped_rows += 1
            continue

print(f"Valid rows: {valid_rows}")
print(f"Skipped rows: {skipped_rows}")

Now, instead of a crash, you get a report. Some Data Engineering decisions are actually this simple: don’t just let bad data break the pipeline, count it, and decide later what to do about it.

Other quality questions worth asking yourself while looking at this dataset:

  • Are there duplicate id values?
  • Do any place values look suspicious, empty, or clearly wrong?
  • Are the time values consistent in format?
  • Do latitude and longitude values fall within valid ranges?

You don’t need to fix any of this yet. Right now, the goal is simply to notice it. Every one of these questions will come back later, once we start building actual validation into the pipeline.

Start Thinking Like a Data Engineer

Notice the shift in this article compared to the last one.

Part 1.1 was about tools. This one was about the data itself, and specifically, about not trusting it.

A beginner assumes a CSV file contains clean, well-formed, complete data, because it usually does in tutorials. A Data Engineer assumes the opposite by default, and treats “the data is actually fine” as something to verify, not something to assume.

That mindset, more than any specific tool or library, is the core of the job.

Commit Your Work

If you have not committed yet, add the raw file and your new script to Git:

git add .
git commit -m "Add first CSV data source and inspection script"

Your project should now look like this:

QuakeFlow/
  .venv/
  data/
    raw/
      all_month.csv
  src/
    main.py
    inspect_data.py
  tests/
  .gitignore
  README.md
  requirements.txt

Key Takeaways

You’ve now:

  • Downloaded your first real dataset
  • Learned the difference between raw and processed data
  • Read a CSV file with Python’s built-in csv module
  • Used DictReader to access columns by name instead of position
  • Learned that every value from a CSV arrives as a string
  • Converted string values into usable data types
  • Handled a data conversion failure instead of letting it crash your script
  • Started asking the kinds of questions a Data Engineer asks about data quality

What Comes Next

Part 1.3 — Turning Inspection Into Ingestion

Right now, inspect_data.py is a script you run manually to poke around a file. That’s useful, but it’s not a pipeline — and doing every type conversion and quality check by hand, one field at a time, doesn’t scale much further than what you just wrote.

That’s exactly the point where we reach for a library built for this problem: pandas. Next, we’ll rebuild this same inspection logic on top of it, turn it into a proper ingestion step, and see directly why “read a CSV, check its types, flag the bad rows” is something pandas was designed to make easier.

After that, in Part 1.4, we’ll go a step further and load the cleaned data into an actual PostgreSQL database — which is also where SQL enters the series for the first time.

Newsletter Updates

Enter your email address below and subscribe to our newsletter

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *


Trending now

Data Engineering for Beginners: Part 1.1 – Setting Up Your Data Engineering Environment
Background
Data Engineering for Beginners: Part 1.2 – Your First Data Source: Working With a CSV