Reading Papers Like a Scientist

Why read papers at all?

Research is a conversation, and papers are how the field talks. Every idea you've met so far, from gradient descent to backpropagation, reached the world as a paper. To contribute to the conversation, you have to be able to listen to it.

There's a problem, though: machine learning publishes an overwhelming amount. Thousands of new ML papers appear on arXiv every month. Nobody reads them all, and nobody reads most of them carefully. The skill isn't reading more. It's reading strategically: deciding quickly what deserves your attention, then reading those few papers deeply.

The anatomy of an ML paper

Most machine-learning papers follow the same skeleton. Once you know what each part is for, you know where to look for what you need.

The single most useful paragraph is usually at the end of the introduction, where strong papers spell out their contributions, often as a bulleted list beginning "Our contributions are…". Those bullets are the authors' claims. Everything else in the paper is either motivation for them or evidence for them.

The three-pass method

In 2007, computer scientist S. Keshav published a two-page guide that has since been assigned to generations of graduate students. His advice: never read a paper once, front to back. Read it in up to three passes, each deeper than the last, and stop as soon as you have what you need.

1. 1

   Pass 1 · The bird's-eye view (5–10 minutes)

   Read the title, abstract and introduction, the section headings, and the conclusion. Glance at the references. Then answer Keshav's five Cs: Category (what kind of paper is it?), Context (what does it build on?), Correctness (do the assumptions look valid?), Contributions (what's new?) and Clarity (is it well written?). Most papers end here, and that's fine.

2. 2

   Pass 2 · The content (up to an hour)

   Read with care, but skip proofs and fiddly details. Study the figures and tables closely: check the axes, the baselines and the error bars. Mark references you'll want to follow up. Afterwards you should be able to summarise the paper's main thrust, with evidence, to a colleague.

3. 3

   Pass 3 · Virtual re-implementation (hours)

   Try to recreate the work yourself: make the same assumptions and see whether you'd reach the same conclusions. Challenge every assumption and notice what's missing. Keshav estimates four or five hours for a beginner, and about an hour for an experienced reader. Reserve this pass for papers you must master, such as ones you're reviewing or building upon.

Dissect an abstract

An abstract compresses the whole paper into a handful of sentences, and good ones follow a predictable sequence of moves. Recognising those moves lets you extract a paper's claims in seconds.

Reading critically

Understanding a paper is only half the job. The other half is judging it. Bring these questions to every paper you read beyond the first pass:

• What exactly is claimed? Find the contributions and restate them in your own words.
• Compared to what? Are the baselines strong, recent and tuned as carefully as the new method?
• Is the effect bigger than the noise? Look for multiple seeds, error bars and confidence intervals (Chamber 7).
• Does the evaluation match the claim? A method claimed to be "robust" but tested on one dataset has only shown something narrower.
• What do the ablations show? Does each component actually earn its place?
• What is missing? Read the limitations section, or notice that there isn't one.
• Is there a simpler explanation? More compute, more data or more tuning can masquerade as a better idea.

A reading workflow

Reading without writing evaporates. Researchers who read a lot keep a reading log, whether that's a reference manager such as Zotero, a spreadsheet or a folder of plain notes. For each paper, capture at least this:

## <Paper title> (<first author>, <year>)
**Pass reached:** 1 / 2 / 3
**Five Cs:** Category · Context · Correctness · Contributions · Clarity
**In three sentences:** what they did, what they found, why it matters.
**Key evidence:** Table/Figure … shows …
**What I doubt:** …
**One question, one idea:** …

The last line matters most. Research ideas rarely arrive from nowhere. They come from the doubt you wrote down while reading someone else's paper.

Watch: advice from Andrew Ng

In this Stanford lecture, Andrew Ng describes how he reads papers efficiently and how to build a machine-learning career. Its owner doesn't allow embedding, so it opens on YouTube.

Watch on YouTube