Change: The Notation of Calculus

Discover: one line that fools a classifier

In 2014 three researchers at Google showed that you can make an image classifier fail by changing each pixel so slightly that no human would notice. Here is the whole recipe:

Add $\boldsymbol{\eta}$ to a photo of a panda, with $\epsilon = 0.007$, and a network that was 57.7% sure it saw a panda becomes 99.3% sure it sees a gibbon. The picture looks unchanged.

You can already read parts of this line. Bold letters are vectors (Chamber 1), $J(\boldsymbol{\theta}, \boldsymbol{x}, y)$ is a function of three arguments (Chamber 4), and $\epsilon$ is a small number. The new piece is the upside-down triangle, $\nabla$, with a subscript. Before learning what it means, make a guess:

$\nabla$ is the notation of calculus, the mathematics of change. Every model you'll ever train is trained with it, and papers use it on almost every page. This chamber teaches you to read it, along with its relatives: $\Delta$, $\lim$, $d$, $\partial$, $\int$ and $\argmin$. It is not a calculus course. You'll derive one derivative by hand, and after that the notation is the point.

Learn: from Δ to d

Change starts with a difference. If $x$ moves from $x_0$ to $x_0 + h$, the change in $x$ is $\Delta x = h$, read “delta x”, with capital delta for “difference” (Chamber 1). If $y = f(x)$, the change in $y$ is $\Delta y = f(x_0 + h) - f(x_0)$. Their ratio is rise over run:

Geometrically, it's the slope of the straight line through two points of the curve, a line called a secant. If $f$ gives a car's position at time $x$, it's the average speed over the interval.

But what is the speed at the instant $x_0$? Setting $h = 0$ gives $\frac{0}{0}$, which means nothing. The way out is a limit:

read “the limit, as h tends to zero, of …”. It asks what the ratio gets closer and closer to as $h$ shrinks, while $h$ itself is never zero. When the change is shrunk in this way, $\Delta$ becomes $d$: $dx$ is an infinitesimally small change in $x$, and $\frac{dy}{dx}$ is what $\frac{\Delta y}{\Delta x}$ becomes in the limit.

Learn: the derivative, from the limit

The derivative of $f$ at $x$ is that limit:

Here is the one derivation this chamber does by hand, for $f(x) = x^2$. Watch each step, because the same moves work for any polynomial.

Step (1) substitutes $f(x) = x^2$, (2) expands the square, (3) cancels the two $x^2$ terms, and (4) divides by $h$. So the secant slope is exactly $2x + h$, for every $h$. As $h \to 0$ the $h$ vanishes and $2x$ stays put:

Two details matter. The division by $h$ is legal because a limit never sets $h = 0$. And the leftover, $h$ itself, is the error of the secant: at $x = 1$ with $h = 0.5$ the secant slope is $2.5$ while the true derivative is $2$.

Now do it with your own hands. Shrink $h$ and watch the secant turn into the tangent, the line that just touches the curve with slope $f'(x_0)$. Get within $0.001$ at three points to clear the challenge, then switch on floating-point mode for a surprise.

Learn: four ways to write a derivative

Calculus was invented twice, by Newton and by Leibniz, and it has carried several notations ever since. You'll meet all of them, sometimes in the same paper:

Leibniz's $\frac{dy}{dx}$ shines in the chain rule. If $y$ depends on $u$, and $u$ depends on $x$, then

It looks like cancelling fractions, which makes it easy to remember, though $\frac{dy}{du}$ isn't really a fraction. An example: for $y = e^{2x}$, let $u = 2x$. Then $\frac{dy}{du} = e^{u}$ and $\frac{du}{dx} = 2$, so $\frac{dy}{dx} = 2e^{2x}$. A neural network is a long chain of functions, and backpropagation is this rule applied link by link.

Learn: many inputs, ∂ and ∇

A loss depends on many parameters at once. To ask how it changes, we change one input and hold every other fixed. That's a partial derivative, written with a curly $\partial$ (read “partial”, sometimes “del”):

In practice you differentiate as usual and treat the other variables as constants. For a single data point $(x, y)$ and a line with slope $w$ and intercept $b$, the squared error is $\ell(w, b) = (wx + b - y)^2$. With the chain rule,

Stack all the partial derivatives into a vector and you have the gradient, $\nabla f$, read “grad f” or “nabla f”. It has one entry per input, so it has the same shape as the input (Chamber 6). When a function has several kinds of arguments, a subscript says which ones are being varied: $\nabla_\theta \mathcal{L}$ is the gradient of the loss with respect to the parameters only. The Adam paper puts it in one line: $g_t = \nabla_\theta f_t(\theta)$ denotes “the vector of partial derivatives of $f_t$, w.r.t $\theta$”.

Here is a short proof that uses exactly this notation. It also explains the adversarial example you met in Discover.

Claim. For fixed $\mathbf{w} \in \mathbb{R}^n$, the function $f(\mathbf{x}) = \mathbf{w}^\top \mathbf{x}$ has gradient $\nabla_{\mathbf{x}} f = \mathbf{w}$.

Proof. Write the dot product as a sum (Chamber 6): $f(\mathbf{x}) = \sum_{i=1}^{n} w_i x_i$. Fix an index $k$ and take the partial derivative with respect to $x_k$. Every term with $i \neq k$ doesn't contain $x_k$, so it's a constant and contributes $0$. The term $w_k x_k$ is a constant times $x_k$, with derivative $w_k$. So $\frac{\partial f}{\partial x_k} = w_k$ for every $k$, and stacking the partials gives $\nabla_{\mathbf{x}} f = (w_1, \ldots, w_n)^\top = \mathbf{w}$. $\blacksquare$

Now think about a linear model. Its cost depends on the input only through $\mathbf{w}^\top \mathbf{x}$, so by the chain rule $\nabla_{\boldsymbol{x}} J$ is this $\mathbf{w}$ times a single number. The signs of the gradient are the signs of $\mathbf{w}$ (or all flipped), and the attack nudges every feature by $\epsilon$ in whichever direction hurts the prediction. Problem 3 runs it.

Two matrices round out the family. The Jacobian collects the first partials of a function with several outputs, one row per output. The Hessian collects the second partials of a single-output function, and describes its curvature. You'll mostly meet them as notation in theory sections: $J \in \mathbb{R}^{m \times n}$ for $\mathbf{f} : \mathbb{R}^n \to \mathbb{R}^m$, and $\nabla^2 f \in \mathbb{R}^{n \times n}$.

Learn: ∫, the continuous sum

The other half of calculus adds things up. Silvanus Thompson's Calculus Made Easy (1910) opens by defusing its two “dreadful symbols”: $d$, “which merely means ‘a little bit of’”, and $\int$, “which is merely a long S, and may be called (if you like) ‘the sum of’”.

That's exactly how to read an integral. Chop the interval from $a$ to $b$ into thin strips of width $\Delta x$, add up height times width, and let the strips get thinner:

The $\Sigma$ of Chamber 5 becomes $\int$ and $\Delta x$ becomes $dx$. Read the left side “the integral from a to b of f of x, d x”. The $dx$ is part of the notation: it says which variable the strips are measured along. Geometrically, it's the area under the curve. For example, $\int_0^1 x\,dx = \frac{1}{2}$, the area of a triangle with base 1 and height 1.

In machine learning, integrals appear mostly in probability, where continuous sums replace finite ones. The expected value in Chamber 8, $\mathbb{E}[X] = \int x\, p(x)\,dx$, is a weighted average written with this notation.

Learn: min, max and argmin

Training a model means finding the parameters that make the loss as small as possible. Papers write this with two operators that look alike but answer different questions:

The subscript names the variable you're allowed to change. The best parameters are then written $\theta^\star = \argmin_\theta \mathcal{L}(\theta)$, read “theta star is the arg min over theta of L of theta”, or more naturally, “the theta that minimises the loss”. Swap min for max and you get $\argmax$: a classifier's prediction is the $\argmax$ of its class probabilities.

Calculus is how you find an argmin. At a smooth minimum the curve is flat, so the derivative (or every partial derivative) is zero there: $\nabla \mathcal{L}(\theta^\star) = \mathbf{0}$. Gradient descent, the subject of the Neural Networks course, walks downhill against $\nabla_\theta \mathcal{L}$ until it gets there.

Argmin is also a general-purpose way to define things. The LIME method for explaining a model's predictions defines an explanation as the solution of an optimisation problem:

Read it: “xi of x is the $g$ in $G$ that minimises L of f, g, pi x, plus omega of g.” The explanation $\xi(x)$ is not a number but a whole simple model $g$, chosen from a set $G$ (Chamber 2) of candidate models, to balance faithfulness to the real model $f$ (the $\mathcal{L}$ term) against simplicity (the $\Omega$ term).

Learn: approximations and big-O

The derivative's most useful job is prediction. Rearranging the secant slope, for small $h$:

This is the first-order approximation: near $x$, a smooth function looks like a straight line. For example, $e^{0.1} \approx e^0 + e^0 \cdot 0.1 = 1.1$, while the true value is $1.1052\ldots$. The same idea with a gradient, $\mathcal{L}(\theta + \mathbf{h}) \approx \mathcal{L}(\theta) + \nabla_\theta \mathcal{L}^\top \mathbf{h}$, is why a small step against the gradient lowers the loss.

How good is an approximation? Big-O notation answers without fussing over constants. In the $x^2$ derivation, the secant slope was $f'(x) + h$: the error is proportional to $h$. In general the forward difference satisfies

read “plus big O of h”: an error no bigger than some constant times $h$ once $h$ is small. The same notation measures cost. The Transformer paper lists self-attention's cost per layer as $O(n^2 \cdot d)$ for $n$ tokens of dimension $d$: double the length of the text, and the work roughly quadruples.

Read beyond the course

Short, free, and each one uses the notation of this chamber on every page.

Papers and lectures

The adversarial examples paper is a good first read at your level. Start with Figure 1 and its caption, the panda. Then read Section 4: one page, with the equation from Discover and a paragraph of results. If you'd like to see why the attack works, read the short Section 3, which uses only a dot product $\mathbf{w}^\top \tilde{\mathbf{x}}$ and a sign. Skip Sections 6 onwards for now.

For the second paper, return to Adam's Algorithm 1, which you met in Chamber 1. This time read its first line inside the loop, the one with the $\nabla$.

Watch

The first video shows where $dx$ and $\int$ come from, by slicing a circle into rings. The second is this chamber's lab in animation: watch for the moment the notation $\frac{ds}{dt}$ is introduced, and for how the derivative of $t^3$ is worked out with a small $dt$, just as you did for $x^2$.

Your turn

Say the symbols, turn them into code, prove two derivatives, then compute: a secant, a minimum and an adversarial attack.

Proofs

The puzzle repeats this chamber's derivation for $x^3$, with a trap or two among the lines. The proof to write combines the limit result with Chamber 6's index notation.