A textbook that is written while you read it, for you.
Ten things Sullivo does, each shown with a real example: a course on inference optimisation, written for one learner, and a research paper read with another. Nothing on this page is a mock-up. The questions can be answered, the figures played with, and the page written on.
Type a subject, or a paragraph about what you want and why: “I write software and want to understand why LLM serving is expensive.” No syllabus to pick from and no level to choose. Sullivo works out the rest from what you tell it and how you answer.
Or bring your own reading: a paper, a textbook chapter, any PDF. That is § 9.
Some subjects people have asked for
learn ›inference optimisation
Category theory
Music theory
The fall of Byzantium
The electrical grid
Linear algebra
Quantum mechanics
The Lean theorem prover
Atomic orbitals
Inference optimisation
AI for business
§ 2 · Calibration
It finds out where you start, rather than guessing.
First, one question about you, written for your subject. The answer sets how much ground the course lays before its first equation.
Then, if you like, a short exploratory test of about six questions, each from a different part of the subject. Every question has “I don't know this yet”, and asks how sure you are. A confident wrong answer is worth knowing about. It is the kind of mistake that quietly breaks everything built on top of it.
Nothing here is graded. It is a map of where to begin.
A real example
Which of these sounds most like you?
Not graded. It tells Sullivo how much scaffolding to lay before the first equation.
pick one to see what it changes
exploratory test · Linear algebra & arithmetic
Multiplying an (M × K) matrix by a (K × N) matrix takes roughly how many floating-point operations?
How sure are you?Fairly sure
Each of the M·N outputs is a K-long dot product: K multiplies and K adds, so 2MKN.
The learner this course was written for answered M · K · N, fairly sure. That is a reasonable thing to count, and it is not a FLOP count. See what chapter I did about it ↓
§ 3 · The plan
Then it plans a whole course, to a goal you can check.
From your answers, Sullivo plans a course: a goal stated as something you will be able to do, the chapters to get there in order, and a capstone project to prove it.
Only the plan is written up front. Each chapter is written when you reach it, so it can use everything the earlier ones found out about you.
A real example
Inference Optimisation: Why Your GPU Is Waiting on Memory
Goal By the end you can look at any model, precision, batch size and context length, and predict which resource — FLOPs or memory bandwidth — limits token rate, then choose the optimisation that actually moves the number.
I The Two Currencies: FLOPs and Bytes
II Where Bytes Live and How Fast They Move
III Arithmetic Intensity and the Roofline
IV Prefill and Decode: One Model, Two Regimes
V Sizing the KV Cache in Bytes
VI Precision as a Bandwidth Lever
VII Batching: Turning a GEMV Back into a GEMM
VIII Continuous Batching and the Scheduler
IX Speculative Decoding: Spending Idle FLOPs
X Attention, Fusion, and Long Context
XI Reading a Real Serving Configuration
Capstone A single-file Python performance model (inferbudget.py) that takes a model config (layers, hidden size, heads, parameter count, precision, KV precision) and a hardware spec (FLOP/s, HBM bandwidth, SRAM size) and returns, for any batch size and context length: FLOPs per token, bytes moved per token, arithmetic intensity, whether prefill and decode are compute- or bandwidth-bound, and the resulting tokens/s ceiling.
§ 4 · The chapters
Chapters written for one reader.
Each chapter is a proper piece of writing, with an argument and examples drawn from your world. New terms are set in bold and defined in the margin. Asides go in the margin too, so the main line stays clear. Maths is typeset, figures can be played with, and quick checks sit in the text where you would want to stop and test yourself.
Below is part of chapter I of the course above, exactly as it was written. It opens like this: “You can rent a modern GPU for roughly the price of a sandwich per hour. In 16-bit precision it performs on the order of 300 trillion floating-point operations per second — 300 TFLOP/s. Now ask that same GPU to serve a 7-billion-parameter language model to one user, and it will emit something like 30 tokens a second. Each token costs it about 14 billion FLOPs. Thirty tokens a second is 420 billion FLOPs a second, which is roughly 0.14% of what the chip can do.”
Start from a line of code you've written a hundred times: C = A @ B, where A is M×K, B is K×N, and C is M×N. Forget the GPU. Count by hand.
Every entry of C is a dot product: multiply K pairs of numbers, add the results. So each entry costs Kmultiply-addmultiply-add — one multiplication immediately followed by an addition, a⋅b+c — the atom of all matrix arithmetic. There are M⋅N entries. The layer therefore performs M⋅N⋅K multiply-adds.
Now the one subtlety in this section, and the reason the exploratory question you answered earlier had a trap in it. Hardware specifications count a multiply-add as twoFLOPFLOP — floating-point operation: a single addition, subtraction, multiplication or division on floating-point numbers — one for the multiply, one for the add. So:
FLOPs=2⋅M⋅K⋅N
Sanity-check it on the smallest case you can picture. M=1,K=2,N=1: the layer computes a1b1+a2b2. Two multiplies, two adds, four FLOPs — and 2⋅1⋅2⋅1=4. ✓
Scale it up. M=1, K=N=4096: that is 2⋅4096⋅4096≈33.6 million FLOPs. At 300 TFLOP/s the arithmetic takes about a tenth of a microsecond, which is a length of time with no intuitive meaning. Hold the number anyway.
Figure 1.1 Count one output cell, then all of them
Click any cell of C to watch its dot product happen, one multiply-add at a time.
written for you · from your exploratory test — you answered M⋅K⋅N and were fairly sure
You counted multiply-adds, not FLOPs
On the exploratory test you said an M×K by K×N matmul costs M⋅K⋅N. You were counting multiply-adds, and that count is exactly right: there are precisely MKN of them. What the question wanted was FLOPs, and a multiply-add is two of those. It feels like a trick, and in a sense it is — the convention is that spec sheets quote 2 FLOPs per multiply-add, so if you compare MKN against a GPU's quoted TFLOP/s you'll conclude the machine is twice as fast as it is. The factor of 2 never changes which resource limits you, because it's a constant. But this course compares counts against hardware numbers constantly, so we carry it everywhere.
The red section is a bridge. Sullivo wrote it because of one answer in the exploratory test: M · K · N, fairly sure. That answer counts multiply-adds, and the course is about to compare FLOP counts against spec sheets. Left alone, it would make every later number wrong by a factor of two. So the chapter stops and deals with it, before building on top of it.
§ 5 · Figures
Figures you play with, not pictures you look at.
Every chapter has figures drawn for it: small programs, each built around one idea, with sliders, buttons and things to drag. Sullivo writes a brief for each (what to show, and what the reader should notice), then draws it, and checks it in a real browser before it reaches you.
These two are from the same chapter. Try them.
Live, from chapter I
Figure 1.2 Two bars for one layer
Same layer, same weights. Drag M and watch the arithmetic bar leave the traffic bar behind.
Figure 1.3 Where the ratio stops improving
Raising M lifts the ratio, but not forever. The ceiling is set by the layer's width, not by the batch.
§ 6 · The test
A test that decides how the next chapter is written.
Each chapter ends with four or five questions on its own ideas. It isn't a grade. It is how the next chapter knows what you are carrying forward.
Whatever you miss is gone over again in the next chapter, in a bridge written around the mistake you actually made. Whatever you answer well is not explained twice. The course you finish is not the course that was planned. It is the one you needed.
A real example
Before you go on · question 3 of 5
In the byte count b(MK+KN+MN) at M=1, which term dominates, and what does it represent?
At M=1, MK and MN are each 4096 elements while KN is 16.8 million. The weights are the traffic, by a factor of four thousand. This is the fact that Chapter 2 turns into a speed limit.
Miss it, and chapter II opens by mending exactly this.
§ 7 · Explain it back
Then it asks you to explain it in your own words.
Understanding a chapter and being able to explain it are different things. After the test, you explain the chapter in your own words. Sullivo plays someone who has never heard of the subject and knows only the chapter's title, and asks the questions a newcomer would ask.
Then an examiner, who has read the chapter, tells you what you got right, and what you could not yet put into words, quoting you. Those gaps go to the next chapter too. A gap in your own explanation counts for more than a wrong multiple-choice answer.
I've honestly never come across "The Two Currencies: FLOPs and Bytes". Could you explain it to me? Start wherever makes sense — I'll stop you if I get lost.
learner
GPUs perform large matrix multiplication. this contains the basic unit of one multiplication and one addition so we can calculate how many operations we doing one large matric multiplication. so if we have two matrixes : A which is of M x K B which is of K x N C = A x B -> hence of the dimensionality MxN …
listener
Got it so far — one thing I'm fuzzy on: you said we don't want the GPU to "lead or lag" relative to this ratio. What does leading or lagging look like in practice — what's the GPU doing in each case?
What the examiner said
Correctly counted FLOPs as 2MKN and knew a multiply-add counts as two FLOPs
Correctly described the traffic side as A, B and C each touched once, with b bytes per value (2 fp16, 1 fp8, 4 fp32)
Assembled the ratio 2MKN / b(MK + KN + MN) and recognised it as the central quantity
Not yet Comparing the layer's ratio to the machine's ratio to see which resource binds.
“this ratio is a central determinant in inference optimisation since we dont want the GPU to lead or lag from this .”
"Don't want it to lead or lag" isn't a mechanism — a machine has its own FLOPs-per-second over bytes-per-second, and you compare the layer's FLOPs-per-byte against it. Below the machine's number the arithmetic units finish and sit waiting on data (memory-bound); above it, memory keeps up and arithmetic is the limit.
§ 8 · The tree beneath it
Every subject rests on others. It shows you which.
Next to your book is the tree beneath it: what your subject rests on, then what those rest on, a few levels down. Each node asks one thing: do you know it?
Say yes and it is cleared. Say no and it is marked shaky, and you can learn it as a course of its own, or take a four-question check if you are not sure. Every tree you grow joins your forest, so you can see how much ground you have covered across subjects.
Click a node to try it.
The real tree under “Inference optimisation”
Inference Optimisation
§ 9 · Follow along
Bring your own reading, and read it with a companion.
Upload a PDF: a paper, a chapter, a manual. Sullivo reads it first and works out what it takes for granted. Then it gives you a short readiness test on exactly that.
If something is missing, it writes you a short primer covering only that: nothing the document teaches itself, and nothing you already know. Then you read the document itself, as it is, with a margin of notes written for you beside the passages they explain: plain-words versions, connections to what you already know, and figures.
And when you are stuck, ask the companion. It has read exactly as far as you have and no further. It will never spoil what comes later. If the author answers your question further on, it tells you where, and gives you a hint to carry until then.
A real reading: a 28-page interpretability paper
Integrated and Cross-Architecture Interpretation of LLM Reasoning · 28 pages · PDF
What it takes for granted, found by reading it
Transformer hidden states & the unembedding (logit lens)
Gaussian RBF kernel & bandwidth saturation
Probability distributions & Jensen–Shannon divergence
Quartiles, IQR & the Tukey outlier rule
Set intersection, union & Jaccard overlap
Mann–Whitney U, rank-biserial effect size & Bonferroni correction
Margin notes it wrote for pages 1–6
§ 10 · Annotate
Write on the page, with whatever you write with.
Turn on annotate and write over the book or your PDF: underline, circle, scribble in the margin, highlight. It works with a mouse, a stylus on a tablet, or a drawing tablet like an XP-Pen or Wacom. Pressing harder draws a thicker line, and the pen's eraser end rubs out.
A finger scrolls rather than draws, so a tablet works the way you'd expect. Your ink is kept with the page, and is there when you come back.
Try it here
The machine you are paying for is idle 99.8% of the time. Where does the time go? Not into arithmetic. Into waiting for numbers to arrive. Before anything can be multiplied by anything, the weights have to be hauled out of memory and into the arithmetic units. A GPU does arithmetic far faster than memory can feed it, and the gap is not small.
try it: draw on the paragraph
Read a whole chapter, then start your own.
The chapter above is open to anyone, figures and checks included. Your own course starts with one chapter, free; no card needed.