Note 10 · Image compression · Est. 1992

A photograph, mostly deleted.

Watch a photo become numbers, lose its color detail, dissolve into waves, and rebuild tiny — the real math of JPEG, computed live in your browser.

Sep 2026 ~8 min 5 scenes + 1 lab Mode: computed live

01One photo, taken apart

The photograph above this text does not exist as a file. Every dot of it was computed by this page the moment you arrived, from a fixed recipe: a coast at golden hour, a lighthouse, a town switching its windows on. Deterministic — the same photo for you as for me, and the same one every number below is measured on.

In 1992, a working group called the Joint Photographic Experts Group published the standard the world now calls JPEG, and it has been the default way photographs travel ever since. Its promise sounds impossible: compression around ten to one is routine, with no loss you can see. The trick is not clever arithmetic. It is knowing, precisely, what you will fail to see.

This article is that whole pipeline, run on this one photo, in front of you: the raw size, the color cut, the wave transform, the rounding, and the final measured file. Nothing is canned — every figure is computed as you scroll.

The photo is 768 × 576 — 442,368 dots. All arithmetic on this page derives from those dimensions and from the pixels this page generated.

Start with the whole picture: 768 × 576 dots, 442,368 of them, each an exact color fixed by the recipe. This is the raw material — there is nothing else in the file.

Zoom in, to the lighthouse. The smooth photograph is a grid of squares. There are no curves anywhere — only tiles of solid color, packed edge to edge.

Zoom onto the red band, forty-eight times. One tile of that grid is three numbers: how much red, how much green, how much blue. Values 0–255 — three bytes. That is all a dot ever is.

So the honest size of this picture is 442,368 × 3 = 1,327,104 bytes — about 1.27 MB. That is the number everything below has to beat.

The JPEG file will keep only a slice of it. Five cuts follow, all computed from this exact photo, in your browser, as you scroll.

02Your eye is the accomplice

Everything JPEG throws away, it throws away because of two facts about your visual system. First: you resolve brightness far better than color. The world could halve the resolution of every hue around you and, outside of hard color edges, you would not notice. Second: you notice coarse structure — where light meets dark — long before you notice fine grain. Sharpness, the thing photographers prize, is mostly the thing your eye is worst at counting.

The first cut uses fact one, and it happens before anything we would call “compression.”

The first move isn't a cut — it's a change of accounting. Each dot's red-green-blue triple is rewritten as three other numbers: Y, its brightness, and Cb, Cr, its two color offsets.

Look at Y alone: the whole black-and-white photograph is in there, at full resolution. Color television was built on exactly this fact — brightness was the signal everyone already owned.

Now the color planes. They are soft — watercolor washes. Edges live in brightness; color mostly says which wash goes where.

So JPEG keeps every Y sample, and only one Cb and one Cr per 2×2 square — “4:2:0”. Three of every four color samples are never written: 1,327,104 samples become 663,552. Half the photo, already gone.

Recombined: the largest change anywhere is — levels of 255 — on the lighthouse's red edge, where averaging is hardest to hide. The average change is —. Find it by eye. I'll wait.

The eye forgives what it never measured.

03Blocks into waves

The brightness plane now gets divided into squares of 8 × 8 samples — 6,912 blocks on this photo — and each block travels the rest of the machine alone.

What happens next is the famous part: the discrete cosine transform. It is only a change of vocabulary — the same 64 numbers, rewritten as 64 weights on fixed wave patterns. Run by itself, it throws nothing away.

Each block: 64 numbers. Zoomed here on the lighthouse's foot, where tower, door, rock, and sea meet — the busiest square on the whole photo.

JPEG rewrites those 64 numbers as 64 weights on this fixed dictionary of waves. Every tile is a pure cosine ripple — and any 8×8 block whatsoever is exactly some mix of these 64.

The dictionary runs rough to fine: a flat average in the corner, faster ripples outward. Photographs live at the rough end; the fine tiles hold grain and hard edges.

Rebuild the busiest block from its single flattest weight: a gray smear — but the smear is already the right gray. One number, one guess, and it isn't wrong.

Fifteen weights in: light side, dark side, the shape of the door. The remaining forty-nine weights are mostly grain. Your eye already stopped caring.

Now the whole photo, rebuilt block by block from its first k waves. Scroll and k runs to 64 — the world sharpens into place. At k = 64 the transform is exact: nothing has been lost yet.

04Where the loss actually lives

The transform loses nothing — run it backwards and your block returns to the last decimal. The compression comes from the next step, and it is almost embarrassingly simple: division, then rounding.

Each weight is divided by an entry in a small table and rounded to the nearest whole number. Flat waves get small divisors; fine waves get big ones. Small weights on fine waves — grain, mostly — divide down to less than half and round to zero. The classic tables ship in the standard's Annex K; every quality setting you have ever chosen is just a number that scales this table up or down.

The busiest block's 64 weights. The corner value is the block's average; the rest measure ripple at each of the dictionary's frequencies. Still reversible — these numbers rebuild the block exactly.

Now divide each weight by its table entry at quality 75 — the classic mapping — and round to the nearest integer.

— of the 64 weights rounded to zero. Across the entire photo, — of all weights are now zero — before any clever coding has happened. This table is the meaning of “quality.”

Decoding multiplies back by the table. Zeros stay zero. The missing waves do not come back smaller — they do not come back. That rounding is the entire loss in “lossy.”

LAB THE QUALITY DIAL MEASURED — YOUR BROWSER'S ENCODER
ORIGINAL CROP · 4× ZOOM
JPEG 75 · SAME CROP
75
— bytesMEASURED FILE SIZE, FULL PHOTO
—RAW : JPEG
—OF RAW KEPT
—WEIGHTS = ZERO
THE LUMINANCE ROUNDING TABLE AT THIS SETTING — SMALL DIVISORS KEEP, LARGE ONES DELETE. CLASSIC SCALING, AS LIBJPEG DOES IT.

05Turn the dial

Everything above is now yours to break. The slider scales the rounding table exactly as encoders do, and the photo is re-encoded by your browser's real JPEG engine on every change.

WHAT TO WATCH
Drag quality toward 10 and the 8×8 grid announces itself; edges grow a shimmering fringe — mosquito noise, the fine waves that survived rounding by error. The byte figure is measured, not modeled: it is the size of the actual JPEG file your browser just wrote for this exact photo.

06Why zeros are almost free

A wall of zeros is cheap to store if you refuse to store it. The block's weights are read out in a zigzag — corner outward, rough to fine — so the zeros cluster at the end of the journey, and one end-of-block symbol can stand in for dozens of them.

The surviving numbers are then written with Huffman codes: short bit patterns for common values, long ones for rare. The measured sizes in the dial above already include all of this — it is the last stage, not a bonus one.

07The whole machine

That is the entire trick, end to end. Your eye's weakness for color set the first cut. The wave dictionary made detail countable. Rounding made the deletion gentle and tunable. Entropy coding made the survivors cheap. Put together, the photo you scrolled past at the top becomes this:

RAW SAMPLES · 1,327,104 B
JPEG QUALITY 75 · —
—RAW : JPEG
—OF RAW KEPT
—NEVER STORED
JPEG doesn't store a photograph. It stores a forecast of your attention — brightness first, rough color second, fine grain last.
Scope: this is baseline sequential JPEG, the mode cameras and browsers write by default. The photo is synthetic by design, so the page ships no assets and every number is reproducible; its grain and glitter are tuned to behave like a sensor's. Successors — WebP, HEIF, AVIF, JPEG XL — inherit this same skeleton with better transforms and better entropy coders. Sources: ITU-T T.81 (1992), Wikipedia · JPEG, Chroma subsampling, libjpeg jcparam.c.