Module 1 · Arcades and foundations

Hardware constraints: sprites, tiles, memory, fixed point

Why 128 bytes of RAM, no FPU and no frame buffer forged the permanent primitives of an engine — tiles, sprites and fixed-point arithmetic — which you still use today.

~16 min

The gist in 20 seconds

The hardware of the 1970s–80s was destitute: the Atari 2600 had 128 bytes of RAM, a 6502 with no multiply or divide, and on cheap machines not even a frame buffer. Out of that came three techniques that outlived all of the hardware: the tilemap (the screen is a grid of indices into a table of tiles, not a million pixels → tens of times less memory), sprites (small bitmaps that the hardware composites over the background) and fixed-point (fractions as integers: store value×2¹⁶, compute in integers, because there is no float in the hardware). This is not archaeology: tiles → texture atlases, fixed-point → neural network quantization, the palette → lookup tables.

The mechanism

Three constraints and three answers to them. All three are about one thing: don't store the raw thing, store a reference and reuse it.

The memory wall and "racing the beam"

The Atari 2600 has 128 bytes of RAM and no frame buffer. You can't "draw the picture into memory and show it": there is nowhere to keep it. So the processor races the beam: the CRT beam physically runs across the screen left to right, line by line top to bottom, ~60 times a second, and the CPU has to set the right registers of the TIA video chip for a scanline cycles before the beam gets there. A scanline gets only ~76 cycles; the code stays just ahead of the beam the whole time — hence the "race": fall behind by a cycle and the line draws garbage. The image exists only as a stream in time, not as an array of pixels. The NES (1983) adds 2 KB of RAM and a separate graphics processor, the PPU, with hardware tiles and sprites — and that modest expansion opens up a whole class of games (scrolling, large worlds).

The tilemap — the screen as a grid of indices

Storing the screen pixel by pixel is expensive. A full-screen buffer for Pac-Man (224×288 px at a byte per pixel) is

224 \times 288 = 64512 bytes \approx 63 KB

— and in 1980 there simply is no such RAM. The fix: cut the screen into a grid of 8×8 px tiles; keep not pixels in memory but a tile index in each cell. The tile artwork itself sits in ROM once and gets reused.

The closest analogy for a programmer is text rendering: a terminal doesn't store a picture of the page, it keeps a grid of character codes (A=65, B=66…), and the letter shapes (the font) live separately, once. A tile is exactly the same thing, except the "font" is not letters but pieces of graphics (wall, dot, corner). Old hardware literally called tiles "characters" and their memory CHR-ROM: the same mechanism as a text display, only the stamps are graphics.

The NES map is a nametable of 32×30 tiles:

32 \times 30 = 960 bytes versus \approx 63 KB (~67 times less)

Bonus: scrolling is "free" — shift the indices and draw one new column of tiles instead of redrawing the whole screen. The downside is that the world looks "checkered" from reused blocks, and that is the recognizable aesthetic of the entire 8-bit era.

Screen = a small table of indices + reusable tiles. Memory goes to the map, not the pixels.

Sprites — moving objects over the background

Whatever moves (the player, enemies, bullets) is not part of the tilemap but a sprite: a small 8×8 or 16×16 bitmap that the video chip composites over the background in hardware. The name comes from sprite (a fairy, a spirit): such images seem to "float" above the background independently of it, like spirits over a stage. On the NES, OAM (object attribute memory) is 256 bytes = 64 sprites at 4 bytes each (y, tile number, attributes, x). But there is a hard limit: no more than 8 sprites per scanline (secondary OAM holds exactly 8). The ninth and beyond on that line are not drawn. To avoid losing objects entirely, games rotate sprite priority every frame — and instead of "gone" you get the familiar flicker in dense scenes: each sprite is visible every other frame.

Fixed-point — fractions without an FPU

The 6502 and the early 68000 have no hardware floating point, and often no multiply either. But physics needs fractional speeds. The trick: store a number as an integer scaled by a power of two. The Q16.16 format gives 16 bits to the integer part and 16 to the fraction; the real value from the "raw" integer r is:

v_{real} = \frac{r}{2^{16}} = r \cdot 2^{- 16}

Addition and subtraction are plain integer ops (1 cycle versus 20+ for emulated float). The integer part (the pixel on screen) comes out with a shift:

posFixed += velFixed; // physics step — integer addition screenX = posFixed >> 16; // integer part = pixel coordinate

Multiplication is the one subtlety: multiplying two Q16.16 numbers adds their fractional parts together (32 bits), and the result has to be shifted back by 16, keeping the intermediate in a wide register so the high bits aren't lost:

c = (a \cdot b) ≫ 16

Q16.16: one 32-bit integer. The top 16 bits are pixels, the bottom ones the subpixel fraction. The "point" is fixed between them.

A worked example. A speed of 0.5 px/tick in Q16.16 is r = 0.5 · 65536 = 32768. Five ticks: posFixed = 5 · 32768 = 163840; the pixel is 163840 >> 16 = 2 (exactly 2.5 px, but on screen a whole 2; the 0.5 fraction accumulates and delivers the third pixel on the sixth tick). No float needed — just addition and a shift.

🕹 Games to play — and what to notice

The same three techniques — tiles, sprites, flicker — can be felt on every machine of the era, from racing the beam to the PPU. For each case: how it was done and what to switch on or count to see the constraint. From "there is no memory at all" to "a little more memory — and here is a new class of games".

Atari 2600 · Combat / Adventure 1977–79 · racing the beam, no frame buffer

128 bytes of RAM, and the CPU draws the picture on the fly as the beam moves. In hardware there are only a couple of sprites (the players) plus a couple of "missiles" plus the "ball" and the playfield background. More than two objects in a row is already a register-rewriting trick. Adventure hides the famous first "Easter egg" in a place where the hardware shouldn't have allowed an extra object at all.

🎮 Play: run Adventure or Combat in Stella. Count how many objects are on screen at once — almost always few, and they flicker when there are "too many" on one line. That is the two-sprite ceiling that comes out of racing the beam.

Pac-Man 1980 · the screen is a tilemap

The maze is a pure tilemap: a ~28×31 grid with a tile index in every cell (wall / dot / empty). The maze itself is static and lives in ROM; what changes in RAM is mostly the state of the dots (eaten or not) — that is already almost a bitmap of a few dozen bytes. The ghosts and Pac-Man are sprites on top of the map.

🎮 Play: in any Pac-Man port, look at the maze as a grid: every "cell" is one tile. The walls repeat — that is the same tile with a different number in the map, not unique artwork.

Super Mario Bros 1985 · NES · tile scrolling + palettes

Huge levels fit in a cartridge because the world is tiles from a shared set (brick, pipe, cloud), and scrolling shifts the nametable and draws one column. The cloud and the bush are the same tile with a different palette. Mario and Luigi are the same sprite, different palette. Reuse everywhere.

🎮 Play: in SMB, notice that the bush and the cloud have the same shape — that is literally one tile recolored by a palette. Palette-swapped enemies (red/gray Koopa) are the same trick for saving ROM.

Mega Man / late NES flicker from the 8-sprites-per-line limit

When more than 8 sprites end up in one horizontal band (a boss + projectiles + the player), the PPU physically can't manage the ninth — the game rotates priority and the objects flicker. That is not a rendering bug but a direct consequence of secondary OAM holding 8 entries.

🎮 Play: in Mega Man (or Contra), get into a boss scene with a pile of bullets at the same height — you'll see the characteristic sprite flicker. That is the "8 per line" limit with your own eyes.

Deep end · theory: precision, range and overflow in fixed-pointskippable

The Qm.n format splits m+n bits into an integer and a fractional part. That settles two parameters at once:

Range versus precision — on a single dial

The step (the smallest representable value) and the maximum are rigidly linked: with n fractional bits

step = 2^{- n}, x_{max} \approx 2^{m - 1}

Move the point right (a larger n) and fractions get finer, but the ceiling before overflow drops. Q16.16 is the compromise "±32768 with a step of 1/65536". Compare with float: it moves the point (the exponent) and therefore gives an enormous dynamic range but a variable absolute precision; fixed-point gives a constant absolute step, which for gameplay physics is actually more convenient (determinism, no ULP drift).

Multiplication and overflow

The product of two Q16.16 values has 32 fractional bits and up to 32 integer bits — you need a 64-bit (or a carefully handled 32-bit) intermediate, otherwise the high bits get cut off. Division is the reverse: shift left by n first, then do integer division. The 6502 doesn't even have integer multiplication — it was done with addition and tables of squares (a·b = ((a+b)² − (a−b)²)/4 using a precomputed x² table).

Why a power of two in the first place

A scale of 2ⁿ turns dividing and multiplying by the scale into a bit shift — one cycle. Any other scale (×1000, say) would need real division. Same reason alignments and buffer sizes are taken as powers of two.

Deep end · engineering: where these techniques live in a modern engineskippable

Tile → texture atlas. Modern 2D rendering packs sprites into a single texture atlas and draws them in a batch (one draw call for hundreds of tiles) — for exactly the same reason: reuse memory and don't poke the GPU per object. Tilemap engines (Tiled, Godot TileMap) are direct descendants of the nametable.
Sprite compositing → compositing in general. Hardware sprites over a background = early hardware compositing; today that is layers and quads on the GPU, and the OS window compositors.
Palette → indexed color / LUT. An index into a palette is a lookup table (LUT). A palette swap is changing one table instead of redrawing. LUTs live on in color grading, tone mapping and shaders.
Fixed-point → deterministic and quantized arithmetic. Lockstep RTS and rollback fighting games still take fixed-point, because float isn't reproducible across platforms. And neural network quantization (int8/int4) is the same Q format with a scale and a zero point.
Racing the beam → beam racing today. The idea of "sync to the beam, don't buffer" came back in low-latency rendering (scanline-synchronous output, VRR, frontbuffer tricks) for minimal latency.

Analogy

A tilemap is LEGO by numbers: instead of drawing every pixel of a wall, you write "cell 5 — brick #1", and the brick itself was molded once. Sprites are figures placed on top of the assembled board and moved separately. Fixed-point is a ruler with fixed markings: the point between whole and fractional parts is "always in the same place", so arithmetic is cheap — but you can't go past the last marking.

Why it matters

These three techniques are not a museum. Whenever you're short of a resource again — GPU memory, bus bandwidth, kilobytes on embedded, bits per model weight — you rediscover exactly them: reuse through an index (tile/atlas/palette), separating "background" from "moving" (static versus dynamic) and quantized integer arithmetic instead of expensive float. The constraints of 1980 are a gym for the "cheap and sufficient" engineering mindset, and it transfers directly to your domain.

🔁 Beyond games — where this transfers

The lesson gives you three transferable moves: an index instead of the data (reference + reuse), separating static from dynamic, and quantized integer arithmetic at a fixed scale.

ML / AI (your domain): the fixed-point Q format is quantization: int8/int4 inference stores weights as integers with a scale and a zero point, exactly like v = r·2⁻ⁿ; choosing n is choosing "range vs precision" during calibration. Palette/index ⇄ the codebook in VQ-VAE and embedding lookup (index → a vector from a table). Tile reuse ⇄ weight sharing (convolutions, tied embeddings). Block coding of tiles ⇄ patches in a ViT. But hold the line: the "tile = token" analogy only holds tight for a discrete representation (BPE tokens, VQ-VAE indices); for a continuous latent (an ordinary VAE, a diffusion latent) there is none — that is not a dictionary of indices but a continuous parameterization. Seeing where an analogy holds and where it breaks matters more than the analogy itself.

Systems / data: index-into-a-table = dictionary encoding in columnar databases (Parquet/Arrow) and LUTs; the atlas = asset packing; "static in ROM, dynamic in RAM" = separating a read-only cache from hot state.

Graphics / compression: tiles = macroblocks in JPEG and video (block DCT), the palette = indexed PNG/GIF; all of it is "store a dictionary, reference it by index".

The principle: when a resource is scarce, don't store the raw thing. Build a dictionary of what repeats, reference it by index, and keep numbers in the smallest sufficient representation.

🔧 Run it and poke at it — on your home machine

What to play is above (🕹). Here — get inside the era's hardware through an emulator with a debugger:

🔧 Poke at it (debug) ~40 min, Mesen/Stella

Open the NES emulator Mesen (it has a PPU debugger). Load Super Mario Bros, open the nametable viewer and the OAM/sprite view: you'll see the screen as a grid of tile indices and the table of 64 sprites live. Set a breakpoint on writes to OAM — you'll catch the moment the game rotates sprite priority. In Stella (Atari 2600), turn on the debugger and watch the game rewrite the TIA registers line by line — racing the beam frame by frame.

🧪 Test it (QA eyes) ~15 min

Look for traces of the constraints: flicker at >8 sprites per line (Mega Man, Contra), palette-swapped enemies (one drawing, different colors), repeating tiles in the background, objects jittering at the screen edge. Note where the artist hides the constraint (a dark background masks flicker) and where it shows through.

Checklist: saw the nametable as a grid of indices; spotted the 8-sprite limit with your eyes (flicker); identified at least one palette swap.

Connections

foundation

The game loop and fixed timestep — fixed-point and a fixed step together give you determinism: integer physics on a constant dt is reproducible bit for bit.

Collision: AABB and tiles — the sprites and tiles from this lesson are precisely the "boxes" and "cells" that collision is computed between.

ECS and data-oriented design — "an index instead of the data" and dense arrays are the direct continuation of this line of thought about memory layout.

Questions worth asking

Fixed-point is just "integers divided by a scale". Why does it need its own name?

Yes, in essence it is scaled integers — but the name pins down a convention: the scale is always 2ⁿ (so multiplying and dividing by it become a one-cycle shift) and it is shared across all quantities, otherwise you can't add them. "Q16.16" communicates both the scale and the bit layout in a single token. The same object is called "quantization with a power-of-two scale" in ML — the substance is identical, only the vocabulary changes.

Why does the hardware flicker at >8 sprites rather than just dropping the extras?

In hardware the ninth sprite on a line really isn't drawn — that is the hard limit of secondary OAM at 8 entries. But if the sprite order were fixed, the same "extra" one would disappear every time. Games rotate priority each frame (a cyclic shift of the list), so the "invisible" one is a different object in turn — each is visible every other frame. The eye assembles this into a semi-transparent flicker instead of "gone". Degradation instead of failure — a deliberate engineering choice.

Tiles give "free scrolling" — why exactly free?

Because when the camera shifts by a tile, almost nothing changes: the old indices stay where they are (they are simply read at an offset), and you only need to draw one new row or column of tiles at the incoming edge. A per-pixel buffer would have to be redrawn entirely for the same shift. The cost of scrolling drops from O(screen) to O(edge). The NES PPU does this in hardware through the scroll registers.

Why would float be worse, even if the hardware had it?

For dynamic range float is better, but 1980s gameplay physics has modest range needs, while determinism and cheapness are critical. Float gives variable absolute precision (the ULP grows with magnitude) and non-reproducibility across platforms and compilers — poison for lockstep and replays. Fixed-point gives a constant step, bit-for-bit repeatability, integer operations. That is why deterministic RTS and fighting games still take fixed-point deliberately, not out of poverty.

If the tiles live in ROM, what is actually held in that tiny RAM?

Not images but mutable state: where the sprites are right now (OAM), which dots have been eaten, the score, timers, scroll indices, the current map reference. Tile and sprite artwork is read-only in ROM/CHR. That is exactly the "static (ROM) / dynamic (RAM)" split: 2 KB of RAM is enough for state because the heavy assets never load into it. The direct ancestor of today's split between immutable assets and hot state.