Porting notes — recurring N64→PC bug classes

A field guide to the bug classes that keep recurring when running big-endian 32-bit N64 game code, unmodified, on a little-endian 64-bit host. Terse by design — each entry compresses a full investigation to a symptom, a fix, and a grep heuristic for finding siblings. Each entry cites a Dxx label; the full evidence and fix for that instance live in dev/findings.md under the same label. Skim the section headers; read the classes relevant to the task at hand.

If you are debugging a crash in this port, read this first — the odds are good that you are looking at one of these.

Contents

A. Pointer-width struct growth (32→64) — the dominant class

A decomp struct with pointer fields, or one pun-allocated into a fixed N64-sized hole / hardcoded byte count, is larger on x86-64. Reading it from ROM bytes misaligns; allocating it N64-sized overruns adjacent state.

B. 16-byte PC Gfx / Vtx vs 8-byte N64

Any buffer reservation, memcpy size, slot stride, or pool budget expressed in N64 Gfx/Vtx units is half-size on PC.

C. Big-endian rodata / ROM data read on little-endian PC

ROM assets and compiled-in .rodata are big-endian. Anything not run through a converter or a runtime bswap fixup reads scrambled.

C2. Port-layer / SDL shims

D. N64 hardware idioms fast3d does not emulate

D2. The HUD/model “X-mirror” (D114/D116) — RESOLVED: it was an upside-down capture

M-33 (finding D168). There was no mirror. gfx_opengl_dump_bound_fbo (port/fast3d/gfx_opengl.cpp) wrote glReadPixels output — bottom-row-first, GL origin — straight into a top-row-first P6 PPM, so every GE_PCDUMP and F12 capture was vertically flipped. Sessions M-6/M-7/M-8/M-11 kept finding textRenderGlyph → GBI → fast3d → buf_vbo → GL each verified non-mirrored — because nothing was mirrored; they were staring at upside-down screenshots of asymmetric content (text, ammo digits, guard skins, the Nintendo logo) and reading “inverted” as “X-mirrored”. The developer confirms the game renders correctly on real hardware.

The lesson worth keeping: when every stage of a pipeline probes clean but the output “looks wrong”, suspect the observation tool before adding a correction. A cosmetic defect that no probe can localise after four sessions is a strong signal that the defect isn’t in the code. Fix: PPM writer now emits rows top-to-bottom; tools_pc/golden/*.png were flipped to match.

D3. GCC/mingw makes an all-non-negative enum UNSIGNED

The N64 toolchain treats enum as signed int; GCC on the PC target gives an enum whose enumerators are all ≥ 0 an unsigned underlying type. Any descending loop that relies on the counter going negative to terminate then spins forever:

for (s = SP_LEVEL_EGYPT; s >= SP_LEVEL_DAM /* == 0 */; s--)   // never ends

D4. N64 “interrupts off” is not free on PC — it must be a real lock

osSetIntMask(OS_IM_NONE) … osSetIntMask(saved) on N64 makes a region atomic w.r.t. every interrupt (audio, VI, SI). libaudio, the scheduler and a few others use it as their only mutual-exclusion primitive. On PC the “audio interrupt” is a real preemptible thread (amMain), so a no-op osSetIntMask shim = no mutual exclusion = concurrent linked-list mutation.

D5. Loop bounds that assume linker adjacency of two file-scope globals

N64 decomp sometimes ends an array walk with end = &nextGlobal; where nextGlobal is the next file-scope definition in the .c. The N64 toolchain emits .data/.bss in source order so &nextGlobal == array + ARRAY_COUNT(array); mingw/GCC on the PC target reorders globals, so end can land before the array (loop runs 0–1 times) or far past it (walk off the end).

D6. Non-void function with no return — latent until -O2

The decomp has a handful of functions declared to return a value (often Gfx *) whose body ends without a return — the IDO/N64 build happened to leave the intended value in $v0 (usually the result of a tail call), and the caller’s x = f(...) kept working by luck. GCC on the PC target does too at -Og, but at -O2 the return register is genuinely undefined and the caller reads garbage. When the value is a display-list cursor, the caller then keeps writing the DL from a stale offset and overwrites whatever the function just emitted — the emitted primitive silently disappears.

D7. va_list is an ARRAY type on x86-64 SysV — never take & of a by-value va_list param

On Windows x64 va_list is a scalar (char *), so passing a va_list parameter by value and later taking its address (&args) to hand a helper a “pointer to the list” happens to work. On the x86-64 System V ABI (Linux/macOS) va_list is __va_list_tag[1] — an array — so a by-value va_list parameter has already decayed to __va_list_tag *. &args is then the address of the local pointer slot, not the argument list, and a helper doing va_arg(*args, …) walks garbage → segfault. Latent at -Og, fatal at -O2.

D8. Latent fixed-size stack-buffer over/under-runs — fatal only under a stack protector

The decomp has a class of local scratch buffers that game code writes a few slots past (or before) the declared bounds — e.g. a traversal stack whose bail check is looser than its array size, or &buf[i*3] with i starting at -1. The N64 build and the MinGW/Windows build have no stack protector, so the overrun lands in adjacent stack scratch and is (in practice) harmless — the Windows build is stable over thousands of frames. Ubuntu’s gcc default is -fstack-protector-strong, which places a canary right after the buffer and turns every such overrun into a fatal *** stack smashing detected *** __stack_chk_fail.

E. Process / method notes