Skip to main content

Codecs and the Decode Contract

Three decoders live behind one interface so the player never cares what it is playing. This chapter explains that interface, then dissects the two bugs that defined its final shape. Both bugs are worth your time even if you never touch this codebase: they are general lessons about wrapping any frame-based decoder.

The interface​

typedef struct {
    const char *name;
    esp_err_t (*init)(void);
    esp_err_t (*decode)(uint8_t *in, size_t in_size, uint8_t *out, size_t *out_size);
    esp_err_t (*close)(void);
    esp_err_t (*get_format)(codec_format_t *fmt);   // rate/channels/bitrate once known
    void      (*reset_input)(void);                  // discard buffered input (seek)
} audio_codec_t;

The player selects a codec by file extension for SD playback, and for streams by the HTTP Content-Type header (with a sniff of the first bytes as fallback). Each codec owns an internal input accumulator, because the transport hands over arbitrary chunks (a TCP read, a 4 KB file block) while decoders consume exact frames; the accumulator is where arbitrary meets exact.

The drain contract​

One rule makes the buffering bounded, and it was added after measuring its absence: decode(NULL, 0, out, &out_size) is a legal call meaning "emit more PCM from what you already have."

Why it must exist: a decoder call can emit at most a few frames of PCM per invocation (the output buffer holds ~3 MP3 frames). If the caller may only push new input, then any input chunk bigger than a frame grows the accumulator on every call, and the accumulator eventually overflows and must throw away data mid-frame. The drain call lets the loop push once, then pull until dry:

for (int pass = 0; pass < MAX_PASSES; pass++) {
    codec->decode(pass == 0 ? buf : NULL, pass == 0 ? n : 0, pcm, &pcm_len);
    if (pcm_len == 0) break;
    write_to_i2s(pcm, pcm_len);   // blocks at the DAC rate, pacing everything
}

Accumulators now idle near one frame of occupancy, permanently. The pass cap bounds pause/stop latency.

MP3: minimp3, and the tail-shear bug​

MP3 decode uses minimp3, a superb single-header decoder, vendored so its exact semantics stay pinned. The wrapper (codec_mp3.c) feeds it from the accumulator frame by frame. Two of its return conventions turned out to be loaded weapons.

The bug​

Symptom: a radio stream decoded normally for a few hundred milliseconds, then went silent forever while consuming compressed data six times faster than real time. No error, no crash; the decoder was "working" furiously and emitting nothing.

Mechanism, reconstructed byte by byte (with checksums proving the input pristine at four checkpoints, then a host-side replay of the exact device chunking):

  1. The accumulator's tail usually ends mid-frame; that is unavoidable with chunked input.
  2. Asked to decode a buffer whose last frame is truncated, minimp3 scans it, finds no completable frame, and returns "skip these N bytes" where N is the whole remainder, including the head of the truncated frame.
  3. The wrapper obeyed and discarded. The frame grid was now sheared: the next buffer began mid-frame.
  4. From then on the decoder latched onto false sync patterns inside audio data (an MP3 sync word is only 11 bits; look-alikes occur constantly), each "frame" failing to decode, each failure consuming a few bytes. Hence silence at 6x consumption.

The deeper cause is a contract mismatch: minimp3's "skip N" verdict is correct for a complete file in memory, where an unparseable region really is garbage. Over a stream, "unparseable" and "not yet complete" are indistinguishable to the decoder, and only the caller knows which situation it is in.

The fix, in two guards​

// 1. Never even attempt a decode without a full worst-case frame buffered.
while (accumulated - offset >= MP3_MAX_FRAME_BYTES && out_space >= FRAME_PCM) { ... }

// 2. Never honor a whole-remainder skip on a short tail.
if (samples == 0 && frame_bytes >= avail && avail < MP3_MAX_FRAME_BYTES)
    break;   // keep the bytes, wait for more input

MP3_MAX_FRAME_BYTES (1728) is the largest legal MPEG-1 Layer III frame plus header slack. Guard 1 also protects minimp3's other trap: calling it on a short tail makes it reset its internal state, silently discarding the bit reservoir (MP3 frames borrow bits from predecessors; lose the reservoir and the next frames decode to silence). After the fix, a 16-second replay of the exact failing capture decoded 599 of 600 frames; the one skip is the legitimate first-frame reservoir warm-up.

The transferable lessons​

  • When wrapping a decoder, read the streaming semantics of every return value, not just the happy path. "Bytes consumed" may include bytes the decoder merely gave up on.
  • A decoder that consumes without emitting is a desynchronized decoder. Instrument ratio of bytes-in to PCM-out; it is the cheapest health metric there is.
  • Reproduce codec bugs on the host. The same C file compiled with a 30-line main() replaying captured bytes turns a flash-and-pray embedded bug into a printf-debuggable one in seconds. That harness pattern is documented in testing.

AAC: esp_audio_codec, same lesson, cleaner fix​

AAC and HE-AAC (what most Quran radio stations broadcast) decode through Espressif's esp_audio_codec library, wrapped in codec_aac.c with the same accumulator and drain contract.

It exhibited the same failure class: a truncated ADTS frame at the tail made the decoder error, and the error path resynced to the next sync word, discarding the truncated frame's head. Stream sheared, playback stalled within a minute.

ADTS allows an exact guard, which is why this fix is three lines instead of a heuristic: every ADTS frame header carries its own total length in bits 30..42. The wrapper parses it and refuses to feed a frame until it is fully buffered:

size_t frame_len = ((hdr[3] & 0x03) << 11) | (hdr[4] << 3) | (hdr[5] >> 5);
if (frame_len > avail) break;   // wait for the rest of this frame

HE-AAC needs one more subtlety: the decoder reports the effective sample rate (SBR doubles it) only after the first frame, so the player applies output format from get_format() after first decode rather than trusting the header.

WAV: the control case​

codec_wav.c is a hundred lines: parse the RIFF header, then pass samples through. It exists mostly to keep the interface honest (three implementations keep an abstraction truthful in a way two rarely do) and to give tests a trivially predictable codec. WAV never buffers, so its drain call returns nothing by definition, which the interface permits.

Memory placement​

Codec state is large (the MP3 accumulator plus minimp3's structures, the AAC library context) and none of it is touched by DMA or ISRs, so it lives in PSRAM via EXT_RAM_BSS_ATTR, reclaiming precious internal RAM. The PCM output buffers the codecs write are also PSRAM; only the I2S driver's own DMA descriptors are internal, and the driver copies into them. Rules per tasks and memory.