The receiver decoded H.264 to YUV420P, converted it to RGBA via a
CPU-intensive swscale pass, then uploaded 4 bytes/pixel to an SDL
texture — only for the GPU to convert back to RGB during rendering.
This eliminated the swscale pass entirely (40-60% of receiver CPU at
1080p) and cut the texture upload by 62%.
- DecodedFrame now carries three YUV420P planes with their strides
instead of a packed RGBA buffer; the decoder copies the planes
directly from the AVFrame (zero conversion for the common software
path). Non-YUV420P decoder output (e.g. NV12 from v4l2m2m) is
converted once to YUV420P.
- The SDL renderer uploads via SDL_UpdateYUVTexture with
SDL_PIXELFORMAT_IYUV; the GPU does the YUV→RGB conversion during
rendering.
- Decoder threading: slice-level with 4 threads (parallelizes within a
frame, no added latency), not frame-level (which buffers multiple
frames — the initial thread_count=0 broke the loopback test because
the H.264 decoder introduced a multi-frame delay before producing
output).
- The round-trip test converts decoded YUV back to RGBA for pixel
comparison via a test-local swscale call (the pipeline itself never
converts).
meson test 5/5 in both configurations, valgrind clean.
Loss recovery for the streaming path:
- PLI over signaling: the depacketizer now reports damaged frames
(DepacketizeResult) and the receiver asks the sender for a keyframe
(SessionPli, rate-limited to one per 500 ms). The sender keeps the
signaling channel open during the session and honors PLIs through
the new thread-safe SenderPipeline::request_keyframe(). Recovery
takes one frame time instead of waiting out the GOP.
- RtpJitterBuffer: reorders RTP packets by sequence number (16 packets
/ 60 ms) before the in-order depacketizer, so Wi-Fi reordering is
not misread as loss; in-order streams release immediately, and a
straggler older than the delivered sequence is discarded.
- Hardware H.264 decode probe: DecoderFactory tries h264_v4l2m2m (the
VideoCore path on the Pi) with an automatic software fallback and a
clear journal line for the chosen path; --swdecode opts out.
Validated: PLI end-to-end with a probe that drops a mid-keyframe
packet over real UDP (receiver logged the damaged frame and the PLI
arrived with the session id); hardware probe fails cleanly and falls
back on this desktop; jitter reordering covered by unit tests.
meson test 5/5 in both build configurations, valgrind clean.
Add FFmpeg-based encoder/decoder with SPS/PPS extradata, Annex-B output
normalization, low-latency libx264 settings, and a round-trip unit test.
Includes review hardening: cached SwsContext, bitrate-only rate control,
std::byte/uin8_t cast helpers, and richer test assertions.
fegger