Four encoder quality improvements, all sender-side:
- CRF rate control (default 22, --crf to override): targets a
constant visual quality level instead of a fixed bitrate. Static
desktop content uses 300-800 kbps (vs. forced 4000+), and the saved
bits go to sharp text and clean motion when they appear. The VBV
max rate (the --bitrate value, now a cap rather than a target)
bounds bursts so the receiver's UDP buffers stay safe. Round-trip
test bitrate dropped from 1390 kb/s to 47 kb/s on synthetic frames
— the encoder uses only what it needs.
- 5-second GOP (was 1 second): 80% fewer keyframe bits freed for
detail frames. Screen content changes incrementally, not
wholesale; PLI feedback recovers from loss in one frame time
regardless of GOP length.
- faster preset (was veryfast): better sub-pixel estimation and
RDO on more decisions. The desktop handles it trivially at 1080p.
- Screen-content x264 tuning: aq-mode=2 (auto-variance AQ moves
bits away from flat areas toward text edges) and psy-rd=1.5
(preserves texture sharpness).
Combined with the earlier veryfast upgrade and sender-side
downscaling, this is roughly 2x the perceived quality at the same
average bandwidth compared to the original ultrafast ABR encoder.
meson test 5/5 in both configurations, valgrind clean.
The receiver decoded H.264 to YUV420P, converted it to RGBA via a
CPU-intensive swscale pass, then uploaded 4 bytes/pixel to an SDL
texture — only for the GPU to convert back to RGB during rendering.
This eliminated the swscale pass entirely (40-60% of receiver CPU at
1080p) and cut the texture upload by 62%.
- DecodedFrame now carries three YUV420P planes with their strides
instead of a packed RGBA buffer; the decoder copies the planes
directly from the AVFrame (zero conversion for the common software
path). Non-YUV420P decoder output (e.g. NV12 from v4l2m2m) is
converted once to YUV420P.
- The SDL renderer uploads via SDL_UpdateYUVTexture with
SDL_PIXELFORMAT_IYUV; the GPU does the YUV→RGB conversion during
rendering.
- Decoder threading: slice-level with 4 threads (parallelizes within a
frame, no added latency), not frame-level (which buffers multiple
frames — the initial thread_count=0 broke the loopback test because
the H.264 decoder introduced a multi-frame delay before producing
output).
- The round-trip test converts decoded YUV back to RGBA for pixel
comparison via a test-local swscale call (the pipeline itself never
converts).
meson test 5/5 in both configurations, valgrind clean.
Loss recovery for the streaming path:
- PLI over signaling: the depacketizer now reports damaged frames
(DepacketizeResult) and the receiver asks the sender for a keyframe
(SessionPli, rate-limited to one per 500 ms). The sender keeps the
signaling channel open during the session and honors PLIs through
the new thread-safe SenderPipeline::request_keyframe(). Recovery
takes one frame time instead of waiting out the GOP.
- RtpJitterBuffer: reorders RTP packets by sequence number (16 packets
/ 60 ms) before the in-order depacketizer, so Wi-Fi reordering is
not misread as loss; in-order streams release immediately, and a
straggler older than the delivered sequence is discarded.
- Hardware H.264 decode probe: DecoderFactory tries h264_v4l2m2m (the
VideoCore path on the Pi) with an automatic software fallback and a
clear journal line for the chosen path; --swdecode opts out.
Validated: PLI end-to-end with a probe that drops a mid-keyframe
packet over real UDP (receiver logged the damaged frame and the PLI
arrived with the session id); hardware probe fails cleanly and falls
back on this desktop; jitter reordering covered by unit tests.
meson test 5/5 in both build configurations, valgrind clean.
Add FFmpeg-based encoder/decoder with SPS/PPS extradata, Annex-B output
normalization, low-latency libx264 settings, and a round-trip unit test.
Includes review hardening: cached SwsContext, bitrate-only rate control,
std::byte/uin8_t cast helpers, and richer test assertions.
fegger