diff --git a/.agents/MEMORY.md b/.agents/MEMORY.md index b7f384f..24ad42b 100644 --- a/.agents/MEMORY.md +++ b/.agents/MEMORY.md @@ -21,6 +21,16 @@ loopback validation pending. See bottom for the run commands. in-band at every keyframe; a receiver now decodes from the bitstream alone (mid-stream join, PLI recovery-ready). `get_extradata()` is empty in this mode; codec round-trip test updated to match the streaming path. + - **Burst-control fixes after the first real run** (receiver saw + `non-existing PPS 0` forever): without VBV, a 2256x1504 IDR burst (~100s + of KB of back-to-back FU-A packets) overflowed the ~208KB default UDP + receive buffer; the sequence gap made drop-on-damage discard whole + keyframes *including their in-band SPS/PPS*, so the receiver never + recovered. Fixes: encoder VBV (`rc_max_rate = bitrate`, + `rc_buffer_size = bitrate*2/fps` — caps any keyframe to ~2 frame periods + of bytes), microsecond encoder time_base (kills the 25fps pts + quantization that duplicated timestamps), and a best-effort 4MB + SO_RCVBUF on the receive socket (kernel clamps to rmem_max). - **Automated validation**: `meson test` 4/4 — new `udp loopback` test encodes synthetic frames, packetizes, sends over a real localhost UDP socket, depacketizes, and decodes 10/10 frames with correct dimensions. @@ -69,15 +79,12 @@ None. ## Forward-looking review notes (for later phases) -- Encoder sets no VBV (`maxrate`/`buffer_size`) — ABR only; add for smoother - UDP streaming in Phase 7. -- Encoder PTS caveat: time_base derives from the configured frame rate - (default 25fps) while portal frames arrive at monitor refresh (often 60Hz), - so pts values quantize and can repeat. RTP timestamps come from the capture - clock instead, so streaming is unaffected; revisit if decoder-side - presentation timing ever matters. - `RtpTransport::start/send` return plain bools (scaffold API); error messages are lost — consider an error channel when signaling lands. +- Very high bitrates can still exceed even a raised receive buffer if + `net.core.rmem_max` is low on the receiver; the encoder VBV bounds + bursts to ~2 frame periods, so this needs `--bitrate` ≳ 100 Mbps to + matter. Document in RUNBOOK. - DMA-BUF-only portal streams are rejected with a clear message (hardware path is Phase 7). - No negative-path tests yet (bad config, bad stride, undersized buffer). diff --git a/docs/RUNBOOK.md b/docs/RUNBOOK.md index 1b1853f..6c6138c 100644 --- a/docs/RUNBOOK.md +++ b/docs/RUNBOOK.md @@ -59,8 +59,16 @@ Two terminals on the same desktop session: ``` Expected: the receiver window shows the captured desktop in near real time. -Stop either side with Ctrl-C. Optional flags: `--port`, `--peer HOST[:PORT]`, -`--bitrate KBPS`, `--target window`. +If the receiver starts after the sender, a few `non-existing PPS` decode +errors are normal until the next keyframe arrives (the GOP is ~0.4s); +video should appear within a second. Stop either side with Ctrl-C. +Optional flags: `--port`, `--peer HOST[:PORT]`, `--bitrate KBPS`, +`--target window`. + +Notes: the encoder runs a VBV that caps keyframe bursts to roughly two +frame periods of bytes, and the receiver requests a 4MB UDP receive +buffer (clamped by `net.core.rmem_max`). For very high `--bitrate` +values, raise `net.core.rmem_max` on the receiver machine. The headless equivalent runs as part of `meson test` (`udp loopback` test): synthetic frames → encode → packetize → localhost UDP → depacketize → diff --git a/src/codec/ffmpeg_encoder.cpp b/src/codec/ffmpeg_encoder.cpp index 92f9e5d..99b3f44 100644 --- a/src/codec/ffmpeg_encoder.cpp +++ b/src/codec/ffmpeg_encoder.cpp @@ -389,10 +389,19 @@ CodecResult> EncoderFactory::create(const EncoderConfig ctx->codec_type = AVMEDIA_TYPE_VIDEO; ctx->width = config.width; ctx->height = config.height; - ctx->time_base = AVRational{config.frame_rate_den, config.frame_rate_num}; + // Microsecond resolution: the capture rate varies at runtime (monitor + // refresh) and must not be quantized to the configured frame rate, or + // consecutive frames get duplicate timestamps. + ctx->time_base = AVRational{1, 1'000'000}; ctx->framerate = AVRational{config.frame_rate_num, config.frame_rate_den}; ctx->pix_fmt = AV_PIX_FMT_YUV420P; ctx->bit_rate = static_cast(config.bitrate_kbps) * 1000; + // VBV keeps the stream CBR-ish: without it a keyframe may take many + // times the average frame size in one burst, overflowing the receiver's + // UDP socket buffer and dropping the packets that carry SPS/PPS. Two + // frame periods of budget keep latency low while bounding the burst. + ctx->rc_max_rate = ctx->bit_rate; + ctx->rc_buffer_size = static_cast(ctx->bit_rate * 2 / config.frame_rate_num); ctx->gop_size = config.frame_rate_num; ctx->max_b_frames = 0; ctx->thread_count = 1; diff --git a/src/network/udp_transport.cpp b/src/network/udp_transport.cpp index a1903f1..3a8accb 100644 --- a/src/network/udp_transport.cpp +++ b/src/network/udp_transport.cpp @@ -69,6 +69,12 @@ class UdpRtpTransport final : public RtpTransport { int reuse = 1; (void)::setsockopt(socket_, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse)); + // Absorb packet bursts: a keyframe arrives back-to-back and UDP has + // no flow control. The kernel clamps this to net.core.rmem_max, so + // very high bitrates may need a raised sysctl on the receiver. + int receive_buffer_bytes = 4 * 1024 * 1024; + (void)::setsockopt(socket_, SOL_SOCKET, SO_RCVBUF, &receive_buffer_bytes, sizeof(receive_buffer_bytes)); + // A zero port skips binding: the OS picks the source port on send. if (local_endpoint.port != 0) { const std::optional address = resolve_ipv4(local_endpoint);