fix(codec): bound keyframe bursts with VBV and microsecond time_base
The first real sender/receiver run failed permanently: the receiver reported 'non-existing PPS 0' for every frame. Without a VBV, a 2256x1504 IDR keyframe bursts hundreds of kilobytes of back-to-back FU-A packets, overflowing the ~208KB default UDP receive buffer; the resulting sequence gap made the depacketizer drop whole keyframes including their in-band SPS/PPS, so the decoder never initialized and never recovered, because every keyframe burst overflowed again. - Encoder: add rc_max_rate = bitrate and rc_buffer_size = bitrate*2/fps, capping any single frame to about two frame periods of bytes (~40KB at the 4Mbps default). - Encoder: switch the time_base to microseconds. It was derived from the configured frame rate, which quantized capture-rate timestamps and duplicated pts; RTP timestamps are now lossless. - Transport: request a 4MB SO_RCVBUF on the receive socket (best-effort; the kernel clamps to net.core.rmem_max). meson test 4/4, valgrind clean (loopback + codec).
This commit is contained in:
+14
-7
@@ -21,6 +21,16 @@ loopback validation pending. See bottom for the run commands.
|
||||
in-band at every keyframe; a receiver now decodes from the bitstream
|
||||
alone (mid-stream join, PLI recovery-ready). `get_extradata()` is empty
|
||||
in this mode; codec round-trip test updated to match the streaming path.
|
||||
- **Burst-control fixes after the first real run** (receiver saw
|
||||
`non-existing PPS 0` forever): without VBV, a 2256x1504 IDR burst (~100s
|
||||
of KB of back-to-back FU-A packets) overflowed the ~208KB default UDP
|
||||
receive buffer; the sequence gap made drop-on-damage discard whole
|
||||
keyframes *including their in-band SPS/PPS*, so the receiver never
|
||||
recovered. Fixes: encoder VBV (`rc_max_rate = bitrate`,
|
||||
`rc_buffer_size = bitrate*2/fps` — caps any keyframe to ~2 frame periods
|
||||
of bytes), microsecond encoder time_base (kills the 25fps pts
|
||||
quantization that duplicated timestamps), and a best-effort 4MB
|
||||
SO_RCVBUF on the receive socket (kernel clamps to rmem_max).
|
||||
- **Automated validation**: `meson test` 4/4 — new `udp loopback` test
|
||||
encodes synthetic frames, packetizes, sends over a real localhost UDP
|
||||
socket, depacketizes, and decodes 10/10 frames with correct dimensions.
|
||||
@@ -69,15 +79,12 @@ None.
|
||||
|
||||
## Forward-looking review notes (for later phases)
|
||||
|
||||
- Encoder sets no VBV (`maxrate`/`buffer_size`) — ABR only; add for smoother
|
||||
UDP streaming in Phase 7.
|
||||
- Encoder PTS caveat: time_base derives from the configured frame rate
|
||||
(default 25fps) while portal frames arrive at monitor refresh (often 60Hz),
|
||||
so pts values quantize and can repeat. RTP timestamps come from the capture
|
||||
clock instead, so streaming is unaffected; revisit if decoder-side
|
||||
presentation timing ever matters.
|
||||
- `RtpTransport::start/send` return plain bools (scaffold API); error
|
||||
messages are lost — consider an error channel when signaling lands.
|
||||
- Very high bitrates can still exceed even a raised receive buffer if
|
||||
`net.core.rmem_max` is low on the receiver; the encoder VBV bounds
|
||||
bursts to ~2 frame periods, so this needs `--bitrate` ≳ 100 Mbps to
|
||||
matter. Document in RUNBOOK.
|
||||
- DMA-BUF-only portal streams are rejected with a clear message (hardware
|
||||
path is Phase 7).
|
||||
- No negative-path tests yet (bad config, bad stride, undersized buffer).
|
||||
|
||||
+10
-2
@@ -59,8 +59,16 @@ Two terminals on the same desktop session:
|
||||
```
|
||||
|
||||
Expected: the receiver window shows the captured desktop in near real time.
|
||||
Stop either side with Ctrl-C. Optional flags: `--port`, `--peer HOST[:PORT]`,
|
||||
`--bitrate KBPS`, `--target window`.
|
||||
If the receiver starts after the sender, a few `non-existing PPS` decode
|
||||
errors are normal until the next keyframe arrives (the GOP is ~0.4s);
|
||||
video should appear within a second. Stop either side with Ctrl-C.
|
||||
Optional flags: `--port`, `--peer HOST[:PORT]`, `--bitrate KBPS`,
|
||||
`--target window`.
|
||||
|
||||
Notes: the encoder runs a VBV that caps keyframe bursts to roughly two
|
||||
frame periods of bytes, and the receiver requests a 4MB UDP receive
|
||||
buffer (clamped by `net.core.rmem_max`). For very high `--bitrate`
|
||||
values, raise `net.core.rmem_max` on the receiver machine.
|
||||
|
||||
The headless equivalent runs as part of `meson test` (`udp loopback` test):
|
||||
synthetic frames → encode → packetize → localhost UDP → depacketize →
|
||||
|
||||
@@ -389,10 +389,19 @@ CodecResult<std::unique_ptr<Encoder>> EncoderFactory::create(const EncoderConfig
|
||||
ctx->codec_type = AVMEDIA_TYPE_VIDEO;
|
||||
ctx->width = config.width;
|
||||
ctx->height = config.height;
|
||||
ctx->time_base = AVRational{config.frame_rate_den, config.frame_rate_num};
|
||||
// Microsecond resolution: the capture rate varies at runtime (monitor
|
||||
// refresh) and must not be quantized to the configured frame rate, or
|
||||
// consecutive frames get duplicate timestamps.
|
||||
ctx->time_base = AVRational{1, 1'000'000};
|
||||
ctx->framerate = AVRational{config.frame_rate_num, config.frame_rate_den};
|
||||
ctx->pix_fmt = AV_PIX_FMT_YUV420P;
|
||||
ctx->bit_rate = static_cast<int64_t>(config.bitrate_kbps) * 1000;
|
||||
// VBV keeps the stream CBR-ish: without it a keyframe may take many
|
||||
// times the average frame size in one burst, overflowing the receiver's
|
||||
// UDP socket buffer and dropping the packets that carry SPS/PPS. Two
|
||||
// frame periods of budget keep latency low while bounding the burst.
|
||||
ctx->rc_max_rate = ctx->bit_rate;
|
||||
ctx->rc_buffer_size = static_cast<int>(ctx->bit_rate * 2 / config.frame_rate_num);
|
||||
ctx->gop_size = config.frame_rate_num;
|
||||
ctx->max_b_frames = 0;
|
||||
ctx->thread_count = 1;
|
||||
|
||||
@@ -69,6 +69,12 @@ class UdpRtpTransport final : public RtpTransport {
|
||||
int reuse = 1;
|
||||
(void)::setsockopt(socket_, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse));
|
||||
|
||||
// Absorb packet bursts: a keyframe arrives back-to-back and UDP has
|
||||
// no flow control. The kernel clamps this to net.core.rmem_max, so
|
||||
// very high bitrates may need a raised sysctl on the receiver.
|
||||
int receive_buffer_bytes = 4 * 1024 * 1024;
|
||||
(void)::setsockopt(socket_, SOL_SOCKET, SO_RCVBUF, &receive_buffer_bytes, sizeof(receive_buffer_bytes));
|
||||
|
||||
// A zero port skips binding: the OS picks the source port on send.
|
||||
if (local_endpoint.port != 0) {
|
||||
const std::optional<sockaddr_in> address = resolve_ipv4(local_endpoint);
|
||||
|
||||
Reference in New Issue
Block a user