fix(codec): bound keyframe bursts with VBV and microsecond time_base
The first real sender/receiver run failed permanently: the receiver reported 'non-existing PPS 0' for every frame. Without a VBV, a 2256x1504 IDR keyframe bursts hundreds of kilobytes of back-to-back FU-A packets, overflowing the ~208KB default UDP receive buffer; the resulting sequence gap made the depacketizer drop whole keyframes including their in-band SPS/PPS, so the decoder never initialized and never recovered, because every keyframe burst overflowed again. - Encoder: add rc_max_rate = bitrate and rc_buffer_size = bitrate*2/fps, capping any single frame to about two frame periods of bytes (~40KB at the 4Mbps default). - Encoder: switch the time_base to microseconds. It was derived from the configured frame rate, which quantized capture-rate timestamps and duplicated pts; RTP timestamps are now lossless. - Transport: request a 4MB SO_RCVBUF on the receive socket (best-effort; the kernel clamps to net.core.rmem_max). meson test 4/4, valgrind clean (loopback + codec).
This commit is contained in:
+14
-7
@@ -21,6 +21,16 @@ loopback validation pending. See bottom for the run commands.
|
|||||||
in-band at every keyframe; a receiver now decodes from the bitstream
|
in-band at every keyframe; a receiver now decodes from the bitstream
|
||||||
alone (mid-stream join, PLI recovery-ready). `get_extradata()` is empty
|
alone (mid-stream join, PLI recovery-ready). `get_extradata()` is empty
|
||||||
in this mode; codec round-trip test updated to match the streaming path.
|
in this mode; codec round-trip test updated to match the streaming path.
|
||||||
|
- **Burst-control fixes after the first real run** (receiver saw
|
||||||
|
`non-existing PPS 0` forever): without VBV, a 2256x1504 IDR burst (~100s
|
||||||
|
of KB of back-to-back FU-A packets) overflowed the ~208KB default UDP
|
||||||
|
receive buffer; the sequence gap made drop-on-damage discard whole
|
||||||
|
keyframes *including their in-band SPS/PPS*, so the receiver never
|
||||||
|
recovered. Fixes: encoder VBV (`rc_max_rate = bitrate`,
|
||||||
|
`rc_buffer_size = bitrate*2/fps` — caps any keyframe to ~2 frame periods
|
||||||
|
of bytes), microsecond encoder time_base (kills the 25fps pts
|
||||||
|
quantization that duplicated timestamps), and a best-effort 4MB
|
||||||
|
SO_RCVBUF on the receive socket (kernel clamps to rmem_max).
|
||||||
- **Automated validation**: `meson test` 4/4 — new `udp loopback` test
|
- **Automated validation**: `meson test` 4/4 — new `udp loopback` test
|
||||||
encodes synthetic frames, packetizes, sends over a real localhost UDP
|
encodes synthetic frames, packetizes, sends over a real localhost UDP
|
||||||
socket, depacketizes, and decodes 10/10 frames with correct dimensions.
|
socket, depacketizes, and decodes 10/10 frames with correct dimensions.
|
||||||
@@ -69,15 +79,12 @@ None.
|
|||||||
|
|
||||||
## Forward-looking review notes (for later phases)
|
## Forward-looking review notes (for later phases)
|
||||||
|
|
||||||
- Encoder sets no VBV (`maxrate`/`buffer_size`) — ABR only; add for smoother
|
|
||||||
UDP streaming in Phase 7.
|
|
||||||
- Encoder PTS caveat: time_base derives from the configured frame rate
|
|
||||||
(default 25fps) while portal frames arrive at monitor refresh (often 60Hz),
|
|
||||||
so pts values quantize and can repeat. RTP timestamps come from the capture
|
|
||||||
clock instead, so streaming is unaffected; revisit if decoder-side
|
|
||||||
presentation timing ever matters.
|
|
||||||
- `RtpTransport::start/send` return plain bools (scaffold API); error
|
- `RtpTransport::start/send` return plain bools (scaffold API); error
|
||||||
messages are lost — consider an error channel when signaling lands.
|
messages are lost — consider an error channel when signaling lands.
|
||||||
|
- Very high bitrates can still exceed even a raised receive buffer if
|
||||||
|
`net.core.rmem_max` is low on the receiver; the encoder VBV bounds
|
||||||
|
bursts to ~2 frame periods, so this needs `--bitrate` ≳ 100 Mbps to
|
||||||
|
matter. Document in RUNBOOK.
|
||||||
- DMA-BUF-only portal streams are rejected with a clear message (hardware
|
- DMA-BUF-only portal streams are rejected with a clear message (hardware
|
||||||
path is Phase 7).
|
path is Phase 7).
|
||||||
- No negative-path tests yet (bad config, bad stride, undersized buffer).
|
- No negative-path tests yet (bad config, bad stride, undersized buffer).
|
||||||
|
|||||||
+10
-2
@@ -59,8 +59,16 @@ Two terminals on the same desktop session:
|
|||||||
```
|
```
|
||||||
|
|
||||||
Expected: the receiver window shows the captured desktop in near real time.
|
Expected: the receiver window shows the captured desktop in near real time.
|
||||||
Stop either side with Ctrl-C. Optional flags: `--port`, `--peer HOST[:PORT]`,
|
If the receiver starts after the sender, a few `non-existing PPS` decode
|
||||||
`--bitrate KBPS`, `--target window`.
|
errors are normal until the next keyframe arrives (the GOP is ~0.4s);
|
||||||
|
video should appear within a second. Stop either side with Ctrl-C.
|
||||||
|
Optional flags: `--port`, `--peer HOST[:PORT]`, `--bitrate KBPS`,
|
||||||
|
`--target window`.
|
||||||
|
|
||||||
|
Notes: the encoder runs a VBV that caps keyframe bursts to roughly two
|
||||||
|
frame periods of bytes, and the receiver requests a 4MB UDP receive
|
||||||
|
buffer (clamped by `net.core.rmem_max`). For very high `--bitrate`
|
||||||
|
values, raise `net.core.rmem_max` on the receiver machine.
|
||||||
|
|
||||||
The headless equivalent runs as part of `meson test` (`udp loopback` test):
|
The headless equivalent runs as part of `meson test` (`udp loopback` test):
|
||||||
synthetic frames → encode → packetize → localhost UDP → depacketize →
|
synthetic frames → encode → packetize → localhost UDP → depacketize →
|
||||||
|
|||||||
@@ -389,10 +389,19 @@ CodecResult<std::unique_ptr<Encoder>> EncoderFactory::create(const EncoderConfig
|
|||||||
ctx->codec_type = AVMEDIA_TYPE_VIDEO;
|
ctx->codec_type = AVMEDIA_TYPE_VIDEO;
|
||||||
ctx->width = config.width;
|
ctx->width = config.width;
|
||||||
ctx->height = config.height;
|
ctx->height = config.height;
|
||||||
ctx->time_base = AVRational{config.frame_rate_den, config.frame_rate_num};
|
// Microsecond resolution: the capture rate varies at runtime (monitor
|
||||||
|
// refresh) and must not be quantized to the configured frame rate, or
|
||||||
|
// consecutive frames get duplicate timestamps.
|
||||||
|
ctx->time_base = AVRational{1, 1'000'000};
|
||||||
ctx->framerate = AVRational{config.frame_rate_num, config.frame_rate_den};
|
ctx->framerate = AVRational{config.frame_rate_num, config.frame_rate_den};
|
||||||
ctx->pix_fmt = AV_PIX_FMT_YUV420P;
|
ctx->pix_fmt = AV_PIX_FMT_YUV420P;
|
||||||
ctx->bit_rate = static_cast<int64_t>(config.bitrate_kbps) * 1000;
|
ctx->bit_rate = static_cast<int64_t>(config.bitrate_kbps) * 1000;
|
||||||
|
// VBV keeps the stream CBR-ish: without it a keyframe may take many
|
||||||
|
// times the average frame size in one burst, overflowing the receiver's
|
||||||
|
// UDP socket buffer and dropping the packets that carry SPS/PPS. Two
|
||||||
|
// frame periods of budget keep latency low while bounding the burst.
|
||||||
|
ctx->rc_max_rate = ctx->bit_rate;
|
||||||
|
ctx->rc_buffer_size = static_cast<int>(ctx->bit_rate * 2 / config.frame_rate_num);
|
||||||
ctx->gop_size = config.frame_rate_num;
|
ctx->gop_size = config.frame_rate_num;
|
||||||
ctx->max_b_frames = 0;
|
ctx->max_b_frames = 0;
|
||||||
ctx->thread_count = 1;
|
ctx->thread_count = 1;
|
||||||
|
|||||||
@@ -69,6 +69,12 @@ class UdpRtpTransport final : public RtpTransport {
|
|||||||
int reuse = 1;
|
int reuse = 1;
|
||||||
(void)::setsockopt(socket_, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse));
|
(void)::setsockopt(socket_, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse));
|
||||||
|
|
||||||
|
// Absorb packet bursts: a keyframe arrives back-to-back and UDP has
|
||||||
|
// no flow control. The kernel clamps this to net.core.rmem_max, so
|
||||||
|
// very high bitrates may need a raised sysctl on the receiver.
|
||||||
|
int receive_buffer_bytes = 4 * 1024 * 1024;
|
||||||
|
(void)::setsockopt(socket_, SOL_SOCKET, SO_RCVBUF, &receive_buffer_bytes, sizeof(receive_buffer_bytes));
|
||||||
|
|
||||||
// A zero port skips binding: the OS picks the source port on send.
|
// A zero port skips binding: the OS picks the source port on send.
|
||||||
if (local_endpoint.port != 0) {
|
if (local_endpoint.port != 0) {
|
||||||
const std::optional<sockaddr_in> address = resolve_ipv4(local_endpoint);
|
const std::optional<sockaddr_in> address = resolve_ipv4(local_endpoint);
|
||||||
|
|||||||
Reference in New Issue
Block a user