feat(network): implement Phase 4 RTP framing with FU-A fragmentation
Add the sc_network library: RFC 3550 RtpHeader/RtpPacket serialize and parse (the receiver tolerates CSRC lists, extension headers, and padding by skipping/stripping them) and RFC 6184 H.264 payloading via H264Packetizer/H264Depacketizer. The packetizer splits Annex-B frames into NAL units (3- and 4-byte start codes), emitting single-NAL packets or FU-A fragments within the configured MTU, with the marker bit closing each frame and randomized SSRC/sequence by default. The depacketizer reassembles access units with 3-byte start codes, so both start-code widths round-trip byte-exactly; frames damaged by sequence gaps or missing fragments are dropped until the Phase 7 loss-recovery work. test_rtp covers header and packet round-trips, malformed-input rejections, splitter behavior, FU-A chunk bounds, full packetize -> depacketize round-trip, gap dropping, marker-only frame separation, sequence wrap, and empty inputs. meson test 3/3, valgrind clean.
This commit is contained in:
+19
-5
@@ -1,13 +1,27 @@
|
||||
# Project Memory — screen_cast
|
||||
|
||||
Last updated: Phase 3 validated and complete; current phase is Phase 4.
|
||||
Last updated: Phase 4 (RTP framing) complete and tested; current phase is
|
||||
Phase 5.
|
||||
|
||||
## Project state
|
||||
|
||||
- **Phase 3 is done and validated on the desktop**: a manual
|
||||
`./build/tools/capture_smoke 10 out.h264` run on Wayland/Hyprland produced
|
||||
a valid 2256x1504 H.264 elementary stream (ffprobe clean). `docs/PHASES.md`
|
||||
is ticked; current phase is Phase 4 — RTP framing.
|
||||
- Phase 4 done: `sc_network` library implements `RtpHeader`/`RtpPacket`
|
||||
(RFC 3550 serialize/parse; tolerates CSRC lists, extension headers, and
|
||||
padding on the receive side) plus `H264Packetizer`/`H264Depacketizer`
|
||||
(RFC 6184 single-NAL + FU-A; STAP-A never emitted, unsupported types mark
|
||||
the frame damaged on receive).
|
||||
- Depacketized access units use **3-byte start codes**, and the splitter
|
||||
keeps a zero byte preceding a start code with the previous NAL, so both
|
||||
3- and 4-byte-start-code streams round-trip byte-exactly (tested).
|
||||
- Loss handling is drop-on-damage: a sequence gap or missing FU fragment
|
||||
marks the frame damaged and it is dropped silently at its marker. Full
|
||||
loss recovery / jitter handling is Phase 7.
|
||||
- `test_rtp` covers header/packet round-trips, malformed rejections, NAL
|
||||
splitting (3- and 4-byte codes), FU-A chunking with MTU bounds, full
|
||||
packetize→depacketize round-trip, gap dropping, frame separation by
|
||||
marker alone, sequence wrap, and random default SSRC/sequence.
|
||||
Valgrind-clean; `meson test` 3/3.
|
||||
- Phase 3 remains validated; earlier review fixes still in place.
|
||||
- The first smoke run emitted `impl_ext_end_proxy called from wrong context`
|
||||
warnings: `pw_context_connect_fd` and `pw_core_disconnect` ran outside the
|
||||
thread-loop lock. Fixed by holding the lock across all pw setup/teardown
|
||||
|
||||
+4
-4
@@ -44,9 +44,9 @@ H.264 bitstream.
|
||||
|
||||
**Goal**: packetize NAL units into RTP and depacketize them.
|
||||
|
||||
- Implement `RtpHeader` and `RtpPacket` serialize/parse.
|
||||
- Add H.264 NAL splitting and FU-A fragmentation.
|
||||
- Unit test for serialization, fragmentation, and reassembly.
|
||||
- [x] Implement `RtpHeader` and `RtpPacket` serialize/parse.
|
||||
- [x] Add H.264 NAL splitting and FU-A fragmentation.
|
||||
- [x] Unit test for serialization, fragmentation, and reassembly.
|
||||
|
||||
**Validation**: unit tests cover single-NAL and fragmented packet paths.
|
||||
|
||||
@@ -89,4 +89,4 @@ where available.
|
||||
|
||||
## Current phase
|
||||
|
||||
Phase 4 — RTP Framing.
|
||||
Phase 5 — Local UDP Sender → Receiver Loopback.
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
#pragma once
|
||||
|
||||
#include "screencast/network/rtp_packet.h"
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <optional>
|
||||
#include <vector>
|
||||
|
||||
namespace sc {
|
||||
|
||||
struct EncodedFrame;
|
||||
|
||||
// Smallest usable MTU: 12-byte RTP header + 2-byte FU-A prefix + 1 data byte.
|
||||
inline constexpr std::size_t kMinimumRtpMtu = 15;
|
||||
|
||||
struct RtpPacketizerConfig {
|
||||
// Zero values ask the packetizer to choose random values, as recommended
|
||||
// by RFC 3550 for SSRC and the initial sequence number.
|
||||
std::uint32_t ssrc = 0;
|
||||
std::uint16_t initial_sequence_number = 0;
|
||||
std::uint8_t payload_type = 96; // dynamic payload type range
|
||||
std::size_t mtu = 1200; // maximum size of a serialized RTP packet
|
||||
};
|
||||
|
||||
// Splits Annex-B encoded frames into RFC 6184 RTP packets using single NAL
|
||||
// unit packets and FU-A fragmentation. STAP-A is never emitted.
|
||||
class H264Packetizer {
|
||||
public:
|
||||
explicit H264Packetizer(const RtpPacketizerConfig& config = RtpPacketizerConfig{});
|
||||
|
||||
// One encoded frame becomes one or more packets sharing the frame's RTP
|
||||
// timestamp; the final packet carries the marker bit. Returns an empty
|
||||
// vector for frames without NAL units or for an unusable MTU.
|
||||
std::vector<RtpPacket> packetize(const EncodedFrame& frame);
|
||||
|
||||
private:
|
||||
RtpPacket next_packet(uint32_t timestamp);
|
||||
void append_fu_a_packets(std::span<const std::byte> nal, uint32_t timestamp, std::vector<RtpPacket>& out);
|
||||
|
||||
RtpPacketizerConfig config_;
|
||||
std::uint16_t next_sequence_number_ = 0;
|
||||
};
|
||||
|
||||
// Reassembles RFC 6184 packet streams (single NAL unit packets and FU-A)
|
||||
// into Annex-B access units. Packets must arrive in order; frames damaged by
|
||||
// sequence gaps or missing fragments are dropped silently.
|
||||
class H264Depacketizer {
|
||||
public:
|
||||
// Feed one packet. Returns the completed access unit (Annex-B with 3-byte
|
||||
// start codes) when the packet closes a frame, nullopt otherwise.
|
||||
std::optional<std::vector<std::byte>> depacketize(const RtpPacket& packet);
|
||||
|
||||
private:
|
||||
void drop_frame();
|
||||
|
||||
std::optional<std::uint16_t> last_sequence_number_;
|
||||
bool frame_started_ = false;
|
||||
bool frame_damaged_ = false;
|
||||
std::uint32_t frame_timestamp_ = 0;
|
||||
std::vector<std::byte> access_unit_;
|
||||
bool fu_active_ = false;
|
||||
std::vector<std::byte> fu_nal_;
|
||||
};
|
||||
|
||||
} // namespace sc
|
||||
@@ -22,3 +22,5 @@ sc_codec_dep = declare_dependency(
|
||||
link_with : sc_codec,
|
||||
include_directories : sc_core_inc,
|
||||
dependencies : [dep_avcodec, dep_avutil, dep_swscale])
|
||||
|
||||
subdir('network')
|
||||
|
||||
@@ -0,0 +1,245 @@
|
||||
#include "screencast/network/h264_packetizer.h"
|
||||
|
||||
#include "screencast/codec/encoder.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <random>
|
||||
|
||||
namespace sc {
|
||||
namespace {
|
||||
|
||||
constexpr uint8_t u8(std::byte value) noexcept {
|
||||
return std::to_integer<uint8_t>(value);
|
||||
}
|
||||
|
||||
constexpr std::byte b8(uint8_t value) noexcept {
|
||||
return std::byte{value};
|
||||
}
|
||||
|
||||
constexpr uint8_t kFuA = 28;
|
||||
|
||||
std::optional<std::size_t> find_start_code(std::span<const std::byte> data, std::size_t from) noexcept {
|
||||
for (std::size_t i = from; i + 3 <= data.size(); ++i) {
|
||||
if (u8(data[i]) == 0x00 && u8(data[i + 1]) == 0x00 && u8(data[i + 2]) == 0x01) {
|
||||
return i;
|
||||
}
|
||||
}
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
// Split an Annex-B stream into NAL unit byte ranges (start codes removed).
|
||||
// A zero byte directly preceding a start code stays with the previous NAL:
|
||||
// it is either its trailing zero padding or part of a 4-byte start code, and
|
||||
// keeping it makes streams with 4-byte start codes round-trip byte-exactly.
|
||||
std::vector<std::span<const std::byte>> split_annex_b_nal_units(std::span<const std::byte> data) {
|
||||
std::vector<std::span<const std::byte>> units;
|
||||
std::optional<std::size_t> start = find_start_code(data, 0);
|
||||
while (start.has_value()) {
|
||||
const std::size_t unit_begin = *start + 3;
|
||||
const std::optional<std::size_t> next = find_start_code(data, unit_begin);
|
||||
const std::size_t unit_end = next.value_or(data.size());
|
||||
if (unit_end > unit_begin) {
|
||||
units.push_back(data.subspan(unit_begin, unit_end - unit_begin));
|
||||
}
|
||||
start = next;
|
||||
}
|
||||
return units;
|
||||
}
|
||||
|
||||
void append_start_code(std::vector<std::byte>& out) {
|
||||
out.push_back(b8(0x00));
|
||||
out.push_back(b8(0x00));
|
||||
out.push_back(b8(0x01));
|
||||
}
|
||||
|
||||
uint32_t random_u32() {
|
||||
static std::mt19937 engine{std::random_device{}()};
|
||||
return static_cast<uint32_t>(engine());
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
H264Packetizer::H264Packetizer(const RtpPacketizerConfig& config) : config_(config) {
|
||||
if (config_.ssrc == 0) {
|
||||
config_.ssrc = random_u32();
|
||||
}
|
||||
next_sequence_number_ = config_.initial_sequence_number != 0 ? config_.initial_sequence_number
|
||||
: static_cast<std::uint16_t>(random_u32());
|
||||
}
|
||||
|
||||
std::vector<RtpPacket> H264Packetizer::packetize(const EncodedFrame& frame) {
|
||||
std::vector<RtpPacket> packets;
|
||||
if (config_.mtu < kMinimumRtpMtu || frame.data.empty()) {
|
||||
return packets;
|
||||
}
|
||||
|
||||
const std::vector<std::span<const std::byte>> units = split_annex_b_nal_units(frame.data);
|
||||
const std::size_t max_single_nal = config_.mtu - 12;
|
||||
|
||||
for (const std::span<const std::byte> unit : units) {
|
||||
if (unit.empty()) {
|
||||
continue;
|
||||
}
|
||||
if (unit.size() <= max_single_nal) {
|
||||
RtpPacket packet = next_packet(frame.rtp_timestamp);
|
||||
packet.payload.assign(unit.begin(), unit.end());
|
||||
packets.push_back(std::move(packet));
|
||||
} else {
|
||||
append_fu_a_packets(unit, frame.rtp_timestamp, packets);
|
||||
}
|
||||
}
|
||||
|
||||
if (!packets.empty()) {
|
||||
packets.back().header.marker = true;
|
||||
}
|
||||
return packets;
|
||||
}
|
||||
|
||||
RtpPacket H264Packetizer::next_packet(uint32_t timestamp) {
|
||||
RtpPacket packet;
|
||||
packet.header.version = 2;
|
||||
packet.header.payload_type = config_.payload_type;
|
||||
packet.header.sequence_number = next_sequence_number_++;
|
||||
packet.header.timestamp = timestamp;
|
||||
packet.header.ssrc = config_.ssrc;
|
||||
return packet;
|
||||
}
|
||||
|
||||
void H264Packetizer::append_fu_a_packets(std::span<const std::byte> nal,
|
||||
uint32_t timestamp,
|
||||
std::vector<RtpPacket>& out) {
|
||||
const std::size_t max_chunk = config_.mtu - 12 - 2;
|
||||
const uint8_t nal_header = u8(nal.front());
|
||||
// The FU indicator keeps the original NAL's F bit (0) and NRI, and
|
||||
// declares type 28; the FU header carries S/E flags plus the real type.
|
||||
const uint8_t fu_indicator = static_cast<uint8_t>((nal_header & 0xE0) | kFuA);
|
||||
const uint8_t nal_type = static_cast<uint8_t>(nal_header & 0x1F);
|
||||
const std::span<const std::byte> payload = nal.subspan(1);
|
||||
|
||||
std::size_t offset = 0;
|
||||
bool first = true;
|
||||
while (true) {
|
||||
const std::size_t chunk = std::min(payload.size() - offset, max_chunk);
|
||||
const bool last = offset + chunk == payload.size();
|
||||
|
||||
RtpPacket packet = next_packet(timestamp);
|
||||
packet.payload.reserve(2 + chunk);
|
||||
packet.payload.push_back(b8(fu_indicator));
|
||||
packet.payload.push_back(b8(static_cast<uint8_t>((first ? 0x80 : 0x00) | (last ? 0x40 : 0x00) | nal_type)));
|
||||
packet.payload.insert(packet.payload.end(),
|
||||
payload.begin() + static_cast<std::ptrdiff_t>(offset),
|
||||
payload.begin() + static_cast<std::ptrdiff_t>(offset + chunk));
|
||||
out.push_back(std::move(packet));
|
||||
|
||||
offset += chunk;
|
||||
first = false;
|
||||
if (last) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
std::optional<std::vector<std::byte>> H264Depacketizer::depacketize(const RtpPacket& packet) {
|
||||
// Track sequence continuity: a gap means packets were lost.
|
||||
if (last_sequence_number_.has_value()) {
|
||||
const std::uint16_t expected = static_cast<std::uint16_t>(*last_sequence_number_ + 1);
|
||||
if (packet.header.sequence_number != expected) {
|
||||
fu_active_ = false;
|
||||
fu_nal_.clear();
|
||||
if (frame_started_) {
|
||||
frame_damaged_ = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
last_sequence_number_ = packet.header.sequence_number;
|
||||
|
||||
// A timestamp change without a closing marker means the previous frame
|
||||
// lost its tail and can no longer be recovered.
|
||||
if (frame_started_ && packet.header.timestamp != frame_timestamp_) {
|
||||
drop_frame();
|
||||
}
|
||||
if (!frame_started_) {
|
||||
frame_started_ = true;
|
||||
frame_damaged_ = false;
|
||||
frame_timestamp_ = packet.header.timestamp;
|
||||
access_unit_.clear();
|
||||
}
|
||||
|
||||
if (!packet.payload.empty()) {
|
||||
const uint8_t type = static_cast<uint8_t>(u8(packet.payload.front()) & 0x1F);
|
||||
if (type >= 1 && type <= 23) {
|
||||
// Single NAL unit packet.
|
||||
if (fu_active_) {
|
||||
// The previous fragmented NAL never received its end packet.
|
||||
frame_damaged_ = true;
|
||||
fu_active_ = false;
|
||||
fu_nal_.clear();
|
||||
}
|
||||
append_start_code(access_unit_);
|
||||
access_unit_.insert(access_unit_.end(), packet.payload.begin(), packet.payload.end());
|
||||
} else if (type == kFuA) {
|
||||
if (packet.payload.size() < 2) {
|
||||
frame_damaged_ = true;
|
||||
} else {
|
||||
const uint8_t fu_header = u8(packet.payload[1]);
|
||||
const bool start = (fu_header & 0x80) != 0;
|
||||
const bool end = (fu_header & 0x40) != 0;
|
||||
const std::span<const std::byte> fragment = std::span<const std::byte>{packet.payload}.subspan(2);
|
||||
if (start) {
|
||||
if (fu_active_) {
|
||||
// The previous fragmented NAL lost its end packet.
|
||||
frame_damaged_ = true;
|
||||
}
|
||||
fu_active_ = true;
|
||||
fu_nal_.clear();
|
||||
fu_nal_.push_back(
|
||||
b8(static_cast<uint8_t>((u8(packet.payload.front()) & 0xE0) | (fu_header & 0x1F))));
|
||||
fu_nal_.insert(fu_nal_.end(), fragment.begin(), fragment.end());
|
||||
} else if (!fu_active_) {
|
||||
// Continuation without a start: the head of the NAL is lost.
|
||||
frame_damaged_ = true;
|
||||
} else {
|
||||
fu_nal_.insert(fu_nal_.end(), fragment.begin(), fragment.end());
|
||||
if (end) {
|
||||
append_start_code(access_unit_);
|
||||
access_unit_.insert(access_unit_.end(), fu_nal_.begin(), fu_nal_.end());
|
||||
fu_active_ = false;
|
||||
fu_nal_.clear();
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Unsupported packetization mode (STAP-A, MTAP, FU-B): the frame
|
||||
// cannot be reconstructed.
|
||||
frame_damaged_ = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (!packet.header.marker) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
if (fu_active_) {
|
||||
// The marker arrived while a NAL was still fragmented.
|
||||
frame_damaged_ = true;
|
||||
fu_active_ = false;
|
||||
fu_nal_.clear();
|
||||
}
|
||||
|
||||
std::optional<std::vector<std::byte>> result;
|
||||
if (!frame_damaged_ && !access_unit_.empty()) {
|
||||
result = std::move(access_unit_);
|
||||
}
|
||||
drop_frame();
|
||||
return result;
|
||||
}
|
||||
|
||||
void H264Depacketizer::drop_frame() {
|
||||
frame_started_ = false;
|
||||
frame_damaged_ = false;
|
||||
access_unit_.clear();
|
||||
fu_active_ = false;
|
||||
fu_nal_.clear();
|
||||
}
|
||||
|
||||
} // namespace sc
|
||||
@@ -0,0 +1,14 @@
|
||||
# Phase 4 RTP framing. Pure C++ with no external dependencies.
|
||||
|
||||
sc_network_sources = files(
|
||||
'rtp_packet.cpp',
|
||||
'h264_packetizer.cpp',
|
||||
)
|
||||
|
||||
sc_network = static_library('sc_network',
|
||||
sc_network_sources,
|
||||
include_directories : sc_core_inc)
|
||||
|
||||
sc_network_dep = declare_dependency(
|
||||
link_with : sc_network,
|
||||
include_directories : sc_core_inc)
|
||||
@@ -0,0 +1,136 @@
|
||||
#include "screencast/network/rtp_packet.h"
|
||||
|
||||
#include <array>
|
||||
#include <cstddef>
|
||||
|
||||
namespace sc {
|
||||
namespace {
|
||||
|
||||
constexpr uint8_t u8(std::byte value) noexcept {
|
||||
return std::to_integer<uint8_t>(value);
|
||||
}
|
||||
|
||||
constexpr std::byte b8(uint8_t value) noexcept {
|
||||
return std::byte{value};
|
||||
}
|
||||
|
||||
void store_u16(std::span<std::byte, 12> out, std::size_t offset, uint16_t value) noexcept {
|
||||
out[offset] = b8(static_cast<uint8_t>(value >> 8));
|
||||
out[offset + 1] = b8(static_cast<uint8_t>(value));
|
||||
}
|
||||
|
||||
void store_u32(std::span<std::byte, 12> out, std::size_t offset, uint32_t value) noexcept {
|
||||
out[offset] = b8(static_cast<uint8_t>(value >> 24));
|
||||
out[offset + 1] = b8(static_cast<uint8_t>(value >> 16));
|
||||
out[offset + 2] = b8(static_cast<uint8_t>(value >> 8));
|
||||
out[offset + 3] = b8(static_cast<uint8_t>(value));
|
||||
}
|
||||
|
||||
uint16_t load_u16(std::span<const std::byte, 12> in, std::size_t offset) noexcept {
|
||||
return static_cast<uint16_t>((static_cast<uint16_t>(u8(in[offset])) << 8) | u8(in[offset + 1]));
|
||||
}
|
||||
|
||||
uint32_t load_u32(std::span<const std::byte, 12> in, std::size_t offset) noexcept {
|
||||
return (static_cast<uint32_t>(u8(in[offset])) << 24) | (static_cast<uint32_t>(u8(in[offset + 1])) << 16) |
|
||||
(static_cast<uint32_t>(u8(in[offset + 2])) << 8) | static_cast<uint32_t>(u8(in[offset + 3]));
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool RtpHeader::serialize(std::span<std::byte, 12> out) const noexcept {
|
||||
// The fixed-extent view can only carry the bare 12-byte header.
|
||||
if (version != 2 || csrc_count != 0 || extension) {
|
||||
return false;
|
||||
}
|
||||
out[0] = b8(static_cast<uint8_t>((version << 6) | (padding ? 0x20 : 0) | csrc_count));
|
||||
out[1] = b8(static_cast<uint8_t>((marker ? 0x80 : 0) | (payload_type & 0x7F)));
|
||||
store_u16(out, 2, sequence_number);
|
||||
store_u32(out, 4, timestamp);
|
||||
store_u32(out, 8, ssrc);
|
||||
return true;
|
||||
}
|
||||
|
||||
std::optional<RtpHeader> RtpHeader::parse(std::span<const std::byte, 12> in) noexcept {
|
||||
RtpHeader header;
|
||||
header.version = static_cast<uint8_t>(u8(in[0]) >> 6);
|
||||
if (header.version != 2) {
|
||||
return std::nullopt;
|
||||
}
|
||||
header.padding = (u8(in[0]) & 0x20) != 0;
|
||||
header.extension = (u8(in[0]) & 0x10) != 0;
|
||||
header.csrc_count = static_cast<uint8_t>(u8(in[0]) & 0x0F);
|
||||
header.marker = (u8(in[1]) & 0x80) != 0;
|
||||
header.payload_type = static_cast<uint8_t>(u8(in[1]) & 0x7F);
|
||||
header.sequence_number = load_u16(in, 2);
|
||||
header.timestamp = load_u32(in, 4);
|
||||
header.ssrc = load_u32(in, 8);
|
||||
return header;
|
||||
}
|
||||
|
||||
std::vector<std::byte> RtpPacket::serialize() const {
|
||||
std::vector<std::byte> bytes;
|
||||
// This sender never emits CSRC lists, extension headers, or padding, so
|
||||
// those flags must stay clear or the wire format could not be honored.
|
||||
if (header.csrc_count != 0 || header.extension || header.padding) {
|
||||
return bytes;
|
||||
}
|
||||
std::array<std::byte, 12> buffer{};
|
||||
if (!header.serialize(buffer)) {
|
||||
return bytes;
|
||||
}
|
||||
bytes.reserve(buffer.size() + payload.size());
|
||||
bytes.insert(bytes.end(), buffer.begin(), buffer.end());
|
||||
bytes.insert(bytes.end(), payload.begin(), payload.end());
|
||||
return bytes;
|
||||
}
|
||||
|
||||
std::optional<RtpPacket> RtpPacket::parse(std::span<const std::byte> in) noexcept {
|
||||
if (in.size() < 12) {
|
||||
return std::nullopt;
|
||||
}
|
||||
const std::optional<RtpHeader> header = RtpHeader::parse(in.first<12>());
|
||||
if (!header.has_value()) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
std::size_t offset = 12 + static_cast<std::size_t>(header->csrc_count) * 4;
|
||||
if (in.size() < offset) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
if (header->extension) {
|
||||
// Skip profile (2 bytes) + length in 32-bit words (2 bytes) + data.
|
||||
if (in.size() < offset + 4) {
|
||||
return std::nullopt;
|
||||
}
|
||||
const uint16_t extension_words =
|
||||
static_cast<uint16_t>((static_cast<uint16_t>(u8(in[offset + 2])) << 8) | u8(in[offset + 3]));
|
||||
const std::size_t extension_bytes = 4 + static_cast<std::size_t>(extension_words) * 4;
|
||||
if (in.size() < offset + extension_bytes) {
|
||||
return std::nullopt;
|
||||
}
|
||||
offset += extension_bytes;
|
||||
}
|
||||
|
||||
std::size_t payload_size = in.size() - offset;
|
||||
if (header->padding) {
|
||||
// RFC 3550: the last byte of the packet holds the padding size,
|
||||
// which includes itself.
|
||||
if (payload_size == 0) {
|
||||
return std::nullopt;
|
||||
}
|
||||
const uint8_t padding_size = u8(in.back());
|
||||
if (padding_size == 0 || padding_size > payload_size) {
|
||||
return std::nullopt;
|
||||
}
|
||||
payload_size -= padding_size;
|
||||
}
|
||||
|
||||
RtpPacket packet;
|
||||
packet.header = *header;
|
||||
packet.payload.assign(in.begin() + static_cast<std::ptrdiff_t>(offset),
|
||||
in.begin() + static_cast<std::ptrdiff_t>(offset + payload_size));
|
||||
return packet;
|
||||
}
|
||||
|
||||
} // namespace sc
|
||||
@@ -9,3 +9,9 @@ test_codec_roundtrip = executable('test_codec_roundtrip',
|
||||
dependencies : sc_codec_dep)
|
||||
|
||||
test('h264 roundtrip', test_codec_roundtrip)
|
||||
|
||||
test_rtp = executable('test_rtp',
|
||||
'network/test_rtp.cpp',
|
||||
dependencies : sc_network_dep)
|
||||
|
||||
test('rtp framing', test_rtp)
|
||||
|
||||
@@ -0,0 +1,378 @@
|
||||
#include "screencast/codec/encoder.h"
|
||||
#include "screencast/network/h264_packetizer.h"
|
||||
#include "screencast/network/rtp_packet.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <cstdint>
|
||||
#include <cstdio>
|
||||
#include <cstdlib>
|
||||
#include <initializer_list>
|
||||
#include <optional>
|
||||
#include <vector>
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void fail(const char* what) {
|
||||
std::fprintf(stderr, "test_rtp: FAIL: %s\n", what);
|
||||
std::abort();
|
||||
}
|
||||
|
||||
void check(bool condition, const char* what) {
|
||||
if (!condition) {
|
||||
fail(what);
|
||||
}
|
||||
}
|
||||
|
||||
std::vector<std::byte> bytes(std::initializer_list<int> values) {
|
||||
std::vector<std::byte> out;
|
||||
out.reserve(values.size());
|
||||
for (const int value : values) {
|
||||
out.push_back(std::byte{static_cast<std::uint8_t>(value)});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
void append_bytes(std::vector<std::byte>& destination, std::initializer_list<int> values) {
|
||||
for (const int value : values) {
|
||||
destination.push_back(std::byte{static_cast<std::uint8_t>(value)});
|
||||
}
|
||||
}
|
||||
|
||||
bool equal_bytes(const std::vector<std::byte>& lhs, const std::vector<std::byte>& rhs) {
|
||||
return lhs.size() == rhs.size() && std::equal(lhs.begin(), lhs.end(), rhs.begin());
|
||||
}
|
||||
|
||||
std::uint8_t u8(std::byte value) {
|
||||
return std::to_integer<std::uint8_t>(value);
|
||||
}
|
||||
|
||||
// Joins NAL units with 3-byte start codes, the canonical form the
|
||||
// depacketizer reconstructs.
|
||||
std::vector<std::byte> annex_b(const std::vector<std::vector<std::byte>>& nals) {
|
||||
std::vector<std::byte> out;
|
||||
for (const std::vector<std::byte>& nal : nals) {
|
||||
append_bytes(out, {0x00, 0x00, 0x01});
|
||||
out.insert(out.end(), nal.begin(), nal.end());
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<std::byte> nal(int header, std::size_t size) {
|
||||
std::vector<std::byte> out;
|
||||
out.reserve(size);
|
||||
out.push_back(std::byte{static_cast<std::uint8_t>(header)});
|
||||
for (std::size_t i = 1; i < size; ++i) {
|
||||
out.push_back(std::byte{static_cast<std::uint8_t>((i * 7 + 1) & 0xFF)});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
sc::EncodedFrame make_frame(const std::vector<std::byte>& data, std::uint32_t rtp_timestamp) {
|
||||
sc::EncodedFrame frame;
|
||||
frame.data = data;
|
||||
frame.rtp_timestamp = rtp_timestamp;
|
||||
frame.is_keyframe = true;
|
||||
return frame;
|
||||
}
|
||||
|
||||
sc::RtpPacketizerConfig test_config(std::uint16_t initial_sequence, std::size_t mtu) {
|
||||
sc::RtpPacketizerConfig config;
|
||||
config.ssrc = 0x12345678;
|
||||
config.initial_sequence_number = initial_sequence;
|
||||
config.mtu = mtu;
|
||||
return config;
|
||||
}
|
||||
|
||||
void test_header_roundtrip() {
|
||||
sc::RtpHeader header;
|
||||
header.marker = true;
|
||||
header.payload_type = 97;
|
||||
header.sequence_number = 0xABCD;
|
||||
header.timestamp = 0x11223344;
|
||||
header.ssrc = 0xDEADBEEF;
|
||||
|
||||
std::array<std::byte, 12> buffer{};
|
||||
check(header.serialize(buffer), "header serialize");
|
||||
const std::optional<sc::RtpHeader> parsed = sc::RtpHeader::parse(buffer);
|
||||
check(parsed.has_value(), "header parse");
|
||||
check(parsed->version == 2, "version");
|
||||
check(parsed->marker, "marker");
|
||||
check(parsed->payload_type == 97, "payload type");
|
||||
check(parsed->sequence_number == 0xABCD, "sequence number");
|
||||
check(parsed->timestamp == 0x11223344, "timestamp");
|
||||
check(parsed->ssrc == 0xDEADBEEF, "ssrc");
|
||||
check(!parsed->padding && !parsed->extension && parsed->csrc_count == 0, "flags");
|
||||
}
|
||||
|
||||
void test_header_rejections() {
|
||||
std::array<std::byte, 12> buffer{};
|
||||
|
||||
sc::RtpHeader bad_version;
|
||||
bad_version.version = 3;
|
||||
check(!bad_version.serialize(buffer), "reject version 3");
|
||||
|
||||
sc::RtpHeader bad_csrc;
|
||||
bad_csrc.csrc_count = 2;
|
||||
check(!bad_csrc.serialize(buffer), "reject csrc count");
|
||||
|
||||
sc::RtpHeader bad_extension;
|
||||
bad_extension.extension = true;
|
||||
check(!bad_extension.serialize(buffer), "reject extension flag");
|
||||
|
||||
const std::array<std::byte, 12> zeros{}; // version 0 on the wire
|
||||
check(!sc::RtpHeader::parse(zeros).has_value(), "reject version 0 input");
|
||||
}
|
||||
|
||||
void test_packet_roundtrip() {
|
||||
sc::RtpPacket packet;
|
||||
packet.header.sequence_number = 7;
|
||||
packet.header.timestamp = 0x0A0B0C0D;
|
||||
packet.header.ssrc = 0x01020304;
|
||||
packet.payload = bytes({0x67, 0x42, 0x00, 0x01, 0xFF});
|
||||
|
||||
const std::vector<std::byte> wire = packet.serialize();
|
||||
check(!wire.empty(), "packet serialize");
|
||||
const std::optional<sc::RtpPacket> parsed = sc::RtpPacket::parse(wire);
|
||||
check(parsed.has_value(), "packet parse");
|
||||
check(parsed->header.sequence_number == 7 && parsed->header.timestamp == 0x0A0B0C0D &&
|
||||
parsed->header.ssrc == 0x01020304 && parsed->header.payload_type == 96,
|
||||
"packet header fields");
|
||||
check(equal_bytes(parsed->payload, packet.payload), "packet payload");
|
||||
}
|
||||
|
||||
void test_packet_parse_tolerances() {
|
||||
// CSRC list is skipped, payload preserved.
|
||||
std::vector<std::byte> csrc_wire = bytes({0x81, 0x60, 0x00, 0x01, 0x00, 0x00, 0x00, 0x02, 0x11, 0x22, 0x33, 0x44});
|
||||
append_bytes(csrc_wire, {0xDE, 0xAD, 0xBE, 0xEF});
|
||||
append_bytes(csrc_wire, {0xAA, 0xBB});
|
||||
const std::optional<sc::RtpPacket> csrc_parsed = sc::RtpPacket::parse(csrc_wire);
|
||||
check(csrc_parsed.has_value(), "csrc parse");
|
||||
check(csrc_parsed->header.csrc_count == 1, "csrc count");
|
||||
check(equal_bytes(csrc_parsed->payload, bytes({0xAA, 0xBB})), "csrc payload preserved");
|
||||
|
||||
// Extension header is skipped, payload preserved.
|
||||
std::vector<std::byte> ext_wire = bytes({0x90, 0x60, 0x00, 0x01, 0x00, 0x00, 0x00, 0x02, 0x11, 0x22, 0x33, 0x44});
|
||||
append_bytes(ext_wire, {0xAB, 0xCD, 0x00, 0x01}); // profile + 1 word
|
||||
append_bytes(ext_wire, {0x55, 0x55, 0x55, 0x55}); // extension data
|
||||
append_bytes(ext_wire, {0xAA, 0xBB});
|
||||
const std::optional<sc::RtpPacket> ext_parsed = sc::RtpPacket::parse(ext_wire);
|
||||
check(ext_parsed.has_value(), "extension parse");
|
||||
check(ext_parsed->header.extension, "extension flag");
|
||||
check(equal_bytes(ext_parsed->payload, bytes({0xAA, 0xBB})), "extension payload preserved");
|
||||
|
||||
// Padding is stripped.
|
||||
std::vector<std::byte> pad_wire = bytes({0xA0, 0x60, 0x00, 0x01, 0x00, 0x00, 0x00, 0x02, 0x11, 0x22, 0x33, 0x44});
|
||||
append_bytes(pad_wire, {0xAA, 0xBB});
|
||||
append_bytes(pad_wire, {0x00, 0x00, 0x03}); // 3 padding bytes, count last
|
||||
const std::optional<sc::RtpPacket> pad_parsed = sc::RtpPacket::parse(pad_wire);
|
||||
check(pad_parsed.has_value(), "padding parse");
|
||||
check(pad_parsed->header.padding, "padding flag");
|
||||
check(equal_bytes(pad_parsed->payload, bytes({0xAA, 0xBB})), "padding stripped");
|
||||
|
||||
// Malformed input is rejected.
|
||||
check(!sc::RtpPacket::parse(bytes({0x80, 0x60, 0x00, 0x01, 0x00, 0x00, 0x00, 0x02, 0x11, 0x22, 0x33})).has_value(),
|
||||
"reject short packet");
|
||||
|
||||
std::vector<std::byte> truncated_csrc =
|
||||
bytes({0x83, 0x60, 0x00, 0x01, 0x00, 0x00, 0x00, 0x02, 0x11, 0x22, 0x33, 0x44});
|
||||
append_bytes(truncated_csrc, {0xAA, 0xBB});
|
||||
check(!sc::RtpPacket::parse(truncated_csrc).has_value(), "reject truncated csrc");
|
||||
|
||||
std::vector<std::byte> overlong_extension =
|
||||
bytes({0x90, 0x60, 0x00, 0x01, 0x00, 0x00, 0x00, 0x02, 0x11, 0x22, 0x33, 0x44});
|
||||
append_bytes(overlong_extension, {0xAB, 0xCD, 0x00, 0x04}); // claims 4 words
|
||||
check(!sc::RtpPacket::parse(overlong_extension).has_value(), "reject overlong extension");
|
||||
|
||||
std::vector<std::byte> zero_padding =
|
||||
bytes({0xA0, 0x60, 0x00, 0x01, 0x00, 0x00, 0x00, 0x02, 0x11, 0x22, 0x33, 0x44});
|
||||
append_bytes(zero_padding, {0xAA, 0x00});
|
||||
check(!sc::RtpPacket::parse(zero_padding).has_value(), "reject zero padding count");
|
||||
|
||||
std::vector<std::byte> oversized_padding =
|
||||
bytes({0xA0, 0x60, 0x00, 0x01, 0x00, 0x00, 0x00, 0x02, 0x11, 0x22, 0x33, 0x44});
|
||||
append_bytes(oversized_padding, {0x00, 0x09});
|
||||
check(!sc::RtpPacket::parse(oversized_padding).has_value(), "reject oversized padding");
|
||||
}
|
||||
|
||||
void test_single_nal_packetization() {
|
||||
const std::vector<std::byte> sps = bytes({0x67, 0x42, 0x00, 0x1F});
|
||||
const std::vector<std::byte> pps = bytes({0x68, 0xCE, 0x06, 0x0D});
|
||||
const sc::EncodedFrame frame = make_frame(annex_b({sps, pps}), 90000);
|
||||
|
||||
sc::H264Packetizer packetizer(test_config(0x0100, 1200));
|
||||
const std::vector<sc::RtpPacket> packets = packetizer.packetize(frame);
|
||||
check(packets.size() == 2, "two packets");
|
||||
check(equal_bytes(packets[0].payload, sps), "sps payload");
|
||||
check(equal_bytes(packets[1].payload, pps), "pps payload");
|
||||
check(!packets[0].header.marker && packets[1].header.marker, "marker placement");
|
||||
check(packets[0].header.sequence_number == 0x0100 && packets[1].header.sequence_number == 0x0101, "sequences");
|
||||
check(packets[0].header.timestamp == 90000 && packets[1].header.timestamp == 90000, "timestamps");
|
||||
check(packets[0].header.ssrc == 0x12345678, "ssrc");
|
||||
}
|
||||
|
||||
void test_three_byte_start_codes() {
|
||||
const std::vector<std::byte> sps = bytes({0x67, 0x01});
|
||||
const std::vector<std::byte> pps = bytes({0x68, 0x02});
|
||||
|
||||
std::vector<std::byte> data = bytes({0x00, 0x00, 0x01});
|
||||
append_bytes(data, {0x67, 0x01});
|
||||
append_bytes(data, {0x00, 0x00, 0x01});
|
||||
append_bytes(data, {0x68, 0x02});
|
||||
|
||||
sc::H264Packetizer packetizer(test_config(1, 1200));
|
||||
const std::vector<sc::RtpPacket> packets = packetizer.packetize(make_frame(data, 1));
|
||||
check(packets.size() == 2, "3-byte start codes split");
|
||||
check(equal_bytes(packets[0].payload, sps) && equal_bytes(packets[1].payload, pps), "3-byte start code payloads");
|
||||
}
|
||||
|
||||
void test_four_byte_start_codes() {
|
||||
// A zero byte preceding a start code belongs to the previous NAL, so
|
||||
// 4-byte start code streams keep their exact byte layout.
|
||||
const std::vector<std::byte> sps_with_trailing_zero = bytes({0x67, 0x01, 0x00});
|
||||
const std::vector<std::byte> pps = bytes({0x68, 0x02});
|
||||
|
||||
std::vector<std::byte> data = bytes({0x00, 0x00, 0x00, 0x01});
|
||||
append_bytes(data, {0x67, 0x01});
|
||||
append_bytes(data, {0x00, 0x00, 0x00, 0x01});
|
||||
append_bytes(data, {0x68, 0x02});
|
||||
|
||||
sc::H264Packetizer packetizer(test_config(1, 1200));
|
||||
const std::vector<sc::RtpPacket> packets = packetizer.packetize(make_frame(data, 1));
|
||||
check(packets.size() == 2, "4-byte start codes split");
|
||||
check(equal_bytes(packets[0].payload, sps_with_trailing_zero), "absorbed trailing zero");
|
||||
check(equal_bytes(packets[1].payload, pps), "4-byte start code last payload");
|
||||
}
|
||||
|
||||
void test_fu_a_fragmentation() {
|
||||
const std::vector<std::byte> big_nal = nal(0x65, 26); // IDR slice, 25 payload bytes
|
||||
const sc::EncodedFrame frame = make_frame(annex_b({big_nal}), 12345);
|
||||
|
||||
// 24-byte packets: 12 header + 2 FU bytes + 10 data per chunk.
|
||||
sc::H264Packetizer packetizer(test_config(0x00F0, 24));
|
||||
const std::vector<sc::RtpPacket> packets = packetizer.packetize(frame);
|
||||
check(packets.size() == 3, "three fu-a packets");
|
||||
for (const sc::RtpPacket& packet : packets) {
|
||||
check(packet.serialize().size() <= 24, "mtu respected");
|
||||
check(u8(packet.payload[0]) == ((0x65 & 0xE0) | 28), "fu indicator");
|
||||
check((u8(packet.payload[1]) & 0x1F) == 5, "nal type preserved");
|
||||
}
|
||||
check((u8(packets[0].payload[1]) & 0x80) != 0, "start flag");
|
||||
check((u8(packets[1].payload[1]) & 0xC0) == 0, "middle flags");
|
||||
check((u8(packets[2].payload[1]) & 0x40) != 0, "end flag");
|
||||
check(!packets[0].header.marker && !packets[1].header.marker && packets[2].header.marker, "fu marker");
|
||||
check(packets[0].header.sequence_number == 0x00F0 && packets[1].header.sequence_number == 0x00F1 &&
|
||||
packets[2].header.sequence_number == 0x00F2,
|
||||
"fu sequences");
|
||||
|
||||
std::vector<std::byte> reassembled;
|
||||
reassembled.push_back(
|
||||
std::byte{static_cast<std::uint8_t>((u8(packets[0].payload[0]) & 0xE0) | (u8(packets[0].payload[1]) & 0x1F))});
|
||||
for (const sc::RtpPacket& packet : packets) {
|
||||
reassembled.insert(reassembled.end(), packet.payload.begin() + 2, packet.payload.end());
|
||||
}
|
||||
check(equal_bytes(reassembled, big_nal), "fu reassembly");
|
||||
}
|
||||
|
||||
void test_depacketize_roundtrip() {
|
||||
const std::vector<std::byte> sps = bytes({0x67, 0x42, 0x00});
|
||||
const std::vector<std::byte> pps = bytes({0x68, 0xCE});
|
||||
const std::vector<std::byte> big = nal(0x65, 40);
|
||||
const std::vector<std::byte> access_unit = annex_b({sps, pps, big});
|
||||
|
||||
// MTU 20: SPS and PPS fit single packets; the 40-byte NAL becomes 7 FU-A
|
||||
// chunks of 6 bytes (39 payload bytes), 9 packets total.
|
||||
sc::H264Packetizer packetizer(test_config(0x1000, 20));
|
||||
const std::vector<sc::RtpPacket> packets = packetizer.packetize(make_frame(access_unit, 3000));
|
||||
check(packets.size() == 9, "round-trip packet count");
|
||||
|
||||
sc::H264Depacketizer depacketizer;
|
||||
std::optional<std::vector<std::byte>> completed;
|
||||
for (const sc::RtpPacket& packet : packets) {
|
||||
if (auto result = depacketizer.depacketize(packet)) {
|
||||
check(!completed.has_value(), "only one completion");
|
||||
completed = std::move(result);
|
||||
}
|
||||
}
|
||||
check(completed.has_value(), "frame completed");
|
||||
check(equal_bytes(*completed, access_unit), "access unit round-trip");
|
||||
}
|
||||
|
||||
void test_depacketizer_drops_gapped_frames() {
|
||||
const std::vector<std::byte> big_nal = nal(0x65, 26);
|
||||
sc::H264Packetizer packetizer(test_config(0x0100, 24));
|
||||
const std::vector<sc::RtpPacket> packets = packetizer.packetize(make_frame(annex_b({big_nal}), 5000));
|
||||
check(packets.size() == 3, "gap test packet count");
|
||||
|
||||
sc::H264Depacketizer depacketizer;
|
||||
check(!depacketizer.depacketize(packets[0]).has_value(), "first fu chunk accepted");
|
||||
// packets[1] is lost in transit; the tail cannot complete the frame.
|
||||
check(!depacketizer.depacketize(packets[2]).has_value(), "tail after gap dropped");
|
||||
}
|
||||
|
||||
void test_depacketizer_separate_frames() {
|
||||
const std::vector<std::byte> f1 = annex_b({bytes({0x67, 0x01})});
|
||||
const std::vector<std::byte> f2 = annex_b({bytes({0x41, 0x02})});
|
||||
|
||||
sc::H264Packetizer packetizer(test_config(0x0001, 1200));
|
||||
const std::vector<sc::RtpPacket> first = packetizer.packetize(make_frame(f1, 90000));
|
||||
const std::vector<sc::RtpPacket> second = packetizer.packetize(make_frame(f2, 90000));
|
||||
|
||||
sc::H264Depacketizer depacketizer;
|
||||
const std::optional<std::vector<std::byte>> au1 = depacketizer.depacketize(first[0]);
|
||||
check(au1.has_value() && equal_bytes(*au1, f1), "first frame");
|
||||
|
||||
// Same RTP timestamp on purpose: the marker alone separates frames.
|
||||
const std::optional<std::vector<std::byte>> au2 = depacketizer.depacketize(second[0]);
|
||||
check(au2.has_value() && equal_bytes(*au2, f2), "second frame with same timestamp");
|
||||
}
|
||||
|
||||
void test_sequence_wrap() {
|
||||
sc::H264Packetizer packetizer(test_config(0xFFFE, 1200));
|
||||
const std::vector<sc::RtpPacket> packets =
|
||||
packetizer.packetize(make_frame(annex_b({bytes({0x67, 0x01}), bytes({0x68, 0x02}), bytes({0x65, 0x03})}), 100));
|
||||
check(packets.size() == 3, "wrap packet count");
|
||||
check(packets[0].header.sequence_number == 0xFFFE && packets[1].header.sequence_number == 0xFFFF &&
|
||||
packets[2].header.sequence_number == 0x0000,
|
||||
"sequence wrap");
|
||||
}
|
||||
|
||||
void test_default_config_randomizes() {
|
||||
sc::H264Packetizer first;
|
||||
sc::H264Packetizer second;
|
||||
const std::vector<sc::RtpPacket> from_first = first.packetize(make_frame(annex_b({bytes({0x67, 0x01})}), 1));
|
||||
const std::vector<sc::RtpPacket> from_second = second.packetize(make_frame(annex_b({bytes({0x67, 0x01})}), 1));
|
||||
check(!from_first.empty() && !from_second.empty(), "default packetize");
|
||||
check(from_first[0].header.ssrc != from_second[0].header.ssrc, "random ssrc");
|
||||
check(from_first[0].header.sequence_number != from_second[0].header.sequence_number, "random sequence");
|
||||
}
|
||||
|
||||
void test_empty_inputs() {
|
||||
sc::H264Packetizer packetizer(test_config(1, 1200));
|
||||
check(packetizer.packetize(sc::EncodedFrame{}).empty(), "empty frame data");
|
||||
check(packetizer.packetize(make_frame(bytes({0x00, 0x01, 0x02}), 1)).empty(), "no start codes");
|
||||
|
||||
sc::H264Packetizer tiny_mtu(test_config(1, 8));
|
||||
check(tiny_mtu.packetize(make_frame(annex_b({bytes({0x67, 0x01})}), 1)).empty(), "unusable mtu");
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
int main() {
|
||||
test_header_roundtrip();
|
||||
test_header_rejections();
|
||||
test_packet_roundtrip();
|
||||
test_packet_parse_tolerances();
|
||||
test_single_nal_packetization();
|
||||
test_three_byte_start_codes();
|
||||
test_four_byte_start_codes();
|
||||
test_fu_a_fragmentation();
|
||||
test_depacketize_roundtrip();
|
||||
test_depacketizer_drops_gapped_frames();
|
||||
test_depacketizer_separate_frames();
|
||||
test_sequence_wrap();
|
||||
test_default_config_randomizes();
|
||||
test_empty_inputs();
|
||||
std::puts("test_rtp: all checks passed");
|
||||
return 0;
|
||||
}
|
||||
Reference in New Issue
Block a user