feat(app): implement Phase 5 local UDP sender->receiver loopback
Wire the first end-to-end pipeline: capture -> encode -> packetize -> UDP -> depacketize -> decode -> render. - UdpRtpTransport: raw POSIX UDP sockets (IPv4 via getaddrinfo), a receive jthread woken by socket close on stop; port 0 skips binding so the sender uses an OS-assigned source port. ASIO stays deferred to the signaling phase per ARCHITECTURE.md. - SdlRenderer: SDL3 window/renderer with RGBA texture upload; the texture is recreated on resolution change. RendererFactory now returns RendererResult so SDL init failures carry a message, mirroring the codec/capture error patterns. - screencast binary: parse_cli plus SenderPipeline/ReceiverPipeline per the app scaffolds; the sender creates its encoder once capture reports real dimensions, the receiver keeps a bounded 3-frame queue to hold latency down and renders on its own thread until the window closes. cli argv signature fixed to 'const char* const*' so main's argv converts implicitly. - Encoder: drop AV_CODEC_FLAG_GLOBAL_HEADER so libx264 repeats SPS/PPS in-band at each keyframe -- the receiver decodes from the bitstream alone, which also makes mid-stream joins and later PLI recovery work without out-of-band parameter negotiation. The round-trip test now exercises exactly that path. - tests: new udp-loopback integration test pushes synthetic frames through a real localhost socket and decodes 10/10 frames with the right dimensions; valgrind clean (loopback + codec). meson test 4/4. Manual validation on the desktop (receiver window shows the captured desktop) is documented in docs/RUNBOOK.md.
This commit is contained in:
+52
-53
@@ -1,51 +1,38 @@
|
||||
# Project Memory — screen_cast
|
||||
|
||||
Last updated: Phase 4 (RTP framing) complete and tested; current phase is
|
||||
Phase 5.
|
||||
Last updated: Phase 5 implemented and automated tests green; manual windowed
|
||||
loopback validation pending. See bottom for the run commands.
|
||||
|
||||
## Project state
|
||||
|
||||
- Phase 4 done: `sc_network` library implements `RtpHeader`/`RtpPacket`
|
||||
(RFC 3550 serialize/parse; tolerates CSRC lists, extension headers, and
|
||||
padding on the receive side) plus `H264Packetizer`/`H264Depacketizer`
|
||||
(RFC 6184 single-NAL + FU-A; STAP-A never emitted, unsupported types mark
|
||||
the frame damaged on receive).
|
||||
- Depacketized access units use **3-byte start codes**, and the splitter
|
||||
keeps a zero byte preceding a start code with the previous NAL, so both
|
||||
3- and 4-byte-start-code streams round-trip byte-exactly (tested).
|
||||
- Loss handling is drop-on-damage: a sequence gap or missing FU fragment
|
||||
marks the frame damaged and it is dropped silently at its marker. Full
|
||||
loss recovery / jitter handling is Phase 7.
|
||||
- `test_rtp` covers header/packet round-trips, malformed rejections, NAL
|
||||
splitting (3- and 4-byte codes), FU-A chunking with MTU bounds, full
|
||||
packetize→depacketize round-trip, gap dropping, frame separation by
|
||||
marker alone, sequence wrap, and random default SSRC/sequence.
|
||||
Valgrind-clean; `meson test` 3/3.
|
||||
- Phase 3 remains validated; earlier review fixes still in place.
|
||||
- The first smoke run emitted `impl_ext_end_proxy called from wrong context`
|
||||
warnings: `pw_context_connect_fd` and `pw_core_disconnect` ran outside the
|
||||
thread-loop lock. Fixed by holding the lock across all pw setup/teardown
|
||||
proxy operations. Re-running the smoke tool should now be warning-free
|
||||
(capture worked both ways; the warnings only meant the first two
|
||||
marshaled messages were rejected and retried from the right context).
|
||||
- `src/capture/pipewire_capture.cpp` now implements the full backend:
|
||||
libportal 0.10 handshake (`create_screencast_session` → `session_start` →
|
||||
`open_pipewire_remote`), PipeWire 1.6 stream on the first portal node,
|
||||
BGRx/BGRA/RGBx/RGBA enumeration, latest-frame slot with condvar handoff.
|
||||
Portal handshake blocks on the caller thread; frames arrive on the pw
|
||||
thread. `stop()` is thread-safe; teardown follows the pw-required order.
|
||||
- Encoder now accepts padded strides for packed RGB formats and the new
|
||||
`PixelFormat::Bgrx` (mapped to `AV_PIX_FMT_BGRA`); planar Yuv420p still
|
||||
requires a packed layout. This was the review note blocking Phase 3.
|
||||
- `tools/capture_smoke` (manual, not in `meson test`) captures N frames →
|
||||
encodes → writes Annex-B including prepended SPS/PPS extradata (verified:
|
||||
libx264 GLOBAL_HEADER extradata is Annex-B).
|
||||
- Earlier review fixes remain in place; valgrind on the codec round-trip
|
||||
test is still clean after the encoder stride changes.
|
||||
- Encoder PTS caveat: the encoder time_base is derived from the configured
|
||||
frame rate (default 25fps), but portal frames arrive at monitor refresh
|
||||
(often 60Hz), so pts values quantize to 40ms units and can repeat. Harmless
|
||||
for the smoke test; revisit when RTP timestamps matter (Phase 4/5).
|
||||
- **Phase 5 (local UDP sender→receiver loopback) is implemented**:
|
||||
- `UdpRtpTransport` (src/network/udp_transport.cpp): raw POSIX sockets,
|
||||
AF_INET, IPv4 via getaddrinfo; port 0 skips binding (sender side); stop()
|
||||
closes the socket to unblock the receive jthread. ASIO was deliberately
|
||||
deferred to Phase 6 (see decisions).
|
||||
- `screencast` binary (src/app/): cli.cpp + main.cpp + pipelines.cpp wiring
|
||||
SenderPipeline (capture→encode→packetize→send) and ReceiverPipeline
|
||||
(recv→depacketize→decode→bounded 3-frame queue→render thread).
|
||||
- `SdlRenderer` (src/render/sdl_renderer.cpp): SDL3 window/renderer/texture,
|
||||
RGBA texture upload, texture recreated on resolution change.
|
||||
`RendererFactory::create` now returns `RendererResult` (error channel
|
||||
added, mirroring codec/capture patterns).
|
||||
- Encoder change: **GLOBAL_HEADER removed** so libx264 repeats SPS/PPS
|
||||
in-band at every keyframe; a receiver now decodes from the bitstream
|
||||
alone (mid-stream join, PLI recovery-ready). `get_extradata()` is empty
|
||||
in this mode; codec round-trip test updated to match the streaming path.
|
||||
- **Automated validation**: `meson test` 4/4 — new `udp loopback` test
|
||||
encodes synthetic frames, packetizes, sends over a real localhost UDP
|
||||
socket, depacketizes, and decodes 10/10 frames with correct dimensions.
|
||||
Valgrind clean (loopback + codec tests).
|
||||
- **Manual validation pending (needs the desktop)**: run the two commands in
|
||||
`docs/RUNBOOK.md` — receiver window should show the captured desktop. Tick
|
||||
`docs/PHASES.md` Phase 5 after this works.
|
||||
- Phase 4 network framing done (RFC 3550 + RFC 6184 single-NAL/FU-A;
|
||||
3-byte canonical start codes; drop-on-damage loss handling).
|
||||
- Phase 3 capture done and validated (PipeWire/portal backend; the
|
||||
`impl_ext_end_proxy` wrong-context warnings were fixed by holding the
|
||||
thread-loop lock across all pw proxy operations).
|
||||
|
||||
## Decisions
|
||||
|
||||
@@ -55,13 +42,19 @@ Phase 5.
|
||||
- Capture: PipeWire + xdg-desktop-portal.
|
||||
- Encode/Decode: FFmpeg (libavcodec, libavutil, libswscale).
|
||||
- H.264 encoder path: software `libx264`, low-latency settings, Annex-B output.
|
||||
- Decoder receives SPS/PPS through `DecoderConfig.extradata`.
|
||||
- Transport: RTP over UDP; signaling via WebSocket/JSON.
|
||||
- **SPS/PPS are sent in-band ahead of every keyframe** (no GLOBAL_HEADER);
|
||||
the decoder starts from the bitstream alone. `DecoderConfig.extradata`
|
||||
remains available if signaling ever negotiates parameters out of band.
|
||||
- Transport: RTP over UDP via raw POSIX sockets for now; **ASIO stays a
|
||||
Phase 6+ option** (the ARCHITECTURE.md dependency table places it with
|
||||
signaling/discovery). IPv4 only at the transport level for now.
|
||||
- Rendering: **SDL3** (`sdl3` pkg-config, 3.4 installed); plain texture
|
||||
upload, no GPU pipeline yet.
|
||||
- Discovery: mDNS/Avahi.
|
||||
- Rendering: SDL2 or SDL3 + OpenGL.
|
||||
- Namespace: `sc`.
|
||||
- Module error results use per-module `std::variant<T, XError>` types
|
||||
(`CodecResult`, `CaptureResult`) since C++20 has no `std::expected`.
|
||||
(`CodecResult`, `CaptureResult`, `RendererResult`) since C++20 has no
|
||||
`std::expected`.
|
||||
|
||||
## Active blockers
|
||||
|
||||
@@ -72,20 +65,26 @@ None.
|
||||
- GUI framework (Qt6 vs. none / CLI only) — deferred to later phase.
|
||||
- Hardware acceleration strategy (VAAPI / Vulkan Video / NVENC) — evaluate after
|
||||
software encode path works.
|
||||
- IPv6 at the transport layer — revisit when LAN streaming lands (Phase 6+).
|
||||
|
||||
## Forward-looking review notes (for later phases)
|
||||
|
||||
- `AV_CODEC_FLAG_GLOBAL_HEADER` suppresses in-band SPS/PPS; a receiver cannot
|
||||
join mid-stream or recover after PLI without parameter sets. Phase 5 must
|
||||
prepend SPS/PPS to keyframes or negotiate them in signaling.
|
||||
- Encoder sets no VBV (`maxrate`/`buffer_size`) — ABR only; add for smoother
|
||||
UDP streaming in Phase 7.
|
||||
- Encoder PTS caveat: time_base derives from the configured frame rate
|
||||
(default 25fps) while portal frames arrive at monitor refresh (often 60Hz),
|
||||
so pts values quantize and can repeat. RTP timestamps come from the capture
|
||||
clock instead, so streaming is unaffected; revisit if decoder-side
|
||||
presentation timing ever matters.
|
||||
- `RtpTransport::start/send` return plain bools (scaffold API); error
|
||||
messages are lost — consider an error channel when signaling lands.
|
||||
- DMA-BUF-only portal streams are rejected with a clear message (hardware
|
||||
path is Phase 7).
|
||||
- No negative-path tests yet (bad config, bad stride, undersized buffer).
|
||||
- `to_annex_b_h264` sniffs AVCC vs Annex-B by content; if an AVCC-emitting
|
||||
encoder is ever added, prefer an explicit config flag over the heuristic.
|
||||
- Region targets are rejected: the desktop portal has no region capture.
|
||||
- `next_frame()` returns `nullopt` on stream error without surfacing the
|
||||
reason (logged to stderr); consider an error channel when the receiver
|
||||
pipeline lands.
|
||||
- `CaptureSession::next_frame()` returns `nullopt` on stream error without
|
||||
surfacing the reason (logged to stderr).
|
||||
- Receiver ignores unknown packetization modes (STAP-A/MTAP/FU-B); senders
|
||||
we control never emit them, but third-party interop would need support.
|
||||
@@ -49,6 +49,23 @@ PipeWire proxy operation ran without the thread-loop lock — all pw
|
||||
calls that send messages (connect, stream, core) must happen under
|
||||
`pw_thread_loop_lock`.
|
||||
|
||||
## Sender / receiver loopback (Phase 5, manual)
|
||||
|
||||
Two terminals on the same desktop session:
|
||||
|
||||
```sh
|
||||
./build/src/app/screencast --receive # terminal 1: window appears
|
||||
./build/src/app/screencast --send # terminal 2: portal source picker
|
||||
```
|
||||
|
||||
Expected: the receiver window shows the captured desktop in near real time.
|
||||
Stop either side with Ctrl-C. Optional flags: `--port`, `--peer HOST[:PORT]`,
|
||||
`--bitrate KBPS`, `--target window`.
|
||||
|
||||
The headless equivalent runs as part of `meson test` (`udp loopback` test):
|
||||
synthetic frames → encode → packetize → localhost UDP → depacketize →
|
||||
decode, no portal or window involved.
|
||||
|
||||
## Formatting
|
||||
|
||||
```sh
|
||||
|
||||
@@ -20,6 +20,8 @@ struct ReceiveCommand {
|
||||
using Command = std::variant<SendCommand, ReceiveCommand>;
|
||||
|
||||
// Parse command line arguments. Prints usage and returns std::nullopt on error.
|
||||
std::optional<Command> parse_cli(int argc, const char* argv[]);
|
||||
// `argv` is `char const* const*` so both `main`'s `char**` and const arrays
|
||||
// convert implicitly.
|
||||
std::optional<Command> parse_cli(int argc, const char* const argv[]);
|
||||
|
||||
} // namespace sc
|
||||
|
||||
@@ -16,6 +16,9 @@ struct SenderPipelineConfig {
|
||||
EncoderConfig encoder;
|
||||
Endpoint local_rtp_endpoint;
|
||||
std::optional<Endpoint> signaling_server;
|
||||
// Where encoded RTP packets are sent. Defaults to the local loopback so
|
||||
// a sender and receiver on one machine work without any configuration.
|
||||
Endpoint peer_rtp_endpoint{"127.0.0.1", 5004};
|
||||
};
|
||||
|
||||
struct ReceiverPipelineConfig {
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
#include <variant>
|
||||
|
||||
namespace sc {
|
||||
|
||||
struct RendererError {
|
||||
std::string message;
|
||||
};
|
||||
|
||||
// C++20 does not provide std::expected. Use a variant-based result type so
|
||||
// fallible renderer operations do not rely on exceptions.
|
||||
template <typename T> using RendererResult = std::variant<T, RendererError>;
|
||||
|
||||
template <typename T> constexpr bool is_renderer_error(const RendererResult<T>& result) noexcept {
|
||||
return std::holds_alternative<RendererError>(result);
|
||||
}
|
||||
|
||||
template <typename T> T& renderer_value(RendererResult<T>& result) {
|
||||
return std::get<T>(result);
|
||||
}
|
||||
|
||||
template <typename T> const T& renderer_value(const RendererResult<T>& result) {
|
||||
return std::get<T>(result);
|
||||
}
|
||||
|
||||
template <typename T> RendererError& renderer_error(RendererResult<T>& result) {
|
||||
return std::get<RendererError>(result);
|
||||
}
|
||||
|
||||
template <typename T> const RendererError& renderer_error(const RendererResult<T>& result) {
|
||||
return std::get<RendererError>(result);
|
||||
}
|
||||
|
||||
} // namespace sc
|
||||
@@ -1,6 +1,7 @@
|
||||
#pragma once
|
||||
|
||||
#include "screencast/codec/decoder.h"
|
||||
#include "screencast/render/error.h"
|
||||
|
||||
#include <memory>
|
||||
#include <string>
|
||||
@@ -29,7 +30,7 @@ class Renderer {
|
||||
|
||||
class RendererFactory {
|
||||
public:
|
||||
static std::unique_ptr<Renderer> create(const RendererConfig& config);
|
||||
static RendererResult<std::unique_ptr<Renderer>> create(const RendererConfig& config);
|
||||
};
|
||||
|
||||
} // namespace sc
|
||||
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
#include "screencast/app/cli.h"
|
||||
|
||||
#include <charconv>
|
||||
#include <cstdio>
|
||||
#include <string_view>
|
||||
|
||||
namespace sc {
|
||||
namespace {
|
||||
|
||||
void print_usage() {
|
||||
std::fputs("usage: screencast --send [--target monitor|window] [--peer HOST[:PORT]] [--bitrate KBPS]\n"
|
||||
" screencast --receive [--port PORT]\n",
|
||||
stderr);
|
||||
}
|
||||
|
||||
bool parse_int(std::string_view text, int& value) {
|
||||
const auto [pointer, error] = std::from_chars(text.data(), text.data() + text.size(), value);
|
||||
return error == std::errc{} && pointer == text.data() + text.size();
|
||||
}
|
||||
|
||||
bool next_argument(int argc, const char* const argv[], int& index, std::string_view& value) {
|
||||
if (index + 1 >= argc) {
|
||||
print_usage();
|
||||
return false;
|
||||
}
|
||||
value = argv[++index];
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::optional<Command> parse_cli(int argc, const char* const argv[]) {
|
||||
enum class Mode {
|
||||
None,
|
||||
Send,
|
||||
Receive,
|
||||
};
|
||||
|
||||
Mode mode = Mode::None;
|
||||
SendCommand send;
|
||||
ReceiveCommand receive;
|
||||
|
||||
for (int index = 1; index < argc; ++index) {
|
||||
const std::string_view argument = argv[index];
|
||||
|
||||
if (argument == "--send") {
|
||||
if (mode != Mode::None) {
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
mode = Mode::Send;
|
||||
} else if (argument == "--receive") {
|
||||
if (mode != Mode::None) {
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
mode = Mode::Receive;
|
||||
} else if (argument == "--target") {
|
||||
std::string_view value;
|
||||
if (!next_argument(argc, argv, index, value)) {
|
||||
return std::nullopt;
|
||||
}
|
||||
if (value != "monitor" && value != "window") {
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
send.target = value;
|
||||
} else if (argument == "--peer") {
|
||||
std::string_view value;
|
||||
if (!next_argument(argc, argv, index, value)) {
|
||||
return std::nullopt;
|
||||
}
|
||||
if (mode == Mode::Receive) {
|
||||
receive.peer_address = value;
|
||||
} else {
|
||||
send.peer_address = value;
|
||||
}
|
||||
} else if (argument == "--bitrate") {
|
||||
std::string_view value;
|
||||
if (!next_argument(argc, argv, index, value) || !parse_int(value, send.bitrate_kbps) ||
|
||||
send.bitrate_kbps <= 0) {
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
} else if (argument == "--port") {
|
||||
std::string_view value;
|
||||
int port = 0;
|
||||
if (!next_argument(argc, argv, index, value) || !parse_int(value, port) || port <= 0 || port > 65535) {
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
receive.local_rtp_port = port;
|
||||
} else {
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
}
|
||||
|
||||
if (mode == Mode::Send) {
|
||||
return Command{std::move(send)};
|
||||
}
|
||||
if (mode == Mode::Receive) {
|
||||
return Command{std::move(receive)};
|
||||
}
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
} // namespace sc
|
||||
@@ -0,0 +1,87 @@
|
||||
#include "screencast/app/cli.h"
|
||||
#include "screencast/app/pipeline.h"
|
||||
|
||||
#include <atomic>
|
||||
#include <charconv>
|
||||
#include <chrono>
|
||||
#include <csignal>
|
||||
#include <string_view>
|
||||
#include <thread>
|
||||
|
||||
namespace {
|
||||
|
||||
std::atomic<bool> g_interrupted{false};
|
||||
|
||||
void handle_interrupt(int) {
|
||||
g_interrupted.store(true);
|
||||
}
|
||||
|
||||
sc::Endpoint parse_endpoint(std::string_view address, uint16_t default_port) {
|
||||
sc::Endpoint endpoint;
|
||||
const std::size_t separator = address.rfind(':');
|
||||
if (separator != std::string_view::npos) {
|
||||
endpoint.address = std::string{address.substr(0, separator)};
|
||||
int port = 0;
|
||||
const auto [pointer, error] =
|
||||
std::from_chars(address.data() + separator + 1, address.data() + address.size(), port);
|
||||
endpoint.port = error == std::errc{} ? static_cast<uint16_t>(port) : default_port;
|
||||
} else {
|
||||
endpoint.address = std::string{address};
|
||||
endpoint.port = default_port;
|
||||
}
|
||||
return endpoint;
|
||||
}
|
||||
|
||||
int run_sender(const sc::SendCommand& command) {
|
||||
sc::SenderPipelineConfig config;
|
||||
if (command.target == "window") {
|
||||
config.capture_target = sc::CaptureTargetWindow{};
|
||||
} else {
|
||||
config.capture_target = sc::CaptureTargetWholeScreen{};
|
||||
}
|
||||
config.peer_rtp_endpoint =
|
||||
command.peer_address.empty() ? sc::Endpoint{"127.0.0.1", 5004} : parse_endpoint(command.peer_address, 5004);
|
||||
config.encoder.bitrate_kbps = command.bitrate_kbps;
|
||||
|
||||
sc::SenderPipeline pipeline{std::move(config)};
|
||||
if (!pipeline.start()) {
|
||||
return 1;
|
||||
}
|
||||
while (!g_interrupted.load()) {
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
pipeline.stop();
|
||||
return 0;
|
||||
}
|
||||
|
||||
int run_receiver(const sc::ReceiveCommand& command) {
|
||||
sc::ReceiverPipelineConfig config;
|
||||
config.local_rtp_endpoint = sc::Endpoint{"0.0.0.0", static_cast<uint16_t>(command.local_rtp_port)};
|
||||
|
||||
sc::ReceiverPipeline pipeline{std::move(config)};
|
||||
if (!pipeline.start()) {
|
||||
return 1;
|
||||
}
|
||||
while (!g_interrupted.load()) {
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
pipeline.stop();
|
||||
return 0;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
const std::optional<sc::Command> command = sc::parse_cli(argc, argv);
|
||||
if (!command.has_value()) {
|
||||
return 1;
|
||||
}
|
||||
|
||||
std::signal(SIGINT, handle_interrupt);
|
||||
std::signal(SIGTERM, handle_interrupt);
|
||||
|
||||
if (const sc::SendCommand* send = std::get_if<sc::SendCommand>(&*command)) {
|
||||
return run_sender(*send);
|
||||
}
|
||||
return run_receiver(std::get<sc::ReceiveCommand>(*command));
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
# The screencast application binary wiring every module together.
|
||||
|
||||
executable('screencast',
|
||||
[
|
||||
'cli.cpp',
|
||||
'main.cpp',
|
||||
'pipelines.cpp',
|
||||
],
|
||||
dependencies : [sc_capture_dep, sc_codec_dep, sc_network_dep, sc_render_dep])
|
||||
@@ -0,0 +1,224 @@
|
||||
#include "screencast/app/pipeline.h"
|
||||
|
||||
#include "screencast/network/h264_packetizer.h"
|
||||
|
||||
#include <chrono>
|
||||
#include <deque>
|
||||
#include <iostream>
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
#include <thread>
|
||||
|
||||
namespace sc {
|
||||
|
||||
class SenderPipeline::Impl {
|
||||
public:
|
||||
explicit Impl(SenderPipelineConfig config) : config_(std::move(config)) {}
|
||||
|
||||
~Impl() {
|
||||
stop();
|
||||
}
|
||||
|
||||
bool start() {
|
||||
auto capture_result = CaptureFactory::create(config_.capture_target);
|
||||
if (is_capture_error(capture_result)) {
|
||||
std::cerr << std::format("screencast: capture failed: {}\n", capture_error(capture_result).message);
|
||||
return false;
|
||||
}
|
||||
capture_ = std::move(capture_value(capture_result));
|
||||
|
||||
if (!transport_->start(config_.local_rtp_endpoint, [](RtpPacket) {})) {
|
||||
std::cerr << "screencast: failed to start the RTP transport\n";
|
||||
return false;
|
||||
}
|
||||
transport_->set_peer(config_.peer_rtp_endpoint);
|
||||
|
||||
run_thread_ = std::jthread([this](std::stop_token stop_token) { run(std::move(stop_token)); });
|
||||
return true;
|
||||
}
|
||||
|
||||
void stop() {
|
||||
if (capture_ != nullptr) {
|
||||
capture_->stop();
|
||||
}
|
||||
run_thread_ = std::jthread{};
|
||||
transport_->stop();
|
||||
}
|
||||
|
||||
private:
|
||||
void run(std::stop_token stop_token) {
|
||||
while (!stop_token.stop_requested()) {
|
||||
const std::optional<CapturedFrame> frame = capture_->next_frame();
|
||||
if (!frame.has_value()) {
|
||||
break;
|
||||
}
|
||||
|
||||
if (encoder_ == nullptr && !create_encoder(*frame)) {
|
||||
break;
|
||||
}
|
||||
|
||||
auto encoded_result = encoder_->encode(*frame);
|
||||
if (is_codec_error(encoded_result)) {
|
||||
std::cerr << std::format("screencast: encode failed: {}\n", codec_error(encoded_result).message);
|
||||
break;
|
||||
}
|
||||
for (const EncodedFrame& encoded : codec_value(encoded_result)) {
|
||||
for (RtpPacket& packet : packetizer_.packetize(encoded)) {
|
||||
(void)transport_->send(packet);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
bool create_encoder(const CapturedFrame& frame) {
|
||||
EncoderConfig config = config_.encoder;
|
||||
config.width = frame.width;
|
||||
config.height = frame.height;
|
||||
|
||||
auto encoder_result = EncoderFactory::create(config);
|
||||
if (is_codec_error(encoder_result)) {
|
||||
std::cerr << std::format("screencast: encoder creation failed: {}\n", codec_error(encoder_result).message);
|
||||
return false;
|
||||
}
|
||||
encoder_ = std::move(codec_value(encoder_result));
|
||||
return true;
|
||||
}
|
||||
|
||||
SenderPipelineConfig config_;
|
||||
std::unique_ptr<CaptureSession> capture_;
|
||||
std::unique_ptr<Encoder> encoder_;
|
||||
H264Packetizer packetizer_;
|
||||
std::unique_ptr<RtpTransport> transport_ = RtpTransportFactory::create();
|
||||
std::jthread run_thread_;
|
||||
};
|
||||
|
||||
SenderPipeline::SenderPipeline(SenderPipelineConfig config) : impl_(std::make_unique<Impl>(std::move(config))) {}
|
||||
|
||||
SenderPipeline::~SenderPipeline() = default;
|
||||
|
||||
bool SenderPipeline::start() {
|
||||
return impl_->start();
|
||||
}
|
||||
|
||||
void SenderPipeline::stop() {
|
||||
impl_->stop();
|
||||
}
|
||||
|
||||
class ReceiverPipeline::Impl {
|
||||
public:
|
||||
explicit Impl(ReceiverPipelineConfig config) : config_(std::move(config)) {}
|
||||
|
||||
~Impl() {
|
||||
stop();
|
||||
}
|
||||
|
||||
bool start() {
|
||||
auto renderer_result = RendererFactory::create(config_.renderer);
|
||||
if (is_renderer_error(renderer_result)) {
|
||||
std::cerr << std::format("screencast: renderer creation failed: {}\n",
|
||||
renderer_error(renderer_result).message);
|
||||
return false;
|
||||
}
|
||||
renderer_ = std::move(renderer_value(renderer_result));
|
||||
|
||||
auto decoder_result = DecoderFactory::create(config_.decoder);
|
||||
if (is_codec_error(decoder_result)) {
|
||||
std::cerr << std::format("screencast: decoder creation failed: {}\n", codec_error(decoder_result).message);
|
||||
return false;
|
||||
}
|
||||
decoder_ = std::move(codec_value(decoder_result));
|
||||
|
||||
if (!transport_->start(config_.local_rtp_endpoint,
|
||||
[this](RtpPacket packet) { on_packet(std::move(packet)); })) {
|
||||
std::cerr << "screencast: failed to start the RTP transport\n";
|
||||
return false;
|
||||
}
|
||||
|
||||
render_thread_ = std::jthread([this](std::stop_token stop_token) { render_loop(std::move(stop_token)); });
|
||||
return true;
|
||||
}
|
||||
|
||||
void stop() {
|
||||
render_thread_ = std::jthread{};
|
||||
transport_->stop();
|
||||
renderer_ = nullptr;
|
||||
decoder_ = nullptr;
|
||||
{
|
||||
std::lock_guard lock(queue_mutex_);
|
||||
queue_.clear();
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
void on_packet(RtpPacket packet) {
|
||||
std::optional<std::vector<std::byte>> access_unit = depacketizer_.depacketize(packet);
|
||||
if (!access_unit.has_value()) {
|
||||
return;
|
||||
}
|
||||
|
||||
EncodedFrame encoded;
|
||||
encoded.data = std::move(*access_unit);
|
||||
encoded.rtp_timestamp = packet.header.timestamp;
|
||||
encoded.is_keyframe = false;
|
||||
|
||||
auto decoded_result = decoder_->decode(encoded);
|
||||
if (is_codec_error(decoded_result)) {
|
||||
std::cerr << std::format("screencast: decode failed: {}\n", codec_error(decoded_result).message);
|
||||
return;
|
||||
}
|
||||
for (DecodedFrame& decoded : codec_value(decoded_result)) {
|
||||
std::lock_guard lock(queue_mutex_);
|
||||
// Keep latency low: drop the oldest frame when the queue is full.
|
||||
if (queue_.size() >= kMaxQueuedFrames) {
|
||||
queue_.pop_front();
|
||||
}
|
||||
queue_.push_back(std::move(decoded));
|
||||
}
|
||||
}
|
||||
|
||||
void render_loop(std::stop_token stop_token) {
|
||||
while (!stop_token.stop_requested()) {
|
||||
if (!renderer_->poll_events()) {
|
||||
break; // window closed
|
||||
}
|
||||
std::optional<DecodedFrame> frame;
|
||||
{
|
||||
std::unique_lock lock(queue_mutex_);
|
||||
if (!queue_.empty()) {
|
||||
frame = std::move(queue_.front());
|
||||
queue_.pop_front();
|
||||
}
|
||||
}
|
||||
if (frame.has_value()) {
|
||||
(void)renderer_->present(*frame);
|
||||
} else {
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(5));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static constexpr std::size_t kMaxQueuedFrames = 3;
|
||||
|
||||
ReceiverPipelineConfig config_;
|
||||
std::unique_ptr<Renderer> renderer_;
|
||||
std::unique_ptr<Decoder> decoder_;
|
||||
H264Depacketizer depacketizer_;
|
||||
std::unique_ptr<RtpTransport> transport_ = RtpTransportFactory::create();
|
||||
std::jthread render_thread_;
|
||||
std::mutex queue_mutex_;
|
||||
std::deque<DecodedFrame> queue_;
|
||||
};
|
||||
|
||||
ReceiverPipeline::ReceiverPipeline(ReceiverPipelineConfig config) : impl_(std::make_unique<Impl>(std::move(config))) {}
|
||||
|
||||
ReceiverPipeline::~ReceiverPipeline() = default;
|
||||
|
||||
bool ReceiverPipeline::start() {
|
||||
return impl_->start();
|
||||
}
|
||||
|
||||
void ReceiverPipeline::stop() {
|
||||
impl_->stop();
|
||||
}
|
||||
|
||||
} // namespace sc
|
||||
@@ -397,7 +397,10 @@ CodecResult<std::unique_ptr<Encoder>> EncoderFactory::create(const EncoderConfig
|
||||
ctx->max_b_frames = 0;
|
||||
ctx->thread_count = 1;
|
||||
ctx->profile = AV_PROFILE_H264_MAIN;
|
||||
ctx->flags |= AV_CODEC_FLAG_LOW_DELAY | AV_CODEC_FLAG_GLOBAL_HEADER;
|
||||
// No GLOBAL_HEADER: SPS/PPS are repeated in-band at every keyframe so a
|
||||
// receiver that joins mid-stream (or recovers after loss) can decode
|
||||
// without out-of-band parameter negotiation.
|
||||
ctx->flags |= AV_CODEC_FLAG_LOW_DELAY;
|
||||
|
||||
if (av_opt_set(ctx->priv_data, "preset", "ultrafast", 0) < 0) {
|
||||
return CodecError{"failed to set libx264 preset"};
|
||||
|
||||
@@ -24,3 +24,6 @@ sc_codec_dep = declare_dependency(
|
||||
dependencies : [dep_avcodec, dep_avutil, dep_swscale])
|
||||
|
||||
subdir('network')
|
||||
subdir('render')
|
||||
|
||||
subdir('app')
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
sc_network_sources = files(
|
||||
'rtp_packet.cpp',
|
||||
'h264_packetizer.cpp',
|
||||
'udp_transport.cpp',
|
||||
)
|
||||
|
||||
sc_network = static_library('sc_network',
|
||||
|
||||
@@ -0,0 +1,163 @@
|
||||
#include "screencast/network/transport.h"
|
||||
|
||||
#include <arpa/inet.h>
|
||||
#include <netdb.h>
|
||||
#include <netinet/in.h>
|
||||
#include <sys/socket.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <array>
|
||||
#include <atomic>
|
||||
#include <cstring>
|
||||
#include <mutex>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <thread>
|
||||
|
||||
namespace sc {
|
||||
namespace {
|
||||
|
||||
constexpr int kInvalidSocket = -1;
|
||||
|
||||
// A UDP datagram cannot exceed 64 KiB on IPv4; one buffer fits any RTP
|
||||
// packet the receiver will ever see.
|
||||
constexpr std::size_t kReceiveBufferSize = 65536;
|
||||
|
||||
std::optional<sockaddr_in> resolve_ipv4(const Endpoint& endpoint) {
|
||||
addrinfo hints{};
|
||||
hints.ai_family = AF_INET;
|
||||
hints.ai_socktype = SOCK_DGRAM;
|
||||
|
||||
addrinfo* result = nullptr;
|
||||
const std::string port = std::to_string(endpoint.port);
|
||||
const char* node = endpoint.address.empty() ? nullptr : endpoint.address.c_str();
|
||||
if (getaddrinfo(node, port.c_str(), &hints, &result) != 0) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
std::optional<sockaddr_in> address;
|
||||
if (result != nullptr && result->ai_family == AF_INET && result->ai_addrlen >= sizeof(sockaddr_in)) {
|
||||
sockaddr_in resolved{};
|
||||
std::memcpy(&resolved, result->ai_addr, sizeof(sockaddr_in));
|
||||
address = resolved;
|
||||
}
|
||||
freeaddrinfo(result);
|
||||
return address;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
class UdpRtpTransport final : public RtpTransport {
|
||||
public:
|
||||
UdpRtpTransport() = default;
|
||||
|
||||
~UdpRtpTransport() override {
|
||||
stop();
|
||||
}
|
||||
|
||||
UdpRtpTransport(const UdpRtpTransport&) = delete;
|
||||
UdpRtpTransport& operator=(const UdpRtpTransport&) = delete;
|
||||
|
||||
bool start(const Endpoint& local_endpoint, ReceiveCallback on_receive) override {
|
||||
if (running_.load()) {
|
||||
return false;
|
||||
}
|
||||
socket_ = ::socket(AF_INET, SOCK_DGRAM, 0);
|
||||
if (socket_ < 0) {
|
||||
return false;
|
||||
}
|
||||
int reuse = 1;
|
||||
(void)::setsockopt(socket_, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse));
|
||||
|
||||
// A zero port skips binding: the OS picks the source port on send.
|
||||
if (local_endpoint.port != 0) {
|
||||
const std::optional<sockaddr_in> address = resolve_ipv4(local_endpoint);
|
||||
if (!address.has_value()) {
|
||||
(void)::close(socket_);
|
||||
socket_ = kInvalidSocket;
|
||||
return false;
|
||||
}
|
||||
if (::bind(socket_, reinterpret_cast<const sockaddr*>(&*address), sizeof(*address)) < 0) {
|
||||
(void)::close(socket_);
|
||||
socket_ = kInvalidSocket;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
running_.store(true);
|
||||
receive_thread_ = std::jthread([this, callback = std::move(on_receive)]() mutable { receive_loop(callback); });
|
||||
return true;
|
||||
}
|
||||
|
||||
bool send(const RtpPacket& packet) override {
|
||||
if (!running_.load() || !has_peer_.load()) {
|
||||
return false;
|
||||
}
|
||||
const std::vector<std::byte> bytes = packet.serialize();
|
||||
if (bytes.empty()) {
|
||||
return false;
|
||||
}
|
||||
sockaddr_in peer{};
|
||||
{
|
||||
// Snapshot the peer so set_peer() can be called concurrently.
|
||||
std::lock_guard lock(peer_mutex_);
|
||||
peer = peer_;
|
||||
}
|
||||
const ssize_t sent =
|
||||
::sendto(socket_, bytes.data(), bytes.size(), 0, reinterpret_cast<const sockaddr*>(&peer), sizeof(peer));
|
||||
return sent == static_cast<ssize_t>(bytes.size());
|
||||
}
|
||||
|
||||
void set_peer(const Endpoint& peer) override {
|
||||
const std::optional<sockaddr_in> address = resolve_ipv4(peer);
|
||||
if (!address.has_value()) {
|
||||
return;
|
||||
}
|
||||
std::lock_guard lock(peer_mutex_);
|
||||
peer_ = *address;
|
||||
has_peer_.store(true);
|
||||
}
|
||||
|
||||
void stop() override {
|
||||
if (!running_.exchange(false)) {
|
||||
return;
|
||||
}
|
||||
if (socket_ >= 0) {
|
||||
(void)::shutdown(socket_, SHUT_RDWR);
|
||||
(void)::close(socket_);
|
||||
socket_ = kInvalidSocket;
|
||||
}
|
||||
// Closing the socket unblocks recvfrom; joining happens implicitly
|
||||
// when the jthread assignment destroys the previous thread.
|
||||
receive_thread_ = std::jthread{};
|
||||
}
|
||||
|
||||
private:
|
||||
void receive_loop(ReceiveCallback& callback) {
|
||||
std::array<std::byte, kReceiveBufferSize> buffer{};
|
||||
while (running_.load()) {
|
||||
const ssize_t received = ::recvfrom(socket_, buffer.data(), buffer.size(), 0, nullptr, nullptr);
|
||||
if (received <= 0) {
|
||||
continue; // closed socket while running_ still true, or error
|
||||
}
|
||||
const std::optional<RtpPacket> packet =
|
||||
RtpPacket::parse(std::span{buffer.data(), static_cast<std::size_t>(received)});
|
||||
if (packet.has_value()) {
|
||||
callback(*packet);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
int socket_ = kInvalidSocket;
|
||||
std::atomic<bool> running_{false};
|
||||
std::atomic<bool> has_peer_{false};
|
||||
std::mutex peer_mutex_;
|
||||
sockaddr_in peer_{};
|
||||
std::jthread receive_thread_;
|
||||
};
|
||||
|
||||
std::unique_ptr<RtpTransport> RtpTransportFactory::create() {
|
||||
return std::make_unique<UdpRtpTransport>();
|
||||
}
|
||||
|
||||
} // namespace sc
|
||||
@@ -0,0 +1,15 @@
|
||||
# Phase 5 renderer: SDL3 software texture upload, no GPU pipeline yet.
|
||||
|
||||
dep_sdl3 = dependency('sdl3')
|
||||
|
||||
sc_render_sources = files('sdl_renderer.cpp')
|
||||
|
||||
sc_render = static_library('sc_render',
|
||||
sc_render_sources,
|
||||
include_directories : sc_core_inc,
|
||||
dependencies : [dep_sdl3])
|
||||
|
||||
sc_render_dep = declare_dependency(
|
||||
link_with : sc_render,
|
||||
include_directories : sc_core_inc,
|
||||
dependencies : [dep_sdl3])
|
||||
@@ -0,0 +1,147 @@
|
||||
#include "screencast/render/renderer.h"
|
||||
|
||||
#include <SDL3/SDL.h>
|
||||
|
||||
#include <string>
|
||||
|
||||
namespace sc {
|
||||
namespace {
|
||||
|
||||
// DecodedFrame pixels are AV_PIX_FMT_RGBA: memory order R, G, B, A, which
|
||||
// matches SDL_PIXELFORMAT_RGBA8888.
|
||||
constexpr SDL_PixelFormat kSdlPixelFormat = SDL_PIXELFORMAT_RGBA8888;
|
||||
|
||||
class SdlRenderer final : public Renderer {
|
||||
public:
|
||||
explicit SdlRenderer(const RendererConfig& config) : config_(config) {}
|
||||
|
||||
~SdlRenderer() override {
|
||||
shutdown();
|
||||
}
|
||||
|
||||
SdlRenderer(const SdlRenderer&) = delete;
|
||||
SdlRenderer& operator=(const SdlRenderer&) = delete;
|
||||
|
||||
const std::string& last_error() const {
|
||||
return last_error_;
|
||||
}
|
||||
|
||||
bool initialize() {
|
||||
if (!SDL_Init(SDL_INIT_VIDEO)) {
|
||||
last_error_ = std::string{"SDL_Init failed: "} + SDL_GetError();
|
||||
return false;
|
||||
}
|
||||
sdl_inited_ = true;
|
||||
|
||||
window_ = SDL_CreateWindow(config_.window_title.c_str(), config_.initial_width, config_.initial_height, 0);
|
||||
if (window_ == nullptr) {
|
||||
last_error_ = std::string{"SDL_CreateWindow failed: "} + SDL_GetError();
|
||||
shutdown();
|
||||
return false;
|
||||
}
|
||||
|
||||
renderer_ = SDL_CreateRenderer(window_, nullptr);
|
||||
if (renderer_ == nullptr) {
|
||||
last_error_ = std::string{"SDL_CreateRenderer failed: "} + SDL_GetError();
|
||||
shutdown();
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool present(const DecodedFrame& frame) override {
|
||||
if (frame.width <= 0 || frame.height <= 0) {
|
||||
return false;
|
||||
}
|
||||
if (frame.width != texture_width_ || frame.height != texture_height_) {
|
||||
if (!recreate_texture(frame.width, frame.height)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
const int pitch = frame.width * 4;
|
||||
if (!SDL_UpdateTexture(texture_, nullptr, frame.rgba_pixels.data(), pitch)) {
|
||||
return false;
|
||||
}
|
||||
if (!SDL_RenderTexture(renderer_, texture_, nullptr, nullptr)) {
|
||||
return false;
|
||||
}
|
||||
SDL_RenderPresent(renderer_);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool poll_events() override {
|
||||
if (closed_) {
|
||||
return false;
|
||||
}
|
||||
SDL_Event event{};
|
||||
while (SDL_PollEvent(&event)) {
|
||||
if (event.type == SDL_EVENT_QUIT ||
|
||||
(event.type == SDL_EVENT_WINDOW_CLOSE_REQUESTED && event.window.windowID == SDL_GetWindowID(window_))) {
|
||||
closed_ = true;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
void shutdown() override {
|
||||
if (texture_ != nullptr) {
|
||||
SDL_DestroyTexture(texture_);
|
||||
texture_ = nullptr;
|
||||
}
|
||||
if (renderer_ != nullptr) {
|
||||
SDL_DestroyRenderer(renderer_);
|
||||
renderer_ = nullptr;
|
||||
}
|
||||
if (window_ != nullptr) {
|
||||
SDL_DestroyWindow(window_);
|
||||
window_ = nullptr;
|
||||
}
|
||||
if (sdl_inited_) {
|
||||
SDL_Quit();
|
||||
sdl_inited_ = false;
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
bool recreate_texture(int width, int height) {
|
||||
if (texture_ != nullptr) {
|
||||
SDL_DestroyTexture(texture_);
|
||||
texture_ = nullptr;
|
||||
}
|
||||
texture_ = SDL_CreateTexture(renderer_, kSdlPixelFormat, SDL_TEXTUREACCESS_STREAMING, width, height);
|
||||
if (texture_ == nullptr) {
|
||||
last_error_ = std::string{"SDL_CreateTexture failed: "} + SDL_GetError();
|
||||
texture_width_ = 0;
|
||||
texture_height_ = 0;
|
||||
return false;
|
||||
}
|
||||
texture_width_ = width;
|
||||
texture_height_ = height;
|
||||
return true;
|
||||
}
|
||||
|
||||
RendererConfig config_;
|
||||
std::string last_error_;
|
||||
bool sdl_inited_ = false;
|
||||
bool closed_ = false;
|
||||
SDL_Window* window_ = nullptr;
|
||||
SDL_Renderer* renderer_ = nullptr;
|
||||
SDL_Texture* texture_ = nullptr;
|
||||
int texture_width_ = 0;
|
||||
int texture_height_ = 0;
|
||||
};
|
||||
|
||||
} // namespace
|
||||
|
||||
RendererResult<std::unique_ptr<Renderer>> RendererFactory::create(const RendererConfig& config) {
|
||||
auto renderer = std::make_unique<SdlRenderer>(config);
|
||||
if (!renderer->initialize()) {
|
||||
return RendererError{renderer->last_error()};
|
||||
}
|
||||
return std::unique_ptr<Renderer>(std::move(renderer));
|
||||
}
|
||||
|
||||
} // namespace sc
|
||||
@@ -0,0 +1,170 @@
|
||||
// Phase 5 integration test: the local UDP sender -> receiver loopback without
|
||||
// the interactive parts. Synthetic frames go through the real chain —
|
||||
// encode, RTP packetization, a localhost UDP socket, depacketization, and
|
||||
// decode (with in-band SPS/PPS, since there is no signaling channel).
|
||||
|
||||
#include "screencast/codec/decoder.h"
|
||||
#include "screencast/codec/encoder.h"
|
||||
#include "screencast/network/h264_packetizer.h"
|
||||
#include "screencast/network/transport.h"
|
||||
|
||||
#include <atomic>
|
||||
#include <chrono>
|
||||
#include <cstdint>
|
||||
#include <cstdio>
|
||||
#include <cstdlib>
|
||||
#include <memory>
|
||||
#include <thread>
|
||||
#include <vector>
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void fail(const char* what) {
|
||||
std::fprintf(stderr, "test_loopback: FAIL: %s\n", what);
|
||||
std::abort();
|
||||
}
|
||||
|
||||
void check(bool condition, const char* what) {
|
||||
if (!condition) {
|
||||
fail(what);
|
||||
}
|
||||
}
|
||||
|
||||
template <typename T> void expect_result(const char* what, const sc::CodecResult<T>& result) {
|
||||
if (sc::is_codec_error(result)) {
|
||||
std::fprintf(stderr, "test_loopback: FAIL: %s: %s\n", what, sc::codec_error(result).message.c_str());
|
||||
std::abort();
|
||||
}
|
||||
}
|
||||
|
||||
constexpr int kWidth = 64;
|
||||
constexpr int kHeight = 64;
|
||||
constexpr int kFrameCount = 10;
|
||||
constexpr uint64_t kNsPerFrame = 40'000'000; // 25 fps
|
||||
|
||||
sc::CapturedFrame make_frame(uint32_t index) {
|
||||
sc::CapturedFrame frame;
|
||||
frame.width = kWidth;
|
||||
frame.height = kHeight;
|
||||
frame.timestamp_ns = static_cast<uint64_t>(index) * kNsPerFrame;
|
||||
frame.pixel_format = sc::PixelFormat::Rgba;
|
||||
frame.stride = kWidth * 4;
|
||||
frame.pixels.resize(static_cast<std::size_t>(kWidth) * kHeight * 4);
|
||||
|
||||
auto* pixels = reinterpret_cast<std::uint8_t*>(frame.pixels.data());
|
||||
for (int y = 0; y < kHeight; ++y) {
|
||||
for (int x = 0; x < kWidth; ++x) {
|
||||
const std::size_t offset = (static_cast<std::size_t>(y) * kWidth + x) * 4;
|
||||
pixels[offset + 0] = static_cast<std::uint8_t>((x + static_cast<int>(index) * 4) & 0xFF);
|
||||
pixels[offset + 1] = static_cast<std::uint8_t>((y + static_cast<int>(index) * 2) & 0xFF);
|
||||
pixels[offset + 2] = static_cast<std::uint8_t>(((x ^ y) + static_cast<int>(index)) & 0xFF);
|
||||
pixels[offset + 3] = 0xFF;
|
||||
}
|
||||
}
|
||||
return frame;
|
||||
}
|
||||
|
||||
// Receiver state driven entirely by the transport's receive thread.
|
||||
struct ReceiverSink {
|
||||
sc::H264Depacketizer depacketizer;
|
||||
std::unique_ptr<sc::Decoder> decoder;
|
||||
std::atomic<int> decoded_frames{0};
|
||||
bool saw_frame = false;
|
||||
int last_width = 0;
|
||||
int last_height = 0;
|
||||
|
||||
void on_packet(sc::RtpPacket packet) {
|
||||
std::optional<std::vector<std::byte>> access_unit = depacketizer.depacketize(packet);
|
||||
if (!access_unit.has_value()) {
|
||||
return;
|
||||
}
|
||||
|
||||
sc::EncodedFrame encoded;
|
||||
encoded.data = std::move(*access_unit);
|
||||
encoded.rtp_timestamp = packet.header.timestamp;
|
||||
|
||||
auto decoded_result = decoder->decode(encoded);
|
||||
if (sc::is_codec_error(decoded_result)) {
|
||||
return;
|
||||
}
|
||||
for (const sc::DecodedFrame& decoded : sc::codec_value(decoded_result)) {
|
||||
last_width = decoded.width;
|
||||
last_height = decoded.height;
|
||||
saw_frame = decoded.width > 0 && !decoded.rgba_pixels.empty();
|
||||
decoded_frames.fetch_add(1);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
} // namespace
|
||||
|
||||
int main() {
|
||||
// Encoder on the sender side.
|
||||
sc::EncoderConfig encoder_config;
|
||||
encoder_config.codec_name = "h264";
|
||||
encoder_config.width = kWidth;
|
||||
encoder_config.height = kHeight;
|
||||
encoder_config.frame_rate_num = 25;
|
||||
encoder_config.frame_rate_den = 1;
|
||||
encoder_config.bitrate_kbps = 2000;
|
||||
|
||||
auto encoder_result = sc::EncoderFactory::create(encoder_config);
|
||||
expect_result("encoder create", encoder_result);
|
||||
auto encoder = std::move(sc::codec_value(encoder_result));
|
||||
|
||||
// The receiver decodes purely from the bitstream (in-band SPS/PPS).
|
||||
sc::DecoderConfig decoder_config;
|
||||
decoder_config.codec_name = "h264";
|
||||
auto decoder_result = sc::DecoderFactory::create(decoder_config);
|
||||
expect_result("decoder create", decoder_result);
|
||||
|
||||
ReceiverSink sink;
|
||||
sink.decoder = std::move(sc::codec_value(decoder_result));
|
||||
|
||||
// Bind the receiver to the first free port in a small range.
|
||||
auto receiver_transport = sc::RtpTransportFactory::create();
|
||||
uint16_t bound_port = 0;
|
||||
for (uint16_t port = 45904; port < 45934; ++port) {
|
||||
if (receiver_transport->start(sc::Endpoint{"0.0.0.0", port},
|
||||
[&sink](sc::RtpPacket packet) { sink.on_packet(std::move(packet)); })) {
|
||||
bound_port = port;
|
||||
break;
|
||||
}
|
||||
}
|
||||
check(bound_port != 0, "receiver transport bound");
|
||||
|
||||
// Sender transport: no bind (OS picks the source port), peer is loopback.
|
||||
auto sender_transport = sc::RtpTransportFactory::create();
|
||||
check(sender_transport->start(sc::Endpoint{"", 0}, [](sc::RtpPacket) {}), "sender transport started");
|
||||
sender_transport->set_peer(sc::Endpoint{"127.0.0.1", bound_port});
|
||||
|
||||
sc::H264Packetizer packetizer;
|
||||
for (uint32_t index = 0; index < kFrameCount; ++index) {
|
||||
const sc::CapturedFrame frame = make_frame(index);
|
||||
|
||||
auto encoded_result = encoder->encode(frame);
|
||||
expect_result("encode", encoded_result);
|
||||
|
||||
for (const sc::EncodedFrame& encoded : sc::codec_value(encoded_result)) {
|
||||
for (const sc::RtpPacket& packet : packetizer.packetize(encoded)) {
|
||||
check(sender_transport->send(packet), "send packet");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Wait (bounded) for the receiver to decode the frames.
|
||||
const auto deadline = std::chrono::steady_clock::now() + std::chrono::seconds(5);
|
||||
while (sink.decoded_frames.load() < kFrameCount && std::chrono::steady_clock::now() < deadline) {
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(10));
|
||||
}
|
||||
|
||||
check(sink.decoded_frames.load() >= kFrameCount / 2, "most frames decoded");
|
||||
check(sink.saw_frame, "decoded frame has pixels");
|
||||
check(sink.last_width == kWidth && sink.last_height == kHeight, "decoded dimensions");
|
||||
|
||||
sender_transport->stop();
|
||||
receiver_transport->stop();
|
||||
|
||||
std::printf("test_loopback: %d frames decoded over UDP loopback\n", sink.decoded_frames.load());
|
||||
return 0;
|
||||
}
|
||||
@@ -85,14 +85,16 @@ int main() {
|
||||
expect_codec_result("encoder create", encoder_result);
|
||||
auto encoder = std::move(sc::codec_value(encoder_result));
|
||||
|
||||
// Without GLOBAL_HEADER the encoder carries no extradata; SPS/PPS are
|
||||
// emitted in-band ahead of every keyframe. This mirrors the streaming
|
||||
// path, where a receiver starts decoding from the bitstream alone.
|
||||
const auto extradata = encoder->get_extradata();
|
||||
assert(!extradata.empty());
|
||||
assert(extradata.empty());
|
||||
|
||||
sc::DecoderConfig decoder_config;
|
||||
decoder_config.codec_name = "h264";
|
||||
decoder_config.width = kWidth;
|
||||
decoder_config.height = kHeight;
|
||||
decoder_config.extradata = extradata;
|
||||
|
||||
auto decoder_result = sc::DecoderFactory::create(decoder_config);
|
||||
expect_codec_result("decoder create", decoder_result);
|
||||
|
||||
@@ -15,3 +15,9 @@ test_rtp = executable('test_rtp',
|
||||
dependencies : sc_network_dep)
|
||||
|
||||
test('rtp framing', test_rtp)
|
||||
|
||||
test_loopback = executable('test_loopback',
|
||||
'app/test_loopback.cpp',
|
||||
dependencies : [sc_codec_dep, sc_network_dep])
|
||||
|
||||
test('udp loopback', test_loopback)
|
||||
|
||||
@@ -97,9 +97,8 @@ int main(int argc, char** argv) {
|
||||
}
|
||||
encoder = std::move(sc::codec_value(encoder_result));
|
||||
|
||||
// The encoder is configured with AV_CODEC_FLAG_GLOBAL_HEADER, so
|
||||
// parameter sets live in extradata; prepend them so the output
|
||||
// file is a self-contained Annex-B stream.
|
||||
// The encoder repeats SPS/PPS in-band at each keyframe, so the
|
||||
// file is a self-contained Annex-B stream without a prefix.
|
||||
const auto extradata = encoder->get_extradata();
|
||||
if (!extradata.empty()) {
|
||||
if (!write_bytes(output, extradata)) {
|
||||
|
||||
Reference in New Issue
Block a user