fegger
10870bc6c9
Wire the first end-to-end pipeline: capture -> encode -> packetize -> UDP -> depacketize -> decode -> render. - UdpRtpTransport: raw POSIX UDP sockets (IPv4 via getaddrinfo), a receive jthread woken by socket close on stop; port 0 skips binding so the sender uses an OS-assigned source port. ASIO stays deferred to the signaling phase per ARCHITECTURE.md. - SdlRenderer: SDL3 window/renderer with RGBA texture upload; the texture is recreated on resolution change. RendererFactory now returns RendererResult so SDL init failures carry a message, mirroring the codec/capture error patterns. - screencast binary: parse_cli plus SenderPipeline/ReceiverPipeline per the app scaffolds; the sender creates its encoder once capture reports real dimensions, the receiver keeps a bounded 3-frame queue to hold latency down and renders on its own thread until the window closes. cli argv signature fixed to 'const char* const*' so main's argv converts implicitly. - Encoder: drop AV_CODEC_FLAG_GLOBAL_HEADER so libx264 repeats SPS/PPS in-band at each keyframe -- the receiver decodes from the bitstream alone, which also makes mid-stream joins and later PLI recovery work without out-of-band parameter negotiation. The round-trip test now exercises exactly that path. - tests: new udp-loopback integration test pushes synthetic frames through a real localhost socket and decodes 10/10 frames with the right dimensions; valgrind clean (loopback + codec). meson test 4/4. Manual validation on the desktop (receiver window shows the captured desktop) is documented in docs/RUNBOOK.md.
4.7 KiB
4.7 KiB
Project Memory — screen_cast
Last updated: Phase 5 implemented and automated tests green; manual windowed loopback validation pending. See bottom for the run commands.
Project state
- Phase 5 (local UDP sender→receiver loopback) is implemented:
UdpRtpTransport(src/network/udp_transport.cpp): raw POSIX sockets, AF_INET, IPv4 via getaddrinfo; port 0 skips binding (sender side); stop() closes the socket to unblock the receive jthread. ASIO was deliberately deferred to Phase 6 (see decisions).screencastbinary (src/app/): cli.cpp + main.cpp + pipelines.cpp wiring SenderPipeline (capture→encode→packetize→send) and ReceiverPipeline (recv→depacketize→decode→bounded 3-frame queue→render thread).SdlRenderer(src/render/sdl_renderer.cpp): SDL3 window/renderer/texture, RGBA texture upload, texture recreated on resolution change.RendererFactory::createnow returnsRendererResult(error channel added, mirroring codec/capture patterns).- Encoder change: GLOBAL_HEADER removed so libx264 repeats SPS/PPS
in-band at every keyframe; a receiver now decodes from the bitstream
alone (mid-stream join, PLI recovery-ready).
get_extradata()is empty in this mode; codec round-trip test updated to match the streaming path.
- Automated validation:
meson test4/4 — newudp loopbacktest encodes synthetic frames, packetizes, sends over a real localhost UDP socket, depacketizes, and decodes 10/10 frames with correct dimensions. Valgrind clean (loopback + codec tests). - Manual validation pending (needs the desktop): run the two commands in
docs/RUNBOOK.md— receiver window should show the captured desktop. Tickdocs/PHASES.mdPhase 5 after this works. - Phase 4 network framing done (RFC 3550 + RFC 6184 single-NAL/FU-A; 3-byte canonical start codes; drop-on-damage loss handling).
- Phase 3 capture done and validated (PipeWire/portal backend; the
impl_ext_end_proxywrong-context warnings were fixed by holding the thread-loop lock across all pw proxy operations).
Decisions
- Language: C++20 with explicit modern-C++ guidelines in
AGENTS.mdandcpp-meson-build/SKILL.md. - Build system: Meson.
- Capture: PipeWire + xdg-desktop-portal.
- Encode/Decode: FFmpeg (libavcodec, libavutil, libswscale).
- H.264 encoder path: software
libx264, low-latency settings, Annex-B output. - SPS/PPS are sent in-band ahead of every keyframe (no GLOBAL_HEADER);
the decoder starts from the bitstream alone.
DecoderConfig.extradataremains available if signaling ever negotiates parameters out of band. - Transport: RTP over UDP via raw POSIX sockets for now; ASIO stays a Phase 6+ option (the ARCHITECTURE.md dependency table places it with signaling/discovery). IPv4 only at the transport level for now.
- Rendering: SDL3 (
sdl3pkg-config, 3.4 installed); plain texture upload, no GPU pipeline yet. - Discovery: mDNS/Avahi.
- Namespace:
sc. - Module error results use per-module
std::variant<T, XError>types (CodecResult,CaptureResult,RendererResult) since C++20 has nostd::expected.
Active blockers
None.
Open questions
- GUI framework (Qt6 vs. none / CLI only) — deferred to later phase.
- Hardware acceleration strategy (VAAPI / Vulkan Video / NVENC) — evaluate after software encode path works.
- IPv6 at the transport layer — revisit when LAN streaming lands (Phase 6+).
Forward-looking review notes (for later phases)
- Encoder sets no VBV (
maxrate/buffer_size) — ABR only; add for smoother UDP streaming in Phase 7. - Encoder PTS caveat: time_base derives from the configured frame rate (default 25fps) while portal frames arrive at monitor refresh (often 60Hz), so pts values quantize and can repeat. RTP timestamps come from the capture clock instead, so streaming is unaffected; revisit if decoder-side presentation timing ever matters.
RtpTransport::start/sendreturn plain bools (scaffold API); error messages are lost — consider an error channel when signaling lands.- DMA-BUF-only portal streams are rejected with a clear message (hardware path is Phase 7).
- No negative-path tests yet (bad config, bad stride, undersized buffer).
to_annex_b_h264sniffs AVCC vs Annex-B by content; if an AVCC-emitting encoder is ever added, prefer an explicit config flag over the heuristic.- Region targets are rejected: the desktop portal has no region capture.
CaptureSession::next_frame()returnsnullopton stream error without surfacing the reason (logged to stderr).- Receiver ignores unknown packetization modes (STAP-A/MTAP/FU-B); senders we control never emit them, but third-party interop would need support.