fegger
d2736b8a94
The receiver window never appeared on Wayland. Two causes: - The window was created on the main thread while event pumping and presenting ran on the render thread. SDL's Wayland backend requires a window's creation, event processing, drawing, and destruction to happen on one thread; a cross-thread surface simply never maps, with no error reported. The renderer now lives entirely on the render thread, with a condition-variable handshake so ReceiverPipeline:: start() still reports renderer failures and timeouts. - On Wayland a window is invisible until the first render commit, so even a healthy receiver showed nothing while waiting for a stream. The renderer presents one blank frame at init: the window is visible immediately, black until video arrives. Also log 'stream started (WxH)' when the first frame decodes and the first rendering failure, which is what made the remaining debugging observable. Validated headlessly end-to-end: a synthetic solid-color RTP feed drove the real receiver over localhost UDP; the window mapped, reported 'stream started (320x240)', and a grim screenshot of the window region showed the fed color (V=198, SATAVG=74 during the red feed). Phase 5 marked complete in docs/PHASES.md; current phase is now Phase 6.
6.2 KiB
6.2 KiB
Project Memory — screen_cast
Last updated: Phase 5 implemented and automated tests green; manual windowed loopback validation pending. See bottom for the run commands.
Project state
- Phase 5 (local UDP sender→receiver loopback) is implemented:
UdpRtpTransport(src/network/udp_transport.cpp): raw POSIX sockets, AF_INET, IPv4 via getaddrinfo; port 0 skips binding (sender side); stop() closes the socket to unblock the receive jthread. ASIO was deliberately deferred to Phase 6 (see decisions).screencastbinary (src/app/): cli.cpp + main.cpp + pipelines.cpp wiring SenderPipeline (capture→encode→packetize→send) and ReceiverPipeline (recv→depacketize→decode→bounded 3-frame queue→render thread).SdlRenderer(src/render/sdl_renderer.cpp): SDL3 window/renderer/texture, RGBA texture upload, texture recreated on resolution change.RendererFactory::createnow returnsRendererResult(error channel added, mirroring codec/capture patterns).- Encoder change: GLOBAL_HEADER removed so libx264 repeats SPS/PPS
in-band at every keyframe; a receiver now decodes from the bitstream
alone (mid-stream join, PLI recovery-ready).
get_extradata()is empty in this mode; codec round-trip test updated to match the streaming path. - Burst-control fixes after the first real run (receiver saw
non-existing PPS 0forever): without VBV, a 2256x1504 IDR burst (~100s of KB of back-to-back FU-A packets) overflowed the ~208KB default UDP receive buffer; the sequence gap made drop-on-damage discard whole keyframes including their in-band SPS/PPS, so the receiver never recovered. Fixes: encoder VBV (rc_max_rate = bitrate,rc_buffer_size = bitrate*2/fps— caps any keyframe to ~2 frame periods of bytes), microsecond encoder time_base (kills the 25fps pts quantization that duplicated timestamps), and a best-effort 4MB SO_RCVBUF on the receive socket (kernel clamps to rmem_max). - Display fixes after the receiver window never appeared: (1) SDL must
own the window from one thread — creating it on main and pumping/presenting
from the render thread left the Wayland surface unmapped; the renderer now
creates/polls/presents/destroys entirely on the render thread (with a
start() init handshake). (2) Wayland windows are invisible until the first
render commit, so the renderer presents a blank frame at init (visible
black window while waiting). (3) The receiver logs
stream started (WxH)on the first decoded frame and any first present failure. - Phase 5 validated end-to-end (headless, without the portal): a
synthetic RTP feed of solid red/green frames drove the real receiver over
localhost UDP; the window mapped, logged
stream started (320x240), and agrimscreenshot of the window region showed the fed color (V=198, SAT=74 during the red feed) — decoded video visibly rendering. The user's real sender run showed capture→encode→send working (1 IDR + 12 P-frames, IDR capped at ~20KB by the VBV).
- Automated validation:
meson test4/4 — newudp loopbacktest encodes synthetic frames, packetizes, sends over a real localhost UDP socket, depacketizes, and decodes 10/10 frames with correct dimensions. Valgrind clean (loopback + codec tests). - Manual validation pending (needs the desktop): run the two commands in
docs/RUNBOOK.md— receiver window should show the captured desktop. Tickdocs/PHASES.mdPhase 5 after this works. - Phase 4 network framing done (RFC 3550 + RFC 6184 single-NAL/FU-A; 3-byte canonical start codes; drop-on-damage loss handling).
- Phase 3 capture done and validated (PipeWire/portal backend; the
impl_ext_end_proxywrong-context warnings were fixed by holding the thread-loop lock across all pw proxy operations).
Decisions
- Language: C++20 with explicit modern-C++ guidelines in
AGENTS.mdandcpp-meson-build/SKILL.md. - Build system: Meson.
- Capture: PipeWire + xdg-desktop-portal.
- Encode/Decode: FFmpeg (libavcodec, libavutil, libswscale).
- H.264 encoder path: software
libx264, low-latency settings, Annex-B output. - SPS/PPS are sent in-band ahead of every keyframe (no GLOBAL_HEADER);
the decoder starts from the bitstream alone.
DecoderConfig.extradataremains available if signaling ever negotiates parameters out of band. - Transport: RTP over UDP via raw POSIX sockets for now; ASIO stays a Phase 6+ option (the ARCHITECTURE.md dependency table places it with signaling/discovery). IPv4 only at the transport level for now.
- Rendering: SDL3 (
sdl3pkg-config, 3.4 installed); plain texture upload, no GPU pipeline yet. - Discovery: mDNS/Avahi.
- Namespace:
sc. - Module error results use per-module
std::variant<T, XError>types (CodecResult,CaptureResult,RendererResult) since C++20 has nostd::expected.
Active blockers
None.
Open questions
- GUI framework (Qt6 vs. none / CLI only) — deferred to later phase.
- Hardware acceleration strategy (VAAPI / Vulkan Video / NVENC) — evaluate after software encode path works.
- IPv6 at the transport layer — revisit when LAN streaming lands (Phase 6+).
Forward-looking review notes (for later phases)
RtpTransport::start/sendreturn plain bools (scaffold API); error messages are lost — consider an error channel when signaling lands.- Very high bitrates can still exceed even a raised receive buffer if
net.core.rmem_maxis low on the receiver; the encoder VBV bounds bursts to ~2 frame periods, so this needs--bitrate≳ 100 Mbps to matter. Document in RUNBOOK. - DMA-BUF-only portal streams are rejected with a clear message (hardware path is Phase 7).
- No negative-path tests yet (bad config, bad stride, undersized buffer).
to_annex_b_h264sniffs AVCC vs Annex-B by content; if an AVCC-emitting encoder is ever added, prefer an explicit config flag over the heuristic.- Region targets are rejected: the desktop portal has no region capture.
CaptureSession::next_frame()returnsnullopton stream error without surfacing the reason (logged to stderr).- Receiver ignores unknown packetization modes (STAP-A/MTAP/FU-B); senders we control never emit them, but third-party interop would need support.