fegger
10870bc6c9
Wire the first end-to-end pipeline: capture -> encode -> packetize -> UDP -> depacketize -> decode -> render. - UdpRtpTransport: raw POSIX UDP sockets (IPv4 via getaddrinfo), a receive jthread woken by socket close on stop; port 0 skips binding so the sender uses an OS-assigned source port. ASIO stays deferred to the signaling phase per ARCHITECTURE.md. - SdlRenderer: SDL3 window/renderer with RGBA texture upload; the texture is recreated on resolution change. RendererFactory now returns RendererResult so SDL init failures carry a message, mirroring the codec/capture error patterns. - screencast binary: parse_cli plus SenderPipeline/ReceiverPipeline per the app scaffolds; the sender creates its encoder once capture reports real dimensions, the receiver keeps a bounded 3-frame queue to hold latency down and renders on its own thread until the window closes. cli argv signature fixed to 'const char* const*' so main's argv converts implicitly. - Encoder: drop AV_CODEC_FLAG_GLOBAL_HEADER so libx264 repeats SPS/PPS in-band at each keyframe -- the receiver decodes from the bitstream alone, which also makes mid-stream joins and later PLI recovery work without out-of-band parameter negotiation. The round-trip test now exercises exactly that path. - tests: new udp-loopback integration test pushes synthetic frames through a real localhost socket and decodes 10/10 frames with the right dimensions; valgrind clean (loopback + codec). meson test 4/4. Manual validation on the desktop (receiver window shows the captured desktop) is documented in docs/RUNBOOK.md.
90 lines
4.7 KiB
Markdown
90 lines
4.7 KiB
Markdown
# Project Memory — screen_cast
|
|
|
|
Last updated: Phase 5 implemented and automated tests green; manual windowed
|
|
loopback validation pending. See bottom for the run commands.
|
|
|
|
## Project state
|
|
|
|
- **Phase 5 (local UDP sender→receiver loopback) is implemented**:
|
|
- `UdpRtpTransport` (src/network/udp_transport.cpp): raw POSIX sockets,
|
|
AF_INET, IPv4 via getaddrinfo; port 0 skips binding (sender side); stop()
|
|
closes the socket to unblock the receive jthread. ASIO was deliberately
|
|
deferred to Phase 6 (see decisions).
|
|
- `screencast` binary (src/app/): cli.cpp + main.cpp + pipelines.cpp wiring
|
|
SenderPipeline (capture→encode→packetize→send) and ReceiverPipeline
|
|
(recv→depacketize→decode→bounded 3-frame queue→render thread).
|
|
- `SdlRenderer` (src/render/sdl_renderer.cpp): SDL3 window/renderer/texture,
|
|
RGBA texture upload, texture recreated on resolution change.
|
|
`RendererFactory::create` now returns `RendererResult` (error channel
|
|
added, mirroring codec/capture patterns).
|
|
- Encoder change: **GLOBAL_HEADER removed** so libx264 repeats SPS/PPS
|
|
in-band at every keyframe; a receiver now decodes from the bitstream
|
|
alone (mid-stream join, PLI recovery-ready). `get_extradata()` is empty
|
|
in this mode; codec round-trip test updated to match the streaming path.
|
|
- **Automated validation**: `meson test` 4/4 — new `udp loopback` test
|
|
encodes synthetic frames, packetizes, sends over a real localhost UDP
|
|
socket, depacketizes, and decodes 10/10 frames with correct dimensions.
|
|
Valgrind clean (loopback + codec tests).
|
|
- **Manual validation pending (needs the desktop)**: run the two commands in
|
|
`docs/RUNBOOK.md` — receiver window should show the captured desktop. Tick
|
|
`docs/PHASES.md` Phase 5 after this works.
|
|
- Phase 4 network framing done (RFC 3550 + RFC 6184 single-NAL/FU-A;
|
|
3-byte canonical start codes; drop-on-damage loss handling).
|
|
- Phase 3 capture done and validated (PipeWire/portal backend; the
|
|
`impl_ext_end_proxy` wrong-context warnings were fixed by holding the
|
|
thread-loop lock across all pw proxy operations).
|
|
|
|
## Decisions
|
|
|
|
- Language: C++20 with explicit modern-C++ guidelines in `AGENTS.md` and
|
|
`cpp-meson-build/SKILL.md`.
|
|
- Build system: Meson.
|
|
- Capture: PipeWire + xdg-desktop-portal.
|
|
- Encode/Decode: FFmpeg (libavcodec, libavutil, libswscale).
|
|
- H.264 encoder path: software `libx264`, low-latency settings, Annex-B output.
|
|
- **SPS/PPS are sent in-band ahead of every keyframe** (no GLOBAL_HEADER);
|
|
the decoder starts from the bitstream alone. `DecoderConfig.extradata`
|
|
remains available if signaling ever negotiates parameters out of band.
|
|
- Transport: RTP over UDP via raw POSIX sockets for now; **ASIO stays a
|
|
Phase 6+ option** (the ARCHITECTURE.md dependency table places it with
|
|
signaling/discovery). IPv4 only at the transport level for now.
|
|
- Rendering: **SDL3** (`sdl3` pkg-config, 3.4 installed); plain texture
|
|
upload, no GPU pipeline yet.
|
|
- Discovery: mDNS/Avahi.
|
|
- Namespace: `sc`.
|
|
- Module error results use per-module `std::variant<T, XError>` types
|
|
(`CodecResult`, `CaptureResult`, `RendererResult`) since C++20 has no
|
|
`std::expected`.
|
|
|
|
## Active blockers
|
|
|
|
None.
|
|
|
|
## Open questions
|
|
|
|
- GUI framework (Qt6 vs. none / CLI only) — deferred to later phase.
|
|
- Hardware acceleration strategy (VAAPI / Vulkan Video / NVENC) — evaluate after
|
|
software encode path works.
|
|
- IPv6 at the transport layer — revisit when LAN streaming lands (Phase 6+).
|
|
|
|
## Forward-looking review notes (for later phases)
|
|
|
|
- Encoder sets no VBV (`maxrate`/`buffer_size`) — ABR only; add for smoother
|
|
UDP streaming in Phase 7.
|
|
- Encoder PTS caveat: time_base derives from the configured frame rate
|
|
(default 25fps) while portal frames arrive at monitor refresh (often 60Hz),
|
|
so pts values quantize and can repeat. RTP timestamps come from the capture
|
|
clock instead, so streaming is unaffected; revisit if decoder-side
|
|
presentation timing ever matters.
|
|
- `RtpTransport::start/send` return plain bools (scaffold API); error
|
|
messages are lost — consider an error channel when signaling lands.
|
|
- DMA-BUF-only portal streams are rejected with a clear message (hardware
|
|
path is Phase 7).
|
|
- No negative-path tests yet (bad config, bad stride, undersized buffer).
|
|
- `to_annex_b_h264` sniffs AVCC vs Annex-B by content; if an AVCC-emitting
|
|
encoder is ever added, prefer an explicit config flag over the heuristic.
|
|
- Region targets are rejected: the desktop portal has no region capture.
|
|
- `CaptureSession::next_frame()` returns `nullopt` on stream error without
|
|
surfacing the reason (logged to stderr).
|
|
- Receiver ignores unknown packetization modes (STAP-A/MTAP/FU-B); senders
|
|
we control never emit them, but third-party interop would need support. |