fegger
7be8d59d07
Add mDNS/DNS-SD discovery and JSON session negotiation so two peers on a LAN connect without hard-coded addresses. - Discovery (Avahi threaded-poll client): the receiver announces _screencast._tcp with its signaling port; senders browse and resolve peers. Strict lock ordering (poll lock before state mutex) keeps the callbacks deadlock-free; name collisions rename via avahi_alternative_service_name. - Signaling: one JSON object per newline-terminated TCP line. The receiver hosts a server (port 5005) and answers session offers with its RTP port; senders connect, offer, and stream to the negotiated endpoint. WebSocket was deferred: no WS library is installed, the skill permits plain TCP, and the wire format is transport-agnostic. - Dual-stack transports: this machine resolves its own services over IPv6, so getaddrinfo now runs AF_UNSPEC and listeners bind IPv6 with IPV6_V6ONLY=0 (IPv4 fallback), covering UDP and TCP alike. The signaling client shutdown now uses shutdown() so a reader blocked in recv() cannot hang the join (a plain close() does not wake it). - CLI: --discover lists receivers (deduped to one entry per host); --send auto-disovers when exactly one receiver is found; --peer targets a receiver directly; --signaling-port overrides the default. Validation: meson test 5/5 (new signaling round-trip test), valgrind clean. End-to-end over loopback: --discover finds the announced receiver, a probe negotiated a session and streamed 60 frames over both IPv4 and IPv6, and the receiver reported stream started. mDNS resolution was verified against avahi-browse as an independent reference.
141 lines
8.1 KiB
Markdown
141 lines
8.1 KiB
Markdown
# Project Memory — screen_cast
|
|
|
|
Last updated: Phase 6 (discovery + signaling) complete and validated over
|
|
loopback; current phase is Phase 7.
|
|
|
|
## Project state
|
|
|
|
- **Phase 6 done**: Avahi mDNS discovery (`_screencast._tcp` — receiver
|
|
announces its signaling port via a threaded-poll Avahi client; senders
|
|
browse+resolve) plus JSON session signaling (offer/answer) over TCP with
|
|
a signaling server on the receiver (port 5005). `--send` auto-discovers
|
|
when exactly one receiver is found; `--discover` lists receivers;
|
|
`--peer` targets a receiver directly. `meson test` 5/5, valgrind clean.
|
|
- **Signaling is newline-delimited JSON over TCP**, not WebSocket: no WS
|
|
library was installed and the rtp-networking skill permits plain TCP.
|
|
The wire format (one JSON object per line: offer/answer with session id,
|
|
codec, rtp port) is transport-agnostic; Phase 7 adds the WS dependency if
|
|
needed.
|
|
- **Dual-stack everywhere**: this machine resolves its own services over
|
|
IPv6 (ULA + link-local), so the UDP transport, signaling client, and both
|
|
listeners now support both families (IPv6 sockets with IPV6_V6ONLY=0 for
|
|
dual-stack listening; AF_UNSPEC getaddrinfo for peers). Validated over
|
|
both 127.0.0.1 and ::1.
|
|
- **Discovery dedupe**: one entry per (service_name, signaling_port) — a
|
|
host with many interfaces otherwise registers dozens of address variants.
|
|
- **mDNS operational lesson (hit during validation)**: `kill -9` on a
|
|
process holding an avahi registration leaves stale daemon records; the
|
|
service then browses but never resolves (timeout for every peer) until
|
|
records expire (~75 min). `systemctl restart avahi-daemon` clears it.
|
|
Recorded in RUNBOOK; always stop with SIGTERM.
|
|
|
|
- **Phase 5 (local UDP sender→receiver loopback) is implemented**:
|
|
- `UdpRtpTransport` (src/network/udp_transport.cpp): raw POSIX sockets,
|
|
AF_INET, IPv4 via getaddrinfo; port 0 skips binding (sender side); stop()
|
|
closes the socket to unblock the receive jthread. ASIO was deliberately
|
|
deferred to Phase 6 (see decisions).
|
|
- `screencast` binary (src/app/): cli.cpp + main.cpp + pipelines.cpp wiring
|
|
SenderPipeline (capture→encode→packetize→send) and ReceiverPipeline
|
|
(recv→depacketize→decode→bounded 3-frame queue→render thread).
|
|
- `SdlRenderer` (src/render/sdl_renderer.cpp): SDL3 window/renderer/texture,
|
|
RGBA texture upload, texture recreated on resolution change.
|
|
`RendererFactory::create` now returns `RendererResult` (error channel
|
|
added, mirroring codec/capture patterns).
|
|
- Encoder change: **GLOBAL_HEADER removed** so libx264 repeats SPS/PPS
|
|
in-band at every keyframe; a receiver now decodes from the bitstream
|
|
alone (mid-stream join, PLI recovery-ready). `get_extradata()` is empty
|
|
in this mode; codec round-trip test updated to match the streaming path.
|
|
- **Burst-control fixes after the first real run** (receiver saw
|
|
`non-existing PPS 0` forever): without VBV, a 2256x1504 IDR burst (~100s
|
|
of KB of back-to-back FU-A packets) overflowed the ~208KB default UDP
|
|
receive buffer; the sequence gap made drop-on-damage discard whole
|
|
keyframes *including their in-band SPS/PPS*, so the receiver never
|
|
recovered. Fixes: encoder VBV (`rc_max_rate = bitrate`,
|
|
`rc_buffer_size = bitrate*2/fps` — caps any keyframe to ~2 frame periods
|
|
of bytes), microsecond encoder time_base (kills the 25fps pts
|
|
quantization that duplicated timestamps), and a best-effort 4MB
|
|
SO_RCVBUF on the receive socket (kernel clamps to rmem_max).
|
|
- **Display fixes after the receiver window never appeared**: (1) SDL must
|
|
own the window from one thread — creating it on main and pumping/presenting
|
|
from the render thread left the Wayland surface unmapped; the renderer now
|
|
creates/polls/presents/destroys entirely on the render thread (with a
|
|
start() init handshake). (2) Wayland windows are invisible until the first
|
|
render commit, so the renderer presents a blank frame at init (visible
|
|
black window while waiting). (3) The receiver logs `stream started (WxH)`
|
|
on the first decoded frame and any first present failure.
|
|
- **Phase 5 validated end-to-end** (headless, without the portal): a
|
|
synthetic RTP feed of solid red/green frames drove the real receiver over
|
|
localhost UDP; the window mapped, logged `stream started (320x240)`, and a
|
|
`grim` screenshot of the window region showed the fed color (V=198,
|
|
SAT=74 during the red feed) — decoded video visibly rendering. The user's
|
|
real sender run showed capture→encode→send working (1 IDR + 12 P-frames,
|
|
IDR capped at ~20KB by the VBV).
|
|
- **Automated validation**: `meson test` 4/4 — new `udp loopback` test
|
|
encodes synthetic frames, packetizes, sends over a real localhost UDP
|
|
socket, depacketizes, and decodes 10/10 frames with correct dimensions.
|
|
Valgrind clean (loopback + codec tests).
|
|
- **Manual validation pending (needs the desktop)**: run the two commands in
|
|
`docs/RUNBOOK.md` — receiver window should show the captured desktop. Tick
|
|
`docs/PHASES.md` Phase 5 after this works.
|
|
- Phase 4 network framing done (RFC 3550 + RFC 6184 single-NAL/FU-A;
|
|
3-byte canonical start codes; drop-on-damage loss handling).
|
|
- Phase 3 capture done and validated (PipeWire/portal backend; the
|
|
`impl_ext_end_proxy` wrong-context warnings were fixed by holding the
|
|
thread-loop lock across all pw proxy operations).
|
|
|
|
## Decisions
|
|
|
|
- Language: C++20 with explicit modern-C++ guidelines in `AGENTS.md` and
|
|
`cpp-meson-build/SKILL.md`.
|
|
- Build system: Meson.
|
|
- Capture: PipeWire + xdg-desktop-portal.
|
|
- Encode/Decode: FFmpeg (libavcodec, libavutil, libswscale).
|
|
- H.264 encoder path: software `libx264`, low-latency settings, Annex-B output.
|
|
- **SPS/PPS are sent in-band ahead of every keyframe** (no GLOBAL_HEADER);
|
|
the decoder starts from the bitstream alone. `DecoderConfig.extradata`
|
|
remains available if signaling ever negotiates parameters out of band.
|
|
- Transport: RTP over UDP via raw POSIX sockets for now; **ASIO stays a
|
|
Phase 6+ option** (the ARCHITECTURE.md dependency table places it with
|
|
signaling/discovery). IPv4 only at the transport level for now.
|
|
- Rendering: **SDL3** (`sdl3` pkg-config, 3.4 installed); plain texture
|
|
upload, no GPU pipeline yet.
|
|
- **Pixel-format convention trap**: FFmpeg names packed formats in memory
|
|
byte order (AV_PIX_FMT_RGBA = R,G,B,A in memory), but SDL names 32-bit
|
|
formats MSB-first (SDL_PIXELFORMAT_RGBA8888 = A,B,G,R in memory).
|
|
FFmpeg RGBA data therefore needs SDL_PIXELFORMAT_ABGR8888 — using
|
|
RGBA8888 paints the alpha byte as red (red-tinted image).
|
|
- Discovery: mDNS/Avahi.
|
|
- Namespace: `sc`.
|
|
- Module error results use per-module `std::variant<T, XError>` types
|
|
(`CodecResult`, `CaptureResult`, `RendererResult`) since C++20 has no
|
|
`std::expected`.
|
|
|
|
## Active blockers
|
|
|
|
None.
|
|
|
|
## Open questions
|
|
|
|
- GUI framework (Qt6 vs. none / CLI only) — deferred to later phase.
|
|
- Hardware acceleration strategy (VAAPI / Vulkan Video / NVENC) — evaluate after
|
|
software encode path works.
|
|
- IPv6 at the transport layer — revisit when LAN streaming lands (Phase 6+).
|
|
|
|
## Forward-looking review notes (for later phases)
|
|
|
|
- `RtpTransport::start/send` return plain bools (scaffold API); error
|
|
messages are lost — consider an error channel when signaling lands.
|
|
- Very high bitrates can still exceed even a raised receive buffer if
|
|
`net.core.rmem_max` is low on the receiver; the encoder VBV bounds
|
|
bursts to ~2 frame periods, so this needs `--bitrate` ≳ 100 Mbps to
|
|
matter. Document in RUNBOOK.
|
|
- DMA-BUF-only portal streams are rejected with a clear message (hardware
|
|
path is Phase 7).
|
|
- No negative-path tests yet (bad config, bad stride, undersized buffer).
|
|
- `to_annex_b_h264` sniffs AVCC vs Annex-B by content; if an AVCC-emitting
|
|
encoder is ever added, prefer an explicit config flag over the heuristic.
|
|
- Region targets are rejected: the desktop portal has no region capture.
|
|
- `CaptureSession::next_frame()` returns `nullopt` on stream error without
|
|
surfacing the reason (logged to stderr).
|
|
- Receiver ignores unknown packetization modes (STAP-A/MTAP/FU-B); senders
|
|
we control never emit them, but third-party interop would need support. |