# Project Memory — screen_cast

Last updated: Phase 6 (discovery + signaling) complete and validated over
loopback; current phase is Phase 7.

## Project state

- **Phase 6 done**: Avahi mDNS discovery (`_screencast._tcp` — receiver
  announces its signaling port via a threaded-poll Avahi client; senders
  browse+resolve) plus JSON session signaling (offer/answer) over TCP with
  a signaling server on the receiver (port 5005). `--send` auto-discovers
  when exactly one receiver is found; `--discover` lists receivers;
  `--peer` targets a receiver directly. `meson test` 5/5, valgrind clean.
- **Signaling is newline-delimited JSON over TCP**, not WebSocket: no WS
  library was installed and the rtp-networking skill permits plain TCP.
  The wire format (one JSON object per line: offer/answer with session id,
  codec, rtp port) is transport-agnostic; Phase 7 adds the WS dependency if
  needed.
- **Dual-stack everywhere**: this machine resolves its own services over
  IPv6 (ULA + link-local), so the UDP transport, signaling client, and both
  listeners now support both families (IPv6 sockets with IPV6_V6ONLY=0 for
  dual-stack listening; AF_UNSPEC getaddrinfo for peers). Validated over
  both 127.0.0.1 and ::1.
- **Discovery dedupe**: one entry per (service_name, signaling_port) — a
  host with many interfaces otherwise registers dozens of address variants.
- **mDNS operational lesson (hit during validation)**: `kill -9` on a
  process holding an avahi registration leaves stale daemon records; the
  service then browses but never resolves (timeout for every peer) until
  records expire (~75 min). `systemctl restart avahi-daemon` clears it.
  Recorded in RUNBOOK; always stop with SIGTERM.

- **Phase 5 (local UDP sender→receiver loopback) is implemented**:
  - `UdpRtpTransport` (src/network/udp_transport.cpp): raw POSIX sockets,
    AF_INET, IPv4 via getaddrinfo; port 0 skips binding (sender side); stop()
    closes the socket to unblock the receive jthread. ASIO was deliberately
    deferred to Phase 6 (see decisions).
  - `screencast` binary (src/app/): cli.cpp + main.cpp + pipelines.cpp wiring
    SenderPipeline (capture→encode→packetize→send) and ReceiverPipeline
    (recv→depacketize→decode→bounded 3-frame queue→render thread).
  - `SdlRenderer` (src/render/sdl_renderer.cpp): SDL3 window/renderer/texture,
    RGBA texture upload, texture recreated on resolution change.
    `RendererFactory::create` now returns `RendererResult` (error channel
    added, mirroring codec/capture patterns).
  - Encoder change: **GLOBAL_HEADER removed** so libx264 repeats SPS/PPS
    in-band at every keyframe; a receiver now decodes from the bitstream
    alone (mid-stream join, PLI recovery-ready). `get_extradata()` is empty
    in this mode; codec round-trip test updated to match the streaming path.
  - **Burst-control fixes after the first real run** (receiver saw
    `non-existing PPS 0` forever): without VBV, a 2256x1504 IDR burst (~100s
    of KB of back-to-back FU-A packets) overflowed the ~208KB default UDP
    receive buffer; the sequence gap made drop-on-damage discard whole
    keyframes *including their in-band SPS/PPS*, so the receiver never
    recovered. Fixes: encoder VBV (`rc_max_rate = bitrate`,
    `rc_buffer_size = bitrate*2/fps` — caps any keyframe to ~2 frame periods
    of bytes), microsecond encoder time_base (kills the 25fps pts
    quantization that duplicated timestamps), and a best-effort 4MB
    SO_RCVBUF on the receive socket (kernel clamps to rmem_max).
  - **Display fixes after the receiver window never appeared**: (1) SDL must
    own the window from one thread — creating it on main and pumping/presenting
    from the render thread left the Wayland surface unmapped; the renderer now
    creates/polls/presents/destroys entirely on the render thread (with a
    start() init handshake). (2) Wayland windows are invisible until the first
    render commit, so the renderer presents a blank frame at init (visible
    black window while waiting). (3) The receiver logs `stream started (WxH)`
    on the first decoded frame and any first present failure.
  - **Phase 5 validated end-to-end** (headless, without the portal): a
    synthetic RTP feed of solid red/green frames drove the real receiver over
    localhost UDP; the window mapped, logged `stream started (320x240)`, and a
    `grim` screenshot of the window region showed the fed color (V=198,
    SAT=74 during the red feed) — decoded video visibly rendering. The user's
    real sender run showed capture→encode→send working (1 IDR + 12 P-frames,
    IDR capped at ~20KB by the VBV).
- **Automated validation**: `meson test` 4/4 — new `udp loopback` test
  encodes synthetic frames, packetizes, sends over a real localhost UDP
  socket, depacketizes, and decodes 10/10 frames with correct dimensions.
  Valgrind clean (loopback + codec tests).
- **Manual validation pending (needs the desktop)**: run the two commands in
  `docs/RUNBOOK.md` — receiver window should show the captured desktop. Tick
  `docs/PHASES.md` Phase 5 after this works.
- Phase 4 network framing done (RFC 3550 + RFC 6184 single-NAL/FU-A;
  3-byte canonical start codes; drop-on-damage loss handling).
- Phase 3 capture done and validated (PipeWire/portal backend; the
  `impl_ext_end_proxy` wrong-context warnings were fixed by holding the
  thread-loop lock across all pw proxy operations).

## Decisions

- Language: C++20 with explicit modern-C++ guidelines in `AGENTS.md` and
  `cpp-meson-build/SKILL.md`.
- Build system: Meson.
- Capture: PipeWire + xdg-desktop-portal.
- Encode/Decode: FFmpeg (libavcodec, libavutil, libswscale).
- H.264 encoder path: software `libx264`, low-latency settings, Annex-B output.
- **SPS/PPS are sent in-band ahead of every keyframe** (no GLOBAL_HEADER);
  the decoder starts from the bitstream alone. `DecoderConfig.extradata`
  remains available if signaling ever negotiates parameters out of band.
- Transport: RTP over UDP via raw POSIX sockets for now; **ASIO stays a
  Phase 6+ option** (the ARCHITECTURE.md dependency table places it with
  signaling/discovery). IPv4 only at the transport level for now.
- Rendering: **SDL3** (`sdl3` pkg-config, 3.4 installed); plain texture
  upload, no GPU pipeline yet.
- **Pixel-format convention trap**: FFmpeg names packed formats in memory
  byte order (AV_PIX_FMT_RGBA = R,G,B,A in memory), but SDL names 32-bit
  formats MSB-first (SDL_PIXELFORMAT_RGBA8888 = A,B,G,R in memory).
  FFmpeg RGBA data therefore needs SDL_PIXELFORMAT_ABGR8888 — using
  RGBA8888 paints the alpha byte as red (red-tinted image).
- Discovery: mDNS/Avahi.
- Namespace: `sc`.
- Module error results use per-module `std::variant<T, XError>` types
  (`CodecResult`, `CaptureResult`, `RendererResult`) since C++20 has no
  `std::expected`.

## Active blockers

None.

## Open questions

- GUI framework (Qt6 vs. none / CLI only) — deferred to later phase.
- Hardware acceleration strategy (VAAPI / Vulkan Video / NVENC) — evaluate after
  software encode path works.
- IPv6 at the transport layer — revisit when LAN streaming lands (Phase 6+).

## Forward-looking review notes (for later phases)

- `RtpTransport::start/send` return plain bools (scaffold API); error
  messages are lost — consider an error channel when signaling lands.
- Very high bitrates can still exceed even a raised receive buffer if
  `net.core.rmem_max` is low on the receiver; the encoder VBV bounds
  bursts to ~2 frame periods, so this needs `--bitrate` ≳ 100 Mbps to
  matter. Document in RUNBOOK.
- DMA-BUF-only portal streams are rejected with a clear message (hardware
  path is Phase 7).
- No negative-path tests yet (bad config, bad stride, undersized buffer).
- `to_annex_b_h264` sniffs AVCC vs Annex-B by content; if an AVCC-emitting
  encoder is ever added, prefer an explicit config flag over the heuristic.
- Region targets are rejected: the desktop portal has no region capture.
- `CaptureSession::next_frame()` returns `nullopt` on stream error without
  surfacing the reason (logged to stderr).
- Receiver ignores unknown packetization modes (STAP-A/MTAP/FU-B); senders
  we control never emit them, but third-party interop would need support.