screen_cast/docs/PHASES.md

# Development Phases

This file breaks the project into incremental milestones. Each phase produces a
working, testable slice of functionality. Do not start a phase until the
previous one is validated.

## Phase 1 — Project Skeleton

**Goal**: configure, compile, and run tests with no real dependencies.

- [x] Meson build files (`meson.build`, `meson_options.txt`).
- [x] Public module headers with `namespace sc`.
- [x] Unit test that exercises a trivial utility function.
- [x] `README.md` with build instructions.

**Validation**: `meson setup build && meson compile -C build && meson test -C build`.

## Phase 2 — Software H.264 Encode / Decode

**Goal**: encode raw pixel buffers to H.264 and decode them back, purely with
FFmpeg software paths.

- [x] Add `libavcodec`, `libavutil`, `libswscale` dependencies.
- [x] Implement `EncoderFactory::create()` and `Encoder::encode()`.
- [x] Implement `DecoderFactory::create()` and `Decoder::decode()`.
- [x] Round-trip test: synthetic RGB frames → H.264 → decoded RGB.

**Validation**: unit test produces visually/structurally correct round-trip
frames.

## Phase 3 — PipeWire Screen Capture

**Goal**: capture the desktop and feed frames into the encoder.

- [x] Add `libpipewire-0.3` dependency.
- [x] Implement `CaptureFactory::create()` using the xdg-desktop-portal.
- [x] Wire `CaptureSession::next_frame()` → `Encoder::encode()` in a local smoke
  test that just writes a few encoded frames to disk.

**Validation**: manual run on a real Linux desktop session produces a valid
H.264 bitstream.

## Phase 4 — RTP Framing

**Goal**: packetize NAL units into RTP and depacketize them.

- [x] Implement `RtpHeader` and `RtpPacket` serialize/parse.
- [x] Add H.264 NAL splitting and FU-A fragmentation.
- [x] Unit test for serialization, fragmentation, and reassembly.

**Validation**: unit tests cover single-NAL and fragmented packet paths.

## Phase 5 — Local UDP Sender → Receiver Loopback

**Goal**: send RTP packets over UDP and render the result locally.

- [x] Implement `RtpTransport` with ASIO or raw UDP sockets.
- [x] Wire sender pipeline: capture → encode → RTP → localhost UDP.
- [x] Wire receiver pipeline: localhost UDP → RTP → decode → renderer.
- [x] Add SDL2/SDL3 dependency and a minimal `Renderer`.

**Validation**: `screencast --send` and `screencast --receive` on the same
machine show the captured desktop in a window.

## Phase 6 — LAN Signaling and Discovery

**Goal**: two peers on the same LAN can find each other and negotiate a
session.

- [x] Implement mDNS/DNS-SD discovery with Avahi.
- [x] Implement JSON signaling (newline-delimited JSON over TCP for now;
      the WebSocket transport is deferred to Phase 7 with its dependency).
- [x] Extend CLI with `--peer` and `--discover`.

**Validation**: two peers connect without hard-coded IP addresses.
Validated over loopback AND on real two-machine hardware: a Raspberry
Pi Zero 2 W (Wi-Fi, headless KMSDRM, systemd autostart) was discovered,
negotiated with, and streamed to from the desktop; the receiver rendered
a synthetic feed fullscreen on its display. On that network the LAN's
6to4 addresses are unreachable between peers, so senders try all
discovered addresses in reachability order (private IPv4 first).

## Phase 7 — Resilience and Polish

**Goal**: loss recovery, hardware acceleration, and polish.

- [x] PLI keyframe feedback over signaling (receiver asks, sender
      re-encodes a keyframe; validated end-to-end with an induced loss).
- [x] Jitter/reorder window on the receiver (16 packets / 60 ms).
- [x] Hardware H.264 decode probe with software fallback (h264_v4l2m2m —
      the Pi's VideoCore path; --swdecode opts out).
- [x] GTK4 sender GUI behind `meson -Dgui=true` (gtkmm; receiver list,
      bitrate, start/stop) plus a waybar widget (`screencast waybar`,
      click-to-toggle, right-click opens the panel; state shared via
      $XDG_RUNTIME_DIR/screencast/sender.json).
- [ ] VAAPI hardware encode probe on the sender (deferred: software
      encode is not the bottleneck).
- Deferred: .desktop file, packaging.

**Validation**: sustained streaming under packet loss; hardware accel smoke
where available. PLI + jitter validated on the desktop (induced-loss probe);
hardware decode validated as clean-fallback on the desktop, hardware path
pending a run on the Pi.

## Phase 8 — Android Receiver App

**Goal**: a native Android receiver app (Kotlin) so a phone can act as the
second receiver — screen on, USB-C DP-alt-mode to HDMI. The app speaks the
existing signaling + RTP protocol; no C++ changes.

- [x] 8.1 Gradle project builds (`android/`, AGP 9 built-in Kotlin, no
      androidx; `gradle :app:assembleDebug`)
- [x] 8.2 Kotlin sources + JVM unit tests (rtp framing, jitter buffer,
      depacketizer — 25 tests green)
- [x] 8.3 Signaling + NSD validated on device (offer → answer over the
      network; mDNS registration via the API-36 RegistrationListener)
      — **correction 2026-09-10**: registration had in fact never
      succeeded (registerService passed protocol `0`, which API 36
      rejects with "Unsupported protocol"; the swallowed failure was
      hidden behind the "Listening…" status). Fixed with
      `PROTOCOL_DNS_SD` + real on-device validation: desktop
      `--discover` lists the phone and a live `--send` session
      decodes and renders.
- [x] 8.4 MediaCodec decode + Surface render validated on device (in-band
      SPS sizing, letterbox fit)
- [x] 8.5 End-to-end on a Fairphone 6 (Android 16): streaming, letterboxed
      fullscreen render, PLI keyframe recovery on Wi-Fi loss
- [x] 8.6 Docs: README receiver section, RUNBOOK build/install/test,
      PHASES + MEMORY updates

**Validation**: `gradle :app:testDebugUnitTest` green; live session
`--send --target window --peer <phone>:5005` rendered fullscreen on the
phone (and HDMI via DP-alt-mode) with loss recovery.

## Phase 9 — iOS Receiver App

**Goal**: a native iOS receiver app (Swift) so an iPhone can act as the second
receiver. The app speaks the existing signaling + RTP protocol; no C++ changes.
Mirrors the Android receiver (Phase 8) source-to-source.

- [x] 9.1 Xcode project scaffolding (`ios/`, XcodeGen spec + bootstrap script;
      Info.plist with local-network + Bonjour privacy keys)
- [x] 9.2 Swift protocol core + XCTest (rtp header/packet, jitter, depacketizer
      ported source-to-source; added signaling JSON, line framing, AVCC, NAL
      extraction tests — the Android side had no signaling tests)
- [x] 9.3 Signaling server (BSD sockets, dual-stack, most-recent-peer, never
      throws) + `NSBonjourServices` advertisement
- [x] 9.4 Media path: UDP (poll-based) → jitter → depacketize → VideoToolbox
      (in-band SPS/PPS, real-time decode) → AVSampleBufferDisplayLayer
      (letterbox) + PLI on damage + pendingOffer for late surface attach
- [ ] 9.5 On-device validation (needs a Mac + Xcode 26 + iPhone 16): unit tests
      green on the simulator; `--discover` lists the phone; `--send --peer
      <ip>:5005` streams letterboxed; PLI recovery on Wi-Fi loss
- [ ] 9.6 Docs: iOS RUNBOOK quirks, README receiver section, PHASES + MEMORY

**Validation**: `ios/bootstrap.sh test` (simulator) green; live session
`--send --peer <iphone>:5005` renders fullscreen letterboxed on the iPhone with
loss recovery. Note: this box is Linux — no Xcode/iOS SDK — so 9.2–9.4 are
authored but only 9.5 can validate the VideoToolbox + local-network paths.

## Current phase

Phase 9 — iOS Receiver App (implementation authored; on-device validation
pending a Mac + Xcode 26 + iPhone 16).