A native iOS receiver so an iPhone can act as the second receiver, speaking the existing signaling + RTP protocol (no C++ changes) and mirroring the Android receiver (Phase 8) source-to-source.
- RTP core (header/packet, jitter buffer, H.264 depacketizer) ported from the Android receiver
- BSD-socket signaling server (dual-stack, most-recent-peer, never-throwing sends) + NSBonjourServices
- VideoToolbox H.264 decode (in-band SPS/PPS, real-time, rebuilds on size change) -> AVSampleBufferDisplayLayer
- PLI keyframe recovery (500 ms) + pendingOffer for late surface attach
- XcodeGen project + bootstrap.sh; XCTest port of the Android suite + new coverage
- .gitignore for generated artifacts; CHANGELOG; PHASES + MEMORY updated
Status: authored; on-device validation pending a Mac + Xcode 26 + iPhone 16.
Android 16 added NsdManager.checkProtocol(): registerService with protocol
0 threw IllegalArgumentException: Unsupported protocol — silently, since
the old code swallowed the exception into a dead reflection fallback and
start() overwrote the failure status with the Listening line. mDNS never
actually advertised (the Phase 8 session streamed via --peer, masking it).
Also advertise after the listening status so a registration failure stays
visible, and log the exception and the registration success.
Validated live on the Fairphone 6: the desktop --discover lists the phone
and a full --send session decodes (in-band SPS 320x240 -> 2496x1040) and
renders.
- SignalingServer.send() never throws: a broken signaling connection
drops the peer instead of killing the RTP reader thread via requestPli()
- H264Decoder: assign the codec before configure/start (no orphaned
instance on failure); feed() reports input-queue timeouts so the
pipeline requests a keyframe instead of dropping the frame silently
- Park offers that arrive without a surface as pendingOffer and configure
on attachSurface(): a codec configured without a surface can never take
one (setOutputSurface refuses it); the late configure sends a PLI
- Forget destroyed surfaces (onSurfaceTextureDestroyed -> detachSurface)
- Show the status overlay again on later messages
- Accumulate NAL bytes in ByteArrayOutputStreams, not boxed ArrayList<Int>
- Delete the dead NSD reflection fallback (ResolutionListener never
existed; the classic API is present from API 16 through 36)
- Hold decoderLock around all MediaCodec calls (not thread-safe) and make
requestPli thread-safe
- Update RUNBOOK quirks (NSD, setOutputSurface) and MEMORY notes
README: phone-as-second-screen section (build, install, sender usage,
multi-monitor caveat). RUNBOOK: Android build/test procedure, device-test
commands, and the API-36 platform quirks found during validation.
PHASES: Phase 8 milestones, all validated. MEMORY: current phase state
and the hard-won Android/portal lessons.
screencast_icon/ holds the single source icon set; meson install ships
the 256 px PNG into the hicolor theme for the GTK panel (themed icon
name) and the waybar CSS, and a custom target embeds it as
icon_png_data.h so the SDL receiver window sets it via SDL_SetWindowIcon
with an optional SDL3_image dependency (built-in find_library, since no
pkg-config file ships with it).
A gtkmm-4.0 control panel (behind -Dgui=true, default off): refresh
shows discovered receivers (grouped and preference-sorted), a bitrate
scale, and start/stop that runs the whole session on a worker thread
so the interactive portal picker never blocks the UI. The CLI and the
GUI now share the new sc_app_core static library holding the
pipelines, session orchestration (negotiation + PLI feedback), and a
state store.
The sender pipeline publishes its state to
$XDG_RUNTIME_DIR/screencast/sender.json (session id, receiver,
bitrate, pid, start time; stale files detected by pid liveness) and
persists the last session for one-click restarts. The new
'screencast waybar' subcommand prints a waybar module line and its
--toggle flag stops a running sender gracefully or spawns a detached
restart of the last receiver.
Waybar on the dev machine is wired: custom/screencast module with
click-to-toggle and right-click panel, plus styles, with a timestamped
backup of both config files. Both binaries are installed to
/usr/local/bin.
Validated: waybar output (idle and streaming states with a synthetic
state file), GUI launches on the desktop (window observed via
hyprctl), meson test 5/5 in both build configurations, formatting
clean.
scripts/pi-hotspot.sh turns the receiver into a WPA2 access point via
NetworkManager (ipv4 shared mode gives the Pi built-in DHCP/NAT at
10.42.0.1), so a sender connects directly with no router in between —
which also sidesteps LAN quirks like unreachable 6to4 addresses, since
the direct link is plain private IPv4. The passphrase is generated on
first use and stored root-only in /etc/screencast-hotspot.conf;
on|off|status subcommands manage it, and off restores normal client
Wi-Fi. The hotspot owns wlan0 while active (documented). No receiver
changes were needed: it already announces on every interface.
nmcli property syntax validated against NetworkManager 1.58 with a
disposable never-activated profile; actual AP bring-up can only be
validated on the Pi.
The full chain — mDNS discovery, signaling negotiation, RTP over
Wi-Fi, software H.264 decode, and fullscreen KMSDRM letterboxed
rendering — is confirmed working on a real two-machine setup
(desktop -> Raspberry Pi Zero 2 W headless receiver). The probe
gradient appeared on the Pi's display.
UDP receive buffers clamp to net.core.rmem_max, whose stock Debian
default (~208 KB) is smaller than one VBV-bounded keyframe burst at
moderate-to-high bitrates. The resulting packet loss damages every
keyframe, drop-on-damage discards them silently, and the receiver
shows a black screen with no journal errors while the sender streams
happily. The install script now raises rmem_max to 4 MB and persists
the sysctl; buffers are allocated lazily, so this costs no RAM at
rest. Wi-Fi-only boards (Pi Zero 2 W) remain bandwidth-bound — the
RUNBOOK points at --bitrate as the tuning knob.
The receiver service never started on the Pi: systemd blocks before
exec in acquire_terminal(), waiting for a controlling terminal that
the console session or getty on tty1 already owns. Symptoms: the
status shows the main PID as "(screencast)" with Tasks:1 and ~40ms
CPU, a silent journal, and no mDNS announcement — the binary never
ran. Reproduced locally with a transient unit (the desktop's Wayland
session owns tty1) and diagnosed by gdb-attaching the stuck process.
The service now owns a dedicated free VT (tty7) so acquisition is
immediate, with a tolerant ExecStartPre chvt to make the screencast
the on-screen console at boot. tty1 keeps the console login;
Ctrl+Alt+F1/Ctrl+Alt+F7 switch between them. The previous advice to
enable Console Autologin actively caused the hang and is removed
from the unit, RUNBOOK, and install script notes.
Validated locally: the same properties via systemd-run give a
running receiver with all ~15 threads alive that announces itself
and is found by --discover; verified unit passes systemd-analyze
verify; graceful stop withdraws mDNS.
A headless receiver has no window manager, so a windowed window is
meaningless there — the screencast should simply fill the screen. The
renderer now goes borderless fullscreen automatically when SDL runs
the KMSDRM backend, and --fullscreen forces the same behavior in
desktop sessions. The video keeps its aspect ratio via
SDL_SetRenderLogicalPresentation(LETTERBOX): a 4:3 desktop on a 16:9
TV renders with black bars instead of a stretched picture, the cursor
is hidden, and the clear-before-draw keeps the bars black.
Verified live on a 3440x1440 monitor: the --fullscreen receiver window
covered the entire display while a 4:3 synthetic feed rendered with
pure-black pillarbox bars (Y=0/SAT=0) and a saturated-red center
(V=254). meson test 5/5 in both build configurations.
The receiver does not need a window manager: SDL3's KMSDRM backend
renders straight to the kernel display pipeline, which is the right
setup for a small-board appliance. But a systemd system service has
no controlling terminal, and SDL's KMSDRM backend expects one for
its VT handling — so the headless path would have failed at boot.
Give the unit StandardInput=tty with TTYPath=/dev/tty1 (harmless in
desktop mode), and document the headless recipe in the unit, the
install script output, and the RUNBOOK: Raspberry Pi OS Lite with
Console Autologin, no desktop enabled (a compositor would hold the
DRM master), and consoleblank=0.
Ship systemd/screencast-receiver.service, a template the install
script substitutes (binary path, run user) and enables so the
receiver starts at boot after the network and avahi-daemon, with
restart-on-failure and a raised start limit for slow-booting boards.
SIGTERM gives the graceful shutdown that withdraws the mDNS
announcement — systemctl stop never triggers the stale-record trap
that kill -9 does. Headless consoles render via KMSDRM (the script
adds the run user to video/render/input); desktop autologin sessions
uncomment two documented Environment lines.
The install script gains SC_RECEIVER_SERVICE=0 to opt out.
Validated: systemd-analyze verify on the substituted unit (only the
expected missing /usr/local/bin path on this dev machine), and a live
lifecycle test as a user unit: the service-managed receiver announced
itself (--discover found it), systemctl stop produced a clean exit
(Result=success, no restart) and the mDNS announcement was withdrawn
immediately.
Add mDNS/DNS-SD discovery and JSON session negotiation so two peers on
a LAN connect without hard-coded addresses.
- Discovery (Avahi threaded-poll client): the receiver announces
_screencast._tcp with its signaling port; senders browse and resolve
peers. Strict lock ordering (poll lock before state mutex) keeps the
callbacks deadlock-free; name collisions rename via
avahi_alternative_service_name.
- Signaling: one JSON object per newline-terminated TCP line. The
receiver hosts a server (port 5005) and answers session offers with
its RTP port; senders connect, offer, and stream to the negotiated
endpoint. WebSocket was deferred: no WS library is installed, the
skill permits plain TCP, and the wire format is transport-agnostic.
- Dual-stack transports: this machine resolves its own services over
IPv6, so getaddrinfo now runs AF_UNSPEC and listeners bind IPv6 with
IPV6_V6ONLY=0 (IPv4 fallback), covering UDP and TCP alike. The
signaling client shutdown now uses shutdown() so a reader blocked in
recv() cannot hang the join (a plain close() does not wake it).
- CLI: --discover lists receivers (deduped to one entry per host);
--send auto-disovers when exactly one receiver is found; --peer
targets a receiver directly; --signaling-port overrides the default.
Validation: meson test 5/5 (new signaling round-trip test), valgrind
clean. End-to-end over loopback: --discover finds the announced
receiver, a probe negotiated a session and streamed 60 frames over
both IPv4 and IPv6, and the receiver reported stream started. mDNS
resolution was verified against avahi-browse as an independent
reference.
The receiver window never appeared on Wayland. Two causes:
- The window was created on the main thread while event pumping and
presenting ran on the render thread. SDL's Wayland backend requires a
window's creation, event processing, drawing, and destruction to
happen on one thread; a cross-thread surface simply never maps, with
no error reported. The renderer now lives entirely on the render
thread, with a condition-variable handshake so ReceiverPipeline::
start() still reports renderer failures and timeouts.
- On Wayland a window is invisible until the first render commit, so
even a healthy receiver showed nothing while waiting for a stream.
The renderer presents one blank frame at init: the window is visible
immediately, black until video arrives.
Also log 'stream started (WxH)' when the first frame decodes and the
first rendering failure, which is what made the remaining debugging
observable.
Validated headlessly end-to-end: a synthetic solid-color RTP feed drove
the real receiver over localhost UDP; the window mapped, reported
'stream started (320x240)', and a grim screenshot of the window region
showed the fed color (V=198, SATAVG=74 during the red feed). Phase 5
marked complete in docs/PHASES.md; current phase is now Phase 6.
The first real sender/receiver run failed permanently: the receiver
reported 'non-existing PPS 0' for every frame. Without a VBV, a
2256x1504 IDR keyframe bursts hundreds of kilobytes of back-to-back
FU-A packets, overflowing the ~208KB default UDP receive buffer; the
resulting sequence gap made the depacketizer drop whole keyframes
including their in-band SPS/PPS, so the decoder never initialized and
never recovered, because every keyframe burst overflowed again.
- Encoder: add rc_max_rate = bitrate and rc_buffer_size =
bitrate*2/fps, capping any single frame to about two frame periods
of bytes (~40KB at the 4Mbps default).
- Encoder: switch the time_base to microseconds. It was derived from
the configured frame rate, which quantized capture-rate timestamps
and duplicated pts; RTP timestamps are now lossless.
- Transport: request a 4MB SO_RCVBUF on the receive socket
(best-effort; the kernel clamps to net.core.rmem_max).
meson test 4/4, valgrind clean (loopback + codec).
Wire the first end-to-end pipeline: capture -> encode -> packetize ->
UDP -> depacketize -> decode -> render.
- UdpRtpTransport: raw POSIX UDP sockets (IPv4 via getaddrinfo), a
receive jthread woken by socket close on stop; port 0 skips binding
so the sender uses an OS-assigned source port. ASIO stays deferred
to the signaling phase per ARCHITECTURE.md.
- SdlRenderer: SDL3 window/renderer with RGBA texture upload; the
texture is recreated on resolution change. RendererFactory now
returns RendererResult so SDL init failures carry a message,
mirroring the codec/capture error patterns.
- screencast binary: parse_cli plus SenderPipeline/ReceiverPipeline
per the app scaffolds; the sender creates its encoder once capture
reports real dimensions, the receiver keeps a bounded 3-frame queue
to hold latency down and renders on its own thread until the window
closes. cli argv signature fixed to 'const char* const*' so main's
argv converts implicitly.
- Encoder: drop AV_CODEC_FLAG_GLOBAL_HEADER so libx264 repeats SPS/PPS
in-band at each keyframe -- the receiver decodes from the bitstream
alone, which also makes mid-stream joins and later PLI recovery
work without out-of-band parameter negotiation. The round-trip test
now exercises exactly that path.
- tests: new udp-loopback integration test pushes synthetic frames
through a real localhost socket and decodes 10/10 frames with the
right dimensions; valgrind clean (loopback + codec). meson test 4/4.
Manual validation on the desktop (receiver window shows the captured
desktop) is documented in docs/RUNBOOK.md.
Add the sc_network library: RFC 3550 RtpHeader/RtpPacket serialize and
parse (the receiver tolerates CSRC lists, extension headers, and
padding by skipping/stripping them) and RFC 6184 H.264 payloading via
H264Packetizer/H264Depacketizer.
The packetizer splits Annex-B frames into NAL units (3- and 4-byte
start codes), emitting single-NAL packets or FU-A fragments within the
configured MTU, with the marker bit closing each frame and randomized
SSRC/sequence by default. The depacketizer reassembles access units
with 3-byte start codes, so both start-code widths round-trip
byte-exactly; frames damaged by sequence gaps or missing fragments
are dropped until the Phase 7 loss-recovery work.
test_rtp covers header and packet round-trips, malformed-input
rejections, splitter behavior, FU-A chunk bounds, full packetize ->
depacketize round-trip, gap dropping, marker-only frame separation,
sequence wrap, and empty inputs. meson test 3/3, valgrind clean.
Implement the xdg-desktop-portal ScreenCast backend via libportal: a
blocking portal handshake (interactive source picker), a PipeWire stream
on the portal's node enumerating BGRx/BGRA/RGBx/RGBA, and a latest-frame
slot handing frames to next_frame(). stop() is thread-safe; teardown
follows the order PipeWire requires. All proxy operations run under the
thread-loop lock to satisfy the protocol extension context checks
('impl_ext_end_proxy called from wrong context' otherwise).
The encoder now accepts padded strides for packed RGB inputs (real
PipeWire row pitches) and maps the new PixelFormat::Bgrx to
AV_PIX_FMT_BGRA.
Add tools/capture_smoke: a manual smoke tool (interactive, not in
meson test) that captures N frames, encodes them, and writes a
self-contained Annex-B elementary stream with prepended SPS/PPS.
Validated manually on Wayland/Hyprland: 2256x1504 H.264 elementary
stream, ffprobe clean. Phase 3 marked complete in docs/PHASES.md.
Allocate packet payloads with av_new_packet so they are freed through the
owning AVBufferRef instead of leaking on every decode. Pad extradata with
AV_INPUT_BUFFER_PADDING_SIZE for FFmpeg's bitstream parsers. Reject
oversized frames before the int cast, handle unexpected EOF in the
EAGAIN-retry loops, and add the missing <limits> include. Document the
valgrind codec check in docs/RUNBOOK.md.
Validated with valgrind: 0 bytes definitely lost, 0 invalid reads.
Add FFmpeg-based encoder/decoder with SPS/PPS extradata, Annex-B output
normalization, low-latency libx264 settings, and a round-trip unit test.
Includes review hardening: cached SwsContext, bitrate-only rate control,
std::byte/uin8_t cast helpers, and richer test assertions.
fegger