fegger
ce52f64e52
Implement the xdg-desktop-portal ScreenCast backend via libportal: a
blocking portal handshake (interactive source picker), a PipeWire stream
on the portal's node enumerating BGRx/BGRA/RGBx/RGBA, and a latest-frame
slot handing frames to next_frame(). stop() is thread-safe; teardown
follows the order PipeWire requires. All proxy operations run under the
thread-loop lock to satisfy the protocol extension context checks
('impl_ext_end_proxy called from wrong context' otherwise).
The encoder now accepts padded strides for packed RGB inputs (real
PipeWire row pitches) and maps the new PixelFormat::Bgrx to
AV_PIX_FMT_BGRA.
Add tools/capture_smoke: a manual smoke tool (interactive, not in
meson test) that captures N frames, encodes them, and writes a
self-contained Annex-B elementary stream with prepended SPS/PPS.
Validated manually on Wayland/Hyprland: 2256x1504 H.264 elementary
stream, ffprobe clean. Phase 3 marked complete in docs/PHASES.md.
3.9 KiB
3.9 KiB
Project Memory — screen_cast
Last updated: Phase 3 validated and complete; current phase is Phase 4.
Project state
- Phase 3 is done and validated on the desktop: a manual
./build/tools/capture_smoke 10 out.h264run on Wayland/Hyprland produced a valid 2256x1504 H.264 elementary stream (ffprobe clean).docs/PHASES.mdis ticked; current phase is Phase 4 — RTP framing. - The first smoke run emitted
impl_ext_end_proxy called from wrong contextwarnings:pw_context_connect_fdandpw_core_disconnectran outside the thread-loop lock. Fixed by holding the lock across all pw setup/teardown proxy operations. Re-running the smoke tool should now be warning-free (capture worked both ways; the warnings only meant the first two marshaled messages were rejected and retried from the right context). src/capture/pipewire_capture.cppnow implements the full backend: libportal 0.10 handshake (create_screencast_session→session_start→open_pipewire_remote), PipeWire 1.6 stream on the first portal node, BGRx/BGRA/RGBx/RGBA enumeration, latest-frame slot with condvar handoff. Portal handshake blocks on the caller thread; frames arrive on the pw thread.stop()is thread-safe; teardown follows the pw-required order.- Encoder now accepts padded strides for packed RGB formats and the new
PixelFormat::Bgrx(mapped toAV_PIX_FMT_BGRA); planar Yuv420p still requires a packed layout. This was the review note blocking Phase 3. tools/capture_smoke(manual, not inmeson test) captures N frames → encodes → writes Annex-B including prepended SPS/PPS extradata (verified: libx264 GLOBAL_HEADER extradata is Annex-B).- Earlier review fixes remain in place; valgrind on the codec round-trip test is still clean after the encoder stride changes.
- Encoder PTS caveat: the encoder time_base is derived from the configured frame rate (default 25fps), but portal frames arrive at monitor refresh (often 60Hz), so pts values quantize to 40ms units and can repeat. Harmless for the smoke test; revisit when RTP timestamps matter (Phase 4/5).
Decisions
- Language: C++20 with explicit modern-C++ guidelines in
AGENTS.mdandcpp-meson-build/SKILL.md. - Build system: Meson.
- Capture: PipeWire + xdg-desktop-portal.
- Encode/Decode: FFmpeg (libavcodec, libavutil, libswscale).
- H.264 encoder path: software
libx264, low-latency settings, Annex-B output. - Decoder receives SPS/PPS through
DecoderConfig.extradata. - Transport: RTP over UDP; signaling via WebSocket/JSON.
- Discovery: mDNS/Avahi.
- Rendering: SDL2 or SDL3 + OpenGL.
- Namespace:
sc. - Module error results use per-module
std::variant<T, XError>types (CodecResult,CaptureResult) since C++20 has nostd::expected.
Active blockers
None.
Open questions
- GUI framework (Qt6 vs. none / CLI only) — deferred to later phase.
- Hardware acceleration strategy (VAAPI / Vulkan Video / NVENC) — evaluate after software encode path works.
Forward-looking review notes (for later phases)
AV_CODEC_FLAG_GLOBAL_HEADERsuppresses in-band SPS/PPS; a receiver cannot join mid-stream or recover after PLI without parameter sets. Phase 5 must prepend SPS/PPS to keyframes or negotiate them in signaling.- Encoder sets no VBV (
maxrate/buffer_size) — ABR only; add for smoother UDP streaming in Phase 7. - DMA-BUF-only portal streams are rejected with a clear message (hardware path is Phase 7).
- No negative-path tests yet (bad config, bad stride, undersized buffer).
to_annex_b_h264sniffs AVCC vs Annex-B by content; if an AVCC-emitting encoder is ever added, prefer an explicit config flag over the heuristic.- Region targets are rejected: the desktop portal has no region capture.
next_frame()returnsnullopton stream error without surfacing the reason (logged to stderr); consider an error channel when the receiver pipeline lands.