feat(capture): implement Phase 3 PipeWire/portal desktop capture

Implement the xdg-desktop-portal ScreenCast backend via libportal: a
blocking portal handshake (interactive source picker), a PipeWire stream
on the portal's node enumerating BGRx/BGRA/RGBx/RGBA, and a latest-frame
slot handing frames to next_frame(). stop() is thread-safe; teardown
follows the order PipeWire requires. All proxy operations run under the
thread-loop lock to satisfy the protocol extension context checks
('impl_ext_end_proxy called from wrong context' otherwise).

The encoder now accepts padded strides for packed RGB inputs (real
PipeWire row pitches) and maps the new PixelFormat::Bgrx to
AV_PIX_FMT_BGRA.

Add tools/capture_smoke: a manual smoke tool (interactive, not in
meson test) that captures N frames, encodes them, and writes a
self-contained Annex-B elementary stream with prepended SPS/PPS.

Validated manually on Wayland/Hyprland: 2256x1504 H.264 elementary
stream, ffprobe clean. Phase 3 marked complete in docs/PHASES.md.
This commit is contained in:
2026-09-07 10:37:14 +02:00
parent ac9dc02b51
commit ce52f64e52
11 changed files with 883 additions and 55 deletions
+36 -27
View File
@@ -1,31 +1,37 @@
# Project Memory — screen_cast
Last updated: Phase 2 codec review fixes applied.
Last updated: Phase 3 validated and complete; current phase is Phase 4.
## Project state
- Phase 2 codec implementation reviewed with valgrind; all confirmed defects
fixed on top of the capture-stub commit:
- decoder leaked every packet payload (`av_malloc` + direct `packet->data`
assignment bypassed the packet's owning `AVBufferRef`); payloads are now
allocated with `av_new_packet`.
- decoder extradata lacked `AV_INPUT_BUFFER_PADDING_SIZE`; FFmpeg's
extradata parser over-read the buffer (valgrind invalid reads). Now
allocated padded and zeroed.
- oversized encoded frames are rejected before the int cast.
- EAGAIN-retry loops in encoder/decoder now handle unexpected EOF instead
of retrying forever.
- fixed "RTP packet" → "AVPacket" error message in the encoder.
- `CaptureFactory::create()` now returns
`CaptureResult<std::unique_ptr<CaptureSession>>` (variant with
`CaptureError`) instead of a nullable unique_ptr; the stub reports
"PipeWire capture is not implemented yet" as an error. New pattern lives in
`include/screencast/capture/error.h`, mirroring `codec/error.h`.
- Validation: `meson test` 2/2 OK; valgrind on `test_codec_roundtrip` is now
clean (0 definite losses, 0 invalid reads); clang-format clean. See
`docs/RUNBOOK.md` for the reusable checks.
- Phase 3 capture stub remains in place; remaining implementation: capture,
transport, rendering, and CLI/pipeline glue.
- **Phase 3 is done and validated on the desktop**: a manual
`./build/tools/capture_smoke 10 out.h264` run on Wayland/Hyprland produced
a valid 2256x1504 H.264 elementary stream (ffprobe clean). `docs/PHASES.md`
is ticked; current phase is Phase 4 — RTP framing.
- The first smoke run emitted `impl_ext_end_proxy called from wrong context`
warnings: `pw_context_connect_fd` and `pw_core_disconnect` ran outside the
thread-loop lock. Fixed by holding the lock across all pw setup/teardown
proxy operations. Re-running the smoke tool should now be warning-free
(capture worked both ways; the warnings only meant the first two
marshaled messages were rejected and retried from the right context).
- `src/capture/pipewire_capture.cpp` now implements the full backend:
libportal 0.10 handshake (`create_screencast_session``session_start`
`open_pipewire_remote`), PipeWire 1.6 stream on the first portal node,
BGRx/BGRA/RGBx/RGBA enumeration, latest-frame slot with condvar handoff.
Portal handshake blocks on the caller thread; frames arrive on the pw
thread. `stop()` is thread-safe; teardown follows the pw-required order.
- Encoder now accepts padded strides for packed RGB formats and the new
`PixelFormat::Bgrx` (mapped to `AV_PIX_FMT_BGRA`); planar Yuv420p still
requires a packed layout. This was the review note blocking Phase 3.
- `tools/capture_smoke` (manual, not in `meson test`) captures N frames →
encodes → writes Annex-B including prepended SPS/PPS extradata (verified:
libx264 GLOBAL_HEADER extradata is Annex-B).
- Earlier review fixes remain in place; valgrind on the codec round-trip
test is still clean after the encoder stride changes.
- Encoder PTS caveat: the encoder time_base is derived from the configured
frame rate (default 25fps), but portal frames arrive at monitor refresh
(often 60Hz), so pts values quantize to 40ms units and can repeat. Harmless
for the smoke test; revisit when RTP timestamps matter (Phase 4/5).
## Decisions
@@ -55,14 +61,17 @@ None.
## Forward-looking review notes (for later phases)
- Encoder rejects non-packed strides in `make_input_frame`; PipeWire/portal
frames usually have alignment-padded strides — Phase 3 must pass the real
stride through to swscale instead of rejecting it.
- `AV_CODEC_FLAG_GLOBAL_HEADER` suppresses in-band SPS/PPS; a receiver cannot
join mid-stream or recover after PLI without parameter sets. Phase 5 must
prepend SPS/PPS to keyframes or negotiate them in signaling.
- Encoder sets no VBV (`maxrate`/`buffer_size`) — ABR only; add for smoother
UDP streaming in Phase 7.
- DMA-BUF-only portal streams are rejected with a clear message (hardware
path is Phase 7).
- No negative-path tests yet (bad config, bad stride, undersized buffer).
- `to_annex_b_h264` sniffs AVCC vs Annex-B by content; if an AVCC-emitting
encoder is ever added, prefer an explicit config flag over the heuristic.
encoder is ever added, prefer an explicit config flag over the heuristic.
- Region targets are rejected: the desktop portal has no region capture.
- `next_frame()` returns `nullopt` on stream error without surfacing the
reason (logged to stderr); consider an error channel when the receiver
pipeline lands.