fegger
ce52f64e52
Implement the xdg-desktop-portal ScreenCast backend via libportal: a
blocking portal handshake (interactive source picker), a PipeWire stream
on the portal's node enumerating BGRx/BGRA/RGBx/RGBA, and a latest-frame
slot handing frames to next_frame(). stop() is thread-safe; teardown
follows the order PipeWire requires. All proxy operations run under the
thread-loop lock to satisfy the protocol extension context checks
('impl_ext_end_proxy called from wrong context' otherwise).
The encoder now accepts padded strides for packed RGB inputs (real
PipeWire row pitches) and maps the new PixelFormat::Bgrx to
AV_PIX_FMT_BGRA.
Add tools/capture_smoke: a manual smoke tool (interactive, not in
meson test) that captures N frames, encodes them, and writes a
self-contained Annex-B elementary stream with prepended SPS/PPS.
Validated manually on Wayland/Hyprland: 2256x1504 H.264 elementary
stream, ffprobe clean. Phase 3 marked complete in docs/PHASES.md.
77 lines
3.9 KiB
Markdown
77 lines
3.9 KiB
Markdown
# Project Memory — screen_cast
|
|
|
|
Last updated: Phase 3 validated and complete; current phase is Phase 4.
|
|
|
|
## Project state
|
|
|
|
- **Phase 3 is done and validated on the desktop**: a manual
|
|
`./build/tools/capture_smoke 10 out.h264` run on Wayland/Hyprland produced
|
|
a valid 2256x1504 H.264 elementary stream (ffprobe clean). `docs/PHASES.md`
|
|
is ticked; current phase is Phase 4 — RTP framing.
|
|
- The first smoke run emitted `impl_ext_end_proxy called from wrong context`
|
|
warnings: `pw_context_connect_fd` and `pw_core_disconnect` ran outside the
|
|
thread-loop lock. Fixed by holding the lock across all pw setup/teardown
|
|
proxy operations. Re-running the smoke tool should now be warning-free
|
|
(capture worked both ways; the warnings only meant the first two
|
|
marshaled messages were rejected and retried from the right context).
|
|
- `src/capture/pipewire_capture.cpp` now implements the full backend:
|
|
libportal 0.10 handshake (`create_screencast_session` → `session_start` →
|
|
`open_pipewire_remote`), PipeWire 1.6 stream on the first portal node,
|
|
BGRx/BGRA/RGBx/RGBA enumeration, latest-frame slot with condvar handoff.
|
|
Portal handshake blocks on the caller thread; frames arrive on the pw
|
|
thread. `stop()` is thread-safe; teardown follows the pw-required order.
|
|
- Encoder now accepts padded strides for packed RGB formats and the new
|
|
`PixelFormat::Bgrx` (mapped to `AV_PIX_FMT_BGRA`); planar Yuv420p still
|
|
requires a packed layout. This was the review note blocking Phase 3.
|
|
- `tools/capture_smoke` (manual, not in `meson test`) captures N frames →
|
|
encodes → writes Annex-B including prepended SPS/PPS extradata (verified:
|
|
libx264 GLOBAL_HEADER extradata is Annex-B).
|
|
- Earlier review fixes remain in place; valgrind on the codec round-trip
|
|
test is still clean after the encoder stride changes.
|
|
- Encoder PTS caveat: the encoder time_base is derived from the configured
|
|
frame rate (default 25fps), but portal frames arrive at monitor refresh
|
|
(often 60Hz), so pts values quantize to 40ms units and can repeat. Harmless
|
|
for the smoke test; revisit when RTP timestamps matter (Phase 4/5).
|
|
|
|
## Decisions
|
|
|
|
- Language: C++20 with explicit modern-C++ guidelines in `AGENTS.md` and
|
|
`cpp-meson-build/SKILL.md`.
|
|
- Build system: Meson.
|
|
- Capture: PipeWire + xdg-desktop-portal.
|
|
- Encode/Decode: FFmpeg (libavcodec, libavutil, libswscale).
|
|
- H.264 encoder path: software `libx264`, low-latency settings, Annex-B output.
|
|
- Decoder receives SPS/PPS through `DecoderConfig.extradata`.
|
|
- Transport: RTP over UDP; signaling via WebSocket/JSON.
|
|
- Discovery: mDNS/Avahi.
|
|
- Rendering: SDL2 or SDL3 + OpenGL.
|
|
- Namespace: `sc`.
|
|
- Module error results use per-module `std::variant<T, XError>` types
|
|
(`CodecResult`, `CaptureResult`) since C++20 has no `std::expected`.
|
|
|
|
## Active blockers
|
|
|
|
None.
|
|
|
|
## Open questions
|
|
|
|
- GUI framework (Qt6 vs. none / CLI only) — deferred to later phase.
|
|
- Hardware acceleration strategy (VAAPI / Vulkan Video / NVENC) — evaluate after
|
|
software encode path works.
|
|
|
|
## Forward-looking review notes (for later phases)
|
|
|
|
- `AV_CODEC_FLAG_GLOBAL_HEADER` suppresses in-band SPS/PPS; a receiver cannot
|
|
join mid-stream or recover after PLI without parameter sets. Phase 5 must
|
|
prepend SPS/PPS to keyframes or negotiate them in signaling.
|
|
- Encoder sets no VBV (`maxrate`/`buffer_size`) — ABR only; add for smoother
|
|
UDP streaming in Phase 7.
|
|
- DMA-BUF-only portal streams are rejected with a clear message (hardware
|
|
path is Phase 7).
|
|
- No negative-path tests yet (bad config, bad stride, undersized buffer).
|
|
- `to_annex_b_h264` sniffs AVCC vs Annex-B by content; if an AVCC-emitting
|
|
encoder is ever added, prefer an explicit config flag over the heuristic.
|
|
- Region targets are rejected: the desktop portal has no region capture.
|
|
- `next_frame()` returns `nullopt` on stream error without surfacing the
|
|
reason (logged to stderr); consider an error channel when the receiver
|
|
pipeline lands. |