feat(capture): implement Phase 3 PipeWire/portal desktop capture
Implement the xdg-desktop-portal ScreenCast backend via libportal: a
blocking portal handshake (interactive source picker), a PipeWire stream
on the portal's node enumerating BGRx/BGRA/RGBx/RGBA, and a latest-frame
slot handing frames to next_frame(). stop() is thread-safe; teardown
follows the order PipeWire requires. All proxy operations run under the
thread-loop lock to satisfy the protocol extension context checks
('impl_ext_end_proxy called from wrong context' otherwise).
The encoder now accepts padded strides for packed RGB inputs (real
PipeWire row pitches) and maps the new PixelFormat::Bgrx to
AV_PIX_FMT_BGRA.
Add tools/capture_smoke: a manual smoke tool (interactive, not in
meson test) that captures N frames, encodes them, and writes a
self-contained Annex-B elementary stream with prepended SPS/PPS.
Validated manually on Wayland/Hyprland: 2256x1504 H.264 elementary
stream, ffprobe clean. Phase 3 marked complete in docs/PHASES.md.
This commit is contained in:
+36
-27
@@ -1,31 +1,37 @@
|
|||||||
# Project Memory — screen_cast
|
# Project Memory — screen_cast
|
||||||
|
|
||||||
Last updated: Phase 2 codec review fixes applied.
|
Last updated: Phase 3 validated and complete; current phase is Phase 4.
|
||||||
|
|
||||||
## Project state
|
## Project state
|
||||||
|
|
||||||
- Phase 2 codec implementation reviewed with valgrind; all confirmed defects
|
- **Phase 3 is done and validated on the desktop**: a manual
|
||||||
fixed on top of the capture-stub commit:
|
`./build/tools/capture_smoke 10 out.h264` run on Wayland/Hyprland produced
|
||||||
- decoder leaked every packet payload (`av_malloc` + direct `packet->data`
|
a valid 2256x1504 H.264 elementary stream (ffprobe clean). `docs/PHASES.md`
|
||||||
assignment bypassed the packet's owning `AVBufferRef`); payloads are now
|
is ticked; current phase is Phase 4 — RTP framing.
|
||||||
allocated with `av_new_packet`.
|
- The first smoke run emitted `impl_ext_end_proxy called from wrong context`
|
||||||
- decoder extradata lacked `AV_INPUT_BUFFER_PADDING_SIZE`; FFmpeg's
|
warnings: `pw_context_connect_fd` and `pw_core_disconnect` ran outside the
|
||||||
extradata parser over-read the buffer (valgrind invalid reads). Now
|
thread-loop lock. Fixed by holding the lock across all pw setup/teardown
|
||||||
allocated padded and zeroed.
|
proxy operations. Re-running the smoke tool should now be warning-free
|
||||||
- oversized encoded frames are rejected before the int cast.
|
(capture worked both ways; the warnings only meant the first two
|
||||||
- EAGAIN-retry loops in encoder/decoder now handle unexpected EOF instead
|
marshaled messages were rejected and retried from the right context).
|
||||||
of retrying forever.
|
- `src/capture/pipewire_capture.cpp` now implements the full backend:
|
||||||
- fixed "RTP packet" → "AVPacket" error message in the encoder.
|
libportal 0.10 handshake (`create_screencast_session` → `session_start` →
|
||||||
- `CaptureFactory::create()` now returns
|
`open_pipewire_remote`), PipeWire 1.6 stream on the first portal node,
|
||||||
`CaptureResult<std::unique_ptr<CaptureSession>>` (variant with
|
BGRx/BGRA/RGBx/RGBA enumeration, latest-frame slot with condvar handoff.
|
||||||
`CaptureError`) instead of a nullable unique_ptr; the stub reports
|
Portal handshake blocks on the caller thread; frames arrive on the pw
|
||||||
"PipeWire capture is not implemented yet" as an error. New pattern lives in
|
thread. `stop()` is thread-safe; teardown follows the pw-required order.
|
||||||
`include/screencast/capture/error.h`, mirroring `codec/error.h`.
|
- Encoder now accepts padded strides for packed RGB formats and the new
|
||||||
- Validation: `meson test` 2/2 OK; valgrind on `test_codec_roundtrip` is now
|
`PixelFormat::Bgrx` (mapped to `AV_PIX_FMT_BGRA`); planar Yuv420p still
|
||||||
clean (0 definite losses, 0 invalid reads); clang-format clean. See
|
requires a packed layout. This was the review note blocking Phase 3.
|
||||||
`docs/RUNBOOK.md` for the reusable checks.
|
- `tools/capture_smoke` (manual, not in `meson test`) captures N frames →
|
||||||
- Phase 3 capture stub remains in place; remaining implementation: capture,
|
encodes → writes Annex-B including prepended SPS/PPS extradata (verified:
|
||||||
transport, rendering, and CLI/pipeline glue.
|
libx264 GLOBAL_HEADER extradata is Annex-B).
|
||||||
|
- Earlier review fixes remain in place; valgrind on the codec round-trip
|
||||||
|
test is still clean after the encoder stride changes.
|
||||||
|
- Encoder PTS caveat: the encoder time_base is derived from the configured
|
||||||
|
frame rate (default 25fps), but portal frames arrive at monitor refresh
|
||||||
|
(often 60Hz), so pts values quantize to 40ms units and can repeat. Harmless
|
||||||
|
for the smoke test; revisit when RTP timestamps matter (Phase 4/5).
|
||||||
|
|
||||||
## Decisions
|
## Decisions
|
||||||
|
|
||||||
@@ -55,14 +61,17 @@ None.
|
|||||||
|
|
||||||
## Forward-looking review notes (for later phases)
|
## Forward-looking review notes (for later phases)
|
||||||
|
|
||||||
- Encoder rejects non-packed strides in `make_input_frame`; PipeWire/portal
|
|
||||||
frames usually have alignment-padded strides — Phase 3 must pass the real
|
|
||||||
stride through to swscale instead of rejecting it.
|
|
||||||
- `AV_CODEC_FLAG_GLOBAL_HEADER` suppresses in-band SPS/PPS; a receiver cannot
|
- `AV_CODEC_FLAG_GLOBAL_HEADER` suppresses in-band SPS/PPS; a receiver cannot
|
||||||
join mid-stream or recover after PLI without parameter sets. Phase 5 must
|
join mid-stream or recover after PLI without parameter sets. Phase 5 must
|
||||||
prepend SPS/PPS to keyframes or negotiate them in signaling.
|
prepend SPS/PPS to keyframes or negotiate them in signaling.
|
||||||
- Encoder sets no VBV (`maxrate`/`buffer_size`) — ABR only; add for smoother
|
- Encoder sets no VBV (`maxrate`/`buffer_size`) — ABR only; add for smoother
|
||||||
UDP streaming in Phase 7.
|
UDP streaming in Phase 7.
|
||||||
|
- DMA-BUF-only portal streams are rejected with a clear message (hardware
|
||||||
|
path is Phase 7).
|
||||||
- No negative-path tests yet (bad config, bad stride, undersized buffer).
|
- No negative-path tests yet (bad config, bad stride, undersized buffer).
|
||||||
- `to_annex_b_h264` sniffs AVCC vs Annex-B by content; if an AVCC-emitting
|
- `to_annex_b_h264` sniffs AVCC vs Annex-B by content; if an AVCC-emitting
|
||||||
encoder is ever added, prefer an explicit config flag over the heuristic.
|
encoder is ever added, prefer an explicit config flag over the heuristic.
|
||||||
|
- Region targets are rejected: the desktop portal has no region capture.
|
||||||
|
- `next_frame()` returns `nullopt` on stream error without surfacing the
|
||||||
|
reason (logged to stderr); consider an error channel when the receiver
|
||||||
|
pipeline lands.
|
||||||
@@ -16,6 +16,9 @@ meson-private/
|
|||||||
*.exe
|
*.exe
|
||||||
compile_commands.json
|
compile_commands.json
|
||||||
|
|
||||||
|
# Smoke test output
|
||||||
|
*.h264
|
||||||
|
|
||||||
# IDE
|
# IDE
|
||||||
.vscode/
|
.vscode/
|
||||||
.idea/
|
.idea/
|
||||||
|
|||||||
+4
-4
@@ -32,9 +32,9 @@ frames.
|
|||||||
|
|
||||||
**Goal**: capture the desktop and feed frames into the encoder.
|
**Goal**: capture the desktop and feed frames into the encoder.
|
||||||
|
|
||||||
- Add `libpipewire-0.3` dependency.
|
- [x] Add `libpipewire-0.3` dependency.
|
||||||
- Implement `CaptureFactory::create()` using the xdg-desktop-portal.
|
- [x] Implement `CaptureFactory::create()` using the xdg-desktop-portal.
|
||||||
- Wire `CaptureSession::next_frame()` → `Encoder::encode()` in a local smoke
|
- [x] Wire `CaptureSession::next_frame()` → `Encoder::encode()` in a local smoke
|
||||||
test that just writes a few encoded frames to disk.
|
test that just writes a few encoded frames to disk.
|
||||||
|
|
||||||
**Validation**: manual run on a real Linux desktop session produces a valid
|
**Validation**: manual run on a real Linux desktop session produces a valid
|
||||||
@@ -89,4 +89,4 @@ where available.
|
|||||||
|
|
||||||
## Current phase
|
## Current phase
|
||||||
|
|
||||||
Phase 3 — PipeWire screen capture.
|
Phase 4 — RTP Framing.
|
||||||
|
|||||||
@@ -27,6 +27,28 @@ FFmpeg may keep some "still reachable" allocations at exit; that is normal.
|
|||||||
This check caught two real defects in the Phase 2 decoder: a per-packet
|
This check caught two real defects in the Phase 2 decoder: a per-packet
|
||||||
payload leak and an unpadded extradata buffer over-read. Keep using it.
|
payload leak and an unpadded extradata buffer over-read. Keep using it.
|
||||||
|
|
||||||
|
## Capture smoke test (Phase 3, manual)
|
||||||
|
|
||||||
|
Requires a running desktop session with `xdg-desktop-portal` and a backend
|
||||||
|
that implements the ScreenCast portal (e.g. Hyprland, GNOME, KDE). The
|
||||||
|
portal shows an interactive source picker, so this cannot run unattended.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
meson compile -C build # builds tools/capture_smoke
|
||||||
|
./build/tools/capture_smoke 10 out.h264 # captures 10 frames
|
||||||
|
ffprobe -v error -show_entries stream=codec_name,width,height out.h264
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: the tool prints the negotiated resolution/format/stride and
|
||||||
|
writes a non-empty file; `ffprobe` reports `codec_name=h264` and the correct
|
||||||
|
width/height. The encoder prepends its Annex-B SPS/PPS extradata so the
|
||||||
|
file is a self-contained elementary stream.
|
||||||
|
|
||||||
|
If you see `impl_ext_end_proxy called from wrong context` warnings, a
|
||||||
|
PipeWire proxy operation ran without the thread-loop lock — all pw
|
||||||
|
calls that send messages (connect, stream, core) must happen under
|
||||||
|
`pw_thread_loop_lock`.
|
||||||
|
|
||||||
## Formatting
|
## Formatting
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ namespace sc {
|
|||||||
|
|
||||||
enum class PixelFormat {
|
enum class PixelFormat {
|
||||||
Rgba,
|
Rgba,
|
||||||
|
Bgrx, // 4 bytes/pixel, memory order B, G, R, unused
|
||||||
Yuv420p,
|
Yuv420p,
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -10,6 +10,9 @@ project('screen_cast', 'cpp',
|
|||||||
# Public and private include directories are declared in `src/meson.build`.
|
# Public and private include directories are declared in `src/meson.build`.
|
||||||
subdir('src')
|
subdir('src')
|
||||||
|
|
||||||
|
# Manual smoke tools
|
||||||
|
subdir('tools')
|
||||||
|
|
||||||
# Tests
|
# Tests
|
||||||
enable_tests = get_option('tests')
|
enable_tests = get_option('tests')
|
||||||
if enable_tests
|
if enable_tests
|
||||||
|
|||||||
+12
-6
@@ -1,11 +1,17 @@
|
|||||||
# Phase 3 capture placeholder. libpipewire-0.3 will be added when the portal
|
# Phase 3 capture backend: PipeWire via the xdg-desktop-portal ScreenCast
|
||||||
# implementation is wired in.
|
# interface, with libportal handling the portal D-Bus handshake.
|
||||||
|
|
||||||
|
dep_pipewire = dependency('libpipewire-0.3')
|
||||||
|
dep_portal = dependency('libportal')
|
||||||
|
|
||||||
sc_capture_sources = files('pipewire_capture.cpp')
|
sc_capture_sources = files('pipewire_capture.cpp')
|
||||||
|
|
||||||
# Built even though nothing links it yet, so the stub stays compile-clean.
|
sc_capture = static_library('sc_capture',
|
||||||
static_library('sc_capture',
|
|
||||||
sc_capture_sources,
|
sc_capture_sources,
|
||||||
include_directories : sc_core_inc)
|
include_directories : sc_core_inc,
|
||||||
|
dependencies : [dep_pipewire, dep_portal])
|
||||||
|
|
||||||
# sc_capture_dep will be declared once the app/pipeline code links against it.
|
sc_capture_dep = declare_dependency(
|
||||||
|
link_with : sc_capture,
|
||||||
|
include_directories : sc_core_inc,
|
||||||
|
dependencies : [dep_pipewire, dep_portal])
|
||||||
@@ -1,13 +1,582 @@
|
|||||||
#include "screencast/capture/capture.h"
|
#include "screencast/capture/capture.h"
|
||||||
|
|
||||||
|
#include <glib.h>
|
||||||
|
#include <libportal/portal-helpers.h>
|
||||||
|
#include <libportal/portal.h>
|
||||||
|
#include <libportal/remote.h>
|
||||||
|
#include <libportal/session.h>
|
||||||
|
|
||||||
|
#include <pipewire/pipewire.h>
|
||||||
|
#include <pipewire/thread-loop.h>
|
||||||
|
#include <spa/param/param.h>
|
||||||
|
#include <spa/param/video/format-utils.h>
|
||||||
|
#include <spa/pod/builder.h>
|
||||||
|
#include <spa/utils/result.h>
|
||||||
|
|
||||||
|
#include <unistd.h>
|
||||||
|
|
||||||
|
#include <array>
|
||||||
|
#include <atomic>
|
||||||
|
#include <chrono>
|
||||||
|
#include <condition_variable>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstring>
|
||||||
|
#include <format>
|
||||||
|
#include <iostream>
|
||||||
#include <memory>
|
#include <memory>
|
||||||
|
#include <mutex>
|
||||||
|
#include <optional>
|
||||||
|
#include <string>
|
||||||
|
#include <utility>
|
||||||
|
|
||||||
namespace sc {
|
namespace sc {
|
||||||
|
namespace {
|
||||||
|
|
||||||
CaptureResult<std::unique_ptr<CaptureSession>> CaptureFactory::create(CaptureTarget /*target*/) {
|
struct GObjectDeleter {
|
||||||
// Phase 3 placeholder. The PipeWire / xdg-desktop-portal capture backend
|
void operator()(gpointer object) const noexcept {
|
||||||
// will be implemented here once the portal integration lands.
|
if (object != nullptr) {
|
||||||
return CaptureError{"PipeWire capture is not implemented yet"};
|
g_object_unref(object);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
using PortalPtr = std::unique_ptr<XdpPortal, GObjectDeleter>;
|
||||||
|
using XdpSessionPtr = std::unique_ptr<XdpSession, GObjectDeleter>;
|
||||||
|
|
||||||
|
class ScopedMainContext {
|
||||||
|
public:
|
||||||
|
ScopedMainContext() : context_(g_main_context_new()) {
|
||||||
|
if (context_ != nullptr) {
|
||||||
|
g_main_context_push_thread_default(context_);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
~ScopedMainContext() {
|
||||||
|
if (context_ != nullptr) {
|
||||||
|
g_main_context_pop_thread_default(context_);
|
||||||
|
g_main_context_unref(context_);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
ScopedMainContext(const ScopedMainContext&) = delete;
|
||||||
|
ScopedMainContext& operator=(const ScopedMainContext&) = delete;
|
||||||
|
|
||||||
|
GMainContext* get() const noexcept {
|
||||||
|
return context_;
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
GMainContext* context_;
|
||||||
|
};
|
||||||
|
|
||||||
|
class ScopedMainLoop {
|
||||||
|
public:
|
||||||
|
explicit ScopedMainLoop(GMainContext* context) : loop_(g_main_loop_new(context, FALSE)) {}
|
||||||
|
|
||||||
|
~ScopedMainLoop() {
|
||||||
|
if (loop_ != nullptr) {
|
||||||
|
g_main_loop_unref(loop_);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
ScopedMainLoop(const ScopedMainLoop&) = delete;
|
||||||
|
ScopedMainLoop& operator=(const ScopedMainLoop&) = delete;
|
||||||
|
|
||||||
|
void run() {
|
||||||
|
g_main_loop_run(loop_);
|
||||||
|
}
|
||||||
|
|
||||||
|
void quit() {
|
||||||
|
g_main_loop_quit(loop_);
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
GMainLoop* loop_;
|
||||||
|
};
|
||||||
|
|
||||||
|
// Owns a raw fd and closes it unless release()d.
|
||||||
|
class OwnedFd {
|
||||||
|
public:
|
||||||
|
explicit OwnedFd(int fd) noexcept : fd_(fd) {}
|
||||||
|
|
||||||
|
~OwnedFd() {
|
||||||
|
if (fd_ >= 0) {
|
||||||
|
close(fd_);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
OwnedFd(const OwnedFd&) = delete;
|
||||||
|
OwnedFd& operator=(const OwnedFd&) = delete;
|
||||||
|
|
||||||
|
OwnedFd(OwnedFd&& other) noexcept : fd_(other.fd_) {
|
||||||
|
other.fd_ = -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
OwnedFd& operator=(OwnedFd&& other) noexcept {
|
||||||
|
if (this != &other) {
|
||||||
|
if (fd_ >= 0) {
|
||||||
|
close(fd_);
|
||||||
|
}
|
||||||
|
fd_ = other.fd_;
|
||||||
|
other.fd_ = -1;
|
||||||
|
}
|
||||||
|
return *this;
|
||||||
|
}
|
||||||
|
|
||||||
|
int get() const noexcept {
|
||||||
|
return fd_;
|
||||||
|
}
|
||||||
|
|
||||||
|
int release() noexcept {
|
||||||
|
const int fd = fd_;
|
||||||
|
fd_ = -1;
|
||||||
|
return fd;
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
int fd_;
|
||||||
|
};
|
||||||
|
|
||||||
|
std::string gerror_message(const GError* error) {
|
||||||
|
if (error == nullptr || error->message == nullptr) {
|
||||||
|
return "unknown error";
|
||||||
|
}
|
||||||
|
return std::string{error->message};
|
||||||
|
}
|
||||||
|
|
||||||
|
uint64_t monotonic_now_ns() {
|
||||||
|
const auto now = std::chrono::steady_clock::now().time_since_epoch();
|
||||||
|
return static_cast<uint64_t>(std::chrono::duration_cast<std::chrono::nanoseconds>(now).count());
|
||||||
|
}
|
||||||
|
|
||||||
|
std::optional<PixelFormat> pixel_format_from_spa(uint32_t spa_format) noexcept {
|
||||||
|
switch (spa_format) {
|
||||||
|
case SPA_VIDEO_FORMAT_BGRx:
|
||||||
|
case SPA_VIDEO_FORMAT_BGRA:
|
||||||
|
return PixelFormat::Bgrx;
|
||||||
|
case SPA_VIDEO_FORMAT_RGBx:
|
||||||
|
case SPA_VIDEO_FORMAT_RGBA:
|
||||||
|
return PixelFormat::Rgba;
|
||||||
|
default:
|
||||||
|
return std::nullopt;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The portal Start response exposes one entry per stream as (u a{sv}); the
|
||||||
|
// first child of the first stream holds the PipeWire node id.
|
||||||
|
std::optional<uint32_t> first_stream_node_id(XdpSession* session) {
|
||||||
|
GVariant* streams = xdp_session_get_streams(session); // transfer full
|
||||||
|
if (streams == nullptr) {
|
||||||
|
return std::nullopt;
|
||||||
|
}
|
||||||
|
|
||||||
|
std::optional<uint32_t> node_id;
|
||||||
|
if (g_variant_n_children(streams) > 0) {
|
||||||
|
GVariant* stream = g_variant_get_child_value(streams, 0);
|
||||||
|
if (stream != nullptr) {
|
||||||
|
if (g_variant_is_of_type(stream, G_VARIANT_TYPE("(ua{sv})"))) {
|
||||||
|
guint32 id = 0;
|
||||||
|
GVariant* properties = nullptr;
|
||||||
|
g_variant_get(stream, "(u@a{sv})", &id, &properties);
|
||||||
|
if (properties != nullptr) {
|
||||||
|
g_variant_unref(properties);
|
||||||
|
}
|
||||||
|
node_id = static_cast<uint32_t>(id);
|
||||||
|
}
|
||||||
|
g_variant_unref(stream);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
g_variant_unref(streams);
|
||||||
|
return node_id;
|
||||||
|
}
|
||||||
|
|
||||||
|
struct SessionCreatedOp {
|
||||||
|
ScopedMainLoop* loop = nullptr;
|
||||||
|
XdpSession* session = nullptr; // owned, transferred by the caller
|
||||||
|
GError* error = nullptr; // owned, cleared by the caller
|
||||||
|
};
|
||||||
|
|
||||||
|
void on_screencast_session_created(GObject* source, GAsyncResult* result, gpointer user_data) {
|
||||||
|
auto* op = static_cast<SessionCreatedOp*>(user_data);
|
||||||
|
op->session = xdp_portal_create_screencast_session_finish(XDP_PORTAL(source), result, &op->error);
|
||||||
|
op->loop->quit();
|
||||||
|
}
|
||||||
|
|
||||||
|
struct SessionStartedOp {
|
||||||
|
ScopedMainLoop* loop = nullptr;
|
||||||
|
bool started = false;
|
||||||
|
GError* error = nullptr; // owned, cleared by the caller
|
||||||
|
};
|
||||||
|
|
||||||
|
void on_screencast_session_started(GObject* source, GAsyncResult* result, gpointer user_data) {
|
||||||
|
auto* op = static_cast<SessionStartedOp*>(user_data);
|
||||||
|
op->started = xdp_session_start_finish(XDP_SESSION(source), result, &op->error) == TRUE;
|
||||||
|
op->loop->quit();
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
// PipeWire screencast session driven by the xdg-desktop-portal ScreenCast
|
||||||
|
// interface (via libportal). The portal handshake runs synchronously on the
|
||||||
|
// thread that calls CaptureFactory::create(); frames are produced on a
|
||||||
|
// PipeWire thread and handed to next_frame() through a latest-frame slot.
|
||||||
|
class PipeWireCaptureSession final : public CaptureSession {
|
||||||
|
public:
|
||||||
|
PipeWireCaptureSession(uint32_t node_id, XdpSessionPtr portal_session, PortalPtr portal)
|
||||||
|
: node_id_(node_id), portal_session_(std::move(portal_session)), portal_(std::move(portal)) {
|
||||||
|
stream_events_.version = PW_VERSION_STREAM_EVENTS;
|
||||||
|
stream_events_.state_changed = &PipeWireCaptureSession::on_state_changed;
|
||||||
|
stream_events_.param_changed = &PipeWireCaptureSession::on_param_changed;
|
||||||
|
stream_events_.process = &PipeWireCaptureSession::on_process;
|
||||||
|
}
|
||||||
|
|
||||||
|
~PipeWireCaptureSession() override {
|
||||||
|
stop();
|
||||||
|
teardown_pipewire();
|
||||||
|
}
|
||||||
|
|
||||||
|
PipeWireCaptureSession(const PipeWireCaptureSession&) = delete;
|
||||||
|
PipeWireCaptureSession& operator=(const PipeWireCaptureSession&) = delete;
|
||||||
|
|
||||||
|
std::optional<CapturedFrame> next_frame() override {
|
||||||
|
std::unique_lock lock(frame_mutex_);
|
||||||
|
frame_cv_.wait(lock, [this] { return stopped_ || has_new_frame_; });
|
||||||
|
if (!has_new_frame_) {
|
||||||
|
return std::nullopt; // stopped (or the stream ended with an error)
|
||||||
|
}
|
||||||
|
has_new_frame_ = false;
|
||||||
|
return std::move(latest_frame_);
|
||||||
|
}
|
||||||
|
|
||||||
|
void stop() override {
|
||||||
|
{
|
||||||
|
std::lock_guard lock(frame_mutex_);
|
||||||
|
stopped_ = true;
|
||||||
|
}
|
||||||
|
frame_cv_.notify_all();
|
||||||
|
|
||||||
|
pw_thread_loop* loop = loop_;
|
||||||
|
if (loop != nullptr) {
|
||||||
|
pw_thread_loop_lock(loop);
|
||||||
|
if (stream_ != nullptr) {
|
||||||
|
pw_stream_disconnect(stream_);
|
||||||
|
}
|
||||||
|
pw_thread_loop_unlock(loop);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Connects the capture stream to the portal's PipeWire node. On success
|
||||||
|
// the fd has been handed to PipeWire; on failure it is closed here.
|
||||||
|
//
|
||||||
|
// Every PipeWire operation that marshals a proxy message must run under
|
||||||
|
// the thread-loop lock; the protocol extension rejects calls from other
|
||||||
|
// contexts ("impl_ext_end_proxy called from wrong context").
|
||||||
|
std::optional<std::string> connect_to_pipewire(OwnedFd fd) {
|
||||||
|
loop_ = pw_thread_loop_new("screencast-capture", nullptr);
|
||||||
|
if (loop_ == nullptr) {
|
||||||
|
return std::string{"failed to create the PipeWire thread loop"};
|
||||||
|
}
|
||||||
|
if (pw_thread_loop_start(loop_) < 0) {
|
||||||
|
return std::string{"failed to start the PipeWire thread loop"};
|
||||||
|
}
|
||||||
|
|
||||||
|
pw_thread_loop_lock(loop_);
|
||||||
|
|
||||||
|
context_ = pw_context_new(pw_thread_loop_get_loop(loop_), nullptr, 0);
|
||||||
|
if (context_ == nullptr) {
|
||||||
|
pw_thread_loop_unlock(loop_);
|
||||||
|
return std::string{"failed to create the PipeWire context"};
|
||||||
|
}
|
||||||
|
|
||||||
|
// pw_context_connect_fd takes ownership of the socket (see core.h).
|
||||||
|
core_ = pw_context_connect_fd(context_, fd.release(), nullptr, 0);
|
||||||
|
if (core_ == nullptr) {
|
||||||
|
pw_thread_loop_unlock(loop_);
|
||||||
|
return std::string{"failed to connect to the PipeWire remote"};
|
||||||
|
}
|
||||||
|
|
||||||
|
pw_properties* props = pw_properties_new(PW_KEY_MEDIA_TYPE, "Video", PW_KEY_MEDIA_CATEGORY, "Capture", nullptr);
|
||||||
|
stream_ = pw_stream_new(core_, "screencast", props);
|
||||||
|
if (stream_ == nullptr) {
|
||||||
|
pw_thread_loop_unlock(loop_);
|
||||||
|
return std::string{"failed to create the capture stream"};
|
||||||
|
}
|
||||||
|
pw_stream_add_listener(stream_, &stream_listener_, &stream_events_, this);
|
||||||
|
|
||||||
|
// Enumerate the formats the portal stream can provide. Properties that
|
||||||
|
// are omitted (size, framerate) mean "any".
|
||||||
|
std::array<uint8_t, 1024> pod_buffer{};
|
||||||
|
struct spa_pod_builder pod_builder = SPA_POD_BUILDER_INIT(pod_buffer.data(), pod_buffer.size());
|
||||||
|
const struct spa_pod* connect_params[1] = {nullptr};
|
||||||
|
{
|
||||||
|
struct spa_pod_frame outer[1];
|
||||||
|
struct spa_pod_frame choice[1];
|
||||||
|
|
||||||
|
spa_pod_builder_push_object(&pod_builder, &outer[0], SPA_TYPE_OBJECT_Format, SPA_PARAM_EnumFormat);
|
||||||
|
spa_pod_builder_add(&pod_builder,
|
||||||
|
SPA_FORMAT_mediaType,
|
||||||
|
SPA_POD_Id(SPA_MEDIA_TYPE_video),
|
||||||
|
SPA_FORMAT_mediaSubtype,
|
||||||
|
SPA_POD_Id(SPA_MEDIA_SUBTYPE_raw),
|
||||||
|
0);
|
||||||
|
spa_pod_builder_prop(&pod_builder, SPA_FORMAT_VIDEO_format, 0);
|
||||||
|
spa_pod_builder_push_choice(&pod_builder, &choice[0], SPA_CHOICE_Enum, 0);
|
||||||
|
spa_pod_builder_id(&pod_builder, SPA_VIDEO_FORMAT_BGRx); // default
|
||||||
|
spa_pod_builder_id(&pod_builder, SPA_VIDEO_FORMAT_BGRA);
|
||||||
|
spa_pod_builder_id(&pod_builder, SPA_VIDEO_FORMAT_RGBx);
|
||||||
|
spa_pod_builder_id(&pod_builder, SPA_VIDEO_FORMAT_RGBA);
|
||||||
|
spa_pod_builder_pop(&pod_builder, &choice[0]);
|
||||||
|
connect_params[0] = static_cast<const struct spa_pod*>(spa_pod_builder_pop(&pod_builder, &outer[0]));
|
||||||
|
}
|
||||||
|
|
||||||
|
const int connect_result = pw_stream_connect(stream_,
|
||||||
|
PW_DIRECTION_INPUT,
|
||||||
|
node_id_,
|
||||||
|
static_cast<enum pw_stream_flags>(PW_STREAM_FLAG_AUTOCONNECT |
|
||||||
|
PW_STREAM_FLAG_MAP_BUFFERS |
|
||||||
|
PW_STREAM_FLAG_DONT_RECONNECT),
|
||||||
|
connect_params,
|
||||||
|
1);
|
||||||
|
|
||||||
|
pw_thread_loop_unlock(loop_);
|
||||||
|
|
||||||
|
if (connect_result < 0) {
|
||||||
|
return std::string{"failed to connect the capture stream: "} + spa_strerror(connect_result);
|
||||||
|
}
|
||||||
|
return std::nullopt;
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
static void on_state_changed(void* data, enum pw_stream_state, enum pw_stream_state state, const char* error) {
|
||||||
|
if (state != PW_STREAM_STATE_ERROR) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
auto* self = static_cast<PipeWireCaptureSession*>(data);
|
||||||
|
std::cerr << std::format("screencast: capture stream error: {}\n", error != nullptr ? error : "unknown error");
|
||||||
|
self->mark_stopped();
|
||||||
|
}
|
||||||
|
|
||||||
|
static void on_param_changed(void* data, uint32_t id, const struct spa_pod* param) {
|
||||||
|
if (param == nullptr || id != SPA_PARAM_Format) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
auto* self = static_cast<PipeWireCaptureSession*>(data);
|
||||||
|
|
||||||
|
struct spa_video_info info = {};
|
||||||
|
if (spa_format_parse(param, &info.media_type, &info.media_subtype) < 0) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (info.media_type != SPA_MEDIA_TYPE_video || info.media_subtype != SPA_MEDIA_SUBTYPE_raw) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (spa_format_video_raw_parse(param, &info.info.raw) < 0) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
self->video_width_.store(info.info.raw.size.width, std::memory_order_relaxed);
|
||||||
|
self->video_height_.store(info.info.raw.size.height, std::memory_order_relaxed);
|
||||||
|
self->video_spa_format_.store(info.info.raw.format, std::memory_order_relaxed);
|
||||||
|
}
|
||||||
|
|
||||||
|
static void on_process(void* data) {
|
||||||
|
auto* self = static_cast<PipeWireCaptureSession*>(data);
|
||||||
|
struct pw_buffer* buffer = nullptr;
|
||||||
|
while ((buffer = pw_stream_dequeue_buffer(self->stream_)) != nullptr) {
|
||||||
|
self->store_frame(*buffer);
|
||||||
|
pw_stream_queue_buffer(self->stream_, buffer);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void store_frame(const struct pw_buffer& pw_buffer) {
|
||||||
|
const struct spa_buffer* buffer = pw_buffer.buffer;
|
||||||
|
if (buffer == nullptr || buffer->n_datas < 1) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const struct spa_data& data = buffer->datas[0];
|
||||||
|
if (data.type == SPA_DATA_DmaBuf && !dma_buf_logged_.exchange(true)) {
|
||||||
|
std::cerr << "screencast: the portal offers DMA-BUF buffers only; software capture "
|
||||||
|
"requires mapped memory (deferred to the hardware-acceleration phase)\n";
|
||||||
|
}
|
||||||
|
if (data.data == nullptr || data.chunk == nullptr) {
|
||||||
|
return; // unmapped (e.g. DMA-BUF) or empty buffer
|
||||||
|
}
|
||||||
|
|
||||||
|
const int width = video_width_.load(std::memory_order_relaxed);
|
||||||
|
const int height = video_height_.load(std::memory_order_relaxed);
|
||||||
|
const uint32_t spa_format = video_spa_format_.load(std::memory_order_relaxed);
|
||||||
|
if (width <= 0 || height <= 0) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const auto pixel_format = pixel_format_from_spa(spa_format);
|
||||||
|
if (!pixel_format.has_value()) {
|
||||||
|
if (!unsupported_format_logged_.exchange(true)) {
|
||||||
|
std::cerr << std::format("screencast: unsupported negotiated pixel format {}\n", spa_format);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const int stride = data.chunk->stride;
|
||||||
|
const int row_bytes = width * 4; // every enumerated format is 4 bytes/pixel
|
||||||
|
if (stride < row_bytes) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const std::size_t required_size = static_cast<std::size_t>(stride) * (static_cast<std::size_t>(height) - 1) +
|
||||||
|
static_cast<std::size_t>(row_bytes);
|
||||||
|
if (data.chunk->size < required_size && data.maxsize < required_size) {
|
||||||
|
return; // buffer smaller than the negotiated frame
|
||||||
|
}
|
||||||
|
|
||||||
|
CapturedFrame frame;
|
||||||
|
frame.width = width;
|
||||||
|
frame.height = height;
|
||||||
|
frame.pixel_format = *pixel_format;
|
||||||
|
frame.stride = stride;
|
||||||
|
frame.timestamp_ns = monotonic_now_ns();
|
||||||
|
frame.pixels.resize(required_size);
|
||||||
|
std::memcpy(frame.pixels.data(), data.data, required_size);
|
||||||
|
|
||||||
|
{
|
||||||
|
std::lock_guard lock(frame_mutex_);
|
||||||
|
latest_frame_ = std::move(frame);
|
||||||
|
has_new_frame_ = true;
|
||||||
|
}
|
||||||
|
frame_cv_.notify_all();
|
||||||
|
}
|
||||||
|
|
||||||
|
void mark_stopped() {
|
||||||
|
{
|
||||||
|
std::lock_guard lock(frame_mutex_);
|
||||||
|
stopped_ = true;
|
||||||
|
}
|
||||||
|
frame_cv_.notify_all();
|
||||||
|
}
|
||||||
|
|
||||||
|
// PipeWire teardown in the order required by the API: destroy the stream
|
||||||
|
// and disconnect the core under the thread-loop lock, then drop the
|
||||||
|
// context, then stop the loop.
|
||||||
|
void teardown_pipewire() noexcept {
|
||||||
|
if (loop_ != nullptr) {
|
||||||
|
pw_thread_loop_lock(loop_);
|
||||||
|
if (stream_ != nullptr) {
|
||||||
|
spa_hook_remove(&stream_listener_);
|
||||||
|
pw_stream_destroy(stream_);
|
||||||
|
stream_ = nullptr;
|
||||||
|
}
|
||||||
|
if (core_ != nullptr) {
|
||||||
|
pw_core_disconnect(core_);
|
||||||
|
core_ = nullptr;
|
||||||
|
}
|
||||||
|
pw_thread_loop_unlock(loop_);
|
||||||
|
}
|
||||||
|
if (context_ != nullptr) {
|
||||||
|
pw_context_destroy(context_);
|
||||||
|
context_ = nullptr;
|
||||||
|
}
|
||||||
|
if (loop_ != nullptr) {
|
||||||
|
pw_thread_loop_stop(loop_);
|
||||||
|
pw_thread_loop_destroy(loop_);
|
||||||
|
loop_ = nullptr;
|
||||||
|
}
|
||||||
|
pw_deinit();
|
||||||
|
}
|
||||||
|
|
||||||
|
uint32_t node_id_ = 0;
|
||||||
|
XdpSessionPtr portal_session_;
|
||||||
|
PortalPtr portal_;
|
||||||
|
|
||||||
|
pw_thread_loop* loop_ = nullptr;
|
||||||
|
pw_context* context_ = nullptr;
|
||||||
|
pw_core* core_ = nullptr;
|
||||||
|
pw_stream* stream_ = nullptr;
|
||||||
|
struct spa_hook stream_listener_ = {};
|
||||||
|
struct pw_stream_events stream_events_ = {};
|
||||||
|
|
||||||
|
std::atomic<int> video_width_{0};
|
||||||
|
std::atomic<int> video_height_{0};
|
||||||
|
std::atomic<uint32_t> video_spa_format_{SPA_VIDEO_FORMAT_UNKNOWN};
|
||||||
|
std::atomic<bool> dma_buf_logged_{false};
|
||||||
|
std::atomic<bool> unsupported_format_logged_{false};
|
||||||
|
|
||||||
|
std::mutex frame_mutex_;
|
||||||
|
std::condition_variable frame_cv_;
|
||||||
|
std::optional<CapturedFrame> latest_frame_;
|
||||||
|
bool has_new_frame_ = false;
|
||||||
|
bool stopped_ = false;
|
||||||
|
};
|
||||||
|
|
||||||
|
CaptureResult<std::unique_ptr<CaptureSession>> CaptureFactory::create(CaptureTarget target) {
|
||||||
|
XdpOutputType outputs = XDP_OUTPUT_NONE;
|
||||||
|
if (std::holds_alternative<CaptureTargetWholeScreen>(target)) {
|
||||||
|
outputs = XDP_OUTPUT_MONITOR;
|
||||||
|
} else if (std::holds_alternative<CaptureTargetWindow>(target)) {
|
||||||
|
outputs = XDP_OUTPUT_WINDOW;
|
||||||
|
} else {
|
||||||
|
return CaptureError{"region capture is not supported by the desktop portal"};
|
||||||
|
}
|
||||||
|
|
||||||
|
pw_init(nullptr, nullptr);
|
||||||
|
|
||||||
|
ScopedMainContext main_context;
|
||||||
|
if (main_context.get() == nullptr) {
|
||||||
|
return CaptureError{"failed to create a GLib main context"};
|
||||||
|
}
|
||||||
|
ScopedMainLoop main_loop(main_context.get());
|
||||||
|
|
||||||
|
PortalPtr portal(xdp_portal_new());
|
||||||
|
if (portal == nullptr) {
|
||||||
|
return CaptureError{"failed to connect to the desktop portal"};
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 1: create the screencast session. The user picks a source in the
|
||||||
|
// portal dialog while this call blocks.
|
||||||
|
SessionCreatedOp created{&main_loop};
|
||||||
|
xdp_portal_create_screencast_session(portal.get(),
|
||||||
|
outputs,
|
||||||
|
XDP_SCREENCAST_FLAG_NONE,
|
||||||
|
XDP_CURSOR_MODE_EMBEDDED,
|
||||||
|
XDP_PERSIST_MODE_NONE,
|
||||||
|
nullptr,
|
||||||
|
nullptr,
|
||||||
|
&on_screencast_session_created,
|
||||||
|
&created);
|
||||||
|
main_loop.run();
|
||||||
|
|
||||||
|
XdpSessionPtr portal_session(created.session);
|
||||||
|
if (portal_session == nullptr) {
|
||||||
|
const std::string message = gerror_message(created.error);
|
||||||
|
g_clear_error(&created.error);
|
||||||
|
return CaptureError{"failed to create the screencast session: " + message};
|
||||||
|
}
|
||||||
|
g_clear_error(&created.error);
|
||||||
|
|
||||||
|
// Step 2: start the session.
|
||||||
|
SessionStartedOp started{&main_loop};
|
||||||
|
xdp_session_start(portal_session.get(), nullptr, nullptr, &on_screencast_session_started, &started);
|
||||||
|
main_loop.run();
|
||||||
|
if (!started.started) {
|
||||||
|
const std::string message = gerror_message(started.error);
|
||||||
|
g_clear_error(&started.error);
|
||||||
|
return CaptureError{"failed to start the screencast session: " + message};
|
||||||
|
}
|
||||||
|
g_clear_error(&started.error);
|
||||||
|
|
||||||
|
// Step 3: find the PipeWire node to capture.
|
||||||
|
const std::optional<uint32_t> node_id = first_stream_node_id(portal_session.get());
|
||||||
|
if (!node_id.has_value()) {
|
||||||
|
return CaptureError{"the screencast session exposes no streams"};
|
||||||
|
}
|
||||||
|
|
||||||
|
OwnedFd remote_fd(xdp_session_open_pipewire_remote(portal_session.get()));
|
||||||
|
if (remote_fd.get() < 0) {
|
||||||
|
return CaptureError{"failed to open the PipeWire remote of the screencast session"};
|
||||||
|
}
|
||||||
|
|
||||||
|
auto session = std::make_unique<PipeWireCaptureSession>(*node_id, std::move(portal_session), std::move(portal));
|
||||||
|
if (auto error = session->connect_to_pipewire(std::move(remote_fd))) {
|
||||||
|
return CaptureError{std::move(*error)};
|
||||||
|
}
|
||||||
|
return std::unique_ptr<CaptureSession>(std::move(session));
|
||||||
}
|
}
|
||||||
|
|
||||||
} // namespace sc
|
} // namespace sc
|
||||||
|
|||||||
@@ -3,6 +3,7 @@
|
|||||||
|
|
||||||
#include "ffmpeg_raii.h"
|
#include "ffmpeg_raii.h"
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
#include <array>
|
#include <array>
|
||||||
#include <atomic>
|
#include <atomic>
|
||||||
#include <cstdint>
|
#include <cstdint>
|
||||||
@@ -18,15 +19,24 @@ namespace {
|
|||||||
using namespace sc::detail;
|
using namespace sc::detail;
|
||||||
|
|
||||||
AVPixelFormat to_ffmpeg_format(PixelFormat format) noexcept {
|
AVPixelFormat to_ffmpeg_format(PixelFormat format) noexcept {
|
||||||
return format == PixelFormat::Rgba ? AV_PIX_FMT_RGBA : AV_PIX_FMT_YUV420P;
|
switch (format) {
|
||||||
|
case PixelFormat::Rgba:
|
||||||
|
return AV_PIX_FMT_RGBA;
|
||||||
|
case PixelFormat::Bgrx:
|
||||||
|
// BGRx is BGR with an unused fourth byte; FFmpeg models that layout as BGRA.
|
||||||
|
return AV_PIX_FMT_BGRA;
|
||||||
|
case PixelFormat::Yuv420p:
|
||||||
|
return AV_PIX_FMT_YUV420P;
|
||||||
|
}
|
||||||
|
return AV_PIX_FMT_NONE;
|
||||||
}
|
}
|
||||||
|
|
||||||
int expected_first_plane_stride(PixelFormat format, int width) noexcept {
|
bool is_packed_rgb(PixelFormat format) noexcept {
|
||||||
return format == PixelFormat::Rgba ? width * 4 : width;
|
return format == PixelFormat::Rgba || format == PixelFormat::Bgrx;
|
||||||
}
|
}
|
||||||
|
|
||||||
std::size_t expected_buffer_size(PixelFormat format, int width, int height) {
|
int minimum_row_bytes(PixelFormat format, int width) noexcept {
|
||||||
return static_cast<std::size_t>(av_image_get_buffer_size(to_ffmpeg_format(format), width, height, 1));
|
return is_packed_rgb(format) ? width * 4 : width;
|
||||||
}
|
}
|
||||||
|
|
||||||
enum class ReceiveStatus {
|
enum class ReceiveStatus {
|
||||||
@@ -244,12 +254,29 @@ class FfmpegEncoder final : public Encoder {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const AVPixelFormat input_format = to_ffmpeg_format(frame.pixel_format);
|
const AVPixelFormat input_format = to_ffmpeg_format(frame.pixel_format);
|
||||||
const int expected_stride = expected_first_plane_stride(frame.pixel_format, frame.width);
|
if (input_format == AV_PIX_FMT_NONE) {
|
||||||
if (frame.stride != 0 && frame.stride != expected_stride) {
|
return CodecError{"unsupported pixel format"};
|
||||||
return CodecError{"unsupported input stride"};
|
|
||||||
}
|
}
|
||||||
|
|
||||||
const std::size_t required_size = expected_buffer_size(frame.pixel_format, frame.width, frame.height);
|
const int row_bytes = minimum_row_bytes(frame.pixel_format, frame.width);
|
||||||
|
std::size_t stride = static_cast<std::size_t>(row_bytes);
|
||||||
|
if (frame.stride != 0) {
|
||||||
|
if (frame.stride < row_bytes) {
|
||||||
|
return CodecError{"input stride is smaller than one pixel row"};
|
||||||
|
}
|
||||||
|
if (!is_packed_rgb(frame.pixel_format) && frame.stride != row_bytes) {
|
||||||
|
return CodecError{"unsupported input stride for planar format"};
|
||||||
|
}
|
||||||
|
stride = static_cast<std::size_t>(frame.stride);
|
||||||
|
}
|
||||||
|
|
||||||
|
std::size_t required_size = 0;
|
||||||
|
if (is_packed_rgb(frame.pixel_format)) {
|
||||||
|
required_size = stride * (static_cast<std::size_t>(frame.height) - 1) + static_cast<std::size_t>(row_bytes);
|
||||||
|
} else {
|
||||||
|
required_size =
|
||||||
|
static_cast<std::size_t>(av_image_get_buffer_size(input_format, frame.width, frame.height, 1));
|
||||||
|
}
|
||||||
if (frame.pixels.size() < required_size) {
|
if (frame.pixels.size() < required_size) {
|
||||||
return CodecError{"input pixel buffer is too small"};
|
return CodecError{"input pixel buffer is too small"};
|
||||||
}
|
}
|
||||||
@@ -274,12 +301,27 @@ class FfmpegEncoder final : public Encoder {
|
|||||||
return CodecError{"failed to create swscale context"};
|
return CodecError{"failed to create swscale context"};
|
||||||
}
|
}
|
||||||
|
|
||||||
std::array<uint8_t*, 4> src{nullptr, nullptr, nullptr, nullptr};
|
std::array<const uint8_t*, 4> src{nullptr, nullptr, nullptr, nullptr};
|
||||||
std::array<int, 4> src_lines{0, 0, 0, 0};
|
std::array<int, 4> src_lines{0, 0, 0, 0};
|
||||||
if (av_image_fill_arrays(
|
if (is_packed_rgb(frame.pixel_format)) {
|
||||||
src.data(), src_lines.data(), as_u8(frame.pixels.data()), input_format, frame.width, frame.height, 1) <
|
// Packed RGB is a single plane whose row pitch may be padded beyond
|
||||||
0) {
|
// the packed row size, so build the source arrays by hand.
|
||||||
return CodecError{"failed to fill input pixel arrays"};
|
src[0] = as_u8(frame.pixels.data());
|
||||||
|
src_lines[0] = static_cast<int>(stride);
|
||||||
|
} else {
|
||||||
|
std::array<uint8_t*, 4> planar_src{nullptr, nullptr, nullptr, nullptr};
|
||||||
|
std::array<int, 4> planar_lines{0, 0, 0, 0};
|
||||||
|
if (av_image_fill_arrays(planar_src.data(),
|
||||||
|
planar_lines.data(),
|
||||||
|
as_u8(frame.pixels.data()),
|
||||||
|
input_format,
|
||||||
|
frame.width,
|
||||||
|
frame.height,
|
||||||
|
1) < 0) {
|
||||||
|
return CodecError{"failed to fill input pixel arrays"};
|
||||||
|
}
|
||||||
|
std::copy(planar_src.begin(), planar_src.end(), src.begin());
|
||||||
|
std::copy(planar_lines.begin(), planar_lines.end(), src_lines.begin());
|
||||||
}
|
}
|
||||||
|
|
||||||
if (sws_scale(scaler_.get(), src.data(), src_lines.data(), 0, frame.height, output->data, output->linesize) <=
|
if (sws_scale(scaler_.get(), src.data(), src_lines.data(), 0, frame.height, output->data, output->linesize) <=
|
||||||
|
|||||||
@@ -0,0 +1,167 @@
|
|||||||
|
// Phase 3 smoke test: capture a few real desktop frames through the portal,
|
||||||
|
// encode them to H.264, and write an Annex-B elementary stream to disk.
|
||||||
|
//
|
||||||
|
// This tool is intentionally manual: it needs a running desktop session and
|
||||||
|
// the user must confirm the source picker dialog, so it is not registered
|
||||||
|
// with `meson test`. Validation steps are documented in docs/RUNBOOK.md.
|
||||||
|
|
||||||
|
#include "screencast/capture/capture.h"
|
||||||
|
#include "screencast/codec/encoder.h"
|
||||||
|
|
||||||
|
#include <charconv>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <format>
|
||||||
|
#include <fstream>
|
||||||
|
#include <iostream>
|
||||||
|
#include <memory>
|
||||||
|
#include <string>
|
||||||
|
#include <string_view>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
int positive_int_arg(std::string_view text) {
|
||||||
|
int value = 0;
|
||||||
|
const auto [ptr, ec] = std::from_chars(text.data(), text.data() + text.size(), value);
|
||||||
|
if (ec != std::errc{} || ptr != text.data() + text.size() || value <= 0) {
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
std::string_view pixel_format_name(sc::PixelFormat format) {
|
||||||
|
switch (format) {
|
||||||
|
case sc::PixelFormat::Rgba:
|
||||||
|
return "rgba";
|
||||||
|
case sc::PixelFormat::Bgrx:
|
||||||
|
return "bgrx";
|
||||||
|
case sc::PixelFormat::Yuv420p:
|
||||||
|
return "yuv420p";
|
||||||
|
}
|
||||||
|
return "unknown";
|
||||||
|
}
|
||||||
|
|
||||||
|
bool write_bytes(std::ofstream& output, const std::vector<std::byte>& bytes) {
|
||||||
|
output.write(reinterpret_cast<const char*>(bytes.data()), static_cast<std::streamsize>(bytes.size()));
|
||||||
|
return output.good();
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
int main(int argc, char** argv) {
|
||||||
|
const int max_frames = argc > 1 ? positive_int_arg(argv[1]) : 10;
|
||||||
|
if (max_frames < 0) {
|
||||||
|
std::cerr << "usage: capture_smoke [frames] [output.h264]\n";
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
const std::string output_path = argc > 2 ? argv[2] : "screencast_smoke.h264";
|
||||||
|
|
||||||
|
std::cout << "creating capture session (choose a source in the portal dialog)...\n";
|
||||||
|
auto capture_result = sc::CaptureFactory::create(sc::CaptureTargetWholeScreen{});
|
||||||
|
if (sc::is_capture_error(capture_result)) {
|
||||||
|
std::cerr << std::format("capture failed: {}\n", sc::capture_error(capture_result).message);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
auto capture = std::move(sc::capture_value(capture_result));
|
||||||
|
|
||||||
|
std::ofstream output(output_path, std::ios::binary | std::ios::trunc);
|
||||||
|
if (!output.is_open()) {
|
||||||
|
std::cerr << std::format("failed to open {} for writing\n", output_path);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
std::unique_ptr<sc::Encoder> encoder;
|
||||||
|
std::uint64_t total_bytes = 0;
|
||||||
|
std::uint64_t encoded_frames = 0;
|
||||||
|
std::uint64_t keyframes = 0;
|
||||||
|
|
||||||
|
for (int captured = 0; captured < max_frames; ++captured) {
|
||||||
|
auto frame = capture->next_frame();
|
||||||
|
if (!frame.has_value()) {
|
||||||
|
std::cout << std::format("capture session ended after {} frames\n", captured);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (encoder == nullptr) {
|
||||||
|
sc::EncoderConfig config;
|
||||||
|
config.width = frame->width;
|
||||||
|
config.height = frame->height;
|
||||||
|
config.frame_rate_num = 25;
|
||||||
|
config.frame_rate_den = 1;
|
||||||
|
config.bitrate_kbps = 8000;
|
||||||
|
|
||||||
|
auto encoder_result = sc::EncoderFactory::create(config);
|
||||||
|
if (sc::is_codec_error(encoder_result)) {
|
||||||
|
std::cerr << std::format("encoder creation failed: {}\n", sc::codec_error(encoder_result).message);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
encoder = std::move(sc::codec_value(encoder_result));
|
||||||
|
|
||||||
|
// The encoder is configured with AV_CODEC_FLAG_GLOBAL_HEADER, so
|
||||||
|
// parameter sets live in extradata; prepend them so the output
|
||||||
|
// file is a self-contained Annex-B stream.
|
||||||
|
const auto extradata = encoder->get_extradata();
|
||||||
|
if (!extradata.empty()) {
|
||||||
|
if (!write_bytes(output, extradata)) {
|
||||||
|
std::cerr << std::format("failed to write parameter sets to {}\n", output_path);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
total_bytes += extradata.size();
|
||||||
|
}
|
||||||
|
|
||||||
|
std::cout << std::format("capturing {}x{} ({}, stride {})\n",
|
||||||
|
frame->width,
|
||||||
|
frame->height,
|
||||||
|
pixel_format_name(frame->pixel_format),
|
||||||
|
frame->stride);
|
||||||
|
}
|
||||||
|
|
||||||
|
auto encoded_result = encoder->encode(*frame);
|
||||||
|
if (sc::is_codec_error(encoded_result)) {
|
||||||
|
std::cerr << std::format("encode failed: {}\n", sc::codec_error(encoded_result).message);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
for (const auto& encoded : sc::codec_value(encoded_result)) {
|
||||||
|
if (!write_bytes(output, encoded.data)) {
|
||||||
|
std::cerr << std::format("failed to write encoded data to {}\n", output_path);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
total_bytes += encoded.data.size();
|
||||||
|
++encoded_frames;
|
||||||
|
keyframes += encoded.is_keyframe ? 1 : 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (encoder != nullptr) {
|
||||||
|
auto flushed = encoder->flush();
|
||||||
|
if (sc::is_codec_error(flushed)) {
|
||||||
|
std::cerr << std::format("flush failed: {}\n", sc::codec_error(flushed).message);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
for (const auto& encoded : sc::codec_value(flushed)) {
|
||||||
|
if (!write_bytes(output, encoded.data)) {
|
||||||
|
std::cerr << std::format("failed to write flushed data to {}\n", output_path);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
total_bytes += encoded.data.size();
|
||||||
|
++encoded_frames;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
capture->stop();
|
||||||
|
output.close();
|
||||||
|
|
||||||
|
if (total_bytes == 0 || encoded_frames == 0) {
|
||||||
|
std::cerr << "no frames were captured\n";
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
std::cout << std::format("wrote {} bytes to {} ({} encoded frames, {} keyframes)\n",
|
||||||
|
total_bytes,
|
||||||
|
output_path,
|
||||||
|
encoded_frames,
|
||||||
|
keyframes);
|
||||||
|
std::cout << std::format("validate with: ffprobe -v error -show_entries stream=codec_name,width,height {}\n",
|
||||||
|
output_path);
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
# Manual smoke tools. These are not registered with `meson test` because they
|
||||||
|
# need an interactive desktop session (portal consent dialog).
|
||||||
|
|
||||||
|
executable('capture_smoke',
|
||||||
|
'capture_smoke.cpp',
|
||||||
|
dependencies : [sc_capture_dep, sc_codec_dep])
|
||||||
Reference in New Issue
Block a user