Compare commits
21 Commits
41f71fd217
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| c43dd3ca4a | |||
| 4f93c7cd20 | |||
| 9a24933203 | |||
| c6722d164b | |||
| bbe4f21a3b | |||
| 455ba6055d | |||
| 69a1cdac1d | |||
| 9ff8e94679 | |||
| 9024930d91 | |||
| 144aab6e5b | |||
| 8f1c2aa868 | |||
| e82e7853d1 | |||
| c78dec139b | |||
| 36d086af4e | |||
| 30538fba73 | |||
| 6516b45b02 | |||
| b4d1411fd9 | |||
| eb86905e67 | |||
| 5c39662cc2 | |||
| 74b3f04082 | |||
| 943596da6d |
@@ -1,10 +1,82 @@
|
||||
# Project Memory — screen_cast
|
||||
|
||||
Last updated: Phase 6 (discovery + signaling) complete and validated over
|
||||
loopback; current phase is Phase 7.
|
||||
Last updated: Phase 8 (Android receiver app) complete and validated on a
|
||||
Fairphone 6; all prior phases done. The Android app's review findings were
|
||||
fixed in a follow-up pass (same day) — see "Android app review" at the
|
||||
bottom of this file.
|
||||
|
||||
## Project state
|
||||
|
||||
- **Phase 9 in progress: iOS receiver app** (`ios/`, Swift, min iOS 17):
|
||||
iPhone as a second receiver. Speaks the same signaling+RTP protocol — no C++
|
||||
changes. Mirrors the Android receiver (Phase 8) source-to-source.
|
||||
- Scaffolding: XcodeGen `project.yml` (Info.plist carries
|
||||
`NSLocalNetworkUsageDescription` + `NSBonjourServices: _screencast._tcp`) +
|
||||
`bootstrap.sh` (downloads XcodeGen from the GitHub release, no Homebrew;
|
||||
generates the project; builds/tests via xcodebuild). `ios/README.md` has
|
||||
build/install/device steps.
|
||||
- App (`Receiver/App`): SwiftUI + `AVSampleBufferDisplayLayer` (`.resizeAspect`
|
||||
= letterbox — the same "size the surface, not a transform" lesson as
|
||||
Android); `ReceiverController` (ObservableObject) drives start/stop on
|
||||
scenePhase.
|
||||
- Protocol core (`Receiver/Rtp`, `Receiver/Signaling`): RtpHeader/RtpPacket/
|
||||
JitterBuffer/H264Depacketizer ported source-to-source; SignalingMessage
|
||||
(JSONSerialization) + LineAssembler + SignalingServer (BSD sockets,
|
||||
dual-stack, most-recent-peer-wins, MSG_NOSIGNAL sends that never throw);
|
||||
`AvccConverter` (Annex-B ↔ AVCC) + `NalExtractor` (SPS/PPS) are new for the
|
||||
VideoToolbox path.
|
||||
- Decode (`Receiver/Decode`): `H264FormatDescription` (Core Foundation H.264
|
||||
config recipe) + `H264VideoToolboxDecoder` (in-band SPS/PPS → session;
|
||||
`kVTDecompressionPropertyKey_RealTime`; session recreated on size change;
|
||||
the output callback may run on a worker thread, so state is lock-guarded)
|
||||
→ `AVSampleBufferRenderSink` (AVSampleBufferDisplayLayerSession).
|
||||
- Support (`Receiver/Support`): `UdpTransport` (poll-based recv so close() can't
|
||||
strand a blocked recvfrom) + `LocalAddress` (getifaddrs for the --peer hint).
|
||||
- Pipeline (`Receiver/Pipeline`): ReceiverPipeline — one serial queue for all
|
||||
state + decode; reader thread only polls/receives UDP; pendingOffer for late
|
||||
surface attach (+ PLI); PLI rate-limited 500 ms; first-frame flag; video
|
||||
size tracked (never reports a placeholder before the first real keyframe —
|
||||
the Android startup-squish lesson).
|
||||
- Tests (`ios/ReceiverTests`): the 25 Android JVM tests ported to XCTest plus
|
||||
new coverage for signaling JSON, line framing, AVCC, NAL extraction.
|
||||
- `.gitignore` excludes the generated `ios/Receiver.xcodeproj/`, `ios/tools/`,
|
||||
`ios/Receiver/Info.plist` (the project.yml `info` block is the source of
|
||||
truth).
|
||||
- **NOT YET VALIDATED** (this box is Linux, no Xcode/iOS SDK): nothing here
|
||||
compiles or runs. 9.5 needs a Mac with Xcode 26 + the iPhone 16.
|
||||
Highest-risk on-device items: (1) local-network + Bonjour consent
|
||||
(NSBonjourServices must be `_screencast._tcp`; iOS 26 tightened the prompt),
|
||||
(2) the H264FormatDescription CF ownership recipe (a bug crashes on the
|
||||
first keyframe — loud, not silent), (3) VideoToolbox decode →
|
||||
AVSampleBufferDisplayLayer render. Run `ios/bootstrap.sh test` first on the
|
||||
Mac.
|
||||
|
||||
- **Phase 8 done: Android receiver app** (`android/`, Kotlin, minSdk 30,
|
||||
app id `screen_cast.receiver`): phone as second receiver (screen → HDMI via
|
||||
USB-C DP-alt-mode). Speaks the existing signaling+RTP protocol — no C++
|
||||
changes. AGP 9 **built-in Kotlin** (no kotlin plugin, no `kotlinOptions`;
|
||||
Kotlin targets `compileOptions`, Java 17); no androidx (framework +
|
||||
org.json + JUnit). 25 JVM tests green. Validated end-to-end on a
|
||||
Fairphone 6 (API 36): offer/answer → MediaCodec → fullscreen letterboxed
|
||||
render; PLI recovery on real Wi-Fi loss. Hard-won API-36 quirks (in
|
||||
RUNBOOK): `INTERNET` permission is REQUIRED for NsdService;
|
||||
`DatagramSocket.localPort` (`.port` is -1 unconnected; `.localAddress` is
|
||||
Inet6Address, not InetSocketAddress); pass the Surface to
|
||||
`MediaCodec.configure()` + `start()`; render via
|
||||
`releaseOutputBuffer(render=true)`; C2 AVC needs a concrete size at
|
||||
configure (in-band SPS reconfigures); `MediaFormat.format()`/
|
||||
`KEY_MIME_TYPE` not public in API 36; NSD: the classic
|
||||
`registerService(info, flags, RegistrationListener)` API exists from API
|
||||
16 through 36 (javap-verified on the android-36 SDK) — an old reflection
|
||||
fallback targeting a never-existent "ResolutionListener" was removed as
|
||||
dead code;
|
||||
**`android._video-scaling`: C2 scales output to the Surface** → letterbox
|
||||
by sizing the TextureView to the video aspect, NOT a transform matrix
|
||||
(double scale). Sender-side: Hyprland + GTK portal's `--target monitor`
|
||||
picks the FIRST output (eDP-1, wrong display) — use `--target window`
|
||||
(hyprland-share-picker). Activity FQN for `am start`:
|
||||
`screen_cast.receiver/screen_cast.ReceiverActivity` (Kotlin package =
|
||||
namespace `screen_cast`).
|
||||
- **Phase 6 done**: Avahi mDNS discovery (`_screencast._tcp` — receiver
|
||||
announces its signaling port via a threaded-poll Avahi client; senders
|
||||
browse+resolve) plus JSON session signaling (offer/answer) over TCP with
|
||||
@@ -65,12 +137,55 @@ loopback; current phase is Phase 7.
|
||||
before the fix). Cross-compiling on the dev machine was considered and
|
||||
dropped — the on-Pi build works and the toolchain/container effort was
|
||||
not needed.
|
||||
- **Pi Wi-Fi hotspot** (`scripts/pi-hotspot.sh on|off|status`): NetworkManager
|
||||
- **App icons wired up** (`screencast_icon/`): single 256 px PNG used
|
||||
everywhere. `meson install` ships it to
|
||||
`share/icons/hicolor/256x256/apps/screencast.png`; the GTK panel sets it
|
||||
via `set_icon_name("screencast")` (GTK4 removed the pixel-buffer window
|
||||
icon — themed names only; `gtk_icon_theme_has_icon` verified OK); the
|
||||
SDL receiver window embeds the PNG at build time (`icon_png_data.h` via
|
||||
`scripts/icon_to_header.py` custom_target) and `SDL_SetWindowIcon` with
|
||||
an optional SDL3_image dependency (no pkg-config ships with it →
|
||||
`meson.get_compiler('cpp').find_library('SDL3_image', required: false)`
|
||||
+ disabler; `-DSC_HAS_WINDOW_ICON` gate; Pi builds without it still
|
||||
work). The waybar widget now shows the icon as `background-image` in
|
||||
`~/.config/waybar/style.css` (▶/⏸ text hidden; `.idle` dimmed,
|
||||
`.streaming` green tint) instead of text glyphs; backup
|
||||
`style.css.bak-20260909-*`. Lesson: icons appear in compositor
|
||||
taskbars/switchers, not title bars (GTK4 CSD and Hyprland decorations
|
||||
don't draw them), so validate via theme lookup, not title-bar
|
||||
screenshots. `sudo pacman -S sdl3_image` done (extra/sdl3_image 3.4.6).
|
||||
- **GTK panel + waybar widget**: `screencast-gui` (gtkmm-4.0, behind
|
||||
`-Dgui=true`; app internals now live in the `sc_app_core` static lib so
|
||||
CLI and GUI share pipelines/session/state). `screencast waybar [--toggle]`
|
||||
prints the waybar module line and toggles streaming via SIGTERM (stop) or
|
||||
a detached re-exec spawn from `last-session.json` (start). State lives in
|
||||
`$XDG_RUNTIME_DIR/screencast/sender.json` (written by the pipeline with
|
||||
session id, receiver, bitrate, pid, start time; stale files are detected
|
||||
by pid-liveness). The waybar config was wired into the user's bar
|
||||
(`custom/screencast` before `custom/timetrack`, with backup) and the
|
||||
binaries installed to /usr/local/bin.
|
||||
- **Pi Wi-Fi hotspot** (`scripts/pi-hotspot.sh on|off|status`): NetworkManager
|
||||
AP mode (WPA2, ipv4 shared → built-in DHCP/NAT, Pi at 10.42.0.1). Takes
|
||||
over wlan0 while active; generated PSK stored in /etc/screencast-hotspot.conf.
|
||||
No application changes needed — the receiver already announces on all
|
||||
interfaces. nmcli property syntax validated against NM 1.58 with a
|
||||
disposable profile; AP bring-up itself can only be validated on the Pi.
|
||||
- **Phase 7 resilience shipped** (pending Pi-side hw-decode run):
|
||||
- PLI over signaling: `SessionPli` message; the depacketizer now returns
|
||||
`DepacketizeResult{access_unit, frame_dropped}`; the receiver
|
||||
rate-limits PLIs to 1/500 ms; the sender keeps the signaling channel
|
||||
open and calls `SenderPipeline::request_keyframe()` (thread-safe
|
||||
atomic → run-loop → encoder). Validated end-to-end with a probe that
|
||||
drops a mid-keyframe packet: receiver logs "frame damaged",
|
||||
PLI arrives with the session id.
|
||||
- `RtpJitterBuffer` (rtp_packet.h): sequence reordering, 16 pkt/60 ms,
|
||||
straggler discard via serial-number arithmetic, overflow flush for
|
||||
genuine loss. Zero added latency on in-order streams.
|
||||
- Hardware decode probe in DecoderFactory (`DecoderConfig.hardware_accel`,
|
||||
default true): h264_v4l2m2m first, software fallback, `--swdecode` opts
|
||||
out. On the desktop the probe fails cleanly ("Could not find a valid
|
||||
device") and falls back; on the Pi it should pick the VideoCore m2m
|
||||
device — NEEDS THE USER'S PI RUN to confirm.
|
||||
- **REAL-HARDWARE VALIDATION (desktop → Pi Zero 2 W over Wi-Fi)**: the full
|
||||
chain works on two machines: mDNS discovery → signaling negotiation →
|
||||
RTP over Wi-Fi → software H.264 decode → fullscreen KMSDRM letterboxed
|
||||
@@ -192,4 +307,95 @@ None.
|
||||
- `CaptureSession::next_frame()` returns `nullopt` on stream error without
|
||||
surfacing the reason (logged to stderr).
|
||||
- Receiver ignores unknown packetization modes (STAP-A/MTAP/FU-B); senders
|
||||
we control never emit them, but third-party interop would need support.
|
||||
we control never emit them, but third-party interop would need support.
|
||||
|
||||
### Android app review (2026-09-10) — findings FIXED same day
|
||||
|
||||
Review pass (25 JVM tests re-run green; rtp/jitter/depacketizer verified
|
||||
faithful to the C++ side source-to-source), followed by a fix pass that
|
||||
landed all findings. No commit yet (user has not asked).
|
||||
|
||||
Fixed:
|
||||
|
||||
- `SignalingServer.send()` no longer throws (mirrors the C++ server, which
|
||||
ignores write failures): a broken signaling TCP no longer kills the
|
||||
`rtp-reader` thread via `requestPli()`; `peerOut` is dropped (closing the
|
||||
socket) on write failure.
|
||||
- `H264Decoder.configure()` assigns the created codec before configuring,
|
||||
so a configure/start failure can no longer orphan the MediaCodec instance
|
||||
(no finalizer; scarce native slots).
|
||||
- Offer-before-surface no longer configures a ByteBuffer-mode decoder: the
|
||||
offer is parked as `pendingOffer` and `attachSurface()` configures later
|
||||
(a surface-less codec can never take one — `setOutputSurface` refuses
|
||||
it, an IllegalStateException crash on the main thread). The late
|
||||
configure sends a PLI (the sender only emits IDRs when asked).
|
||||
- `ReceiverActivity.onSurfaceTextureDestroyed` → `pipeline.detachSurface()`;
|
||||
the pipeline/decoder forget dead surfaces instead of configuring against
|
||||
them (frames decode unrendered until the next attach).
|
||||
- The status overlay reappears: `onStatus` sets `visibility = VISIBLE`
|
||||
(it used to write into a GONE view after the first frame).
|
||||
- `H264Decoder.feed()` returns false on input-queue timeout → the pipeline
|
||||
requests a PLI instead of silently dropping the frame (no flush — the
|
||||
codec is healthy).
|
||||
- The depacketizer accumulates into `ByteArrayOutputStream`s instead of
|
||||
boxed `ArrayList<Int>` (was ~MB/s of Integer allocations on keyframes).
|
||||
- NSD reflection fallback deleted (dead code — see the quirk note above).
|
||||
- Thread-safety: all codec calls now run under `decoderLock` (MediaCodec is
|
||||
not thread-safe; feed/drain previously raced attachSurface);
|
||||
`requestPli()` is thread-safe via `pliLock`.
|
||||
|
||||
Validation: `gradle :app:assembleDebug :app:testDebugUnitTest` green
|
||||
(25/25, full --rerun-tasks rebuild, only two pre-existing warnings);
|
||||
`meson test` 5/5 unchanged (no C++ touched).
|
||||
|
||||
### Discovery bug found and fixed on-device (2026-09-10, same day)
|
||||
|
||||
The user reported the desktop sender never discovered the phone.
|
||||
Root cause (found live on the Fairphone 6): `registerService(info, 0,
|
||||
listener)` — Android 16's `NsdManager.checkProtocol()` rejects protocol
|
||||
`0` with `IllegalArgumentException: Unsupported protocol`. The old code
|
||||
swallowed it into the dead reflection fallback, and `start()` posted the
|
||||
"Listening…" status AFTER advertiseNsd, overwriting the failure text — so
|
||||
mDNS never advertised and Phase 8's NSD validation was only ever "no
|
||||
crash" (8.5 streamed via --peer, masking it).
|
||||
|
||||
Fixes in `ReceiverPipeline.advertiseNsd()`/`start()`:
|
||||
- pass `NsdManager.PROTOCOL_DNS_SD`;
|
||||
- advertise AFTER the listening status so a failure stays visible;
|
||||
- `Log.e` the registration exception; `Log.i` on registered success.
|
||||
|
||||
On-device validation (adb, live): `mDNS registered:
|
||||
screencast._screencast._tcp` in logcat; `dumpsys servicediscovery` shows
|
||||
the active Advertiser (key diagnostic: `mClientRequests` empty == no
|
||||
request ever issued); desktop `avahi-browse` and `screencast --discover`
|
||||
list the phone at 192.168.178.29:5005; a 25s `--send --target monitor
|
||||
--peer 192.168.178.29:5005` session decoded (in-band SPS reconfigured
|
||||
320x240 → 2496x1040) and rendered (screencap mean brightness 0.51).
|
||||
Note: with both the Pi and the phone on the LAN, plain `--send` refuses
|
||||
(two receivers found) — target the phone with `--peer`.
|
||||
|
||||
Follow-up from the same on-device session — startup squish eliminated:
|
||||
`H264Decoder.outputSize()` previously reported the 320x240 configure()
|
||||
placeholder until the codec parsed the SPS, so `fitVideo` sized the
|
||||
TextureView 1488x1116 (1.33 aspect) and the first rendered frame(s) of
|
||||
2.4 content were visibly squished. It now returns null until
|
||||
`INFO_OUTPUT_FORMAT_CHANGED` fires; the first frame renders into the
|
||||
fullscreen surface and the correct letterbox (2484x1035) follows within
|
||||
~40ms. Validated live: no placeholder "video size" line in logcat.
|
||||
|
||||
Commits: 9a24933 (PROTOCOL_DNS_SD discovery fix + docs), plus the
|
||||
outputSize fix (see git log).
|
||||
|
||||
Still open (accepted, needs a device or a new test dep):
|
||||
|
||||
- Untested on device: `setOutputSurface` mid-session (surface switch) and
|
||||
the whole pendingOffer path; API-35 `detachOutputSurface()` could replace
|
||||
the render-flag approach.
|
||||
- No JVM tests for `SignalingMessage`/`SignalingServer` (would need the
|
||||
`org.json:json` test dependency; android.jar stubs throw).
|
||||
- Interop caveats by design: 2048-byte datagram buffer (our MTU is 1200),
|
||||
RTP timestamps (90 kHz) fed as µs, unauthenticated offers (SRTP is a
|
||||
future phase).
|
||||
- Verified NOT a bug: reusing one `DatagramPacket` without resetting its
|
||||
length — modern JVMs recv by buffer capacity (JDK-21 probe + the
|
||||
successful on-device streaming confirm it).
|
||||
@@ -19,6 +19,11 @@ compile_commands.json
|
||||
# Smoke test output
|
||||
*.h264
|
||||
|
||||
# iOS (generated by bootstrap.sh / XcodeGen; project.yml is the source of truth)
|
||||
ios/Receiver.xcodeproj/
|
||||
ios/tools/
|
||||
ios/Receiver/Info.plist
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
@@ -29,3 +34,4 @@ compile_commands.json
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
hotspot.txt
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
# Changelog
|
||||
|
||||
Notable changes to `screen_cast`. Loosely follows [Keep a Changelog].
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- **Phase 9 — iOS receiver app** (`ios/`): a native Swift receiver so an iPhone
|
||||
can act as the second receiver. Speaks the existing signaling + RTP protocol
|
||||
(no C++ changes), mirroring the Android receiver (Phase 8) source-to-source.
|
||||
- RTP core (header/packet, jitter buffer, H.264 depacketizer) ported
|
||||
source-to-source from the Android receiver.
|
||||
- BSD-socket signaling server (dual-stack, most-recent-peer, never-throwing
|
||||
sends) + `NSBonjourServices` advertisement.
|
||||
- VideoToolbox H.264 decode (in-band SPS/PPS, real-time, session rebuilt on
|
||||
resolution change) rendered via `AVSampleBufferDisplayLayer` (letterbox).
|
||||
- PLI keyframe recovery (rate-limited 500 ms) and `pendingOffer` for late
|
||||
surface attach; no placeholder sizing before the first real keyframe.
|
||||
- XcodeGen project + `bootstrap.sh`; XCTest port of the Android suite plus new
|
||||
signaling / line-framing / AVCC / NAL-extraction coverage.
|
||||
|
||||
> **Status:** authored; on-device validation pending a Mac + Xcode 26 + iPhone 16
|
||||
> (local-network/Bonjour consent, the H.264 format-description recipe, and the
|
||||
> VideoToolbox render path are the on-device verification items).
|
||||
|
||||
[Keep a Changelog]: https://keepachangelog.com/en/1.1.0/
|
||||
@@ -3,9 +3,10 @@
|
||||
A native Linux peer-to-peer screencast application.
|
||||
|
||||
- **Send** your desktop or a window to another Linux machine.
|
||||
- **Receive** a stream and render it in a window.
|
||||
- **Receive** a stream and render it in a window — or fullscreen headless.
|
||||
- Discover receivers on the LAN via **mDNS/Avahi**; negotiate sessions with
|
||||
**JSON signaling**; stream **H.264 over RTP/UDP**.
|
||||
- Control it from the **CLI**, a **GTK4 panel**, or a **waybar widget**.
|
||||
|
||||
Built with **C++20**, **Meson**, **PipeWire**, **FFmpeg**, and **SDL3**.
|
||||
|
||||
@@ -19,6 +20,7 @@ Requirements:
|
||||
- SDL3 development package (`sdl3`)
|
||||
- nlohmann JSON (`nlohmann_json`) and Avahi client (`avahi-client`)
|
||||
- For the sender only: PipeWire dev (`libpipewire-0.3`) and libportal
|
||||
- Optional, for the GUI: gtkmm-4.0
|
||||
|
||||
Build and run tests:
|
||||
|
||||
@@ -32,11 +34,72 @@ Stream between two machines:
|
||||
|
||||
```sh
|
||||
screencast --receive # machine A: announces itself, opens a window
|
||||
screencast --send # machine B: discovers A, negotiates, streams
|
||||
screencast --send # machine B: discovers A, negotiates, streams
|
||||
```
|
||||
|
||||
See `docs/RUNBOOK.md` for all modes, flags, and validation procedures.
|
||||
|
||||
## Modes and flags
|
||||
|
||||
```text
|
||||
screencast --send [--target monitor|window] [--peer HOST[:PORT]] [--bitrate MAX_KBPS] [--crf 0-51] [--fps 1-60]
|
||||
screencast --receive [--port PORT] [--signaling-port PORT] [--fullscreen] [--swdecode]
|
||||
screencast --discover [--timeout SECONDS]
|
||||
screencast waybar [--toggle] # for waybar widgets
|
||||
```
|
||||
|
||||
| Mode | Flag | Default | Meaning |
|
||||
|-------------|------------------|---------------|------------------------------------------------|
|
||||
| `--send` | `--target` | `monitor` | `monitor` or `window` (portal source picker) |
|
||||
| | `--peer` | auto-discover | target a receiver directly, `HOST[:PORT]` |
|
||||
| | `--bitrate` | `4000` | VBV max bitrate (kbps) |
|
||||
| | `--crf` | `22` | encoder quality (lower = better) |
|
||||
| | `--fps` | capture rate | frame-rate cap, 1-60 |
|
||||
| `--receive` | `--port` | `5004` | local RTP UDP port |
|
||||
| | `--signaling-port` | `5005` | TCP signaling port announced via mDNS |
|
||||
| | `--fullscreen` | off | force fullscreen (automatic under KMSDRM) |
|
||||
| | `--swdecode` | off | skip the hardware decode probe |
|
||||
| `--discover`| `--timeout` | `3` | seconds to wait for mDNS responses |
|
||||
| `waybar` | `--toggle` | print status | toggle streaming (stop / restart last session) |
|
||||
|
||||
`--send` without `--peer` discovers receivers on the LAN and requires that
|
||||
exactly one is found; use `--discover` to list them.
|
||||
|
||||
## GUI and waybar widget
|
||||
|
||||
Build the GUI alongside the CLI (`meson configure build -Dgui=true`, then
|
||||
recompile; installs as `screencast-gui`):
|
||||
|
||||
```sh
|
||||
screencast-gui # receiver list → pick one → bitrate → Start
|
||||
screencast waybar # one JSON line for a waybar custom module
|
||||
screencast waybar --toggle
|
||||
```
|
||||
|
||||
The waybar module shows the app icon (dimmed while idle, highlighted while
|
||||
streaming; tooltip: receiver, bitrate, elapsed); left-click toggles
|
||||
streaming to the last receiver, right-click opens the panel. All front-ends
|
||||
agree on state because the sender publishes it to
|
||||
`$XDG_RUNTIME_DIR/screencast/sender.json`. The icon (from `screencast_icon/`)
|
||||
is installed into the hicolor theme by `meson install`, used by the GTK
|
||||
panel and the waybar CSS, and embedded in the SDL receiver window.
|
||||
|
||||
## Under the hood: resilience
|
||||
|
||||
The receiver absorbs loss in two stages and recovers actively:
|
||||
|
||||
1. **Jitter window**: RTP packets are re-ordered by sequence number in a
|
||||
small buffer (16 packets / 60 ms), so Wi-Fi reordering is not misread as
|
||||
loss. In-order streams release immediately (zero added latency).
|
||||
2. **PLI feedback**: when a frame arrives genuinely damaged, the receiver
|
||||
drops it and asks the sender for a keyframe over the signaling channel
|
||||
(rate-limited to one request per 500 ms); the sender re-encodes a
|
||||
keyframe immediately, so recovery takes one frame time.
|
||||
|
||||
**Hardware decode**: the receiver probes `h264_v4l2m2m` (the VideoCore path
|
||||
on Raspberry Pi) and falls back to software automatically; `--swdecode`
|
||||
forces software.
|
||||
|
||||
## Receiver on a small ARM board (e.g. Raspberry Pi Zero 2 W)
|
||||
|
||||
The receiver does not need the sender's PipeWire/portal capture stack. On
|
||||
@@ -53,13 +116,60 @@ The script installs the dependencies, builds a receiver-only binary
|
||||
or newer (GCC 13+ for C++20 `<format>`), builds SDL3 from source when the
|
||||
distribution does not package it, and enables `avahi-daemon`. It also
|
||||
installs and enables a systemd service so the receiver starts at boot —
|
||||
`systemctl status screencast-receiver` to check on it.
|
||||
`systemctl status screencast-receiver` to check on it (stop with
|
||||
`sudo systemctl stop screencast-receiver`; always SIGTERM, never `kill -9`,
|
||||
or stale mDNS records linger).
|
||||
|
||||
Performance note: decoding is software H.264; on very small boards expect
|
||||
smooth playback for modest resolutions and reduced frame rates at high
|
||||
resolutions. Hardware decode is planned for Phase 7. On a headless console
|
||||
the receiver runs fullscreen automatically with aspect-preserving
|
||||
letterboxing.
|
||||
On a headless console the receiver runs fullscreen automatically with
|
||||
aspect-preserving letterboxing — no window manager needed (SDL3 KMSDRM; it
|
||||
renders on its own VT, tty7, so `Ctrl+Alt+F1` returns to the console).
|
||||
See `docs/RUNBOOK.md` for the headless setup details.
|
||||
|
||||
## Direct link: receiver as a Wi-Fi hotspot
|
||||
|
||||
The receiver can act as a Wi-Fi access point, so a sender connects to the
|
||||
board directly with no router in between:
|
||||
|
||||
```sh
|
||||
sudo scripts/pi-hotspot.sh on # prints SSID + generated password
|
||||
sudo scripts/pi-hotspot.sh status # shows the saved credentials
|
||||
sudo scripts/pi-hotspot.sh off # back to normal router Wi-Fi
|
||||
```
|
||||
|
||||
While active the Pi is reachable at `10.42.0.1`; on the sender, join the
|
||||
hotspot's Wi-Fi and run `screencast --send`.
|
||||
|
||||
## Android receiver: a phone as the second screen
|
||||
|
||||
A native receiver app (Kotlin, minSdk 30) lets a phone act as the second
|
||||
receiver — and with a USB-C DisplayPort-alt-mode cable, the phone screen
|
||||
mirrors straight to HDMI. The app speaks the same signaling + RTP protocol,
|
||||
so the sender needs no changes.
|
||||
|
||||
```sh
|
||||
cd android
|
||||
gradle :app:assembleDebug # or :app:installDebug with a device attached
|
||||
adb install app/build/outputs/apk/debug/app-debug.apk
|
||||
```
|
||||
|
||||
Launch the app (it advertises `_screencast._tcp` and shows its IP + ports),
|
||||
then on the sender:
|
||||
|
||||
```sh
|
||||
screencast --send --peer <phone-ip>:5005
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- The sender downscales to the phone's display resolution automatically
|
||||
(from the signaling answer).
|
||||
- The phone renders letterboxed (fit-within), fullscreen, screen kept on.
|
||||
- The app needs only the `INTERNET` permission; no camera/location.
|
||||
- mDNS discovery works on the same L2 segment; across subnets use `--peer`
|
||||
(the app prints its IP and the exact fallback command).
|
||||
- `--target monitor` captures the portal's default output — on multi-monitor
|
||||
Hyprland/GTK-portal setups that may not be the one you want; use
|
||||
`--target window` and pick a window on the target display.
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -69,8 +179,12 @@ See `docs/ARCHITECTURE.md` for module boundaries and design rules.
|
||||
|
||||
Development is split into phases in `docs/PHASES.md`.
|
||||
|
||||
Current phase: **Phase 7 — Resilience and polish**.
|
||||
Current phase: **Phase 8 — Android receiver app** (complete): the Kotlin
|
||||
receiver app streams to a phone (validated on a Fairphone 6, phone → HDMI
|
||||
via USB-C DP-alt-mode). Phase 7 items (PLI, jitter, hardware decode,
|
||||
GUI/waybar) are complete and validated; deferred items remain the VAAPI
|
||||
hardware encode probe and packaging.
|
||||
|
||||
## License
|
||||
|
||||
MIT — see `LICENSE` (to be added).
|
||||
MIT — see `LICENSE` (to be added).
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
.gradle/
|
||||
.kotlin/
|
||||
build/
|
||||
local.properties
|
||||
.idea/
|
||||
captures/
|
||||
.cxx/
|
||||
@@ -0,0 +1,32 @@
|
||||
plugins {
|
||||
id("com.android.application")
|
||||
}
|
||||
|
||||
android {
|
||||
namespace = "screen_cast"
|
||||
compileSdk = 36
|
||||
|
||||
defaultConfig {
|
||||
applicationId = "screen_cast.receiver"
|
||||
minSdk = 30
|
||||
targetSdk = 36
|
||||
versionCode = 1
|
||||
versionName = "0.1.0"
|
||||
}
|
||||
|
||||
// No release signing configured: the app is sideloaded as a debug build.
|
||||
buildTypes {
|
||||
release {
|
||||
isMinifyEnabled = false
|
||||
}
|
||||
}
|
||||
|
||||
compileOptions {
|
||||
sourceCompatibility = JavaVersion.VERSION_17
|
||||
targetCompatibility = JavaVersion.VERSION_17
|
||||
}
|
||||
}
|
||||
|
||||
dependencies {
|
||||
testImplementation("junit:junit:4.13.2")
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
|
||||
|
||||
<!-- NsdService and all socket I/O require INTERNET. No other permissions:
|
||||
no camera, no location, no network state. The screen is kept on, which
|
||||
keeps Wi-Fi up. -->
|
||||
<uses-permission android:name="android.permission.INTERNET" />
|
||||
|
||||
<application
|
||||
android:label="screencast"
|
||||
android:icon="@mipmap/ic_launcher"
|
||||
android:allowBackup="false"
|
||||
android:theme="@style/Theme.Screencast">
|
||||
|
||||
<activity
|
||||
android:name=".ReceiverActivity"
|
||||
android:exported="true"
|
||||
android:launchMode="singleTask"
|
||||
android:screenOrientation="sensorLandscape"
|
||||
android:configChanges="orientation|screenSize|screenLayout|smallestScreenSize|keyboard|keyboardHidden|navigation|uiMode|density"
|
||||
android:stateNotNeeded="true">
|
||||
<intent-filter>
|
||||
<action android:name="android.intent.action.MAIN" />
|
||||
<category android:name="android.intent.category.LAUNCHER" />
|
||||
</intent-filter>
|
||||
</activity>
|
||||
</application>
|
||||
</manifest>
|
||||
@@ -0,0 +1,164 @@
|
||||
package screen_cast
|
||||
|
||||
import android.app.Activity
|
||||
import android.graphics.SurfaceTexture
|
||||
import android.os.Bundle
|
||||
import android.os.Handler
|
||||
import android.os.Looper
|
||||
import android.view.Surface
|
||||
import android.view.TextureView
|
||||
import android.view.View
|
||||
import android.view.WindowManager
|
||||
import android.widget.TextView
|
||||
import java.net.Inet4Address
|
||||
import java.net.NetworkInterface
|
||||
import kotlin.math.min
|
||||
import screen_cast.pipeline.ReceiverPipeline
|
||||
|
||||
/**
|
||||
* Fullscreen receiver: a letterboxed TextureView plus a status overlay.
|
||||
* The pipeline lives in the activity lifetime — created on demand, stopped
|
||||
* when the app leaves the foreground, released on destroy.
|
||||
*
|
||||
* For the USB-C → HDMI use case the app only guarantees the screen is on,
|
||||
* undimmed, landscape, and immersive; the display mirroring itself is the
|
||||
* OS behavior of a DisplayPort-alt-mode port.
|
||||
*/
|
||||
class ReceiverActivity : Activity() {
|
||||
companion object {
|
||||
private const val TAG = "ReceiverActivity"
|
||||
}
|
||||
|
||||
private val ui = Handler(Looper.getMainLooper())
|
||||
private var pipeline: ReceiverPipeline? = null
|
||||
|
||||
private lateinit var videoView: TextureView
|
||||
private lateinit var statusView: TextView
|
||||
|
||||
override fun onCreate(savedInstanceState: Bundle?) {
|
||||
super.onCreate(savedInstanceState)
|
||||
setContentView(R.layout.activity_receiver)
|
||||
videoView = findViewById(R.id.video)
|
||||
statusView = findViewById(R.id.status)
|
||||
videoView.isOpaque = true
|
||||
|
||||
videoView.surfaceTextureListener = object : TextureView.SurfaceTextureListener {
|
||||
override fun onSurfaceTextureAvailable(surface: SurfaceTexture, width: Int, height: Int) {
|
||||
pipeline?.attachSurface(Surface(surface))
|
||||
fitVideo()
|
||||
}
|
||||
|
||||
override fun onSurfaceTextureSizeChanged(surface: SurfaceTexture, width: Int, height: Int) {
|
||||
fitVideo()
|
||||
}
|
||||
|
||||
override fun onSurfaceTextureDestroyed(surface: SurfaceTexture): Boolean {
|
||||
// The TextureView is about to release this SurfaceTexture; the
|
||||
// pipeline must forget it or a later offer would configure the
|
||||
// decoder against a dead surface.
|
||||
pipeline?.detachSurface()
|
||||
return true
|
||||
}
|
||||
|
||||
override fun onSurfaceTextureUpdated(surface: SurfaceTexture) = Unit
|
||||
}
|
||||
}
|
||||
|
||||
override fun onResume() {
|
||||
super.onResume()
|
||||
applyFullscreen()
|
||||
ensurePipeline()
|
||||
pipeline?.start()
|
||||
}
|
||||
|
||||
override fun onPause() {
|
||||
super.onPause()
|
||||
pipeline?.stop()
|
||||
}
|
||||
|
||||
override fun onDestroy() {
|
||||
super.onDestroy()
|
||||
pipeline?.stop()
|
||||
pipeline = null
|
||||
}
|
||||
|
||||
private fun ensurePipeline() {
|
||||
if (pipeline != null) return
|
||||
val pipeline = ReceiverPipeline(
|
||||
context = applicationContext,
|
||||
localIp = localIpv4(),
|
||||
displaySize = {
|
||||
val metrics = resources.displayMetrics
|
||||
metrics.widthPixels to metrics.heightPixels
|
||||
},
|
||||
onStatus = { text ->
|
||||
ui.post {
|
||||
statusView.text = text
|
||||
statusView.visibility = View.VISIBLE
|
||||
}
|
||||
},
|
||||
onFirstFrame = { ui.post { statusView.visibility = View.GONE } },
|
||||
onVideoSize = { _, _ -> ui.post { fitVideo() } },
|
||||
)
|
||||
// The surface may already exist by the time the pipeline starts.
|
||||
val surfaceTexture = videoView.surfaceTexture
|
||||
if (surfaceTexture != null) {
|
||||
pipeline.attachSurface(Surface(surfaceTexture))
|
||||
}
|
||||
this.pipeline = pipeline
|
||||
}
|
||||
|
||||
/**
|
||||
* Fit-within (letterbox) by sizing the TextureView to the video's aspect
|
||||
* ratio, centered on the black window background. The C2 decoder scales
|
||||
* its output to the Surface (android._video-scaling), so a surface with
|
||||
* the video's aspect ratio renders 1:1 without distortion; a transform
|
||||
* matrix on top would double-scale the already-stretched buffer.
|
||||
*/
|
||||
private fun fitVideo() {
|
||||
val (videoWidth, videoHeight) = pipeline?.videoSize() ?: return
|
||||
val parent = videoView.parent as? View ?: return
|
||||
val parentWidth = parent.width
|
||||
val parentHeight = parent.height
|
||||
if (videoWidth <= 0 || videoHeight <= 0 || parentWidth <= 0 || parentHeight <= 0) return
|
||||
|
||||
val scale = min(parentWidth.toFloat() / videoWidth, parentHeight.toFloat() / videoHeight)
|
||||
val width = (videoWidth * scale).toInt()
|
||||
val height = (videoHeight * scale).toInt()
|
||||
val params = videoView.layoutParams
|
||||
if (params.width == width && params.height == height) return
|
||||
params.width = width
|
||||
params.height = height
|
||||
if (params is android.widget.FrameLayout.LayoutParams) {
|
||||
params.gravity = android.view.Gravity.CENTER
|
||||
}
|
||||
videoView.layoutParams = params
|
||||
android.util.Log.i(TAG, "fitVideo: ${width}x$height in ${parentWidth}x$parentHeight")
|
||||
}
|
||||
|
||||
private fun applyFullscreen() {
|
||||
window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
|
||||
val controller = window.insetsController ?: return
|
||||
controller.systemBarsBehavior = android.view.WindowInsetsController.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
|
||||
controller.hide(
|
||||
android.view.WindowInsets.Type.statusBars() or android.view.WindowInsets.Type.navigationBars(),
|
||||
)
|
||||
}
|
||||
|
||||
private fun localIpv4(): String {
|
||||
return try {
|
||||
val interfaces = NetworkInterface.getNetworkInterfaces() ?: return "unknown"
|
||||
for (iface in interfaces) {
|
||||
if (!iface.isUp || iface.isLoopback) continue
|
||||
for (address in iface.inetAddresses) {
|
||||
if (address is Inet4Address && !address.isLoopbackAddress) {
|
||||
return address.hostAddress ?: "unknown"
|
||||
}
|
||||
}
|
||||
}
|
||||
"unknown"
|
||||
} catch (e: Exception) {
|
||||
"unknown"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,179 @@
|
||||
package screen_cast.decode
|
||||
|
||||
import android.media.MediaCodec
|
||||
import android.media.MediaFormat
|
||||
import android.view.Surface
|
||||
|
||||
/**
|
||||
* MediaCodec H.264 decoder that renders directly onto a Surface (no CPU
|
||||
* pixels). The stream is self-describing: the sender repeats SPS/PPS
|
||||
* in-band at every keyframe, so no out-of-band codec data is needed.
|
||||
*/
|
||||
class H264Decoder {
|
||||
companion object {
|
||||
// The C++ sender's VBV bounds keyframes to ~2 frame periods of bytes;
|
||||
// 8 MiB is far beyond anything the negotiated rates can produce.
|
||||
private const val MAX_INPUT_SIZE = 8 * 1024 * 1024
|
||||
}
|
||||
|
||||
private var codec: MediaCodec? = null
|
||||
private var configured = false
|
||||
// True once the codec parsed the in-band SPS (INFO_OUTPUT_FORMAT_CHANGED).
|
||||
// Before that, outputFormat still carries the configure() placeholder and
|
||||
// must not drive layout — sizing the view to it squishes the first frame(s).
|
||||
@Volatile
|
||||
private var realFormatSeen = false
|
||||
@Volatile
|
||||
private var renderSurface: Surface? = null
|
||||
|
||||
/** Creates and configures the decoder. Width/height of 0 = unknown (the bitstream decides). */
|
||||
@Synchronized
|
||||
fun configure(width: Int, height: Int) {
|
||||
if (configured) return
|
||||
// 0x0 (the sender's offer) means the size is unknown: the Qualcomm
|
||||
// C2 AVC decoder requires a concrete size at configure() and
|
||||
// reconfigures from the in-band SPS of the first keyframe (the
|
||||
// standard adaptive-resolution pattern).
|
||||
val format = if (width > 0 && height > 0) {
|
||||
MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_AVC, width, height)
|
||||
} else {
|
||||
MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_AVC, 320, 240)
|
||||
}
|
||||
format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, MAX_INPUT_SIZE)
|
||||
// Assign before configuring so a configure()/start() failure cannot
|
||||
// orphan the created instance: MediaCodec has no finalizer and each
|
||||
// unreleased instance holds a scarce native codec slot. The caller
|
||||
// follows an exception with release(), which is a no-op on null.
|
||||
val created = MediaCodec.createDecoderByType(MediaFormat.MIMETYPE_VIDEO_AVC)
|
||||
codec = created
|
||||
try {
|
||||
// The surface must go into configure(): the codec must start in
|
||||
// surface mode. setOutputSurface() afterwards only switches an
|
||||
// already-surface-mode codec; a codec configured without a
|
||||
// surface can never take one.
|
||||
created.configure(format, renderSurface, null, 0)
|
||||
created.start()
|
||||
} catch (e: Exception) {
|
||||
codec = null
|
||||
try {
|
||||
created.release()
|
||||
} catch (ignored: Exception) {
|
||||
}
|
||||
throw e
|
||||
}
|
||||
configured = true
|
||||
realFormatSeen = false
|
||||
}
|
||||
|
||||
/** Points the decoder at a (possibly new) render surface. */
|
||||
@Synchronized
|
||||
fun attachSurface(surface: Surface) {
|
||||
renderSurface = surface
|
||||
// Legal only in surface mode: setOutputSurface() dynamically switches
|
||||
// an output surface on a codec configured WITH one (per the platform
|
||||
// docs); on a ByteBuffer-mode codec it throws IllegalStateException.
|
||||
codec?.setOutputSurface(surface)
|
||||
}
|
||||
|
||||
/**
|
||||
* Forgets a destroyed render surface: frames still decode (returning
|
||||
* their buffers and keeping the queue moving) but are not rendered
|
||||
* until the next [attachSurface]. API 35's detachOutputSurface() is the
|
||||
* codec-side equivalent.
|
||||
*/
|
||||
@Synchronized
|
||||
fun detachSurface() {
|
||||
renderSurface = null
|
||||
}
|
||||
|
||||
/**
|
||||
* Queues one access unit. [presentationUs] must be monotonic (the RTP timestamp).
|
||||
* @return false when the codec's input queue was full and the frame was
|
||||
* dropped; the caller should request a keyframe, not flush (the codec
|
||||
* is healthy, merely busy).
|
||||
*/
|
||||
fun feed(data: ByteArray, presentationUs: Long, isKeyFrame: Boolean): Boolean {
|
||||
val c = codec ?: throw IllegalStateException("decoder not configured")
|
||||
val index = c.dequeueInputBuffer(10_000)
|
||||
if (index < 0) return false
|
||||
val buffer = c.getInputBuffer(index) ?: return false
|
||||
buffer.clear()
|
||||
buffer.put(data)
|
||||
val flags = if (isKeyFrame) MediaCodec.BUFFER_FLAG_KEY_FRAME else 0
|
||||
c.queueInputBuffer(index, 0, data.size, presentationUs, flags)
|
||||
return true
|
||||
}
|
||||
|
||||
/**
|
||||
* Non-blocking drain of available output frames; in surface mode each
|
||||
* released buffer is rendered to the output surface by the codec.
|
||||
* @throws IllegalStateException when the decoder signals a fatal error;
|
||||
* the caller should flush() and request a keyframe.
|
||||
*/
|
||||
fun drain() {
|
||||
val c = codec ?: return
|
||||
val info = MediaCodec.BufferInfo()
|
||||
while (true) {
|
||||
val index = c.dequeueOutputBuffer(info, 0)
|
||||
when {
|
||||
index >= 0 -> {
|
||||
// releaseOutputBuffer(render=true) is what puts the
|
||||
// frame on the surface (and frees the pool slot).
|
||||
c.releaseOutputBuffer(index, renderSurface != null)
|
||||
}
|
||||
index == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED -> {
|
||||
// The codec parsed the SPS size; outputFormat is ready.
|
||||
realFormatSeen = true
|
||||
android.util.Log.i(
|
||||
"H264Decoder", "output format: " +
|
||||
c.outputFormat.getInteger(MediaFormat.KEY_WIDTH) +
|
||||
"x" +
|
||||
c.outputFormat.getInteger(MediaFormat.KEY_HEIGHT)
|
||||
)
|
||||
}
|
||||
index == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED -> Unit
|
||||
else -> break // no more frames right now
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Resets decoder state; the next keyframe re-primes it (in-band SPS/PPS). */
|
||||
fun flush() {
|
||||
val c = codec ?: return
|
||||
try {
|
||||
c.flush()
|
||||
} catch (e: Exception) {
|
||||
// A codec in an error state may refuse the flush; the caller
|
||||
// follows up with a keyframe request either way.
|
||||
}
|
||||
}
|
||||
|
||||
/** The decoded resolution, once the first keyframe configured the codec. */
|
||||
@Synchronized
|
||||
fun outputSize(): Pair<Int, Int>? {
|
||||
val c = codec ?: return null
|
||||
if (!realFormatSeen) return null // still the configure() placeholder
|
||||
return try {
|
||||
val format = c.outputFormat
|
||||
val width = format.getInteger(MediaFormat.KEY_WIDTH)
|
||||
val height = format.getInteger(MediaFormat.KEY_HEIGHT)
|
||||
if (width > 0 && height > 0) width to height else null
|
||||
} catch (e: Exception) {
|
||||
null
|
||||
}
|
||||
}
|
||||
|
||||
@Synchronized
|
||||
fun release() {
|
||||
val c = codec ?: return
|
||||
try {
|
||||
c.stop()
|
||||
} catch (e: Exception) {
|
||||
// ignore
|
||||
}
|
||||
c.release()
|
||||
codec = null
|
||||
configured = false
|
||||
renderSurface = null
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,383 @@
|
||||
package screen_cast.pipeline
|
||||
|
||||
import android.content.Context
|
||||
import android.net.nsd.NsdManager
|
||||
import android.net.nsd.NsdServiceInfo
|
||||
import android.view.Surface
|
||||
import screen_cast.decode.H264Decoder
|
||||
import screen_cast.rtp.H264Depacketizer
|
||||
import screen_cast.rtp.JitterBuffer
|
||||
import screen_cast.rtp.RtpPacket
|
||||
import screen_cast.signaling.SessionAnswer
|
||||
import screen_cast.signaling.SessionOffer
|
||||
import screen_cast.signaling.SessionPli
|
||||
import screen_cast.signaling.SignalingServer
|
||||
import java.net.DatagramPacket
|
||||
import java.net.DatagramSocket
|
||||
import java.net.InetSocketAddress
|
||||
import java.net.SocketException
|
||||
import java.util.concurrent.atomic.AtomicBoolean
|
||||
|
||||
/**
|
||||
* The receiver pipeline, mirroring the C++ ReceiverPipeline:
|
||||
*
|
||||
* NSD advertise + signaling server (offer → answer)
|
||||
* UDP RTP → jitter buffer → depacketize → MediaCodec → Surface
|
||||
*
|
||||
* Recovery matches the C++ receiver: a damaged frame is dropped and a PLI
|
||||
* (rate-limited to one per 500 ms) asks the sender for a keyframe.
|
||||
*/
|
||||
class ReceiverPipeline(
|
||||
private val context: Context,
|
||||
private val localIp: String,
|
||||
private val displaySize: () -> Pair<Int, Int>,
|
||||
private val onStatus: (String) -> Unit,
|
||||
private val onFirstFrame: () -> Unit,
|
||||
private val onVideoSize: (Int, Int) -> Unit,
|
||||
) {
|
||||
companion object {
|
||||
private const val TAG = "ReceiverPipeline"
|
||||
private const val SERVICE_NAME = "screencast"
|
||||
private const val SERVICE_TYPE = "_screencast._tcp"
|
||||
private const val DESIRED_UDP_PORT = 5004
|
||||
private const val DESIRED_SIGNALING_PORT = 5005
|
||||
private const val PLI_MIN_INTERVAL_MS = 500L
|
||||
}
|
||||
|
||||
private val nsdManager: NsdManager = context.getSystemService(Context.NSD_SERVICE) as NsdManager
|
||||
private val decoderLock = Any()
|
||||
|
||||
@Volatile private var running = false
|
||||
private var udpSocket: DatagramSocket? = null
|
||||
private var udpPort = 0
|
||||
private var signalingPort = 0
|
||||
private var signaling: SignalingServer? = null
|
||||
private var readerThread: Thread? = null
|
||||
@Volatile private var registrationListener: NsdManager.RegistrationListener? = null
|
||||
|
||||
// Guarded by decoderLock: currentSurface, pendingOffer, decoder.
|
||||
@Volatile private var decoder: H264Decoder? = null
|
||||
@Volatile private var depacketizer = H264Depacketizer()
|
||||
private val jitter = JitterBuffer()
|
||||
private var currentSurface: Surface? = null
|
||||
private var pendingOffer: SessionOffer? = null
|
||||
@Volatile private var activeSession = ""
|
||||
private val firstFrameSeen = AtomicBoolean(false)
|
||||
private val pliLock = Any()
|
||||
private var lastPliAtMs = 0L
|
||||
@Volatile private var videoWidth = 0
|
||||
@Volatile private var videoHeight = 0
|
||||
|
||||
/** The current decoded resolution (0, 0 until the first keyframe). */
|
||||
fun videoSize(): Pair<Int, Int> = videoWidth to videoHeight
|
||||
|
||||
/** Binds the ports, advertises the service, and starts reading RTP. */
|
||||
@Synchronized
|
||||
fun start() {
|
||||
if (running) return
|
||||
|
||||
try {
|
||||
// Plain DatagramSocket: consistent semantics across platforms
|
||||
// (Android's DatagramChannel.receive() returns a SocketAddress,
|
||||
// not the byte count). Port 5004 when free; otherwise ephemeral.
|
||||
val socket = try {
|
||||
DatagramSocket(InetSocketAddress(DESIRED_UDP_PORT))
|
||||
} catch (e: Exception) {
|
||||
DatagramSocket()
|
||||
}
|
||||
udpPort = socket.localPort
|
||||
udpSocket = socket
|
||||
} catch (e: Exception) {
|
||||
onStatus("Failed to bind the media port: ${e.message}")
|
||||
return
|
||||
}
|
||||
|
||||
val server = SignalingServer(
|
||||
onOffer = { offer -> onOffer(offer) },
|
||||
onPli = { /* the receiver never receives PLIs */ },
|
||||
)
|
||||
try {
|
||||
signalingPort = server.start(DESIRED_SIGNALING_PORT)
|
||||
signaling = server
|
||||
} catch (e: Exception) {
|
||||
onStatus("Failed to start signaling: ${e.message}")
|
||||
try {
|
||||
udpSocket?.close()
|
||||
} catch (ignored: Exception) {
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
running = true
|
||||
readerThread = Thread({ readLoop() }, "rtp-reader").also { it.start() }
|
||||
// Advertise AFTER the listening status: a registration failure must
|
||||
// not be overwritten by it (both post to the same overlay).
|
||||
onStatus(
|
||||
"Listening on $localIp (media :$udpPort, signaling :$signalingPort)\n" +
|
||||
"Waiting for a sender… (fall back to: screencast --send --peer $localIp:$signalingPort)",
|
||||
)
|
||||
advertiseNsd(signalingPort)
|
||||
}
|
||||
|
||||
/**
|
||||
* Points the decoder (current or future) at a render surface. A pending
|
||||
* offer (accepted while no surface existed) configures its decoder now.
|
||||
*/
|
||||
fun attachSurface(surface: Surface) {
|
||||
var configuredFromPending = false
|
||||
synchronized(decoderLock) {
|
||||
currentSurface = surface
|
||||
decoder?.attachSurface(surface)
|
||||
val offer = pendingOffer
|
||||
if (offer != null) {
|
||||
val created = configureDecoderLocked(offer.width, offer.height)
|
||||
if (created != null) {
|
||||
pendingOffer = null
|
||||
decoder = created
|
||||
configuredFromPending = true
|
||||
} else {
|
||||
// Keep the offer: the next attachSurface retries the
|
||||
// configure instead of abandoning the session.
|
||||
onStatus("Could not start the decoder")
|
||||
}
|
||||
}
|
||||
}
|
||||
if (configuredFromPending) {
|
||||
// The sender is already streaming P-frames; the late-configured
|
||||
// decoder needs a keyframe (SPS/PPS + IDR) to start producing
|
||||
// output — the sender only emits one when asked.
|
||||
requestPli()
|
||||
}
|
||||
}
|
||||
|
||||
/** Forgets a destroyed render surface so a later offer cannot configure against it. */
|
||||
fun detachSurface() {
|
||||
synchronized(decoderLock) {
|
||||
currentSurface = null
|
||||
decoder?.detachSurface()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates and configures a decoder for the stream size; caller holds
|
||||
* [decoderLock]. Returns null on failure with the codec released.
|
||||
*/
|
||||
private fun configureDecoderLocked(width: Int, height: Int): H264Decoder? {
|
||||
val fresh = H264Decoder()
|
||||
try {
|
||||
// Surface first: configure() needs it before the codec starts.
|
||||
currentSurface?.let { fresh.attachSurface(it) }
|
||||
fresh.configure(width, height)
|
||||
} catch (e: Exception) {
|
||||
fresh.release()
|
||||
return null
|
||||
}
|
||||
return fresh
|
||||
}
|
||||
|
||||
/** Stops listening; the pipeline can be started again. */
|
||||
@Synchronized
|
||||
fun stop() {
|
||||
if (!running) return
|
||||
running = false
|
||||
activeSession = ""
|
||||
try {
|
||||
udpSocket?.close() // unblocks the reader's receive()
|
||||
} catch (ignored: Exception) {
|
||||
}
|
||||
udpSocket = null
|
||||
readerThread?.join(1000)
|
||||
readerThread = null
|
||||
signaling?.close()
|
||||
signaling = null
|
||||
unregisterNsd()
|
||||
synchronized(decoderLock) {
|
||||
decoder?.release()
|
||||
decoder = null
|
||||
pendingOffer = null
|
||||
}
|
||||
firstFrameSeen.set(false)
|
||||
onStatus("Stopped")
|
||||
}
|
||||
|
||||
private fun onOffer(offer: SessionOffer) {
|
||||
if (offer.codec != "h264") {
|
||||
onStatus("Unsupported codec: ${offer.codec}")
|
||||
return
|
||||
}
|
||||
// The offer's width/height are informational (the C++ sender leaves
|
||||
// them 0); the bitstream carries SPS/PPS at every keyframe.
|
||||
var decoderReady = true
|
||||
synchronized(decoderLock) {
|
||||
pendingOffer = null
|
||||
decoder?.release()
|
||||
decoder = null
|
||||
if (currentSurface != null) {
|
||||
decoder = configureDecoderLocked(offer.width, offer.height)
|
||||
decoderReady = decoder != null
|
||||
} else {
|
||||
// No surface yet (the window is between surfaces): remember
|
||||
// the offer; attachSurface configures the decoder. Configuring
|
||||
// without a surface is not an option — that codec could never
|
||||
// take one afterwards (setOutputSurface refuses it).
|
||||
pendingOffer = offer
|
||||
}
|
||||
}
|
||||
if (!decoderReady) {
|
||||
onStatus("Could not start the decoder")
|
||||
return
|
||||
}
|
||||
// New session: pristine reassembly state.
|
||||
depacketizer = H264Depacketizer()
|
||||
jitter.clear()
|
||||
firstFrameSeen.set(false)
|
||||
videoWidth = 0
|
||||
videoHeight = 0
|
||||
activeSession = offer.sessionId
|
||||
android.util.Log.i(TAG, "offer: session=${offer.sessionId} ${offer.width}x${offer.height} @${offer.frameRateNum}/${offer.frameRateDen}")
|
||||
|
||||
val (displayWidth, displayHeight) = displaySize()
|
||||
signaling?.send(
|
||||
SessionAnswer(
|
||||
sessionId = offer.sessionId,
|
||||
rtpAddress = "", // the sender targets the address of its own signaling connection
|
||||
rtpPort = udpPort,
|
||||
displayWidth = displayWidth,
|
||||
displayHeight = displayHeight,
|
||||
),
|
||||
)
|
||||
onStatus(
|
||||
if (decoder != null) "Session ${offer.sessionId} negotiated — waiting for the first frame…"
|
||||
else "Session ${offer.sessionId} negotiated — waiting for the display surface…"
|
||||
)
|
||||
}
|
||||
|
||||
private fun readLoop() {
|
||||
val socket = udpSocket ?: return
|
||||
val buffer = ByteArray(2048) // MTU 1200 + headroom
|
||||
val datagram = DatagramPacket(buffer, buffer.size)
|
||||
while (running) {
|
||||
try {
|
||||
socket.receive(datagram)
|
||||
} catch (e: SocketException) {
|
||||
break // socket closed
|
||||
} catch (e: Exception) {
|
||||
continue
|
||||
}
|
||||
val received = datagram.length
|
||||
if (received <= 0) continue
|
||||
|
||||
val bytes = if (received == buffer.size) buffer.copyOf() else buffer.copyOf(received)
|
||||
val packet = RtpPacket.parse(bytes) ?: continue
|
||||
for (released in jitter.push(packet)) {
|
||||
handleDepacketized(released)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun handleDepacketized(packet: RtpPacket) {
|
||||
val result = depacketizer.depacketize(packet)
|
||||
|
||||
val accessUnit = result.accessUnit
|
||||
if (accessUnit != null) {
|
||||
val presentationUs = packet.header.timestamp.toLong() and 0xFFFFFFFFL
|
||||
try {
|
||||
// MediaCodec is not thread-safe: attachSurface() (main
|
||||
// thread) and onOffer (signaling thread) synchronize on the
|
||||
// same lock around their codec calls.
|
||||
synchronized(decoderLock) {
|
||||
val activeDecoder = decoder ?: return
|
||||
if (!activeDecoder.feed(accessUnit, presentationUs, result.isKeyFrame)) {
|
||||
// Input queue full: the dropped frame corrupts the
|
||||
// GOP until the next keyframe — ask for one. No
|
||||
// flush; the codec is healthy, merely busy.
|
||||
requestPli()
|
||||
}
|
||||
activeDecoder.drain()
|
||||
if (firstFrameSeen.compareAndSet(false, true)) {
|
||||
onFirstFrame()
|
||||
}
|
||||
updateVideoSize()
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
onStatus("Decoder error (${e.message}) — requesting a keyframe…")
|
||||
synchronized(decoderLock) { decoder?.flush() }
|
||||
requestPli()
|
||||
}
|
||||
}
|
||||
if (result.frameDropped) {
|
||||
requestPli()
|
||||
}
|
||||
}
|
||||
|
||||
private fun updateVideoSize() {
|
||||
val size = synchronized(decoderLock) { decoder?.outputSize() } ?: return
|
||||
if (size.first != videoWidth || size.second != videoHeight) {
|
||||
videoWidth = size.first
|
||||
videoHeight = size.second
|
||||
android.util.Log.i(TAG, "video size: ${size.first}x${size.second}")
|
||||
onVideoSize(size.first, size.second)
|
||||
}
|
||||
}
|
||||
|
||||
// Rate-limited keyframe request, callable from any thread (the reader
|
||||
// thread and the main thread's attachSurface both reach it).
|
||||
private fun requestPli() {
|
||||
val session = activeSession
|
||||
if (session.isEmpty()) return
|
||||
val now = System.currentTimeMillis()
|
||||
synchronized(pliLock) {
|
||||
if (now - lastPliAtMs < PLI_MIN_INTERVAL_MS) return
|
||||
lastPliAtMs = now
|
||||
}
|
||||
android.util.Log.i(TAG, "PLI requested for session $session")
|
||||
signaling?.send(SessionPli(session))
|
||||
}
|
||||
|
||||
private fun advertiseNsd(port: Int) {
|
||||
val info = NsdServiceInfo().apply {
|
||||
setServiceName(SERVICE_NAME)
|
||||
setServiceType(SERVICE_TYPE)
|
||||
setPort(port)
|
||||
}
|
||||
// The classic RegistrationListener API exists from API 16 through 36
|
||||
// (verified against the android-36 SDK with javap), so no fallback is
|
||||
// needed — the previously reflected "ResolutionListener" never
|
||||
// existed at any API level and could only ever fail.
|
||||
val listener = object : NsdManager.RegistrationListener {
|
||||
override fun onServiceRegistered(serviceInfo: NsdServiceInfo) {
|
||||
// Registered: the sender's mDNS browser should see it now.
|
||||
android.util.Log.i(TAG, "mDNS registered: ${serviceInfo.serviceName}.${SERVICE_TYPE}")
|
||||
}
|
||||
|
||||
override fun onServiceUnregistered(serviceInfo: NsdServiceInfo) = Unit
|
||||
|
||||
override fun onRegistrationFailed(serviceInfo: NsdServiceInfo, errorCode: Int) {
|
||||
onStatus("mDNS registration failed — reach this receiver with --peer $localIp:$port")
|
||||
}
|
||||
|
||||
override fun onUnregistrationFailed(serviceInfo: NsdServiceInfo, errorCode: Int) = Unit
|
||||
}
|
||||
try {
|
||||
// PROTOCOL_DNS_SD is mandatory on API 36: NsdManager.checkProtocol()
|
||||
// rejects anything else (the historical 0 threw
|
||||
// "IllegalArgumentException: Unsupported protocol", which the
|
||||
// old code swallowed — mDNS never advertised on this phone).
|
||||
nsdManager.registerService(info, NsdManager.PROTOCOL_DNS_SD, listener)
|
||||
registrationListener = listener
|
||||
} catch (e: Exception) {
|
||||
android.util.Log.e(TAG, "mDNS registration failed", e)
|
||||
onStatus("mDNS unavailable (${e.message}) — reach this receiver with --peer $localIp:$port")
|
||||
}
|
||||
}
|
||||
|
||||
private fun unregisterNsd() {
|
||||
val listener = registrationListener ?: return
|
||||
try {
|
||||
nsdManager.unregisterService(listener)
|
||||
} catch (ignored: Exception) {
|
||||
// already unregistered
|
||||
}
|
||||
registrationListener = null
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,179 @@
|
||||
package screen_cast.rtp
|
||||
|
||||
import java.io.ByteArrayOutputStream
|
||||
|
||||
/** Result of feeding one packet to the depacketizer. */
|
||||
data class DepacketizeResult(
|
||||
/** Completed access unit (Annex-B with 3-byte start codes) when the frame closed undamaged. */
|
||||
val accessUnit: ByteArray? = null,
|
||||
/** True when this call discarded a frame as damaged (packet loss or unsupported packetization). */
|
||||
val frameDropped: Boolean = false,
|
||||
/** True when the completed access unit carries SPS/PPS (a keyframe). */
|
||||
val isKeyFrame: Boolean = false,
|
||||
)
|
||||
|
||||
/**
|
||||
* Reassembles RFC 6184 packet streams (single NAL unit packets and FU-A)
|
||||
* into Annex-B access units. Packets must arrive in order; frames damaged
|
||||
* by sequence gaps or missing fragments are reported via DepacketizeResult.
|
||||
*
|
||||
* Mirrors the C++ `H264Depacketizer` (same state machine and start codes).
|
||||
*/
|
||||
class H264Depacketizer {
|
||||
companion object {
|
||||
private const val FU_A = 28
|
||||
}
|
||||
|
||||
private var lastSequenceNumber: Int? = null
|
||||
private var frameStarted = false
|
||||
private var frameDamaged = false
|
||||
private var frameTimestamp = 0
|
||||
// Growable byte accumulators: keyframes reach hundreds of KB, and boxing
|
||||
// each byte (the ArrayList<Int> this replaced) churned the GC hard.
|
||||
private val accessUnit = ByteArrayOutputStream()
|
||||
private var fuActive = false
|
||||
private val fuNal = ByteArrayOutputStream()
|
||||
|
||||
/** Feed one packet (in sequence order, from the jitter buffer). */
|
||||
fun depacketize(packet: RtpPacket): DepacketizeResult {
|
||||
var result = DepacketizeResult()
|
||||
|
||||
// Track sequence continuity: a gap means packets were lost.
|
||||
lastSequenceNumber?.let { last ->
|
||||
val expected = (last + 1) and 0xFFFF
|
||||
if (packet.header.sequenceNumber != expected) {
|
||||
fuActive = false
|
||||
fuNal.reset()
|
||||
if (frameStarted) {
|
||||
frameDamaged = true
|
||||
}
|
||||
}
|
||||
}
|
||||
lastSequenceNumber = packet.header.sequenceNumber
|
||||
|
||||
// A timestamp change without a closing marker means the previous frame
|
||||
// lost its tail and can no longer be recovered.
|
||||
if (frameStarted && packet.header.timestamp != frameTimestamp) {
|
||||
dropFrame()
|
||||
result = result.copy(frameDropped = true)
|
||||
}
|
||||
if (!frameStarted) {
|
||||
frameStarted = true
|
||||
frameDamaged = false
|
||||
frameTimestamp = packet.header.timestamp
|
||||
accessUnit.reset()
|
||||
}
|
||||
|
||||
val payload = packet.payload
|
||||
if (payload.isNotEmpty()) {
|
||||
val type = payload[0].toInt() and 0x1F
|
||||
when {
|
||||
type in 1..23 -> {
|
||||
// Single NAL unit packet.
|
||||
if (fuActive) {
|
||||
// The previous fragmented NAL never received its end packet.
|
||||
frameDamaged = true
|
||||
fuActive = false
|
||||
fuNal.reset()
|
||||
}
|
||||
appendStartCode()
|
||||
accessUnit.write(payload)
|
||||
}
|
||||
|
||||
type == FU_A -> {
|
||||
if (payload.size < 2) {
|
||||
frameDamaged = true
|
||||
} else {
|
||||
val fuHeader = payload[1].toInt() and 0xFF
|
||||
val start = fuHeader and 0x80 != 0
|
||||
val end = fuHeader and 0x40 != 0
|
||||
val fragment = payload.copyOfRange(2, payload.size)
|
||||
when {
|
||||
start -> {
|
||||
if (fuActive) {
|
||||
// The previous fragmented NAL lost its end packet.
|
||||
frameDamaged = true
|
||||
}
|
||||
fuActive = true
|
||||
fuNal.reset()
|
||||
// The FU indicator keeps the original NAL's F bit (0) and NRI,
|
||||
// and declares type 28; the FU header carries S/E plus the real type.
|
||||
fuNal.write((payload[0].toInt() and 0xE0) or (fuHeader and 0x1F))
|
||||
fuNal.write(fragment)
|
||||
}
|
||||
|
||||
!fuActive -> {
|
||||
// Continuation without a start: the head of the NAL is lost.
|
||||
frameDamaged = true
|
||||
}
|
||||
|
||||
else -> {
|
||||
fuNal.write(fragment)
|
||||
if (end) {
|
||||
appendStartCode()
|
||||
fuNal.writeTo(accessUnit)
|
||||
fuActive = false
|
||||
fuNal.reset()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
else -> {
|
||||
// Unsupported packetization mode (STAP-A, MTAP, FU-B): the frame
|
||||
// cannot be reconstructed.
|
||||
frameDamaged = true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!packet.header.marker) {
|
||||
return result
|
||||
}
|
||||
|
||||
if (fuActive) {
|
||||
// The marker arrived while a NAL was still fragmented.
|
||||
frameDamaged = true
|
||||
fuActive = false
|
||||
fuNal.reset()
|
||||
}
|
||||
|
||||
if (!frameDamaged && accessUnit.size() > 0) {
|
||||
val unit = accessUnit.toByteArray()
|
||||
result = result.copy(accessUnit = unit, isKeyFrame = containsParameterSets(unit))
|
||||
} else {
|
||||
// The frame that just ended is unusable.
|
||||
result = result.copy(frameDropped = true)
|
||||
}
|
||||
dropFrame()
|
||||
return result
|
||||
}
|
||||
|
||||
private fun appendStartCode() {
|
||||
accessUnit.write(0)
|
||||
accessUnit.write(0)
|
||||
accessUnit.write(1)
|
||||
}
|
||||
|
||||
/** The sender repeats SPS/PPS in-band at every keyframe; sniff for NAL types 7/8. */
|
||||
private fun containsParameterSets(unit: ByteArray): Boolean {
|
||||
for (i in 0..unit.size - 4) {
|
||||
if (unit[i] == 0.toByte() && unit[i + 1] == 0.toByte() && unit[i + 2] == 1.toByte()) {
|
||||
val nalType = unit[i + 3].toInt() and 0x1F
|
||||
if (nalType == 7 || nalType == 8) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
private fun dropFrame() {
|
||||
frameStarted = false
|
||||
frameDamaged = false
|
||||
accessUnit.reset()
|
||||
fuActive = false
|
||||
fuNal.reset()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
package screen_cast.rtp
|
||||
|
||||
import java.util.TreeMap
|
||||
import java.util.concurrent.TimeUnit
|
||||
|
||||
/**
|
||||
* Reorders RTP packets by sequence number before depacketization so that a
|
||||
* reordering link (Wi-Fi) is not read as loss. Delivery stays in order;
|
||||
* only aged-out or overflowing buffers release out of order, which the
|
||||
* downstream gap detection still handles for genuine loss.
|
||||
*
|
||||
* Mirrors the C++ `RtpJitterBuffer` (same defaults and semantics).
|
||||
*/
|
||||
class JitterBuffer(
|
||||
private val maxDepth: Int = 16,
|
||||
private val maxDelayMs: Long = 60,
|
||||
) {
|
||||
private class BufferEntry(val timeNanos: Long, val packet: RtpPacket)
|
||||
|
||||
private val lock = Any()
|
||||
private val buffer = TreeMap<Int, BufferEntry>()
|
||||
private var nextExpected: Int? = null
|
||||
|
||||
/** Insert one packet and return the packets now ready for in-order delivery. */
|
||||
fun push(packet: RtpPacket): List<RtpPacket> = synchronized(lock) {
|
||||
val released = ArrayList<RtpPacket>()
|
||||
val sequence = packet.header.sequenceNumber
|
||||
val now = System.nanoTime()
|
||||
|
||||
val expected0 = nextExpected ?: sequence.also { nextExpected = it }
|
||||
|
||||
// Serial-number comparison: a distance >= 32768 means the packet is
|
||||
// older than what we already delivered (duplicate or straggler).
|
||||
val distance = (sequence - expected0 + 65536) % 65536
|
||||
if (distance < 32768) {
|
||||
buffer[sequence] = BufferEntry(now, packet)
|
||||
|
||||
// Release the consecutive run from the expected sequence.
|
||||
var expected = expected0
|
||||
while (true) {
|
||||
val entry = buffer[expected] ?: break
|
||||
released.add(entry.packet)
|
||||
buffer.remove(expected)
|
||||
expected = (expected + 1) and 0xFFFF
|
||||
}
|
||||
nextExpected = expected
|
||||
|
||||
// A missing packet stalls the run: age out the backlog (or bound
|
||||
// the buffer) and release what is there in order, so genuine loss
|
||||
// reaches the depacketizer's gap detection rather than blocking.
|
||||
if (buffer.isNotEmpty()) {
|
||||
val head = buffer.firstEntry().value
|
||||
val headAgeMs = TimeUnit.NANOSECONDS.toMillis(now - head.timeNanos)
|
||||
if (headAgeMs > maxDelayMs || buffer.size > maxDepth) {
|
||||
released.addAll(buffer.values.map { it.packet })
|
||||
nextExpected = (buffer.lastKey() + 1) and 0xFFFF
|
||||
buffer.clear()
|
||||
}
|
||||
}
|
||||
}
|
||||
released
|
||||
}
|
||||
|
||||
/** Discard everything still buffered. */
|
||||
fun clear() {
|
||||
synchronized(lock) {
|
||||
buffer.clear()
|
||||
nextExpected = null
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
package screen_cast.rtp
|
||||
|
||||
/** Minimal RTP header (RFC 3550) without extensions. */
|
||||
data class RtpHeader(
|
||||
val version: Int = 2,
|
||||
val padding: Boolean = false,
|
||||
val extension: Boolean = false,
|
||||
val csrcCount: Int = 0,
|
||||
val marker: Boolean = false,
|
||||
val payloadType: Int = 96,
|
||||
val sequenceNumber: Int = 0,
|
||||
val timestamp: Int = 0,
|
||||
val ssrc: Int = 0,
|
||||
) {
|
||||
/** Serializes the bare 12-byte header; requires a version-2, extension-less header. */
|
||||
fun serialize(): ByteArray {
|
||||
val out = ByteArray(12)
|
||||
out[0] = (((version and 0x0F) shl 6) or (if (padding) 0x20 else 0) or (if (extension) 0x10 else 0) or (csrcCount and 0x0F)).toByte()
|
||||
out[1] = ((if (marker) 0x80 else 0) or (payloadType and 0x7F)).toByte()
|
||||
out[2] = (sequenceNumber ushr 8).toByte()
|
||||
out[3] = (sequenceNumber and 0xFF).toByte()
|
||||
out[4] = (timestamp ushr 24).toByte()
|
||||
out[5] = (timestamp ushr 16).toByte()
|
||||
out[6] = (timestamp ushr 8).toByte()
|
||||
out[7] = (timestamp and 0xFF).toByte()
|
||||
out[8] = (ssrc ushr 24).toByte()
|
||||
out[9] = (ssrc ushr 16).toByte()
|
||||
out[10] = (ssrc ushr 8).toByte()
|
||||
out[11] = (ssrc and 0xFF).toByte()
|
||||
return out
|
||||
}
|
||||
|
||||
companion object {
|
||||
fun parse(input: ByteArray): RtpHeader? {
|
||||
if (input.size < 12) return null
|
||||
val b0 = input[0].toInt() and 0xFF
|
||||
val b1 = input[1].toInt() and 0xFF
|
||||
val version = b0 ushr 6
|
||||
if (version != 2) return null
|
||||
return RtpHeader(
|
||||
version = version,
|
||||
padding = b0 and 0x20 != 0,
|
||||
extension = b0 and 0x10 != 0,
|
||||
csrcCount = b0 and 0x0F,
|
||||
marker = b1 and 0x80 != 0,
|
||||
payloadType = b1 and 0x7F,
|
||||
sequenceNumber = ((input[2].toInt() and 0xFF) shl 8) or (input[3].toInt() and 0xFF),
|
||||
timestamp = ((input[4].toInt() and 0xFF) shl 24) or ((input[5].toInt() and 0xFF) shl 16) or
|
||||
((input[6].toInt() and 0xFF) shl 8) or (input[7].toInt() and 0xFF),
|
||||
ssrc = ((input[8].toInt() and 0xFF) shl 24) or ((input[9].toInt() and 0xFF) shl 16) or
|
||||
((input[10].toInt() and 0xFF) shl 8) or (input[11].toInt() and 0xFF),
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
package screen_cast.rtp
|
||||
|
||||
/** An RTP packet: 12-byte base header (plus optional CSRC/extension) and payload. */
|
||||
class RtpPacket(val header: RtpHeader, val payload: ByteArray) {
|
||||
companion object {
|
||||
/**
|
||||
* Parses a full RTP datagram. Honors CSRC lists, one-level extension
|
||||
* headers, and RFC 3550 padding, mirroring the C++ receiver.
|
||||
*/
|
||||
fun parse(input: ByteArray): RtpPacket? {
|
||||
if (input.size < 12) return null
|
||||
val header = RtpHeader.parse(input) ?: return null
|
||||
|
||||
var offset = 12 + header.csrcCount * 4
|
||||
if (input.size < offset) return null
|
||||
|
||||
if (header.extension) {
|
||||
if (input.size < offset + 4) return null
|
||||
val extensionWords =
|
||||
((input[offset + 2].toInt() and 0xFF) shl 8) or (input[offset + 3].toInt() and 0xFF)
|
||||
offset += 4 + extensionWords * 4
|
||||
if (input.size < offset) return null
|
||||
}
|
||||
|
||||
var payloadSize = input.size - offset
|
||||
if (header.padding) {
|
||||
// RFC 3550: the last byte holds the padding size, including itself.
|
||||
if (payloadSize == 0) return null
|
||||
val paddingSize = input[input.size - 1].toInt() and 0xFF
|
||||
if (paddingSize == 0 || paddingSize > payloadSize) return null
|
||||
payloadSize -= paddingSize
|
||||
}
|
||||
return RtpPacket(header, input.copyOfRange(offset, offset + payloadSize))
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
package screen_cast.signaling
|
||||
|
||||
import org.json.JSONObject
|
||||
|
||||
// JSON wire format shared with the C++ implementation: one JSON object per
|
||||
// newline-terminated TCP line (offer / answer / pli).
|
||||
|
||||
data class SessionOffer(
|
||||
val sessionId: String,
|
||||
val codec: String,
|
||||
val width: Int,
|
||||
val height: Int,
|
||||
val frameRateNum: Int,
|
||||
val frameRateDen: Int,
|
||||
val rtpAddress: String,
|
||||
val rtpPort: Int,
|
||||
) : SignalingMessage
|
||||
|
||||
data class SessionAnswer(
|
||||
val sessionId: String,
|
||||
val rtpAddress: String,
|
||||
val rtpPort: Int,
|
||||
val displayWidth: Int,
|
||||
val displayHeight: Int,
|
||||
) : SignalingMessage
|
||||
|
||||
data class SessionPli(val sessionId: String) : SignalingMessage
|
||||
|
||||
sealed interface SignalingMessage {
|
||||
companion object {
|
||||
private const val MAX_MESSAGE_BYTES = 64 * 1024
|
||||
|
||||
fun parse(line: String): SignalingMessage? {
|
||||
if (line.length > MAX_MESSAGE_BYTES) return null
|
||||
val json = try {
|
||||
JSONObject(line)
|
||||
} catch (e: Exception) {
|
||||
return null
|
||||
}
|
||||
when (val type = json.optString("type")) {
|
||||
"offer" ->
|
||||
return SessionOffer(
|
||||
sessionId = json.optString("session_id"),
|
||||
codec = json.optString("codec"),
|
||||
width = json.optInt("width"),
|
||||
height = json.optInt("height"),
|
||||
frameRateNum = json.optInt("frame_rate_num", 30),
|
||||
frameRateDen = json.optInt("frame_rate_den", 1),
|
||||
rtpAddress = json.optString("rtp_address"),
|
||||
rtpPort = json.optInt("rtp_port"),
|
||||
)
|
||||
|
||||
"answer" ->
|
||||
return SessionAnswer(
|
||||
sessionId = json.optString("session_id"),
|
||||
rtpAddress = json.optString("rtp_address"),
|
||||
rtpPort = json.optInt("rtp_port"),
|
||||
displayWidth = json.optInt("display_width"),
|
||||
displayHeight = json.optInt("display_height"),
|
||||
)
|
||||
|
||||
"pli" -> return SessionPli(sessionId = json.optString("session_id"))
|
||||
|
||||
else -> return null
|
||||
}
|
||||
}
|
||||
|
||||
fun serialize(message: SignalingMessage): String {
|
||||
val json = JSONObject()
|
||||
when (message) {
|
||||
is SessionOffer -> {
|
||||
json.put("type", "offer")
|
||||
json.put("session_id", message.sessionId)
|
||||
json.put("codec", message.codec)
|
||||
json.put("width", message.width)
|
||||
json.put("height", message.height)
|
||||
json.put("frame_rate_num", message.frameRateNum)
|
||||
json.put("frame_rate_den", message.frameRateDen)
|
||||
json.put("rtp_address", message.rtpAddress)
|
||||
json.put("rtp_port", message.rtpPort)
|
||||
}
|
||||
|
||||
is SessionAnswer -> {
|
||||
json.put("type", "answer")
|
||||
json.put("session_id", message.sessionId)
|
||||
json.put("rtp_address", message.rtpAddress)
|
||||
json.put("rtp_port", message.rtpPort)
|
||||
json.put("display_width", message.displayWidth)
|
||||
json.put("display_height", message.displayHeight)
|
||||
}
|
||||
|
||||
is SessionPli -> {
|
||||
json.put("type", "pli")
|
||||
json.put("session_id", message.sessionId)
|
||||
}
|
||||
}
|
||||
return json.toString() + "\n"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,144 @@
|
||||
package screen_cast.signaling
|
||||
|
||||
import java.io.BufferedInputStream
|
||||
import java.io.ByteArrayOutputStream
|
||||
import java.io.OutputStream
|
||||
import java.net.InetSocketAddress
|
||||
import java.net.ServerSocket
|
||||
import java.net.Socket
|
||||
|
||||
/**
|
||||
* Newline-delimited JSON signaling server (the receiver side). Keeps the
|
||||
* most recent connection as its active peer, mirroring the C++ server:
|
||||
* `onOffer` may answer synchronously (the sender blocks on the answer).
|
||||
*/
|
||||
class SignalingServer(
|
||||
private val onOffer: (SessionOffer) -> Unit,
|
||||
private val onPli: (SessionPli) -> Unit,
|
||||
) {
|
||||
private val server = ServerSocket()
|
||||
private val peerLock = Any()
|
||||
private var peerOut: OutputStream? = null
|
||||
private var acceptThread: Thread? = null
|
||||
private var readerThread: Thread? = null
|
||||
@Volatile
|
||||
private var running = false
|
||||
|
||||
/** Binds the port (SO_REUSEADDR) and starts accepting. Returns the bound port. */
|
||||
fun start(port: Int): Int {
|
||||
server.reuseAddress = true
|
||||
server.bind(InetSocketAddress(port))
|
||||
running = true
|
||||
acceptThread = Thread({ acceptLoop() }, "signaling-accept")
|
||||
acceptThread!!.start()
|
||||
return server.localPort
|
||||
}
|
||||
|
||||
/**
|
||||
* Sends to the current peer; never throws. A lost control message (a
|
||||
* PLI, an answer) is recoverable — the session re-negotiates or the next
|
||||
* keyframe arrives — but an exception here would kill the RTP reader
|
||||
* thread, which reaches send() from requestPli(). Mirrors the C++ server,
|
||||
* which ignores write failures.
|
||||
*/
|
||||
fun send(message: SignalingMessage) {
|
||||
val bytes = try {
|
||||
SignalingMessage.serialize(message).toByteArray(Charsets.UTF_8)
|
||||
} catch (e: Exception) {
|
||||
return // unreachable for our message types; serialize is total
|
||||
}
|
||||
synchronized(peerLock) {
|
||||
val out = peerOut ?: return
|
||||
try {
|
||||
out.write(bytes)
|
||||
out.flush()
|
||||
} catch (e: Exception) {
|
||||
// The peer is gone (e.g. the sender died while media still
|
||||
// flows). Dropping peerOut also closes the socket; the next
|
||||
// accepted offer installs a fresh one.
|
||||
peerOut = null
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fun close() {
|
||||
running = false
|
||||
try {
|
||||
server.close()
|
||||
} catch (e: Exception) {
|
||||
// already closed
|
||||
}
|
||||
synchronized(peerLock) {
|
||||
try {
|
||||
peerOut?.close()
|
||||
} catch (e: Exception) {
|
||||
// peer already gone
|
||||
}
|
||||
peerOut = null
|
||||
}
|
||||
acceptThread?.join(500)
|
||||
readerThread?.join(500)
|
||||
}
|
||||
|
||||
private fun acceptLoop() {
|
||||
while (running) {
|
||||
val socket = try {
|
||||
server.accept()
|
||||
} catch (e: Exception) {
|
||||
break // listening socket closed
|
||||
}
|
||||
synchronized(peerLock) {
|
||||
try {
|
||||
peerOut?.close()
|
||||
} catch (e: Exception) {
|
||||
// previous sender already gone
|
||||
}
|
||||
peerOut = socket.getOutputStream()
|
||||
}
|
||||
readerThread = Thread({ readLoop(socket) }, "signaling-reader").also { it.start() }
|
||||
}
|
||||
}
|
||||
|
||||
private fun readLoop(socket: Socket) {
|
||||
val input = BufferedInputStream(socket.getInputStream())
|
||||
val line = ByteArrayOutputStream()
|
||||
val chunk = ByteArray(4096)
|
||||
while (running) {
|
||||
val read = try {
|
||||
input.read(chunk)
|
||||
} catch (e: Exception) {
|
||||
break
|
||||
}
|
||||
if (read < 0) break
|
||||
for (i in 0 until read) {
|
||||
val b = chunk[i].toInt() and 0xFF
|
||||
if (b == '\n'.code) {
|
||||
val text = line.toString(Charsets.UTF_8.name())
|
||||
line.reset()
|
||||
if (text.isNotEmpty()) {
|
||||
dispatch(text)
|
||||
}
|
||||
} else if (b != '\r'.code) {
|
||||
if (line.size() < 64 * 1024) {
|
||||
line.write(b)
|
||||
} else {
|
||||
line.reset() // hostile or broken peer: drop the oversized line
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun dispatch(line: String) {
|
||||
val message = SignalingMessage.parse(line) ?: return
|
||||
try {
|
||||
when (message) {
|
||||
is SessionOffer -> onOffer(message)
|
||||
is SessionPli -> onPli(message)
|
||||
is SessionAnswer -> Unit // the receiver never receives answers
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
// A broken callback must not kill the reader thread.
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
|
||||
android:layout_width="match_parent"
|
||||
android:layout_height="match_parent"
|
||||
android:background="@android:color/black">
|
||||
|
||||
<TextureView
|
||||
android:id="@+id/video"
|
||||
android:layout_width="match_parent"
|
||||
android:layout_height="match_parent" />
|
||||
|
||||
<TextView
|
||||
android:id="@+id/status"
|
||||
android:layout_width="wrap_content"
|
||||
android:layout_height="wrap_content"
|
||||
android:layout_gravity="top|start"
|
||||
android:layout_margin="16dp"
|
||||
android:background="#66000000"
|
||||
android:maxLines="3"
|
||||
android:ellipsize="end"
|
||||
android:paddingHorizontal="12dp"
|
||||
android:paddingVertical="8dp"
|
||||
android:textColor="@android:color/white"
|
||||
android:textSize="13sp"
|
||||
tools:text="Listening… waiting for a sender"
|
||||
xmlns:tools="http://schemas.android.com/tools" />
|
||||
</FrameLayout>
|
||||
|
After Width: | Height: | Size: 4.4 KiB |
|
After Width: | Height: | Size: 2.7 KiB |
|
After Width: | Height: | Size: 6.2 KiB |
|
After Width: | Height: | Size: 10 KiB |
|
After Width: | Height: | Size: 16 KiB |
@@ -0,0 +1,9 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<resources>
|
||||
<style name="Theme.Screencast" parent="@android:style/Theme.Material.NoActionBar">
|
||||
<item name="android:windowFullscreen">true</item>
|
||||
<item name="android:windowBackground">@android:color/black</item>
|
||||
<item name="android:statusBarColor">@android:color/black</item>
|
||||
<item name="android:navigationBarColor">@android:color/black</item>
|
||||
</style>
|
||||
</resources>
|
||||
@@ -0,0 +1,138 @@
|
||||
package screen_cast.rtp
|
||||
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertFalse
|
||||
import org.junit.Assert.assertNull
|
||||
import org.junit.Assert.assertTrue
|
||||
import org.junit.Test
|
||||
|
||||
class H264DepacketizerTest {
|
||||
companion object {
|
||||
private val START = byteArrayOf(0x00, 0x00, 0x01)
|
||||
|
||||
private fun packet(seq: Int, ts: Int, payload: ByteArray, marker: Boolean = false) =
|
||||
RtpPacket(
|
||||
RtpHeader(sequenceNumber = seq, timestamp = ts, marker = marker),
|
||||
payload,
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun single_nal_two_packets() {
|
||||
val depacketizer = H264Depacketizer()
|
||||
// SPS (type 7) then a slice (type 5), closed by the marker.
|
||||
val first = depacketizer.depacketize(packet(1, 100, byteArrayOf(0x67, 0xAA.toByte(), 0xBB.toByte())))
|
||||
assertNull(first.accessUnit)
|
||||
val second = depacketizer.depacketize(packet(2, 100, byteArrayOf(0x41, 0x01, 0x02), marker = true))
|
||||
assertTrue(
|
||||
second.accessUnit
|
||||
?.contentEquals(START + byteArrayOf(0x67, 0xAA.toByte(), 0xBB.toByte()) + START + byteArrayOf(0x41, 0x01, 0x02)) == true,
|
||||
)
|
||||
assertTrue(second.isKeyFrame)
|
||||
assertFalse(second.frameDropped)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun fu_a_reassembly() {
|
||||
val depacketizer = H264Depacketizer()
|
||||
// NAL: header 0x41 (type 1, NRI 2) + payload 0x11 0x22 0x33 0x44.
|
||||
// FU indicator = (0x41 & 0xE0) | 28 = 0x5C; the depacketizer rebuilds
|
||||
// the NAL header from indicator-NRI | FU-type, so the FU header must
|
||||
// carry the original type (1).
|
||||
val indicator = 0x5C.toByte()
|
||||
val start = packet(1, 100, byteArrayOf(indicator, 0x81.toByte(), 0x11))
|
||||
val mid = packet(2, 100, byteArrayOf(indicator, 0x01, 0x22))
|
||||
val last = packet(3, 100, byteArrayOf(indicator, 0x41, 0x33, 0x44), marker = true)
|
||||
|
||||
assertNull(depacketizer.depacketize(start).accessUnit)
|
||||
assertNull(depacketizer.depacketize(mid).accessUnit)
|
||||
val done = depacketizer.depacketize(last)
|
||||
assertTrue(done.accessUnit?.contentEquals(START + byteArrayOf(0x41, 0x11, 0x22, 0x33, 0x44)) == true)
|
||||
assertFalse(done.frameDropped)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun drops_gapped_frames() {
|
||||
val depacketizer = H264Depacketizer()
|
||||
val first = depacketizer.depacketize(packet(1, 100, byteArrayOf(0x41, 0x01)))
|
||||
assertNull(first.accessUnit)
|
||||
// seq 2 is missing; the frame must be reported dropped, not delivered.
|
||||
val tail = depacketizer.depacketize(packet(3, 100, byteArrayOf(0x41, 0x02), marker = true))
|
||||
assertNull(tail.accessUnit)
|
||||
assertTrue(tail.frameDropped)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun separate_frames_by_marker() {
|
||||
val depacketizer = H264Depacketizer()
|
||||
val au1 = depacketizer.depacketize(packet(1, 100, byteArrayOf(0x41, 0xAA.toByte()), marker = true))
|
||||
assertTrue(au1.accessUnit?.contentEquals(START + byteArrayOf(0x41, 0xAA.toByte())) == true)
|
||||
val au2 = depacketizer.depacketize(packet(2, 200, byteArrayOf(0x41, 0xBB.toByte()), marker = true))
|
||||
assertTrue(au2.accessUnit?.contentEquals(START + byteArrayOf(0x41, 0xBB.toByte())) == true)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun drops_fu_without_start() {
|
||||
val depacketizer = H264Depacketizer()
|
||||
// Continuation (no S bit) without any start packet.
|
||||
val result = depacketizer.depacketize(packet(1, 100, byteArrayOf(0x7C.toByte(), 0x41.toByte(), 0x11), marker = true))
|
||||
assertNull(result.accessUnit)
|
||||
assertTrue(result.frameDropped)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun drops_still_fragmented_at_marker() {
|
||||
val depacketizer = H264Depacketizer()
|
||||
val start = depacketizer.depacketize(packet(1, 100, byteArrayOf(0x7C.toByte(), 0x81.toByte(), 0x11)))
|
||||
assertNull(start.accessUnit)
|
||||
// Marker arrives while the FU-A NAL is still open.
|
||||
val result = depacketizer.depacketize(packet(2, 100, byteArrayOf(0x41, 0x01), marker = true))
|
||||
assertNull(result.accessUnit)
|
||||
assertTrue(result.frameDropped)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun drops_unsupported_packetization() {
|
||||
val depacketizer = H264Depacketizer()
|
||||
val stapA = 24.toByte() // STAP-A
|
||||
val result = depacketizer.depacketize(
|
||||
packet(1, 100, byteArrayOf(stapA, 0x00, 0x05, 0x41, 0x01), marker = true),
|
||||
)
|
||||
assertNull(result.accessUnit)
|
||||
assertTrue(result.frameDropped)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun drops_timestamp_change_without_marker() {
|
||||
val depacketizer = H264Depacketizer()
|
||||
val first = depacketizer.depacketize(packet(1, 100, byteArrayOf(0x41, 0x01)))
|
||||
assertNull(first.accessUnit)
|
||||
// The stale frame is reported dropped, but this packet starts (and
|
||||
// closes) the next frame — matching the C++ depacketizer.
|
||||
val result = depacketizer.depacketize(packet(2, 200, byteArrayOf(0x41, 0x02), marker = true))
|
||||
assertTrue(result.accessUnit?.contentEquals(START + byteArrayOf(0x41, 0x02)) == true)
|
||||
assertTrue(result.frameDropped)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun keyframe_detection_requires_parameter_sets() {
|
||||
val depacketizer = H264Depacketizer()
|
||||
val plain = depacketizer.depacketize(packet(1, 100, byteArrayOf(0x41, 0x01), marker = true))
|
||||
assertFalse(plain.isKeyFrame)
|
||||
|
||||
val depacketizer2 = H264Depacketizer()
|
||||
val withSps = depacketizer2.depacketize(
|
||||
packet(1, 100, byteArrayOf(0x67, 0xAA.toByte(), 0x88.toByte(), 0x68, 0xBB.toByte(), 0x41, 0x01), marker = true),
|
||||
)
|
||||
assertTrue(withSps.isKeyFrame)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun drops_short_fu_packets() {
|
||||
val depacketizer = H264Depacketizer()
|
||||
// FU-A packet without its FU header byte.
|
||||
val result = depacketizer.depacketize(packet(1, 100, byteArrayOf(0x7C.toByte()), marker = true))
|
||||
assertNull(result.accessUnit)
|
||||
assertTrue(result.frameDropped)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
package screen_cast.rtp
|
||||
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertTrue
|
||||
import org.junit.Test
|
||||
|
||||
class JitterBufferTest {
|
||||
private fun packet(seq: Int, ts: Int = 100) =
|
||||
RtpPacket(RtpHeader(sequenceNumber = seq, timestamp = ts), byteArrayOf(seq.toByte()))
|
||||
|
||||
@Test
|
||||
fun in_order_releases_immediately() {
|
||||
val jitter = JitterBuffer()
|
||||
assertEquals(listOf(1), jitter.push(packet(1)).map { it.header.sequenceNumber })
|
||||
assertEquals(listOf(2), jitter.push(packet(2)).map { it.header.sequenceNumber })
|
||||
assertEquals(listOf(3), jitter.push(packet(3)).map { it.header.sequenceNumber })
|
||||
}
|
||||
|
||||
@Test
|
||||
fun reorders_out_of_order_packets() {
|
||||
val jitter = JitterBuffer()
|
||||
assertEquals(listOf(1), jitter.push(packet(1)).map { it.header.sequenceNumber })
|
||||
assertTrue(jitter.push(packet(3)).isEmpty())
|
||||
val released = jitter.push(packet(2)).map { it.header.sequenceNumber }
|
||||
assertEquals(listOf(2, 3), released)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun overflow_releases_in_order_and_advances() {
|
||||
val jitter = JitterBuffer(maxDepth = 4)
|
||||
assertTrue(jitter.push(packet(1)).map { it.header.sequenceNumber } == listOf(1))
|
||||
// seq 2 is lost; 3..6 stay buffered (within the depth bound).
|
||||
for (seq in 3..6) {
|
||||
assertTrue(jitter.push(packet(seq)).isEmpty())
|
||||
}
|
||||
// seq 7 overflows the buffer: 3..7 flush in order.
|
||||
assertEquals(listOf(3, 4, 5, 6, 7), jitter.push(packet(7)).map { it.header.sequenceNumber })
|
||||
// Delivery continues in order afterwards.
|
||||
assertEquals(listOf(8), jitter.push(packet(8)).map { it.header.sequenceNumber })
|
||||
assertEquals(listOf(9), jitter.push(packet(9)).map { it.header.sequenceNumber })
|
||||
}
|
||||
|
||||
@Test
|
||||
fun discards_stragglers() {
|
||||
val jitter = JitterBuffer(maxDepth = 4)
|
||||
jitter.push(packet(1))
|
||||
for (seq in 3..8) {
|
||||
jitter.push(packet(seq))
|
||||
}
|
||||
assertTrue(jitter.push(packet(9)).isNotEmpty())
|
||||
// seq 4 is now far behind the expected sequence: discarded, not delivered.
|
||||
assertTrue(jitter.push(packet(4)).isEmpty())
|
||||
// In-order delivery continues from 10.
|
||||
assertEquals(listOf(10), jitter.push(packet(10)).map { it.header.sequenceNumber })
|
||||
}
|
||||
|
||||
@Test
|
||||
fun clear_resets_state() {
|
||||
val jitter = JitterBuffer()
|
||||
jitter.push(packet(5))
|
||||
jitter.clear()
|
||||
// A completely different sequence now starts fresh.
|
||||
assertEquals(listOf(100), jitter.push(packet(100)).map { it.header.sequenceNumber })
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
package screen_cast.rtp
|
||||
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertFalse
|
||||
import org.junit.Assert.assertNull
|
||||
import org.junit.Assert.assertTrue
|
||||
import org.junit.Test
|
||||
|
||||
class RtpHeaderTest {
|
||||
@Test
|
||||
fun roundtrip() {
|
||||
val header = RtpHeader(
|
||||
version = 2,
|
||||
padding = false,
|
||||
extension = false,
|
||||
csrcCount = 0,
|
||||
marker = true,
|
||||
payloadType = 96,
|
||||
sequenceNumber = 0xABCD,
|
||||
timestamp = 0xDEADBEEF.toInt(),
|
||||
ssrc = 0x12345678,
|
||||
)
|
||||
val wire = header.serialize()
|
||||
assertEquals(12, wire.size)
|
||||
val parsed = RtpHeader.parse(wire)
|
||||
assertEquals(header, parsed)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun rejects_bad_version() {
|
||||
val wire = RtpHeader().serialize().copyOf()
|
||||
wire[0] = (wire[0].toInt() and 0x3F or (1 shl 6)).toByte()
|
||||
assertNull(RtpHeader.parse(wire))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun rejects_short_input() {
|
||||
val header = RtpHeader()
|
||||
assertNull(RtpHeader.parse(header.serialize().copyOfRange(0, 11)))
|
||||
assertNull(RtpHeader.parse(ByteArray(0)))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun preserves_flags() {
|
||||
val header = RtpHeader(padding = true, csrcCount = 2, marker = true, payloadType = 63)
|
||||
val parsed = RtpHeader.parse(header.serialize())
|
||||
assertEquals(true, parsed?.padding)
|
||||
assertEquals(2, parsed?.csrcCount)
|
||||
assertEquals(true, parsed?.marker)
|
||||
assertEquals(63, parsed?.payloadType)
|
||||
}
|
||||
}
|
||||
|
||||
class RtpPacketTest {
|
||||
private fun header(seq: Int, marker: Boolean = false) =
|
||||
RtpHeader(sequenceNumber = seq, payloadType = 96, marker = marker)
|
||||
|
||||
@Test
|
||||
fun roundtrip_with_payload() {
|
||||
val packet = RtpPacket(header(seq = 7), byteArrayOf(0x11, 0x22, 0x33))
|
||||
val wire = packet.header.serialize() + packet.payload
|
||||
val parsed = RtpPacket.parse(wire)
|
||||
assertEquals(header(seq = 7), parsed?.header)
|
||||
assertTrue(parsed?.payload?.contentEquals(byteArrayOf(0x11, 0x22, 0x33)) == true)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun skips_csrc_list() {
|
||||
val header = RtpHeader(csrcCount = 1, sequenceNumber = 3)
|
||||
val wire = header.serialize() + byteArrayOf(0x0A, 0x00, 0x00, 0x01) + byteArrayOf(0x99.toByte())
|
||||
val parsed = RtpPacket.parse(wire)
|
||||
assertEquals(1, parsed?.header?.csrcCount)
|
||||
assertTrue(parsed?.payload?.contentEquals(byteArrayOf(0x99.toByte())) == true)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun skips_extension_header() {
|
||||
val header = RtpHeader(extension = true, sequenceNumber = 4)
|
||||
// profile=0x0001, length=1 word, one word of data
|
||||
val wire = header.serialize() +
|
||||
byteArrayOf(0x00, 0x01, 0x00, 0x01, 0xDE.toByte(), 0xAD.toByte(), 0xBE.toByte(), 0xEF.toByte()) +
|
||||
byteArrayOf(0x77)
|
||||
val parsed = RtpPacket.parse(wire)
|
||||
assertTrue(parsed?.payload?.contentEquals(byteArrayOf(0x77)) == true)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun strips_padding() {
|
||||
val header = RtpHeader(padding = true, sequenceNumber = 5)
|
||||
// Payload byte, one padding zero, size byte (2 = padding incl. itself).
|
||||
val wire = header.serialize() + byteArrayOf(0x55, 0x00, 0x02)
|
||||
val parsed = RtpPacket.parse(wire)
|
||||
assertTrue(parsed?.payload?.contentEquals(byteArrayOf(0x55)) == true)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun rejects_truncated_csrc_and_extension() {
|
||||
val csrc = RtpHeader(csrcCount = 1).serialize()
|
||||
assertNull(RtpPacket.parse(csrc)) // 12 bytes, needs 16
|
||||
val ext = RtpHeader(extension = true).serialize() + byteArrayOf(0x00, 0x01)
|
||||
assertNull(RtpPacket.parse(ext)) // extension length field cut off
|
||||
}
|
||||
|
||||
@Test
|
||||
fun rejects_bad_padding() {
|
||||
val zeroPad = RtpHeader(padding = true).serialize() + byteArrayOf(0x00)
|
||||
assertNull(RtpPacket.parse(zeroPad))
|
||||
val oversized = RtpHeader(padding = true).serialize() + byteArrayOf(0x00, 0x00, 0x05)
|
||||
assertNull(RtpPacket.parse(oversized))
|
||||
assertNull(RtpPacket.parse(ByteArray(11)))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
plugins {
|
||||
id("com.android.application") version "9.4.0" apply false
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
org.gradle.jvmargs=-Xmx2g -Dfile.encoding=UTF-8
|
||||
org.gradle.caching=true
|
||||
kotlin.code.style=official
|
||||
@@ -0,0 +1,17 @@
|
||||
pluginManagement {
|
||||
repositories {
|
||||
google()
|
||||
mavenCentral()
|
||||
gradlePluginPortal()
|
||||
}
|
||||
}
|
||||
|
||||
dependencyResolutionManagement {
|
||||
repositories {
|
||||
google()
|
||||
mavenCentral()
|
||||
}
|
||||
}
|
||||
|
||||
rootProject.name = "screen_cast-android"
|
||||
include(":app")
|
||||
@@ -82,17 +82,83 @@ discovered addresses in reachability order (private IPv4 first).
|
||||
|
||||
## Phase 7 — Resilience and Polish
|
||||
|
||||
**Goal**: loss recovery, hardware acceleration, and packaging.
|
||||
**Goal**: loss recovery, hardware acceleration, and polish.
|
||||
|
||||
- NACK / PLI feedback loop.
|
||||
- Jitter buffer on the receiver.
|
||||
- VAAPI/NVENC hardware encode probes and fallback.
|
||||
- Optional GUI target behind `meson -Dgui=true`.
|
||||
- `.desktop` file, icon, packaging notes.
|
||||
- [x] PLI keyframe feedback over signaling (receiver asks, sender
|
||||
re-encodes a keyframe; validated end-to-end with an induced loss).
|
||||
- [x] Jitter/reorder window on the receiver (16 packets / 60 ms).
|
||||
- [x] Hardware H.264 decode probe with software fallback (h264_v4l2m2m —
|
||||
the Pi's VideoCore path; --swdecode opts out).
|
||||
- [x] GTK4 sender GUI behind `meson -Dgui=true` (gtkmm; receiver list,
|
||||
bitrate, start/stop) plus a waybar widget (`screencast waybar`,
|
||||
click-to-toggle, right-click opens the panel; state shared via
|
||||
$XDG_RUNTIME_DIR/screencast/sender.json).
|
||||
- [ ] VAAPI hardware encode probe on the sender (deferred: software
|
||||
encode is not the bottleneck).
|
||||
- Deferred: .desktop file, packaging.
|
||||
|
||||
**Validation**: sustained streaming under packet loss; hardware accel smoke
|
||||
where available.
|
||||
where available. PLI + jitter validated on the desktop (induced-loss probe);
|
||||
hardware decode validated as clean-fallback on the desktop, hardware path
|
||||
pending a run on the Pi.
|
||||
|
||||
## Phase 8 — Android Receiver App
|
||||
|
||||
**Goal**: a native Android receiver app (Kotlin) so a phone can act as the
|
||||
second receiver — screen on, USB-C DP-alt-mode to HDMI. The app speaks the
|
||||
existing signaling + RTP protocol; no C++ changes.
|
||||
|
||||
- [x] 8.1 Gradle project builds (`android/`, AGP 9 built-in Kotlin, no
|
||||
androidx; `gradle :app:assembleDebug`)
|
||||
- [x] 8.2 Kotlin sources + JVM unit tests (rtp framing, jitter buffer,
|
||||
depacketizer — 25 tests green)
|
||||
- [x] 8.3 Signaling + NSD validated on device (offer → answer over the
|
||||
network; mDNS registration via the API-36 RegistrationListener)
|
||||
— **correction 2026-09-10**: registration had in fact never
|
||||
succeeded (registerService passed protocol `0`, which API 36
|
||||
rejects with "Unsupported protocol"; the swallowed failure was
|
||||
hidden behind the "Listening…" status). Fixed with
|
||||
`PROTOCOL_DNS_SD` + real on-device validation: desktop
|
||||
`--discover` lists the phone and a live `--send` session
|
||||
decodes and renders.
|
||||
- [x] 8.4 MediaCodec decode + Surface render validated on device (in-band
|
||||
SPS sizing, letterbox fit)
|
||||
- [x] 8.5 End-to-end on a Fairphone 6 (Android 16): streaming, letterboxed
|
||||
fullscreen render, PLI keyframe recovery on Wi-Fi loss
|
||||
- [x] 8.6 Docs: README receiver section, RUNBOOK build/install/test,
|
||||
PHASES + MEMORY updates
|
||||
|
||||
**Validation**: `gradle :app:testDebugUnitTest` green; live session
|
||||
`--send --target window --peer <phone>:5005` rendered fullscreen on the
|
||||
phone (and HDMI via DP-alt-mode) with loss recovery.
|
||||
|
||||
## Phase 9 — iOS Receiver App
|
||||
|
||||
**Goal**: a native iOS receiver app (Swift) so an iPhone can act as the second
|
||||
receiver. The app speaks the existing signaling + RTP protocol; no C++ changes.
|
||||
Mirrors the Android receiver (Phase 8) source-to-source.
|
||||
|
||||
- [x] 9.1 Xcode project scaffolding (`ios/`, XcodeGen spec + bootstrap script;
|
||||
Info.plist with local-network + Bonjour privacy keys)
|
||||
- [x] 9.2 Swift protocol core + XCTest (rtp header/packet, jitter, depacketizer
|
||||
ported source-to-source; added signaling JSON, line framing, AVCC, NAL
|
||||
extraction tests — the Android side had no signaling tests)
|
||||
- [x] 9.3 Signaling server (BSD sockets, dual-stack, most-recent-peer, never
|
||||
throws) + `NSBonjourServices` advertisement
|
||||
- [x] 9.4 Media path: UDP (poll-based) → jitter → depacketize → VideoToolbox
|
||||
(in-band SPS/PPS, real-time decode) → AVSampleBufferDisplayLayer
|
||||
(letterbox) + PLI on damage + pendingOffer for late surface attach
|
||||
- [ ] 9.5 On-device validation (needs a Mac + Xcode 26 + iPhone 16): unit tests
|
||||
green on the simulator; `--discover` lists the phone; `--send --peer
|
||||
<ip>:5005` streams letterboxed; PLI recovery on Wi-Fi loss
|
||||
- [ ] 9.6 Docs: iOS RUNBOOK quirks, README receiver section, PHASES + MEMORY
|
||||
|
||||
**Validation**: `ios/bootstrap.sh test` (simulator) green; live session
|
||||
`--send --peer <iphone>:5005` renders fullscreen letterboxed on the iPhone with
|
||||
loss recovery. Note: this box is Linux — no Xcode/iOS SDK — so 9.2–9.4 are
|
||||
authored but only 9.5 can validate the VideoToolbox + local-network paths.
|
||||
|
||||
## Current phase
|
||||
|
||||
Phase 7 — Resilience and Polish.
|
||||
Phase 9 — iOS Receiver App (implementation authored; on-device validation
|
||||
pending a Mac + Xcode 26 + iPhone 16).
|
||||
|
||||
@@ -134,6 +134,54 @@ A window manager only comes with the desktop-session alternative, where
|
||||
the receiver runs inside it (uncomment the `Environment=` lines in the
|
||||
service file as described above).
|
||||
|
||||
## GTK panel and waybar widget (Phase 7)
|
||||
|
||||
Build the GUI alongside the CLI (`meson configure build -Dgui=true`, then
|
||||
recompile; installs as `screencast-gui`):
|
||||
|
||||
```sh
|
||||
screencast-gui # receiver list → pick one → bitrate → Start
|
||||
screencast waybar # one JSON line for a waybar custom module
|
||||
screencast waybar --toggle # stop a running sender / restart the last
|
||||
```
|
||||
|
||||
The waybar module (installed to this machine's `~/.config/waybar/` — see
|
||||
`custom/screencast` there): the app icon as the module face (dimmed while
|
||||
idle, green tint while streaming; the `▶`/`⏸` text from the JSON is hidden
|
||||
in the CSS), tooltip with receiver, bitrate, elapsed; left-click toggles
|
||||
streaming to the last receiver via the state file, right-click opens the
|
||||
panel. The sender publishes its state to `$XDG_RUNTIME_DIR/screencast/sender.json`,
|
||||
so every front-end — CLI, GUI, widget — agrees on what is running.
|
||||
|
||||
**Icons**: `screencast_icon/screencast_256.png` is the single app icon.
|
||||
`meson install` puts it in `share/icons/hicolor/256x256/apps/screencast.png`;
|
||||
the GTK panel uses it by themed name (`set_icon_name` — GTK4 removed the
|
||||
pixel-buffer window icon, so themed icons only), and the waybar CSS
|
||||
references the installed path. The SDL receiver window embeds the PNG at
|
||||
build time (`icon_png_data.h` generated by `scripts/icon_to_header.py`) and
|
||||
sets it via `SDL_SetWindowIcon` when SDL3_image is available (optional
|
||||
dependency; no pkg-config ships with it, so meson finds the library
|
||||
directly). Icons surface in compositor taskbars/switchers, not in title
|
||||
bars (neither GTK4's CSD nor Hyprland's decorations draw them).
|
||||
Restart waybar after changing `style.css` to pick up icon changes.
|
||||
|
||||
## Under the hood: resilience (Phase 7)
|
||||
|
||||
The receiver absorbs loss in two stages and recovers actively:
|
||||
|
||||
1. **Jitter window**: RTP packets are re-ordered by sequence number in a
|
||||
small buffer (16 packets / 60 ms), so Wi-Fi reordering is not misread as
|
||||
loss. In-order streams release immediately (zero added latency).
|
||||
2. **PLI feedback**: when a frame arrives genuinely damaged, the receiver
|
||||
drops it and asks the sender for a keyframe over the signaling channel
|
||||
(rate-limited to one request per 500 ms). The sender re-encodes a
|
||||
keyframe immediately — recovery takes one frame time instead of
|
||||
waiting out the rest of the GOP.
|
||||
|
||||
**Hardware decode**: the receiver probes `h264_v4l2m2m` (the VideoCore
|
||||
path on Raspberry Pi) and falls back to software automatically; the
|
||||
journal says which path is active. `--swdecode` forces software.
|
||||
|
||||
## Sender / receiver loopback (Phase 5, manual)
|
||||
|
||||
Two terminals on the same desktop session:
|
||||
@@ -161,6 +209,69 @@ The headless equivalent runs as part of `meson test` (`udp loopback` test):
|
||||
synthetic frames → encode → packetize → localhost UDP → depacketize →
|
||||
decode, no portal or window involved.
|
||||
|
||||
## Android receiver app (Phase 8)
|
||||
|
||||
Toolchain: JDK 21, system `gradle` (no wrapper), Android SDK with android-36.
|
||||
AGP 9 has **built-in Kotlin** — do not apply `org.jetbrains.kotlin.android`
|
||||
and do not use `kotlinOptions {}`.
|
||||
|
||||
```sh
|
||||
cd android
|
||||
gradle :app:assembleDebug # -> app/build/outputs/apk/debug/app-debug.apk
|
||||
gradle :app:testDebugUnitTest # 25 JVM tests (rtp, jitter, depacketizer)
|
||||
```
|
||||
|
||||
Device test (validated on a Fairphone 6, Android 16 / API 36):
|
||||
|
||||
```sh
|
||||
adb install -r app/build/outputs/apk/debug/app-debug.apk
|
||||
adb shell am start -n screen_cast.receiver/screen_cast.ReceiverActivity
|
||||
# activity FQN is screen_cast.ReceiverActivity (namespace), not
|
||||
# screen_cast.receiver.ReceiverActivity
|
||||
adb shell uiautomator dump /sdcard/ui.xml && adb shell cat /sdcard/ui.xml # status text
|
||||
adb exec-out screencap -p > phone.png # visual check
|
||||
```
|
||||
|
||||
On the sender: `screencast --send --peer <phone-ip>:5005` (add `--target
|
||||
window` to pick a specific window; the portal's monitor source picks the
|
||||
first output on multi-monitor Hyprland/GTK-portal setups).
|
||||
|
||||
Platform quirks found while validating (Android 16 / API 36):
|
||||
|
||||
- `android.permission.INTERNET` is required — NsdService rejects
|
||||
registration without it.
|
||||
- `NsdManager.registerService` on Android 16 validates the protocol
|
||||
argument (`NsdManager.checkProtocol`): the historical `0` throws
|
||||
`IllegalArgumentException: Unsupported protocol` — pass
|
||||
`NsdManager.PROTOCOL_DNS_SD`. This was silent for a long time: the app
|
||||
swallowed the exception and the failure status was overwritten by the
|
||||
"Listening…" line, so mDNS never advertised even though the UI looked
|
||||
fine (discovery only worked via `--peer`). Diagnosed via
|
||||
`adb shell dumpsys servicediscovery` (the client's `mClientRequests`
|
||||
stays empty when no request was ever issued) plus logging the
|
||||
exception.
|
||||
- `DatagramSocket.localPort` gives the bound port; `.port` is -1 for
|
||||
unconnected datagram sockets, and `.localAddress` is an `InetAddress`
|
||||
(Inet6Address on Android), not an `InetSocketAddress`.
|
||||
- `MediaCodec`: pass the output Surface to `configure()` — the codec must
|
||||
start in surface mode (`setOutputSurface` afterwards only switches an
|
||||
already-surface-mode codec to a new surface; a codec configured without
|
||||
a surface can never take one, per the API docs); call `start()`; render
|
||||
with `releaseOutputBuffer(index, render=true)`; the C2 AVC decoder
|
||||
requires a concrete width/height at configure (use a placeholder — the
|
||||
in-band SPS reconfigures it).
|
||||
- `MediaFormat.format()` / `KEY_MIME_TYPE` are not in the API-36 public
|
||||
surface.
|
||||
- NSD: the classic `registerService(info, flags, RegistrationListener)`
|
||||
API exists from API 16 through 36 (verified with javap on the android-36
|
||||
SDK jar) — no fallback is needed. A reflection fallback targeting a
|
||||
pre-36 "ResolutionListener" was removed: that class never existed at
|
||||
any API level, so the fallback could only ever fail.
|
||||
- The C2 decoder scales its output to the Surface (`android._video-scaling`)
|
||||
— letterbox by **sizing the TextureView to the video aspect** (centered
|
||||
on a black window background), not with a transform matrix (double scale).
|
||||
- mDNS does not cross subnets; `--peer` is the reliable path.
|
||||
|
||||
## Formatting
|
||||
|
||||
```sh
|
||||
|
||||
@@ -10,7 +10,9 @@ namespace sc {
|
||||
struct SendCommand {
|
||||
std::string_view target = "monitor"; // monitor, window
|
||||
std::string_view peer_address; // optional; empty means auto-discover
|
||||
int bitrate_kbps = 4000;
|
||||
int bitrate_kbps = 4000; // VBV max bitrate
|
||||
int crf = 22; // constant rate factor (quality)
|
||||
int fps = 0; // 0 = no cap (capture rate)
|
||||
};
|
||||
|
||||
struct ReceiveCommand {
|
||||
@@ -18,13 +20,18 @@ struct ReceiveCommand {
|
||||
int local_rtp_port = 5004;
|
||||
int signaling_port = 5005;
|
||||
bool fullscreen = false;
|
||||
bool software_decode = false; // --swdecode disables the hardware probe
|
||||
};
|
||||
|
||||
struct DiscoverCommand {
|
||||
int timeout_seconds = 3;
|
||||
};
|
||||
|
||||
using Command = std::variant<SendCommand, ReceiveCommand, DiscoverCommand>;
|
||||
struct WaybarCommand {
|
||||
bool toggle = false; // toggle streaming instead of printing status
|
||||
};
|
||||
|
||||
using Command = std::variant<SendCommand, ReceiveCommand, DiscoverCommand, WaybarCommand>;
|
||||
|
||||
// Parse command line arguments. Prints usage and returns std::nullopt on error.
|
||||
// `argv` is `char const* const*` so both `main`'s `char**` and const arrays
|
||||
|
||||
@@ -19,6 +19,18 @@ struct SenderPipelineConfig {
|
||||
// Where encoded RTP packets are sent. Defaults to the local loopback so
|
||||
// a sender and receiver on one machine work without any configuration.
|
||||
Endpoint peer_rtp_endpoint{"127.0.0.1", 5004};
|
||||
// Session identity from signaling; written to the sender state file for
|
||||
// status widgets (waybar) and one-click restarts.
|
||||
std::string session_id;
|
||||
// The receiver's display resolution (0 = unknown). The sender downscales
|
||||
// to this before encoding so bitrate is not spent on pixels the display
|
||||
// cannot show.
|
||||
int max_encode_width = 0;
|
||||
int max_encode_height = 0;
|
||||
// Cap the encoding frame rate (0 = no cap; use the capture rate).
|
||||
// Lowering the frame rate halves the bandwidth at the same quality
|
||||
// level — useful on constrained links.
|
||||
int max_frame_rate = 0;
|
||||
};
|
||||
|
||||
struct ReceiverPipelineConfig {
|
||||
@@ -39,6 +51,10 @@ class SenderPipeline {
|
||||
bool start();
|
||||
void stop();
|
||||
|
||||
// Ask the sender to encode its next frame as a keyframe. Thread-safe;
|
||||
// used by the PLI feedback path.
|
||||
void request_keyframe();
|
||||
|
||||
private:
|
||||
class Impl;
|
||||
std::unique_ptr<Impl> impl_;
|
||||
|
||||
@@ -10,11 +10,21 @@
|
||||
|
||||
namespace sc {
|
||||
|
||||
// Decoded video frame in YUV420P planar format (the native decoder output,
|
||||
// passed to the renderer without any colorspace conversion).
|
||||
struct DecodedFrame {
|
||||
int width = 0;
|
||||
int height = 0;
|
||||
uint64_t capture_timestamp_ns = 0;
|
||||
std::vector<std::byte> rgba_pixels;
|
||||
|
||||
// YUV420P planes; each row is `stride` bytes wide and may be padded
|
||||
// beyond the picture width.
|
||||
std::vector<std::byte> plane_y;
|
||||
std::vector<std::byte> plane_u;
|
||||
std::vector<std::byte> plane_v;
|
||||
int stride_y = 0;
|
||||
int stride_u = 0;
|
||||
int stride_v = 0;
|
||||
};
|
||||
|
||||
struct DecoderConfig {
|
||||
@@ -22,6 +32,9 @@ struct DecoderConfig {
|
||||
int width = 0;
|
||||
int height = 0;
|
||||
std::vector<std::byte> extradata; // SPS/PPS for H.264
|
||||
// Probe a hardware decoder (v4l2 mem2mem) first and fall back to the
|
||||
// software decoder automatically. Disable with --swdecode.
|
||||
bool hardware_accel = true;
|
||||
};
|
||||
|
||||
class Decoder {
|
||||
|
||||
@@ -23,6 +23,12 @@ struct EncoderConfig {
|
||||
int height = 0;
|
||||
int frame_rate_num = 30;
|
||||
int frame_rate_den = 1;
|
||||
// CRF (constant rate factor, 0-51): the target visual quality level.
|
||||
// Lower = better quality. 23 is x264's default; screen content looks
|
||||
// good at 20-24.
|
||||
int crf = 22;
|
||||
// Maximum bitrate in kbps (the VBV cap). The encoder uses fewer bits on
|
||||
// static content and more on motion, but never exceeds this.
|
||||
int bitrate_kbps = 4000;
|
||||
bool hardware_accel = false;
|
||||
};
|
||||
|
||||
@@ -42,14 +42,23 @@ class H264Packetizer {
|
||||
std::uint16_t next_sequence_number_ = 0;
|
||||
};
|
||||
|
||||
// Result of feeding one packet to the depacketizer.
|
||||
struct DepacketizeResult {
|
||||
// The completed access unit (Annex-B with 3-byte start codes) when the
|
||||
// packet closed an undamaged frame.
|
||||
std::optional<std::vector<std::byte>> access_unit;
|
||||
// True when this call discarded a frame as damaged (packet loss or an
|
||||
// unsupported packetization). Pipelines use it to request a keyframe.
|
||||
bool frame_dropped = false;
|
||||
};
|
||||
|
||||
// Reassembles RFC 6184 packet streams (single NAL unit packets and FU-A)
|
||||
// into Annex-B access units. Packets must arrive in order; frames damaged by
|
||||
// sequence gaps or missing fragments are dropped silently.
|
||||
// sequence gaps or missing fragments are reported via DepacketizeResult.
|
||||
class H264Depacketizer {
|
||||
public:
|
||||
// Feed one packet. Returns the completed access unit (Annex-B with 3-byte
|
||||
// start codes) when the packet closes a frame, nullopt otherwise.
|
||||
std::optional<std::vector<std::byte>> depacketize(const RtpPacket& packet);
|
||||
// Feed one packet.
|
||||
DepacketizeResult depacketize(const RtpPacket& packet);
|
||||
|
||||
private:
|
||||
void drop_frame();
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
#pragma once
|
||||
|
||||
#include <chrono>
|
||||
#include <cstdint>
|
||||
#include <map>
|
||||
#include <mutex>
|
||||
#include <optional>
|
||||
#include <span>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
namespace sc {
|
||||
@@ -31,4 +35,29 @@ struct RtpPacket {
|
||||
static std::optional<RtpPacket> parse(std::span<const std::byte> in) noexcept;
|
||||
};
|
||||
|
||||
// Reorders RTP packets by sequence number before depacketization so that a
|
||||
// reordering link (Wi-Fi) does not read as loss. Delivery stays in order;
|
||||
// only aged-out or overflowing buffers release out of order, which the
|
||||
// downstream gap detection still handles for genuine loss.
|
||||
class RtpJitterBuffer {
|
||||
public:
|
||||
explicit RtpJitterBuffer(std::size_t max_depth = 16,
|
||||
std::chrono::milliseconds max_delay = std::chrono::milliseconds{60});
|
||||
|
||||
// Insert one packet and return the packets now ready for in-order
|
||||
// delivery. In-order streams release immediately (zero added latency);
|
||||
// a straggler older than the next expected sequence is discarded.
|
||||
std::vector<RtpPacket> push(RtpPacket packet);
|
||||
|
||||
// Discard everything still buffered.
|
||||
void clear();
|
||||
|
||||
private:
|
||||
std::size_t max_depth_;
|
||||
std::chrono::milliseconds max_delay_;
|
||||
std::mutex mutex_;
|
||||
std::map<std::uint16_t, std::pair<std::chrono::steady_clock::time_point, RtpPacket>> buffer_;
|
||||
std::optional<std::uint16_t> next_expected_;
|
||||
};
|
||||
|
||||
} // namespace sc
|
||||
|
||||
@@ -29,9 +29,19 @@ struct SessionAnswer {
|
||||
// The address may be empty: the sender then targets the address of its
|
||||
// signaling connection and the port carried here.
|
||||
Endpoint rtp_endpoint;
|
||||
// The receiver's display resolution (0 = unknown). The sender may
|
||||
// downscale to this to avoid encoding pixels the display cannot show.
|
||||
int display_width = 0;
|
||||
int display_height = 0;
|
||||
};
|
||||
|
||||
using SignalingMessage = std::variant<SessionOffer, SessionAnswer>;
|
||||
// Picture Loss Indication: the receiver asks the sender for a keyframe
|
||||
// after discarding a damaged frame.
|
||||
struct SessionPli {
|
||||
std::string session_id;
|
||||
};
|
||||
|
||||
using SignalingMessage = std::variant<SessionOffer, SessionAnswer, SessionPli>;
|
||||
|
||||
class SignalingChannel {
|
||||
public:
|
||||
|
||||
@@ -24,6 +24,11 @@ class Renderer {
|
||||
// Present one decoded frame. Returns false if the window was closed.
|
||||
virtual bool present(const DecodedFrame& frame) = 0;
|
||||
|
||||
// The display resolution (native monitor size in fullscreen mode).
|
||||
// Used by the receiver to tell the sender what resolution to encode at.
|
||||
virtual int display_width() const = 0;
|
||||
virtual int display_height() const = 0;
|
||||
|
||||
// Pump events (window close, resize). Non-blocking.
|
||||
virtual bool poll_events() = 0;
|
||||
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
# screencast iOS receiver
|
||||
|
||||
A native iOS receiver app (Swift) so an iPhone can act as a second receiver.
|
||||
It speaks the **existing signaling + RTP protocol unchanged** — no C++ changes —
|
||||
mirroring the Android receiver (Phase 8) source-to-source.
|
||||
|
||||
- **Min iOS:** 17 (validated on iPhone 16 / iOS 26.6.1)
|
||||
- **No third-party dependencies** — only Apple system frameworks (SwiftUI,
|
||||
AVFoundation, VideoToolbox, CoreMedia, BSD sockets).
|
||||
|
||||
## Pipeline
|
||||
|
||||
```
|
||||
Bonjour advertise (_screencast._tcp) + TCP signaling server (offer → answer)
|
||||
UDP RTP → jitter buffer (16 pkt / 60 ms) → depacketize (single-NAL + FU-A)
|
||||
→ VideoToolbox H.264 (in-band SPS/PPS) → AVSampleBufferDisplayLayer (letterbox)
|
||||
```
|
||||
|
||||
Recovery matches the C++ receiver: a damaged frame is dropped and a PLI
|
||||
(rate-limited to one per 500 ms) asks the sender for a keyframe.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
Receiver/
|
||||
App/ SwiftUI app, video surface (AVSampleBufferDisplayLayer), controller
|
||||
Rtp/ RtpHeader, RtpPacket, JitterBuffer, H264Depacketizer, Avcc, NAL extraction
|
||||
Signaling/ SignalingMessage (JSON), LineAssembler, SignalingServer (BSD sockets)
|
||||
Decode/ RenderSink, H.264 format description, VideoToolbox decoder
|
||||
Support/ UdpTransport (poll-based), LocalAddress
|
||||
Pipeline/ ReceiverPipeline (coordinates everything)
|
||||
ReceiverTests/
|
||||
XCTest port of the Android JVM tests + added coverage (signaling, line
|
||||
framing, AVCC, NAL extraction)
|
||||
```
|
||||
|
||||
## Build and test (on a Mac with Xcode 26)
|
||||
|
||||
```sh
|
||||
./bootstrap.sh # installs XcodeGen, generates the project, builds for device
|
||||
./bootstrap.sh test # runs the unit tests on the simulator
|
||||
```
|
||||
|
||||
`bootstrap.sh` downloads XcodeGen from the GitHub release into `tools/` (no
|
||||
Homebrew). Override the simulator with `IOS_DEST="platform=iOS Simulator,name=…"`.
|
||||
|
||||
## Install on a device
|
||||
|
||||
Free provisioning (or your team) is set in Xcode — this can't be scripted:
|
||||
|
||||
1. Open `Receiver.xcodeproj`.
|
||||
2. Select the **Receiver** target → *Signing & Capabilities* → set your Apple ID
|
||||
(a 7-day development certificate is fine for sideloading).
|
||||
3. Run to the iPhone (USB, trust the computer).
|
||||
|
||||
The first run prompts for **Local Network** access — allow it, or the sender
|
||||
will never discover the phone (silent failure, like the Android
|
||||
`PROTOCOL_DNS_SD` bug).
|
||||
|
||||
## Stream to the phone
|
||||
|
||||
On the sender (Linux desktop):
|
||||
|
||||
```sh
|
||||
screencast --send --peer <phone-ip>:5005 # target the phone directly
|
||||
screencast --discover # or list receivers; the phone shows up
|
||||
```
|
||||
|
||||
Expected: the phone shows the captured desktop, letterboxed to its screen, with
|
||||
PLI recovery on Wi-Fi loss.
|
||||
|
||||
## Testing notes
|
||||
|
||||
The pure protocol core (RTP, jitter, depacketizer, signaling JSON, line framing,
|
||||
AVCC, NAL extraction) is unit-tested in the simulator. The VideoToolbox decode
|
||||
path and the local-network/Bonjour flow are on-device validation items — the
|
||||
highest-risk parts to verify on the borrowed Mac + iPhone.
|
||||
@@ -0,0 +1,88 @@
|
||||
import SwiftUI
|
||||
import UIKit
|
||||
|
||||
/// Owns the pipeline and mirrors its status to the UI. The pipeline posts its
|
||||
/// callbacks to the main thread, so the @Published mutations happen there.
|
||||
final class ReceiverController: ObservableObject {
|
||||
@Published var status = "Starting…"
|
||||
@Published var showStatus = true
|
||||
|
||||
private let pipeline: ReceiverPipeline
|
||||
private let pixelSize: CGSize
|
||||
|
||||
init() {
|
||||
let localIP = LocalAddress.primaryIPv4()
|
||||
let size = ReceiverController.screenPixelSize()
|
||||
pixelSize = size
|
||||
pipeline = ReceiverPipeline(
|
||||
localIP: localIP,
|
||||
displaySize: { [weak self] in self?.pixelSize ?? .zero },
|
||||
onStatus: { [weak self] text in self?.apply(status: text) },
|
||||
onFirstFrame: { [weak self] in self?.apply(showStatus: false) },
|
||||
onVideoSize: { _ in })
|
||||
}
|
||||
|
||||
func start() { pipeline.start() }
|
||||
func stop() { pipeline.stop() }
|
||||
func bindSink(_ sink: RenderSink) { pipeline.attachSink(sink) }
|
||||
|
||||
private func apply(status: String? = nil, showStatus: Bool? = nil) {
|
||||
if let s = status { self.status = s }
|
||||
if let v = showStatus { self.showStatus = v }
|
||||
}
|
||||
|
||||
static func screenPixelSize() -> CGSize {
|
||||
let scale = UIScreen.main.scale
|
||||
let b = UIScreen.main.bounds
|
||||
return CGSize(width: b.width * scale, height: b.height * scale)
|
||||
}
|
||||
}
|
||||
|
||||
struct ContentView: View {
|
||||
@StateObject private var controller = ReceiverController()
|
||||
@Environment(\.scenePhase) private var scenePhase
|
||||
|
||||
var body: some View {
|
||||
ZStack {
|
||||
Color.black.ignoresSafeArea()
|
||||
VideoSurfaceView { sink in controller.bindSink(sink) }
|
||||
.ignoresSafeArea()
|
||||
|
||||
if controller.showStatus {
|
||||
VStack {
|
||||
Spacer()
|
||||
Text(controller.status)
|
||||
.font(.footnote)
|
||||
.foregroundStyle(.white)
|
||||
.multilineTextAlignment(.center)
|
||||
.padding(.horizontal, 24)
|
||||
.padding(.vertical, 12)
|
||||
.background(.black.opacity(0.55), in: RoundedRectangle(cornerRadius: 10))
|
||||
Spacer()
|
||||
Spacer()
|
||||
}
|
||||
.transition(.opacity)
|
||||
}
|
||||
}
|
||||
.onAppear { controller.start() }
|
||||
.onChange(of: scenePhase) { _, phase in
|
||||
switch phase {
|
||||
case .active: controller.start()
|
||||
case .inactive, .background: controller.stop()
|
||||
@unknown default: break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@main
|
||||
struct ReceiverApp: App {
|
||||
var body: some Scene {
|
||||
WindowGroup {
|
||||
ContentView()
|
||||
.preferredColorScheme(.dark)
|
||||
.statusBarHidden(true)
|
||||
.persistentSystemOverlays(.hidden)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
import SwiftUI
|
||||
import AVFoundation
|
||||
|
||||
/// Hosts the `AVSampleBufferDisplayLayer` and exposes it as a `RenderSink`.
|
||||
/// The layer keeps the full screen and letterboxes via `.resizeAspect`, so the
|
||||
/// decoded stream (already downscaled to the display size by the sender) is
|
||||
/// shown 1:1 with no distortion.
|
||||
struct VideoSurfaceView: UIViewRepresentable {
|
||||
let onSink: (RenderSink) -> Void
|
||||
|
||||
func makeCoordinator() -> Coordinator {
|
||||
Coordinator()
|
||||
}
|
||||
|
||||
func makeUIView(context: Context) -> UIView {
|
||||
let view = UIView(frame: .zero)
|
||||
view.backgroundColor = .black
|
||||
|
||||
let layer = AVSampleBufferDisplayLayer()
|
||||
layer.videoGravity = .resizeAspect
|
||||
view.layer.addSublayer(layer)
|
||||
|
||||
let sink = AVSampleBufferRenderSink(layer: layer)
|
||||
context.coordinator.sink = sink
|
||||
onSink(sink)
|
||||
return view
|
||||
}
|
||||
|
||||
func updateUIView(_ uiView: UIView, context: Context) {}
|
||||
|
||||
final class Coordinator {
|
||||
var sink: AVSampleBufferRenderSink?
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
import CoreMedia
|
||||
import Foundation
|
||||
|
||||
/// Builds a `CMVideoFormatDescription` for H.264 from in-band SPS + PPS.
|
||||
///
|
||||
/// VideoToolbox needs the parameter sets up front; the sender repeats them
|
||||
/// before every keyframe, so any keyframe carries a complete set. This is the
|
||||
/// Core Foundation recipe for an H.264 "config" format description.
|
||||
enum H264FormatDescription {
|
||||
static func create(sps: [UInt8], pps: [UInt8]) -> CMVideoFormatDescription? {
|
||||
let pointersArray = CFArrayCreateMutable(kCFAllocatorDefault, 0, &kCFTypeArrayCallBacks)
|
||||
for ps in [sps, pps] {
|
||||
guard let descriptor = makeParameterSetDescriptor(ps) else {
|
||||
CFRelease(pointersArray)
|
||||
return nil
|
||||
}
|
||||
CFArrayAppendValue(pointersArray, descriptor.takeUnretainedValue())
|
||||
CFRelease(descriptor) // the array now owns it
|
||||
}
|
||||
|
||||
let key = kCMFormatDescriptionExtension_SampleDescriptionPointers as CFString
|
||||
let attrs = CFDictionaryCreateMutable(kCFAllocatorDefault, 0, &kCFDictionaryKeyCallBacks, &kCFDictionaryValueCallBacks)
|
||||
CFDictionarySetValue(attrs, key, pointersArray)
|
||||
|
||||
var config: Unmanaged<CMVideoFormatDescription>?
|
||||
let status = CMVideoFormatDescriptionCreate(
|
||||
kCFAllocatorDefault,
|
||||
kCMVideoCodecType_H264,
|
||||
0, 0, 0,
|
||||
attrs,
|
||||
&config)
|
||||
CFRelease(attrs)
|
||||
CFRelease(pointersArray)
|
||||
guard status == noErr, let c = config else { return nil }
|
||||
return c.takeRetainedValue()
|
||||
}
|
||||
|
||||
private static func makeParameterSetDescriptor(_ ps: [UInt8]) -> Unmanaged<CMVideoFormatDescription>? {
|
||||
let cfData = Data(ps) as CFData
|
||||
let oneElement = CFArrayCreateMutable(kCFAllocatorDefault, 0, &kCFTypeArrayCallBacks)
|
||||
CFArrayAppendValue(oneElement, cfData)
|
||||
let key = kCMFormatDescriptionExtension_SampleDescriptionPointers as CFString
|
||||
let attrs = CFDictionaryCreateMutable(kCFAllocatorDefault, 0, &kCFDictionaryKeyCallBacks, &kCFDictionaryValueCallBacks)
|
||||
CFDictionarySetValue(attrs, key, oneElement)
|
||||
|
||||
var desc: Unmanaged<CMVideoFormatDescription>?
|
||||
let status = CMVideoFormatDescriptionCreateForCodecType(
|
||||
kCFAllocatorDefault,
|
||||
kCMVideoCodecType_H264,
|
||||
attrs,
|
||||
&desc)
|
||||
CFRelease(oneElement)
|
||||
CFRelease(attrs)
|
||||
guard status == noErr, let d = desc else { return nil }
|
||||
return d
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,174 @@
|
||||
import VideoToolbox
|
||||
import CoreMedia
|
||||
import CoreGraphics
|
||||
import Foundation
|
||||
|
||||
/// Hardware H.264 decoder using VideoToolbox. The stream is self-describing:
|
||||
/// the sender repeats SPS/PPS in-band at every keyframe, so no out-of-band
|
||||
/// codec data is needed. On a keyframe the in-band SPS/PPS (re)establish the
|
||||
/// stream size; when that size changes, the decode session is recreated — the
|
||||
/// Android C2 "in-band SPS reconfigure" pattern, which also avoids the startup
|
||||
/// squish (we never report a placeholder size before the first real keyframe).
|
||||
///
|
||||
/// The decode output callback may run on a worker thread, so session and
|
||||
/// format-description access is guarded by a lock.
|
||||
final class H264VideoToolboxDecoder {
|
||||
private let renderSink: RenderSink
|
||||
private let stateLock = NSLock()
|
||||
private var session: VTDecompressionSession?
|
||||
private var formatDescription: CMVideoFormatDescription?
|
||||
private var streamSize = CGSize.zero
|
||||
private var realSizeSeen = false
|
||||
|
||||
init(renderSink: RenderSink) {
|
||||
self.renderSink = renderSink
|
||||
}
|
||||
|
||||
/// The decoded resolution, once the first keyframe configured the session
|
||||
/// (nil before that — the caller must not drive layout off a placeholder).
|
||||
func outputSize() -> CGSize? {
|
||||
guard realSizeSeen, streamSize != .zero else { return nil }
|
||||
return streamSize
|
||||
}
|
||||
|
||||
/// Decodes one access unit (Annex-B) and renders the output. Returns false
|
||||
/// when a decode error occurred and the caller should request a keyframe.
|
||||
func decode(accessUnit annexB: [UInt8], rtpTimestamp: Int, isKeyFrame: Bool) -> Bool {
|
||||
if isKeyFrame {
|
||||
guard let sets = NalExtractor.parameterSets(annexB) else { return false }
|
||||
configureIfNeeded(sps: sets.sps, pps: sets.pps)
|
||||
}
|
||||
stateLock.lock()
|
||||
let session = self.session
|
||||
let cd = self.formatDescription
|
||||
stateLock.unlock()
|
||||
guard let session, let cd else { return false }
|
||||
guard let avcc = AvccConverter.toAvcc(annexB) else { return false }
|
||||
|
||||
let pts = CMTime(value: CMTimeValue(rtpTimestamp), timescale: 90000)
|
||||
guard let blockBuffer = makeBlockBuffer(avcc) else { return false }
|
||||
|
||||
var infoFlags: VTDecodeInfoFlags = []
|
||||
let status = VTDecompressionSessionDecodeFrame(
|
||||
session,
|
||||
blockBuffer,
|
||||
isKeyFrame ? kVTDecodeFrameFlags_EnableFastPath : 0,
|
||||
&infoFlags,
|
||||
pts)
|
||||
CFRelease(blockBuffer)
|
||||
|
||||
if status != noErr {
|
||||
// Recoverable: the next keyframe (SPS/PPS + IDR) re-primes it.
|
||||
teardownSession()
|
||||
return false
|
||||
}
|
||||
realSizeSeen = true
|
||||
return true
|
||||
}
|
||||
|
||||
func release() {
|
||||
stateLock.lock()
|
||||
teardownSessionLocked()
|
||||
if let cd = formatDescription { CFRelease(cd) }
|
||||
formatDescription = nil
|
||||
stateLock.unlock()
|
||||
renderSink.detach()
|
||||
streamSize = .zero
|
||||
realSizeSeen = false
|
||||
}
|
||||
|
||||
// MARK: - Internals
|
||||
|
||||
private func configureIfNeeded(sps: [UInt8], pps: [UInt8]) {
|
||||
guard let cd = H264FormatDescription.create(sps: sps, pps: pps) else { return }
|
||||
var dims = CMVideoDimensions()
|
||||
guard CMVideoFormatDescriptionGetDimensions(cd, &dims) == noErr else {
|
||||
CFRelease(cd)
|
||||
return
|
||||
}
|
||||
let size = CGSize(width: CGFloat(dims.width), height: CGFloat(dims.height))
|
||||
stateLock.lock()
|
||||
// Same size and a live session: keep it (the sender only changes the
|
||||
// stream when the receiver's display size changes).
|
||||
if session != nil && size == streamSize {
|
||||
CFRelease(cd)
|
||||
stateLock.unlock()
|
||||
return
|
||||
}
|
||||
teardownSessionLocked()
|
||||
|
||||
var outSession: VTDecompressionSession?
|
||||
let status = VTDecompressionSessionCreate(
|
||||
kCFAllocatorDefault,
|
||||
cd,
|
||||
vtOutputCallback,
|
||||
Unmanaged.passUnretained(self).toOpaque(),
|
||||
&outSession)
|
||||
guard status == noErr, let newSession = outSession else {
|
||||
CFRelease(cd)
|
||||
stateLock.unlock()
|
||||
return
|
||||
}
|
||||
// Low-latency decode: emit as soon as the frame is complete.
|
||||
VTSessionSetProperty(newSession, kVTDecompressionPropertyKey_RealTime, kCFBooleanTrue)
|
||||
|
||||
renderSink.setFormatDescription(cd)
|
||||
formatDescription = cd // we hold the +1 from H264FormatDescription.create
|
||||
session = newSession
|
||||
streamSize = size
|
||||
realSizeSeen = false
|
||||
stateLock.unlock()
|
||||
}
|
||||
|
||||
// Caller holds stateLock.
|
||||
private func teardownSessionLocked() {
|
||||
if let s = session {
|
||||
VTDecompressionSessionInvalidate(s)
|
||||
CFRelease(s)
|
||||
}
|
||||
session = nil
|
||||
}
|
||||
|
||||
// Caller does not hold the lock.
|
||||
private func teardownSession() {
|
||||
stateLock.lock()
|
||||
teardownSessionLocked()
|
||||
stateLock.unlock()
|
||||
}
|
||||
|
||||
private func makeBlockBuffer(_ bytes: [UInt8]) -> CMBlockBuffer? {
|
||||
let cfData = Data(bytes) as CFData
|
||||
var blockBuffer: CMBlockBuffer?
|
||||
let status = CMBlockBufferCreateWithData(kCFAllocatorDefault, cfData, &blockBuffer)
|
||||
guard status == noErr, let bb = blockBuffer else { return nil }
|
||||
return bb
|
||||
}
|
||||
|
||||
// Runs on whatever thread VideoToolbox uses for the output callback.
|
||||
private func handleOutput(_ pixelBuffer: CVPixelBuffer?, _ presentationTime: CMTime?) {
|
||||
stateLock.lock()
|
||||
let cd = formatDescription
|
||||
stateLock.unlock()
|
||||
guard let cd, let pixelBuffer else { return }
|
||||
let pts = presentationTime ?? CMTime(value: 0, timescale: 600)
|
||||
var sampleBuffer: CMSampleBuffer?
|
||||
let status = CMSampleBufferCreate(kCFAllocatorDefault, nil, 0, pts, .invalid, 1, 0, nil, &sampleBuffer)
|
||||
guard status == noErr, let sb = sampleBuffer else { return }
|
||||
guard CMSampleBufferSetDataBufferFromPixelBuffer(sb, pixelBuffer) == noErr else {
|
||||
CFRelease(sb)
|
||||
return
|
||||
}
|
||||
renderSink.enqueue(sb)
|
||||
}
|
||||
}
|
||||
|
||||
/// C-compatible decode output callback; recovers the decoder from the refCon.
|
||||
private func vtOutputCallback(_ refCon: UnsafeMutableRawPointer?,
|
||||
_ pixelBuffer: CVPixelBuffer?,
|
||||
_ presentationTime: CMTime?,
|
||||
_ duration: CMTime?,
|
||||
_ infoFlags: VTDecodeInfoFlags) {
|
||||
guard let refCon = refCon else { return }
|
||||
let decoder = Unmanaged<H264VideoToolboxDecoder>.fromOpaque(refCon).takeUnretainedValue()
|
||||
decoder.handleOutput(pixelBuffer, presentationTime: presentationTime)
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
import AVFoundation
|
||||
import CoreMedia
|
||||
|
||||
/// A render target for decoded frames. The decoder enqueues a `CMSampleBuffer`
|
||||
/// (wrapping a CVPixelBuffer) per frame; the sink renders it. Mirrors the role
|
||||
/// of the Android `Surface` + MediaCodec surface-mode rendering.
|
||||
protocol RenderSink: AnyObject {
|
||||
var isAttached: Bool { get }
|
||||
func attach()
|
||||
func detach()
|
||||
/// Updates the layer's video format description (called when the in-band
|
||||
/// SPS/PPS establish a (new) stream size).
|
||||
func setFormatDescription(_ formatDescription: CMVideoFormatDescription)
|
||||
/// Enqueues one decoded frame for display.
|
||||
func enqueue(_ sampleBuffer: CMSampleBuffer)
|
||||
}
|
||||
|
||||
/// Renders decoded CVPixelBuffers via `AVSampleBufferDisplayLayer`, which does
|
||||
/// the video-scaling for us. `.resizeAspect` gives letterbox directly, so the
|
||||
/// host view can stay full-screen without a manual transform (the same lesson
|
||||
/// as the Android TextureView sizing: size the surface to the aspect, not a
|
||||
/// transform matrix).
|
||||
final class AVSampleBufferRenderSink: RenderSink {
|
||||
private let layer: AVSampleBufferDisplayLayer
|
||||
private var session: AVSampleBufferDisplayLayerSession?
|
||||
private let lock = NSLock()
|
||||
private(set) var isAttached = false
|
||||
|
||||
init(layer: AVSampleBufferDisplayLayer) {
|
||||
self.layer = layer
|
||||
}
|
||||
|
||||
func attach() {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
guard session == nil else { return }
|
||||
let s = AVSampleBufferDisplayLayerSession(layer)
|
||||
s.start()
|
||||
session = s
|
||||
isAttached = true
|
||||
}
|
||||
|
||||
func detach() {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
session?.stop()
|
||||
session = nil
|
||||
isAttached = false
|
||||
}
|
||||
|
||||
func setFormatDescription(_ formatDescription: CMVideoFormatDescription) {
|
||||
layer.formatDescription = formatDescription
|
||||
}
|
||||
|
||||
func enqueue(_ sampleBuffer: CMSampleBuffer) {
|
||||
lock.lock()
|
||||
let s = session
|
||||
lock.unlock()
|
||||
s?.enqueue(sampleBuffer)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,268 @@
|
||||
import Foundation
|
||||
import CoreGraphics
|
||||
|
||||
/// The receiver pipeline, mirroring the C++ ReceiverPipeline and the Kotlin
|
||||
/// receiver:
|
||||
///
|
||||
/// signaling server (offer → answer)
|
||||
/// UDP RTP → jitter buffer → depacketize → VideoToolbox → render sink
|
||||
///
|
||||
/// Recovery matches the C++ receiver: a damaged frame is dropped and a PLI
|
||||
/// (rate-limited to one per 500 ms) asks the sender for a keyframe.
|
||||
///
|
||||
/// Concurrency: all shared state and every decode call run on a single serial
|
||||
/// queue; the reader thread only polls/receives UDP and forwards datagrams to
|
||||
/// that queue. UI callbacks are marshalled to the main thread.
|
||||
final class ReceiverPipeline {
|
||||
static let desiredUdpPort: UInt16 = 5004
|
||||
static let desiredSignalingPort: UInt16 = 5005
|
||||
static let pliMinIntervalMs: Int = 500
|
||||
static let receiveBufferSize: Int32 = 4 * 1024 * 1024
|
||||
|
||||
private let localIP: String
|
||||
private let displaySize: () -> CGSize
|
||||
private let onStatus: (String) -> Void
|
||||
private let onFirstFrame: () -> Void
|
||||
private let onVideoSize: (CGSize) -> Void
|
||||
|
||||
private let queue = DispatchQueue(label: "sc.receiver.pipeline")
|
||||
|
||||
private var running = false
|
||||
private var udp: UdpTransport?
|
||||
private var signaling: SignalingServer?
|
||||
private var readerThread: Thread?
|
||||
|
||||
private let jitter = JitterBuffer()
|
||||
private var depacketizer = H264Depacketizer()
|
||||
private var decoder: H264VideoToolboxDecoder?
|
||||
private var renderSink: RenderSink?
|
||||
private var pendingOffer: SignalingMessage?
|
||||
private var activeSession = ""
|
||||
private var firstFrameSeen = false
|
||||
private var currentVideoSize = CGSize.zero
|
||||
private var lastPliAtMs: Int64 = 0
|
||||
|
||||
init(localIP: String,
|
||||
displaySize: @escaping () -> CGSize,
|
||||
onStatus: @escaping (String) -> Void,
|
||||
onFirstFrame: @escaping () -> Void,
|
||||
onVideoSize: @escaping (CGSize) -> Void) {
|
||||
self.localIP = localIP
|
||||
self.displaySize = displaySize
|
||||
self.onStatus = onStatus
|
||||
self.onFirstFrame = onFirstFrame
|
||||
self.onVideoSize = onVideoSize
|
||||
}
|
||||
|
||||
/// The current decoded resolution (zero until the first keyframe).
|
||||
func videoSize() -> CGSize {
|
||||
return queue.sync { currentVideoSize }
|
||||
}
|
||||
|
||||
// MARK: - Lifecycle
|
||||
|
||||
/// Binds the ports, starts the signaling server, and starts reading RTP.
|
||||
func start() {
|
||||
queue.async { [weak self] in
|
||||
guard let self, !self.running else { return }
|
||||
|
||||
let udp = UdpTransport()
|
||||
guard udp.bind(preferredPort: Self.desiredUdpPort, receiveBufferSize: Self.receiveBufferSize) else {
|
||||
self.postStatus("Failed to bind the media port")
|
||||
return
|
||||
}
|
||||
|
||||
let signaling = SignalingServer(
|
||||
onOffer: { [weak self] offer in self?.queue.async { self?.handleOffer(offer) } },
|
||||
onPli: { _ in })
|
||||
guard signaling.start(Self.desiredSignalingPort) else {
|
||||
udp.close()
|
||||
self.postStatus("Failed to start signaling")
|
||||
return
|
||||
}
|
||||
|
||||
self.running = true
|
||||
self.udp = udp
|
||||
self.signaling = signaling
|
||||
|
||||
let thread = Thread { [weak self] in self?.readLoop(udp: udp) }
|
||||
thread.name = "rtp-reader"
|
||||
self.readerThread = thread
|
||||
thread.start()
|
||||
|
||||
let ip = self.localIP
|
||||
let mediaPort = udp.port
|
||||
let sigPort = signaling.port
|
||||
self.postStatus("Listening on \(ip) (media :\(mediaPort), signaling :\(sigPort))\nWaiting for a sender… (fall back to: screencast --send --peer \(ip):\(sigPort))")
|
||||
}
|
||||
}
|
||||
|
||||
/// Points the (current or future) decoder at a render sink. A pending offer
|
||||
/// (accepted while no sink existed) configures its decoder now and requests
|
||||
/// a keyframe, since the sender only emits one when asked.
|
||||
func attachSink(_ sink: RenderSink) {
|
||||
queue.async { [weak self] in
|
||||
guard let self else { return }
|
||||
self.renderSink = sink
|
||||
sink.attach()
|
||||
guard let offer = self.pendingOffer else { return }
|
||||
self.pendingOffer = nil
|
||||
// The decoder configures on the first keyframe; the sink is already
|
||||
// attached, so creation cannot fail. A late-configured decoder needs
|
||||
// a keyframe (the sender only emits one when asked).
|
||||
self.decoder = H264VideoToolboxDecoder(renderSink: sink)
|
||||
self.requestPli()
|
||||
}
|
||||
}
|
||||
|
||||
/// Forgets a destroyed render sink so a later offer cannot render into it.
|
||||
func detachSink() {
|
||||
queue.async { [weak self] in
|
||||
guard let self else { return }
|
||||
self.decoder?.release()
|
||||
self.decoder = nil
|
||||
self.renderSink?.detach()
|
||||
self.renderSink = nil
|
||||
}
|
||||
}
|
||||
|
||||
/// Stops listening; the pipeline can be started again.
|
||||
func stop() {
|
||||
queue.async { [weak self] in
|
||||
guard let self, self.running else { return }
|
||||
self.running = false
|
||||
self.activeSession = ""
|
||||
self.udp?.close()
|
||||
self.udp = nil
|
||||
self.readerThread?.join()
|
||||
self.readerThread = nil
|
||||
self.signaling?.close()
|
||||
self.signaling = nil
|
||||
self.decoder?.release()
|
||||
self.decoder = nil
|
||||
self.pendingOffer = nil
|
||||
self.firstFrameSeen = false
|
||||
self.currentVideoSize = .zero
|
||||
self.postStatus("Stopped")
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Media path
|
||||
|
||||
private func readLoop(udp: UdpTransport) {
|
||||
while true {
|
||||
switch udp.poll(timeoutMs: 100) {
|
||||
case 1:
|
||||
if let data = udp.receiveDatagram() {
|
||||
queue.async { [weak self] in self?.processDatagram(data) }
|
||||
}
|
||||
case 0:
|
||||
continue // timeout: re-check on the next poll
|
||||
default:
|
||||
return // closed/errored: the socket was closed by stop()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func processDatagram(_ data: [UInt8]) {
|
||||
guard let packet = RtpPacket.parse(data) else { return }
|
||||
for released in jitter.push(packet) {
|
||||
handleDepacketized(released)
|
||||
}
|
||||
}
|
||||
|
||||
private func handleDepacketized(_ packet: RtpPacket) {
|
||||
let result = depacketizer.depacketize(packet)
|
||||
|
||||
if let accessUnit = result.accessUnit {
|
||||
let presentation = Int(packet.header.timestamp & 0xFFFFFFFF)
|
||||
guard let decoder = decoder else { return }
|
||||
if !decoder.decode(accessUnit: accessUnit, rtpTimestamp: presentation, isKeyFrame: result.isKeyFrame) {
|
||||
// Input/decode trouble: the dropped frame corrupts the GOP
|
||||
// until the next keyframe — ask for one.
|
||||
requestPli()
|
||||
}
|
||||
if !firstFrameSeen {
|
||||
firstFrameSeen = true
|
||||
postFirstFrame()
|
||||
}
|
||||
postVideoSizeIfChanged()
|
||||
}
|
||||
|
||||
if result.frameDropped {
|
||||
requestPli()
|
||||
}
|
||||
}
|
||||
|
||||
private func postVideoSizeIfChanged() {
|
||||
guard let size = decoder?.outputSize(), size != .zero else { return }
|
||||
if size != currentVideoSize {
|
||||
currentVideoSize = size
|
||||
postVideoSize(size)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Signaling
|
||||
|
||||
private func handleOffer(_ offer: SignalingMessage) {
|
||||
guard case let .offer(sessionId, codec, _, _, _, _, _, _) = offer else { return }
|
||||
if codec != "h264" {
|
||||
postStatus("Unsupported codec: \(codec)")
|
||||
return
|
||||
}
|
||||
// New session: pristine decoder, reassembly state, and session.
|
||||
decoder?.release()
|
||||
decoder = nil
|
||||
pendingOffer = nil
|
||||
if renderSink != nil {
|
||||
decoder = H264VideoToolboxDecoder(renderSink: renderSink!)
|
||||
} else {
|
||||
// No surface yet: park the offer; attachSink configures later.
|
||||
pendingOffer = offer
|
||||
}
|
||||
|
||||
depacketizer = H264Depacketizer()
|
||||
jitter.clear()
|
||||
firstFrameSeen = false
|
||||
currentVideoSize = .zero
|
||||
activeSession = sessionId
|
||||
|
||||
let size = displaySize()
|
||||
let answer = SignalingMessage.answer(
|
||||
sessionId: sessionId,
|
||||
rtpAddress: "", // the sender targets the address of its own signaling connection
|
||||
rtpPort: Int(udp?.port ?? 0),
|
||||
displayWidth: Int(size.width),
|
||||
displayHeight: Int(size.height))
|
||||
signaling?.send(answer)
|
||||
|
||||
if decoder != nil {
|
||||
postStatus("Session \(sessionId) negotiated — waiting for the first frame…")
|
||||
} else {
|
||||
postStatus("Session \(sessionId) negotiated — waiting for the display surface…")
|
||||
}
|
||||
}
|
||||
|
||||
/// Rate-limited keyframe request, callable from any thread (it always runs
|
||||
/// on the pipeline queue in practice).
|
||||
private func requestPli() {
|
||||
let session = activeSession
|
||||
guard !session.isEmpty else { return }
|
||||
let now = Int64(Date().timeIntervalSince1970 * 1000)
|
||||
if now - lastPliAtMs < Int64(Self.pliMinIntervalMs) { return }
|
||||
lastPliAtMs = now
|
||||
signaling?.send(.pli(sessionId: session))
|
||||
}
|
||||
|
||||
// MARK: - UI callbacks (main thread)
|
||||
|
||||
private func postStatus(_ text: String) {
|
||||
DispatchQueue.main.async { [onStatus] in onStatus(text) }
|
||||
}
|
||||
private func postFirstFrame() {
|
||||
DispatchQueue.main.async { [onFirstFrame] in onFirstFrame() }
|
||||
}
|
||||
private func postVideoSize(_ size: CGSize) {
|
||||
DispatchQueue.main.async { [onVideoSize] in onVideoSize(size) }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
import Foundation
|
||||
|
||||
/// Converts between Annex-B (start-code delimited) and AVCC (4-byte
|
||||
/// big-endian length prefixed) H.264 representations.
|
||||
///
|
||||
/// VideoToolbox consumes AVCC: each NAL unit is preceded by a 32-bit length.
|
||||
/// Our depacketizer emits Annex-B (the project's canonical 3-byte start codes),
|
||||
/// so the decoder feeds AVCC derived here.
|
||||
enum AvccConverter {
|
||||
/// Splits an Annex-B access unit into its NAL units (start codes removed).
|
||||
static func nalUnits(_ annexB: [UInt8]) -> [[UInt8]] {
|
||||
guard annexB.count >= 4 else { return [] }
|
||||
let n = annexB.count
|
||||
|
||||
// Locate every start code (3-byte `00 00 01` and 4-byte `00 00 00 01`).
|
||||
var offsets: [Int] = []
|
||||
var i = 0
|
||||
while i + 2 < n {
|
||||
if annexB[i] == 0 && annexB[i + 1] == 0 && annexB[i + 2] == 1 {
|
||||
offsets.append(i)
|
||||
i += 3
|
||||
continue
|
||||
}
|
||||
if i + 3 < n, annexB[i + 2] == 0, annexB[i + 3] == 1 {
|
||||
offsets.append(i)
|
||||
i += 4
|
||||
continue
|
||||
}
|
||||
i += 1
|
||||
}
|
||||
guard !offsets.isEmpty else { return [] }
|
||||
|
||||
var units: [[UInt8]] = []
|
||||
for (idx, offset) in offsets.enumerated() {
|
||||
let nalStart = offset + 3
|
||||
let nalEnd = idx + 1 < offsets.count ? offsets[idx + 1] : n
|
||||
let nal = Array(annexB[nalStart..<nalEnd])
|
||||
if !nal.isEmpty { units.append(nal) }
|
||||
}
|
||||
return units
|
||||
}
|
||||
|
||||
/// Returns the AVCC form of an Annex-B access unit, or nil if it has no NALs.
|
||||
static func toAvcc(_ annexB: [UInt8]) -> [UInt8]? {
|
||||
let units = nalUnits(annexB)
|
||||
guard !units.isEmpty else { return nil }
|
||||
var out: [UInt8] = []
|
||||
out.reserveCapacity(annexB.count + units.count * 4)
|
||||
for nal in units {
|
||||
let len = nal.count
|
||||
out.append(UInt8((len >> 24) & 0xFF))
|
||||
out.append(UInt8((len >> 16) & 0xFF))
|
||||
out.append(UInt8((len >> 8) & 0xFF))
|
||||
out.append(UInt8(len & 0xFF))
|
||||
out.append(contentsOf: nal)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
/// Splits an AVCC byte array (4-byte big-endian length prefixes) into NAL units.
|
||||
static func fromAvcc(_ avcc: [UInt8]) -> [[UInt8]] {
|
||||
var units: [[UInt8]] = []
|
||||
var i = 0
|
||||
let n = avcc.count
|
||||
while i + 4 <= n {
|
||||
let len = (Int(avcc[i]) << 24) | (Int(avcc[i + 1]) << 16) | (Int(avcc[i + 2]) << 8) | Int(avcc[i + 3])
|
||||
guard len > 0, i + 4 + len <= n else { break }
|
||||
units.append(Array(avcc[(i + 4)..<(i + 4 + len)]))
|
||||
i += 4 + len
|
||||
}
|
||||
return units
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,160 @@
|
||||
import Foundation
|
||||
|
||||
/// Result of feeding one packet to the depacketizer.
|
||||
struct DepacketizeResult {
|
||||
/// Completed access unit (Annex-B with 3-byte start codes) when the frame closed undamaged.
|
||||
var accessUnit: [UInt8]?
|
||||
/// True when this call discarded a frame as damaged (packet loss or unsupported packetization).
|
||||
var frameDropped = false
|
||||
/// True when the completed access unit carries SPS/PPS (a keyframe).
|
||||
var isKeyFrame = false
|
||||
}
|
||||
|
||||
/// Reassembles RFC 6184 packet streams (single NAL unit packets and FU-A)
|
||||
/// into Annex-B access units. Packets must arrive in order; frames damaged by
|
||||
/// sequence gaps or missing fragments are reported via DepacketizeResult.
|
||||
///
|
||||
/// Mirrors the C++ `H264Depacketizer` (same state machine and start codes).
|
||||
final class H264Depacketizer {
|
||||
static let fuA = 28
|
||||
|
||||
private var lastSequenceNumber: Int?
|
||||
private var frameStarted = false
|
||||
private var frameDamaged = false
|
||||
private var frameTimestamp = 0
|
||||
// Growable byte accumulators: keyframes reach hundreds of KB.
|
||||
private var accessUnit: [UInt8] = []
|
||||
private var fuActive = false
|
||||
private var fuNal: [UInt8] = []
|
||||
|
||||
/// Feed one packet (in sequence order, from the jitter buffer).
|
||||
func depacketize(_ packet: RtpPacket) -> DepacketizeResult {
|
||||
var result = DepacketizeResult()
|
||||
|
||||
// Track sequence continuity: a gap means packets were lost.
|
||||
if let last = lastSequenceNumber {
|
||||
let expected = (last + 1) & 0xFFFF
|
||||
if packet.header.sequenceNumber != expected {
|
||||
fuActive = false
|
||||
fuNal.removeAll(keepingCapacity: true)
|
||||
if frameStarted { frameDamaged = true }
|
||||
}
|
||||
}
|
||||
lastSequenceNumber = packet.header.sequenceNumber
|
||||
|
||||
// A timestamp change without a closing marker means the previous frame
|
||||
// lost its tail and can no longer be recovered.
|
||||
if frameStarted && packet.header.timestamp != frameTimestamp {
|
||||
dropFrame()
|
||||
result.frameDropped = true
|
||||
}
|
||||
if !frameStarted {
|
||||
frameStarted = true
|
||||
frameDamaged = false
|
||||
frameTimestamp = packet.header.timestamp
|
||||
accessUnit.removeAll(keepingCapacity: true)
|
||||
}
|
||||
|
||||
let payload = packet.payload
|
||||
if !payload.isEmpty {
|
||||
let type = Int(payload[0]) & 0x1F
|
||||
if type >= 1 && type <= 23 {
|
||||
// Single NAL unit packet.
|
||||
if fuActive {
|
||||
frameDamaged = true
|
||||
fuActive = false
|
||||
fuNal.removeAll(keepingCapacity: true)
|
||||
}
|
||||
appendStartCode()
|
||||
accessUnit.append(contentsOf: payload)
|
||||
} else if type == Self.fuA {
|
||||
if payload.count < 2 {
|
||||
frameDamaged = true
|
||||
} else {
|
||||
let fuHeader = Int(payload[1])
|
||||
let start = fuHeader & 0x80 != 0
|
||||
let end = fuHeader & 0x40 != 0
|
||||
let fragment = Array(payload[2...])
|
||||
if start {
|
||||
if fuActive {
|
||||
// The previous fragmented NAL lost its end packet.
|
||||
frameDamaged = true
|
||||
}
|
||||
fuActive = true
|
||||
fuNal.removeAll(keepingCapacity: true)
|
||||
// The FU indicator keeps the original NAL's F bit (0) and NRI,
|
||||
// and declares type 28; the FU header carries S/E plus the real type.
|
||||
fuNal.append(UInt8((Int(payload[0]) & 0xE0) | (fuHeader & 0x1F)))
|
||||
fuNal.append(contentsOf: fragment)
|
||||
} else if !fuActive {
|
||||
// Continuation without a start: the head of the NAL is lost.
|
||||
frameDamaged = true
|
||||
} else {
|
||||
fuNal.append(contentsOf: fragment)
|
||||
if end {
|
||||
appendStartCode()
|
||||
accessUnit.append(contentsOf: fuNal)
|
||||
fuActive = false
|
||||
fuNal.removeAll(keepingCapacity: true)
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Unsupported packetization mode (STAP-A, MTAP, FU-B).
|
||||
frameDamaged = true
|
||||
}
|
||||
}
|
||||
|
||||
if !packet.header.marker {
|
||||
return result
|
||||
}
|
||||
|
||||
if fuActive {
|
||||
// The marker arrived while a NAL was still fragmented.
|
||||
frameDamaged = true
|
||||
fuActive = false
|
||||
fuNal.removeAll(keepingCapacity: true)
|
||||
}
|
||||
|
||||
if !frameDamaged && !accessUnit.isEmpty {
|
||||
let unit = accessUnit
|
||||
result.accessUnit = unit
|
||||
result.isKeyFrame = Self.containsParameterSets(unit)
|
||||
} else {
|
||||
// The frame that just ended is unusable.
|
||||
result.frameDropped = true
|
||||
}
|
||||
dropFrame()
|
||||
return result
|
||||
}
|
||||
|
||||
private func appendStartCode() {
|
||||
accessUnit.append(0)
|
||||
accessUnit.append(0)
|
||||
accessUnit.append(1)
|
||||
}
|
||||
|
||||
/// The sender repeats SPS/PPS in-band at every keyframe; sniff for NAL types 7/8.
|
||||
private static func containsParameterSets(_ unit: [UInt8]) -> Bool {
|
||||
guard unit.count >= 4 else { return false }
|
||||
var i = 0
|
||||
while i <= unit.count - 4 {
|
||||
if unit[i] == 0 && unit[i + 1] == 0 && unit[i + 2] == 1 {
|
||||
let nalType = Int(unit[i + 3]) & 0x1F
|
||||
if nalType == 7 || nalType == 8 {
|
||||
return true
|
||||
}
|
||||
}
|
||||
i += 1
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
private func dropFrame() {
|
||||
frameStarted = false
|
||||
frameDamaged = false
|
||||
accessUnit.removeAll(keepingCapacity: true)
|
||||
fuActive = false
|
||||
fuNal.removeAll(keepingCapacity: true)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
import Foundation
|
||||
|
||||
/// Reorders RTP packets by sequence number before depacketization so that a
|
||||
/// reordering link (Wi-Fi) is not read as loss. Delivery stays in order; only
|
||||
/// aged-out or overflowing buffers release out of order, which the downstream
|
||||
/// gap detection still handles for genuine loss.
|
||||
///
|
||||
/// Mirrors the C++ `RtpJitterBuffer` (same defaults and semantics) and the
|
||||
/// Kotlin receiver.
|
||||
final class JitterBuffer {
|
||||
private struct Entry {
|
||||
let timeNs: Int64
|
||||
let packet: RtpPacket
|
||||
}
|
||||
|
||||
private let maxDepth: Int
|
||||
private let maxDelayNs: Int64
|
||||
private var buffer: [Int: Entry] = [:]
|
||||
private var nextExpected: Int?
|
||||
|
||||
init(maxDepth: Int = 16, maxDelayMs: Int = 60) {
|
||||
self.maxDepth = maxDepth
|
||||
self.maxDelayNs = Int64(maxDelayMs) * 1_000_000
|
||||
}
|
||||
|
||||
/// Insert one packet and return the packets now ready for in-order delivery.
|
||||
func push(_ packet: RtpPacket) -> [RtpPacket] {
|
||||
var released: [RtpPacket] = []
|
||||
let sequence = packet.header.sequenceNumber
|
||||
let now = Self.nowNanos()
|
||||
|
||||
let expected0: Int
|
||||
if let e = nextExpected {
|
||||
expected0 = e
|
||||
} else {
|
||||
expected0 = sequence
|
||||
nextExpected = sequence
|
||||
}
|
||||
|
||||
// Serial-number comparison: a distance >= 32768 means the packet is
|
||||
// older than what we already delivered (duplicate or straggler).
|
||||
let distance = (sequence - expected0 + 65536) % 65536
|
||||
if distance < 32768 {
|
||||
buffer[sequence] = Entry(timeNs: now, packet: packet)
|
||||
|
||||
// Release the consecutive run from the expected sequence.
|
||||
var expected = expected0
|
||||
while let entry = buffer[expected] {
|
||||
released.append(entry.packet)
|
||||
buffer.removeValue(forKey: expected)
|
||||
expected = (expected + 1) & 0xFFFF
|
||||
}
|
||||
nextExpected = expected
|
||||
|
||||
// A missing packet stalls the run: age out the backlog (or bound
|
||||
// the buffer) and release what is there in order, so genuine loss
|
||||
// reaches the depacketizer's gap detection rather than blocking.
|
||||
if !buffer.isEmpty {
|
||||
let keys = buffer.keys.sorted()
|
||||
let head = keys.first!
|
||||
let headAgeNs = now - buffer[head]!.timeNs
|
||||
if headAgeNs > maxDelayNs || buffer.count > maxDepth {
|
||||
for key in keys {
|
||||
released.append(buffer[key]!.packet)
|
||||
}
|
||||
nextExpected = (keys.last! + 1) & 0xFFFF
|
||||
buffer.removeAll(keepingCapacity: true)
|
||||
}
|
||||
}
|
||||
}
|
||||
return released
|
||||
}
|
||||
|
||||
/// Discard everything still buffered.
|
||||
func clear() {
|
||||
buffer.removeAll(keepingCapacity: true)
|
||||
nextExpected = nil
|
||||
}
|
||||
|
||||
private static func nowNanos() -> Int64 {
|
||||
return Int64(DispatchTime.now().uptimeNanoseconds)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
import Foundation
|
||||
|
||||
/// The in-band SPS (type 7) and PPS (type 8) NAL units extracted from an
|
||||
/// access unit. The sender repeats both ahead of every keyframe, so any
|
||||
/// keyframe carries them; they are the source for the VideoToolbox format
|
||||
/// description.
|
||||
struct NalSets {
|
||||
let sps: [UInt8]
|
||||
let pps: [UInt8]
|
||||
}
|
||||
|
||||
/// Extracts parameter sets from an Annex-B access unit.
|
||||
enum NalExtractor {
|
||||
static func parameterSets(_ annexB: [UInt8]) -> NalSets? {
|
||||
var sps: [UInt8]?
|
||||
var pps: [UInt8]?
|
||||
for nal in AvccConverter.nalUnits(annexB) {
|
||||
guard !nal.isEmpty else { continue }
|
||||
let type = Int(nal[0]) & 0x1F
|
||||
if type == 7 && sps == nil {
|
||||
sps = nal
|
||||
} else if type == 8 && pps == nil {
|
||||
pps = nal
|
||||
}
|
||||
}
|
||||
guard let s = sps, let p = pps else { return nil }
|
||||
return NalSets(sps: s, pps: p)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
import Foundation
|
||||
|
||||
/// Minimal RTP header (RFC 3550) without extensions, mirroring the C++
|
||||
/// `RtpHeader` and the Kotlin receiver.
|
||||
struct RtpHeader: Equatable {
|
||||
var version = 2
|
||||
var padding = false
|
||||
var extensionHeader = false
|
||||
var csrcCount = 0
|
||||
var marker = false
|
||||
var payloadType = 96
|
||||
var sequenceNumber = 0
|
||||
var timestamp = 0
|
||||
var ssrc = 0
|
||||
|
||||
/// Serializes the bare 12-byte header; requires a version-2, extension-less header.
|
||||
func serialize() -> [UInt8] {
|
||||
var out = [UInt8](repeating: 0, count: 12)
|
||||
out[0] = UInt8((version & 0x0F) << 6
|
||||
| (padding ? 0x20 : 0)
|
||||
| (extensionHeader ? 0x10 : 0)
|
||||
| (csrcCount & 0x0F))
|
||||
out[1] = UInt8((marker ? 0x80 : 0) | (payloadType & 0x7F))
|
||||
out[2] = UInt8((sequenceNumber >> 8) & 0xFF)
|
||||
out[3] = UInt8(sequenceNumber & 0xFF)
|
||||
out[4] = UInt8((timestamp >> 24) & 0xFF)
|
||||
out[5] = UInt8((timestamp >> 16) & 0xFF)
|
||||
out[6] = UInt8((timestamp >> 8) & 0xFF)
|
||||
out[7] = UInt8(timestamp & 0xFF)
|
||||
out[8] = UInt8((ssrc >> 24) & 0xFF)
|
||||
out[9] = UInt8((ssrc >> 16) & 0xFF)
|
||||
out[10] = UInt8((ssrc >> 8) & 0xFF)
|
||||
out[11] = UInt8(ssrc & 0xFF)
|
||||
return out
|
||||
}
|
||||
|
||||
/// Parses a 12-byte header from the start of a datagram.
|
||||
static func parse(_ input: [UInt8]) -> RtpHeader? {
|
||||
guard input.count >= 12 else { return nil }
|
||||
let b0 = Int(input[0])
|
||||
let b1 = Int(input[1])
|
||||
let version = b0 >> 6
|
||||
guard version == 2 else { return nil }
|
||||
return RtpHeader(
|
||||
version: version,
|
||||
padding: b0 & 0x20 != 0,
|
||||
extensionHeader: b0 & 0x10 != 0,
|
||||
csrcCount: b0 & 0x0F,
|
||||
marker: b1 & 0x80 != 0,
|
||||
payloadType: b1 & 0x7F,
|
||||
sequenceNumber: (Int(input[2]) << 8) | Int(input[3]),
|
||||
timestamp: (Int(input[4]) << 24) | (Int(input[5]) << 16) | (Int(input[6]) << 8) | Int(input[7]),
|
||||
ssrc: (Int(input[8]) << 24) | (Int(input[9]) << 16) | (Int(input[10]) << 8) | Int(input[11]),
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
import Foundation
|
||||
|
||||
/// An RTP packet: 12-byte base header (plus optional CSRC/extension) and payload.
|
||||
struct RtpPacket {
|
||||
let header: RtpHeader
|
||||
let payload: [UInt8]
|
||||
|
||||
init(header: RtpHeader, payload: [UInt8]) {
|
||||
self.header = header
|
||||
self.payload = payload
|
||||
}
|
||||
|
||||
/// Parses a full RTP datagram. Honors CSRC lists, one-level extension
|
||||
/// headers, and RFC 3550 padding, mirroring the C++ receiver.
|
||||
static func parse(_ input: [UInt8]) -> RtpPacket? {
|
||||
guard input.count >= 12, let header = RtpHeader.parse(input) else { return nil }
|
||||
|
||||
var offset = 12 + header.csrcCount * 4
|
||||
guard input.count >= offset else { return nil }
|
||||
|
||||
if header.extensionHeader {
|
||||
guard input.count >= offset + 4 else { return nil }
|
||||
let extensionWords = (Int(input[offset + 2]) << 8) | Int(input[offset + 3])
|
||||
offset += 4 + extensionWords * 4
|
||||
guard input.count >= offset else { return nil }
|
||||
}
|
||||
|
||||
var payloadSize = input.count - offset
|
||||
if header.padding {
|
||||
// RFC 3550: the last byte holds the padding size, including itself.
|
||||
guard payloadSize > 0 else { return nil }
|
||||
let paddingSize = Int(input[input.count - 1])
|
||||
guard paddingSize != 0, paddingSize <= payloadSize else { return nil }
|
||||
payloadSize -= paddingSize
|
||||
}
|
||||
return RtpPacket(header: header, payload: Array(input[offset..<(offset + payloadSize)]))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
import Foundation
|
||||
|
||||
/// Splits a byte stream into newline-terminated lines, dropping CR and
|
||||
/// enforcing the 64 KB line cap — the same framing the C++ and Kotlin
|
||||
/// receivers use. Extracted so the logic is testable in isolation.
|
||||
final class LineAssembler {
|
||||
private var pending: [UInt8] = []
|
||||
private let maxLine = SignalingMessage.maxMessageBytes
|
||||
|
||||
/// Feed a chunk of received bytes; returns the complete lines it contained.
|
||||
func feed(_ chunk: [UInt8]) -> [String] {
|
||||
var lines: [String] = []
|
||||
for b in chunk {
|
||||
if b == 0x0A { // \n
|
||||
let text = String(bytes: pending, encoding: .utf8) ?? ""
|
||||
pending.removeAll(keepingCapacity: true)
|
||||
if !text.isEmpty { lines.append(text) }
|
||||
} else if b != 0x0D { // \r
|
||||
if pending.count < maxLine {
|
||||
pending.append(b)
|
||||
} else {
|
||||
// Hostile or broken peer: drop the oversized line.
|
||||
pending.removeAll(keepingCapacity: true)
|
||||
}
|
||||
}
|
||||
}
|
||||
return lines
|
||||
}
|
||||
|
||||
func reset() {
|
||||
pending.removeAll(keepingCapacity: true)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
import Foundation
|
||||
|
||||
/// JSON wire format shared with the C++ implementation: one JSON object per
|
||||
/// newline-terminated TCP line (offer / answer / pli). Mirrors the C++
|
||||
/// `SignalingMessage` and the Kotlin receiver.
|
||||
enum SignalingMessage {
|
||||
case offer(sessionId: String,
|
||||
codec: String,
|
||||
width: Int,
|
||||
height: Int,
|
||||
frameRateNum: Int,
|
||||
frameRateDen: Int,
|
||||
rtpAddress: String,
|
||||
rtpPort: Int)
|
||||
case answer(sessionId: String,
|
||||
rtpAddress: String,
|
||||
rtpPort: Int,
|
||||
displayWidth: Int,
|
||||
displayHeight: Int)
|
||||
case pli(sessionId: String)
|
||||
|
||||
static let maxMessageBytes = 64 * 1024
|
||||
|
||||
static func parse(_ line: String) -> SignalingMessage? {
|
||||
guard line.count <= maxMessageBytes else { return nil }
|
||||
guard let data = line.data(using: .utf8),
|
||||
let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
|
||||
let type = obj["type"] as? String else {
|
||||
return nil
|
||||
}
|
||||
switch type {
|
||||
case "offer":
|
||||
return .offer(
|
||||
sessionId: obj["session_id"] as? String ?? "",
|
||||
codec: obj["codec"] as? String ?? "",
|
||||
width: obj["width"] as? Int ?? 0,
|
||||
height: obj["height"] as? Int ?? 0,
|
||||
frameRateNum: obj["frame_rate_num"] as? Int ?? 30,
|
||||
frameRateDen: obj["frame_rate_den"] as? Int ?? 1,
|
||||
rtpAddress: obj["rtp_address"] as? String ?? "",
|
||||
rtpPort: obj["rtp_port"] as? Int ?? 0)
|
||||
case "answer":
|
||||
return .answer(
|
||||
sessionId: obj["session_id"] as? String ?? "",
|
||||
rtpAddress: obj["rtp_address"] as? String ?? "",
|
||||
rtpPort: obj["rtp_port"] as? Int ?? 0,
|
||||
displayWidth: obj["display_width"] as? Int ?? 0,
|
||||
displayHeight: obj["display_height"] as? Int ?? 0)
|
||||
case "pli":
|
||||
return .pli(sessionId: obj["session_id"] as? String ?? "")
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
static func serialize(_ message: SignalingMessage) -> String {
|
||||
let json: [String: Any]
|
||||
switch message {
|
||||
case let .offer(sessionId, codec, width, height, frameRateNum, frameRateDen, rtpAddress, rtpPort):
|
||||
json = [
|
||||
"type": "offer",
|
||||
"session_id": sessionId,
|
||||
"codec": codec,
|
||||
"width": width,
|
||||
"height": height,
|
||||
"frame_rate_num": frameRateNum,
|
||||
"frame_rate_den": frameRateDen,
|
||||
"rtp_address": rtpAddress,
|
||||
"rtp_port": rtpPort,
|
||||
]
|
||||
case let .answer(sessionId, rtpAddress, rtpPort, displayWidth, displayHeight):
|
||||
json = [
|
||||
"type": "answer",
|
||||
"session_id": sessionId,
|
||||
"rtp_address": rtpAddress,
|
||||
"rtp_port": rtpPort,
|
||||
"display_width": displayWidth,
|
||||
"display_height": displayHeight,
|
||||
]
|
||||
case let .pli(sessionId):
|
||||
json = ["type": "pli", "session_id": sessionId]
|
||||
}
|
||||
guard let data = try? JSONSerialization.data(withJSONObject: json),
|
||||
let s = String(data: data, encoding: .utf8) else {
|
||||
return "{}\n" // unreachable for our message types; serialize is total
|
||||
}
|
||||
return s + "\n"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,212 @@
|
||||
import Foundation
|
||||
#if canImport(Darwin)
|
||||
import Darwin
|
||||
#endif
|
||||
|
||||
/// Small socket helpers shared by the signaling server and the UDP transport.
|
||||
enum SocketUtils {
|
||||
/// Creates a bound, listening TCP socket. Prefers a dual-stack IPv6
|
||||
/// listener (both families), falls back to IPv4-only. Returns -1 on failure.
|
||||
static func makeStreamListener(port: UInt16) -> Int32 {
|
||||
for family in [AF_INET6, AF_INET] {
|
||||
let fd = socket(family, SOCK_STREAM, 0)
|
||||
guard fd >= 0 else { continue }
|
||||
var yes: Int32 = 1
|
||||
setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &yes, socklen_t(MemoryLayout<Int32>.size))
|
||||
if family == AF_INET6 {
|
||||
var no: Int32 = 0
|
||||
setsockopt(fd, IPPROTO_IPV6, IPV6_V6ONLY, &no, socklen_t(MemoryLayout<Int32>.size))
|
||||
}
|
||||
|
||||
let bound: Int32
|
||||
if family == AF_INET6 {
|
||||
var a = sockaddr_in6()
|
||||
a.sin6_family = sa_family_t(AF_INET6)
|
||||
a.sin6_port = port.bigEndian
|
||||
a.sin6_addr = in6addr_any
|
||||
bound = withUnsafePointer(to: &a) { p in
|
||||
p.withMemoryRebound(to: sockaddr.self, capacity: 1) { bind(fd, $0, socklen_t(MemoryLayout<sockaddr_in6>.size)) }
|
||||
}
|
||||
} else {
|
||||
var a = sockaddr_in()
|
||||
a.sin_family = sa_family_t(AF_INET)
|
||||
a.sin_port = port.bigEndian
|
||||
a.sin_addr = in_addr(s_addr: INADDR_ANY)
|
||||
bound = withUnsafePointer(to: &a) { p in
|
||||
p.withMemoryRebound(to: sockaddr.self, capacity: 1) { bind(fd, $0, socklen_t(MemoryLayout<sockaddr_in>.size)) }
|
||||
}
|
||||
}
|
||||
if bound != 0 { close(fd); continue }
|
||||
if listen(fd, 16) != 0 { close(fd); continue }
|
||||
return fd
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
/// The local port of a bound socket (the port is at byte offset 2 for both
|
||||
/// IPv4 and IPv6 sockets).
|
||||
static func boundPort(_ fd: Int32) -> UInt16 {
|
||||
var a = sockaddr_storage()
|
||||
var len = socklen_t(MemoryLayout<sockaddr_storage>.size)
|
||||
guard getsockname(fd, &a, &len) == 0 else { return 0 }
|
||||
return withUnsafeBytes(of: &a) { raw in
|
||||
raw.load(fromByteOffset: 2, as: UInt16.self).bigEndian
|
||||
}
|
||||
}
|
||||
|
||||
/// A UDP socket bound to [port] (or 0 for ephemeral) for receiving, with a
|
||||
/// generous receive buffer (the C++ sender's VBV bounds bursts, but a larger
|
||||
/// buffer absorbs a burst on a lossy link).
|
||||
static func makeUdpReceiver(port: UInt16, receiveBufferSize: Int32) -> Int32 {
|
||||
for family in [AF_INET6, AF_INET] {
|
||||
let fd = socket(family, SOCK_DGRAM, 0)
|
||||
guard fd >= 0 else { continue }
|
||||
var yes: Int32 = 1
|
||||
setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &yes, socklen_t(MemoryLayout<Int32>.size))
|
||||
if family == AF_INET6 {
|
||||
var no: Int32 = 0
|
||||
setsockopt(fd, IPPROTO_IPV6, IPV6_V6ONLY, &no, socklen_t(MemoryLayout<Int32>.size))
|
||||
}
|
||||
let bound: Int32
|
||||
if family == AF_INET6 {
|
||||
var a = sockaddr_in6()
|
||||
a.sin6_family = sa_family_t(AF_INET6)
|
||||
a.sin6_port = port.bigEndian
|
||||
a.sin6_addr = in6addr_any
|
||||
bound = withUnsafePointer(to: &a) { p in
|
||||
p.withMemoryRebound(to: sockaddr.self, capacity: 1) { bind(fd, $0, socklen_t(MemoryLayout<sockaddr_in6>.size)) }
|
||||
}
|
||||
} else {
|
||||
var a = sockaddr_in()
|
||||
a.sin_family = sa_family_t(AF_INET)
|
||||
a.sin_port = port.bigEndian
|
||||
a.sin_addr = in_addr(s_addr: INADDR_ANY)
|
||||
bound = withUnsafePointer(to: &a) { p in
|
||||
p.withMemoryRebound(to: sockaddr.self, capacity: 1) { bind(fd, $0, socklen_t(MemoryLayout<sockaddr_in>.size)) }
|
||||
}
|
||||
}
|
||||
if bound != 0 { close(fd); continue }
|
||||
setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &receiveBufferSize, socklen_t(MemoryLayout<Int32>.size))
|
||||
return fd
|
||||
}
|
||||
return -1
|
||||
}
|
||||
}
|
||||
|
||||
/// Newline-delimited JSON signaling server (the receiver side). Keeps the most
|
||||
/// recent connection as its active peer, mirroring the C++ server: `onOffer`
|
||||
/// may answer synchronously (the sender blocks on the answer).
|
||||
final class SignalingServer {
|
||||
private let onOffer: (SignalingMessage) -> Void
|
||||
private let onPli: (SignalingMessage) -> Void
|
||||
private var listenFd: Int32 = -1
|
||||
private(set) var port: UInt16 = 0
|
||||
private var peerFd: Int32 = -1
|
||||
private let peerLock = NSLock()
|
||||
private let assembler = LineAssembler()
|
||||
private var acceptThread: Thread?
|
||||
private var readerThread: Thread?
|
||||
private var running = false
|
||||
|
||||
init(onOffer: @escaping (SignalingMessage) -> Void,
|
||||
onPli: @escaping (SignalingMessage) -> Void) {
|
||||
self.onOffer = onOffer
|
||||
self.onPli = onPli
|
||||
}
|
||||
|
||||
/// Binds the port (SO_REUSEADDR) and starts accepting. Returns true on success.
|
||||
func start(_ preferredPort: UInt16) -> Bool {
|
||||
guard let fd = SocketUtils.makeStreamListener(port: preferredPort), fd >= 0 else { return false }
|
||||
listenFd = fd
|
||||
port = SocketUtils.boundPort(fd)
|
||||
running = true
|
||||
let thread = Thread { [weak self] in self?.acceptLoop() }
|
||||
thread.name = "signaling-accept"
|
||||
acceptThread = thread
|
||||
thread.start()
|
||||
return true
|
||||
}
|
||||
|
||||
/// Sends to the current peer; never throws. A lost control message is
|
||||
/// recoverable — the session re-negotiates or the next keyframe arrives —
|
||||
/// but an exception here would kill the RTP reader thread that reaches
|
||||
/// send() from requestPli(). Mirrors the C++ server, which ignores write
|
||||
/// failures.
|
||||
func send(_ message: SignalingMessage) {
|
||||
guard let bytes = SignalingMessage.serialize(message).data(using: .utf8) else { return }
|
||||
peerLock.lock()
|
||||
let fd = peerFd
|
||||
peerLock.unlock()
|
||||
guard fd >= 0 else { return }
|
||||
bytes.withUnsafeBytes { raw in
|
||||
_ = send(fd, raw.baseAddress, raw.count, Int32(MSG_NOSIGNAL))
|
||||
}
|
||||
}
|
||||
|
||||
func close() {
|
||||
running = false
|
||||
peerLock.lock()
|
||||
let peer = peerFd
|
||||
peerFd = -1
|
||||
peerLock.unlock()
|
||||
if peer >= 0 { close(peer) }
|
||||
if listenFd >= 0 { close(listenFd) }
|
||||
listenFd = -1
|
||||
}
|
||||
|
||||
private func acceptLoop() {
|
||||
while running {
|
||||
var addr = sockaddr()
|
||||
var len = socklen_t(MemoryLayout<sockaddr>.size)
|
||||
let client = accept(listenFd, &addr, &len)
|
||||
if client < 0 {
|
||||
if !running { break }
|
||||
continue
|
||||
}
|
||||
// Most-recent-connection-wins: close the previous peer.
|
||||
peerLock.lock()
|
||||
let old = peerFd
|
||||
peerFd = client
|
||||
peerLock.unlock()
|
||||
if old >= 0 { close(old) }
|
||||
|
||||
let thread = Thread { [weak self] in self?.readLoop(fd: client) }
|
||||
thread.name = "signaling-reader"
|
||||
readerThread = thread
|
||||
thread.start()
|
||||
}
|
||||
}
|
||||
|
||||
private func readLoop(fd: Int32) {
|
||||
assembler.reset()
|
||||
var buffer = [UInt8](repeating: 0, count: 4096)
|
||||
while running {
|
||||
let read = buffer.withUnsafeMutableBytes { raw in
|
||||
recv(fd, raw.baseAddress, raw.count, 0)
|
||||
}
|
||||
guard read > 0 else { break }
|
||||
for line in assembler.feed(Array(buffer.prefix(read))) {
|
||||
dispatch(line)
|
||||
}
|
||||
}
|
||||
// Peer closed; if it is still our active peer, mark it gone.
|
||||
peerLock.lock()
|
||||
if peerFd == fd {
|
||||
peerFd = -1
|
||||
}
|
||||
peerLock.unlock()
|
||||
}
|
||||
|
||||
private func dispatch(_ line: String) {
|
||||
guard let message = SignalingMessage.parse(line) else { return }
|
||||
// A broken callback must not kill the reader thread.
|
||||
switch message {
|
||||
case .offer:
|
||||
onOffer(message)
|
||||
case .pli:
|
||||
onPli(message)
|
||||
case .answer:
|
||||
break // the receiver never receives answers
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
import Foundation
|
||||
#if canImport(Darwin)
|
||||
import Darwin
|
||||
#endif
|
||||
|
||||
/// Finds the primary IPv4 address (e.g. on Wi-Fi) for the status overlay and
|
||||
/// the `--peer` hint. Best-effort; the sender reaches us by IP over the LAN.
|
||||
enum LocalAddress {
|
||||
static func primaryIPv4() -> String {
|
||||
var result = "unknown"
|
||||
var fallback = "unknown"
|
||||
var ptr: UnsafeMutablePointer<ifaddrs>?
|
||||
guard getifaddrs(&ptr) == 0 else { return result }
|
||||
defer { freeifaddrs(ptr) }
|
||||
|
||||
var current = ptr
|
||||
while let iface = current {
|
||||
let next = iface.pointee.ifa_next
|
||||
current = next
|
||||
|
||||
guard let sa = iface.pointee.ifa_addr else { continue }
|
||||
guard sa.pointee.sa_family == sa_family_t(AF_INET) else { continue }
|
||||
if Int32(iface.pointee.ifa_flags) & IFF_LOOPBACK == 0 {
|
||||
let inaddr = sa.assumingMemoryBound(to: sockaddr_in.self).pointee
|
||||
var host = [CChar](repeating: 0, count: Int(INET_ADDRSTRLEN))
|
||||
if inet_ntop(AF_INET, &inaddr.sin_addr, &host, socklen_t(INET_ADDRSTRLEN)) != nil {
|
||||
let ip = String(cString: host)
|
||||
if ip.hasPrefix("169.254") {
|
||||
continue // link-local; prefer a routable address
|
||||
}
|
||||
fallback = ip
|
||||
let name = String(cString: iface.pointee.ifa_name)
|
||||
if name.hasPrefix("en") || name.hasPrefix("wlan") {
|
||||
return ip // Wi-Fi / Ethernet: good enough for the hint
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
import Foundation
|
||||
#if canImport(Darwin)
|
||||
import Darwin
|
||||
#endif
|
||||
|
||||
/// A UDP socket for receiving RTP. Binds to a preferred port (or an ephemeral
|
||||
/// port when it is busy), then delivers complete datagrams one at a time.
|
||||
/// Mirrors the C++ `UdpRtpTransport` receive path and the Kotlin `DatagramSocket`
|
||||
/// reader. Uses poll(2) with a timeout so `close()` from another thread cannot
|
||||
/// strand a blocked recv (a close does not reliably unblock a POSIX recvfrom).
|
||||
final class UdpTransport {
|
||||
private var fd: Int32 = -1
|
||||
private(set) var port: UInt16 = 0
|
||||
|
||||
/// Binds to [preferredPort] (or an ephemeral port when it is busy).
|
||||
@discardableResult
|
||||
func bind(preferredPort: UInt16, receiveBufferSize: Int32) -> Bool {
|
||||
guard fd < 0 else { return true }
|
||||
for p in [preferredPort, UInt16(0)] {
|
||||
if let f = SocketUtils.makeUdpReceiver(port: p, receiveBufferSize: receiveBufferSize), f >= 0 {
|
||||
fd = f
|
||||
port = SocketUtils.boundPort(fd)
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
/// Waits up to [timeoutMs] for a datagram. Returns 1 when one is ready to
|
||||
/// read, 0 on timeout, and -1 when the socket is closed/errored.
|
||||
func poll(timeoutMs: Int32) -> Int32 {
|
||||
guard fd >= 0 else { return -1 }
|
||||
var pfd = pollfd(fd: fd, events: poll_events_t(POLLIN), revents: 0)
|
||||
let r = withUnsafeMutablePointer(to: &pfd) { poll($0, 1, timeoutMs) }
|
||||
if r > 0 {
|
||||
if pfd.revents & poll_events_t(POLLERR) != 0 { return -1 }
|
||||
return 1
|
||||
}
|
||||
return r == 0 ? 0 : -1
|
||||
}
|
||||
|
||||
/// Reads one ready datagram. Returns nil on error or empty read.
|
||||
func receiveDatagram() -> [UInt8]? {
|
||||
guard fd >= 0 else { return nil }
|
||||
var buf = [UInt8](repeating: 0, count: 4096)
|
||||
let read = buf.withUnsafeMutableBytes { raw in
|
||||
recvfrom(fd, raw.baseAddress, raw.count, 0, nil, nil)
|
||||
}
|
||||
guard read > 0 else { return nil }
|
||||
return Array(buf.prefix(read))
|
||||
}
|
||||
|
||||
func close() {
|
||||
if fd >= 0 {
|
||||
close(fd)
|
||||
fd = -1
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import XCTest
|
||||
@testable import Receiver
|
||||
|
||||
final class AvccConverterTests: XCTestCase {
|
||||
func testNalUnits() {
|
||||
let annexB: [UInt8] = [0, 0, 1, 0x41, 0x11, 0, 0, 1, 0x67, 0xAA]
|
||||
XCTAssertEqual(AvccConverter.nalUnits(annexB), [[0x41, 0x11], [0x67, 0xAA]])
|
||||
}
|
||||
|
||||
func testToAvcc() {
|
||||
let annexB: [UInt8] = [0, 0, 1, 0x41, 0x11, 0, 0, 1, 0x67, 0xAA]
|
||||
let avcc = AvccConverter.toAvcc(annexB)
|
||||
XCTAssertEqual(avcc, [0, 0, 0, 2, 0x41, 0x11, 0, 0, 0, 2, 0x67, 0xAA])
|
||||
}
|
||||
|
||||
func testFromAvccRoundtrip() {
|
||||
let units: [[UInt8]] = [[0x41, 0x11], [0x67, 0xAA, 0xBB]]
|
||||
let annexB = [0, 0, 1] + units[0] + [0, 0, 1] + units[1]
|
||||
let avcc = AvccConverter.toAvcc(annexB)
|
||||
XCTAssertEqual(AvccConverter.fromAvcc(avcc ?? []), units)
|
||||
}
|
||||
|
||||
func testEmptyReturnsNil() {
|
||||
XCTAssertNil(AvccConverter.toAvcc([UInt8]()))
|
||||
XCTAssertNil(AvccConverter.toAvcc([0, 0, 1]))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
import XCTest
|
||||
@testable import Receiver
|
||||
|
||||
final class H264DepacketizerTests: XCTestCase {
|
||||
private let start: [UInt8] = [0x00, 0x00, 0x01]
|
||||
|
||||
private func packet(seq: Int, ts: Int, payload: [UInt8], marker: Bool = false) -> RtpPacket {
|
||||
RtpPacket(header: RtpHeader(sequenceNumber: seq, timestamp: ts, marker: marker), payload: payload)
|
||||
}
|
||||
|
||||
func testSingleNalTwoPackets() {
|
||||
let d = H264Depacketizer()
|
||||
// SPS (type 7) then a slice (type 5), closed by the marker.
|
||||
let first = d.depacketize(packet(seq: 1, ts: 100, payload: [0x67, 0xAA, 0xBB]))
|
||||
XCTAssertNil(first.accessUnit)
|
||||
let second = d.depacketize(packet(seq: 2, ts: 100, payload: [0x41, 0x01, 0x02], marker: true))
|
||||
XCTAssertEqual(second.accessUnit, start + [0x67, 0xAA, 0xBB] + start + [0x41, 0x01, 0x02])
|
||||
XCTAssertTrue(second.isKeyFrame)
|
||||
XCTAssertFalse(second.frameDropped)
|
||||
}
|
||||
|
||||
func testFuAReassembly() {
|
||||
let d = H264Depacketizer()
|
||||
// NAL: header 0x41 (type 1, NRI 2) + payload 0x11 0x22 0x33 0x44.
|
||||
// FU indicator = (0x41 & 0xE0) | 28 = 0x5C.
|
||||
let indicator: UInt8 = 0x5C
|
||||
_ = d.depacketize(packet(seq: 1, ts: 100, payload: [indicator, 0x81, 0x11]))
|
||||
_ = d.depacketize(packet(seq: 2, ts: 100, payload: [indicator, 0x01, 0x22]))
|
||||
let done = d.depacketize(packet(seq: 3, ts: 100, payload: [indicator, 0x41, 0x33, 0x44], marker: true))
|
||||
XCTAssertEqual(done.accessUnit, start + [0x41, 0x11, 0x22, 0x33, 0x44])
|
||||
XCTAssertFalse(done.frameDropped)
|
||||
}
|
||||
|
||||
func testDropsGappedFrames() {
|
||||
let d = H264Depacketizer()
|
||||
_ = d.depacketize(packet(seq: 1, ts: 100, payload: [0x41, 0x01]))
|
||||
// seq 2 is missing; the frame must be reported dropped, not delivered.
|
||||
let tail = d.depacketize(packet(seq: 3, ts: 100, payload: [0x41, 0x02], marker: true))
|
||||
XCTAssertNil(tail.accessUnit)
|
||||
XCTAssertTrue(tail.frameDropped)
|
||||
}
|
||||
|
||||
func testSeparateFramesByMarker() {
|
||||
let d = H264Depacketizer()
|
||||
let au1 = d.depacketize(packet(seq: 1, ts: 100, payload: [0x41, 0xAA], marker: true))
|
||||
XCTAssertEqual(au1.accessUnit, start + [0x41, 0xAA])
|
||||
let au2 = d.depacketize(packet(seq: 2, ts: 200, payload: [0x41, 0xBB], marker: true))
|
||||
XCTAssertEqual(au2.accessUnit, start + [0x41, 0xBB])
|
||||
}
|
||||
|
||||
func testDropsFuWithoutStart() {
|
||||
let d = H264Depacketizer()
|
||||
// Continuation (no S bit) without any start packet.
|
||||
let result = d.depacketize(packet(seq: 1, ts: 100, payload: [0x7C, 0x41, 0x11], marker: true))
|
||||
XCTAssertNil(result.accessUnit)
|
||||
XCTAssertTrue(result.frameDropped)
|
||||
}
|
||||
|
||||
func testDropsStillFragmentedAtMarker() {
|
||||
let d = H264Depacketizer()
|
||||
_ = d.depacketize(packet(seq: 1, ts: 100, payload: [0x7C, 0x81, 0x11]))
|
||||
// Marker arrives while the FU-A NAL is still open.
|
||||
let result = d.depacketize(packet(seq: 2, ts: 100, payload: [0x41, 0x01], marker: true))
|
||||
XCTAssertNil(result.accessUnit)
|
||||
XCTAssertTrue(result.frameDropped)
|
||||
}
|
||||
|
||||
func testDropsUnsupportedPacketization() {
|
||||
let d = H264Depacketizer()
|
||||
let stapA: UInt8 = 24 // STAP-A
|
||||
let result = d.depacketize(packet(seq: 1, ts: 100, payload: [stapA, 0x00, 0x05, 0x41, 0x01], marker: true))
|
||||
XCTAssertNil(result.accessUnit)
|
||||
XCTAssertTrue(result.frameDropped)
|
||||
}
|
||||
|
||||
func testDropsTimestampChangeWithoutMarker() {
|
||||
let d = H264Depacketizer()
|
||||
_ = d.depacketize(packet(seq: 1, ts: 100, payload: [0x41, 0x01]))
|
||||
// The stale frame is reported dropped, but this packet starts (and
|
||||
// closes) the next frame — matching the C++ depacketizer.
|
||||
let result = d.depacketize(packet(seq: 2, ts: 200, payload: [0x41, 0x02], marker: true))
|
||||
XCTAssertEqual(result.accessUnit, start + [0x41, 0x02])
|
||||
XCTAssertTrue(result.frameDropped)
|
||||
}
|
||||
|
||||
func testKeyframeDetectionRequiresParameterSets() {
|
||||
let d = H264Depacketizer()
|
||||
let plain = d.depacketize(packet(seq: 1, ts: 100, payload: [0x41, 0x01], marker: true))
|
||||
XCTAssertFalse(plain.isKeyFrame)
|
||||
|
||||
let d2 = H264Depacketizer()
|
||||
let withSps = d2.depacketize(packet(seq: 1, ts: 100, payload: [0x67, 0xAA, 0x88, 0x68, 0xBB, 0x41, 0x01], marker: true))
|
||||
XCTAssertTrue(withSps.isKeyFrame)
|
||||
}
|
||||
|
||||
func testDropsShortFuPackets() {
|
||||
let d = H264Depacketizer()
|
||||
// FU-A packet without its FU header byte.
|
||||
let result = d.depacketize(packet(seq: 1, ts: 100, payload: [0x7C], marker: true))
|
||||
XCTAssertNil(result.accessUnit)
|
||||
XCTAssertTrue(result.frameDropped)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
import XCTest
|
||||
@testable import Receiver
|
||||
|
||||
final class JitterBufferTests: XCTestCase {
|
||||
private func packet(seq: Int, ts: Int = 100) -> RtpPacket {
|
||||
RtpPacket(header: RtpHeader(sequenceNumber: seq, timestamp: ts), payload: [UInt8(seq)])
|
||||
}
|
||||
|
||||
func testInOrderReleasesImmediately() {
|
||||
let jitter = JitterBuffer()
|
||||
XCTAssertEqual(jitter.push(packet(seq: 1)).map { $0.header.sequenceNumber }, [1])
|
||||
XCTAssertEqual(jitter.push(packet(seq: 2)).map { $0.header.sequenceNumber }, [2])
|
||||
XCTAssertEqual(jitter.push(packet(seq: 3)).map { $0.header.sequenceNumber }, [3])
|
||||
}
|
||||
|
||||
func testReordersOutOfOrderPackets() {
|
||||
let jitter = JitterBuffer()
|
||||
XCTAssertEqual(jitter.push(packet(seq: 1)).map { $0.header.sequenceNumber }, [1])
|
||||
XCTAssertTrue(jitter.push(packet(seq: 3)).isEmpty)
|
||||
XCTAssertEqual(jitter.push(packet(seq: 2)).map { $0.header.sequenceNumber }, [2, 3])
|
||||
}
|
||||
|
||||
func testOverflowReleasesInOrderAndAdvances() {
|
||||
let jitter = JitterBuffer(maxDepth: 4)
|
||||
XCTAssertEqual(jitter.push(packet(seq: 1)).map { $0.header.sequenceNumber }, [1])
|
||||
// seq 2 is lost; 3..6 stay buffered (within the depth bound).
|
||||
for seq in 3...6 {
|
||||
XCTAssertTrue(jitter.push(packet(seq: seq)).isEmpty)
|
||||
}
|
||||
// seq 7 overflows the buffer: 3..7 flush in order.
|
||||
XCTAssertEqual(jitter.push(packet(seq: 7)).map { $0.header.sequenceNumber }, [3, 4, 5, 6, 7])
|
||||
// Delivery continues in order afterwards.
|
||||
XCTAssertEqual(jitter.push(packet(seq: 8)).map { $0.header.sequenceNumber }, [8])
|
||||
XCTAssertEqual(jitter.push(packet(seq: 9)).map { $0.header.sequenceNumber }, [9])
|
||||
}
|
||||
|
||||
func testDiscardsStragglers() {
|
||||
let jitter = JitterBuffer(maxDepth: 4)
|
||||
_ = jitter.push(packet(seq: 1))
|
||||
for seq in 3...8 {
|
||||
_ = jitter.push(packet(seq: seq))
|
||||
}
|
||||
XCTAssertTrue(jitter.push(packet(seq: 9)).isNotEmpty)
|
||||
// seq 4 is now far behind the expected sequence: discarded, not delivered.
|
||||
XCTAssertTrue(jitter.push(packet(seq: 4)).isEmpty)
|
||||
// In-order delivery continues from 10.
|
||||
XCTAssertEqual(jitter.push(packet(seq: 10)).map { $0.header.sequenceNumber }, [10])
|
||||
}
|
||||
|
||||
func testClearResetsState() {
|
||||
let jitter = JitterBuffer()
|
||||
_ = jitter.push(packet(seq: 5))
|
||||
jitter.clear()
|
||||
// A completely different sequence now starts fresh.
|
||||
XCTAssertEqual(jitter.push(packet(seq: 100)).map { $0.header.sequenceNumber }, [100])
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
import XCTest
|
||||
@testable import Receiver
|
||||
|
||||
final class LineAssemblerTests: XCTestCase {
|
||||
private func bytes(_ s: String) -> [UInt8] { Array(s.utf8) }
|
||||
|
||||
func testSingleLine() {
|
||||
let a = LineAssembler()
|
||||
XCTAssertEqual(a.feed(bytes("hello")), [])
|
||||
XCTAssertEqual(a.feed(bytes("\n")), ["hello"])
|
||||
}
|
||||
|
||||
func testMultipleLinesInOneChunk() {
|
||||
let a = LineAssembler()
|
||||
XCTAssertEqual(a.feed(bytes("one\ntwo\nthree\n")), ["one", "two", "three"])
|
||||
}
|
||||
|
||||
func testDropsCR() {
|
||||
let a = LineAssembler()
|
||||
XCTAssertEqual(a.feed(bytes("line\r\n")), ["line"])
|
||||
}
|
||||
|
||||
func testIgnoresEmptyLine() {
|
||||
let a = LineAssembler()
|
||||
XCTAssertEqual(a.feed(bytes("\n")), [])
|
||||
}
|
||||
|
||||
func testSplitAcrossChunks() {
|
||||
let a = LineAssembler()
|
||||
let payload = "{\"session_id\":\"x\"}"
|
||||
var lines: [String] = []
|
||||
lines += a.feed(Array(payload.prefix(5)))
|
||||
lines += a.feed(Array(payload.suffix(from: 5)))
|
||||
lines += a.feed(["\n".utf8.first!])
|
||||
XCTAssertEqual(lines, [payload])
|
||||
}
|
||||
|
||||
func testDropsOversizedLine() {
|
||||
let a = LineAssembler()
|
||||
let big = String(repeating: "a", count: SignalingMessage.maxMessageBytes + 1)
|
||||
_ = a.feed(Array(big.utf8))
|
||||
XCTAssertEqual(a.feed(bytes("\n")), [])
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
import XCTest
|
||||
@testable import Receiver
|
||||
|
||||
final class NalExtractorTests: XCTestCase {
|
||||
func testExtractsSpsAndPps() {
|
||||
// Annex-B: SPS (type 7), PPS (type 8), slice (type 5).
|
||||
let annexB: [UInt8] = [0, 0, 1, 0x67, 0xAA, 0, 0, 1, 0x68, 0xBB, 0, 0, 1, 0x41, 0x01]
|
||||
let sets = NalExtractor.parameterSets(annexB)
|
||||
XCTAssertEqual(sets?.sps, [0x67, 0xAA])
|
||||
XCTAssertEqual(sets?.pps, [0x68, 0xBB])
|
||||
}
|
||||
|
||||
func testReturnsNilWithoutBoth() {
|
||||
let annexB: [UInt8] = [0, 0, 1, 0x67, 0xAA, 0, 0, 1, 0x41, 0x01] // SPS but no PPS
|
||||
XCTAssertNil(NalExtractor.parameterSets(annexB))
|
||||
}
|
||||
|
||||
func testPicksFirstOfEach() {
|
||||
let annexB: [UInt8] = [0, 0, 1, 0x67, 0x11, 0, 0, 1, 0x68, 0x22, 0, 0, 1, 0x67, 0x33, 0, 0, 1, 0x68, 0x44]
|
||||
let sets = NalExtractor.parameterSets(annexB)
|
||||
XCTAssertEqual(sets?.sps, [0x67, 0x11])
|
||||
XCTAssertEqual(sets?.pps, [0x68, 0x22])
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
import XCTest
|
||||
@testable import Receiver
|
||||
|
||||
final class RtpHeaderTests: XCTestCase {
|
||||
func testRoundtrip() {
|
||||
let header = RtpHeader(version: 2, padding: false, extensionHeader: false, csrcCount: 0,
|
||||
marker: true, payloadType: 96, sequenceNumber: 0xABCD,
|
||||
timestamp: 0xDEADBEEF, ssrc: 0x12345678)
|
||||
let wire = header.serialize()
|
||||
XCTAssertEqual(wire.count, 12)
|
||||
XCTAssertEqual(RtpHeader.parse(wire), header)
|
||||
}
|
||||
|
||||
func testRejectsBadVersion() {
|
||||
var wire = RtpHeader().serialize()
|
||||
wire[0] = (wire[0] & 0x3F) | (1 << 6)
|
||||
XCTAssertNil(RtpHeader.parse(wire))
|
||||
}
|
||||
|
||||
func testRejectsShortInput() {
|
||||
let header = RtpHeader()
|
||||
XCTAssertNil(RtpHeader.parse(Array(header.serialize().prefix(11))))
|
||||
XCTAssertNil(RtpHeader.parse([]))
|
||||
}
|
||||
|
||||
func testPreservesFlags() {
|
||||
let header = RtpHeader(padding: true, csrcCount: 2, marker: true, payloadType: 63)
|
||||
let parsed = RtpHeader.parse(header.serialize())
|
||||
XCTAssertEqual(parsed?.padding, true)
|
||||
XCTAssertEqual(parsed?.csrcCount, 2)
|
||||
XCTAssertEqual(parsed?.marker, true)
|
||||
XCTAssertEqual(parsed?.payloadType, 63)
|
||||
}
|
||||
}
|
||||
|
||||
final class RtpPacketTests: XCTestCase {
|
||||
private func header(seq: Int, marker: Bool = false) -> RtpHeader {
|
||||
RtpHeader(sequenceNumber: seq, payloadType: 96, marker: marker)
|
||||
}
|
||||
|
||||
func testRoundtripWithPayload() {
|
||||
let packet = RtpPacket(header: header(seq: 7), payload: [0x11, 0x22, 0x33])
|
||||
let wire = packet.header.serialize() + packet.payload
|
||||
let parsed = RtpPacket.parse(wire)
|
||||
XCTAssertEqual(parsed?.header, header(seq: 7))
|
||||
XCTAssertEqual(parsed?.payload, [0x11, 0x22, 0x33])
|
||||
}
|
||||
|
||||
func testSkipsCsrcList() {
|
||||
let header = RtpHeader(csrcCount: 1, sequenceNumber: 3)
|
||||
let wire = header.serialize() + [0x0A, 0x00, 0x00, 0x01] + [0x99]
|
||||
let parsed = RtpPacket.parse(wire)
|
||||
XCTAssertEqual(parsed?.header.csrcCount, 1)
|
||||
XCTAssertEqual(parsed?.payload, [0x99])
|
||||
}
|
||||
|
||||
func testSkipsExtensionHeader() {
|
||||
let header = RtpHeader(extensionHeader: true, sequenceNumber: 4)
|
||||
// profile=0x0001, length=1 word, one word of data.
|
||||
let wire = header.serialize() + [0x00, 0x01, 0x00, 0x01, 0xDE, 0xAD, 0xBE, 0xEF] + [0x77]
|
||||
let parsed = RtpPacket.parse(wire)
|
||||
XCTAssertEqual(parsed?.payload, [0x77])
|
||||
}
|
||||
|
||||
func testStripsPadding() {
|
||||
let header = RtpHeader(padding: true, sequenceNumber: 5)
|
||||
// Payload byte, one padding zero, size byte (2 = padding incl. itself).
|
||||
let wire = header.serialize() + [0x55, 0x00, 0x02]
|
||||
let parsed = RtpPacket.parse(wire)
|
||||
XCTAssertEqual(parsed?.payload, [0x55])
|
||||
}
|
||||
|
||||
func testRejectsTruncatedCsrcAndExtension() {
|
||||
let csrc = RtpHeader(csrcCount: 1).serialize()
|
||||
XCTAssertNil(RtpPacket.parse(csrc)) // 12 bytes, needs 16
|
||||
let ext = RtpHeader(extensionHeader: true).serialize() + [0x00, 0x01]
|
||||
XCTAssertNil(RtpPacket.parse(ext)) // extension length field cut off
|
||||
}
|
||||
|
||||
func testRejectsBadPadding() {
|
||||
let zeroPad = RtpHeader(padding: true).serialize() + [0x00]
|
||||
XCTAssertNil(RtpPacket.parse(zeroPad))
|
||||
let oversized = RtpHeader(padding: true).serialize() + [0x00, 0x00, 0x05]
|
||||
XCTAssertNil(RtpPacket.parse(oversized))
|
||||
XCTAssertNil(RtpPacket.parse([UInt8](repeating: 0, count: 11)))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
import XCTest
|
||||
@testable import Receiver
|
||||
|
||||
final class SignalingMessageTests: XCTestCase {
|
||||
func testOfferRoundtrip() {
|
||||
let offer = SignalingMessage.offer(sessionId: "s-1", codec: "h264", width: 0, height: 0,
|
||||
frameRateNum: 30, frameRateDen: 1, rtpAddress: "10.0.0.5", rtpPort: 1234)
|
||||
let line = SignalingMessage.serialize(offer).trimmingCharacters(in: .newlines)
|
||||
let parsed = SignalingMessage.parse(line)
|
||||
guard case let .offer(sid, codec, w, h, num, den, addr, port) = parsed else { return XCTFail() }
|
||||
XCTAssertEqual(sid, "s-1")
|
||||
XCTAssertEqual(codec, "h264")
|
||||
XCTAssertEqual(w, 0)
|
||||
XCTAssertEqual(h, 0)
|
||||
XCTAssertEqual(num, 30)
|
||||
XCTAssertEqual(den, 1)
|
||||
XCTAssertEqual(addr, "10.0.0.5")
|
||||
XCTAssertEqual(port, 1234)
|
||||
}
|
||||
|
||||
func testAnswerRoundtrip() {
|
||||
let answer = SignalingMessage.answer(sessionId: "s-2", rtpAddress: "", rtpPort: 5004,
|
||||
displayWidth: 1179, displayHeight: 2556)
|
||||
let line = SignalingMessage.serialize(answer).trimmingCharacters(in: .newlines)
|
||||
let parsed = SignalingMessage.parse(line)
|
||||
guard case let .answer(sid, addr, port, dw, dh) = parsed else { return XCTFail() }
|
||||
XCTAssertEqual(sid, "s-2")
|
||||
XCTAssertEqual(addr, "")
|
||||
XCTAssertEqual(port, 5004)
|
||||
XCTAssertEqual(dw, 1179)
|
||||
XCTAssertEqual(dh, 2556)
|
||||
}
|
||||
|
||||
func testPliRoundtrip() {
|
||||
let pli = SignalingMessage.pli(sessionId: "s-3")
|
||||
let line = SignalingMessage.serialize(pli).trimmingCharacters(in: .newlines)
|
||||
let parsed = SignalingMessage.parse(line)
|
||||
guard case let .pli(sid) = parsed else { return XCTFail() }
|
||||
XCTAssertEqual(sid, "s-3")
|
||||
}
|
||||
|
||||
func testRejectsUnknownType() {
|
||||
XCTAssertNil(SignalingMessage.parse("{\"type\":\"bogus\"}"))
|
||||
}
|
||||
|
||||
func testRejectsInvalidJson() {
|
||||
XCTAssertNil(SignalingMessage.parse("not json"))
|
||||
}
|
||||
|
||||
func testRejectsOversizedLine() {
|
||||
let big = String(repeating: "a", count: SignalingMessage.maxMessageBytes + 1)
|
||||
XCTAssertNil(SignalingMessage.parse(big))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Bootstrap for the iOS receiver. Intended to be run on a Mac with Xcode 26.
|
||||
# It installs XcodeGen (if absent) from the GitHub release (no Homebrew needed),
|
||||
# generates the Xcode project, and builds/tests.
|
||||
#
|
||||
# Usage:
|
||||
# ./bootstrap.sh # generate + build for device
|
||||
# ./bootstrap.sh test # generate + run the unit tests (simulator)
|
||||
# ./bootstrap.sh build-sim # generate + build for the simulator
|
||||
# ./bootstrap.sh generate # just generate the project
|
||||
#
|
||||
# Device install/signing is intentionally left to Xcode: open
|
||||
# Receiver.xcodeproj, set your Apple ID on the Receiver target, and hit Run.
|
||||
set -euo pipefail
|
||||
|
||||
IOS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
TOOLS_DIR="$IOS_DIR/tools"
|
||||
XCODEGEN_BIN="$TOOLS_DIR/xcodegen"
|
||||
PROJ="$IOS_DIR/Receiver.xcodeproj"
|
||||
|
||||
# 1. Xcode
|
||||
if ! command -v xcodebuild >/dev/null 2>&1; then
|
||||
echo "error: xcodebuild not found. Install Xcode (26.x) and select its CLI tools." >&2
|
||||
echo " xcode-select --install (or: sudo xcode-select -s /Applications/Xcode.app/Contents/Developer)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 2. XcodeGen: prefer a system copy, then a local install, else download latest.
|
||||
if ! command -v xcodegen >/dev/null 2>&1 && [ ! -x "$XCODEGEN_BIN" ]; then
|
||||
echo "Installing XcodeGen into $TOOLS_DIR ..."
|
||||
mkdir -p "$TOOLS_DIR"
|
||||
url="https://github.com/yonaskolb/XcodeGen/releases/latest/download/xcodegen.zip"
|
||||
tmp="$(mktemp -d)"
|
||||
curl -fL "$url" -o "$tmp/xcodegen.zip"
|
||||
unzip -o -q "$tmp/xcodegen.zip" -d "$TOOLS_DIR"
|
||||
# The release zip extracts to a flat binary named "xcodegen".
|
||||
if [ ! -x "$XCODEGEN_BIN" ] && [ -f "$TOOLS_DIR/xcodegen" ]; then
|
||||
chmod +x "$XCODEGEN_BIN"
|
||||
fi
|
||||
rm -rf "$tmp"
|
||||
fi
|
||||
XCODEGEN="$(command -v xcodegen || echo "$XCODEGEN_BIN")"
|
||||
if [ ! -x "$XCODEGEN" ] && [ ! -x "$XCODEGEN_BIN" ]; then
|
||||
echo "error: XcodeGen not found and download failed." >&2
|
||||
exit 1
|
||||
fi
|
||||
XCODEGEN="${XCODEGEN_BIN}"
|
||||
|
||||
generate() {
|
||||
"$XCODEGEN_BIN" generate --spec "$IOS_DIR/project.yml" --project "$IOS_DIR"
|
||||
echo "Generated $PROJ"
|
||||
}
|
||||
|
||||
dest_sim() { echo "${IOS_DEST:-platform=iOS Simulator,name=iPhone 16}"; }
|
||||
|
||||
command="${1:-build}"
|
||||
generate
|
||||
|
||||
case "$command" in
|
||||
generate)
|
||||
;;
|
||||
build)
|
||||
xcodebuild -project "$PROJ" -scheme Receiver -destination "generic/platform=iOS" build
|
||||
;;
|
||||
build-sim)
|
||||
xcodebuild -project "$PROJ" -scheme Receiver -destination "$(dest_sim)" build
|
||||
;;
|
||||
test)
|
||||
xcodebuild test -project "$PROJ" -scheme Receiver -destination "$(dest_sim)" \
|
||||
-only-testing:ReceiverTests
|
||||
;;
|
||||
*)
|
||||
echo "usage: bootstrap.sh [generate|build|build-sim|test]" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo "Done."
|
||||
@@ -0,0 +1,71 @@
|
||||
name: Receiver
|
||||
options:
|
||||
bundleIdPrefix: screencast
|
||||
deploymentTarget:
|
||||
iOS: "17.0"
|
||||
createIntermediateGroups: true
|
||||
generateEmptySchemes: false
|
||||
|
||||
targets:
|
||||
Receiver:
|
||||
type: application
|
||||
platform: iOS
|
||||
deploymentTarget: "17.0"
|
||||
sources:
|
||||
- path: Receiver
|
||||
settings:
|
||||
base:
|
||||
PRODUCT_BUNDLE_IDENTIFIER: screencast.receiver
|
||||
PRODUCT_NAME: Receiver
|
||||
TARGETED_DEVICE_FAMILY: 1
|
||||
CODE_SIGN_STYLE: Automatic
|
||||
DEVELOPMENT_TEAM: ""
|
||||
IPHONEOS_DEPLOYMENT_TARGET: "17.0"
|
||||
ENABLE_USER_SCRIPT_SANDBOXING: YES
|
||||
info:
|
||||
path: Receiver/Info.plist
|
||||
properties:
|
||||
CFBundleDisplayName: screencast
|
||||
CFBundleName: screencast
|
||||
CFBundleShortVersionString: "0.9.0"
|
||||
CFBundleVersion: "1"
|
||||
UILaunchScreen: {}
|
||||
UISupportedInterfaceOrientations:
|
||||
- UIInterfaceOrientationLandscapeLeft
|
||||
- UIInterfaceOrientationLandscapeRight
|
||||
UISupportedInterfaceOrientations~ipad:
|
||||
- UIInterfaceOrientationLandscapeLeft
|
||||
- UIInterfaceOrientationLandscapeRight
|
||||
- UIInterfaceOrientationPortrait
|
||||
- UIInterfaceOrientationPortraitUpsideDown
|
||||
# Local-network privacy: required for both Bonjour and the TCP/UDP
|
||||
# sockets. Without NSBonjourServices the sender never resolves us.
|
||||
NSLocalNetworkUsageDescription: "screencast receives a video stream on your local network."
|
||||
NSBonjourServices:
|
||||
- "_screencast._tcp"
|
||||
|
||||
ReceiverTests:
|
||||
type: bundle.unit-test
|
||||
platform: iOS
|
||||
deploymentTarget: "17.0"
|
||||
sources:
|
||||
- path: ReceiverTests
|
||||
dependencies:
|
||||
- target: Receiver
|
||||
settings:
|
||||
base:
|
||||
PRODUCT_BUNDLE_IDENTIFIER: screencast.receiver.tests
|
||||
BUNDLE_LOADER: "$(TEST_HOST)"
|
||||
TEST_HOST: "$(BUILT_PRODUCTS_DIR)/Receiver.app/Receiver"
|
||||
|
||||
schemes:
|
||||
Receiver:
|
||||
build:
|
||||
targets:
|
||||
Receiver: all
|
||||
test:
|
||||
config: Debug
|
||||
targets:
|
||||
- ReceiverTests
|
||||
archive:
|
||||
enabled: false
|
||||
@@ -10,6 +10,11 @@ project('screen_cast', 'cpp',
|
||||
# Public and private include directories are declared in `src/meson.build`.
|
||||
subdir('src')
|
||||
|
||||
# The GUI is a sender panel and needs the capture backend.
|
||||
if get_option('gui') and not get_option('sender')
|
||||
error('The GUI is a sender panel and requires -Dsender=true')
|
||||
endif
|
||||
|
||||
# Manual smoke tools are sender-side.
|
||||
if get_option('sender')
|
||||
subdir('tools')
|
||||
@@ -21,7 +26,13 @@ if enable_tests
|
||||
subdir('tests')
|
||||
endif
|
||||
|
||||
# Install the application icon into the hicolor theme: the waybar widget CSS
|
||||
# and any future .desktop file reference it.
|
||||
install_data('screencast_icon/screencast_256.png',
|
||||
install_dir : get_option('datadir') / 'icons/hicolor/256x256/apps',
|
||||
rename : 'screencast.png')
|
||||
|
||||
# Summary for the user
|
||||
summary({
|
||||
'tests': enable_tests,
|
||||
}, section: 'Build options')
|
||||
}, section: 'Build options')
|
||||
|
After Width: | Height: | Size: 4.4 KiB |
|
After Width: | Height: | Size: 2.7 KiB |
|
After Width: | Height: | Size: 6.2 KiB |
|
After Width: | Height: | Size: 10 KiB |
|
After Width: | Height: | Size: 16 KiB |
|
After Width: | Height: | Size: 398 KiB |
|
After Width: | Height: | Size: 9.1 KiB |
|
After Width: | Height: | Size: 25 KiB |
|
After Width: | Height: | Size: 1.7 KiB |
|
After Width: | Height: | Size: 2.7 KiB |
|
After Width: | Height: | Size: 84 KiB |
|
After Width: | Height: | Size: 3.8 KiB |
@@ -0,0 +1,39 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate a C++ header that embeds an image file as a byte array.
|
||||
|
||||
Usage: icon_to_header.py INPUT_IMAGE OUTPUT_HEADER
|
||||
"""
|
||||
|
||||
import sys
|
||||
|
||||
|
||||
def main() -> int:
|
||||
if len(sys.argv) != 3:
|
||||
print(f"usage: {sys.argv[0]} INPUT_IMAGE OUTPUT_HEADER", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
with open(sys.argv[1], "rb") as src:
|
||||
data = src.read()
|
||||
|
||||
words = ", ".join(f"0x{byte:02x}" for byte in data)
|
||||
header = f"""// GENERATED FILE - do not edit by hand.
|
||||
// Produced by scripts/icon_to_header.py from screencast_icon/screencast_256.png.
|
||||
|
||||
#pragma once
|
||||
|
||||
#include <array>
|
||||
#include <cstdint>
|
||||
|
||||
namespace sc {{
|
||||
inline constexpr std::array<std::uint8_t, {len(data)}> app_icon_png = {{
|
||||
{words}
|
||||
}};
|
||||
}} // namespace sc
|
||||
"""
|
||||
with open(sys.argv[2], "w") as dst:
|
||||
dst.write(header)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -8,9 +8,11 @@ namespace sc {
|
||||
namespace {
|
||||
|
||||
void print_usage() {
|
||||
std::fputs("usage: screencast --send [--target monitor|window] [--peer HOST[:PORT]] [--bitrate KBPS]\n"
|
||||
" screencast --receive [--port PORT] [--signaling-port PORT] [--fullscreen]\n"
|
||||
std::fputs("usage: screencast --send [--target monitor|window] [--peer HOST[:PORT]] [--bitrate MAX_KBPS] [--crf "
|
||||
"0-51] [--fps 1-60]\n"
|
||||
" screencast --receive [--port PORT] [--signaling-port PORT] [--fullscreen] [--swdecode]\n"
|
||||
" screencast --discover [--timeout SECONDS]\n"
|
||||
" screencast waybar [--toggle] # for waybar widgets\n"
|
||||
"\n"
|
||||
"--send without --peer discovers a receiver on the LAN and requires\n"
|
||||
"that exactly one is found.\n",
|
||||
@@ -39,12 +41,14 @@ std::optional<Command> parse_cli(int argc, const char* const argv[]) {
|
||||
Send,
|
||||
Receive,
|
||||
Discover,
|
||||
Waybar,
|
||||
};
|
||||
|
||||
Mode mode = Mode::None;
|
||||
SendCommand send;
|
||||
ReceiveCommand receive;
|
||||
DiscoverCommand discover;
|
||||
WaybarCommand waybar;
|
||||
|
||||
for (int index = 1; index < argc; ++index) {
|
||||
const std::string_view argument = argv[index];
|
||||
@@ -67,6 +71,14 @@ std::optional<Command> parse_cli(int argc, const char* const argv[]) {
|
||||
return std::nullopt;
|
||||
}
|
||||
mode = Mode::Discover;
|
||||
} else if (argument == "waybar") {
|
||||
if (mode != Mode::None) {
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
mode = Mode::Waybar;
|
||||
} else if (argument == "--toggle") {
|
||||
waybar.toggle = true;
|
||||
} else if (argument == "--target") {
|
||||
std::string_view value;
|
||||
if (!next_argument(argc, argv, index, value)) {
|
||||
@@ -94,6 +106,20 @@ std::optional<Command> parse_cli(int argc, const char* const argv[]) {
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
} else if (argument == "--crf") {
|
||||
std::string_view value;
|
||||
if (!next_argument(argc, argv, index, value) || !parse_int(value, send.crf) || send.crf < 0 ||
|
||||
send.crf > 51) {
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
} else if (argument == "--fps") {
|
||||
std::string_view value;
|
||||
if (!next_argument(argc, argv, index, value) || !parse_int(value, send.fps) || send.fps < 0 ||
|
||||
send.fps > 60) {
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
} else if (argument == "--port") {
|
||||
std::string_view value;
|
||||
int port = 0;
|
||||
@@ -112,6 +138,8 @@ std::optional<Command> parse_cli(int argc, const char* const argv[]) {
|
||||
receive.signaling_port = port;
|
||||
} else if (argument == "--fullscreen") {
|
||||
receive.fullscreen = true;
|
||||
} else if (argument == "--swdecode") {
|
||||
receive.software_decode = true;
|
||||
} else if (argument == "--timeout") {
|
||||
std::string_view value;
|
||||
if (!next_argument(argc, argv, index, value) || !parse_int(value, discover.timeout_seconds) ||
|
||||
@@ -134,6 +162,9 @@ std::optional<Command> parse_cli(int argc, const char* const argv[]) {
|
||||
if (mode == Mode::Discover) {
|
||||
return Command{std::move(discover)};
|
||||
}
|
||||
if (mode == Mode::Waybar) {
|
||||
return Command{std::move(waybar)};
|
||||
}
|
||||
print_usage();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
@@ -1,9 +1,17 @@
|
||||
#include "screencast/app/cli.h"
|
||||
#include "screencast/app/pipeline.h"
|
||||
|
||||
#include "sender_session.h"
|
||||
#include "state_store.h"
|
||||
|
||||
#include "screencast/network/discovery.h"
|
||||
#include "screencast/network/signaling.h"
|
||||
|
||||
#include <nlohmann/json.hpp>
|
||||
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <atomic>
|
||||
#include <charconv>
|
||||
@@ -53,49 +61,6 @@ sc::Endpoint parse_endpoint(std::string_view address, std::uint16_t default_port
|
||||
|
||||
#endif // SC_HAS_SENDER
|
||||
|
||||
std::string make_session_id() {
|
||||
static std::mt19937 engine{std::random_device{}()};
|
||||
std::string id;
|
||||
for (int i = 0; i < 16; ++i) {
|
||||
id += "0123456789abcdef"[static_cast<std::size_t>(engine()) & 0xF];
|
||||
}
|
||||
return id;
|
||||
}
|
||||
|
||||
bool is_private_ipv4(std::string_view host) {
|
||||
if (host.rfind("192.168.", 0) == 0 || host.rfind("10.", 0) == 0) {
|
||||
return true;
|
||||
}
|
||||
if (host.rfind("172.", 0) == 0) {
|
||||
const std::size_t second = host.find('.', 5);
|
||||
if (second != std::string_view::npos) {
|
||||
const int octet = std::stoi(std::string{host.substr(5, second - 5)});
|
||||
return octet >= 16 && octet <= 31;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// Ordering for trying a receiver's addresses: private IPv4 first (LANs,
|
||||
// most reliable), then public IPv4, ULA, and global IPv6. 6to4 (2002::) and
|
||||
// link-local addresses last: 6to4 is frequently unreachable between LAN
|
||||
// peers, and link-local needs a scope id to even route.
|
||||
int address_preference(std::string_view host) {
|
||||
if (host.find(':') == std::string_view::npos) {
|
||||
return is_private_ipv4(host) ? 0 : 1;
|
||||
}
|
||||
if (host.rfind("fd", 0) == 0 || host.rfind("fc", 0) == 0) {
|
||||
return 2;
|
||||
}
|
||||
if (host.rfind("2002:", 0) == 0) {
|
||||
return 4;
|
||||
}
|
||||
if (host.rfind("fe80:", 0) == 0) {
|
||||
return 5;
|
||||
}
|
||||
return 3;
|
||||
}
|
||||
|
||||
std::vector<sc::DiscoveredPeer> discover_peers(int timeout_seconds, std::string& error) {
|
||||
std::vector<sc::DiscoveredPeer> peers;
|
||||
std::mutex mutex;
|
||||
@@ -140,155 +105,172 @@ std::vector<sc::DiscoveredPeer> discover_peers(int timeout_seconds, std::string&
|
||||
|
||||
#ifdef SC_HAS_SENDER
|
||||
|
||||
// Offer, wait for the answer, and stream to the negotiated endpoint. The
|
||||
// channel must already be connected.
|
||||
int negotiate_and_stream(sc::SignalingChannel& channel, const sc::Endpoint& signaling, const sc::SendCommand& command) {
|
||||
std::promise<sc::SessionAnswer> answer_promise;
|
||||
auto answer_future = answer_promise.get_future();
|
||||
std::atomic<bool> answered{false};
|
||||
channel.on_message([&](const sc::SignalingMessage& message) {
|
||||
if (const sc::SessionAnswer* answer = std::get_if<sc::SessionAnswer>(&message)) {
|
||||
if (!answered.exchange(true)) {
|
||||
answer_promise.set_value(*answer);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
sc::SessionOffer offer;
|
||||
offer.session_id = make_session_id();
|
||||
offer.codec_name = "h264";
|
||||
offer.frame_rate_num = 25;
|
||||
offer.frame_rate_den = 1;
|
||||
channel.send(offer);
|
||||
|
||||
if (answer_future.wait_for(std::chrono::seconds(5)) != std::future_status::ready) {
|
||||
std::cerr << "screencast: the receiver did not answer the session offer\n";
|
||||
channel.disconnect();
|
||||
return 1;
|
||||
}
|
||||
const sc::SessionAnswer answer = answer_future.get();
|
||||
channel.disconnect();
|
||||
|
||||
if (answer.session_id != offer.session_id) {
|
||||
std::cerr << "screencast: session mismatch in the receiver's answer\n";
|
||||
return 1;
|
||||
}
|
||||
|
||||
// Stream to the negotiated RTP endpoint. An empty address means
|
||||
// "the address you reached me on".
|
||||
const sc::Endpoint rtp_endpoint = answer.rtp_endpoint.address.empty()
|
||||
? sc::Endpoint{signaling.address, answer.rtp_endpoint.port}
|
||||
: answer.rtp_endpoint;
|
||||
|
||||
sc::SenderPipelineConfig config;
|
||||
if (command.target == "window") {
|
||||
config.capture_target = sc::CaptureTargetWindow{};
|
||||
} else {
|
||||
config.capture_target = sc::CaptureTargetWholeScreen{};
|
||||
}
|
||||
config.peer_rtp_endpoint = rtp_endpoint;
|
||||
config.encoder.bitrate_kbps = command.bitrate_kbps;
|
||||
|
||||
std::cout << std::format("screencast: session {} established; streaming to {}:{}\n",
|
||||
offer.session_id,
|
||||
rtp_endpoint.address,
|
||||
rtp_endpoint.port);
|
||||
|
||||
sc::SenderPipeline pipeline{std::move(config)};
|
||||
if (!pipeline.start()) {
|
||||
return 1;
|
||||
}
|
||||
while (!g_interrupted.load()) {
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
pipeline.stop();
|
||||
return 0;
|
||||
}
|
||||
|
||||
int run_sender(const sc::SendCommand& command) {
|
||||
// Find the receiver's signaling endpoint: explicit --peer, or discover
|
||||
// exactly one receiver on the LAN. A receiver may resolve to several
|
||||
// addresses; try them in reachability order until the signaling
|
||||
// connection succeeds.
|
||||
auto channel_result = sc::SignalingFactory::create_client();
|
||||
if (sc::is_network_error(channel_result)) {
|
||||
std::cerr << std::format("screencast: {}\n", sc::network_error(channel_result).message);
|
||||
return 1;
|
||||
}
|
||||
auto channel = std::move(sc::network_value(channel_result));
|
||||
|
||||
sc::Endpoint signaling;
|
||||
sc::CaptureTarget target = sc::CaptureTargetWholeScreen{};
|
||||
if (command.target == "window") {
|
||||
target = sc::CaptureTargetWindow{};
|
||||
}
|
||||
|
||||
if (!command.peer_address.empty()) {
|
||||
signaling = parse_endpoint(command.peer_address, kDefaultSignalingPort);
|
||||
if (!channel->connect(signaling)) {
|
||||
std::cerr << std::format(
|
||||
"screencast: failed to connect to the receiver at {}:{}\n", signaling.address, signaling.port);
|
||||
} else {
|
||||
std::string error;
|
||||
std::vector<sc::DiscoveredPeer> peers = discover_peers(kSenderDiscoveryTimeoutSeconds, error);
|
||||
if (!error.empty()) {
|
||||
std::cerr << std::format("screencast: discovery failed: {}\n", error);
|
||||
return 1;
|
||||
}
|
||||
return negotiate_and_stream(*channel, signaling, command);
|
||||
}
|
||||
|
||||
std::string error;
|
||||
std::vector<sc::DiscoveredPeer> peers = discover_peers(kSenderDiscoveryTimeoutSeconds, error);
|
||||
if (!error.empty()) {
|
||||
std::cerr << std::format("screencast: discovery failed: {}\n", error);
|
||||
return 1;
|
||||
}
|
||||
if (peers.empty()) {
|
||||
std::cerr << "screencast: no receiver found on the LAN; run 'screencast --receive' on the "
|
||||
"target machine, or pass --peer\n";
|
||||
return 1;
|
||||
}
|
||||
|
||||
// Group the addresses by receiver (name + signaling port): one entry
|
||||
// per address, but they are all the same host.
|
||||
std::map<std::pair<std::string, std::uint16_t>, std::vector<std::string>> receivers;
|
||||
for (const sc::DiscoveredPeer& peer : peers) {
|
||||
receivers[{peer.service_name, peer.signaling_port}].push_back(peer.host);
|
||||
}
|
||||
if (receivers.size() > 1) {
|
||||
for (const auto& [key, hosts] : receivers) {
|
||||
std::cerr << std::format(
|
||||
"screencast: {} at {}\n",
|
||||
key.first,
|
||||
std::accumulate(hosts.begin(), hosts.end(), std::string{}, [](std::string lhs, const std::string& rhs) {
|
||||
return lhs.empty() ? rhs : lhs + ", " + rhs;
|
||||
}));
|
||||
if (peers.empty()) {
|
||||
std::cerr << "screencast: no receiver found on the LAN; run 'screencast --receive' on the "
|
||||
"target machine, or pass --peer\n";
|
||||
return 1;
|
||||
}
|
||||
std::cerr << "screencast: multiple receivers found; pass --peer to choose one\n";
|
||||
return 1;
|
||||
}
|
||||
|
||||
auto [name, port] = receivers.begin()->first;
|
||||
std::vector<std::string> hosts = receivers.begin()->second;
|
||||
std::sort(hosts.begin(), hosts.end(), [](const std::string& lhs, const std::string& rhs) {
|
||||
return address_preference(lhs) < address_preference(rhs);
|
||||
});
|
||||
std::cout << std::format("screencast: found receiver '{}'\n", name);
|
||||
|
||||
bool connected = false;
|
||||
std::string tried;
|
||||
for (const std::string& host : hosts) {
|
||||
if (channel->connect(sc::Endpoint{host, port})) {
|
||||
signaling = sc::Endpoint{host, port};
|
||||
connected = true;
|
||||
break;
|
||||
// Group the addresses by receiver (name + signaling port): one entry
|
||||
// per address, but they are all the same host.
|
||||
std::map<std::pair<std::string, std::uint16_t>, std::vector<std::string>> receivers;
|
||||
for (const sc::DiscoveredPeer& peer : peers) {
|
||||
receivers[{peer.service_name, peer.signaling_port}].push_back(peer.host);
|
||||
}
|
||||
tried += (tried.empty() ? "" : ", ") + host;
|
||||
if (receivers.size() > 1) {
|
||||
for (const auto& [key, hosts] : receivers) {
|
||||
std::cerr << std::format(
|
||||
"screencast: {} at {}\n",
|
||||
key.first,
|
||||
std::accumulate(
|
||||
hosts.begin(), hosts.end(), std::string{}, [](std::string lhs, const std::string& rhs) {
|
||||
return lhs.empty() ? rhs : lhs + ", " + rhs;
|
||||
}));
|
||||
}
|
||||
std::cerr << "screencast: multiple receivers found; pass --peer to choose one\n";
|
||||
return 1;
|
||||
}
|
||||
|
||||
auto [name, port] = receivers.begin()->first;
|
||||
std::vector<std::string> hosts = receivers.begin()->second;
|
||||
std::sort(hosts.begin(), hosts.end(), [](const std::string& lhs, const std::string& rhs) {
|
||||
return sc::address_preference(lhs) < sc::address_preference(rhs);
|
||||
});
|
||||
std::cout << std::format("screencast: found receiver '{}'\n", name);
|
||||
signaling = sc::Endpoint{hosts.front(), port};
|
||||
}
|
||||
if (!connected) {
|
||||
std::cerr << std::format("screencast: could not reach the receiver (tried {})\n", tried);
|
||||
|
||||
auto session_result = sc::SenderSession::start(signaling, command.bitrate_kbps, command.crf, command.fps, target);
|
||||
if (auto* error = std::get_if<std::string>(&session_result)) {
|
||||
std::cerr << std::format("screencast: {}\n", *error);
|
||||
return 1;
|
||||
}
|
||||
return negotiate_and_stream(*channel, signaling, command);
|
||||
auto session = std::move(std::get<sc::SenderSession>(session_result));
|
||||
std::cout << std::format("screencast: session {} streaming to {}\n", session.session_id(), session.receiver());
|
||||
|
||||
while (!g_interrupted.load()) {
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
session.stop();
|
||||
return 0;
|
||||
}
|
||||
|
||||
#endif // SC_HAS_SENDER
|
||||
|
||||
#ifdef SC_HAS_SENDER
|
||||
void spawn_detached_sender(const std::string& peer, int bitrate_kbps) {
|
||||
char self_path[4096] = {};
|
||||
const ssize_t length = readlink("/proc/self/exe", self_path, sizeof(self_path) - 1);
|
||||
if (length <= 0) {
|
||||
return;
|
||||
}
|
||||
self_path[length] = '\0';
|
||||
|
||||
const pid_t child = fork();
|
||||
if (child != 0) {
|
||||
return; // parent returns immediately; the child streams detached
|
||||
}
|
||||
setsid();
|
||||
(void)freopen("/dev/null", "w", stdout);
|
||||
const std::string bitrate = std::to_string(bitrate_kbps);
|
||||
(void)execl(self_path,
|
||||
"screencast",
|
||||
"--send",
|
||||
"--peer",
|
||||
peer.c_str(),
|
||||
"--bitrate",
|
||||
bitrate.c_str(),
|
||||
static_cast<char*>(nullptr));
|
||||
_exit(127);
|
||||
}
|
||||
#endif
|
||||
|
||||
void print_waybar_status() {
|
||||
const auto state = sc::read_sender_state();
|
||||
|
||||
// Built with nlohmann::json so all escaping (newlines in tooltips,
|
||||
// non-ASCII characters) is handled correctly. Hand-rolled format
|
||||
// strings produced literal control characters that broke waybar's
|
||||
// JSON parser.
|
||||
nlohmann::json json = nlohmann::json::object();
|
||||
|
||||
if (!state.has_value()) {
|
||||
json["text"] = "\u23f8"; // pause symbol
|
||||
json["alt"] = "idle";
|
||||
json["class"] = "idle";
|
||||
json["tooltip"] = "screencast idle \u2014 click to stream to the last receiver\nright-click: open the panel";
|
||||
std::cout << json.dump() << '\n';
|
||||
return;
|
||||
}
|
||||
|
||||
const std::int64_t elapsed_ms =
|
||||
std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch())
|
||||
.count() -
|
||||
state->started_epoch_ms;
|
||||
const std::int64_t minutes = elapsed_ms / 60000;
|
||||
const std::int64_t seconds = (elapsed_ms / 1000) % 60;
|
||||
|
||||
json["text"] = "\u25b6"; // play symbol
|
||||
json["alt"] = "streaming";
|
||||
json["class"] = "streaming";
|
||||
json["tooltip"] = std::format("screencast \u2192 {}\n{} kbps \u00b7 {}m {:02}s\nsession {}",
|
||||
state->receiver,
|
||||
state->bitrate_kbps,
|
||||
minutes,
|
||||
seconds,
|
||||
state->session_id);
|
||||
std::cout << json.dump() << '\n';
|
||||
}
|
||||
|
||||
int run_waybar(const sc::WaybarCommand& command) {
|
||||
if (command.toggle) {
|
||||
if (const auto state = sc::read_sender_state(); state.has_value()) {
|
||||
(void)kill(static_cast<pid_t>(state->pid), SIGTERM);
|
||||
// Give the graceful shutdown time to withdraw its state file so
|
||||
// the immediately following status print reflects reality.
|
||||
for (int attempt = 0; attempt < 20; ++attempt) {
|
||||
if (!sc::read_sender_state().has_value()) {
|
||||
break;
|
||||
}
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(50));
|
||||
}
|
||||
} else {
|
||||
#ifdef SC_HAS_SENDER
|
||||
if (const auto last = sc::read_last_session(); last.has_value()) {
|
||||
spawn_detached_sender(last->peer, last->bitrate_kbps);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
}
|
||||
print_waybar_status();
|
||||
return 0;
|
||||
}
|
||||
|
||||
int run_receiver(const sc::ReceiveCommand& command) {
|
||||
sc::ReceiverPipelineConfig config;
|
||||
config.local_rtp_endpoint = sc::Endpoint{"0.0.0.0", static_cast<std::uint16_t>(command.local_rtp_port)};
|
||||
config.signaling_port = static_cast<std::uint16_t>(command.signaling_port);
|
||||
config.decoder.hardware_accel = !command.software_decode;
|
||||
// Fullscreen is automatic under KMSDRM (headless); the flag forces it
|
||||
// on desktop sessions.
|
||||
config.renderer.fullscreen = command.fullscreen;
|
||||
@@ -323,7 +305,7 @@ int run_discover(const sc::DiscoverCommand& command) {
|
||||
}
|
||||
for (auto& [key, hosts] : receivers) {
|
||||
std::sort(hosts.begin(), hosts.end(), [](const std::string& lhs, const std::string& rhs) {
|
||||
return address_preference(lhs) < address_preference(rhs);
|
||||
return sc::address_preference(lhs) < sc::address_preference(rhs);
|
||||
});
|
||||
std::string joined =
|
||||
std::accumulate(hosts.begin(), hosts.end(), std::string{}, [](std::string lhs, const std::string& rhs) {
|
||||
@@ -357,5 +339,8 @@ int main(int argc, char* argv[]) {
|
||||
if (const sc::ReceiveCommand* receive = std::get_if<sc::ReceiveCommand>(&*command)) {
|
||||
return run_receiver(*receive);
|
||||
}
|
||||
if (const sc::WaybarCommand* waybar = std::get_if<sc::WaybarCommand>(&*command)) {
|
||||
return run_waybar(*waybar);
|
||||
}
|
||||
return run_discover(std::get<sc::DiscoverCommand>(*command));
|
||||
}
|
||||
|
||||
@@ -1,3 +1,32 @@
|
||||
# Shared application internals used by both the CLI binary and the GUI:
|
||||
# pipelines, session orchestration, and the state store. The sender
|
||||
# pipeline is only compiled when the sender is built; receiver-only
|
||||
# targets must not require the capture backend.
|
||||
|
||||
sc_app_core_sources = files(
|
||||
'pipelines.cpp',
|
||||
'sender_session.cpp',
|
||||
'state_store.cpp',
|
||||
)
|
||||
|
||||
sc_app_core_args = []
|
||||
sc_app_core_deps = [dep_json]
|
||||
if build_sender
|
||||
sc_app_core_args += ['-DSC_HAS_SENDER=1']
|
||||
sc_app_core_deps += [sc_capture_dep]
|
||||
endif
|
||||
|
||||
sc_app_core = static_library('sc_app_core',
|
||||
sc_app_core_sources,
|
||||
include_directories : [sc_core_inc, include_directories('.')],
|
||||
cpp_args : sc_app_core_args,
|
||||
dependencies : sc_app_core_deps)
|
||||
|
||||
sc_app_core_dep = declare_dependency(
|
||||
link_with : sc_app_core,
|
||||
include_directories : include_directories('.'),
|
||||
dependencies : [dep_json])
|
||||
|
||||
# The screencast application binary wiring every module together.
|
||||
# Receiver-only builds (-Dsender=false) exclude the capture backend and the
|
||||
# sender pipeline.
|
||||
@@ -5,10 +34,9 @@
|
||||
screencast_sources = [
|
||||
'cli.cpp',
|
||||
'main.cpp',
|
||||
'pipelines.cpp',
|
||||
]
|
||||
|
||||
screencast_dependencies = [sc_codec_dep, sc_network_dep, sc_render_dep]
|
||||
screencast_dependencies = [sc_app_core_dep, sc_codec_dep, sc_network_dep, sc_render_dep]
|
||||
screencast_arguments = []
|
||||
|
||||
if build_sender
|
||||
@@ -16,7 +44,7 @@ if build_sender
|
||||
screencast_arguments += ['-DSC_HAS_SENDER=1']
|
||||
endif
|
||||
|
||||
screencast = executable('screencast',
|
||||
executable('screencast',
|
||||
screencast_sources,
|
||||
cpp_args : screencast_arguments,
|
||||
dependencies : screencast_dependencies,
|
||||
|
||||
@@ -1,11 +1,16 @@
|
||||
#include "screencast/app/pipeline.h"
|
||||
|
||||
#include "state_store.h"
|
||||
|
||||
#include "screencast/network/discovery.h"
|
||||
#include "screencast/network/h264_packetizer.h"
|
||||
#include "screencast/network/signaling.h"
|
||||
|
||||
#include <unistd.h>
|
||||
|
||||
#include <unistd.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <chrono>
|
||||
#include <condition_variable>
|
||||
@@ -32,7 +37,7 @@ std::string receiver_service_name() {
|
||||
|
||||
class SenderPipeline::Impl {
|
||||
public:
|
||||
explicit Impl(SenderPipelineConfig config) : config_(std::move(config)) {}
|
||||
explicit Impl(SenderPipelineConfig config) : config_(std::move(config)), current_crf_(config_.encoder.crf) {}
|
||||
|
||||
~Impl() {
|
||||
stop();
|
||||
@@ -53,10 +58,24 @@ class SenderPipeline::Impl {
|
||||
transport_->set_peer(config_.peer_rtp_endpoint);
|
||||
|
||||
run_thread_ = std::jthread([this](std::stop_token stop_token) { run(std::move(stop_token)); });
|
||||
|
||||
// Publish the session for status widgets and one-click restarts.
|
||||
write_sender_state(SenderState{
|
||||
.session_id = config_.session_id,
|
||||
.receiver = std::format("{}:{}", config_.peer_rtp_endpoint.address, config_.peer_rtp_endpoint.port),
|
||||
.bitrate_kbps = config_.encoder.bitrate_kbps,
|
||||
.pid = ::getpid(),
|
||||
.started_epoch_ms = 0});
|
||||
std::string restart_peer = std::format("{}:5005", config_.peer_rtp_endpoint.address);
|
||||
if (config_.signaling_server.has_value()) {
|
||||
restart_peer = std::format("{}:{}", config_.signaling_server->address, config_.signaling_server->port);
|
||||
}
|
||||
write_last_session(LastSession{.peer = restart_peer, .bitrate_kbps = config_.encoder.bitrate_kbps});
|
||||
return true;
|
||||
}
|
||||
|
||||
void stop() {
|
||||
remove_sender_state();
|
||||
if (capture_ != nullptr) {
|
||||
capture_->stop();
|
||||
}
|
||||
@@ -64,9 +83,23 @@ class SenderPipeline::Impl {
|
||||
transport_->stop();
|
||||
}
|
||||
|
||||
void request_keyframe() {
|
||||
keyframe_requested_.store(true);
|
||||
pli_count_.fetch_add(1, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
private:
|
||||
void run(std::stop_token stop_token) {
|
||||
auto last_adaptation = std::chrono::steady_clock::now();
|
||||
auto last_pli_check = last_adaptation;
|
||||
auto last_pli_count = 0;
|
||||
auto last_frame_time = last_adaptation;
|
||||
|
||||
while (!stop_token.stop_requested()) {
|
||||
if (keyframe_requested_.exchange(false) && encoder_ != nullptr) {
|
||||
encoder_->request_keyframe();
|
||||
}
|
||||
|
||||
const std::optional<CapturedFrame> frame = capture_->next_frame();
|
||||
if (!frame.has_value()) {
|
||||
break;
|
||||
@@ -76,6 +109,17 @@ class SenderPipeline::Impl {
|
||||
break;
|
||||
}
|
||||
|
||||
// Frame rate capping: skip frames that arrive faster than the
|
||||
// configured target. 0 = no cap (use the monitor rate).
|
||||
if (config_.max_frame_rate > 0) {
|
||||
const auto now = std::chrono::steady_clock::now();
|
||||
const auto min_interval = std::chrono::microseconds(1'000'000 / config_.max_frame_rate);
|
||||
if (now - last_frame_time < min_interval) {
|
||||
continue;
|
||||
}
|
||||
last_frame_time = now;
|
||||
}
|
||||
|
||||
auto encoded_result = encoder_->encode(*frame);
|
||||
if (is_codec_error(encoded_result)) {
|
||||
std::cerr << std::format("screencast: encode failed: {}\n", codec_error(encoded_result).message);
|
||||
@@ -86,13 +130,70 @@ class SenderPipeline::Impl {
|
||||
(void)transport_->send(packet);
|
||||
}
|
||||
}
|
||||
|
||||
// Adaptive quality: evaluate the link every 5 seconds by
|
||||
// checking how many PLIs the receiver sent. Frequent PLIs
|
||||
// mean the receiver is dropping frames — the link is
|
||||
// saturated, so increase the CRF (lower quality, fewer
|
||||
// bits). When the link is quiet, try lowering the CRF to
|
||||
// probe for better quality.
|
||||
const auto now = std::chrono::steady_clock::now();
|
||||
if (now - last_pli_check >= std::chrono::seconds(5)) {
|
||||
const auto current_pli = pli_count_.load(std::memory_order_relaxed);
|
||||
const auto pli_delta = current_pli - last_pli_count;
|
||||
const auto seconds = std::chrono::duration_cast<std::chrono::seconds>(now - last_pli_check).count();
|
||||
const auto pli_per_second = static_cast<double>(pli_delta) / static_cast<double>(seconds);
|
||||
last_pli_count = current_pli;
|
||||
last_pli_check = now;
|
||||
|
||||
if (pli_per_second > 0.5 && current_crf_ < config_.encoder.crf + 10) {
|
||||
// Saturated: degrade quality (higher CRF = fewer bits)
|
||||
current_crf_ += 2;
|
||||
std::cerr << std::format(
|
||||
"screencast: link saturated ({} PLI/s); adapting CRF to {}\n", pli_per_second, current_crf_);
|
||||
if (!restart_encoder(*frame)) {
|
||||
break;
|
||||
}
|
||||
} else if (pli_per_second < 0.1 && current_crf_ > config_.encoder.crf) {
|
||||
// Stable: try better quality (lower CRF = more bits)
|
||||
current_crf_ -= 1;
|
||||
std::cerr << std::format("screencast: link stable; probing CRF {}\n", current_crf_);
|
||||
if (!restart_encoder(*frame)) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
bool restart_encoder(const CapturedFrame& frame) {
|
||||
// Create a new encoder with the adjusted CRF; the next encoded
|
||||
// frame is a keyframe, so the receiver recovers immediately.
|
||||
encoder_ = nullptr;
|
||||
return create_encoder(frame);
|
||||
}
|
||||
|
||||
bool create_encoder(const CapturedFrame& frame) {
|
||||
EncoderConfig config = config_.encoder;
|
||||
config.width = frame.width;
|
||||
config.height = frame.height;
|
||||
config.crf = current_crf_;
|
||||
|
||||
// Downscale to the receiver's display when the capture is larger,
|
||||
// preserving aspect ratio and rounding to even values (YUV420P
|
||||
// requires even dimensions for the chroma planes).
|
||||
if (config_.max_encode_width > 0 && config_.max_encode_height > 0 &&
|
||||
(frame.width > config_.max_encode_width || frame.height > config_.max_encode_height)) {
|
||||
const double scale = std::min(static_cast<double>(config_.max_encode_width) / frame.width,
|
||||
static_cast<double>(config_.max_encode_height) / frame.height);
|
||||
config.width = std::max(2, static_cast<int>(frame.width * scale) & ~1);
|
||||
config.height = std::max(2, static_cast<int>(frame.height * scale) & ~1);
|
||||
std::cerr << std::format("screencast: downscaling {}x{} to {}x{} for the receiver's display\n",
|
||||
frame.width,
|
||||
frame.height,
|
||||
config.width,
|
||||
config.height);
|
||||
}
|
||||
|
||||
auto encoder_result = EncoderFactory::create(config);
|
||||
if (is_codec_error(encoder_result)) {
|
||||
@@ -104,13 +205,20 @@ class SenderPipeline::Impl {
|
||||
}
|
||||
|
||||
SenderPipelineConfig config_;
|
||||
int current_crf_;
|
||||
std::unique_ptr<CaptureSession> capture_;
|
||||
std::unique_ptr<Encoder> encoder_;
|
||||
H264Packetizer packetizer_;
|
||||
std::unique_ptr<RtpTransport> transport_ = RtpTransportFactory::create();
|
||||
std::jthread run_thread_;
|
||||
std::atomic<bool> keyframe_requested_{false};
|
||||
std::atomic<int> pli_count_{0};
|
||||
};
|
||||
|
||||
void SenderPipeline::request_keyframe() {
|
||||
impl_->request_keyframe();
|
||||
}
|
||||
|
||||
SenderPipeline::SenderPipeline(SenderPipelineConfig config) : impl_(std::make_unique<Impl>(std::move(config))) {}
|
||||
|
||||
SenderPipeline::~SenderPipeline() = default;
|
||||
@@ -238,6 +346,7 @@ class ReceiverPipeline::Impl {
|
||||
signaling_ = nullptr;
|
||||
discovery_ = nullptr;
|
||||
transport_->stop();
|
||||
jitter_.clear();
|
||||
renderer_ = nullptr;
|
||||
decoder_ = nullptr;
|
||||
{
|
||||
@@ -248,13 +357,24 @@ class ReceiverPipeline::Impl {
|
||||
|
||||
private:
|
||||
void on_packet(RtpPacket packet) {
|
||||
std::optional<std::vector<std::byte>> access_unit = depacketizer_.depacketize(packet);
|
||||
if (!access_unit.has_value()) {
|
||||
// Absorb reordering (Wi-Fi) before the in-order depacketizer, so a
|
||||
// late packet is not misread as loss.
|
||||
for (RtpPacket& ordered : jitter_.push(std::move(packet))) {
|
||||
deliver_packet(ordered);
|
||||
}
|
||||
}
|
||||
|
||||
void deliver_packet(const RtpPacket& packet) {
|
||||
const DepacketizeResult result = depacketizer_.depacketize(packet);
|
||||
if (result.frame_dropped) {
|
||||
maybe_send_pli();
|
||||
}
|
||||
if (!result.access_unit.has_value()) {
|
||||
return;
|
||||
}
|
||||
|
||||
EncodedFrame encoded;
|
||||
encoded.data = std::move(*access_unit);
|
||||
encoded.data = std::move(*result.access_unit);
|
||||
encoded.rtp_timestamp = packet.header.timestamp;
|
||||
encoded.is_keyframe = false;
|
||||
|
||||
@@ -282,14 +402,38 @@ class ReceiverPipeline::Impl {
|
||||
if (offer == nullptr) {
|
||||
return;
|
||||
}
|
||||
session_id_ = offer->session_id;
|
||||
SessionAnswer answer;
|
||||
answer.session_id = offer->session_id;
|
||||
// Empty address: the sender targets the address of its signaling
|
||||
// connection, which reaches this RTP port.
|
||||
answer.rtp_endpoint = Endpoint{"", config_.local_rtp_endpoint.port};
|
||||
// Tell the sender what display it is rendering to so it can
|
||||
// downscale instead of encoding pixels the display cannot show.
|
||||
if (renderer_ != nullptr) {
|
||||
answer.display_width = renderer_->display_width();
|
||||
answer.display_height = renderer_->display_height();
|
||||
}
|
||||
signaling_->send(answer);
|
||||
}
|
||||
|
||||
// Ask the sender for a keyframe after a damaged frame, rate-limited so
|
||||
// sustained loss cannot flood the signaling channel.
|
||||
void maybe_send_pli() {
|
||||
if (signaling_ == nullptr || session_id_.empty()) {
|
||||
return;
|
||||
}
|
||||
const auto now = std::chrono::steady_clock::now();
|
||||
if (now - last_pli_time_ < kPliMinInterval) {
|
||||
return;
|
||||
}
|
||||
last_pli_time_ = now;
|
||||
SessionPli pli;
|
||||
pli.session_id = session_id_;
|
||||
signaling_->send(pli);
|
||||
std::cerr << std::format("screencast: frame damaged; requesting a keyframe\n");
|
||||
}
|
||||
|
||||
void render_loop(std::stop_token stop_token) {
|
||||
while (!stop_token.stop_requested()) {
|
||||
if (!renderer_->poll_events()) {
|
||||
@@ -315,17 +459,21 @@ class ReceiverPipeline::Impl {
|
||||
}
|
||||
|
||||
static constexpr std::size_t kMaxQueuedFrames = 3;
|
||||
static constexpr std::chrono::milliseconds kPliMinInterval{500};
|
||||
|
||||
ReceiverPipelineConfig config_;
|
||||
std::unique_ptr<Renderer> renderer_;
|
||||
std::unique_ptr<Decoder> decoder_;
|
||||
H264Depacketizer depacketizer_;
|
||||
RtpJitterBuffer jitter_;
|
||||
std::unique_ptr<RtpTransport> transport_ = RtpTransportFactory::create();
|
||||
std::unique_ptr<SignalingChannel> signaling_;
|
||||
std::unique_ptr<DiscoveryService> discovery_;
|
||||
std::jthread render_thread_;
|
||||
std::mutex queue_mutex_;
|
||||
std::deque<DecodedFrame> queue_;
|
||||
std::string session_id_;
|
||||
std::chrono::steady_clock::time_point last_pli_time_{};
|
||||
bool stream_started_ = false;
|
||||
bool present_error_logged_ = false;
|
||||
};
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
#include "sender_session.h"
|
||||
|
||||
#include "screencast/network/signaling.h"
|
||||
|
||||
#include <atomic>
|
||||
#include <charconv>
|
||||
#include <chrono>
|
||||
#include <format>
|
||||
#include <future>
|
||||
#include <memory>
|
||||
#include <string_view>
|
||||
#include <utility>
|
||||
|
||||
namespace sc {
|
||||
|
||||
#ifdef SC_HAS_SENDER
|
||||
|
||||
namespace {
|
||||
|
||||
bool is_private_ipv4(std::string_view host) {
|
||||
if (host.rfind("192.168.", 0) == 0 || host.rfind("10.", 0) == 0) {
|
||||
return true;
|
||||
}
|
||||
if (host.rfind("172.", 0) == 0) {
|
||||
const std::size_t second = host.find('.', 5);
|
||||
if (second != std::string_view::npos) {
|
||||
int octet = 0;
|
||||
const auto [pointer, error] = std::from_chars(host.data() + 5, host.data() + second, octet);
|
||||
if (error == std::errc{}) {
|
||||
return octet >= 16 && octet <= 31;
|
||||
}
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::variant<SenderSession, std::string>
|
||||
SenderSession::start(const Endpoint& signaling_endpoint, int bitrate_kbps, int crf, int max_fps, CaptureTarget target) {
|
||||
auto channel_result = SignalingFactory::create_client();
|
||||
if (is_network_error(channel_result)) {
|
||||
return network_error(channel_result).message;
|
||||
}
|
||||
auto channel = std::move(network_value(channel_result));
|
||||
if (!channel->connect(signaling_endpoint)) {
|
||||
return std::format(
|
||||
"failed to connect to the receiver at {}:{}", signaling_endpoint.address, signaling_endpoint.port);
|
||||
}
|
||||
|
||||
// Offer + answer.
|
||||
static std::atomic<std::uint32_t> session_counter{0};
|
||||
const std::string session_id = std::format(
|
||||
"sc-{:x}-{:x}",
|
||||
static_cast<std::uint32_t>(std::chrono::steady_clock::now().time_since_epoch().count()) & 0xffffffffU,
|
||||
session_counter.fetch_add(1));
|
||||
|
||||
std::promise<SessionAnswer> answer_promise;
|
||||
auto answer_future = answer_promise.get_future();
|
||||
std::atomic<bool> answered{false};
|
||||
channel->on_message([&](const SignalingMessage& message) {
|
||||
if (const SessionAnswer* answer = std::get_if<SessionAnswer>(&message)) {
|
||||
if (!answered.exchange(true)) {
|
||||
answer_promise.set_value(*answer);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
SessionOffer offer;
|
||||
offer.session_id = session_id;
|
||||
offer.codec_name = "h264";
|
||||
offer.frame_rate_num = 25;
|
||||
offer.frame_rate_den = 1;
|
||||
channel->send(offer);
|
||||
|
||||
if (answer_future.wait_for(std::chrono::seconds(5)) != std::future_status::ready) {
|
||||
channel->disconnect();
|
||||
return std::string{"the receiver did not answer the session offer"};
|
||||
}
|
||||
const SessionAnswer answer = answer_future.get();
|
||||
if (answer.session_id != session_id) {
|
||||
channel->disconnect();
|
||||
return std::string{"session mismatch in the receiver's answer"};
|
||||
}
|
||||
if (answer.rtp_endpoint.port == 0) {
|
||||
channel->disconnect();
|
||||
return std::string{"the receiver did not provide an RTP port"};
|
||||
}
|
||||
|
||||
// Stream to the negotiated endpoint; an empty address means "the
|
||||
// address you reached me on".
|
||||
const Endpoint rtp_endpoint = answer.rtp_endpoint.address.empty()
|
||||
? Endpoint{signaling_endpoint.address, answer.rtp_endpoint.port}
|
||||
: answer.rtp_endpoint;
|
||||
|
||||
SenderPipelineConfig config;
|
||||
config.capture_target = target;
|
||||
config.peer_rtp_endpoint = rtp_endpoint;
|
||||
config.signaling_server = signaling_endpoint;
|
||||
config.encoder.bitrate_kbps = bitrate_kbps;
|
||||
config.encoder.crf = crf;
|
||||
config.max_frame_rate = max_fps;
|
||||
config.session_id = session_id;
|
||||
config.max_encode_width = answer.display_width;
|
||||
config.max_encode_height = answer.display_height;
|
||||
|
||||
auto pipeline = std::make_unique<SenderPipeline>(std::move(config));
|
||||
if (!pipeline->start()) {
|
||||
channel->disconnect();
|
||||
return std::string{"failed to start the sender pipeline"};
|
||||
}
|
||||
|
||||
// PLI feedback over the still-open channel.
|
||||
channel->on_message([pipeline = pipeline.get(), session_id](const SignalingMessage& message) {
|
||||
const SessionPli* pli = std::get_if<SessionPli>(&message);
|
||||
if (pli != nullptr && pli->session_id == session_id) {
|
||||
pipeline->request_keyframe();
|
||||
}
|
||||
});
|
||||
|
||||
SenderSession session;
|
||||
session.channel_ = std::move(channel);
|
||||
session.pipeline_ = std::move(pipeline);
|
||||
session.session_id_ = session_id;
|
||||
session.receiver_ = std::format("{}:{}", rtp_endpoint.address, rtp_endpoint.port);
|
||||
return session;
|
||||
}
|
||||
|
||||
SenderSession::SenderSession(SenderSession&& other) noexcept
|
||||
: channel_(std::move(other.channel_)),
|
||||
pipeline_(std::move(other.pipeline_)),
|
||||
session_id_(std::move(other.session_id_)),
|
||||
receiver_(std::move(other.receiver_)),
|
||||
stopped_(other.stopped_) {}
|
||||
|
||||
SenderSession& SenderSession::operator=(SenderSession&& other) noexcept {
|
||||
stop();
|
||||
channel_ = std::move(other.channel_);
|
||||
pipeline_ = std::move(other.pipeline_);
|
||||
session_id_ = std::move(other.session_id_);
|
||||
receiver_ = std::move(other.receiver_);
|
||||
stopped_ = other.stopped_;
|
||||
return *this;
|
||||
}
|
||||
|
||||
SenderSession::~SenderSession() {
|
||||
stop();
|
||||
}
|
||||
|
||||
void SenderSession::stop() {
|
||||
if (stopped_) {
|
||||
return;
|
||||
}
|
||||
stopped_ = true;
|
||||
// The channel first: disconnecting joins its reader threads, so no PLI
|
||||
// callback can race the pipeline teardown it points at.
|
||||
if (channel_ != nullptr) {
|
||||
channel_->disconnect();
|
||||
channel_ = nullptr;
|
||||
}
|
||||
if (pipeline_ != nullptr) {
|
||||
pipeline_->stop();
|
||||
pipeline_ = nullptr;
|
||||
}
|
||||
}
|
||||
|
||||
#else
|
||||
|
||||
// Receiver-only builds have no sender pipeline; address_preference is the
|
||||
// inline header function, and the rest of SenderSession is compiled out.
|
||||
|
||||
#endif // SC_HAS_SENDER
|
||||
|
||||
} // namespace sc
|
||||
@@ -0,0 +1,72 @@
|
||||
#pragma once
|
||||
|
||||
#include "screencast/app/pipeline.h"
|
||||
|
||||
#include <cstdint>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
#include <variant>
|
||||
|
||||
namespace sc {
|
||||
|
||||
// Ordering for trying a receiver's addresses: private IPv4 first (LANs,
|
||||
// most reliable), then public IPv4, ULA, and global IPv6. 6to4 (2002::) and
|
||||
// link-local addresses last: 6to4 is frequently unreachable between LAN
|
||||
// peers, and link-local needs a scope id to even route. Inline so that
|
||||
// receiver-only builds (no sender session) can still use it for discovery.
|
||||
inline int address_preference(std::string_view host) {
|
||||
if (host.find(':') == std::string_view::npos) {
|
||||
const bool priv = host.rfind("192.168.", 0) == 0 || host.rfind("10.", 0) == 0 ||
|
||||
(host.rfind("172.", 0) == 0 && host.find('.', 5) != std::string_view::npos);
|
||||
return priv ? 0 : 1;
|
||||
}
|
||||
if (host.rfind("fd", 0) == 0 || host.rfind("fc", 0) == 0) {
|
||||
return 2;
|
||||
}
|
||||
if (host.rfind("2002:", 0) == 0) {
|
||||
return 4;
|
||||
}
|
||||
if (host.rfind("fe80:", 0) == 0) {
|
||||
return 5;
|
||||
}
|
||||
return 3;
|
||||
}
|
||||
|
||||
// A running sender session: it negotiated over signaling (keeping the
|
||||
// channel open for PLI feedback) and drives the sender pipeline. Shared by
|
||||
// the CLI, the GUI, and the waybar widget's one-click restart.
|
||||
class SenderSession {
|
||||
public:
|
||||
// Blocking: connects, offers, waits for the answer, and starts the
|
||||
// pipeline — which includes the portal's interactive source picker.
|
||||
// Returns an error message on failure.
|
||||
static std::variant<SenderSession, std::string>
|
||||
start(const Endpoint& signaling_endpoint, int bitrate_kbps, int crf, int max_fps, CaptureTarget target);
|
||||
|
||||
SenderSession() = default;
|
||||
~SenderSession();
|
||||
|
||||
SenderSession(SenderSession&& other) noexcept;
|
||||
SenderSession& operator=(SenderSession&& other) noexcept;
|
||||
|
||||
// Graceful stop: withdraws state and closes the signaling channel.
|
||||
void stop();
|
||||
|
||||
[[nodiscard]] const std::string& session_id() const {
|
||||
return session_id_;
|
||||
}
|
||||
|
||||
[[nodiscard]] const std::string& receiver() const {
|
||||
return receiver_;
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<class SignalingChannel> channel_;
|
||||
std::unique_ptr<SenderPipeline> pipeline_;
|
||||
std::string session_id_;
|
||||
std::string receiver_;
|
||||
bool stopped_ = false;
|
||||
};
|
||||
|
||||
} // namespace sc
|
||||
@@ -0,0 +1,140 @@
|
||||
#include "state_store.h"
|
||||
|
||||
#include <nlohmann/json.hpp>
|
||||
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <chrono>
|
||||
#include <csignal>
|
||||
#include <cstdlib>
|
||||
#include <filesystem>
|
||||
#include <fstream>
|
||||
#include <sstream>
|
||||
|
||||
namespace sc {
|
||||
namespace {
|
||||
|
||||
namespace fs = std::filesystem;
|
||||
|
||||
fs::path runtime_dir() {
|
||||
const char* xdg = std::getenv("XDG_RUNTIME_DIR");
|
||||
if (xdg != nullptr && *xdg != '\0') {
|
||||
return fs::path{xdg} / "screencast";
|
||||
}
|
||||
return fs::temp_directory_path() / ("screencast-" + std::to_string(::getuid()));
|
||||
}
|
||||
|
||||
fs::path config_dir() {
|
||||
const char* xdg = std::getenv("XDG_CONFIG_HOME");
|
||||
if (xdg != nullptr && *xdg != '\0') {
|
||||
return fs::path{xdg} / "screencast";
|
||||
}
|
||||
const char* home = std::getenv("HOME");
|
||||
return fs::path{home != nullptr ? home : "."} / ".config" / "screencast";
|
||||
}
|
||||
|
||||
std::optional<std::string> read_file(const fs::path& path) {
|
||||
std::ifstream file{path, std::ios::binary};
|
||||
if (!file.is_open()) {
|
||||
return std::nullopt;
|
||||
}
|
||||
std::ostringstream buffer;
|
||||
buffer << file.rdbuf();
|
||||
return buffer.str();
|
||||
}
|
||||
|
||||
bool write_file(const fs::path& path, std::string_view contents) {
|
||||
std::error_code error;
|
||||
fs::create_directories(path.parent_path(), error);
|
||||
std::ofstream file{path, std::ios::binary | std::ios::trunc};
|
||||
if (!file.is_open()) {
|
||||
return false;
|
||||
}
|
||||
file.write(contents.data(), static_cast<std::streamsize>(contents.size()));
|
||||
return file.good();
|
||||
}
|
||||
|
||||
std::int64_t epoch_ms() {
|
||||
const auto now = std::chrono::system_clock::now().time_since_epoch();
|
||||
return std::chrono::duration_cast<std::chrono::milliseconds>(now).count();
|
||||
}
|
||||
|
||||
bool process_alive(int pid) {
|
||||
if (pid <= 0) {
|
||||
return false;
|
||||
}
|
||||
return ::kill(static_cast<pid_t>(pid), 0) == 0;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool write_sender_state(const SenderState& state) {
|
||||
nlohmann::json json;
|
||||
json["session_id"] = state.session_id;
|
||||
json["receiver"] = state.receiver;
|
||||
json["bitrate_kbps"] = state.bitrate_kbps;
|
||||
json["pid"] = state.pid;
|
||||
json["started_epoch_ms"] = state.started_epoch_ms != 0 ? state.started_epoch_ms : epoch_ms();
|
||||
return write_file(runtime_dir() / "sender.json", json.dump() + "\n");
|
||||
}
|
||||
|
||||
bool remove_sender_state() {
|
||||
std::error_code error;
|
||||
const bool removed = fs::remove(runtime_dir() / "sender.json", error);
|
||||
return removed || !fs::exists(runtime_dir() / "sender.json");
|
||||
}
|
||||
|
||||
std::optional<SenderState> read_sender_state() {
|
||||
const std::optional<std::string> contents = read_file(runtime_dir() / "sender.json");
|
||||
if (!contents.has_value()) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
const nlohmann::json json = nlohmann::json::parse(*contents, nullptr, /*allow_exceptions=*/false);
|
||||
if (json.is_discarded() || !json.is_object()) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
SenderState state;
|
||||
state.session_id = json.value("session_id", std::string{});
|
||||
state.receiver = json.value("receiver", std::string{});
|
||||
state.bitrate_kbps = json.value("bitrate_kbps", 0);
|
||||
state.pid = json.value("pid", 0);
|
||||
state.started_epoch_ms = json.value("started_epoch_ms", std::int64_t{0});
|
||||
|
||||
// A state file without a live process is a crash remnant: not streaming.
|
||||
if (!process_alive(state.pid)) {
|
||||
(void)remove_sender_state();
|
||||
return std::nullopt;
|
||||
}
|
||||
return state;
|
||||
}
|
||||
|
||||
void write_last_session(const LastSession& last) {
|
||||
nlohmann::json json;
|
||||
json["peer"] = last.peer;
|
||||
json["bitrate_kbps"] = last.bitrate_kbps;
|
||||
(void)write_file(config_dir() / "last-session.json", json.dump() + "\n");
|
||||
}
|
||||
|
||||
std::optional<LastSession> read_last_session() {
|
||||
const std::optional<std::string> contents = read_file(config_dir() / "last-session.json");
|
||||
if (!contents.has_value()) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
const nlohmann::json json = nlohmann::json::parse(*contents, nullptr, /*allow_exceptions=*/false);
|
||||
if (json.is_discarded() || !json.is_object()) {
|
||||
return std::nullopt;
|
||||
}
|
||||
LastSession last;
|
||||
last.peer = json.value("peer", std::string{});
|
||||
last.bitrate_kbps = json.value("bitrate_kbps", 0);
|
||||
if (last.peer.empty() || last.bitrate_kbps <= 0) {
|
||||
return std::nullopt;
|
||||
}
|
||||
return last;
|
||||
}
|
||||
|
||||
} // namespace sc
|
||||
@@ -0,0 +1,35 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstdint>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
|
||||
namespace sc {
|
||||
|
||||
// Live state of a running sender, written by the pipeline so status
|
||||
// widgets (waybar) and one-click restarts work for every front-end.
|
||||
struct SenderState {
|
||||
std::string session_id;
|
||||
std::string receiver; // "host:port" of the RTP endpoint
|
||||
int bitrate_kbps = 0;
|
||||
int pid = 0;
|
||||
std::int64_t started_epoch_ms = 0;
|
||||
};
|
||||
|
||||
// The last successfully started session, persisted across reboots so a
|
||||
// single click can restart streaming without picking anything.
|
||||
struct LastSession {
|
||||
std::string peer; // "host" or "host:port" for the signaling endpoint
|
||||
int bitrate_kbps = 0;
|
||||
};
|
||||
|
||||
bool write_sender_state(const SenderState& state);
|
||||
bool remove_sender_state();
|
||||
// Returns nullopt when no sender is running (a stale state file is
|
||||
// treated as not streaming and is cleaned up).
|
||||
std::optional<SenderState> read_sender_state();
|
||||
|
||||
void write_last_session(const LastSession& last);
|
||||
std::optional<LastSession> read_last_session();
|
||||
|
||||
} // namespace sc
|
||||
@@ -5,6 +5,8 @@
|
||||
#include <array>
|
||||
#include <cstdint>
|
||||
#include <cstring>
|
||||
#include <format>
|
||||
#include <iostream>
|
||||
#include <limits>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
@@ -145,34 +147,61 @@ class FfmpegDecoder final : public Decoder {
|
||||
? static_cast<uint64_t>(av_rescale_q(input->pts, ctx_->time_base, AVRational{1, 1'000'000'000}))
|
||||
: fallback_timestamp_ns;
|
||||
|
||||
const std::size_t buffer_size =
|
||||
static_cast<std::size_t>(av_image_get_buffer_size(AV_PIX_FMT_RGBA, frame.width, frame.height, 1));
|
||||
frame.rgba_pixels.resize(buffer_size);
|
||||
|
||||
if (!ensure_scaler(input->width, input->height, static_cast<AVPixelFormat>(input->format))) {
|
||||
return CodecError{"failed to create swscale context"};
|
||||
const AVPixelFormat pixel_format = static_cast<AVPixelFormat>(input->format);
|
||||
if (pixel_format != AV_PIX_FMT_YUV420P) {
|
||||
// Hardware decoders (e.g. v4l2m2m) may emit NV12 or other planar
|
||||
// variants; convert to YUV420P once. The common software path
|
||||
// (YUV420P) skips this entirely.
|
||||
if (!ensure_scaler(input->width, input->height, pixel_format)) {
|
||||
return CodecError{"failed to create format conversion context"};
|
||||
}
|
||||
AvFramePtr converted(av_frame_alloc(), AvFrameDeleter{});
|
||||
converted->width = input->width;
|
||||
converted->height = input->height;
|
||||
converted->format = AV_PIX_FMT_YUV420P;
|
||||
if (av_frame_get_buffer(converted.get(), 0) < 0) {
|
||||
return CodecError{"failed to allocate converted frame"};
|
||||
}
|
||||
if (sws_scale(scaler_.get(),
|
||||
input->data,
|
||||
input->linesize,
|
||||
0,
|
||||
input->height,
|
||||
converted->data,
|
||||
converted->linesize) <= 0) {
|
||||
return CodecError{"failed to convert decoded frame to YUV420P"};
|
||||
}
|
||||
return copy_yuv420p_planes(converted.get(), std::move(frame));
|
||||
}
|
||||
|
||||
std::array<uint8_t*, 4> dst{nullptr, nullptr, nullptr, nullptr};
|
||||
std::array<int, 4> dst_lines{0, 0, 0, 0};
|
||||
if (av_image_fill_arrays(dst.data(),
|
||||
dst_lines.data(),
|
||||
as_u8(frame.rgba_pixels.data()),
|
||||
AV_PIX_FMT_RGBA,
|
||||
frame.width,
|
||||
frame.height,
|
||||
1) < 0) {
|
||||
return CodecError{"failed to fill output pixel arrays"};
|
||||
}
|
||||
return copy_yuv420p_planes(input, std::move(frame));
|
||||
}
|
||||
|
||||
if (sws_scale(scaler_.get(), input->data, input->linesize, 0, input->height, dst.data(), dst_lines.data()) <=
|
||||
0) {
|
||||
return CodecError{"failed to convert decoded frame to RGBA"};
|
||||
}
|
||||
// Copies the three YUV420P planes with their strides (which may include
|
||||
// alignment padding). SDL's UpdateYUVTexture accepts arbitrary pitches.
|
||||
CodecResult<DecodedFrame> copy_yuv420p_planes(const AVFrame* input, DecodedFrame frame) const {
|
||||
const int width = input->width;
|
||||
const int height = input->height;
|
||||
frame.stride_y = input->linesize[0];
|
||||
frame.stride_u = input->linesize[1];
|
||||
frame.stride_v = input->linesize[2];
|
||||
|
||||
const std::size_t y_size = static_cast<std::size_t>(frame.stride_y) * height;
|
||||
const std::size_t uv_size = static_cast<std::size_t>(frame.stride_u) * ((height + 1) / 2);
|
||||
const std::size_t v_size = static_cast<std::size_t>(frame.stride_v) * ((height + 1) / 2);
|
||||
|
||||
frame.plane_y.resize(y_size);
|
||||
frame.plane_u.resize(uv_size);
|
||||
frame.plane_v.resize(v_size);
|
||||
|
||||
std::memcpy(frame.plane_y.data(), input->data[0], y_size);
|
||||
std::memcpy(frame.plane_u.data(), input->data[1], uv_size);
|
||||
std::memcpy(frame.plane_v.data(), input->data[2], v_size);
|
||||
|
||||
return frame;
|
||||
}
|
||||
|
||||
// Only used when the decoder emits a non-YUV420P format (hardware paths).
|
||||
bool ensure_scaler(int width, int height, AVPixelFormat input_format) const {
|
||||
if (scaler_ != nullptr && scaler_input_width_ == width && scaler_input_height_ == height &&
|
||||
scaler_input_format_ == input_format) {
|
||||
@@ -180,7 +209,7 @@ class FfmpegDecoder final : public Decoder {
|
||||
}
|
||||
|
||||
scaler_.reset(sws_getContext(
|
||||
width, height, input_format, width, height, AV_PIX_FMT_RGBA, SWS_BILINEAR, nullptr, nullptr, nullptr));
|
||||
width, height, input_format, width, height, AV_PIX_FMT_YUV420P, SWS_BILINEAR, nullptr, nullptr, nullptr));
|
||||
if (scaler_ == nullptr) {
|
||||
return false;
|
||||
}
|
||||
@@ -199,19 +228,18 @@ class FfmpegDecoder final : public Decoder {
|
||||
mutable AVPixelFormat scaler_input_format_ = AV_PIX_FMT_NONE;
|
||||
};
|
||||
|
||||
CodecResult<std::unique_ptr<Decoder>> DecoderFactory::create(const DecoderConfig& config) {
|
||||
if (config.codec_name != "h264") {
|
||||
return CodecError{"only h264 is supported in phase 2"};
|
||||
}
|
||||
namespace {
|
||||
|
||||
const AVCodec* codec = avcodec_find_decoder(AV_CODEC_ID_H264);
|
||||
// Opens the given decoder implementation for the given config. Returns an
|
||||
// error string on failure (used for the hardware probe + fallback).
|
||||
std::optional<std::string> open_decoder(const DecoderConfig& config, const AVCodec* codec, AvCodecContextPtr& ctx) {
|
||||
if (codec == nullptr) {
|
||||
return CodecError{"h264 decoder not found"};
|
||||
return std::string{"decoder not found"};
|
||||
}
|
||||
|
||||
AvCodecContextPtr ctx(avcodec_alloc_context3(codec), AvCodecContextDeleter{});
|
||||
ctx.reset(avcodec_alloc_context3(codec));
|
||||
if (ctx == nullptr) {
|
||||
return CodecError{"failed to allocate decoder context"};
|
||||
return std::string{"failed to allocate decoder context"};
|
||||
}
|
||||
|
||||
ctx->codec_type = AVMEDIA_TYPE_VIDEO;
|
||||
@@ -221,28 +249,66 @@ CodecResult<std::unique_ptr<Decoder>> DecoderFactory::create(const DecoderConfig
|
||||
if (config.height > 0) {
|
||||
ctx->height = config.height;
|
||||
}
|
||||
ctx->thread_count = 1;
|
||||
// Slice-level threading parallelizes within a single frame (no added
|
||||
// latency), unlike frame-level threading which buffers multiple frames
|
||||
// before producing output — unacceptable for a live stream.
|
||||
ctx->thread_count = 4;
|
||||
ctx->thread_type = FF_THREAD_SLICE;
|
||||
|
||||
if (!config.extradata.empty()) {
|
||||
if (config.extradata.size() > static_cast<std::size_t>(std::numeric_limits<int>::max())) {
|
||||
return CodecError{"decoder extradata is too large"};
|
||||
return std::string{"decoder extradata is too large"};
|
||||
}
|
||||
// FFmpeg bitstream parsers may read past the end of extradata, so the
|
||||
// buffer must include the padding they require.
|
||||
ctx->extradata = static_cast<uint8_t*>(av_malloc(config.extradata.size() + AV_INPUT_BUFFER_PADDING_SIZE));
|
||||
if (ctx->extradata == nullptr) {
|
||||
return CodecError{"failed to allocate decoder extradata"};
|
||||
return std::string{"failed to allocate decoder extradata"};
|
||||
}
|
||||
std::memcpy(ctx->extradata, config.extradata.data(), config.extradata.size());
|
||||
std::memset(ctx->extradata + config.extradata.size(), 0, AV_INPUT_BUFFER_PADDING_SIZE);
|
||||
ctx->extradata_size = static_cast<int>(config.extradata.size());
|
||||
}
|
||||
|
||||
int open_ret = avcodec_open2(ctx.get(), codec, nullptr);
|
||||
const int open_ret = avcodec_open2(ctx.get(), codec, nullptr);
|
||||
if (open_ret < 0) {
|
||||
return CodecError{std::string{"failed to open h264 decoder: "} + ffmpeg_error(open_ret)};
|
||||
return std::string{"failed to open decoder: "} + ffmpeg_error(open_ret);
|
||||
}
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
CodecResult<std::unique_ptr<Decoder>> DecoderFactory::create(const DecoderConfig& config) {
|
||||
if (config.codec_name != "h264") {
|
||||
return CodecError{"only h264 is supported"};
|
||||
}
|
||||
|
||||
// Hardware first (v4l2 mem2mem: the Pi's VideoCore H.264 decoder), with
|
||||
// an automatic software fallback. The kernel's bitstream parser reads
|
||||
// the stream dimensions from the in-band SPS, so they are not required
|
||||
// up front.
|
||||
if (config.hardware_accel) {
|
||||
if (const AVCodec* hw = avcodec_find_decoder_by_name("h264_v4l2m2m"); hw != nullptr) {
|
||||
AvCodecContextPtr ctx{nullptr, AvCodecContextDeleter{}};
|
||||
if (auto error = open_decoder(config, hw, ctx)) {
|
||||
std::cerr << std::format("screencast: hardware decode unavailable ({}); falling back to "
|
||||
"software\n",
|
||||
*error);
|
||||
} else {
|
||||
std::cerr << "screencast: using hardware H.264 decode (h264_v4l2m2m)\n";
|
||||
return std::make_unique<FfmpegDecoder>(std::move(ctx), config);
|
||||
}
|
||||
} else {
|
||||
std::cerr << "screencast: hardware decoder not compiled in; using software decode\n";
|
||||
}
|
||||
}
|
||||
|
||||
const AVCodec* codec = avcodec_find_decoder(AV_CODEC_ID_H264);
|
||||
AvCodecContextPtr ctx{nullptr, AvCodecContextDeleter{}};
|
||||
if (auto error = open_decoder(config, codec, ctx)) {
|
||||
return CodecError{std::move(*error)};
|
||||
}
|
||||
return std::make_unique<FfmpegDecoder>(std::move(ctx), config);
|
||||
}
|
||||
|
||||
|
||||
@@ -286,8 +286,12 @@ class FfmpegEncoder final : public Encoder {
|
||||
return CodecError{"failed to allocate AVFrame"};
|
||||
}
|
||||
|
||||
output->width = frame.width;
|
||||
output->height = frame.height;
|
||||
// The output frame is at the encoder's configured dimensions (which
|
||||
// may be smaller than the capture when downscaling to the receiver's
|
||||
// display); sws_scale handles both the format conversion and the
|
||||
// resolution change in one pass.
|
||||
output->width = config_.width;
|
||||
output->height = config_.height;
|
||||
output->format = AV_PIX_FMT_YUV420P;
|
||||
output->time_base = ctx_->time_base;
|
||||
output->pts =
|
||||
@@ -338,8 +342,18 @@ class FfmpegEncoder final : public Encoder {
|
||||
return true;
|
||||
}
|
||||
|
||||
scaler_.reset(sws_getContext(
|
||||
width, height, input_format, width, height, AV_PIX_FMT_YUV420P, SWS_BILINEAR, nullptr, nullptr, nullptr));
|
||||
// Scale to the encoder's configured output (which may be smaller
|
||||
// than the input when downscaling to the receiver's display).
|
||||
scaler_.reset(sws_getContext(width,
|
||||
height,
|
||||
input_format,
|
||||
config_.width,
|
||||
config_.height,
|
||||
AV_PIX_FMT_YUV420P,
|
||||
SWS_BILINEAR,
|
||||
nullptr,
|
||||
nullptr,
|
||||
nullptr));
|
||||
if (scaler_ == nullptr) {
|
||||
return false;
|
||||
}
|
||||
@@ -367,7 +381,10 @@ CodecResult<std::unique_ptr<Encoder>> EncoderFactory::create(const EncoderConfig
|
||||
return CodecError{"encoder frame rate must be positive"};
|
||||
}
|
||||
if (config.bitrate_kbps <= 0) {
|
||||
return CodecError{"encoder bitrate must be positive"};
|
||||
return CodecError{"encoder VBV max bitrate must be positive"};
|
||||
}
|
||||
if (config.crf < 0 || config.crf > 51) {
|
||||
return CodecError{"encoder CRF must be between 0 and 51"};
|
||||
}
|
||||
if (config.codec_name != "h264" && config.codec_name != "libx264") {
|
||||
return CodecError{"only h264 is supported in phase 2"};
|
||||
@@ -395,14 +412,32 @@ CodecResult<std::unique_ptr<Encoder>> EncoderFactory::create(const EncoderConfig
|
||||
ctx->time_base = AVRational{1, 1'000'000};
|
||||
ctx->framerate = AVRational{config.frame_rate_num, config.frame_rate_den};
|
||||
ctx->pix_fmt = AV_PIX_FMT_YUV420P;
|
||||
ctx->bit_rate = static_cast<int64_t>(config.bitrate_kbps) * 1000;
|
||||
// VBV keeps the stream CBR-ish: without it a keyframe may take many
|
||||
// times the average frame size in one burst, overflowing the receiver's
|
||||
// UDP socket buffer and dropping the packets that carry SPS/PPS. Two
|
||||
// frame periods of budget keep latency low while bounding the burst.
|
||||
ctx->rc_max_rate = ctx->bit_rate;
|
||||
ctx->rc_buffer_size = static_cast<int>(ctx->bit_rate * 2 / config.frame_rate_num);
|
||||
ctx->gop_size = config.frame_rate_num;
|
||||
|
||||
// CRF rate control: target a constant visual quality level instead of
|
||||
// a fixed bitrate, and let the encoder use fewer bits on static screen
|
||||
// content and more on motion or text. The VBV max rate still caps the
|
||||
// peak so bursts cannot overflow the receiver's UDP buffers.
|
||||
// CRF rate control: set x264's CRF directly as a private option.
|
||||
// FFmpeg's global_quality + AV_CODEC_FLAG_QSCALE path divides by
|
||||
// FF_QP2LAMBDA, giving a wrong CRF value (16/118 ≈ 0.1, essentially
|
||||
// lossless) — hence the "-qscale is ignored" warning. Setting the
|
||||
// private "crf" option bypasses that and gives x264 the exact value.
|
||||
const std::string crf_value = std::to_string(config.crf);
|
||||
if (av_opt_set(ctx->priv_data, "crf", crf_value.c_str(), 0) < 0) {
|
||||
return CodecError{"failed to set CRF quality"};
|
||||
}
|
||||
ctx->rc_max_rate = static_cast<int64_t>(config.bitrate_kbps) * 1000;
|
||||
// VBV: one frame period of budget keeps bursts tight — a two-frame
|
||||
// buffer lets a keyframe spike beyond what a constrained link can
|
||||
// absorb in real time, causing packet loss that cascades into PLI
|
||||
// storms. The tighter buffer trades a small quality dip on keyframes
|
||||
// for much better behavior on slow paths.
|
||||
ctx->rc_buffer_size = static_cast<int>(ctx->rc_max_rate / config.frame_rate_num);
|
||||
|
||||
// A long GOP saves the keyframe overhead for screen content (which
|
||||
// changes incrementally); PLI feedback recovers from loss within one
|
||||
// frame time regardless of the GOP length.
|
||||
ctx->gop_size = config.frame_rate_num * 5;
|
||||
ctx->max_b_frames = 0;
|
||||
ctx->thread_count = 1;
|
||||
ctx->profile = AV_PROFILE_H264_MAIN;
|
||||
@@ -411,7 +446,7 @@ CodecResult<std::unique_ptr<Encoder>> EncoderFactory::create(const EncoderConfig
|
||||
// without out-of-band parameter negotiation.
|
||||
ctx->flags |= AV_CODEC_FLAG_LOW_DELAY;
|
||||
|
||||
if (av_opt_set(ctx->priv_data, "preset", "ultrafast", 0) < 0) {
|
||||
if (av_opt_set(ctx->priv_data, "preset", "faster", 0) < 0) {
|
||||
return CodecError{"failed to set libx264 preset"};
|
||||
}
|
||||
if (av_opt_set(ctx->priv_data, "tune", "zerolatency", 0) < 0) {
|
||||
@@ -420,6 +455,15 @@ CodecResult<std::unique_ptr<Encoder>> EncoderFactory::create(const EncoderConfig
|
||||
if (av_opt_set(ctx->priv_data, "forced-idr", "1", 0) < 0) {
|
||||
return CodecError{"failed to enable forced IDR keyframes"};
|
||||
}
|
||||
// Screen-content tuning: auto-variance AQ allocates bits away from
|
||||
// flat areas and toward text edges; higher psy-rd preserves texture
|
||||
// sharpness at the cost of slight rate efficiency.
|
||||
if (av_opt_set(ctx->priv_data, "aq-mode", "2", 0) < 0) {
|
||||
return CodecError{"failed to set adaptive quantization mode"};
|
||||
}
|
||||
if (av_opt_set(ctx->priv_data, "psy-rd", "1.5", 0) < 0) {
|
||||
return CodecError{"failed to set psychovisual rate-distortion strength"};
|
||||
}
|
||||
|
||||
int open_ret = avcodec_open2(ctx.get(), codec, nullptr);
|
||||
if (open_ret < 0) {
|
||||
|
||||
@@ -0,0 +1,402 @@
|
||||
// GTK4 sender panel: discover receivers on the LAN, pick one, choose a
|
||||
// quality preset, and start/stop streaming. The blocking parts (portal
|
||||
// source picker, negotiation) run on worker threads so the UI stays
|
||||
// responsive.
|
||||
|
||||
#include "screencast/network/discovery.h"
|
||||
|
||||
#include "sender_session.h"
|
||||
#include "state_store.h"
|
||||
|
||||
#include <gtkmm.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <chrono>
|
||||
#include <cstdint>
|
||||
#include <map>
|
||||
#include <memory>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <thread>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
namespace {
|
||||
|
||||
using sc::DiscoveredPeer;
|
||||
using sc::Endpoint;
|
||||
|
||||
struct ReceiverRow {
|
||||
std::string name;
|
||||
std::string host;
|
||||
std::uint16_t signaling_port = 0;
|
||||
};
|
||||
|
||||
// Quality presets: CRF (visual quality), max bitrate (VBV cap), and frame
|
||||
// rate cap. Each maps to a real-world CLI invocation. The frame rate is a
|
||||
// critical bandwidth lever for desktop content — 15-25 fps is perfectly
|
||||
// smooth for screencasting and cuts bandwidth 2-4x vs. monitor rate.
|
||||
struct QualityPreset {
|
||||
std::string label;
|
||||
int crf;
|
||||
int bitrate_kbps;
|
||||
int fps; // 0 = no cap (use monitor rate)
|
||||
};
|
||||
|
||||
const std::vector<QualityPreset>& quality_presets() {
|
||||
static const std::vector<QualityPreset> presets = {
|
||||
{"Low bandwidth", 26, 3000, 15},
|
||||
{"Balanced", 22, 5000, 20},
|
||||
{"Sharp", 18, 8000, 25},
|
||||
{"Very sharp", 16, 12000, 30},
|
||||
{"Maximum", 14, 20000, 0},
|
||||
};
|
||||
return presets;
|
||||
}
|
||||
|
||||
class SenderWindow : public Gtk::ApplicationWindow {
|
||||
public:
|
||||
SenderWindow() {
|
||||
set_title("screencast");
|
||||
set_default_size(440, 440);
|
||||
// GTK4 only supports themed icons: the PNG is installed into the
|
||||
// hicolor theme (see meson.build) and picked up by name.
|
||||
set_icon_name("screencast");
|
||||
|
||||
auto* box = Gtk::make_managed<Gtk::Box>(Gtk::Orientation::VERTICAL, 8);
|
||||
set_child(*box);
|
||||
box->set_margin(12);
|
||||
|
||||
// Receiver list header + refresh.
|
||||
auto* header = Gtk::make_managed<Gtk::Box>(Gtk::Orientation::HORIZONTAL, 8);
|
||||
box->append(*header);
|
||||
auto* title = Gtk::make_managed<Gtk::Label>();
|
||||
title->set_text("Receivers");
|
||||
title->set_hexpand(true);
|
||||
title->set_halign(Gtk::Align::START);
|
||||
header->append(*title);
|
||||
refresh_button_ = Gtk::make_managed<Gtk::Button>();
|
||||
refresh_button_->set_label("Refresh");
|
||||
refresh_button_->signal_clicked().connect(sigc::mem_fun(*this, &SenderWindow::on_refresh));
|
||||
header->append(*refresh_button_);
|
||||
|
||||
scrolled_ = Gtk::make_managed<Gtk::ScrolledWindow>();
|
||||
scrolled_->set_policy(Gtk::PolicyType::NEVER, Gtk::PolicyType::AUTOMATIC);
|
||||
scrolled_->set_vexpand(true);
|
||||
box->append(*scrolled_);
|
||||
receiver_list_ = Gtk::make_managed<Gtk::ListBox>();
|
||||
receiver_list_->set_selection_mode(Gtk::SelectionMode::SINGLE);
|
||||
receiver_list_->signal_row_selected().connect(
|
||||
[this](Gtk::ListBoxRow*) { Glib::signal_idle().connect_once([this] { update_sensitivity(); }); });
|
||||
scrolled_->set_child(*receiver_list_);
|
||||
|
||||
// Quality preset dropdown.
|
||||
auto* quality_box = Gtk::make_managed<Gtk::Box>(Gtk::Orientation::HORIZONTAL, 8);
|
||||
box->append(*quality_box);
|
||||
auto* quality_label = Gtk::make_managed<Gtk::Label>();
|
||||
quality_label->set_text("Quality:");
|
||||
quality_label->set_halign(Gtk::Align::START);
|
||||
quality_box->append(*quality_label);
|
||||
quality_combo_ = Gtk::make_managed<Gtk::DropDown>();
|
||||
auto preset_list = Gtk::StringList::create({"placeholder"});
|
||||
preset_list->remove(0);
|
||||
for (const QualityPreset& preset : quality_presets()) {
|
||||
preset_list->append(preset.label);
|
||||
}
|
||||
quality_combo_->set_model(preset_list);
|
||||
quality_combo_->set_selected(1); // Standard
|
||||
quality_combo_->property_selected().signal_changed().connect([this] { on_preset_changed(); });
|
||||
quality_combo_->set_hexpand(true);
|
||||
quality_box->append(*quality_combo_);
|
||||
|
||||
// Fine-tuning: bitrate cap and frame rate, updated by the preset.
|
||||
auto* detail_box = Gtk::make_managed<Gtk::Box>(Gtk::Orientation::HORIZONTAL, 8);
|
||||
box->append(*detail_box);
|
||||
bitrate_label_ = Gtk::make_managed<Gtk::Label>();
|
||||
bitrate_label_->set_hexpand(true);
|
||||
bitrate_label_->set_halign(Gtk::Align::START);
|
||||
detail_box->append(*bitrate_label_);
|
||||
bitrate_scale_ = Gtk::make_managed<Gtk::Scale>(Gtk::Orientation::HORIZONTAL);
|
||||
bitrate_scale_->set_range(500.0, 20000.0);
|
||||
bitrate_scale_->set_value(5000.0);
|
||||
bitrate_scale_->set_increments(500.0, 1000.0);
|
||||
bitrate_scale_->set_draw_value(false);
|
||||
bitrate_scale_->set_hexpand(true);
|
||||
bitrate_scale_->signal_value_changed().connect([this] { update_quality_labels(); });
|
||||
detail_box->append(*bitrate_scale_);
|
||||
|
||||
auto* fps_box = Gtk::make_managed<Gtk::Box>(Gtk::Orientation::HORIZONTAL, 8);
|
||||
box->append(*fps_box);
|
||||
auto* fps_label = Gtk::make_managed<Gtk::Label>();
|
||||
fps_label->set_text("Frame rate:");
|
||||
fps_label->set_halign(Gtk::Align::START);
|
||||
fps_box->append(*fps_label);
|
||||
auto fps_adjustment = Gtk::Adjustment::create(20.0, 5.0, 60.0, 1.0, 5.0, 0.0);
|
||||
fps_spin_ = Gtk::make_managed<Gtk::SpinButton>(fps_adjustment, 1.0, 0);
|
||||
fps_spin_->signal_value_changed().connect([this] { update_quality_labels(); });
|
||||
fps_box->append(*fps_spin_);
|
||||
auto* fps_hint = Gtk::make_managed<Gtk::Label>();
|
||||
fps_hint->set_text("0 = uncapped (monitor rate)");
|
||||
fps_hint->set_halign(Gtk::Align::END);
|
||||
fps_hint->set_hexpand(true);
|
||||
fps_hint->set_sensitive(false);
|
||||
fps_box->append(*fps_hint);
|
||||
|
||||
start_button_ = Gtk::make_managed<Gtk::Button>();
|
||||
start_button_->set_label("Start");
|
||||
start_button_->signal_clicked().connect(sigc::mem_fun(*this, &SenderWindow::on_start_stop));
|
||||
box->append(*start_button_);
|
||||
|
||||
status_label_ = Gtk::make_managed<Gtk::Label>();
|
||||
status_label_->set_wrap(true);
|
||||
status_label_->set_halign(Gtk::Align::START);
|
||||
status_label_->set_valign(Gtk::Align::START);
|
||||
status_label_->set_vexpand(true);
|
||||
box->append(*status_label_);
|
||||
|
||||
// Per-second status refresh (elapsed time, liveness heartbeat).
|
||||
Glib::signal_timeout().connect_seconds(
|
||||
[this]() -> bool {
|
||||
update_status();
|
||||
return true;
|
||||
},
|
||||
1);
|
||||
|
||||
update_quality_labels();
|
||||
update_sensitivity();
|
||||
on_refresh();
|
||||
}
|
||||
|
||||
~SenderWindow() override {
|
||||
if (worker_.joinable()) {
|
||||
worker_.join();
|
||||
}
|
||||
}
|
||||
|
||||
private:
|
||||
int current_bitrate() const {
|
||||
return static_cast<int>(bitrate_scale_->get_value());
|
||||
}
|
||||
|
||||
int current_crf() const {
|
||||
// CRF from the preset, adjusted by the bitrate slider's distance
|
||||
// from the preset's default: moving the slider up from the preset
|
||||
// means the user wants more headroom, so we keep the preset's CRF
|
||||
// (the slider fine-tunes the cap, not the quality target).
|
||||
const auto index = quality_combo_->get_selected();
|
||||
if (index >= quality_presets().size()) {
|
||||
return 22;
|
||||
}
|
||||
return quality_presets()[index].crf;
|
||||
}
|
||||
|
||||
void on_preset_changed() {
|
||||
const auto index = quality_combo_->get_selected();
|
||||
if (index >= quality_presets().size()) {
|
||||
return;
|
||||
}
|
||||
const QualityPreset& preset = quality_presets()[index];
|
||||
bitrate_scale_->set_value(static_cast<double>(preset.bitrate_kbps));
|
||||
fps_spin_->set_value(preset.fps);
|
||||
update_quality_labels();
|
||||
}
|
||||
|
||||
void update_quality_labels() {
|
||||
bitrate_label_->set_text(std::format("Max bitrate: {} kbps · CRF {}", current_bitrate(), current_crf()));
|
||||
}
|
||||
|
||||
int current_fps() const {
|
||||
return static_cast<int>(fps_spin_->get_value());
|
||||
}
|
||||
|
||||
void clear_receiver_rows() {
|
||||
for (Gtk::Widget* row : receiver_rows_) {
|
||||
receiver_list_->remove(*row);
|
||||
}
|
||||
receiver_rows_.clear();
|
||||
receivers_.clear();
|
||||
}
|
||||
|
||||
void on_refresh() {
|
||||
refresh_button_->set_sensitive(false);
|
||||
status("discovering receivers…");
|
||||
clear_receiver_rows();
|
||||
|
||||
worker_ = std::jthread([this](std::stop_token stop_token) {
|
||||
std::vector<DiscoveredPeer> peers;
|
||||
std::mutex mutex;
|
||||
std::string error;
|
||||
|
||||
auto discovery_result = sc::DiscoveryFactory::create_avahi();
|
||||
if (sc::is_network_error(discovery_result)) {
|
||||
error = sc::network_error(discovery_result).message;
|
||||
} else {
|
||||
auto discovery = std::move(sc::network_value(discovery_result));
|
||||
if (!discovery->browse([&](const DiscoveredPeer& peer) {
|
||||
std::lock_guard lock(mutex);
|
||||
const bool known = std::any_of(peers.begin(), peers.end(), [&](const DiscoveredPeer& existing) {
|
||||
return existing.service_name == peer.service_name &&
|
||||
existing.signaling_port == peer.signaling_port;
|
||||
});
|
||||
if (!known) {
|
||||
peers.push_back(peer);
|
||||
}
|
||||
})) {
|
||||
error = discovery->last_error();
|
||||
}
|
||||
if (!stop_token.stop_requested()) {
|
||||
std::this_thread::sleep_for(std::chrono::seconds(3));
|
||||
}
|
||||
discovery->stop();
|
||||
}
|
||||
|
||||
Glib::signal_idle().connect_once([this, peers = std::move(peers), error = std::move(error)]() mutable {
|
||||
receivers_ = group_receivers(std::move(peers));
|
||||
for (const ReceiverRow& row : receivers_) {
|
||||
auto* label = Gtk::make_managed<Gtk::Label>();
|
||||
label->set_text(std::format("{}\n{}", row.name, row.host));
|
||||
label->set_halign(Gtk::Align::START);
|
||||
receiver_list_->append(*label);
|
||||
receiver_rows_.push_back(label);
|
||||
}
|
||||
refresh_button_->set_sensitive(true);
|
||||
update_sensitivity();
|
||||
status(error.empty() ? (receivers_.empty() ? "no receivers found"
|
||||
: std::format("{} receiver(s)", receivers_.size()))
|
||||
: "discovery failed: " + error);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
static std::vector<ReceiverRow> group_receivers(std::vector<DiscoveredPeer> peers) {
|
||||
std::map<std::pair<std::string, std::uint16_t>, std::vector<std::string>> grouped;
|
||||
for (DiscoveredPeer& peer : peers) {
|
||||
grouped[{peer.service_name, peer.signaling_port}].push_back(std::move(peer.host));
|
||||
}
|
||||
std::vector<ReceiverRow> rows;
|
||||
for (auto& [key, hosts] : grouped) {
|
||||
std::sort(hosts.begin(), hosts.end(), [](const std::string& lhs, const std::string& rhs) {
|
||||
return sc::address_preference(lhs) < sc::address_preference(rhs);
|
||||
});
|
||||
ReceiverRow row;
|
||||
row.name = key.first;
|
||||
row.host = std::move(hosts.front());
|
||||
row.signaling_port = key.second;
|
||||
rows.push_back(std::move(row));
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
void on_start_stop() {
|
||||
if (session_.has_value()) {
|
||||
auto session = std::move(*session_);
|
||||
session_ = std::nullopt;
|
||||
start_button_->set_sensitive(false);
|
||||
status("stopping…");
|
||||
worker_ = std::jthread([this, session = std::move(session)](std::stop_token) mutable {
|
||||
session.stop();
|
||||
Glib::signal_idle().connect_once([this] {
|
||||
start_button_->set_label("Start");
|
||||
update_sensitivity();
|
||||
status("idle");
|
||||
});
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
const int index = selected_index();
|
||||
if (index < 0) {
|
||||
status("pick a receiver first");
|
||||
return;
|
||||
}
|
||||
const ReceiverRow& receiver = receivers_[static_cast<std::size_t>(index)];
|
||||
const Endpoint signaling{receiver.host, receiver.signaling_port};
|
||||
const int bitrate = current_bitrate();
|
||||
const int crf = current_crf();
|
||||
const int fps = current_fps();
|
||||
|
||||
start_button_->set_sensitive(false);
|
||||
status(std::format("connecting to {}… (choose a source in the portal dialog)", receiver.name));
|
||||
|
||||
worker_ = std::jthread([this, signaling, bitrate, crf, fps](std::stop_token) {
|
||||
auto result = sc::SenderSession::start(signaling, bitrate, crf, fps, sc::CaptureTargetWholeScreen{});
|
||||
if (auto* error = std::get_if<std::string>(&result)) {
|
||||
Glib::signal_idle().connect_once([this, message = *error] {
|
||||
start_button_->set_label("Start");
|
||||
update_sensitivity();
|
||||
status("failed: " + message);
|
||||
});
|
||||
return;
|
||||
}
|
||||
// sigc++ slots require copyable lambdas; the move-only session
|
||||
// travels via shared_ptr.
|
||||
auto session = std::make_shared<sc::SenderSession>(std::move(std::get<sc::SenderSession>(result)));
|
||||
Glib::signal_idle().connect_once([this, session] {
|
||||
const std::string receiver_text = session->receiver();
|
||||
const std::string session_id = session->session_id();
|
||||
session_ = std::move(*session);
|
||||
start_button_->set_label("Stop");
|
||||
update_sensitivity();
|
||||
status(std::format("streaming to {} (session {})", receiver_text, session_id));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
int selected_index() const {
|
||||
Gtk::ListBoxRow* row = receiver_list_->get_selected_row();
|
||||
return row != nullptr ? row->get_index() : -1;
|
||||
}
|
||||
|
||||
void update_sensitivity() {
|
||||
if (session_.has_value()) {
|
||||
start_button_->set_sensitive(true);
|
||||
refresh_button_->set_sensitive(false);
|
||||
return;
|
||||
}
|
||||
refresh_button_->set_sensitive(true);
|
||||
start_button_->set_sensitive(selected_index() >= 0);
|
||||
}
|
||||
|
||||
void update_status() {
|
||||
if (!session_.has_value()) {
|
||||
return;
|
||||
}
|
||||
const auto state = sc::read_sender_state();
|
||||
if (state.has_value()) {
|
||||
const std::int64_t elapsed_ms = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::system_clock::now().time_since_epoch())
|
||||
.count() -
|
||||
state->started_epoch_ms;
|
||||
status(std::format("streaming to {}\nsession {}\nelapsed {}m {:02}s",
|
||||
state->receiver,
|
||||
state->session_id,
|
||||
elapsed_ms / 60000,
|
||||
(elapsed_ms / 1000) % 60));
|
||||
}
|
||||
}
|
||||
|
||||
void status(const std::string& text) {
|
||||
status_label_->set_text(text);
|
||||
}
|
||||
|
||||
std::vector<ReceiverRow> receivers_;
|
||||
std::vector<Gtk::Widget*> receiver_rows_;
|
||||
std::optional<sc::SenderSession> session_;
|
||||
std::jthread worker_;
|
||||
Gtk::Button* refresh_button_ = nullptr;
|
||||
Gtk::ScrolledWindow* scrolled_ = nullptr;
|
||||
Gtk::ListBox* receiver_list_ = nullptr;
|
||||
Gtk::DropDown* quality_combo_ = nullptr;
|
||||
Gtk::Scale* bitrate_scale_ = nullptr;
|
||||
Gtk::SpinButton* fps_spin_ = nullptr;
|
||||
Gtk::Label* bitrate_label_ = nullptr;
|
||||
Gtk::Button* start_button_ = nullptr;
|
||||
Gtk::Label* status_label_ = nullptr;
|
||||
};
|
||||
|
||||
} // namespace
|
||||
|
||||
int main(int argc, char** argv) {
|
||||
auto app = Gtk::Application::create("io.github.screen_cast.panel");
|
||||
return app->make_window_and_run<SenderWindow>(argc, argv);
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
# GTK4 sender panel, behind the `gui` option (default: false). The core
|
||||
# CLI and tests never need GTK.
|
||||
|
||||
dep_gtkmm = dependency('gtkmm-4.0')
|
||||
|
||||
screencast_gui_sources = files(
|
||||
'gui.cpp',
|
||||
)
|
||||
|
||||
executable('screencast-gui',
|
||||
screencast_gui_sources,
|
||||
include_directories : [sc_core_inc, include_directories('../app')],
|
||||
dependencies : [dep_gtkmm, sc_app_core_dep, sc_capture_dep, sc_codec_dep, sc_network_dep,
|
||||
sc_render_dep],
|
||||
install : true)
|
||||
@@ -1,6 +1,16 @@
|
||||
# Core public headers / include dependency.
|
||||
sc_core_inc = include_directories('../include')
|
||||
|
||||
# Embed the application icon so the GUI window and the SDL window icon work
|
||||
# from the build tree, after install, and on headless targets without any
|
||||
# runtime path lookup.
|
||||
icon_to_header = find_program('../scripts/icon_to_header.py')
|
||||
icon_png_data = custom_target('icon_png_data',
|
||||
input : files('../screencast_icon/screencast_256.png'),
|
||||
output : 'icon_png_data.h',
|
||||
command : [icon_to_header, '@INPUT@', '@OUTPUT@'])
|
||||
sc_icon_dep = declare_dependency(sources : [icon_png_data])
|
||||
|
||||
# The sender needs the PipeWire / xdg-desktop-portal capture backend;
|
||||
# receiver-only builds skip it entirely.
|
||||
build_sender = get_option('sender')
|
||||
@@ -32,3 +42,7 @@ subdir('network')
|
||||
subdir('render')
|
||||
|
||||
subdir('app')
|
||||
|
||||
if get_option('gui')
|
||||
subdir('gui')
|
||||
endif
|
||||
|
||||
@@ -139,7 +139,9 @@ void H264Packetizer::append_fu_a_packets(std::span<const std::byte> nal,
|
||||
}
|
||||
}
|
||||
|
||||
std::optional<std::vector<std::byte>> H264Depacketizer::depacketize(const RtpPacket& packet) {
|
||||
DepacketizeResult H264Depacketizer::depacketize(const RtpPacket& packet) {
|
||||
DepacketizeResult result;
|
||||
|
||||
// Track sequence continuity: a gap means packets were lost.
|
||||
if (last_sequence_number_.has_value()) {
|
||||
const std::uint16_t expected = static_cast<std::uint16_t>(*last_sequence_number_ + 1);
|
||||
@@ -157,6 +159,7 @@ std::optional<std::vector<std::byte>> H264Depacketizer::depacketize(const RtpPac
|
||||
// lost its tail and can no longer be recovered.
|
||||
if (frame_started_ && packet.header.timestamp != frame_timestamp_) {
|
||||
drop_frame();
|
||||
result.frame_dropped = true;
|
||||
}
|
||||
if (!frame_started_) {
|
||||
frame_started_ = true;
|
||||
@@ -216,7 +219,7 @@ std::optional<std::vector<std::byte>> H264Depacketizer::depacketize(const RtpPac
|
||||
}
|
||||
|
||||
if (!packet.header.marker) {
|
||||
return std::nullopt;
|
||||
return result;
|
||||
}
|
||||
|
||||
if (fu_active_) {
|
||||
@@ -226,9 +229,11 @@ std::optional<std::vector<std::byte>> H264Depacketizer::depacketize(const RtpPac
|
||||
fu_nal_.clear();
|
||||
}
|
||||
|
||||
std::optional<std::vector<std::byte>> result;
|
||||
if (!frame_damaged_ && !access_unit_.empty()) {
|
||||
result = std::move(access_unit_);
|
||||
result.access_unit = std::move(access_unit_);
|
||||
} else {
|
||||
// The frame that just ended is unusable.
|
||||
result.frame_dropped = true;
|
||||
}
|
||||
drop_frame();
|
||||
return result;
|
||||
|
||||
@@ -133,4 +133,59 @@ std::optional<RtpPacket> RtpPacket::parse(std::span<const std::byte> in) noexcep
|
||||
return packet;
|
||||
}
|
||||
|
||||
RtpJitterBuffer::RtpJitterBuffer(std::size_t max_depth, std::chrono::milliseconds max_delay)
|
||||
: max_depth_(max_depth), max_delay_(max_delay) {}
|
||||
|
||||
std::vector<RtpPacket> RtpJitterBuffer::push(RtpPacket packet) {
|
||||
std::vector<RtpPacket> released;
|
||||
const std::uint16_t sequence = packet.header.sequence_number;
|
||||
const auto now = std::chrono::steady_clock::now();
|
||||
|
||||
std::lock_guard lock(mutex_);
|
||||
if (!next_expected_.has_value()) {
|
||||
next_expected_ = sequence;
|
||||
}
|
||||
|
||||
// Serial-number comparison: a difference >= 32768 means the packet is
|
||||
// older than what we already delivered (a duplicate or a late straggler).
|
||||
const std::uint16_t distance = static_cast<std::uint16_t>(sequence - *next_expected_);
|
||||
if (distance >= 32768) {
|
||||
return released; // discard the straggler
|
||||
}
|
||||
|
||||
buffer_[sequence] = {now, std::move(packet)};
|
||||
|
||||
// Release the consecutive run from the expected sequence.
|
||||
while (true) {
|
||||
const auto entry = buffer_.find(*next_expected_);
|
||||
if (entry == buffer_.end()) {
|
||||
break;
|
||||
}
|
||||
released.push_back(std::move(entry->second.second));
|
||||
buffer_.erase(entry);
|
||||
++(*next_expected_);
|
||||
}
|
||||
|
||||
// A missing packet stalls the run: age out the backlog (or bound the
|
||||
// buffer) and release what is there in order, so genuine loss reaches
|
||||
// the depacketizer's gap detection rather than blocking forever.
|
||||
if (!buffer_.empty()) {
|
||||
const auto head_age = now - buffer_.begin()->second.first;
|
||||
if (head_age > max_delay_ || buffer_.size() > max_depth_) {
|
||||
for (auto& entry : buffer_) {
|
||||
released.push_back(std::move(entry.second.second));
|
||||
}
|
||||
next_expected_ = static_cast<std::uint16_t>(buffer_.rbegin()->first + 1);
|
||||
buffer_.clear();
|
||||
}
|
||||
}
|
||||
return released;
|
||||
}
|
||||
|
||||
void RtpJitterBuffer::clear() {
|
||||
std::lock_guard lock(mutex_);
|
||||
buffer_.clear();
|
||||
next_expected_.reset();
|
||||
}
|
||||
|
||||
} // namespace sc
|
||||
|
||||
@@ -115,12 +115,17 @@ std::string serialize_message(const SignalingMessage& message) {
|
||||
json["frame_rate_den"] = offer->frame_rate_den;
|
||||
json["rtp_address"] = offer->rtp_endpoint.address;
|
||||
json["rtp_port"] = offer->rtp_endpoint.port;
|
||||
} else {
|
||||
const SessionAnswer& answer = std::get<SessionAnswer>(message);
|
||||
} else if (const SessionAnswer* answer = std::get_if<SessionAnswer>(&message)) {
|
||||
json["type"] = "answer";
|
||||
json["session_id"] = answer.session_id;
|
||||
json["rtp_address"] = answer.rtp_endpoint.address;
|
||||
json["rtp_port"] = answer.rtp_endpoint.port;
|
||||
json["session_id"] = answer->session_id;
|
||||
json["rtp_address"] = answer->rtp_endpoint.address;
|
||||
json["rtp_port"] = answer->rtp_endpoint.port;
|
||||
json["display_width"] = answer->display_width;
|
||||
json["display_height"] = answer->display_height;
|
||||
} else {
|
||||
const SessionPli& pli = std::get<SessionPli>(message);
|
||||
json["type"] = "pli";
|
||||
json["session_id"] = pli.session_id;
|
||||
}
|
||||
return json.dump() + "\n";
|
||||
}
|
||||
@@ -169,8 +174,19 @@ std::optional<SignalingMessage> parse_message(std::string_view line) {
|
||||
SessionAnswer answer;
|
||||
answer.session_id = session_id;
|
||||
answer.rtp_endpoint = rtp_endpoint;
|
||||
if (json.contains("display_width") && json.at("display_width").is_number_integer()) {
|
||||
answer.display_width = json.at("display_width").get<int>();
|
||||
}
|
||||
if (json.contains("display_height") && json.at("display_height").is_number_integer()) {
|
||||
answer.display_height = json.at("display_height").get<int>();
|
||||
}
|
||||
return answer;
|
||||
}
|
||||
if (type == "pli") {
|
||||
SessionPli pli;
|
||||
pli.session_id = session_id;
|
||||
return pli;
|
||||
}
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
|
||||
@@ -2,14 +2,30 @@
|
||||
|
||||
dep_sdl3 = dependency('sdl3')
|
||||
|
||||
# SDL3_image (window icon) is optional: no pkg-config file ships with it,
|
||||
# so find the library directly. Builds without it (e.g. the Pi receiver)
|
||||
# simply skip the window icon.
|
||||
cc = meson.get_compiler('cpp')
|
||||
dep_sdl3_image = cc.find_library('SDL3_image', required: false)
|
||||
if not dep_sdl3_image.found()
|
||||
dep_sdl3_image = disabler()
|
||||
endif
|
||||
|
||||
sc_render_args = []
|
||||
if dep_sdl3_image.found() and cc.has_header('SDL3_image/SDL_image.h', dependencies : dep_sdl3_image)
|
||||
sc_render_args += ['-DSC_HAS_WINDOW_ICON=1']
|
||||
endif
|
||||
|
||||
sc_render_sources = files('sdl_renderer.cpp')
|
||||
|
||||
sc_render = static_library('sc_render',
|
||||
sc_render_sources,
|
||||
include_directories : sc_core_inc,
|
||||
dependencies : [dep_sdl3])
|
||||
cpp_args : sc_render_args,
|
||||
dependencies : [dep_sdl3, dep_sdl3_image, sc_icon_dep])
|
||||
|
||||
sc_render_dep = declare_dependency(
|
||||
link_with : sc_render,
|
||||
include_directories : sc_core_inc,
|
||||
dependencies : [dep_sdl3])
|
||||
compile_args : sc_render_args,
|
||||
dependencies : [dep_sdl3, dep_sdl3_image])
|
||||
@@ -4,13 +4,23 @@
|
||||
|
||||
#include <string>
|
||||
|
||||
#ifdef SC_HAS_WINDOW_ICON
|
||||
|
||||
#include <SDL3_image/SDL_image.h>
|
||||
|
||||
#include <cstdio>
|
||||
|
||||
#include "icon_png_data.h"
|
||||
|
||||
#endif
|
||||
|
||||
namespace sc {
|
||||
namespace {
|
||||
|
||||
// DecodedFrame pixels are AV_PIX_FMT_RGBA: memory order R, G, B, A. SDL
|
||||
// names 32-bit formats MSB-first, so that byte order is SDL's ABGR8888 —
|
||||
// using RGBA8888 would read the alpha byte as red and tint the image red.
|
||||
constexpr SDL_PixelFormat kSdlPixelFormat = SDL_PIXELFORMAT_ABGR8888;
|
||||
// DecodedFrame carries YUV420P planar data; SDL_PIXELFORMAT_IYUV is the
|
||||
// matching SDL texture format. The GPU does the YUV→RGB conversion during
|
||||
// rendering, eliminating a CPU-side swscale pass.
|
||||
constexpr SDL_PixelFormat kSdlPixelFormat = SDL_PIXELFORMAT_IYUV;
|
||||
|
||||
class SdlRenderer final : public Renderer {
|
||||
public:
|
||||
@@ -52,6 +62,10 @@ class SdlRenderer final : public Renderer {
|
||||
(void)SDL_HideCursor();
|
||||
}
|
||||
|
||||
#ifdef SC_HAS_WINDOW_ICON
|
||||
set_window_icon();
|
||||
#endif
|
||||
|
||||
renderer_ = SDL_CreateRenderer(window_, nullptr);
|
||||
if (renderer_ == nullptr) {
|
||||
last_error_ = std::string{"SDL_CreateRenderer failed: "} + SDL_GetError();
|
||||
@@ -59,6 +73,14 @@ class SdlRenderer final : public Renderer {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Cache the display resolution: in fullscreen (KMSDRM on the Pi) this
|
||||
// is the native monitor size; in windowed mode it is the window.
|
||||
int window_width = 0;
|
||||
int window_height = 0;
|
||||
SDL_GetWindowSize(window_, &window_width, &window_height);
|
||||
display_width_ = window_width;
|
||||
display_height_ = window_height;
|
||||
|
||||
// Commit the surface once: on Wayland a window only becomes visible
|
||||
// after the first present, and the receiver must be visible while it
|
||||
// waits for the stream to start.
|
||||
@@ -73,8 +95,16 @@ class SdlRenderer final : public Renderer {
|
||||
return true;
|
||||
}
|
||||
|
||||
int display_width() const override {
|
||||
return display_width_;
|
||||
}
|
||||
|
||||
int display_height() const override {
|
||||
return display_height_;
|
||||
}
|
||||
|
||||
bool present(const DecodedFrame& frame) override {
|
||||
if (frame.width <= 0 || frame.height <= 0) {
|
||||
if (frame.width <= 0 || frame.height <= 0 || frame.plane_y.empty()) {
|
||||
return false;
|
||||
}
|
||||
if (frame.width != texture_width_ || frame.height != texture_height_) {
|
||||
@@ -83,8 +113,14 @@ class SdlRenderer final : public Renderer {
|
||||
}
|
||||
}
|
||||
|
||||
const int pitch = frame.width * 4;
|
||||
if (!SDL_UpdateTexture(texture_, nullptr, frame.rgba_pixels.data(), pitch)) {
|
||||
if (!SDL_UpdateYUVTexture(texture_,
|
||||
nullptr,
|
||||
reinterpret_cast<const uint8_t*>(frame.plane_y.data()),
|
||||
frame.stride_y,
|
||||
reinterpret_cast<const uint8_t*>(frame.plane_u.data()),
|
||||
frame.stride_u,
|
||||
reinterpret_cast<const uint8_t*>(frame.plane_v.data()),
|
||||
frame.stride_v)) {
|
||||
return false;
|
||||
}
|
||||
// Clear first so the letterbox bars stay black between frames.
|
||||
@@ -133,6 +169,29 @@ class SdlRenderer final : public Renderer {
|
||||
}
|
||||
|
||||
private:
|
||||
#ifdef SC_HAS_WINDOW_ICON
|
||||
// The icon is embedded at build time (see icon_png_data.h), so it works
|
||||
// without any runtime file lookup. Harmless under KMSDRM, where there is
|
||||
// no window manager to display it.
|
||||
void set_window_icon() {
|
||||
auto* stream = SDL_IOFromConstMem(sc::app_icon_png.data(), sc::app_icon_png.size());
|
||||
if (stream == nullptr) {
|
||||
std::fprintf(stderr, "screencast: failed to create icon stream: %s\n", SDL_GetError());
|
||||
return;
|
||||
}
|
||||
// closeio=true: SDL_image consumes the stream, success or failure.
|
||||
SDL_Surface* icon_surface = IMG_Load_IO(stream, true);
|
||||
if (icon_surface == nullptr) {
|
||||
std::fprintf(stderr, "screencast: failed to load window icon: %s\n", SDL_GetError());
|
||||
return;
|
||||
}
|
||||
if (SDL_SetWindowIcon(window_, icon_surface) != 0) {
|
||||
std::fprintf(stderr, "screencast: failed to set window icon: %s\n", SDL_GetError());
|
||||
}
|
||||
SDL_DestroySurface(icon_surface);
|
||||
}
|
||||
#endif
|
||||
|
||||
bool recreate_texture(int width, int height) {
|
||||
if (texture_ != nullptr) {
|
||||
SDL_DestroyTexture(texture_);
|
||||
@@ -164,6 +223,8 @@ class SdlRenderer final : public Renderer {
|
||||
SDL_Texture* texture_ = nullptr;
|
||||
int texture_width_ = 0;
|
||||
int texture_height_ = 0;
|
||||
int display_width_ = 0;
|
||||
int display_height_ = 0;
|
||||
};
|
||||
|
||||
} // namespace
|
||||
|
||||
@@ -74,13 +74,13 @@ struct ReceiverSink {
|
||||
int last_height = 0;
|
||||
|
||||
void on_packet(sc::RtpPacket packet) {
|
||||
std::optional<std::vector<std::byte>> access_unit = depacketizer.depacketize(packet);
|
||||
if (!access_unit.has_value()) {
|
||||
const sc::DepacketizeResult result = depacketizer.depacketize(packet);
|
||||
if (!result.access_unit.has_value()) {
|
||||
return;
|
||||
}
|
||||
|
||||
sc::EncodedFrame encoded;
|
||||
encoded.data = std::move(*access_unit);
|
||||
encoded.data = std::move(*result.access_unit);
|
||||
encoded.rtp_timestamp = packet.header.timestamp;
|
||||
|
||||
auto decoded_result = decoder->decode(encoded);
|
||||
@@ -90,7 +90,7 @@ struct ReceiverSink {
|
||||
for (const sc::DecodedFrame& decoded : sc::codec_value(decoded_result)) {
|
||||
last_width = decoded.width;
|
||||
last_height = decoded.height;
|
||||
saw_frame = decoded.width > 0 && !decoded.rgba_pixels.empty();
|
||||
saw_frame = decoded.width > 0 && !decoded.plane_y.empty();
|
||||
decoded_frames.fetch_add(1);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,12 @@
|
||||
#include "screencast/codec/decoder.h"
|
||||
#include "screencast/codec/encoder.h"
|
||||
|
||||
#include <algorithm>
|
||||
// swscale is a C library; without the extern wrapper its functions get
|
||||
// C++ mangled and the linker cannot find them.
|
||||
extern "C" {
|
||||
#include <libswscale/swscale.h>
|
||||
}
|
||||
|
||||
#include <cassert>
|
||||
#include <cstdint>
|
||||
#include <cstdlib>
|
||||
@@ -47,10 +52,45 @@ sc::CapturedFrame make_frame(uint32_t index) {
|
||||
return frame;
|
||||
}
|
||||
|
||||
// Convert a decoded YUV420P frame back to RGBA for pixel comparison with
|
||||
// the original capture. Test-only; the pipeline itself never converts.
|
||||
std::vector<std::byte> decoded_to_rgba(const sc::DecodedFrame& decoded) {
|
||||
const std::size_t rgba_size = static_cast<std::size_t>(decoded.width) * decoded.height * 4;
|
||||
std::vector<std::byte> rgba(rgba_size);
|
||||
|
||||
const uint8_t* src_planes[4] = {
|
||||
reinterpret_cast<const uint8_t*>(decoded.plane_y.data()),
|
||||
reinterpret_cast<const uint8_t*>(decoded.plane_u.data()),
|
||||
reinterpret_cast<const uint8_t*>(decoded.plane_v.data()),
|
||||
nullptr,
|
||||
};
|
||||
const int src_strides[4] = {decoded.stride_y, decoded.stride_u, decoded.stride_v, 0};
|
||||
|
||||
uint8_t* dst_planes[4] = {reinterpret_cast<uint8_t*>(rgba.data()), nullptr, nullptr, nullptr};
|
||||
const int dst_strides[4] = {decoded.width * 4, 0, 0, 0};
|
||||
|
||||
SwsContext* scaler = sws_getContext(decoded.width,
|
||||
decoded.height,
|
||||
AV_PIX_FMT_YUV420P,
|
||||
decoded.width,
|
||||
decoded.height,
|
||||
AV_PIX_FMT_RGBA,
|
||||
SWS_BILINEAR,
|
||||
nullptr,
|
||||
nullptr,
|
||||
nullptr);
|
||||
assert(scaler != nullptr);
|
||||
(void)sws_scale(scaler, src_planes, src_strides, 0, decoded.height, dst_planes, dst_strides);
|
||||
sws_freeContext(scaler);
|
||||
|
||||
return rgba;
|
||||
}
|
||||
|
||||
int max_channel_difference(const sc::DecodedFrame& decoded, const sc::CapturedFrame& expected) {
|
||||
const auto* decoded_pixels = reinterpret_cast<const std::uint8_t*>(decoded.rgba_pixels.data());
|
||||
const auto decoded_rgba = decoded_to_rgba(decoded);
|
||||
const auto* decoded_pixels = reinterpret_cast<const std::uint8_t*>(decoded_rgba.data());
|
||||
const auto* expected_pixels = reinterpret_cast<const std::uint8_t*>(expected.pixels.data());
|
||||
const std::size_t count = std::min(decoded.rgba_pixels.size(), expected.pixels.size());
|
||||
const std::size_t count = std::min(decoded_rgba.size(), expected.pixels.size());
|
||||
|
||||
int max_diff = 0;
|
||||
for (std::size_t i = 0; i < count; ++i) {
|
||||
|
||||