fegger
c43dd3ca4a
A native iOS receiver so an iPhone can act as the second receiver, speaking the existing signaling + RTP protocol (no C++ changes) and mirroring the Android receiver (Phase 8) source-to-source. - RTP core (header/packet, jitter buffer, H.264 depacketizer) ported from the Android receiver - BSD-socket signaling server (dual-stack, most-recent-peer, never-throwing sends) + NSBonjourServices - VideoToolbox H.264 decode (in-band SPS/PPS, real-time, rebuilds on size change) -> AVSampleBufferDisplayLayer - PLI keyframe recovery (500 ms) + pendingOffer for late surface attach - XcodeGen project + bootstrap.sh; XCTest port of the Android suite + new coverage - .gitignore for generated artifacts; CHANGELOG; PHASES + MEMORY updated Status: authored; on-device validation pending a Mac + Xcode 26 + iPhone 16.
78 lines
2.9 KiB
Markdown
78 lines
2.9 KiB
Markdown
# screencast iOS receiver
|
|
|
|
A native iOS receiver app (Swift) so an iPhone can act as a second receiver.
|
|
It speaks the **existing signaling + RTP protocol unchanged** — no C++ changes —
|
|
mirroring the Android receiver (Phase 8) source-to-source.
|
|
|
|
- **Min iOS:** 17 (validated on iPhone 16 / iOS 26.6.1)
|
|
- **No third-party dependencies** — only Apple system frameworks (SwiftUI,
|
|
AVFoundation, VideoToolbox, CoreMedia, BSD sockets).
|
|
|
|
## Pipeline
|
|
|
|
```
|
|
Bonjour advertise (_screencast._tcp) + TCP signaling server (offer → answer)
|
|
UDP RTP → jitter buffer (16 pkt / 60 ms) → depacketize (single-NAL + FU-A)
|
|
→ VideoToolbox H.264 (in-band SPS/PPS) → AVSampleBufferDisplayLayer (letterbox)
|
|
```
|
|
|
|
Recovery matches the C++ receiver: a damaged frame is dropped and a PLI
|
|
(rate-limited to one per 500 ms) asks the sender for a keyframe.
|
|
|
|
## Layout
|
|
|
|
```
|
|
Receiver/
|
|
App/ SwiftUI app, video surface (AVSampleBufferDisplayLayer), controller
|
|
Rtp/ RtpHeader, RtpPacket, JitterBuffer, H264Depacketizer, Avcc, NAL extraction
|
|
Signaling/ SignalingMessage (JSON), LineAssembler, SignalingServer (BSD sockets)
|
|
Decode/ RenderSink, H.264 format description, VideoToolbox decoder
|
|
Support/ UdpTransport (poll-based), LocalAddress
|
|
Pipeline/ ReceiverPipeline (coordinates everything)
|
|
ReceiverTests/
|
|
XCTest port of the Android JVM tests + added coverage (signaling, line
|
|
framing, AVCC, NAL extraction)
|
|
```
|
|
|
|
## Build and test (on a Mac with Xcode 26)
|
|
|
|
```sh
|
|
./bootstrap.sh # installs XcodeGen, generates the project, builds for device
|
|
./bootstrap.sh test # runs the unit tests on the simulator
|
|
```
|
|
|
|
`bootstrap.sh` downloads XcodeGen from the GitHub release into `tools/` (no
|
|
Homebrew). Override the simulator with `IOS_DEST="platform=iOS Simulator,name=…"`.
|
|
|
|
## Install on a device
|
|
|
|
Free provisioning (or your team) is set in Xcode — this can't be scripted:
|
|
|
|
1. Open `Receiver.xcodeproj`.
|
|
2. Select the **Receiver** target → *Signing & Capabilities* → set your Apple ID
|
|
(a 7-day development certificate is fine for sideloading).
|
|
3. Run to the iPhone (USB, trust the computer).
|
|
|
|
The first run prompts for **Local Network** access — allow it, or the sender
|
|
will never discover the phone (silent failure, like the Android
|
|
`PROTOCOL_DNS_SD` bug).
|
|
|
|
## Stream to the phone
|
|
|
|
On the sender (Linux desktop):
|
|
|
|
```sh
|
|
screencast --send --peer <phone-ip>:5005 # target the phone directly
|
|
screencast --discover # or list receivers; the phone shows up
|
|
```
|
|
|
|
Expected: the phone shows the captured desktop, letterboxed to its screen, with
|
|
PLI recovery on Wi-Fi loss.
|
|
|
|
## Testing notes
|
|
|
|
The pure protocol core (RTP, jitter, depacketizer, signaling JSON, line framing,
|
|
AVCC, NAL extraction) is unit-tested in the simulator. The VideoToolbox decode
|
|
path and the local-network/Bonjour flow are on-device validation items — the
|
|
highest-risk parts to verify on the borrowed Mac + iPhone.
|