Android: remote transcription via whisper.cpp server (Phase 2)

- new Transcriber interface makes the recorder service engine-agnostic;
  WhisperEngine (local JNI) and new RemoteWhisperEngine (POST /inference,
  verbose_json, in-memory WAV upload via new WavEncoder) implement it
- SessionConfig carries serverUrl; when set, BOTH the live pass and the
  final pass transcribe remotely (server owns the model, e.g. large-v3
  on GPU); local engine remains the offline fallback
- UI: Transcribe on: phone/server dropdown + server URL field,
  persisted in SharedPreferences; model controls grey out in server
  mode; manual Transcribe also routes to the server
- network security config permits cleartext HTTP for user-configured
  LAN/tailnet servers (documented; HTTPS works either way)
- WavEncoder round-trip unit test (11 total green); validated phone ->
  Tailscale -> R9700 with large-v3 before the app-side change
This commit is contained in:
2026-09-07 13:26:02 +02:00
parent a1df9d4037
commit 50d00286d7
10 changed files with 328 additions and 44 deletions
+22 -15
View File
@@ -4,10 +4,22 @@ Native Android app with the same functionality as the desktop meetrec:
record meetings and transcribe them on-device with Whisper. Nothing leaves
the phone — no cloud, no telemetry.
Status: **M2 (live transcript + automatic final pass)** — record a meeting
and watch the live rolling transcript while recording; on Stop the final
(better) pass runs automatically and saves `.txt`/`.srt`/`.json` next to
the WAV, mirroring the desktop two-pass design.
Status: **M2.5 (remote transcription)** — record on the phone while a
remote whisper.cpp server (GPU) provides live and final transcripts.
The phone's local whisper.cpp engine remains as the offline fallback.
## Transcription targets
**phone** — the app's local whisper.cpp engine (JNI/NEON): pick a model
(tiny/base for live, e.g. small for the final pass). Measured on the
Fairphone 6: ~0.60.8× realtime, so keep local models small.
**server** — a remote [whisper.cpp server](../server/whisper-server/)
(Docker Compose, Vulkan GPU) transcribes BOTH the live pass and the final
pass; the model lives on the server (e.g. large-v3). The phone only
records and displays. Set the server URL in the app (persisted; e.g.
`http://100.103.83.12:8085` over Tailscale — the phone needs Tailscale
too). Local model selection is greyed out in this mode.
## Requirements
@@ -30,20 +42,15 @@ release tag so the JNI layer never breaks on upstream churn.
## Test on device
1. Launch **MeetRec**, pick a model (`tiny` is fine for a first test) and tap
**Download** (model comes from Hugging Face; tiny is ~75 MB).
2. Tap **Load engine**.
3. Tap **Record** — live transcript lines appear every few seconds (they use
the cheap *live model*, `tiny`/`base`, on a 60 s rolling window).
4. Tap **Stop** — the final pass runs automatically with the selected model
and writes `<stem>.txt/.srt/.json` next to the WAV.
1. Pick **Transcribe on: phone or server**. For server: enter the URL
(needs Tailscale on the phone for a Tailscale-only server).
2. Phone mode: Download + Load engine first (tiny is fine for a start).
3. Tap **Record** — live transcript lines appear every few seconds.
4. Tap **Stop** — the final pass runs automatically and writes
`<stem>.txt/.srt/.json` next to the WAV.
5. Share the transcript via the share sheet. You can also pick any PCM WAV
file and transcribe it manually.
Tip: use `tiny`/`base` as the live model (they keep up on a phone CPU) and
a larger model like `small` for the final pass. The **Live model** dropdown
also has `off` to disable live transcription (longest battery life).
Expect roughly realtime transcription with `tiny`/`base` on the
Fairphone 6's CPU; `small` is noticeably slower — use it for final passes
only (the live/final split comes with the recorder milestones).