Files

T

fegger 50d00286d7 Android: remote transcription via whisper.cpp server (Phase 2)

- new Transcriber interface makes the recorder service engine-agnostic;
  WhisperEngine (local JNI) and new RemoteWhisperEngine (POST /inference,
  verbose_json, in-memory WAV upload via new WavEncoder) implement it
- SessionConfig carries serverUrl; when set, BOTH the live pass and the
  final pass transcribe remotely (server owns the model, e.g. large-v3
  on GPU); local engine remains the offline fallback
- UI: Transcribe on: phone/server dropdown + server URL field,
  persisted in SharedPreferences; model controls grey out in server
  mode; manual Transcribe also routes to the server
- network security config permits cleartext HTTP for user-configured
  LAN/tailnet servers (documented; HTTPS works either way)
- WavEncoder round-trip unit test (11 total green); validated phone ->
  Tailscale -> R9700 with large-v3 before the app-side change

2026-09-07 13:26:02 +02:00

3.4 KiB

Raw Blame History

MeetRec for Android

Native Android app with the same functionality as the desktop meetrec: record meetings and transcribe them on-device with Whisper. Nothing leaves the phone — no cloud, no telemetry.

Status: M2.5 (remote transcription) — record on the phone while a remote whisper.cpp server (GPU) provides live and final transcripts. The phone's local whisper.cpp engine remains as the offline fallback.

Transcription targets

phone — the app's local whisper.cpp engine (JNI/NEON): pick a model (tiny/base for live, e.g. small for the final pass). Measured on the Fairphone 6: ~0.6–0.8× realtime, so keep local models small.

server — a remote whisper.cpp server (Docker Compose, Vulkan GPU) transcribes BOTH the live pass and the final pass; the model lives on the server (e.g. large-v3). The phone only records and displays. Set the server URL in the app (persisted; e.g. http://100.103.83.12:8085 over Tailscale — the phone needs Tailscale too). Local model selection is greyed out in this mode.

Requirements

Android Studio (or: SDK Platform 36, Build Tools 36, NDK 27.1, CMake 3.22.1)
JDK 17+
Device running Android 10+ (developed against a Fairphone 6 / Snapdragon 7s Gen 3, Android 15+)

Build

cd android
./tools/fetch-whisper.sh     # vendors whisper.cpp v1.9.3 into third_party/
./gradlew :app:assembleDebug # or open the android/ folder in Android Studio
adb install -r app/build/outputs/apk/debug/app-debug.apk

third_party/whisper.cpp is gitignored — the fetch script pins the exact release tag so the JNI layer never breaks on upstream churn.

Test on device

Pick Transcribe on: phone or server. For server: enter the URL (needs Tailscale on the phone for a Tailscale-only server).
Phone mode: Download + Load engine first (tiny is fine for a start).
Tap Record — live transcript lines appear every few seconds.
Tap Stop — the final pass runs automatically and writes <stem>.txt/.srt/.json next to the WAV.
Share the transcript via the share sheet. You can also pick any PCM WAV file and transcribe it manually.

Expect roughly realtime transcription with tiny/base on the Fairphone 6's CPU; small is noticeably slower — use it for final passes only (the live/final split comes with the recorder milestones).

Performance notes

The engine runs on CPU via NEON, using up to 4 threads.
GPU/NPU acceleration (e.g. Snapdragon NPU via the QNN backend) is a stretch goal, not wired up yet.

Module layout

app/          Compose UI + RecorderService (record → live pass → final pass)
core/recording/ MeetingRecorder, WavWriter/WavReader, RollingWindow,
              linear resampler — pure Kotlin, unit tested on the JVM
core/whisper/ whisper.cpp JNI wrapper + WhisperEngine + TranscriptFiles
              (txt/srt/json writers, desktop-compatible formats)
tools/        fetch-whisper.sh — vendor the pinned whisper.cpp release

Recording mirrors the desktop app: audio is captured at the input's native rate (WAV on disk keeps it), while the 60 s rolling window is resampled to Whisper's 16 kHz for the upcoming live pass. Recording continues while the app is backgrounded via the microphone foreground service.

Models live in the app's private storage (filesDir/models), shared files with the desktop app's ~/.cache/meetrec/whisper-cpp naming (ggml-tiny.bin … ggml-large-v3.bin).

3.4 KiB Raw Blame History Unescape Escape