Files

T

fegger 46b0f128bc Android M2: live rolling transcript and automatic final pass

- RecorderService: rolling-window live pass with a dedicated live model
  (off/tiny/base, beam 1, 8 s ticks, time-based dedupe) and an automatic
  final pass with the selected model (beam 5) writing txt/srt/json
- TranscriptFiles in core/whisper: desktop-compatible outputs, 5 JVM tests
- WhisperEngine exposes modelName; UI: live-model dropdown, merged
  transcript view, share sheet; falls back to manual path without engine
- live loop failures now logged (MeetRec tag) and surfaced in the UI
  (was silently swallowed), plus final-pass timing logs

On-device measurements (Fairphone 6): tiny live ~0.6x realtime, small
final ~0.8x realtime — motivates the planned whisper-server engine.

2026-09-07 12:14:24 +02:00

3.1 KiB

Raw Blame History

MeetRec for Android

Native Android app with the same functionality as the desktop meetrec: record meetings and transcribe them on-device with Whisper. Nothing leaves the phone — no cloud, no telemetry.

Status: M2 (live transcript + automatic final pass) — record a meeting and watch the live rolling transcript while recording; on Stop the final (better) pass runs automatically and saves .txt/.srt/.json next to the WAV, mirroring the desktop two-pass design.

Requirements

Android Studio (or: SDK Platform 36, Build Tools 36, NDK 27.1, CMake 3.22.1)
JDK 17+
Device running Android 10+ (developed against a Fairphone 6 / Snapdragon 7s Gen 3, Android 15+)

Build

cd android
./tools/fetch-whisper.sh     # vendors whisper.cpp v1.9.3 into third_party/
./gradlew :app:assembleDebug # or open the android/ folder in Android Studio
adb install -r app/build/outputs/apk/debug/app-debug.apk

third_party/whisper.cpp is gitignored — the fetch script pins the exact release tag so the JNI layer never breaks on upstream churn.

Test on device

Launch MeetRec, pick a model (tiny is fine for a first test) and tap Download (model comes from Hugging Face; tiny is ~75 MB).
Tap Load engine.
Tap Record — live transcript lines appear every few seconds (they use the cheap live model, tiny/base, on a 60 s rolling window).
Tap Stop — the final pass runs automatically with the selected model and writes <stem>.txt/.srt/.json next to the WAV.
Share the transcript via the share sheet. You can also pick any PCM WAV file and transcribe it manually.

Tip: use tiny/base as the live model (they keep up on a phone CPU) and a larger model like small for the final pass. The Live model dropdown also has off to disable live transcription (longest battery life).

Expect roughly realtime transcription with tiny/base on the Fairphone 6's CPU; small is noticeably slower — use it for final passes only (the live/final split comes with the recorder milestones).

Performance notes

The engine runs on CPU via NEON, using up to 4 threads.
GPU/NPU acceleration (e.g. Snapdragon NPU via the QNN backend) is a stretch goal, not wired up yet.

Module layout

app/          Compose UI + RecorderService (record → live pass → final pass)
core/recording/ MeetingRecorder, WavWriter/WavReader, RollingWindow,
              linear resampler — pure Kotlin, unit tested on the JVM
core/whisper/ whisper.cpp JNI wrapper + WhisperEngine + TranscriptFiles
              (txt/srt/json writers, desktop-compatible formats)
tools/        fetch-whisper.sh — vendor the pinned whisper.cpp release

Recording mirrors the desktop app: audio is captured at the input's native rate (WAV on disk keeps it), while the 60 s rolling window is resampled to Whisper's 16 kHz for the upcoming live pass. Recording continues while the app is backgrounded via the microphone foreground service.

Models live in the app's private storage (filesDir/models), shared files with the desktop app's ~/.cache/meetrec/whisper-cpp naming (ggml-tiny.bin … ggml-large-v3.bin).

3.1 KiB Raw Blame History