fegger
2cf785746e
- meetrec-server: Ollama integration (chat API, gemma4:12b, num_ctx 32768); German structured summary (topic/points/decisions/to-dos) written to summary.md; agenda coverage returns strict JSON (covered, time, evidence) parsed defensively; both run automatically in a background thread after upload plus manual trigger endpoints (POST /summary, POST /agenda) with status tracking in the index - phone: agenda input on the Record tab (one item per line, persisted) is uploaded with the recording; Library detail shows the summary and a per-item agenda checklist with timestamps and evidence quotes, with polling while the server generates and manual re-trigger buttons - validated end-to-end against the live Ollama server: crafted German test meeting produced a correct structured summary and perfect agenda discrimination (covered items with correct timestamps + quotes, undiscussed item correctly false)
113 lines
4.7 KiB
Markdown
113 lines
4.7 KiB
Markdown
# MeetRec for Android
|
||
|
||
Native Android app with the same functionality as the desktop meetrec:
|
||
record meetings and transcribe them on-device with Whisper. Nothing leaves
|
||
the phone — no cloud, no telemetry.
|
||
|
||
Status: **M2.5 (remote transcription)** — record on the phone while a
|
||
remote whisper.cpp server (GPU) provides live and final transcripts.
|
||
The phone's local whisper.cpp engine remains as the offline fallback.
|
||
|
||
## Transcription targets
|
||
|
||
**phone** — the app's local whisper.cpp engine (JNI/NEON): pick a model
|
||
(tiny/base for live, e.g. small for the final pass). Measured on the
|
||
Fairphone 6: ~0.6–0.8× realtime, so keep local models small.
|
||
|
||
**server** — a remote [whisper.cpp server](../whisper-server/)
|
||
(Docker Compose, Vulkan GPU) transcribes BOTH the live pass and the final
|
||
pass; the model lives on the server (e.g. large-v3). The phone only
|
||
records and displays. Set the server URL in the app (persisted; e.g.
|
||
`http://100.103.83.12:8085` over Tailscale — the phone needs Tailscale
|
||
too). Local model selection is greyed out in this mode.
|
||
|
||
## Recording library (meetrec-server)
|
||
|
||
Finished recordings (WAV + txt/srt/json) upload automatically to the
|
||
[meetrec-server](../meetrec-server/) storage service. Set the **Library
|
||
URL** in the app (`http://100.103.83.12:8090`, persisted) or leave it
|
||
empty to disable upload. Uploads run in the background after the final
|
||
pass; the status line reports the result.
|
||
|
||
The **Library** tab browses everything:
|
||
|
||
- *On this phone* — local recordings (with or without transcripts)
|
||
- *On the server* — stored bundles with date, duration, language
|
||
- Detail view: timestamped transcript, audio playback (streamed from
|
||
the server or from the local file), share, and delete with
|
||
confirmation. Refresh reloads both lists.
|
||
|
||
## Summaries and agenda (Ollama)
|
||
|
||
Enter **agenda items on the Record tab (one per line, persisted)** before
|
||
recording. After the recording uploads, meetrec-server automatically:
|
||
|
||
1. writes a German **summary** (gemma4:12b via your Ollama server):
|
||
topic, key points, decisions, to-dos — shown on the recording's
|
||
detail page
|
||
2. checks **which agenda items were actually discussed** — the detail
|
||
page shows a ✓/✗ checklist with the timestamp and a quote as evidence
|
||
|
||
Both can be re-triggered from the detail page ("Neu erstellen" /
|
||
"Agenda neu prüfen"). Note: very long meetings may exceed the LLM's
|
||
context window (the transcript is then truncated).
|
||
|
||
## Requirements
|
||
|
||
- Android Studio (or: SDK Platform 36, Build Tools 36, NDK 27.1, CMake 3.22.1)
|
||
- JDK 17+
|
||
- Device running Android 10+ (developed against a Fairphone 6 / Snapdragon
|
||
7s Gen 3, Android 15+)
|
||
|
||
## Build
|
||
|
||
```sh
|
||
cd android
|
||
./tools/fetch-whisper.sh # vendors whisper.cpp v1.9.3 into third_party/
|
||
./gradlew :app:assembleDebug # or open the android/ folder in Android Studio
|
||
adb install -r app/build/outputs/apk/debug/app-debug.apk
|
||
```
|
||
|
||
`third_party/whisper.cpp` is gitignored — the fetch script pins the exact
|
||
release tag so the JNI layer never breaks on upstream churn.
|
||
|
||
## Test on device
|
||
|
||
1. Pick **Transcribe on: phone or server**. For server: enter the URL
|
||
(needs Tailscale on the phone for a Tailscale-only server).
|
||
2. Phone mode: Download + Load engine first (tiny is fine for a start).
|
||
3. Tap **Record** — live transcript lines appear every few seconds.
|
||
4. Tap **Stop** — the final pass runs automatically and writes
|
||
`<stem>.txt/.srt/.json` next to the WAV.
|
||
5. Share the transcript via the share sheet. You can also pick any PCM WAV
|
||
file and transcribe it manually.
|
||
|
||
Expect roughly realtime transcription with `tiny`/`base` on the
|
||
Fairphone 6's CPU; `small` is noticeably slower — use it for final passes
|
||
only (the live/final split comes with the recorder milestones).
|
||
|
||
## Performance notes
|
||
|
||
- The engine runs on CPU via NEON, using up to 4 threads.
|
||
- GPU/NPU acceleration (e.g. Snapdragon NPU via the QNN backend) is a
|
||
stretch goal, not wired up yet.
|
||
|
||
## Module layout
|
||
|
||
```
|
||
app/ Compose UI + RecorderService (record → live pass → final pass)
|
||
core/recording/ MeetingRecorder, WavWriter/WavReader, RollingWindow,
|
||
linear resampler — pure Kotlin, unit tested on the JVM
|
||
core/whisper/ whisper.cpp JNI wrapper + WhisperEngine + TranscriptFiles
|
||
(txt/srt/json writers, desktop-compatible formats)
|
||
tools/ fetch-whisper.sh — vendor the pinned whisper.cpp release
|
||
```
|
||
|
||
Recording mirrors the desktop app: audio is captured at the input's native
|
||
rate (WAV on disk keeps it), while the 60 s rolling window is resampled to
|
||
Whisper's 16 kHz for the upcoming live pass. Recording continues while the
|
||
app is backgrounded via the microphone foreground service.
|
||
|
||
Models live in the app's private storage (`filesDir/models`), shared files
|
||
with the desktop app's `~/.cache/meetrec/whisper-cpp` naming
|
||
(ggml-tiny.bin … ggml-large-v3.bin). |