fegger
76611af7e0
- whisper-server stack: second container (port 8086) running the English-trained small.en-tdrz model with -tdrz; image patched (speaker-turn.patch) to expose speaker_turn_next per segment in verbose_json like the cli example does - core/whisper Diarization: merges the tdrz pass's TURN TIMES onto the quality transcript as alternating 'Sprecher 1/2:' labels, splitting segments when a turn falls inside them; no turns detected = no labels (never mislabels); 6 unit tests - RemoteWhisperEngine gains a diarize flag (sends tinydiarize=true, parses speaker_turn_next); WhisperEngine.Segment carries the flag - RecorderService: optional second pass on the diarize server after the final pass; failures keep the unlabeled transcript - Settings: Diarize server URL (persisted; empty disables) - validated infrastructure locally: patched image builds, tdrz model downloads from akashmjn/tinydiarize-whisper.cpp, speaker_turn_next present in responses; synthetic espeak audio does not trigger the model's turn tokens — real two-person speech needed for the end-to-end check
whisper.cpp server (Docker Compose)
The transcription backend for meetrec clients: the same pinned whisper.cpp
release (v1.9.3) as the Android app, exposed as an HTTP inference API with
Vulkan GPU support (AMD Radeon AI PRO R9700) and reachable over
Tailscale at 100.103.83.12:8085.
The Fairphone 6 transcribes at roughly 0.6–0.8× realtime on-device; the R9700 (Strix Halo, RDNA 3.5, ~256 GB/s shared memory) is bandwidth-bound friendly for Whisper — expect large-v3 at many times realtime.
Start
RENDER_GID=$(getent group render | cut -d: -f3) docker compose up -d --build
docker compose logs -f # watch the model download, then "running"
The RENDER_GID lookup passes the host's render group into the container
so the GPU device is accessible. Verify the GPU is actually used from the
startup log — it should print ggml_vulkan: Found 1 Vulkan devices (and
VULKAN = 1 in the system info); if the GPU is unavailable the server
transparently falls back to CPU.
Configuration lives in docker-compose.yml:
| Env | Default | Meaning |
|---|---|---|
MODEL |
large-v3 |
tiny/base/small/medium/large-v3 (downloaded to ./models on first start) |
THREADS |
8 |
CPU threads per inference |
PORT |
8085 |
Port inside the container |
Try it
curl http://100.103.83.12:8085/inference \
-F file=@meeting.wav \
-F response_format=verbose_json \
-F language=auto
POST /inference accepts multipart fields file (PCM WAV, any rate),
language (auto supported), response_format
(text/json/srt/vtt/verbose_json), temperature. With
verbose_json the response carries the detected language and segments
with start/end/text (seconds).
Notes:
- Beam size is a server-start setting (v1.9.3 has no per-request override), so the live/final beam split of the clients doesn't apply here — one beam for all requests.
- The server also has a
/loadendpoint to swap models at runtime. - No authentication: the compose file binds
100.103.83.12:8085(Tailscale interface only) for that reason. Do not switch this to0.0.0.0unless the host is otherwise firewalled. - The image builds with CPU feature auto-detection (
-march=native): build it on the machine that runs it (--buildfrom the server, not by exporting an image from another host). - GPU backend: Vulkan via the RADV driver (mesa-vulkan-drivers in the
image). For maximum performance a ROCm/HIP build is the alternative
(heavier image, needs a ROCm base image and
gfx1151target support for Strix Halo) — add later if Vulkan benchmarks are insufficient. NO_GPU=1in the environment forces CPU-only inference.
Client status
- Desktop
meetrec:--engine whisper-server --server-url http://100.103.83.12:8085. - Android app: Transcribe on: server (phone records, server transcribes; local JNI stays as the offline fallback).
- Recording library: meetrec-server stores recordings
- transcripts; Android auto-uploads after the final pass.