meetrec/server/whisper-server/README.md

whisper.cpp server (Docker Compose)

The transcription backend for meetrec clients: the same pinned whisper.cpp release (v1.9.3) as the Android app, exposed as an HTTP inference API with Vulkan GPU support (AMD Radeon AI PRO R9700) and reachable over Tailscale at 100.103.83.12:8080.

The Fairphone 6 transcribes at roughly 0.6–0.8× realtime on-device; the R9700 (Strix Halo, RDNA 3.5, ~256 GB/s shared memory) is bandwidth-bound friendly for Whisper — expect large-v3 at many times realtime.

Start

RENDER_GID=$(getent group render | cut -d: -f3) docker compose up -d --build
docker compose logs -f          # watch the model download, then "running"

The RENDER_GID lookup passes the host's render group into the container so the GPU device is accessible. Verify the GPU is actually used from the startup log — it should print ggml_vulkan: Found 1 Vulkan devices (and VULKAN = 1 in the system info); if the GPU is unavailable the server transparently falls back to CPU.

Configuration lives in docker-compose.yml:

Env	Default	Meaning
MODEL	large-v3	tiny/base/small/medium/large-v3 (downloaded to ./models on first start)
THREADS	8	CPU threads per inference
PORT	8080	Port inside the container

Try it

curl http://100.103.83.12:8080/inference \
  -F file=@meeting.wav \
  -F response_format=verbose_json \
  -F language=auto

POST /inference accepts multipart fields file (PCM WAV, any rate), language (auto supported), response_format (text/json/srt/vtt/verbose_json), temperature. With verbose_json the response carries the detected language and segments with start/end/text (seconds).

Notes:

• Beam size is a server-start setting (v1.9.3 has no per-request override), so the live/final beam split of the clients doesn't apply here — one beam for all requests.
• The server also has a /load endpoint to swap models at runtime.
• No authentication: the compose file binds 100.103.83.12:8080 (Tailscale interface only) for that reason. Do not switch this to 0.0.0.0 unless the host is otherwise firewalled.
• The image builds with CPU feature auto-detection (-march=native): build it on the machine that runs it (--build from the server, not by exporting an image from another host).
• GPU backend: Vulkan via the RADV driver (mesa-vulkan-drivers in the image). For maximum performance a ROCm/HIP build is the alternative (heavier image, needs a ROCm base image and gfx1151 target support for Strix Halo) — add later if Vulkan benchmarks are insufficient.
• NO_GPU=1 in the environment forces CPU-only inference.

Client status

• Desktop meetrec: a whisper-server engine is planned (--engine whisper-server --server-url http://100.103.83.12:8080).
• Android app: a remote engine option is planned (phone records, server transcribes; local JNI stays as the offline fallback).