# whisper.cpp server (Docker Compose)

The transcription backend for meetrec clients: the same pinned whisper.cpp
release (v1.9.3) as the Android app, exposed as an HTTP inference API with
**Vulkan GPU support** (AMD Radeon AI PRO R9700) and reachable over
Tailscale at `100.103.83.12:8085`.

The Fairphone 6 transcribes at roughly 0.6–0.8× realtime on-device; the
R9700 (Strix Halo, RDNA 3.5, ~256 GB/s shared memory) is bandwidth-bound
friendly for Whisper — expect large-v3 at many times realtime.

## Start

```sh
RENDER_GID=$(getent group render | cut -d: -f3) docker compose up -d --build
docker compose logs -f          # watch the model download, then "running"
```

The `RENDER_GID` lookup passes the host's render group into the container
so the GPU device is accessible. Verify the GPU is actually used from the
startup log — it should print `ggml_vulkan: Found 1 Vulkan devices` (and
`VULKAN = 1` in the system info); if the GPU is unavailable the server
transparently falls back to CPU.

Configuration lives in `docker-compose.yml`: 

| Env        | Default   | Meaning                                    |
| ---------- | --------- | ------------------------------------------ |
| `MODEL`    | `large-v3`| `tiny`/`base`/`small`/`medium`/`large-v3` (downloaded to `./models` on first start) |
| `THREADS`  | `8`       | CPU threads per inference                  |
| `PORT`     | `8085`    | Port inside the container                  |

## Try it

```sh
curl http://100.103.83.12:8085/inference \
  -F file=@meeting.wav \
  -F response_format=verbose_json \
  -F language=auto
```

`POST /inference` accepts multipart fields `file` (PCM WAV, any rate),
`language` (`auto` supported), `response_format`
(`text`/`json`/`srt`/`vtt`/`verbose_json`), `temperature`. With
`verbose_json` the response carries the detected language and segments
with `start`/`end`/`text` (seconds).

Notes:

- **Beam size is a server-start setting** (v1.9.3 has no per-request
  override), so the live/final beam split of the clients doesn't apply
  here — one beam for all requests.
- The server also has a `/load` endpoint to swap models at runtime.
- **No authentication**: the compose file binds `100.103.83.12:8085`
  (Tailscale interface only) for that reason. Do not switch this to
  `0.0.0.0` unless the host is otherwise firewalled.
- The image builds with CPU feature auto-detection (`-march=native`):
  build it on the machine that runs it (`--build` from the server, not
  by exporting an image from another host).
- **GPU backend**: Vulkan via the RADV driver (mesa-vulkan-drivers in the
  image). For maximum performance a ROCm/HIP build is the alternative
  (heavier image, needs a ROCm base image and `gfx1151` target support
  for Strix Halo) — add later if Vulkan benchmarks are insufficient.
- `NO_GPU=1` in the environment forces CPU-only inference.

## Client status

- Desktop `meetrec`: `--engine whisper-server --server-url http://100.103.83.12:8085`.
- Android app: **Transcribe on: server** (phone records, server transcribes;
  local JNI stays as the offline fallback).
- Recording library: [meetrec-server](../meetrec-server/) stores recordings
  + transcripts; Android auto-uploads after the final pass.