fegger
ef59fc0fb4
- server/whisper-server: compose stack built from the pinned whisper.cpp v1.9.3 release, same as the Android JNI layer - Vulkan GPU backend (AMD Radeon AI PRO R9700 / RADV) with transparent CPU fallback and NO_GPU override; GGML models auto-download on first start (MODEL env, default large-v3) - API bound to the Tailscale interface only (100.103.83.12:8080) since whisper-server has no authentication; render-group GID passthrough for /dev/dri - validated locally: image builds, entrypoint downloads tiny, POST /inference returns verbose_json with language + segments; GPU-less fallback confirmed
71 lines
3.1 KiB
Markdown
71 lines
3.1 KiB
Markdown
# whisper.cpp server (Docker Compose)
|
||
|
||
The transcription backend for meetrec clients: the same pinned whisper.cpp
|
||
release (v1.9.3) as the Android app, exposed as an HTTP inference API with
|
||
**Vulkan GPU support** (AMD Radeon AI PRO R9700) and reachable over
|
||
Tailscale at `100.103.83.12:8080`.
|
||
|
||
The Fairphone 6 transcribes at roughly 0.6–0.8× realtime on-device; the
|
||
R9700 (Strix Halo, RDNA 3.5, ~256 GB/s shared memory) is bandwidth-bound
|
||
friendly for Whisper — expect large-v3 at many times realtime.
|
||
|
||
## Start
|
||
|
||
```sh
|
||
RENDER_GID=$(getent group render | cut -d: -f3) docker compose up -d --build
|
||
docker compose logs -f # watch the model download, then "running"
|
||
```
|
||
|
||
The `RENDER_GID` lookup passes the host's render group into the container
|
||
so the GPU device is accessible. Verify the GPU is actually used from the
|
||
startup log — it should print `ggml_vulkan: Found 1 Vulkan devices` (and
|
||
`VULKAN = 1` in the system info); if the GPU is unavailable the server
|
||
transparently falls back to CPU.
|
||
|
||
Configuration lives in `docker-compose.yml`:
|
||
|
||
| Env | Default | Meaning |
|
||
| ---------- | --------- | ------------------------------------------ |
|
||
| `MODEL` | `large-v3`| `tiny`/`base`/`small`/`medium`/`large-v3` (downloaded to `./models` on first start) |
|
||
| `THREADS` | `8` | CPU threads per inference |
|
||
| `PORT` | `8080` | Port inside the container |
|
||
|
||
## Try it
|
||
|
||
```sh
|
||
curl http://100.103.83.12:8080/inference \
|
||
-F file=@meeting.wav \
|
||
-F response_format=verbose_json \
|
||
-F language=auto
|
||
```
|
||
|
||
`POST /inference` accepts multipart fields `file` (PCM WAV, any rate),
|
||
`language` (`auto` supported), `response_format`
|
||
(`text`/`json`/`srt`/`vtt`/`verbose_json`), `temperature`. With
|
||
`verbose_json` the response carries the detected language and segments
|
||
with `start`/`end`/`text` (seconds).
|
||
|
||
Notes:
|
||
|
||
- **Beam size is a server-start setting** (v1.9.3 has no per-request
|
||
override), so the live/final beam split of the clients doesn't apply
|
||
here — one beam for all requests.
|
||
- The server also has a `/load` endpoint to swap models at runtime.
|
||
- **No authentication**: the compose file binds `100.103.83.12:8080`
|
||
(Tailscale interface only) for that reason. Do not switch this to
|
||
`0.0.0.0` unless the host is otherwise firewalled.
|
||
- The image builds with CPU feature auto-detection (`-march=native`):
|
||
build it on the machine that runs it (`--build` from the server, not
|
||
by exporting an image from another host).
|
||
- **GPU backend**: Vulkan via the RADV driver (mesa-vulkan-drivers in the
|
||
image). For maximum performance a ROCm/HIP build is the alternative
|
||
(heavier image, needs a ROCm base image and `gfx1151` target support
|
||
for Strix Halo) — add later if Vulkan benchmarks are insufficient.
|
||
- `NO_GPU=1` in the environment forces CPU-only inference.
|
||
|
||
## Client status
|
||
|
||
- Desktop `meetrec`: a `whisper-server` engine is planned
|
||
(`--engine whisper-server --server-url http://100.103.83.12:8080`).
|
||
- Android app: a remote engine option is planned (phone records, server
|
||
transcribes; local JNI stays as the offline fallback). |