fegger
6b697f8d94
- server/meetrec-server: FastAPI storage API (upload bundle with wav/ txt/srt/json + metadata incl. agenda, list, fetch, download, delete); file-based index.json, no database; Docker Compose on port 8090, Tailscale-only bind like whisper-server; Ollama env prepared for M3c (gemma4:12b, German) - phone: StorageClient (stdlib multipart upload); RecorderService uploads the bundle in the background after the final pass and publishes UploadState (Uploading/Done/Error) to the UI - app: Library URL setting (persisted, default http://100.103.83.12:8090, empty disables upload); status line reports upload progress - storage API validated locally end-to-end: upload, list, metadata, download, path-traversal rejected, delete
72 lines
3.2 KiB
Markdown
72 lines
3.2 KiB
Markdown
# whisper.cpp server (Docker Compose)
|
||
|
||
The transcription backend for meetrec clients: the same pinned whisper.cpp
|
||
release (v1.9.3) as the Android app, exposed as an HTTP inference API with
|
||
**Vulkan GPU support** (AMD Radeon AI PRO R9700) and reachable over
|
||
Tailscale at `100.103.83.12:8085`.
|
||
|
||
The Fairphone 6 transcribes at roughly 0.6–0.8× realtime on-device; the
|
||
R9700 (Strix Halo, RDNA 3.5, ~256 GB/s shared memory) is bandwidth-bound
|
||
friendly for Whisper — expect large-v3 at many times realtime.
|
||
|
||
## Start
|
||
|
||
```sh
|
||
RENDER_GID=$(getent group render | cut -d: -f3) docker compose up -d --build
|
||
docker compose logs -f # watch the model download, then "running"
|
||
```
|
||
|
||
The `RENDER_GID` lookup passes the host's render group into the container
|
||
so the GPU device is accessible. Verify the GPU is actually used from the
|
||
startup log — it should print `ggml_vulkan: Found 1 Vulkan devices` (and
|
||
`VULKAN = 1` in the system info); if the GPU is unavailable the server
|
||
transparently falls back to CPU.
|
||
|
||
Configuration lives in `docker-compose.yml`:
|
||
|
||
| Env | Default | Meaning |
|
||
| ---------- | --------- | ------------------------------------------ |
|
||
| `MODEL` | `large-v3`| `tiny`/`base`/`small`/`medium`/`large-v3` (downloaded to `./models` on first start) |
|
||
| `THREADS` | `8` | CPU threads per inference |
|
||
| `PORT` | `8085` | Port inside the container |
|
||
|
||
## Try it
|
||
|
||
```sh
|
||
curl http://100.103.83.12:8085/inference \
|
||
-F file=@meeting.wav \
|
||
-F response_format=verbose_json \
|
||
-F language=auto
|
||
```
|
||
|
||
`POST /inference` accepts multipart fields `file` (PCM WAV, any rate),
|
||
`language` (`auto` supported), `response_format`
|
||
(`text`/`json`/`srt`/`vtt`/`verbose_json`), `temperature`. With
|
||
`verbose_json` the response carries the detected language and segments
|
||
with `start`/`end`/`text` (seconds).
|
||
|
||
Notes:
|
||
|
||
- **Beam size is a server-start setting** (v1.9.3 has no per-request
|
||
override), so the live/final beam split of the clients doesn't apply
|
||
here — one beam for all requests.
|
||
- The server also has a `/load` endpoint to swap models at runtime.
|
||
- **No authentication**: the compose file binds `100.103.83.12:8085`
|
||
(Tailscale interface only) for that reason. Do not switch this to
|
||
`0.0.0.0` unless the host is otherwise firewalled.
|
||
- The image builds with CPU feature auto-detection (`-march=native`):
|
||
build it on the machine that runs it (`--build` from the server, not
|
||
by exporting an image from another host).
|
||
- **GPU backend**: Vulkan via the RADV driver (mesa-vulkan-drivers in the
|
||
image). For maximum performance a ROCm/HIP build is the alternative
|
||
(heavier image, needs a ROCm base image and `gfx1151` target support
|
||
for Strix Halo) — add later if Vulkan benchmarks are insufficient.
|
||
- `NO_GPU=1` in the environment forces CPU-only inference.
|
||
|
||
## Client status
|
||
|
||
- Desktop `meetrec`: `--engine whisper-server --server-url http://100.103.83.12:8085`.
|
||
- Android app: **Transcribe on: server** (phone records, server transcribes;
|
||
local JNI stays as the offline fallback).
|
||
- Recording library: [meetrec-server](../meetrec-server/) stores recordings
|
||
+ transcripts; Android auto-uploads after the final pass. |