fegger/meetrec
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add whisper-server engine: transcribe on a remote whisper.cpp server

Browse Source 
- WhisperServerEngine posts multipart/form-data to POST /inference
  (verbose_json) with the same transcribe() contract as the other
  engines, so live and final passes work unchanged; stdlib-only
  multipart builder; HTTP and connection errors surface clean messages
- CLI: --engine whisper-server + --server-url (or MEETREC_SERVER_URL)
- GUI: engine dropdown gains whisper-server; model/device/compute grey
  out (the server owns the model), new Server URL field
- model lives server-side, beam size is a server-start setting in
  whisper.cpp v1.9.3 (documented); verbose_json reports language names
  (german) rather than ISO codes
- validated against the live R9700 server over Tailscale: array input,
  native-rate file input (server-side resample), full final_transcribe
  pipeline, and unreachable-server handling
This commit is contained in:
Florian Egger
2026-09-07 12:59:14 +02:00
parent e97e8b1d33
commit a1df9d4037
3 changed files with 160 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+13 -2
View File
@@ -86,6 +86,8 @@ meetrec-cli --list-sources # find your input de
meetrec-cli -s "Conference Mic" --model small --live 8 -o meeting
meetrec-cli -s monitor --model base --language en # record system audio
meetrec-cli --engine whisper-cpp -o meeting # transcribe on the GPU (Vulkan)
MEETREC_SERVER_URL=http://100.103.83.12:8085 \
meetrec-cli --engine whisper-server -o meeting # transcribe on a remote server
python meetrec.py --help # all options
```
@@ -93,7 +95,8 @@ Key options:
| Option | Meaning |
| --------------- | ---------------------------------------------------------------- |
| `--engine` | transcription backend: `faster-whisper` (default) or `whisper-cpp` |
| `--engine` | transcription backend: `faster-whisper` (default), `whisper-cpp`, or `whisper-server` |
| `--server-url` | whisper-server base URL, e.g. `http://100.103.83.12:8085` (or `MEETREC_SERVER_URL`) |
| `-s, --source` | `default` or a substring of a device name (`--list-sources`) |
| `-m, --model` | `tiny` / `base` / `small` / `medium` / `large-v3` (default `small`) |
| `-o, --output` | output file stem (default `meeting`) |
@@ -118,8 +121,9 @@ Meetrec supports two interchangeable transcription backends — `--engine` on th
| ------ | ------- | ----------- |
| `faster-whisper` (default) | CTranslate2 | NVIDIA CUDA (`--device cuda`); otherwise CPU |
| `whisper-cpp` | [whisper.cpp](https://github.com/ggml-org/whisper.cpp) | **AMD / Intel / NVIDIA via Vulkan**; falls back to CPU |
| `whisper-server` | remote whisper.cpp HTTP server | whatever the server has — see [server/whisper-server/](server/whisper-server/) (Docker Compose, Vulkan GPU, Tailscale) |
On AMD hardware (e.g. the Radeon iGPU in Ryzen AI 300 laptops) the `faster-whisper` path is CPU-only — use the `whisper-cpp` engine there, which runs on the GPU when whisper.cpp is built with `-DWHISPER_VULKAN=ON` (`./install.sh --whisper-cpp` does this for you).
On AMD hardware (e.g. the Radeon iGPU in Ryzen AI 300 laptops) the `faster-whisper` path is CPU-only — use the `whisper-cpp` engine there, which runs on the GPU when whisper.cpp is built with `-DWHISPER_VULKAN=ON` (`./install.sh --whisper-cpp` does this for you). The `whisper-server` engine moves the work to a server entirely (ideal for slow clients such as phones) and can run `large-v3` on a GPU.
Notes on `whisper-cpp`:
@@ -127,6 +131,13 @@ Notes on `whisper-cpp`:
- GGML models (`ggml-*.bin`) download automatically to `~/.cache/meetrec/whisper-cpp` on first use.
- VAD works like on faster-whisper: whisper.cpp's Silero VAD model (~1 MB) downloads automatically and is used for both live and final passes.
- Live mode spawns `whisper-cli` for every rolling-window pass, so the GGML model is reloaded on each live tick — slightly heavier than faster-whisper, which loads once per recording.
Notes on `whisper-server`:
- The model lives on the server — `--model`, `--device`, `--compute-type` do not apply; beam size is a server-start setting in whisper.cpp v1.9.3 (no per-request override).
- Run your own with the Docker Compose stack in `server/whisper-server/` — Vulkan GPU (AMD/NVIDIA/Intel), model auto-download, Tailscale-only binding.
- The server has no authentication: keep it on Tailscale or behind a VPN/firewall.
- `verbose_json` reports language names ("german") rather than ISO codes.
- `--device` / `--compute-type` do not apply (GPU vs. CPU is decided by the whisper.cpp build).
## Project layout
meetrec.py
+103 -7
View File
@@ -41,6 +41,7 @@ import sys
import tempfile
import threading
import time
import urllib.error
import urllib.request
import wave
from pathlib import Path
@@ -324,6 +325,8 @@ def final_transcribe(model, wav_path: str, out_stem: str, language,
# faster-whisper CTranslate2 — NVIDIA CUDA or CPU (default)
# whisper-cpp whisper.cpp CLI — GPU via Vulkan (AMD, Intel, NVIDIA)
# or CPU; preferred on AMD hardware (e.g. Ryzen AI laptops)
# whisper-server remote whisper.cpp HTTP server (server/whisper-server) —
# the model lives on the server; ideal for phones/laptops
#
# Both expose: transcribe(audio, beam_size, vad_filter, language,
# vad_parameters) -> (segments, info)
@@ -493,11 +496,95 @@ class WhisperCppEngine:
return segs, info
MULTIPART_BOUNDARY = "meetrec-9f3e1c7a5b2d"
def _multipart_body(fields: dict, file_name: str, file_obj) -> bytes:
"""Build a multipart/form-data body with stdlib only."""
parts = []
for name, value in fields.items():
parts.append(
f"--{MULTIPART_BOUNDARY}\r\n"
f'Content-Disposition: form-data; name="{name}"\r\n\r\n'
f"{value}\r\n".encode())
parts.append(
f"--{MULTIPART_BOUNDARY}\r\n"
f'Content-Disposition: form-data; name="file"; '
f'filename="{file_name}"\r\n'
f"Content-Type: audio/wav\r\n\r\n".encode())
parts.append(file_obj.read())
parts.append(b"\r\n")
parts.append(f"--{MULTIPART_BOUNDARY}--\r\n".encode())
return b"".join(parts)
class WhisperServerEngine:
"""Transcription via a remote whisper.cpp server (whisper-server).
Use the Docker Compose stack in server/whisper-server/ to run one
(Vulkan GPU on AMD, or CPU). The model lives on the server, so
--model / --device / --compute-type do not apply, and beam size is a
server-start setting in whisper.cpp v1.9.3 (no per-request override) —
beam_size is accepted for interface compatibility but ignored.
vad_filter/vad_parameters are likewise accepted but unused; the server
runs its own pipeline. Note: verbose_json reports language names like
"german" rather than ISO codes.
"""
def __init__(self, server_url: str):
self.server_url = server_url.rstrip("/")
self.model_name = None # the server owns the model
def transcribe(self, audio, beam_size=5, vad_filter=False,
language=None, vad_parameters=None, timeout=600):
with tempfile.TemporaryDirectory(prefix="meetrec-") as td:
if isinstance(audio, (str, os.PathLike)):
wav = str(audio) # server resamples to 16 kHz itself
else:
wav = os.path.join(td, "in.wav")
write_wav_float(wav, audio)
with open(wav, "rb") as f:
body = _multipart_body(
{"response_format": "verbose_json",
"language": language or "auto"},
"audio.wav", f)
req = urllib.request.Request(
self.server_url + "/inference", data=body,
headers={"Content-Type":
"multipart/form-data; boundary=" + MULTIPART_BOUNDARY})
try:
with urllib.request.urlopen(req, timeout=timeout) as r:
data = json.load(r)
except urllib.error.HTTPError as e:
raise RuntimeError(f"server returned {e.code}: "
f"{e.read()[:300]!r}") from None
except urllib.error.URLError as e:
raise RuntimeError(f"cannot reach whisper server at "
f"{self.server_url}: {e.reason}") from None
segs = [SimpleNamespace(start=s["start"], end=s["end"],
text=s["text"].strip())
for s in data.get("segments", [])
if s.get("text", "").strip()]
info = SimpleNamespace(
language=data.get("language") or language or "unknown",
language_probability=1.0, # not reported by verbose_json
duration=data.get("duration", 0.0),
)
return segs, info
def load_engine(model_name, engine="faster-whisper",
device="auto", compute_type="int8"):
device="auto", compute_type="int8", server_url=None):
"""Create a transcription engine. See the Engines section above."""
if engine == "whisper-cpp":
return WhisperCppEngine(model_name)
if engine == "whisper-server":
url = server_url or os.environ.get("MEETREC_SERVER_URL")
if not url:
raise RuntimeError("no server URL: pass --server-url or set "
"MEETREC_SERVER_URL (e.g. http://100.103.83.12:8085)")
return WhisperServerEngine(url)
if engine == "faster-whisper":
from faster_whisper import WhisperModel
return WhisperModel(model_name, device=device,
@@ -528,10 +615,13 @@ def main():
ap.add_argument("--language", default=None,
help="force language code, e.g. en, de (default: autodetect)")
ap.add_argument("--engine", default="faster-whisper",
choices=["faster-whisper", "whisper-cpp"],
choices=["faster-whisper", "whisper-cpp", "whisper-server"],
help="transcription backend (default: faster-whisper). "
"whisper-cpp can use a GPU via Vulkan — preferred "
"on AMD hardware")
"whisper-cpp can use a GPU via Vulkan; whisper-server "
"transcribes on a remote whisper.cpp server")
ap.add_argument("--server-url", default=None,
help="whisper-server base URL, e.g. http://100.103.83.12:8085 "
"(or set MEETREC_SERVER_URL); whisper-server engine only")
ap.add_argument("--device", default="auto",
help="compute device: auto / cpu / cuda (default: auto); "
"faster-whisper only")
@@ -548,12 +638,18 @@ def main():
device = resolve_source(args.source)
print(f"loading Whisper model {args.model!r} via {args.engine!r} "
f"(first run downloads it to ~/.cache)...")
if args.engine == "whisper-server":
print(f"using whisper server at "
f"{args.server_url or os.environ.get('MEETREC_SERVER_URL')} "
f"(model lives on the server)")
else:
print(f"loading Whisper model {args.model!r} via {args.engine!r} "
f"(first run downloads it to ~/.cache)...")
try:
model = load_engine(args.model, engine=args.engine,
device=args.device,
compute_type=args.compute_type)
compute_type=args.compute_type,
server_url=args.server_url)
except Exception as e:
print(f"error: {e}", file=sys.stderr)
sys.exit(1)
meetrec_gui.py
+44 -17
View File
@@ -50,18 +50,20 @@ class ModelLoader(QThread):
failed = Signal(str)
def __init__(self, model_name, engine="faster-whisper", device="auto",
compute_type="int8", parent=None):
compute_type="int8", server_url=None, parent=None):
super().__init__(parent)
self.model_name = model_name
self.engine = engine
self.device = device
self.compute_type = compute_type
self.server_url = server_url
def run(self):
try:
self.model = meetrec.load_engine(
self.model_name, engine=self.engine,
device=self.device, compute_type=self.compute_type)
device=self.device, compute_type=self.compute_type,
server_url=self.server_url)
self.loaded.emit(self.model)
except Exception as e:
self.failed.emit(str(e))
@@ -104,7 +106,7 @@ class MeetRecWindow(QWidget):
"pt", "ru", "zh", "ja", "ko"]
DEVICES = ["auto", "cpu", "cuda"]
COMPUTE_TYPES = ["int8", "int8_float16", "float16", "float32"]
ENGINES = ["faster-whisper", "whisper-cpp"]
ENGINES = ["faster-whisper", "whisper-cpp", "whisper-server"]
def __init__(self):
super().__init__()
@@ -151,7 +153,8 @@ class MeetRecWindow(QWidget):
# row 2: model + language
r2 = QHBoxLayout()
r2.addWidget(QLabel("Model:"))
self.model_lbl = QLabel("Model:")
r2.addWidget(self.model_lbl)
self.model_cb = QComboBox()
self.model_cb.addItems(self.MODELS)
self.model_cb.setCurrentText("small")
@@ -192,6 +195,17 @@ class MeetRecWindow(QWidget):
r2b.addStretch(1)
root.addLayout(r2b)
# row 2c: whisper-server URL
r2c = QHBoxLayout()
self.server_lbl = QLabel("Server:")
r2c.addWidget(self.server_lbl)
self.server_url = QLineEdit("http://100.103.83.12:8085")
self.server_url.setToolTip(
"whisper.cpp server (see server/whisper-server/) — "
"the model lives on the server")
r2c.addWidget(self.server_url, 1)
root.addLayout(r2c)
# row 3: live options + output name
r3 = QHBoxLayout()
r3.addWidget(QLabel("Live:"))
@@ -287,18 +301,26 @@ class MeetRecWindow(QWidget):
self.src.setCurrentText(keep)
def _engine_changed(self, _idx):
# --device/--compute-type only apply to faster-whisper (CTranslate2);
# whisper.cpp picks GPU (Vulkan) or CPU on its own.
gpu_opts = self.engine_cb.currentText() != "whisper-cpp"
self.device_lbl.setText("Device:" if gpu_opts else "Device (n/a):")
self.compute_lbl.setText("Compute:" if gpu_opts else "Compute (n/a):")
self.device_cb.setEnabled(gpu_opts)
self.compute_cb.setEnabled(gpu_opts)
# What applies per engine:
# model: faster-whisper + whisper-cpp (server owns the
# model otherwise)
# device/compute: faster-whisper only (CTranslate2)
# server url: whisper-server only
engine = self.engine_cb.currentText()
has_model = engine != "whisper-server"
has_dc = engine == "faster-whisper"
self.model_lbl.setText("Model:" if has_model else "Model (n/a):")
self.device_lbl.setText("Device:" if has_dc else "Device (n/a):")
self.compute_lbl.setText("Compute:" if has_dc else "Compute (n/a):")
self.model_cb.setEnabled(has_model)
self.device_cb.setEnabled(has_dc)
self.compute_cb.setEnabled(has_dc)
self.server_url.setEnabled(engine == "whisper-server")
def _set_inputs_enabled(self, enabled: bool):
for w in (self.src, self.model_cb, self.engine_cb, self.device_cb,
self.compute_cb, self.lang, self.live_cb, self.live_spin,
self.out):
self.compute_cb, self.server_url, self.lang, self.live_cb,
self.live_spin, self.out):
w.setEnabled(enabled)
if enabled:
self._engine_changed(-1) # whisper-cpp: keep device/compute off
@@ -341,18 +363,23 @@ class MeetRecWindow(QWidget):
self.elapsed.setText("00:00:00")
key = (self.engine_cb.currentText(), self._args.model,
self.device_cb.currentText(), self.compute_cb.currentText())
self.device_cb.currentText(), self.compute_cb.currentText(),
self.server_url.text().strip())
if self.model is None or self.model_key != key:
self.state = "loading"
self.status.setText(
f"Loading Whisper model {self._args.model!r} via {key[0]} "
"(first run downloads it, this can take a minute)...")
if key[0] == "whisper-server":
self.status.setText(f"Using whisper server {key[4]} ...")
else:
self.status.setText(
f"Loading Whisper model {self._args.model!r} via {key[0]} "
"(first run downloads it, this can take a minute)...")
self._pending_key = key
self.loader = ModelLoader(
self._args.model,
engine=key[0],
device=self.device_cb.currentText(),
compute_type=self.compute_cb.currentText(),
server_url=key[4] or None,
parent=self)
self.loader.loaded.connect(self._model_ready)
self.loader.failed.connect(self._model_failed)