Add whisper.cpp engine with Vulkan GPU support (AMD/Intel/NVIDIA)
- meetrec.py: engine layer with a common transcribe() contract; WhisperCppEngine shells out to whisper-cli, GGML + Silero VAD models auto-download to ~/.cache/meetrec; faster-whisper import now lazy; new --engine CLI option - meetrec_gui.py: Engine dropdown; Device/Compute greyed out for whisper-cpp; model reload keyed on (engine, model, device, compute) - install.sh: --whisper-cpp builds whisper.cpp with -DWHISPER_VULKAN=ON, installs whisper-cli into the prefix and wires WHISPER_CPP_BIN via a generated whisper-cpp.env; build failures degrade to a warning - launchers source whisper-cpp.env; README documents engines - Recorder.stop() is now idempotent (GUI close path called it twice)
This commit is contained in:
@@ -48,11 +48,14 @@ Options:
|
||||
|
||||
```sh
|
||||
./install.sh --model base # pre-download a different model (tiny/base/small/medium/large-v3)
|
||||
./install.sh --whisper-cpp # also build whisper.cpp with its Vulkan GPU backend
|
||||
./install.sh --no-model # skip the model pre-download
|
||||
./install.sh --uninstall # remove everything the installer created
|
||||
PREFIX=/opt/meetrec ./install.sh # install into a different prefix
|
||||
```
|
||||
|
||||
On an **AMD GPU** (e.g. Ryzen AI 300 laptops with a Radeon iGPU), use `--whisper-cpp` and select the `whisper-cpp` engine — see [Engines](#engines) below. The build needs `git`, `cmake`, `g++`, and Vulkan headers (`sudo pacman -S vulkan-headers` on Arch).
|
||||
|
||||
### Manual install (venv)
|
||||
|
||||
```sh
|
||||
@@ -79,6 +82,7 @@ Pick a microphone (or type a name substring), model, language, and compute devic
|
||||
meetrec-cli --list-sources # find your input devices
|
||||
meetrec-cli -s "Conference Mic" --model small --live 8 -o meeting
|
||||
meetrec-cli -s monitor --model base --language en # record system audio
|
||||
meetrec-cli --engine whisper-cpp -o meeting # transcribe on the GPU (Vulkan)
|
||||
python meetrec.py --help # all options
|
||||
```
|
||||
|
||||
@@ -86,13 +90,14 @@ Key options:
|
||||
|
||||
| Option | Meaning |
|
||||
| --------------- | ---------------------------------------------------------------- |
|
||||
| `--engine` | transcription backend: `faster-whisper` (default) or `whisper-cpp` |
|
||||
| `-s, --source` | `default` or a substring of a device name (`--list-sources`) |
|
||||
| `-m, --model` | `tiny` / `base` / `small` / `medium` / `large-v3` (default `small`) |
|
||||
| `-o, --output` | output file stem (default `meeting`) |
|
||||
| `--live SEC` | live-transcription interval in seconds; `0` disables (default 8) |
|
||||
| `--language` | force a language code, e.g. `en`, `de` (default: autodetect) |
|
||||
| `--device` | `auto` / `cpu` / `cuda` (default `auto`) |
|
||||
| `--compute-type`| `int8` / `int8_float16` / `float16` / `float32` (default `int8`) |
|
||||
| `--device` | `auto` / `cpu` / `cuda` (default `auto`); faster-whisper only |
|
||||
| `--compute-type`| `int8` / `int8_float16` / `float16` / `float32` (default `int8`); faster-whisper only |
|
||||
|
||||
`Ctrl+C` stops recording and runs the final transcription.
|
||||
|
||||
@@ -100,7 +105,26 @@ Key options:
|
||||
|
||||
- **Record system audio**: sink monitors appear as `<sink name>.monitor`; use `-s monitor` (or a matching substring).
|
||||
- **Model size vs. speed/accuracy**: `tiny`/`base` are fast and rough, `small` is a good default, `medium`/`large-v3` are most accurate but much slower on CPU.
|
||||
- Models are cached in `~/.cache/huggingface` and shared between the CLI and the GUI.
|
||||
- Models are cached in `~/.cache/huggingface` (faster-whisper) and `~/.cache/meetrec/whisper-cpp` (whisper.cpp) and shared between the CLI and the GUI.
|
||||
|
||||
## Engines
|
||||
|
||||
Meetrec supports two interchangeable transcription backends — `--engine` on the CLI, the **Engine** dropdown in the GUI:
|
||||
|
||||
| Engine | Backend | GPU support |
|
||||
| ------ | ------- | ----------- |
|
||||
| `faster-whisper` (default) | CTranslate2 | NVIDIA CUDA (`--device cuda`); otherwise CPU |
|
||||
| `whisper-cpp` | [whisper.cpp](https://github.com/ggml-org/whisper.cpp) | **AMD / Intel / NVIDIA via Vulkan**; falls back to CPU |
|
||||
|
||||
On AMD hardware (e.g. the Radeon iGPU in Ryzen AI 300 laptops) the `faster-whisper` path is CPU-only — use the `whisper-cpp` engine there, which runs on the GPU when whisper.cpp is built with `-DWHISPER_VULKAN=ON` (`./install.sh --whisper-cpp` does this for you).
|
||||
|
||||
Notes on `whisper-cpp`:
|
||||
|
||||
- The `whisper-cli` binary is looked up in `WHISPER_CPP_BIN`, then `PATH`, then `~/.local/share/meetrec/whisper-cpp/bin/whisper-cli` (where `install.sh --whisper-cpp` puts it).
|
||||
- GGML models (`ggml-*.bin`) download automatically to `~/.cache/meetrec/whisper-cpp` on first use.
|
||||
- VAD works like on faster-whisper: whisper.cpp's Silero VAD model (~1 MB) downloads automatically and is used for both live and final passes.
|
||||
- Live mode spawns `whisper-cli` for every rolling-window pass, so the GGML model is reloaded on each live tick — slightly heavier than faster-whisper, which loads once per recording.
|
||||
- `--device` / `--compute-type` do not apply (GPU vs. CPU is decided by the whisper.cpp build).
|
||||
|
||||
## Project layout
|
||||
|
||||
|
||||
@@ -7,4 +7,7 @@ if [ ! -x "$PY" ]; then
|
||||
echo "MeetRec is not installed. Run ./install.sh first." >&2
|
||||
exit 1
|
||||
fi
|
||||
# Engine hint generated by install.sh --whisper-cpp (points WHISPER_CPP_BIN at
|
||||
# the locally built whisper-cli).
|
||||
if [ -f "$APP_DIR/whisper-cpp.env" ]; then . "$APP_DIR/whisper-cpp.env"; fi
|
||||
exec "$PY" "$APP_DIR/meetrec_gui.py" "$@"
|
||||
|
||||
@@ -7,4 +7,7 @@ if [ ! -x "$PY" ]; then
|
||||
echo "MeetRec is not installed. Run ./install.sh first." >&2
|
||||
exit 1
|
||||
fi
|
||||
# Engine hint generated by install.sh --whisper-cpp (points WHISPER_CPP_BIN at
|
||||
# the locally built whisper-cli).
|
||||
if [ -f "$APP_DIR/whisper-cpp.env" ]; then . "$APP_DIR/whisper-cpp.env"; fi
|
||||
exec "$PY" "$APP_DIR/meetrec.py" "$@"
|
||||
|
||||
+61
-2
@@ -4,6 +4,7 @@
|
||||
#
|
||||
# Usage:
|
||||
# ./install.sh install, pre-downloading the 'small' model
|
||||
# ./install.sh --whisper-cpp also build whisper.cpp (Vulkan GPU backend)
|
||||
# ./install.sh --model base pre-download a different Whisper model
|
||||
# ./install.sh --no-model skip the model pre-download
|
||||
# ./install.sh --uninstall remove everything the installer created
|
||||
@@ -26,6 +27,7 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODEL="small"
|
||||
DOWNLOAD_MODEL=1
|
||||
UNINSTALL=0
|
||||
WHISPER_CPP=0
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
@@ -35,9 +37,10 @@ while [ $# -gt 0 ]; do
|
||||
shift 2
|
||||
;;
|
||||
--no-model) DOWNLOAD_MODEL=0; shift ;;
|
||||
--whisper-cpp) WHISPER_CPP=1; shift ;;
|
||||
--uninstall) UNINSTALL=1; shift ;;
|
||||
-h|--help)
|
||||
sed -n '2,13p' "$0" | sed 's/^# \{0,1\}//'
|
||||
sed -n '2,14p' "$0" | sed 's/^# \{0,1\}//'
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
@@ -58,7 +61,7 @@ if [ "$UNINSTALL" = 1 ]; then
|
||||
if command -v update-desktop-database >/dev/null 2>&1; then
|
||||
update-desktop-database "$DESKTOP_DIR" 2>/dev/null || true
|
||||
fi
|
||||
echo "Done. (Whisper models remain in ~/.cache/huggingface — remove that if you want them gone.)"
|
||||
echo "Done. (Whisper models remain in ~/.cache/huggingface and ~/.cache/meetrec — remove those if you want them gone.)"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
@@ -101,10 +104,52 @@ fi
|
||||
echo "==> Installing Python dependencies (numpy, sounddevice, faster-whisper, pyside6)"
|
||||
"$VENV/bin/pip" install -r "$APP_DIR/requirements.txt"
|
||||
|
||||
# ----------------------------------------------------------- whisper.cpp build
|
||||
|
||||
if [ "$WHISPER_CPP" = 1 ]; then
|
||||
echo "==> Building whisper.cpp (Vulkan GPU backend)"
|
||||
build_ok=1
|
||||
for t in git cmake g++ make; do
|
||||
command -v "$t" >/dev/null 2>&1 || {
|
||||
echo "warning: '$t' not found — skipping the whisper.cpp build." >&2
|
||||
build_ok=0
|
||||
}
|
||||
done
|
||||
if [ "$build_ok" = 1 ]; then
|
||||
if command -v pkg-config >/dev/null 2>&1 && ! pkg-config --exists vulkan; then
|
||||
echo "warning: Vulkan dev headers not found — the build may fail."
|
||||
echo " On Arch: sudo pacman -S vulkan-headers"
|
||||
fi
|
||||
SRC="$APP_DIR/src/whisper.cpp"
|
||||
if [ -d "$SRC/.git" ]; then
|
||||
git -C "$SRC" pull --ff-only || true
|
||||
else
|
||||
git clone --depth 1 https://github.com/ggml-org/whisper.cpp "$SRC"
|
||||
fi
|
||||
# The GPU engine is optional: a failed build must not leave the core
|
||||
# install half-done, so degrade to a warning and continue.
|
||||
if ! (cmake -S "$SRC" -B "$SRC/build" -DCMAKE_BUILD_TYPE=Release \
|
||||
-DWHISPER_VULKAN=ON \
|
||||
&& cmake --build "$SRC/build" -j"$(nproc)" \
|
||||
&& install -Dm755 "$SRC/build/bin/whisper-cli" \
|
||||
"$PREFIX/share/meetrec/whisper-cpp/bin/whisper-cli"); then
|
||||
echo "warning: whisper.cpp build failed — continuing without the GPU engine." >&2
|
||||
echo " faster-whisper (CPU) still works; retry the build later with ./install.sh --whisper-cpp" >&2
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "==> Installing launchers into $BIN_DIR"
|
||||
install -m 0755 "$SCRIPT_DIR/bin/meetrec" "$BIN_DIR/meetrec"
|
||||
install -m 0755 "$SCRIPT_DIR/bin/meetrec-cli" "$BIN_DIR/meetrec-cli"
|
||||
|
||||
if [ -x "$PREFIX/share/meetrec/whisper-cpp/bin/whisper-cli" ]; then
|
||||
printf 'WHISPER_CPP_BIN="%s"\n' \
|
||||
"$PREFIX/share/meetrec/whisper-cpp/bin/whisper-cli" \
|
||||
> "$APP_DIR/whisper-cpp.env"
|
||||
echo " (whisper.cpp engine installed — select 'whisper-cpp' to use the GPU)"
|
||||
fi
|
||||
|
||||
echo "==> Installing XDG desktop entry"
|
||||
install -m 0644 "$SCRIPT_DIR/share/applications/meetrec.desktop" \
|
||||
"$DESKTOP_DIR/meetrec.desktop"
|
||||
@@ -120,6 +165,20 @@ WhisperModel("$MODEL", device="cpu", compute_type="int8")
|
||||
EOF
|
||||
fi
|
||||
|
||||
if [ "$WHISPER_CPP" = 1 ] && [ "$DOWNLOAD_MODEL" = 1 ]; then
|
||||
echo "==> Pre-downloading whisper.cpp model ggml-$MODEL.bin"
|
||||
CACHE="$HOME/.cache/meetrec/whisper-cpp"
|
||||
mkdir -p "$CACHE"
|
||||
if command -v curl >/dev/null 2>&1; then
|
||||
curl -fL --retry 3 -C - -o "$CACHE/ggml-$MODEL.bin.part" \
|
||||
"https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-$MODEL.bin" \
|
||||
&& mv "$CACHE/ggml-$MODEL.bin.part" "$CACHE/ggml-$MODEL.bin" \
|
||||
|| echo "warning: ggml model download failed — it will retry on first use."
|
||||
else
|
||||
echo "warning: curl not found — skipping ggml model pre-download"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ----------------------------------------------------------------- summary
|
||||
|
||||
echo
|
||||
|
||||
+216
-7
@@ -23,23 +23,28 @@ Usage:
|
||||
python meetrec.py --list-sources
|
||||
python meetrec.py -s "Conference Mic" --model small --live 8 -o meeting
|
||||
python meetrec.py -s monitor --model base --language en # record system audio
|
||||
python meetrec.py --engine whisper-cpp -o meeting # GPU via Vulkan (AMD/NVIDIA)
|
||||
|
||||
Ctrl+C to stop -> final transcripts are written.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import queue
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import urllib.request
|
||||
import wave
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import numpy as np
|
||||
import sounddevice as sd
|
||||
from faster_whisper import WhisperModel
|
||||
|
||||
SR = 16000 # sample rate — what Whisper wants
|
||||
LIVE_WINDOW = 60 # seconds of rolling audio kept for live transcription
|
||||
@@ -175,6 +180,8 @@ class Recorder:
|
||||
self.stream.start()
|
||||
|
||||
def stop(self):
|
||||
if self.stop_event.is_set():
|
||||
return # idempotent: the GUI close path may call this twice
|
||||
self.stop_event.set()
|
||||
self.q.put(None)
|
||||
if self.stream:
|
||||
@@ -287,6 +294,196 @@ def final_transcribe(model, wav_path: str, out_stem: str, language,
|
||||
on_done()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Engines
|
||||
# --------------------------------------------------------------------------
|
||||
#
|
||||
# Two interchangeable transcription backends:
|
||||
# faster-whisper CTranslate2 — NVIDIA CUDA or CPU (default)
|
||||
# whisper-cpp whisper.cpp CLI — GPU via Vulkan (AMD, Intel, NVIDIA)
|
||||
# or CPU; preferred on AMD hardware (e.g. Ryzen AI laptops)
|
||||
#
|
||||
# Both expose: transcribe(audio, beam_size, vad_filter, language,
|
||||
# vad_parameters) -> (segments, info)
|
||||
# where `audio` is a 1-D float32 @16 kHz array or a WAV path, segments have
|
||||
# .start/.end/.text and info has .language/.language_probability/.duration.
|
||||
|
||||
WHISPER_CPP_MODEL_FILES = {
|
||||
"tiny": "ggml-tiny.bin",
|
||||
"base": "ggml-base.bin",
|
||||
"small": "ggml-small.bin",
|
||||
"medium": "ggml-medium.bin",
|
||||
"large-v3": "ggml-large-v3.bin",
|
||||
}
|
||||
WHISPER_CPP_MODEL_URL = ("https://huggingface.co/ggerganov/whisper.cpp/"
|
||||
"resolve/main/{name}")
|
||||
WHISPER_CPP_VAD_FILE = "ggml-silero-v6.2.0.bin"
|
||||
WHISPER_CPP_VAD_URL = ("https://huggingface.co/ggml-org/whisper-vad/"
|
||||
"resolve/main/" + WHISPER_CPP_VAD_FILE)
|
||||
|
||||
|
||||
def _download_file(url: str, dest: Path) -> None:
|
||||
tmp = dest.with_suffix(dest.suffix + ".part")
|
||||
req = urllib.request.Request(url, headers={"User-Agent": "meetrec"})
|
||||
with urllib.request.urlopen(req, timeout=60) as r:
|
||||
total = int(r.headers.get("Content-Length") or 0)
|
||||
done = 0
|
||||
with open(tmp, "wb") as f:
|
||||
while True:
|
||||
chunk = r.read(1024 * 1024)
|
||||
if not chunk:
|
||||
break
|
||||
f.write(chunk)
|
||||
done += len(chunk)
|
||||
if total:
|
||||
print(f"\r {done/1e6:.0f}/{total/1e6:.0f} MB", end="",
|
||||
flush=True)
|
||||
if total:
|
||||
print(flush=True)
|
||||
os.replace(tmp, dest)
|
||||
|
||||
|
||||
def find_whisper_cpp_bin() -> str:
|
||||
"""Locate the whisper.cpp 'whisper-cli' binary."""
|
||||
candidates = [
|
||||
os.environ.get("WHISPER_CPP_BIN"),
|
||||
shutil.which("whisper-cli"),
|
||||
str(Path.home() / ".local/share/meetrec/whisper-cpp/bin/whisper-cli"),
|
||||
]
|
||||
for c in candidates:
|
||||
if c and os.path.isfile(c) and os.access(c, os.X_OK):
|
||||
return c
|
||||
raise RuntimeError(
|
||||
"whisper-cli not found. Build whisper.cpp (e.g. './install.sh "
|
||||
"--whisper-cpp', or cmake -DWHISPER_VULKAN=ON) or set WHISPER_CPP_BIN "
|
||||
"to the binary.")
|
||||
|
||||
|
||||
def _whisper_cpp_cache() -> Path:
|
||||
cache = Path(os.environ.get(
|
||||
"MEETREC_CACHE", str(Path.home() / ".cache/meetrec")))
|
||||
d = cache / "whisper-cpp"
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def _cached_ggml(fname: str, url: str) -> str:
|
||||
"""Return the path of a cached GGML file, downloading it if missing."""
|
||||
p = _whisper_cpp_cache() / fname
|
||||
if p.exists() and p.stat().st_size > 0:
|
||||
return str(p)
|
||||
print(f"downloading {fname} to {p} ...", flush=True)
|
||||
_download_file(url, p)
|
||||
return str(p)
|
||||
|
||||
|
||||
def whisper_cpp_model_path(model_name: str) -> str:
|
||||
"""Return the local path of a whisper.cpp GGML model, downloading it."""
|
||||
try:
|
||||
fname = WHISPER_CPP_MODEL_FILES[model_name]
|
||||
except KeyError:
|
||||
raise RuntimeError(
|
||||
f"unknown model {model_name!r}; use one of: "
|
||||
f"{', '.join(WHISPER_CPP_MODEL_FILES)}")
|
||||
return _cached_ggml(fname, WHISPER_CPP_MODEL_URL.format(name=fname))
|
||||
|
||||
|
||||
def write_wav_float(path: str, audio: np.ndarray) -> None:
|
||||
"""Write a 1-D float32 mono @16 kHz array as 16-bit PCM WAV."""
|
||||
audio = np.asarray(audio, dtype=np.float32).ravel()
|
||||
pcm = (np.clip(audio, -1.0, 1.0) * 32767).astype(np.int16)
|
||||
with wave.open(path, "wb") as w:
|
||||
w.setnchannels(1)
|
||||
w.setsampwidth(2)
|
||||
w.setframerate(SR)
|
||||
w.writeframes(pcm.tobytes())
|
||||
|
||||
|
||||
def _parse_ts(s: str) -> float:
|
||||
"""whisper.cpp timestamp 'HH:MM:SS,mmm' -> seconds."""
|
||||
h, m, rest = s.split(":")
|
||||
sec, ms = rest.split(",")
|
||||
return int(h) * 3600 + int(m) * 60 + int(sec) + int(ms) / 1000.0
|
||||
|
||||
|
||||
class WhisperCppEngine:
|
||||
"""Transcription via the whisper.cpp 'whisper-cli' binary.
|
||||
|
||||
Build whisper.cpp with -DWHISPER_VULKAN=ON to run on a GPU (Radeon,
|
||||
Intel, or NVIDIA); it falls back to CPU otherwise. GPU vs. CPU is decided
|
||||
by the build, so --device/--compute-type do not apply. vad_filter=True
|
||||
enables whisper.cpp's Silero VAD (auto-downloaded); vad_parameters are
|
||||
accepted but not mapped.
|
||||
"""
|
||||
|
||||
def __init__(self, model_name: str, cli: str = None):
|
||||
self.model_name = model_name
|
||||
self.cli = cli or find_whisper_cpp_bin()
|
||||
self.model_path = whisper_cpp_model_path(model_name)
|
||||
|
||||
def transcribe(self, audio, beam_size=5, vad_filter=False,
|
||||
language=None, vad_parameters=None):
|
||||
# vad_parameters accepted for interface compatibility; Silero VAD uses
|
||||
# whisper.cpp's own defaults
|
||||
with tempfile.TemporaryDirectory(prefix="meetrec-") as td:
|
||||
if isinstance(audio, (str, os.PathLike)):
|
||||
wav = str(audio)
|
||||
with wave.open(wav, "rb") as w:
|
||||
duration = w.getnframes() / w.getframerate()
|
||||
else:
|
||||
wav = os.path.join(td, "in.wav")
|
||||
write_wav_float(wav, audio)
|
||||
duration = len(audio) / SR
|
||||
|
||||
cmd = [self.cli, "-m", self.model_path, "-f", wav,
|
||||
"-oj", "-of", os.path.join(td, "out"),
|
||||
"-bs", str(max(1, int(beam_size))), "-np",
|
||||
"-l", language or "auto"]
|
||||
if vad_filter:
|
||||
cmd += ["--vad", "-vm", _cached_ggml(
|
||||
WHISPER_CPP_VAD_FILE, WHISPER_CPP_VAD_URL)]
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if proc.returncode != 0:
|
||||
raise RuntimeError(
|
||||
(proc.stderr or proc.stdout).strip()
|
||||
or f"whisper-cli exited {proc.returncode}")
|
||||
with open(os.path.join(td, "out.json")) as f:
|
||||
data = json.load(f)
|
||||
|
||||
result = data.get("result", {})
|
||||
segs = []
|
||||
for item in data.get("transcription", []):
|
||||
off = item.get("offsets", {})
|
||||
if "from" in off and "to" in off: # milliseconds (current)
|
||||
a, b = off["from"] / 1000.0, off["to"] / 1000.0
|
||||
else: # 'HH:MM:SS,mmm' (older)
|
||||
ts = item.get("timestamps", {})
|
||||
a = _parse_ts(ts.get("from", "00:00:00,000"))
|
||||
b = _parse_ts(ts.get("to", "00:00:00,000"))
|
||||
text = item.get("text", "").strip()
|
||||
if text:
|
||||
segs.append(SimpleNamespace(start=a, end=b, text=text))
|
||||
info = SimpleNamespace(
|
||||
language=result.get("language") or language or "unknown",
|
||||
language_probability=1.0, # not reported by whisper.cpp
|
||||
duration=duration,
|
||||
)
|
||||
return segs, info
|
||||
|
||||
|
||||
def load_engine(model_name, engine="faster-whisper",
|
||||
device="auto", compute_type="int8"):
|
||||
"""Create a transcription engine. See the Engines section above."""
|
||||
if engine == "whisper-cpp":
|
||||
return WhisperCppEngine(model_name)
|
||||
if engine == "faster-whisper":
|
||||
from faster_whisper import WhisperModel
|
||||
return WhisperModel(model_name, device=device,
|
||||
compute_type=compute_type)
|
||||
raise RuntimeError(f"unknown engine {engine!r}; use 'faster-whisper' "
|
||||
"or 'whisper-cpp'")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Main
|
||||
# --------------------------------------------------------------------------
|
||||
@@ -308,10 +505,17 @@ def main():
|
||||
"(default: 8)")
|
||||
ap.add_argument("--language", default=None,
|
||||
help="force language code, e.g. en, de (default: autodetect)")
|
||||
ap.add_argument("--engine", default="faster-whisper",
|
||||
choices=["faster-whisper", "whisper-cpp"],
|
||||
help="transcription backend (default: faster-whisper). "
|
||||
"whisper-cpp can use a GPU via Vulkan — preferred "
|
||||
"on AMD hardware")
|
||||
ap.add_argument("--device", default="auto",
|
||||
help="compute device: auto / cpu / cuda (default: auto)")
|
||||
help="compute device: auto / cpu / cuda (default: auto); "
|
||||
"faster-whisper only")
|
||||
ap.add_argument("--compute-type", default="int8",
|
||||
help="int8 / int8_float16 / float16 / float32 (default: int8)")
|
||||
help="int8 / int8_float16 / float16 / float32 "
|
||||
"(default: int8); faster-whisper only")
|
||||
ap.add_argument("--list-sources", action="store_true",
|
||||
help="list available input devices and exit")
|
||||
args = ap.parse_args()
|
||||
@@ -322,10 +526,15 @@ def main():
|
||||
|
||||
device = resolve_source(args.source)
|
||||
|
||||
print(f"loading Whisper model {args.model!r} "
|
||||
f"(first run downloads it to ~/.cache/huggingface)...", flush=True)
|
||||
model = WhisperModel(args.model, device=args.device,
|
||||
compute_type=args.compute_type)
|
||||
print(f"loading Whisper model {args.model!r} via {args.engine!r} "
|
||||
f"(first run downloads it to ~/.cache)...")
|
||||
try:
|
||||
model = load_engine(args.model, engine=args.engine,
|
||||
device=args.device,
|
||||
compute_type=args.compute_type)
|
||||
except Exception as e:
|
||||
print(f"error: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
rec = Recorder(device, args.output + ".wav")
|
||||
rec.start()
|
||||
|
||||
+43
-15
@@ -32,7 +32,6 @@ import sounddevice as sd
|
||||
import meetrec
|
||||
from meetrec import (
|
||||
Recorder,
|
||||
WhisperModel,
|
||||
final_transcribe,
|
||||
fmt_hms,
|
||||
live_worker,
|
||||
@@ -50,18 +49,19 @@ class ModelLoader(QThread):
|
||||
loaded = Signal(object)
|
||||
failed = Signal(str)
|
||||
|
||||
def __init__(self, model_name, device="auto", compute_type="int8",
|
||||
parent=None):
|
||||
def __init__(self, model_name, engine="faster-whisper", device="auto",
|
||||
compute_type="int8", parent=None):
|
||||
super().__init__(parent)
|
||||
self.model_name = model_name
|
||||
self.engine = engine
|
||||
self.device = device
|
||||
self.compute_type = compute_type
|
||||
|
||||
def run(self):
|
||||
try:
|
||||
self.model = WhisperModel(self.model_name,
|
||||
device=self.device,
|
||||
compute_type=self.compute_type)
|
||||
self.model = meetrec.load_engine(
|
||||
self.model_name, engine=self.engine,
|
||||
device=self.device, compute_type=self.compute_type)
|
||||
self.loaded.emit(self.model)
|
||||
except Exception as e:
|
||||
self.failed.emit(str(e))
|
||||
@@ -104,6 +104,7 @@ class MeetRecWindow(QWidget):
|
||||
"pt", "ru", "zh", "ja", "ko"]
|
||||
DEVICES = ["auto", "cpu", "cuda"]
|
||||
COMPUTE_TYPES = ["int8", "int8_float16", "float16", "float32"]
|
||||
ENGINES = ["faster-whisper", "whisper-cpp"]
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
@@ -111,7 +112,7 @@ class MeetRecWindow(QWidget):
|
||||
self.setMinimumSize(700, 600)
|
||||
|
||||
self.model = None
|
||||
self.model_name = None # model currently loaded into memory
|
||||
self.model_key = None # (engine, model, device, compute) loaded
|
||||
self.rec = None
|
||||
self.state = "idle" # idle | loading | record | finalizing
|
||||
self.decay = 0.0
|
||||
@@ -165,14 +166,25 @@ class MeetRecWindow(QWidget):
|
||||
r2.addWidget(self.lang)
|
||||
root.addLayout(r2)
|
||||
|
||||
# row 2b: compute device + precision
|
||||
# row 2b: engine + compute device + precision
|
||||
r2b = QHBoxLayout()
|
||||
r2b.addWidget(QLabel("Device:"))
|
||||
r2b.addWidget(QLabel("Engine:"))
|
||||
self.engine_cb = QComboBox()
|
||||
self.engine_cb.addItems(self.ENGINES)
|
||||
self.engine_cb.setCurrentText("faster-whisper")
|
||||
self.engine_cb.activated.connect(self._engine_changed)
|
||||
self.engine_cb.setToolTip("faster-whisper: NVIDIA CUDA or CPU.\n"
|
||||
"whisper-cpp: GPU via Vulkan (AMD, Intel, "
|
||||
"NVIDIA) or CPU — preferred on AMD.")
|
||||
r2b.addWidget(self.engine_cb)
|
||||
self.device_lbl = QLabel("Device:")
|
||||
r2b.addWidget(self.device_lbl)
|
||||
self.device_cb = QComboBox()
|
||||
self.device_cb.addItems(self.DEVICES)
|
||||
self.device_cb.setCurrentText("auto")
|
||||
r2b.addWidget(self.device_cb)
|
||||
r2b.addWidget(QLabel("Compute:"))
|
||||
self.compute_lbl = QLabel("Compute:")
|
||||
r2b.addWidget(self.compute_lbl)
|
||||
self.compute_cb = QComboBox()
|
||||
self.compute_cb.addItems(self.COMPUTE_TYPES)
|
||||
self.compute_cb.setCurrentText("int8")
|
||||
@@ -274,10 +286,22 @@ class MeetRecWindow(QWidget):
|
||||
else:
|
||||
self.src.setCurrentText(keep)
|
||||
|
||||
def _engine_changed(self, _idx):
|
||||
# --device/--compute-type only apply to faster-whisper (CTranslate2);
|
||||
# whisper.cpp picks GPU (Vulkan) or CPU on its own.
|
||||
gpu_opts = self.engine_cb.currentText() != "whisper-cpp"
|
||||
self.device_lbl.setText("Device:" if gpu_opts else "Device (n/a):")
|
||||
self.compute_lbl.setText("Compute:" if gpu_opts else "Compute (n/a):")
|
||||
self.device_cb.setEnabled(gpu_opts)
|
||||
self.compute_cb.setEnabled(gpu_opts)
|
||||
|
||||
def _set_inputs_enabled(self, enabled: bool):
|
||||
for w in (self.src, self.model_cb, self.device_cb, self.compute_cb,
|
||||
self.lang, self.live_cb, self.live_spin, self.out):
|
||||
for w in (self.src, self.model_cb, self.engine_cb, self.device_cb,
|
||||
self.compute_cb, self.lang, self.live_cb, self.live_spin,
|
||||
self.out):
|
||||
w.setEnabled(enabled)
|
||||
if enabled:
|
||||
self._engine_changed(-1) # whisper-cpp: keep device/compute off
|
||||
|
||||
def _to_idle(self):
|
||||
self.state = "idle"
|
||||
@@ -316,13 +340,17 @@ class MeetRecWindow(QWidget):
|
||||
self.level.setValue(0)
|
||||
self.elapsed.setText("00:00:00")
|
||||
|
||||
if self.model is None or self.model_name != self._args.model:
|
||||
key = (self.engine_cb.currentText(), self._args.model,
|
||||
self.device_cb.currentText(), self.compute_cb.currentText())
|
||||
if self.model is None or self.model_key != key:
|
||||
self.state = "loading"
|
||||
self.status.setText(
|
||||
f"Loading Whisper model {self._args.model!r} "
|
||||
f"Loading Whisper model {self._args.model!r} via {key[0]} "
|
||||
"(first run downloads it, this can take a minute)...")
|
||||
self._pending_key = key
|
||||
self.loader = ModelLoader(
|
||||
self._args.model,
|
||||
engine=key[0],
|
||||
device=self.device_cb.currentText(),
|
||||
compute_type=self.compute_cb.currentText(),
|
||||
parent=self)
|
||||
@@ -334,7 +362,7 @@ class MeetRecWindow(QWidget):
|
||||
|
||||
def _model_ready(self, model):
|
||||
self.model = model
|
||||
self.model_name = self._args.model
|
||||
self.model_key = self._pending_key
|
||||
self._start_recording()
|
||||
|
||||
def _model_failed(self, err):
|
||||
|
||||
Reference in New Issue
Block a user