M3d: speaker labels via tinydiarize two-pass merge
- whisper-server stack: second container (port 8086) running the English-trained small.en-tdrz model with -tdrz; image patched (speaker-turn.patch) to expose speaker_turn_next per segment in verbose_json like the cli example does - core/whisper Diarization: merges the tdrz pass's TURN TIMES onto the quality transcript as alternating 'Sprecher 1/2:' labels, splitting segments when a turn falls inside them; no turns detected = no labels (never mislabels); 6 unit tests - RemoteWhisperEngine gains a diarize flag (sends tinydiarize=true, parses speaker_turn_next); WhisperEngine.Segment carries the flag - RecorderService: optional second pass on the diarize server after the final pass; failures keep the unlabeled transcript - Settings: Diarize server URL (persisted; empty disables) - validated infrastructure locally: patched image builds, tdrz model downloads from akashmjn/tinydiarize-whisper.cpp, speaker_turn_next present in responses; synthetic espeak audio does not trigger the model's turn tokens — real two-person speech needed for the end-to-end check
This commit is contained in:
@@ -1,14 +1,16 @@
|
||||
# whisper.cpp inference server for meetrec, with Vulkan GPU support
|
||||
# whisper.cpp inference servers for meetrec, with Vulkan GPU support
|
||||
# (AMD Radeon AI PRO R9700), reachable over Tailscale.
|
||||
#
|
||||
# RENDER_GID=$(getent group render | cut -d: -f3) docker compose up -d --build
|
||||
#
|
||||
# Security: whisper.cpp's server has NO authentication. The port below is
|
||||
# bound ONLY to the Tailscale interface (100.103.83.12), so the API is
|
||||
# never exposed to the LAN or the internet. Tailscale must own that IP
|
||||
# before the container starts, otherwise the bind fails — start order:
|
||||
# tailscale first, then `docker compose up -d`. If you would rather
|
||||
# tolerate LAN exposure, use "8085:8085" instead.
|
||||
# Two services:
|
||||
# whisper-server port 8085 — large-v3, quality transcripts
|
||||
# whisper-server-tdrz port 8086 — small.en-tdrz, tinydiarize speaker
|
||||
# turns (English-trained, 2-speaker, best-effort)
|
||||
#
|
||||
# The image is patched to expose `speaker_turn_next` per segment in
|
||||
# verbose_json (speaker-turn.patch); clients merge the turn times onto
|
||||
# the better transcript as "Sprecher 1/2" labels.
|
||||
|
||||
services:
|
||||
whisper-server:
|
||||
@@ -45,4 +47,41 @@ services:
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
|
||||
whisper-server-tdrz:
|
||||
# tinydiarize variant: English-only small.en-tdrz, used ONLY for its
|
||||
# speaker-turn timestamps (meetrec merges them onto the better
|
||||
# transcript from the main server). 2-speaker detection, best-effort
|
||||
# on non-English audio.
|
||||
build: .
|
||||
image: meetrec-whisper-server:latest
|
||||
container_name: whisper-server-tdrz
|
||||
restart: unless-stopped
|
||||
|
||||
environment:
|
||||
MODEL: small.en-tdrz
|
||||
MODEL_URL: https://huggingface.co/akashmjn/tinydiarize-whisper.cpp/resolve/main/ggml-small.en-tdrz.bin
|
||||
THREADS: 8
|
||||
HOST: 0.0.0.0
|
||||
PORT: 8086
|
||||
TDRZ: 1
|
||||
|
||||
volumes:
|
||||
- ./models:/models # shared with the main server (distinct files)
|
||||
|
||||
ports:
|
||||
- "100.103.83.12:8086:8086" # tailscale-only
|
||||
|
||||
devices:
|
||||
- /dev/dri:/dev/dri
|
||||
group_add:
|
||||
- video
|
||||
- "${RENDER_GID:-110}"
|
||||
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "curl -s -o /dev/null http://localhost:8086/ || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
Reference in New Issue
Block a user