Our Discord voice bot answers to the name “Rina”, and it kept writing that name down wrong: “Inha”, “Nina”, “Bina”. We traced the errors to two causes, a recording that starts in the middle of the word and a proper noun that whisper.cpp kept getting wrong even with the name in its prompt. We replaced the local speech-to-text engine on our DGX Spark, whisper.cpp large-v3 (q5_0), with Qwen3-ASR-1.7B and gave it the name as context. The name then came out right in 5 of the 6 recorded segments we had. On a 200-sentence Korean benchmark the character error rate (CER, averaged per utterance) fell from 5.52 % to 4.36 %, and median latency rose from 0.56 s to 1.03 s. The new server imitates the whisper.cpp HTTP API, so the two other programs that use the same engine did not change. The clipped word onsets are a separate bug in the voice-receive path. We traced it in the code and patched it, but as of this writing no voice segment has been recorded since the patch, so we do not yet know whether it works.
One relay, three callers
We described this voice assistant in an earlier post: text-to-speech, the language model and speech-to-text all run on our own hardware. The speech-to-text part sits on an NVIDIA DGX Spark (GB10) behind a small relay server. The voice bot itself is built on OpenClaw and runs on a Mac mini. When someone speaks in the Discord voice channel, OpenClaw’s Discord voice receiver records the segment, a script on the Mac mini sends the audio to the relay, and the transcribed text starts the bot’s turn.
Two other programs send audio to the same relay: a conversation app for Even Realities G2 smart glasses, and the transcription of Slack voice messages. That shared relay set a constraint for everything that follows. If the replacement engine spoke the same HTTP API as whisper.cpp’s server, only the engine behind the relay would change, and none of the three callers would need an edit.
The engine in production until 2026-10-29 was whisper.cpp with the large-v3 model quantized to q5_0, its built-in voice-activity detection (--vad) turned on, and --prompt "리나" (the name written in Korean) to bias it toward the name.
Two sources of the errors
The bot keeps the audio of each segment it transcribes for debugging. We took the three segments from the evening of 10/29 that prompted this work and three older ones from 10/15, and ran all six through three engines. The whisper.cpp column is what the production setup produced at the time. The Korean outputs are given as transcribed, with a rough English gloss; the name is 리나 (Rina).
| Segment | Length | Level of first 100 ms | whisper.cpp (production) | Qwen3-ASR-1.7B + context “리나” | Cohere Transcribe |
|---|---|---|---|---|---|
| 10/15 06:35:09 | 1.78 s | −80.2 dBFS | 비나야 들려? (“Bina, can you hear?”) | 리나야 들려? (“Rina, can you hear?”) | 이나야, 틀려? (“Ina, wrong?”) |
| 10/15 06:36:14 | 1.30 s | −96.3 dBFS | 리니 (“Rini”) | 들리니. (“Can you hear?”) | 니니. (“Nini.”) |
| 10/15 06:37:27 | 1.96 s | −28.4 dBFS | 니 아리나라고 불렀어 (“ni, I called (it) Arina”) | 네, 리나라고 불렀어. (“Yes, I called Rina.”) | 니야, 리나라고 불렀어. (“Niya, I called Rina.”) |
| 10/29 21:26:32 | 0.88 s | −19.6 dBFS | 인하 (“Inha”) | 리나. (“Rina.”) | 이나. (“Ina.”) |
| 10/29 21:27:23 | 2.00 s | −18.1 dBFS | 니나 넓은 거지 (“Nina, it’s wide”) | 리나 너 부은 거지. (“Rina, you’re swollen”) | 지나도 없군 거지. (no clear meaning) |
| 10/29 21:27:48 | 4.58 s | −50.8 dBFS | 아니 리나 널 부른 거라고 (“No, Rina, I was calling you”) | 아니, 리나 널 부른 거라고. (same words as the whisper.cpp output) | 아니, 니나 널 부른 거라고? (“No, Nina, I was calling you?”) |
Two rows need a caveat. We could not establish what was actually said in the 10/15 06:36:14 segment, so we do not score it for any engine; Qwen3-ASR’s output there contains no name at all. In the 10/29 21:27:23 segment Qwen3-ASR got the name, but we have not confirmed the rest of the sentence, so we count only the name. So the “5 of 6” above is a plain count: Qwen3-ASR’s output contains the name in 5 of the 6 segments, and the sixth is the unscored 06:36:14 row, whose output has no name in it.
The errors split into two causes.
The word onset was cut off. The two wrong segments from 10/29 start at speech level: their first 100 ms already measure −19.6 and −18.1 dBFS. The 4.58 s segment, which whisper.cpp and Qwen3-ASR both got right, starts with silence at −50.8 dBFS. A recording that starts at full speech level began in the middle of a word. We prepended 300 ms of silence to the clipped segments and transcribed them again, and the results barely changed. A missing syllable cannot be recovered by any engine. Where the clipping happens is the subject of a later section.
The name is uncommon. The first two 10/15 segments start with silence (−80.2 and −96.3 dBFS), and whisper.cpp still wrote “Bina” and “Rini”. We tried four changes to whisper.cpp one at a time: voice-activity detection off, the prompt removed, beam search with width 5, and the large-v3-turbo model. Each gave the same or a worse result. whisper.cpp did not correct the name even with --prompt "리나". Qwen3-ASR, given the same word as context, did use it.
Candidate engines on a Korean benchmark
We measured the candidates on a fixed 200-sentence Korean set: 100 sentences from the FLEURS ko_kr test split and 100 from the Zeroth Korean test split. CER is computed on normalized text that keeps only letters and digits. We report it two ways: macro averages the CER of each utterance, and micro treats the whole set as one long string. Requests were sent one at a time.
| Engine | CER macro | CER micro | p50 latency | p90 latency | How it was measured |
|---|---|---|---|---|---|
| whisper.cpp large-v3-turbo | 6.37 % | 7.11 % | 0.35 s | 0.49 s | measured 10/02 |
| whisper.cpp large-v3 q5_0 | 5.52 % | 6.19 % | 0.56 s | 0.75 s | measured 10/02; in production until 10/29 |
| Cohere Transcribe 03-2026 | 7.29 % | 8.26 % | 0.20 s | 0.39 s | in-process, no context input |
| Qwen3-ASR-1.7B, no context | 4.73 % | 5.41 % | 0.99 s | 1.63 s | in-process |
| Qwen3-ASR-1.7B + context “리나” + VAD | 4.36 % | 5.08 % | 1.03 s | 1.68 s | the deployed server, over HTTP |
| OpenAI gpt-4o-transcribe | 4.41 % | 5.09 % | 1.01 s | 1.34 s | cloud, for reference |
The rows were not all measured the same way. The two whisper.cpp rows come from an earlier run on 10/02, Cohere and Qwen3-ASR without context were timed inside the process with no server in between, and the deployed configuration was timed through its HTTP server. The latencies are therefore close but not strictly comparable. They also describe benchmark sentences 4.7 to 24 s long. The utterances in the voice channel are 1 to 5 s long, and after the deployment those were transcribed in 0.1 to 0.45 s.
Cohere Transcribe was the fastest and the least accurate on the benchmark, so we dropped it. Qwen3-ASR without context already beat the production whisper.cpp by 0.79 points of CER (calculated: 5.52 − 4.73). With the name as context and voice-activity detection in front of it, it went down to 4.36 %, although the benchmark sentences never contain the name. That last row changes two things at once, the context and the VAD, and we did not separate their effects. The cloud model we used as a reference landed almost exactly where the deployed local engine did.
A side effect we had to rule out
The G2 glasses app sends the last two transcribed lines, up to 120 characters, as the prompt of each request so that the engine has context. While testing, we gave Qwen3-ASR a short name segment together with two unrelated sentences as context, and it returned one of the context sentences instead of what was said. If that happened often, the glasses app would show the user text from earlier in the conversation as if it were new speech. So we compared both engines under the same conditions.
| Condition | whisper.cpp | Qwen3-ASR |
|---|---|---|
| 120 benchmark sentences, previous two sentences as context: CER macro | 4.91 % | 4.98 % |
| 120 clips of 1.2 s: outputs that copied the context | 1 | 0 |
| 6 name segments × 4 contexts: outputs that appear to copy the context | 3 / 24 | 1 / 24 |
Both engines can copy context into the output when the audio is short. Qwen3-ASR did not do it more often than whisper.cpp did. The last row was judged by reading the outputs, which is why it says “appear to”. We left the glasses app’s settings as they were.
A server that looks like whisper.cpp
The new server answers the same two endpoints as whisper.cpp’s server: GET /, and POST /inference with a multipart file, language, response_format and an optional prompt, returning {"text": ...}. The relay and the three callers did not change.
- Model. Qwen3-ASR-1.7B in its
-hfcheckpoint, loaded once in bf16. It takes 4.2 GB of GPU memory, and the server handles one request at a time. Theqwen-asrpip package would not import with transformers 5.12 (an error incheck_model_inputs()), so we used the built-in support for the-hfcheckpoint in transformers 5.13 and later, in a separate virtual environment (transformers 5.21.1). That kept the text-to-speech environment on the same machine untouched. - Default context. If a caller sends no prompt, the server uses “리나” as context. This takes the place of whisper.cpp’s
--prompt. If a caller sends its own context, as the glasses app does, that context is used instead. - Silence gate. Qwen3-ASR produces sentences such as 그렇죠. (“Right.”) from digital silence. So the server first asks Silero VAD whether the audio contains speech at all. If it does not, the model is not run and the server returns an empty string. The audio is not trimmed to the speech regions. On the 200 benchmark sentences the gate wrongly discarded none. An empty result makes the Mac mini script exit with an error, and the bot does not start a turn for that segment, which is the same behavior it had with whisper.cpp.
- Which VAD build. The PyTorch build of Silero VAD took 4.4 s on one 17 s clip when called from the request thread. The ONNX build took 0.04 s, so we use ONNX. Loading Silero VAD also sets PyTorch’s CPU thread count to 1, which slowed the ASR step itself from 1.35 s to around 1.8 s. The server restores the thread count after loading it.
- No fallback. If the audio cannot be decoded the server returns 400, and if the model fails it returns 500, each with the cause. The relay passes either on as 502. The server does not fall back to another engine. Every setting is read from a configuration file, and the server refuses to start if any is missing. Logs rotate at 1 MB × 3 files.
- Rollback. The whisper.cpp service and its model were stopped, not deleted. Switching back is three steps, which the deployment record puts at about a minute.
We deployed it on 2026-10-29 at 23:25 KST and checked it through the real path, not with a unit test. The relay’s health check named the new engine. The Mac mini’s own transcription script, run on the “Inha” segment from 21:26:32, now returned “리나.” All six segments sent through the relay gave the outputs in the Qwen3-ASR column above, and a silent clip returned an empty string. When we checked again on 2026-10-31 at 00:20 KST, the relay was still reporting Qwen3-ASR-1.7B.
The clipped onsets: traced and patched, not yet confirmed
After the swap the name came out right far more often, but the cut-off word onsets remained. The Discord client on the speaking side had every voice setting, input sensitivity included, on automatic, so there was nothing to tune there. We read the receive path on the bot’s side instead.
The receiver in @discordjs/voice 0.19.2 raises a “speaking start” event as soon as a speaker’s first packet arrives. If nothing has subscribed to that speaker’s audio yet, it drops the packet. In OpenClaw’s Discord voice code (we checked versions 2026.10.8 and 2026.10.9), an audio worker receives the event and forwards it to the main thread. The main thread goes through several asynchronous steps and then tells the worker to capture, and only then does the worker subscribe. Every packet that arrives during that round trip is lost. If the same person speaks again within one second, the bot reuses the capture already in progress, so the loss happens only at the start of a new segment.
The patch makes the worker subscribe on its own as soon as it sees “speaking start”, hold the incoming Opus packets in a buffer, and replay them ahead of the live packets when the main thread’s capture request arrives. Each replay writes a log line with the number of packets it replayed. A buffer that nobody claims is emptied at the next “speaking start” and released after 2000 ms, so it cannot leak into a later capture. The buffer holds at most 100 packets per speaker, which at 20 ms per packet is 2 s of audio (calculated).
We applied it to the Mac mini’s OpenClaw install on the morning of 2026-10-30. On the first restart the bot failed to join the voice channel with a timeout. On the second restart, with the same patched file, it joined. The patch does not touch the join step, and we think the first failure was a transient connection timeout, but that is a guess: there were no logs from earlier restarts to compare with.
Here is the part we cannot report yet. As of 2026-10-31 00:20 KST, no voice segment has been recorded since the patch went in. The most recent recording is from 06:42 on 10/30, before it. We do not know whether the patch reduces the clipping.
We also know how we will judge it, and why the obvious measure does not work. The level of the first 100 ms cannot settle the question, because the Discord client has its own input gate and only sends packets once that gate opens, so even an unclipped recording can start at speech level. The four recordings from just before the patch started at −24.0, −20.5, −110 and −56.8 dBFS. Instead we will read the replay count in the log for each segment. Each replayed packet is 20 ms of audio that would have been dropped before. If the count is zero for every segment, the round trip was not causing the loss, and the cause is somewhere else.
Once that is settled we plan to open an issue on the OpenClaw repository with the cause and the proposed change. We have not opened it yet. Until a change lands upstream, the local patch carries a maintenance cost: updating OpenClaw installs it into a fresh directory, and the patch disappears without any error. Every update has to be followed by a check that the patched file is still in place.
The case against this swap
A cloud model does the same job. OpenAI’s gpt-4o-transcribe scored 4.41 % CER at 1.01 s median latency, within a few hundredths of the deployed local engine (4.36 %, 1.03 s), and its p90 was lower (1.34 s against 1.68 s). Our reason for staying local is not that local was more accurate. Three programs were already wired to the local relay, and the change could be made by replacing the engine behind it without touching any of them. We have not compared cost or privacy in numbers here.
Latency nearly doubled. Median latency on the benchmark went from 0.56 s to 1.03 s, about 1.84 times (calculated). On the short utterances the voice channel actually produces, the deployed engine stayed between 0.1 and 0.45 s. Utterances longer than 10 s run about 0.5 s slower than they did on whisper.cpp. We have not tried serving the model with vLLM.
The name test is small. It rests on six segments, and for one of them we do not know what was said. The benchmark covers general Korean, not names.
whisper.cpp was not exhausted. We tried four changes, one of which was a different model. We did not fine-tune whisper on the name.
The engine cannot restore what the receiver drops. The prepended-silence test showed that. Better name recognition does not remove the clipping.
Next steps
- Read the replay counts from the first real voice sessions after the patch and decide from them whether the clipping came from the receive round trip.
- If it did, open the issue upstream with the cause and the change.
- If the assistant ever needs real-time streaming transcription, Qwen3-ASR served through vLLM’s streaming mode is a candidate. We have not tried vLLM serving yet.
