---
title: "Voice assistant"
canonical: https://docs.qvac.tether.io/sdk/v0.23/ai-capabilities/voice-assistant/
collection: "SDK"
package: "@qvac/sdk"
line: v0.23
current_line: false
---

# Voice assistant (/sdk/v0.23/ai-capabilities/voice-assistant)



## Overview

A voice assistant chains three AI capabilities into a continuous conversation loop:

<Mermaid
  chart="flowchart LR
    Mic([Microphone]) -->|PCM audio| ASR[ASR<br/>Whisper + Silero VAD]
    ASR -->|utterance| LLM[LLM<br/>text generation]
    LLM -->|response text| TTS[TTS<br/>Supertonic]
    TTS -->|audio| Spk([Speakers])
    Spk -.conversation loop.-> Mic"
/>

Each stage maps to a QVAC capability — [Transcription](/sdk/v0.23/ai-capabilities/transcription) (ASR), [Text generation](/sdk/v0.23/ai-capabilities/text-generation) (LLM), and [Text-to-Speech](/sdk/v0.23/ai-capabilities/text-to-speech) (TTS).

Compared to using each capability individually, the key differences are:

* You need to coordinate **three model loads** simultaneously (Whisper + VAD, LLM, and TTS bundle) — they all stay loaded for the duration of the session.
* VAD parameters need **conservative tuning** to avoid the assistant transcribing its own TTS output (self-hearing feedback loop).
* You should **gate the microphone during TTS playback** and apply a short post-playback cooldown so room reverb doesn't bleed into the next utterance.
* You should **filter short or non-linguistic transcripts** (e.g. `"."`, `"[BLANK_AUDIO]"`) since Whisper hallucinates them from near-silent audio.

## Functions

Use the following sequence of function calls:

1. [`loadModel()`](/sdk/v0.23/reference/api#loadmodel) three times — once per `modelType` (`"whisper"`, `"llm"`, `"tts"`).
2. [`transcribeStream()`](/sdk/v0.23/reference/api#transcribestream) — open a streaming session that emits utterances on VAD-detected pauses.
3. [`completion()`](/sdk/v0.23/reference/api#completion) — generate a response from the rolling conversation history (streamed).
4. [`textToSpeech()`](/sdk/v0.23/reference/api#texttospeech) — synthesize the response into a PCM buffer.
5. [`unloadModel()`](/sdk/v0.23/reference/api#unloadmodel) for each loaded model on shutdown.

For how to use each function, see [SDK — API reference](/sdk/v0.23/reference/api/).

## Models

You load four model bundles in total:

* A `qvac-fabric-speech.cpp`-compatible model for transcription, plus a Silero VAD model.
* A `llama.cpp`-compatible LLM for response generation.
* A Supertonic TTS bundle (text encoder, duration predictor, vector estimator, vocoder, unicode indexer, config, and voice style).

Recommended defaults (used in the example below):

| Stage | Model                    |
| ----- | ------------------------ |
| ASR   | `WHISPER_TINY`           |
| VAD   | `VAD_SILERO_5_1_2`       |
| LLM   | `LLAMA_3_2_1B_INST_Q4_0` |
| TTS   | Supertonic2 (English)    |

For models available as constants, see [SDK — Models](/sdk/v0.23/#models).

## Example

This example is **desktop-only**. Mobile (React Native / Expo) needs a different audio path and isn't covered here.

### Requirements

* **FFmpeg** (with `ffplay`) on `PATH` — `ffmpeg` captures mic audio, `ffplay` plays back TTS output.
* **Microphone** access (on macOS, the running shell needs mic permission in *System Settings → Privacy & Security → Microphone*).
* **Speakers** connected and selected as the default output device.

<Accordions type="multiple">
  <Accordion title="Installing FFmpeg">
    | Platform         | Command                                                                                              |
    | ---------------- | ---------------------------------------------------------------------------------------------------- |
    | macOS (Homebrew) | `brew install ffmpeg`                                                                                |
    | Debian / Ubuntu  | `sudo apt update && sudo apt install ffmpeg`                                                         |
    | Fedora / RHEL    | `sudo dnf install ffmpeg` (enable [RPM Fusion](https://rpmfusion.org/Configuration) first if needed) |
    | Arch Linux       | `sudo pacman -S ffmpeg`                                                                              |

    Verify the install with:

    ```bash
    ffmpeg -version
    ```
  </Accordion>

  <Accordion title="Selecting a microphone">
    By default the example uses the system default mic on each OS:

    * **macOS:** AVFoundation audio device `:0`.
    * **Linux:** PulseAudio source `default`.

    To use a different mic, set the `MIC_DEVICE` environment variable:

    ```bash
    # macOS — pick by index (list with `ffmpeg -f avfoundation -list_devices true -i ""`)
    MIC_DEVICE=":1" npx tsx examples/voice-assistant/voice-assistant.ts

    # Linux — pick a PulseAudio source (list with `pactl list short sources`)
    MIC_DEVICE="alsa_input.usb-Blue_Microphones_Yeti-00" \
      npx tsx examples/voice-assistant/voice-assistant.ts
    ```
  </Accordion>
</Accordions>

### Running it

The following script implements the full loop with VAD tuning, mic gating during playback, and short-utterance filtering:

<Tabs>
  <Tab value="js" label="JavaScript" default>
    <WrapCode>
      ```js file=<rootDir>/packages/sdk/dist/examples/voice-assistant/voice-assistant.js title="voice-assistant.js" lineNumbers
      /**
       * Real-time Voice Assistant: mic → Whisper (with Silero VAD) → Llama → Supertonic TTS.
       *
       * Usage: npx tsx examples/voice-assistant/voice-assistant.ts
       *
       * Speak a question; the VAD detects when you pause, the utterance is
       * transcribed, sent to the LLM, and the response is spoken back. The loop
       * continues until you press Ctrl+C. While the assistant is speaking, mic
       * audio is dropped so it does not hear itself.
       *
       * Requirements: FFmpeg installed, microphone access, speakers.
       */
      import { loadModel, unloadModel, transcribeStream, completion, textToSpeech, WHISPER_TINY, VAD_SILERO_5_1_2, LLAMA_3_2_1B_INST_Q4_0, TTS_EN_SUPERTONIC_Q8_0 } from '@qvac/sdk';
      import { spawnSync } from 'child_process';
      import { startMicrophone } from '../audio/mic-input.js';
      import { createWavHeader, int16ArrayToBuffer, playAudio } from '../tts/utils.js';
      const MIC_SAMPLE_RATE = 16000;
      const TTS_SAMPLE_RATE = 44100;
      const SYSTEM_PROMPT = 'You are a concise, friendly voice assistant. Keep responses under two sentences. ' +
          'Never use markdown, lists, or code blocks — your output will be spoken aloud.';
      // VAD parameters tuned for conversational speech without the assistant looping
      // on its own echo. These defaults are deliberately conservative:
      //   - threshold 0.6: less sensitive than Silero's default; avoids triggering
      //     on TTS reverb bleeding into the mic or low-level background noise.
      //   - min_speech_duration_ms 300: drops short clicks/breaths and stray words.
      //   - min_silence_duration_ms 700: requires a longer quiet tail before
      //     committing a segment. Crucial for preventing self-hearing feedback loops
      //     where Whisper hallucinates content from near-silent audio.
      //   - max_speech_duration_s 15: caps runaway utterances.
      //   - speech_pad_ms 200: padding improves accuracy on utterance edges.
      // If the assistant cuts you off mid-sentence, raise min_silence_duration_ms.
      // If it keeps hallucinating / talking to itself, raise threshold to 0.7 and/or
      // min_silence_duration_ms to 900.
      const VAD_PARAMS = {
          threshold: 0.6,
          min_speech_duration_ms: 300,
          min_silence_duration_ms: 700,
          max_speech_duration_s: 15.0,
          speech_pad_ms: 200
      };
      // Short grace period after TTS playback before we start listening again.
      // Gives the speaker amp / room reverb a moment to fully settle so the first
      // post-playback mic frames don't get transcribed as the tail of our own voice.
      const POST_PLAYBACK_COOLDOWN_MS = 300;
      // Minimum characters for an utterance to be considered meaningful. Whisper
      // frequently hallucinates single words like "you", ".", or "Thanks." from
      // silence or faint noise; these short phantoms are the main driver of the
      // self-hearing feedback loop, so we drop them.
      const MIN_UTTERANCE_CHARS = 3;
      function isMeaningfulTranscript(text) {
          const trimmed = text.trim();
          if (trimmed.length === 0)
              return false;
          if (trimmed.includes('[No speech detected]'))
              return false;
          // Whisper sometimes emits non-linguistic cues on silence, e.g. "[BLANK_AUDIO]".
          if (/^\[[^\]]+\]$/.test(trimmed))
              return false;
          // Strip punctuation/whitespace for the length check so ". . ." is rejected.
          const letters = trimmed.replace(/[^\p{L}\p{N}]/gu, '');
          if (letters.length < MIN_UTTERANCE_CHARS)
              return false;
          return true;
      }
      function sleep(ms) {
          return new Promise((resolve) => setTimeout(resolve, ms));
      }
      // ── Main ──
      for (const tool of ['ffmpeg', 'ffplay']) {
          const r = spawnSync(tool, ['-version'], { stdio: 'ignore' });
          if (r.error || r.status !== 0) {
              console.error(`✖ ${tool} not found on PATH. Install ffmpeg (ffplay ships with it) and retry.`);
              process.exit(1);
          }
      }
      console.log('▸ Loading whisper-tiny + Silero VAD...');
      const asrModelId = await loadModel({
          modelSrc: WHISPER_TINY,
          modelConfig: {
              vadModelSrc: VAD_SILERO_5_1_2,
              audio_format: 'f32le',
              strategy: 'greedy',
              n_threads: 4,
              language: 'en',
              no_timestamps: true,
              suppress_blank: true,
              suppress_nst: true,
              temperature: 0.0,
              vad_params: VAD_PARAMS
          }
      });
      console.log('▸ Loading Llama 3.2 1B...');
      const llmModelId = await loadModel({
          modelSrc: LLAMA_3_2_1B_INST_Q4_0,
          modelConfig: {
              ctx_size: 4096
          }
      });
      console.log('▸ Loading Supertonic TTS...');
      const ttsModelId = await loadModel({
          modelSrc: TTS_EN_SUPERTONIC_Q8_0,
          modelConfig: {
              ttsEngine: 'supertonic',
              language: 'en',
              voice: 'F1',
              ttsSpeed: 1.05,
              ttsNumInferenceSteps: 5
          }
      });
      console.log('▸ All models loaded.\n');
      const ffmpeg = startMicrophone({
          sampleRate: MIC_SAMPLE_RATE,
          format: 'f32le'
      });
      const session = await transcribeStream({ modelId: asrModelId });
      const history = [{ role: 'system', content: SYSTEM_PROMPT }];
      // Dropped-chunk gate: while the assistant is speaking we stop feeding the mic
      // stream into the ASR session. Using a flag (rather than pausing the ffmpeg
      // pipe) keeps the pipe drained so we never accumulate stale audio, and the
      // VAD starts fresh on the next user turn.
      let isSpeaking = false;
      ffmpeg.stdout.on('data', (chunk) => {
          if (isSpeaking)
              return;
          session.write(chunk);
      });
      let shuttingDown = false;
      async function cleanup() {
          if (shuttingDown)
              return;
          shuttingDown = true;
          console.log('\n\n▸ Stopping...');
          ffmpeg.kill();
          try {
              session.end();
          }
          catch {
              // session may already be closed
          }
          await unloadModel({ modelId: ttsModelId }).catch(() => { });
          await unloadModel({ modelId: llmModelId }).catch(() => { });
          await unloadModel({ modelId: asrModelId }).catch(() => { });
          console.log('▸ Done.');
          process.exit(0);
      }
      process.on('SIGINT', () => void cleanup());
      process.on('SIGTERM', () => void cleanup());
      console.log('▸ Listening. Speak a question and pause. Ctrl+C to quit.\n');
      for await (const rawText of session) {
          if (!isMeaningfulTranscript(rawText))
              continue;
          const userText = rawText.trim();
          console.log(`▸ You: ${userText}`);
          history.push({ role: 'user', content: userText });
          isSpeaking = true;
          try {
              console.log('▸ Assistant:');
              const llmResult = completion({
                  modelId: llmModelId,
                  history,
                  stream: true
              });
              let assistantText = '';
              for await (const token of llmResult.tokenStream) {
                  process.stdout.write(token);
                  assistantText += token;
              }
              process.stdout.write('\n');
              history.push({ role: 'assistant', content: assistantText });
              const spoken = assistantText.trim();
              if (spoken.length > 0) {
                  const ttsResult = textToSpeech({
                      modelId: ttsModelId,
                      text: spoken,
                      inputType: 'text',
                      stream: false
                  });
                  const samples = await ttsResult.buffer;
                  const audioData = int16ArrayToBuffer(samples);
                  const wavBuffer = Buffer.concat([
                      createWavHeader(audioData.length, TTS_SAMPLE_RATE),
                      audioData
                  ]);
                  playAudio(wavBuffer);
                  // Cooldown keeps the mic gated briefly so speaker tail / room reverb
                  // doesn't feed into the next VAD segment.
                  await sleep(POST_PLAYBACK_COOLDOWN_MS);
              }
          }
          catch (turnError) {
              console.error('\n✖ Turn failed:', turnError instanceof Error ? turnError.message : turnError);
          }
          finally {
              isSpeaking = false;
              console.log('\n▸ Listening...\n');
          }
      }
      ```
    </WrapCode>
  </Tab>

  <Tab value="ts" label="TypeScript">
    <WrapCode>
      ```ts file=<rootDir>/packages/sdk/examples/voice-assistant/voice-assistant.ts title="voice-assistant.ts" lineNumbers
      /**
       * Real-time Voice Assistant: mic → Whisper (with Silero VAD) → Llama → Supertonic TTS.
       *
       * Usage: npx tsx examples/voice-assistant/voice-assistant.ts
       *
       * Speak a question; the VAD detects when you pause, the utterance is
       * transcribed, sent to the LLM, and the response is spoken back. The loop
       * continues until you press Ctrl+C. While the assistant is speaking, mic
       * audio is dropped so it does not hear itself.
       *
       * Requirements: FFmpeg installed, microphone access, speakers.
       */
      import {
        loadModel,
        unloadModel,
        transcribeStream,
        completion,
        textToSpeech,
        WHISPER_TINY,
        VAD_SILERO_5_1_2,
        LLAMA_3_2_1B_INST_Q4_0,
        TTS_EN_SUPERTONIC_Q8_0
      } from '@qvac/sdk'
      import { spawnSync } from 'child_process'
      import { startMicrophone } from '../audio/mic-input.js'
      import { createWavHeader, int16ArrayToBuffer, playAudio } from '../tts/utils.js'

      const MIC_SAMPLE_RATE = 16000
      const TTS_SAMPLE_RATE = 44100

      const SYSTEM_PROMPT =
        'You are a concise, friendly voice assistant. Keep responses under two sentences. ' +
        'Never use markdown, lists, or code blocks — your output will be spoken aloud.'

      // VAD parameters tuned for conversational speech without the assistant looping
      // on its own echo. These defaults are deliberately conservative:
      //   - threshold 0.6: less sensitive than Silero's default; avoids triggering
      //     on TTS reverb bleeding into the mic or low-level background noise.
      //   - min_speech_duration_ms 300: drops short clicks/breaths and stray words.
      //   - min_silence_duration_ms 700: requires a longer quiet tail before
      //     committing a segment. Crucial for preventing self-hearing feedback loops
      //     where Whisper hallucinates content from near-silent audio.
      //   - max_speech_duration_s 15: caps runaway utterances.
      //   - speech_pad_ms 200: padding improves accuracy on utterance edges.
      // If the assistant cuts you off mid-sentence, raise min_silence_duration_ms.
      // If it keeps hallucinating / talking to itself, raise threshold to 0.7 and/or
      // min_silence_duration_ms to 900.
      const VAD_PARAMS = {
        threshold: 0.6,
        min_speech_duration_ms: 300,
        min_silence_duration_ms: 700,
        max_speech_duration_s: 15.0,
        speech_pad_ms: 200
      }

      // Short grace period after TTS playback before we start listening again.
      // Gives the speaker amp / room reverb a moment to fully settle so the first
      // post-playback mic frames don't get transcribed as the tail of our own voice.
      const POST_PLAYBACK_COOLDOWN_MS = 300

      // Minimum characters for an utterance to be considered meaningful. Whisper
      // frequently hallucinates single words like "you", ".", or "Thanks." from
      // silence or faint noise; these short phantoms are the main driver of the
      // self-hearing feedback loop, so we drop them.
      const MIN_UTTERANCE_CHARS = 3

      function isMeaningfulTranscript(text: string): boolean {
        const trimmed = text.trim()
        if (trimmed.length === 0) return false
        if (trimmed.includes('[No speech detected]')) return false
        // Whisper sometimes emits non-linguistic cues on silence, e.g. "[BLANK_AUDIO]".
        if (/^\[[^\]]+\]$/.test(trimmed)) return false
        // Strip punctuation/whitespace for the length check so ". . ." is rejected.
        const letters = trimmed.replace(/[^\p{L}\p{N}]/gu, '')
        if (letters.length < MIN_UTTERANCE_CHARS) return false
        return true
      }

      function sleep(ms: number) {
        return new Promise((resolve) => setTimeout(resolve, ms))
      }

      // ── Main ──

      for (const tool of ['ffmpeg', 'ffplay']) {
        const r = spawnSync(tool, ['-version'], { stdio: 'ignore' })
        if (r.error || r.status !== 0) {
          console.error(`✖ ${tool} not found on PATH. Install ffmpeg (ffplay ships with it) and retry.`)
          process.exit(1)
        }
      }

      console.log('▸ Loading whisper-tiny + Silero VAD...')
      const asrModelId = await loadModel({
        modelSrc: WHISPER_TINY,
        modelConfig: {
          vadModelSrc: VAD_SILERO_5_1_2,
          audio_format: 'f32le',
          strategy: 'greedy',
          n_threads: 4,
          language: 'en',
          no_timestamps: true,
          suppress_blank: true,
          suppress_nst: true,
          temperature: 0.0,
          vad_params: VAD_PARAMS
        }
      })

      console.log('▸ Loading Llama 3.2 1B...')
      const llmModelId = await loadModel({
        modelSrc: LLAMA_3_2_1B_INST_Q4_0,
        modelConfig: {
          ctx_size: 4096
        }
      })

      console.log('▸ Loading Supertonic TTS...')
      const ttsModelId = await loadModel({
        modelSrc: TTS_EN_SUPERTONIC_Q8_0,
        modelConfig: {
          ttsEngine: 'supertonic',
          language: 'en',
          voice: 'F1',
          ttsSpeed: 1.05,
          ttsNumInferenceSteps: 5
        }
      })

      console.log('▸ All models loaded.\n')

      const ffmpeg = startMicrophone({
        sampleRate: MIC_SAMPLE_RATE,
        format: 'f32le'
      })
      const session = await transcribeStream({ modelId: asrModelId })

      const history: Array<{
        role: 'system' | 'user' | 'assistant'
        content: string
      }> = [{ role: 'system', content: SYSTEM_PROMPT }]

      // Dropped-chunk gate: while the assistant is speaking we stop feeding the mic
      // stream into the ASR session. Using a flag (rather than pausing the ffmpeg
      // pipe) keeps the pipe drained so we never accumulate stale audio, and the
      // VAD starts fresh on the next user turn.
      let isSpeaking = false

      ffmpeg.stdout.on('data', (chunk: Buffer) => {
        if (isSpeaking) return
        session.write(chunk)
      })

      let shuttingDown = false
      async function cleanup() {
        if (shuttingDown) return
        shuttingDown = true
        console.log('\n\n▸ Stopping...')
        ffmpeg.kill()
        try {
          session.end()
        } catch {
          // session may already be closed
        }
        await unloadModel({ modelId: ttsModelId }).catch(() => {})
        await unloadModel({ modelId: llmModelId }).catch(() => {})
        await unloadModel({ modelId: asrModelId }).catch(() => {})
        console.log('▸ Done.')
        process.exit(0)
      }

      process.on('SIGINT', () => void cleanup())
      process.on('SIGTERM', () => void cleanup())

      console.log('▸ Listening. Speak a question and pause. Ctrl+C to quit.\n')

      for await (const rawText of session) {
        if (!isMeaningfulTranscript(rawText)) continue
        const userText = rawText.trim()

        console.log(`▸ You: ${userText}`)
        history.push({ role: 'user', content: userText })

        isSpeaking = true
        try {
          console.log('▸ Assistant:')
          const llmResult = completion({
            modelId: llmModelId,
            history,
            stream: true
          })
          let assistantText = ''
          for await (const token of llmResult.tokenStream) {
            process.stdout.write(token)
            assistantText += token
          }
          process.stdout.write('\n')
          history.push({ role: 'assistant', content: assistantText })

          const spoken = assistantText.trim()
          if (spoken.length > 0) {
            const ttsResult = textToSpeech({
              modelId: ttsModelId,
              text: spoken,
              inputType: 'text',
              stream: false
            })
            const samples = await ttsResult.buffer
            const audioData = int16ArrayToBuffer(samples)
            const wavBuffer = Buffer.concat([
              createWavHeader(audioData.length, TTS_SAMPLE_RATE),
              audioData
            ])
            playAudio(wavBuffer)
            // Cooldown keeps the mic gated briefly so speaker tail / room reverb
            // doesn't feed into the next VAD segment.
            await sleep(POST_PLAYBACK_COOLDOWN_MS)
          }
        } catch (turnError) {
          console.error('\n✖ Turn failed:', turnError instanceof Error ? turnError.message : turnError)
        } finally {
          isSpeaking = false
          console.log('\n▸ Listening...\n')
        }
      }
      ```
    </WrapCode>
  </Tab>

  <Tab value="python" label="Python">
    <WrapCode>
      ```python file=<rootDir>/packages/sdk-python/examples/voice_assistant.py title="voice_assistant.py" lineNumbers
      """Python port of packages/sdk/examples/voice-assistant/voice-assistant.ts.

      Real-time Voice Assistant: mic → Whisper (with Silero VAD) → Llama →
      Supertonic TTS.

      Speak a question; the VAD detects when you pause, the utterance is
      transcribed, sent to the LLM, and the response is spoken back. The loop
      continues until you press Ctrl+C. While the assistant is speaking, mic audio
      is dropped so it does not hear itself.

      The microphone and playback helpers the TypeScript example imports from
      `audio/mic-input` and `tts/utils` are repeated inline. Playback runs in a
      thread so the mic pipe keeps draining while the assistant speaks, which the
      TypeScript example gets from Node's event loop.

      Requirements: FFmpeg installed, microphone access, speakers.

      RUN: python examples/voice_assistant.py
      """

      from __future__ import annotations

      import asyncio
      import os
      import platform
      import re
      import shutil
      import struct
      import subprocess
      import sys

      from tetherto.qvac_sdk import (
          Client,
          TextToSpeechRequest,
          completion,
          load_model,
          text_to_speech,
          transcribe_stream_session,
          unload_model,
      )
      from tetherto.qvac_sdk.models import (
          LLAMA_3_2_1B_INST_Q4_0,
          TTS_EN_SUPERTONIC_Q8_0,
          VAD_SILERO_5_1_2,
          WHISPER_TINY,
      )

      MIC_SAMPLE_RATE = 16000
      TTS_SAMPLE_RATE = 44100

      SYSTEM_PROMPT = (
          "You are a concise, friendly voice assistant. Keep responses under two "
          "sentences. Never use markdown, lists, or code blocks — your output will "
          "be spoken aloud."
      )

      # VAD parameters tuned for conversational speech without the assistant looping
      # on its own echo. These defaults are deliberately conservative:
      #   - threshold 0.6: less sensitive than Silero's default; avoids triggering
      #     on TTS reverb bleeding into the mic or low-level background noise.
      #   - min_speech_duration_ms 300: drops short clicks/breaths and stray words.
      #   - min_silence_duration_ms 700: requires a longer quiet tail before
      #     committing a segment. Crucial for preventing self-hearing feedback loops
      #     where Whisper hallucinates content from near-silent audio.
      #   - max_speech_duration_s 15: caps runaway utterances.
      #   - speech_pad_ms 200: padding improves accuracy on utterance edges.
      # If the assistant cuts you off mid-sentence, raise min_silence_duration_ms.
      # If it keeps hallucinating / talking to itself, raise threshold to 0.7 and/or
      # min_silence_duration_ms to 900.
      VAD_PARAMS = {
          "threshold": 0.6,
          "min_speech_duration_ms": 300,
          "min_silence_duration_ms": 700,
          "max_speech_duration_s": 15.0,
          "speech_pad_ms": 200,
      }

      # Short grace period after TTS playback before we start listening again.
      # Gives the speaker amp / room reverb a moment to fully settle so the first
      # post-playback mic frames don't get transcribed as the tail of our own voice.
      POST_PLAYBACK_COOLDOWN_MS = 300

      # Minimum characters for an utterance to be considered meaningful. Whisper
      # frequently hallucinates single words like "you", ".", or "Thanks." from
      # silence or faint noise; these short phantoms are the main driver of the
      # self-hearing feedback loop, so we drop them.
      MIN_UTTERANCE_CHARS = 3


      def is_meaningful_transcript(text: str) -> bool:
          trimmed = text.strip()
          if len(trimmed) == 0:
              return False
          if "[No speech detected]" in trimmed:
              return False
          # Whisper sometimes emits non-linguistic cues on silence, e.g. "[BLANK_AUDIO]".
          if re.fullmatch(r"\[[^\]]+\]", trimmed):
              return False
          # Strip punctuation/whitespace for the length check so ". . ." is rejected.
          letters = re.sub(r"[\W_]", "", trimmed)
          return len(letters) >= MIN_UTTERANCE_CHARS


      # ── Microphone (audio/mic-input.ts) ──


      def list_windows_audio_devices() -> list[str]:
          # ffmpeg lists DirectShow devices on stderr and always exits non-zero
          # because `-i dummy` is not a real input.
          result = subprocess.run(
              [
                  "ffmpeg",
                  "-hide_banner",
                  "-f",
                  "dshow",
                  "-list_devices",
                  "true",
                  "-i",
                  "dummy",
              ],
              capture_output=True,
              text=True,
              check=False,
          )
          devices = []
          section = None
          for line in result.stderr.splitlines():
              # Old ffmpeg (≤ 4.3) prints section headers; new ffmpeg (≥ 4.4) tags
              # each device line with "(audio)" or "(video)".
              if "DirectShow audio devices" in line:
                  section = "audio"
                  continue
              if "DirectShow video devices" in line:
                  section = "video"
                  continue
              line_section = section
              if "(audio)" in line:
                  line_section = "audio"
              elif "(video)" in line:
                  line_section = "video"
              if line_section != "audio":
                  continue
              # First quoted token is the device name. Alternative-name lines start
              # with `@device_...` and are filtered out.
              match = re.search(r'"([^"]+)"', line)
              if not match or match[1].startswith("@device"):
                  continue
              devices.append(match[1])
          return devices


      def get_audio_input_args() -> list[str]:
          override = os.environ.get("MIC_DEVICE")
          system = platform.system()
          if system == "Darwin":
              # AVFoundation device spec is `<video>:<audio>`. `:0` = default audio.
              return ["-f", "avfoundation", "-i", override or ":0"]
          if system == "Linux":
              # PulseAudio source name. `default` follows the system default mic.
              return ["-f", "pulse", "-i", override or "default"]
          if system == "Windows":
              if override:
                  return ["-f", "dshow", "-i", f"audio={override}"]
              devices = list_windows_audio_devices()
              if not devices:
                  raise RuntimeError(
                      "No Windows audio input device found. List devices with:\n"
                      "  ffmpeg -hide_banner -f dshow -list_devices true -i dummy\n"
                      'Then set MIC_DEVICE="Your Device Name" and retry.'
                  )
              return ["-f", "dshow", "-i", f"audio={devices[0]}"]
          raise RuntimeError(f"Unsupported platform: {system}")


      async def start_microphone(sample_rate: int) -> asyncio.subprocess.Process:
          """Capture mono f32le audio from the default mic through ffmpeg."""
          return await asyncio.create_subprocess_exec(
              "ffmpeg",
              *get_audio_input_args(),
              "-ar",
              str(sample_rate),
              "-ac",
              "1",
              "-sample_fmt",
              "flt",
              "-f",
              "f32le",
              "pipe:1",
              stdin=subprocess.DEVNULL,
              stdout=subprocess.PIPE,
              stderr=subprocess.DEVNULL,
          )


      # ── Playback (tts/utils.ts) ──


      def create_wav_header(data_length: int, sample_rate: int) -> bytes:
          """Create a WAV header for 16-bit mono PCM audio."""
          return (
              b"RIFF"
              + struct.pack("<I", 36 + data_length)
              + b"WAVE"
              + b"fmt "
              + struct.pack("<IHHIIHH", 16, 1, 1, sample_rate, sample_rate * 2, 2, 16)
              + b"data"
              + struct.pack("<I", data_length)
          )


      def int16_array_to_buffer(samples) -> bytes:
          """Convert int16 PCM samples to little-endian bytes."""
          return struct.pack(
              f"<{len(samples)}h",
              *(max(-32768, min(32767, round(s))) for s in samples),
          )


      def play_audio(audio_buffer: bytes) -> None:
          """Play a WAV buffer by streaming it into ffplay over stdin."""
          try:
              result = subprocess.run(
                  [
                      "ffplay",
                      "-hide_banner",
                      "-loglevel",
                      "error",
                      "-autoexit",
                      "-nodisp",
                      "-i",
                      "pipe:0",
                  ],
                  input=audio_buffer,
                  check=False,
              )
          except FileNotFoundError:
              raise RuntimeError(
                  "ffplay not found on PATH. Install ffmpeg (ffplay ships with it) and retry."
              ) from None
          if result.returncode != 0:
              raise RuntimeError(f"ffplay exited with code {result.returncode}")


      # ── Main ──


      async def main() -> int:
          for tool in ("ffmpeg", "ffplay"):
              if shutil.which(tool) is None:
                  print(
                      f"✖ {tool} not found on PATH. Install ffmpeg (ffplay ships with it) and retry.",
                      file=sys.stderr,
                  )
                  return 1

          async with Client() as client:
              t = client.transport
              model_ids: list[str] = []
              ffmpeg = None
              try:
                  print("▸ Loading whisper-tiny + Silero VAD...")
                  asr_model_id = await load_model(
                      t,
                      model_src=WHISPER_TINY,
                      model_config={
                          "vadModelSrc": VAD_SILERO_5_1_2,
                          "audio_format": "f32le",
                          "strategy": "greedy",
                          "n_threads": 4,
                          "language": "en",
                          "no_timestamps": True,
                          "suppress_blank": True,
                          "suppress_nst": True,
                          "temperature": 0.0,
                          "vad_params": VAD_PARAMS,
                      },
                  )
                  model_ids.append(asr_model_id)

                  print("▸ Loading Llama 3.2 1B...")
                  llm_model_id = await load_model(
                      t,
                      model_src=LLAMA_3_2_1B_INST_Q4_0,
                      model_config={"ctx_size": 4096},
                  )
                  model_ids.append(llm_model_id)

                  print("▸ Loading Supertonic TTS...")
                  tts_model_id = await load_model(
                      t,
                      model_src=TTS_EN_SUPERTONIC_Q8_0,
                      model_config={
                          "ttsEngine": "supertonic",
                          "language": "en",
                          "voice": "F1",
                          "ttsSpeed": 1.05,
                          "ttsNumInferenceSteps": 5,
                      },
                  )
                  model_ids.append(tts_model_id)

                  print("▸ All models loaded.\n")

                  ffmpeg = await start_microphone(MIC_SAMPLE_RATE)
                  session = transcribe_stream_session(t, model_id=asr_model_id)

                  history = [{"role": "system", "content": SYSTEM_PROMPT}]

                  # Dropped-chunk gate: while the assistant is speaking we stop
                  # feeding the mic stream into the ASR session. Using a flag (rather
                  # than pausing the ffmpeg pipe) keeps the pipe drained so we never
                  # accumulate stale audio, and the VAD starts fresh on the next turn.
                  is_speaking = False

                  async def pump_mic() -> None:
                      while chunk := await ffmpeg.stdout.read(4096):
                          if not is_speaking:
                              session.write(chunk)

                  pump = asyncio.create_task(pump_mic())

                  print("▸ Listening. Speak a question and pause. Ctrl+C to quit.\n")

                  async for raw_text in session:
                      if not is_meaningful_transcript(raw_text):
                          continue
                      user_text = raw_text.strip()

                      print(f"▸ You: {user_text}")
                      history.append({"role": "user", "content": user_text})

                      is_speaking = True
                      try:
                          print("▸ Assistant:")
                          run = completion(t, model_id=llm_model_id, history=history)
                          assistant_text = ""
                          async for event in run.events:
                              if event.type == "contentDelta":
                                  sys.stdout.write(event.text)
                                  sys.stdout.flush()
                                  assistant_text += event.text
                          sys.stdout.write("\n")
                          history.append({"role": "assistant", "content": assistant_text})

                          spoken = assistant_text.strip()
                          if spoken:
                              request = TextToSpeechRequest.model_validate(
                                  {
                                      "type": "textToSpeech",
                                      "modelId": tts_model_id,
                                      "text": spoken,
                                      "inputType": "text",
                                      "stream": False,
                                  }
                              )
                              samples: list[float] = []
                              async for response in text_to_speech(t, request):
                                  samples.extend(response.buffer)
                              audio_data = int16_array_to_buffer(samples)
                              wav_buffer = (
                                  create_wav_header(len(audio_data), TTS_SAMPLE_RATE)
                                  + audio_data
                              )
                              await asyncio.to_thread(play_audio, wav_buffer)
                              # Cooldown keeps the mic gated briefly so speaker tail /
                              # room reverb doesn't feed into the next VAD segment.
                              await asyncio.sleep(POST_PLAYBACK_COOLDOWN_MS / 1000)
                      except Exception as turn_error:
                          print(f"\n✖ Turn failed: {turn_error}", file=sys.stderr)
                      finally:
                          is_speaking = False
                          print("\n▸ Listening...\n")

                  pump.cancel()
              except asyncio.CancelledError:
                  pass
              except Exception as error:
                  print(f"✖ {error}", file=sys.stderr)
                  return 1
              finally:
                  print("\n\n▸ Stopping...")
                  if ffmpeg is not None and ffmpeg.returncode is None:
                      ffmpeg.kill()
                  for model_id in reversed(model_ids):
                      try:
                          await unload_model(t, model_id)
                      except Exception:
                          pass
                  print("▸ Done.")
          return 0


      if __name__ == "__main__":
          try:
              sys.exit(asyncio.run(main()))
          except KeyboardInterrupt:
              sys.exit(0)
      ```
    </WrapCode>
  </Tab>

  <Tab value="kotlin" label="Kotlin">
    <WrapCode>
      ```kotlin file=<rootDir>/packages/sdk-kotlin/examples/src/VoiceAssistant.kt title="VoiceAssistant.kt" lineNumbers
      // Kotlin port of packages/sdk/examples/voice-assistant/voice-assistant.ts.
      //
      // Real-time Voice Assistant: mic → Whisper (with Silero VAD) → Llama →
      // Supertonic TTS.
      //
      // Speak a question; the VAD detects when you pause, the utterance is
      // transcribed, sent to the LLM, and the response is spoken back. The loop
      // continues until you press Ctrl+C. While the assistant is speaking, mic audio
      // is dropped so it does not hear itself.
      //
      // The microphone and playback helpers the TypeScript example imports from
      // `audio/mic-input` and `tts/utils` are repeated inline, on the JDK's
      // `javax.sound.sampled` (a `TargetDataLine` for the mic, a `SourceDataLine` for
      // the speakers) instead of the ffmpeg/ffplay subprocesses that JS and Python
      // spawn, so no external tool is needed. Set `MIC_DEVICE` to part of a mixer
      // name to pick a microphone other than the system default.
      //
      // Kotlin notes: the microphone is a cold `Flow<ByteArray>` handed straight to
      // the duplex `client.speech.transcribeStream()`; it reads on `Dispatchers.IO`
      // and keeps draining the mic while a turn runs, which JS gets from Node's event
      // loop and Python from a pump task. Ctrl+C ends the JVM: the transport's
      // shutdown hook then stops the worker, which releases the loaded models.
      //
      // Requirements: microphone access, speakers.
      //
      // RUN: ./gradlew -p examples run -Pexample=VoiceAssistant

      import io.tether.qvac.sdk.QvacClient
      import io.tether.qvac.sdk.QvacCompletionEvent
      import io.tether.qvac.sdk.QvacMessage
      import io.tether.qvac.sdk.completion
      import io.tether.qvac.sdk.generated.ModelConstant
      import io.tether.qvac.sdk.generated.Models
      import io.tether.qvac.sdk.generated.schema.TranscribeStreamRequest
      import io.tether.qvac.sdk.jvm.JvmBareKitTransport
      import io.tether.qvac.sdk.models
      import io.tether.qvac.sdk.run
      import io.tether.qvac.sdk.speech
      import java.nio.ByteBuffer
      import java.nio.ByteOrder
      import java.util.concurrent.atomic.AtomicBoolean
      import javax.sound.sampled.AudioFormat
      import javax.sound.sampled.AudioSystem
      import javax.sound.sampled.DataLine
      import javax.sound.sampled.TargetDataLine
      import kotlin.math.roundToInt
      import kotlinx.coroutines.CancellationException
      import kotlinx.coroutines.Dispatchers
      import kotlinx.coroutines.currentCoroutineContext
      import kotlinx.coroutines.delay
      import kotlinx.coroutines.flow.Flow
      import kotlinx.coroutines.flow.flow
      import kotlinx.coroutines.flow.flowOn
      import kotlinx.coroutines.isActive
      import kotlinx.coroutines.runBlocking
      import kotlinx.coroutines.withContext
      import kotlinx.serialization.json.JsonObject
      import kotlinx.serialization.json.buildJsonObject
      import kotlinx.serialization.json.put

      private const val MIC_SAMPLE_RATE = 16000
      private const val TTS_SAMPLE_RATE = 44100

      private const val SYSTEM_PROMPT =
          "You are a concise, friendly voice assistant. Keep responses under two " +
              "sentences. Never use markdown, lists, or code blocks — your output will " +
              "be spoken aloud."

      // VAD parameters tuned for conversational speech without the assistant looping
      // on its own echo. These defaults are deliberately conservative:
      //   - threshold 0.6: less sensitive than Silero's default; avoids triggering
      //     on TTS reverb bleeding into the mic or low-level background noise.
      //   - min_speech_duration_ms 300: drops short clicks/breaths and stray words.
      //   - min_silence_duration_ms 700: requires a longer quiet tail before
      //     committing a segment. Crucial for preventing self-hearing feedback loops
      //     where Whisper hallucinates content from near-silent audio.
      //   - max_speech_duration_s 15: caps runaway utterances.
      //   - speech_pad_ms 200: padding improves accuracy on utterance edges.
      // If the assistant cuts you off mid-sentence, raise min_silence_duration_ms.
      // If it keeps hallucinating / talking to itself, raise threshold to 0.7 and/or
      // min_silence_duration_ms to 900.
      private val VAD_PARAMS = buildJsonObject {
          put("threshold", 0.6)
          put("min_speech_duration_ms", 300)
          put("min_silence_duration_ms", 700)
          put("max_speech_duration_s", 15.0)
          put("speech_pad_ms", 200)
      }

      // Short grace period after TTS playback before we start listening again.
      // Gives the speaker amp / room reverb a moment to fully settle so the first
      // post-playback mic frames don't get transcribed as the tail of our own voice.
      private const val POST_PLAYBACK_COOLDOWN_MS = 300L

      // Minimum characters for an utterance to be considered meaningful. Whisper
      // frequently hallucinates single words like "you", ".", or "Thanks." from
      // silence or faint noise; these short phantoms are the main driver of the
      // self-hearing feedback loop, so we drop them.
      private const val MIN_UTTERANCE_CHARS = 3

      private fun isMeaningfulTranscript(text: String): Boolean {
          val trimmed = text.trim()
          if (trimmed.isEmpty()) return false
          if ("[No speech detected]" in trimmed) return false
          // Whisper sometimes emits non-linguistic cues on silence, e.g. "[BLANK_AUDIO]".
          if (Regex("""\[[^\]]+\]""").matches(trimmed)) return false
          // Strip punctuation/whitespace for the length check so ". . ." is rejected.
          val letters = trimmed.replace(Regex("""(?U)[\W_]"""), "")
          return letters.length >= MIN_UTTERANCE_CHARS
      }

      // ── Microphone (audio/mic-input.ts) ──

      /** Open the `MIC_DEVICE` mixer's input line when set, else the system default microphone. */
      private fun openMicrophone(format: AudioFormat): TargetDataLine {
          val info = DataLine.Info(TargetDataLine::class.java, format)
          val override = System.getenv("MIC_DEVICE")
              ?: return AudioSystem.getLine(info) as TargetDataLine
          val mixer = AudioSystem.getMixerInfo()
              .map(AudioSystem::getMixer)
              .firstOrNull { override in it.mixerInfo.name && it.isLineSupported(info) }
              ?: error("No audio input device matches MIC_DEVICE=\"$override\"")
          return mixer.getLine(info) as TargetDataLine
      }

      /**
       * Capture mono audio from the mic as f32le chunks. Samples are read as 16-bit
       * PCM (which every Java Sound mixer supports) and converted to the `f32le`
       * format the Whisper model is loaded with. Chunks read while [muted] returns
       * true are dropped, so the line keeps draining.
       */
      private fun microphone(sampleRate: Int, muted: () -> Boolean): Flow<ByteArray> = flow {
          val line = openMicrophone(AudioFormat(sampleRate.toFloat(), 16, 1, true, false))
          line.open()
          try {
              line.start()
              val buffer = ByteArray(4096)
              while (currentCoroutineContext().isActive) {
                  val count = line.read(buffer, 0, buffer.size)
                  if (count > 0 && !muted()) emit(s16leToF32le(buffer, count))
              }
          } finally {
              line.close()
          }
      }.flowOn(Dispatchers.IO)

      private fun s16leToF32le(pcm: ByteArray, length: Int): ByteArray {
          val input = ByteBuffer.wrap(pcm, 0, length).order(ByteOrder.LITTLE_ENDIAN)
          val output = ByteBuffer.allocate(length / 2 * 4).order(ByteOrder.LITTLE_ENDIAN)
          while (input.remaining() >= 2) output.putFloat(input.getShort() / 32768f)
          return output.array()
      }

      // ── Playback (tts/utils.ts) ──

      /** Convert int16 PCM samples to little-endian bytes. */
      private fun int16ArrayToBuffer(samples: List<Double>): ByteArray {
          val buffer = ByteBuffer.allocate(samples.size * 2).order(ByteOrder.LITTLE_ENDIAN)
          for (s in samples) buffer.putShort(s.roundToInt().coerceIn(-32768, 32767).toShort())
          return buffer.array()
      }

      /** Play 16-bit mono PCM on the default speakers and block until it has been heard. */
      private fun playAudio(pcm: ByteArray, sampleRate: Int) {
          val line = AudioSystem.getSourceDataLine(AudioFormat(sampleRate.toFloat(), 16, 1, true, false))
          line.open()
          try {
              line.start()
              line.write(pcm, 0, pcm.size)
              line.drain()
          } finally {
              line.close()
          }
      }

      // ── Main ──

      fun main(): Unit = runBlocking {
          val client = QvacClient(JvmBareKitTransport.connect())
          val modelIds = mutableListOf<String>()
          try {
              println("▸ Loading whisper-tiny + Silero VAD...")
              val asrModelId = client.loadModel(
                  Models.WHISPER_TINY,
                  buildJsonObject {
                      put("vadModelSrc", Models.VAD_SILERO_5_1_2.src)
                      put("audio_format", "f32le")
                      put("strategy", "greedy")
                      put("n_threads", 4)
                      put("language", "en")
                      put("no_timestamps", true)
                      put("suppress_blank", true)
                      put("suppress_nst", true)
                      put("temperature", 0.0)
                      put("vad_params", VAD_PARAMS)
                  },
              )
              modelIds += asrModelId

              println("▸ Loading Llama 3.2 1B...")
              val llmModelId = client.loadModel(
                  Models.LLAMA_3_2_1B_INST_Q4_0,
                  buildJsonObject { put("ctx_size", 4096) },
              )
              modelIds += llmModelId

              println("▸ Loading Supertonic TTS...")
              val ttsModelId = client.loadModel(
                  Models.TTS_EN_SUPERTONIC_Q8_0,
                  buildJsonObject {
                      put("ttsEngine", "supertonic")
                      put("language", "en")
                      put("voice", "F1")
                      put("ttsSpeed", 1.05)
                      put("ttsNumInferenceSteps", 5)
                  },
              )
              modelIds += ttsModelId

              println("▸ All models loaded.\n")

              val history = mutableListOf(QvacMessage.system(SYSTEM_PROMPT))

              // Dropped-chunk gate: while the assistant is speaking we stop feeding
              // the mic stream into the ASR session. Using a flag (rather than
              // stopping the mic line) keeps the line drained so we never accumulate
              // stale audio, and the VAD starts fresh on the next turn.
              val isSpeaking = AtomicBoolean(false)
              val mic = microphone(MIC_SAMPLE_RATE) { isSpeaking.get() }

              println("▸ Listening. Speak a question and pause. Ctrl+C to quit.\n")

              client.speech.transcribeStream(TranscribeStreamRequest(modelId = asrModelId), mic).collect { response ->
                  response.error?.let { error(it) }
                  val rawText = response.text ?: return@collect
                  if (!isMeaningfulTranscript(rawText)) return@collect
                  val userText = rawText.trim()

                  println("▸ You: $userText")
                  history += QvacMessage.user(userText)

                  isSpeaking.set(true)
                  try {
                      println("▸ Assistant:")
                      val run = client.completion.run(modelId = llmModelId, history = history.toList())
                      val assistantText = StringBuilder()
                      run.events.collect { event ->
                          if (event is QvacCompletionEvent.ContentDelta) {
                              print(event.text)
                              System.out.flush()
                              assistantText.append(event.text)
                          }
                      }
                      println()
                      history += QvacMessage.assistant(assistantText.toString())

                      val spoken = assistantText.trim().toString()
                      if (spoken.isNotEmpty()) {
                          val speech = client.speech.synthesize(ttsModelId, spoken)
                          val audioData = int16ArrayToBuffer(speech.samples)
                          withContext(Dispatchers.IO) { playAudio(audioData, TTS_SAMPLE_RATE) }
                          // Cooldown keeps the mic gated briefly so speaker tail /
                          // room reverb doesn't feed into the next VAD segment.
                          delay(POST_PLAYBACK_COOLDOWN_MS)
                      }
                  } catch (cancelled: CancellationException) {
                      throw cancelled
                  } catch (turnError: Exception) {
                      System.err.println("\n✖ Turn failed: ${turnError.message}")
                  } finally {
                      isSpeaking.set(false)
                      println("\n▸ Listening...\n")
                  }
              }
          } finally {
              println("\n\n▸ Stopping...")
              for (modelId in modelIds.asReversed()) {
                  runCatching { client.models.unload(modelId) }
              }
              client.close()
              println("▸ Done.")
          }
      }

      /** Load [model] with [modelConfig] and return its model id. */
      private suspend fun QvacClient.loadModel(model: ModelConstant, modelConfig: JsonObject): String {
          val loaded = models.load(model.src, model.engine, model.name, modelConfig = modelConfig)
          return loaded.modelId ?: error(loaded.error ?: "loadModel failed")
      }
      ```
    </WrapCode>
  </Tab>
</Tabs>

Speak into the mic; transcriptions and the assistant's spoken responses will follow. Press `Ctrl+C` to quit. Models are downloaded on first run (\~1 GB total) and cached locally; subsequent runs work fully offline.

### Helpers

The JS and TS versions import two helpers: `mic-input` from `../audio/` and `utils` from `../tts/`, relative to the script. The Python and Kotlin examples include them inline.

#### mic-input

Captures microphone audio through `ffmpeg`:

<Tabs>
  <Tab value="js" label="JavaScript" default>
    <WrapCode>
      ```js file=<rootDir>/packages/sdk/dist/examples/audio/mic-input.js title="mic-input.js" lineNumbers
      /**
       * Shared microphone-capture helpers for the transcription and
       * voice-assistant examples. Wraps ffmpeg so the examples don't each
       * re-implement device selection.
       *
       * The default device is the system default mic. Override on any OS
       * with the MIC_DEVICE environment variable. On Windows, if no override
       * is set, the first DirectShow audio device reported by ffmpeg is used.
       */
      import { spawn, spawnSync } from 'child_process';
      import { platform } from 'os';
      // Parses the DirectShow audio device list printed by:
      //   ffmpeg -hide_banner -f dshow -list_devices true -i dummy
      // Output goes to stderr; ffmpeg always exits non-zero because `-i dummy`
      // isn't a real device — that's expected.
      //
      // Handles both ffmpeg output formats:
      //
      // Old (ffmpeg ≤ 4.3):
      //   [dshow @ ...] DirectShow audio devices
      //   [dshow @ ...]  "Microphone (Realtek Audio)"
      //   [dshow @ ...]     Alternative name "@device_cm_..."
      //
      // New (ffmpeg ≥ 4.4):
      //   [dshow @ ...] "Integrated Camera" (video)
      //   [dshow @ ...]   Alternative name "@device_pnp_..."
      //   [dshow @ ...] "Microphone (Realtek Audio)" (audio)
      //   [dshow @ ...]   Alternative name "@device_cm_..."
      //
      // The newer format dropped the "DirectShow audio devices" header and tags
      // each device line with `(audio)` or `(video)` instead.
      export function listWindowsAudioDevices() {
          const result = spawnSync('ffmpeg', ['-hide_banner', '-f', 'dshow', '-list_devices', 'true', '-i', 'dummy'], { encoding: 'utf8' });
          const stderr = result.stderr ?? '';
          const devices = [];
          let section = null;
          for (const line of stderr.split(/\r?\n/)) {
              // Old-format section headers.
              if (line.includes('DirectShow audio devices')) {
                  section = 'audio';
                  continue;
              }
              if (line.includes('DirectShow video devices')) {
                  section = 'video';
                  continue;
              }
              // New-format per-line tag wins over the section state.
              const hasAudioTag = /\(audio\)/.test(line);
              const hasVideoTag = /\(video\)/.test(line);
              const lineSection = hasAudioTag ? 'audio' : hasVideoTag ? 'video' : section;
              if (lineSection !== 'audio')
                  continue;
              // First quoted token is the device name. Alternative-name lines start
              // with `@device_...` and are filtered out.
              const match = line.match(/"([^"]+)"/);
              if (!match)
                  continue;
              const name = match[1];
              if (!name || name.startsWith('@device'))
                  continue;
              devices.push(name);
          }
          return devices;
      }
      export function getAudioInputArgs() {
          const env = process.env;
          const override = env['MIC_DEVICE'];
          switch (platform()) {
              case 'darwin':
                  // AVFoundation device spec is `<video>:<audio>`. `:0` = default audio.
                  // Override with e.g. MIC_DEVICE=":1" to pick another device, or run
                  // `ffmpeg -f avfoundation -list_devices true -i ""` to enumerate.
                  return ['-f', 'avfoundation', '-i', override ?? ':0'];
              case 'linux':
                  // PulseAudio source name. `default` follows the system default mic.
                  // Override with MIC_DEVICE="alsa_input.pci-..." etc.; list with
                  // `pactl list short sources`.
                  return ['-f', 'pulse', '-i', override ?? 'default'];
              case 'win32': {
                  if (override)
                      return ['-f', 'dshow', '-i', `audio=${override}`];
                  const devices = listWindowsAudioDevices();
                  const deviceName = devices[0];
                  if (!deviceName) {
                      throw new Error('No Windows audio input device found. List devices with:\n' +
                          '  ffmpeg -hide_banner -f dshow -list_devices true -i dummy\n' +
                          'Then set MIC_DEVICE="Your Device Name" and retry.');
                  }
                  return ['-f', 'dshow', '-i', `audio=${deviceName}`];
              }
              default:
                  throw new Error(`Unsupported platform: ${platform()}`);
          }
      }
      export function startMicrophone(options) {
          const formatArgs = options.format === 'f32le'
              ? ['-sample_fmt', 'flt', '-f', 'f32le']
              : ['-sample_fmt', 's16', '-f', 's16le'];
          const ffmpeg = spawn('ffmpeg', [
              ...getAudioInputArgs(),
              '-ar',
              String(options.sampleRate),
              '-ac',
              '1',
              ...formatArgs,
              'pipe:1'
          ], { stdio: ['ignore', 'pipe', 'ignore'] });
          if (!ffmpeg.stdout)
              throw new Error('Failed to open microphone');
          return ffmpeg;
      }
      ```
    </WrapCode>
  </Tab>

  <Tab value="ts" label="TypeScript">
    <WrapCode>
      ```ts file=<rootDir>/packages/sdk/examples/audio/mic-input.ts title="mic-input.ts" lineNumbers
      /**
       * Shared microphone-capture helpers for the transcription and
       * voice-assistant examples. Wraps ffmpeg so the examples don't each
       * re-implement device selection.
       *
       * The default device is the system default mic. Override on any OS
       * with the MIC_DEVICE environment variable. On Windows, if no override
       * is set, the first DirectShow audio device reported by ffmpeg is used.
       */
      import { spawn, spawnSync } from 'child_process'
      import { platform } from 'os'

      export type MicFormat = 'f32le' | 's16le'

      // Parses the DirectShow audio device list printed by:
      //   ffmpeg -hide_banner -f dshow -list_devices true -i dummy
      // Output goes to stderr; ffmpeg always exits non-zero because `-i dummy`
      // isn't a real device — that's expected.
      //
      // Handles both ffmpeg output formats:
      //
      // Old (ffmpeg ≤ 4.3):
      //   [dshow @ ...] DirectShow audio devices
      //   [dshow @ ...]  "Microphone (Realtek Audio)"
      //   [dshow @ ...]     Alternative name "@device_cm_..."
      //
      // New (ffmpeg ≥ 4.4):
      //   [dshow @ ...] "Integrated Camera" (video)
      //   [dshow @ ...]   Alternative name "@device_pnp_..."
      //   [dshow @ ...] "Microphone (Realtek Audio)" (audio)
      //   [dshow @ ...]   Alternative name "@device_cm_..."
      //
      // The newer format dropped the "DirectShow audio devices" header and tags
      // each device line with `(audio)` or `(video)` instead.
      export function listWindowsAudioDevices() {
        const result = spawnSync(
          'ffmpeg',
          ['-hide_banner', '-f', 'dshow', '-list_devices', 'true', '-i', 'dummy'],
          { encoding: 'utf8' }
        )
        const stderr = result.stderr ?? ''
        const devices: string[] = []
        let section: 'audio' | 'video' | null = null

        for (const line of stderr.split(/\r?\n/)) {
          // Old-format section headers.
          if (line.includes('DirectShow audio devices')) {
            section = 'audio'
            continue
          }
          if (line.includes('DirectShow video devices')) {
            section = 'video'
            continue
          }

          // New-format per-line tag wins over the section state.
          const hasAudioTag = /\(audio\)/.test(line)
          const hasVideoTag = /\(video\)/.test(line)
          const lineSection = hasAudioTag ? 'audio' : hasVideoTag ? 'video' : section
          if (lineSection !== 'audio') continue

          // First quoted token is the device name. Alternative-name lines start
          // with `@device_...` and are filtered out.
          const match = line.match(/"([^"]+)"/)
          if (!match) continue
          const name = match[1]
          if (!name || name.startsWith('@device')) continue

          devices.push(name)
        }

        return devices
      }

      export function getAudioInputArgs() {
        const env = process.env as Record<string, string | undefined>
        const override = env['MIC_DEVICE']
        switch (platform()) {
          case 'darwin':
            // AVFoundation device spec is `<video>:<audio>`. `:0` = default audio.
            // Override with e.g. MIC_DEVICE=":1" to pick another device, or run
            // `ffmpeg -f avfoundation -list_devices true -i ""` to enumerate.
            return ['-f', 'avfoundation', '-i', override ?? ':0']
          case 'linux':
            // PulseAudio source name. `default` follows the system default mic.
            // Override with MIC_DEVICE="alsa_input.pci-..." etc.; list with
            // `pactl list short sources`.
            return ['-f', 'pulse', '-i', override ?? 'default']
          case 'win32': {
            if (override) return ['-f', 'dshow', '-i', `audio=${override}`]
            const devices = listWindowsAudioDevices()
            const deviceName = devices[0]
            if (!deviceName) {
              throw new Error(
                'No Windows audio input device found. List devices with:\n' +
                  '  ffmpeg -hide_banner -f dshow -list_devices true -i dummy\n' +
                  'Then set MIC_DEVICE="Your Device Name" and retry.'
              )
            }
            return ['-f', 'dshow', '-i', `audio=${deviceName}`]
          }
          default:
            throw new Error(`Unsupported platform: ${platform()}`)
        }
      }

      export interface StartMicrophoneOptions {
        sampleRate: number
        format: MicFormat
      }

      export function startMicrophone(options: StartMicrophoneOptions) {
        const formatArgs =
          options.format === 'f32le'
            ? ['-sample_fmt', 'flt', '-f', 'f32le']
            : ['-sample_fmt', 's16', '-f', 's16le']
        const ffmpeg = spawn(
          'ffmpeg',
          [
            ...getAudioInputArgs(),
            '-ar',
            String(options.sampleRate),
            '-ac',
            '1',
            ...formatArgs,
            'pipe:1'
          ],
          { stdio: ['ignore', 'pipe', 'ignore'] }
        )
        if (!ffmpeg.stdout) throw new Error('Failed to open microphone')
        return ffmpeg
      }
      ```
    </WrapCode>
  </Tab>
</Tabs>

#### utils

Builds a WAV from the PCM samples and plays it through `ffplay`:

<Tabs>
  <Tab value="js" label="JavaScript" default>
    <WrapCode>
      ```js file=<rootDir>/packages/sdk/dist/examples/tts/utils.js title="utils.js" lineNumbers
      import { writeFileSync, unlinkSync } from 'fs';
      import { spawn, spawnSync } from 'child_process';
      import { platform, tmpdir } from 'os';
      import { join } from 'path';
      /**
       * Create WAV header for 16-bit PCM audio
       */
      export function createWavHeader(dataLength, sampleRate) {
          const header = Buffer.alloc(44);
          // RIFF header
          header.write('RIFF', 0);
          header.writeUInt32LE(36 + dataLength, 4);
          header.write('WAVE', 8);
          // fmt chunk
          header.write('fmt ', 12);
          header.writeUInt32LE(16, 16); // fmt chunk size
          header.writeUInt16LE(1, 20); // PCM format
          header.writeUInt16LE(1, 22); // mono
          header.writeUInt32LE(sampleRate, 24);
          header.writeUInt32LE(sampleRate * 2, 28); // byte rate
          header.writeUInt16LE(2, 32); // block align
          header.writeUInt16LE(16, 34); // bits per sample
          // data chunk
          header.write('data', 36);
          header.writeUInt32LE(dataLength, 40);
          return header;
      }
      /**
       * Convert Int16Array to Buffer
       */
      export function int16ArrayToBuffer(samples) {
          const buffer = Buffer.alloc(samples.length * 2);
          for (let i = 0; i < samples.length; i++) {
              const value = Math.max(-32768, Math.min(32767, Math.round(samples[i] ?? 0)));
              buffer.writeInt16LE(value, i * 2);
          }
          return buffer;
      }
      /**
       * Create and save WAV file
       */
      export function createWav(audioBuffer, sampleRate, filename) {
          const audioData = int16ArrayToBuffer(audioBuffer);
          const wavHeader = createWavHeader(audioData.length, sampleRate);
          const wavFile = Buffer.concat([wavHeader, audioData]);
          writeFileSync(filename, wavFile);
          console.log(`▸ WAV file saved as: ${filename}`);
      }
      /**
       * Play a WAV buffer by streaming it into ffplay over stdin.
       *
       * ffplay ships with ffmpeg and is cross-platform (macOS/Linux/Windows), so
       * we avoid the old "write to /tmp then shell out to afplay/aplay/powershell"
       * dance — no temp files, no platform switch, no hardcoded /tmp path (which
       * doesn't exist on Windows). Requires ffplay on PATH.
       */
      /**
       * Play one mono s16le PCM chunk (as a minimal WAV) and wait for the player to finish.
       * Chunks are played sequentially when awaited in order — suitable for streaming TTS output.
       */
      export function playPcmInt16Chunk(samples, sampleRate) {
          if (samples.length === 0) {
              return Promise.resolve();
          }
          const audioData = int16ArrayToBuffer(samples);
          const wavHeader = createWavHeader(audioData.length, sampleRate);
          const wavFile = Buffer.concat([wavHeader, audioData]);
          // `os.tmpdir()` resolves to the OS-specific temp directory (e.g. `%TEMP%`
          // on Windows), so the Windows branch below no longer tries to read a
          // POSIX-only `/tmp/...` path.
          const tempFile = join(tmpdir(), `qvac-tts-chunk-${Date.now()}-${Math.random().toString(16).slice(2)}.wav`);
          writeFileSync(tempFile, wavFile);
          const currentPlatform = platform();
          let audioPlayer;
          let args;
          switch (currentPlatform) {
              case 'darwin':
                  audioPlayer = 'afplay';
                  args = [tempFile];
                  break;
              case 'linux':
                  audioPlayer = 'aplay';
                  args = [tempFile];
                  break;
              case 'win32':
                  audioPlayer = 'powershell';
                  args = [
                      '-Command',
                      `Add-Type -AssemblyName presentationCore; (New-Object Media.SoundPlayer).LoadStream([System.IO.File]::ReadAllBytes('${tempFile}')).PlaySync()`
                  ];
                  break;
              default:
                  audioPlayer = 'aplay';
                  args = [tempFile];
          }
          return new Promise(function (resolve, reject) {
              const proc = spawn(audioPlayer, args, { stdio: 'ignore' });
              proc.on('error', function (err) {
                  try {
                      unlinkSync(tempFile);
                  }
                  catch {
                      // ignore
                  }
                  reject(err);
              });
              proc.on('close', function (code) {
                  try {
                      unlinkSync(tempFile);
                  }
                  catch {
                      // ignore
                  }
                  if (code === 0) {
                      resolve();
                  }
                  else {
                      reject(new Error(`Audio player exited with code ${code}`));
                  }
              });
          });
      }
      export function playAudio(audioBuffer) {
          const result = spawnSync('ffplay', ['-hide_banner', '-loglevel', 'error', '-autoexit', '-nodisp', '-i', 'pipe:0'], {
              input: audioBuffer,
              stdio: ['pipe', 'inherit', 'inherit']
          });
          if (result.error) {
              const code = result.error.code;
              if (code === 'ENOENT') {
                  throw new Error('ffplay not found on PATH. Install ffmpeg (ffplay ships with it) and retry.');
              }
              throw new Error(`ffplay failed: ${result.error.message}`);
          }
          if (result.status !== 0) {
              throw new Error(`ffplay exited with code ${result.status}`);
          }
      }
      ```
    </WrapCode>
  </Tab>

  <Tab value="ts" label="TypeScript">
    <WrapCode>
      ```ts file=<rootDir>/packages/sdk/examples/tts/utils.ts title="utils.ts" lineNumbers
      import { writeFileSync, unlinkSync } from 'fs'
      import { spawn, spawnSync } from 'child_process'
      import { platform, tmpdir } from 'os'
      import { join } from 'path'

      /**
       * Create WAV header for 16-bit PCM audio
       */
      export function createWavHeader(dataLength: number, sampleRate: number): Buffer {
        const header = Buffer.alloc(44)

        // RIFF header
        header.write('RIFF', 0)
        header.writeUInt32LE(36 + dataLength, 4)
        header.write('WAVE', 8)

        // fmt chunk
        header.write('fmt ', 12)
        header.writeUInt32LE(16, 16) // fmt chunk size
        header.writeUInt16LE(1, 20) // PCM format
        header.writeUInt16LE(1, 22) // mono
        header.writeUInt32LE(sampleRate, 24)
        header.writeUInt32LE(sampleRate * 2, 28) // byte rate
        header.writeUInt16LE(2, 32) // block align
        header.writeUInt16LE(16, 34) // bits per sample

        // data chunk
        header.write('data', 36)
        header.writeUInt32LE(dataLength, 40)

        return header
      }

      /**
       * Convert Int16Array to Buffer
       */
      export function int16ArrayToBuffer(samples: number[]): Buffer {
        const buffer = Buffer.alloc(samples.length * 2)
        for (let i = 0; i < samples.length; i++) {
          const value = Math.max(-32768, Math.min(32767, Math.round(samples[i] ?? 0)))
          buffer.writeInt16LE(value, i * 2)
        }
        return buffer
      }

      /**
       * Create and save WAV file
       */
      export function createWav(audioBuffer: number[], sampleRate: number, filename: string): void {
        const audioData = int16ArrayToBuffer(audioBuffer)
        const wavHeader = createWavHeader(audioData.length, sampleRate)
        const wavFile = Buffer.concat([wavHeader, audioData])

        writeFileSync(filename, wavFile)
        console.log(`▸ WAV file saved as: ${filename}`)
      }

      /**
       * Play a WAV buffer by streaming it into ffplay over stdin.
       *
       * ffplay ships with ffmpeg and is cross-platform (macOS/Linux/Windows), so
       * we avoid the old "write to /tmp then shell out to afplay/aplay/powershell"
       * dance — no temp files, no platform switch, no hardcoded /tmp path (which
       * doesn't exist on Windows). Requires ffplay on PATH.
       */
      /**
       * Play one mono s16le PCM chunk (as a minimal WAV) and wait for the player to finish.
       * Chunks are played sequentially when awaited in order — suitable for streaming TTS output.
       */
      export function playPcmInt16Chunk(samples: number[], sampleRate: number): Promise<void> {
        if (samples.length === 0) {
          return Promise.resolve()
        }

        const audioData = int16ArrayToBuffer(samples)
        const wavHeader = createWavHeader(audioData.length, sampleRate)
        const wavFile = Buffer.concat([wavHeader, audioData])
        // `os.tmpdir()` resolves to the OS-specific temp directory (e.g. `%TEMP%`
        // on Windows), so the Windows branch below no longer tries to read a
        // POSIX-only `/tmp/...` path.
        const tempFile = join(
          tmpdir(),
          `qvac-tts-chunk-${Date.now()}-${Math.random().toString(16).slice(2)}.wav`
        )
        writeFileSync(tempFile, wavFile)

        const currentPlatform = platform()
        let audioPlayer: string
        let args: string[]

        switch (currentPlatform) {
          case 'darwin':
            audioPlayer = 'afplay'
            args = [tempFile]
            break
          case 'linux':
            audioPlayer = 'aplay'
            args = [tempFile]
            break
          case 'win32':
            audioPlayer = 'powershell'
            args = [
              '-Command',
              `Add-Type -AssemblyName presentationCore; (New-Object Media.SoundPlayer).LoadStream([System.IO.File]::ReadAllBytes('${tempFile}')).PlaySync()`
            ]
            break
          default:
            audioPlayer = 'aplay'
            args = [tempFile]
        }

        return new Promise(function (resolve, reject) {
          const proc = spawn(audioPlayer, args, { stdio: 'ignore' })
          proc.on('error', function (err) {
            try {
              unlinkSync(tempFile)
            } catch {
              // ignore
            }
            reject(err)
          })
          proc.on('close', function (code) {
            try {
              unlinkSync(tempFile)
            } catch {
              // ignore
            }
            if (code === 0) {
              resolve()
            } else {
              reject(new Error(`Audio player exited with code ${code}`))
            }
          })
        })
      }

      export function playAudio(audioBuffer: Buffer): void {
        const result = spawnSync(
          'ffplay',
          ['-hide_banner', '-loglevel', 'error', '-autoexit', '-nodisp', '-i', 'pipe:0'],
          {
            input: audioBuffer,
            stdio: ['pipe', 'inherit', 'inherit']
          }
        )

        if (result.error) {
          const code = (result.error as NodeJS.ErrnoException).code
          if (code === 'ENOENT') {
            throw new Error('ffplay not found on PATH. Install ffmpeg (ffplay ships with it) and retry.')
          }
          throw new Error(`ffplay failed: ${result.error.message}`)
        }
        if (result.status !== 0) {
          throw new Error(`ffplay exited with code ${result.status}`)
        }
      }
      ```
    </WrapCode>
  </Tab>
</Tabs>

### Tuning

The defaults are deliberately conservative to prevent the assistant from hearing its own TTS output and looping on itself (a classic failure mode when mic and speakers share the same room). The relevant VAD parameters in the script:

```ts
{
  threshold: 0.6,              // less sensitive than Silero's default
  min_speech_duration_ms: 300, // drops short clicks / breaths / stray words
  min_silence_duration_ms: 700,// long quiet tail before committing a segment
  max_speech_duration_s: 15.0, // caps runaway utterances
  speech_pad_ms: 200,          // edge padding improves accuracy
}
```

Plus three additional safeguards:

* **Mic gate during TTS:** incoming audio is dropped while the assistant speaks, so it cannot transcribe its own output.
* **Post-playback cooldown** (`POST_PLAYBACK_COOLDOWN_MS = 300`): keeps the mic gated for a moment after playback so speaker/room reverb doesn't bleed into the next VAD segment.
* **Minimum utterance length** (`MIN_UTTERANCE_CHARS = 3`): drops single-character or two-letter phantom transcripts like `"you"` or `"."` that Whisper hallucinates from near-silent audio.

### Troubleshooting

If you run into common issues, adjust the values above:

| Symptom                                     | Fix                                                                                                              |
| ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| Assistant cuts you off mid-sentence         | Raise `min_silence_duration_ms` to `900-1000`                                                                    |
| Assistant talks over itself / loops forever | Raise `threshold` to `0.7`; raise `min_silence_duration_ms` to `900`; raise `POST_PLAYBACK_COOLDOWN_MS` to `500` |
| Slow to respond after you stop talking      | Lower `min_silence_duration_ms` to `500`                                                                         |
| Picks up background typing / keyboard       | Raise `threshold` to `0.7` and `min_speech_duration_ms` to `400`                                                 |
| Short commands ("yes", "no") are ignored    | Lower `MIN_UTTERANCE_CHARS` to `2`                                                                               |

If you're running with headphones (mic cannot hear the speaker), you can loosen everything: `threshold: 0.5`, `min_silence_duration_ms: 500`, `POST_PLAYBACK_COOLDOWN_MS: 0`.

### Customizing

* **Different ASR model:** swap `WHISPER_TINY` for a larger Whisper model for better transcription accuracy (e.g. `WHISPER_BASE_Q8_0`, `WHISPER_SMALL_Q8_0`, `WHISPER_LARGE_V3_TURBO`, etc.).
* **Different LLM:** swap `LLAMA_3_2_1B_INST_Q4_0` for any LLM constant from `@qvac/sdk`. Larger models give better answers at the cost of latency.
* **Different voice:** replace the Supertonic constants with another TTS model (e.g. Chatterbox — see [Text-to-Speech](/sdk/v0.23/ai-capabilities/text-to-speech)).
* **System prompt:** edit `SYSTEM_PROMPT` at the top of the script. The default instructs the LLM to be concise and avoid markdown so responses are pleasant to listen to.

<Callout type="success">
  **Tip:** all examples throughout this documentation are self-contained and runnable. For instructions on how to run them, see the [JS/TS quickstart](/sdk/v0.23/js-ts-sdk#quickstart), the [Python quickstart](/sdk/v0.23/python-sdk#quickstart), or the [Kotlin quickstart](/sdk/v0.23/kotlin-sdk#quickstart).
</Callout>
