@qvac/tts-ggml

Overview

Bare module that adds support for text-to-speech in QVAC, backed by the qvac-tts.cpp GGML library.

It runs in-process with a persistent native engine — the GGUFs, the S3Gen preload, the ggml backend, and any voice-conditioning tensors are loaded once and reused across every synthesis call. GPU acceleration (Metal on macOS/iOS, Vulkan / OpenCL on Linux/Windows/Android) is opt-in via config: { useGPU: true }; the default is CPU.

Models

Five engine families are wrapped — Chatterbox, Supertonic, Parler-TTS, CosyVoice3, and Audio8 — each with its own model layout under models/:

Chatterbox multilingual (23 languages): Arabic (ar), Danish (da), German (de), Greek (el), English (en), Spanish (es), Finnish (fi), French (fr), Hebrew (he), Hindi (hi), Italian (it), Japanese (ja), Korean (ko), Malay (ms), Dutch (nl), Norwegian (no), Polish (pl), Portuguese (pt), Russian (ru), Swedish (sv), Swahili (sw), Turkish (tr), Chinese (zh).

Supertonic 3 (31 languages + na): ar, bg, hr, cs, da, nl, en, et, fi, fr, de, el, hi, hu, id, it, ja, ko, lv, lt, pl, pt, ro, ru, sk, sl, es, sv, tr, uk, vi. Pass language: 'na' when the input language is unknown.

Point the addon at a custom location via files.modelDir (engine auto-detected from the GGUF filenames present), or pass explicit files.t3Model + files.s3genModel (Chatterbox), files.supertonicModel (Supertonic), files.parlerModel (Parler-TTS), files.cosyvoiceModelDir (CosyVoice3), or files.audio8Lm + files.audio8CodecDecoder (+ files.audio8CodecEncoder for voice cloning) (Audio8). Use npm run download-models:registry to acquire registry-published Chatterbox, Supertonic, Parler, and Audio8 models. CosyVoice3 must currently be staged from local converted artifacts.

Requirement

Bare $\geq$ v1.19

Installation

npm i @qvac/tts-ggml

Prebuilds are published for Linux x64/arm64, macOS x64/arm64, Windows x64, Android arm64, iOS arm64 devices, and iOS x64/arm64 simulators. Unsupported targets require an explicit source build; installation does not automatically compile a local addon.

Quickstart

npm i -g bare

mkdir qvac-tts-quickstart
cd qvac-tts-quickstart
npm init -y

npm i @qvac/tts-ggml bare-fs bare-path

"use strict";

const fs = require("bare-fs");
const TTSGgml = require("@qvac/tts-ggml");

const SAMPLE_RATE = 24000;

async function main() {
  const model = new TTSGgml({
    files: { modelDir: "./models" }, // contains chatterbox-{t3-turbo,s3gen}.gguf
    referenceAudio: "./reference.wav", // optional voice cloning
    config: { language: "en" },
    opts: { stats: true },
  });

  try {
    console.log("Loading Chatterbox TTS model...");
    await model.load();
    console.log("Model loaded.");

    const textToSynthesize =
      "Hello world! This is a test of the Chatterbox TTS system.";
    console.log(`Running TTS on: "${textToSynthesize}"`);

    const response = await model.run({
      input: textToSynthesize,
      type: "text",
    });

    let pcm = [];
    await response
      .onUpdate((data) => {
        if (data && data.outputArray)
          pcm = pcm.concat(Array.from(data.outputArray));
      })
      .await();

    console.log("TTS finished!");
    if (response.stats) {
      console.log(`Inference stats: ${JSON.stringify(response.stats)}`);
    }

    console.log(`Generated ${pcm.length} audio samples at ${SAMPLE_RATE}Hz`);
  } catch (err) {
    console.error("Error during TTS processing:", err);
  } finally {
    console.log("Unloading model...");
    await model.unload();
    console.log("Model unloaded.");
  }
}

main().catch(console.error);

bare index.js

Usage

1. Import the Model Class

const TTSGgml = require("@qvac/tts-ggml");

2. Create the Model Instance

const model = new TTSGgml({
  files: { modelDir: "./models" },
  referenceAudio: "./voices/me.wav",
  config: { language: "en", useGPU: false },
  opts: { stats: true },
});

The most common constructor options:

See the package README for the full option set (GPU/backend, KV-cache, and Android-specific options).

3. Load the Model

await model.load();

load() constructs the native engine — it loads T3, preloads S3Gen, and bakes voice conditioning. Subsequent run() calls reuse all of it.

4. Run TTS Synthesis

Pass the text to synthesize to the run method and process the generated audio output asynchronously:

try {
  const textToSynthesize = "Hello world! This is a test of the TTS system.";
  let audioSamples = [];

  const response = await model.run({
    input: textToSynthesize,
    type: "text",
  });

  await response
    .onUpdate((data) => {
      if (data && data.outputArray) {
        audioSamples = audioSamples.concat(Array.from(data.outputArray));
      }
    })
    .await();

  console.log(`Total audio samples generated: ${audioSamples.length}`);

  // audioSamples now contains the complete audio as PCM data (16-bit, 24 kHz, mono)
  if (response.stats) {
    console.log(`Inference stats: ${JSON.stringify(response.stats)}`);
  }
} catch (error) {
  console.error("TTS synthesis failed:", error);
}

5. Release Resources

Unload the model when finished:

try {
  await model.unload();
} catch (error) {
  console.error("Failed to unload model:", error);
}

Streaming

Sentence streaming — runStreaming(asyncIterable)

Use when your text arrives as discrete sentences (e.g. buffered LLM output) and you want the audio to flow sentence-by-sentence. One onUpdate event per input yield:

async function* sentencesOverTime() {
  yield "First sentence.";
  await new Promise((r) => setTimeout(r, 200));
  yield "The second arrives shortly after.";
}

const response = await model.runStreaming(sentencesOverTime());
await response
  .onUpdate((data) => {
    // data.outputArray   — Int16 PCM for this sentence's audio
    // data.chunkIndex    — 0-based index of the yielded sentence
    // data.sentenceChunk — the sentence text that produced this audio
  })
  .await();

runStreaming(textStream, options) accepts a string, string array, iterable, or async iterable. Async iterables default to accumulateSentences: true; strings, arrays, and synchronous iterables default to one synthesis job per item. Set accumulateSentences explicitly to override that behavior. Select sentenceDelimiterPreset: "latin", "multilingual", or "cjk", or provide a sentenceDelimiter regular expression. maxBufferScalars forces a flush at the buffer limit and flushAfterMs flushes incomplete text after the configured delay. Parler description fields and Audio8 referenceAudio / referenceText can also be fixed for the full streaming response.

Chunk streaming — streamChunkTokens

Use when you want the fastest possible first-audio-out within a single utterance. The C++ engine splits each synthesis into chunks of streamChunkTokens speech tokens and emits audio per chunk:

const model = new TTSGgml({
  files: { modelDir: "./models" },
  streamChunkTokens: 25, // ~1 s of audio per chunk
  streamFirstChunkTokens: 10, // smaller first chunk = faster first-audio-out
  cfmSteps: 1,
  config: { language: "en" },
});

await model.load();

const response = await model.run({
  input: "A long sentence produces many chunks...",
});
await response
  .onUpdate((data) => {
    if (data && data.outputArray) playPcmChunk(data.outputArray);
  })
  .await();

Voice cloning

Pass a mono WAV with ≥ 5 s of clean speech. The engine does the loudness normalisation, resampling, and all conditioning natively at load() time:

const model = new TTSGgml({
  files: { modelDir: "./models" },
  referenceAudio: "./voices/me.wav",
  config: { language: "en" },
});

Alternatively point at a pre-baked profile directory via voiceDir. When both are supplied, missing tensors in voiceDir are backfilled from referenceAudio.

Audio8 clones from the recording plus what is said in it: supply referenceAudio and referenceText (the verbatim transcript) together with files.audio8CodecEncoder, which encodes the reference to codes in-process — no enrolment step, no voice profile. The reference can also be switched per call. CosyVoice3 does not take a reference — it always speaks with its baked default voice.

Speech enhancement (LavaSR)

Opt-in neural post-processing that bandwidth-extends the synthesized audio to 48 kHz with a synthesised high band, using the LavaSR Vocos enhancer run on the CPU/GGML path. It is fully backward compatible — provide no enhancer GGUF and nothing changes. Enhancement is enabled simply by supplying the enhancer GGUF; there is no separate on/off flag.

const model = new TTSGgml({
  engine: TTSGgml.ENGINE_SUPERTONIC,
  // Providing the enhancer GGUF is what turns enhancement on:
  files: {
    supertonicModel,
    lavasrEnhancer: "models/lavasr/lavasr-enhancer.gguf",
  },
  config: { language: "en" },
});
// The output callback now reports 48000:
//   response.onUpdate(d => { /* d.outputArray; d.sampleRate === 48000 */ })

The GGUF path may instead be given as an enhancer: { type: 'lavasr', enhancerPath } block.

• Works for Chatterbox, Supertonic, Parler, and CosyVoice3 (Audio8 is not supported) — on the batch path, sentence-level streaming, and native chunk streaming where the selected engine supports it.
• For native chunk streaming the enhancer runs over a sliding window with look-ahead + crossfade, so each emitted chunk is bandwidth-extended seam-free; this adds ~0.34 s of look-ahead latency.
• The enhancer always runs at 48 kHz internally. By default the emitted audio is 48 kHz; set config.outputSampleRate to resample the enhanced output (sampleRate reports the actual rate).

Denoiser

LavaSR's first stage — the UL-UNAS denoiser that cleans the signal before the enhancer bandwidth-extends it — is enabled the same way, via files.lavasrDenoiser (or a denoiser: { type: 'lavasr', denoiserPath } block), and runs before the enhancer on the batch path:

const model = new TTSGgml({
  engine: TTSGgml.ENGINE_SUPERTONIC,
  files: {
    supertonicModel,
    lavasrDenoiser: "models/lavasr/lavasr-denoiser.gguf", // cleaned first…
    lavasrEnhancer: "models/lavasr/lavasr-enhancer.gguf", // …then upsampled
  },
  config: { language: "en" },
});

• The denoiser forward runs at 16 kHz internally (resampled in/out), so it is rate-preserving — the emitted audio keeps the engine's sample rate. With no denoiser path the output is unchanged.
• Denoiser + Chatterbox native chunk streaming (streamChunkTokens > 0) is rejected up front; use batch synthesis, or drop the denoiser for streaming.

Output Format

Audio is received via the onUpdate callback of the response object as raw PCM samples.

response.onUpdate((data) => {
  data.outputArray; // Int16Array — mono PCM
  data.sampleRate; // actual output sample rate
  data.chunkIndex; // present on sentence-streaming events only
  data.sentenceChunk; // present on sentence-streaming events only
});

When synthesis completes and opts: { stats: true } was set, response.stats reports performance:

response.stats.totalTime; // seconds
response.stats.realTimeFactor; // synthesis time / audio duration; < 1 means streaming is possible
response.stats.audioDurationMs;
response.stats.totalSamples;
response.stats.tokensPerSecond;
response.stats.backendDevice; // 0 CPU, 1 GPU
response.stats.backendId; // 0 CPU, 1 Metal, 2 CUDA, 3 Vulkan, 4 OpenCL, 99 other
response.stats.enhancerBackendDevice; // -1 absent, 0 CPU, 1 GPU
response.stats.enhancerBackendId;

Audio Format Specifications:

• Sample Rate: engine-native by default (24 kHz for Chatterbox/CosyVoice3, 44.1 kHz for Supertonic/Parler/Audio8, 48 kHz with the enhancer), configurable via config.outputSampleRate
• Format: 16-bit signed PCM, mono channel
• Data Type: Int16Array containing raw audio samples

Public helpers and errors

Import splitTtsText from @qvac/tts-ggml/text-chunker and accumulateTextStream from @qvac/tts-ggml/text-stream-accumulator. The legacy @qvac/tts-ggml/lib/textStreamAccumulator.js path remains supported.

QvacErrorAddonTTSGgml and ERR_CODES are exported from @qvac/tts-ggml. Error codes are:

More resources

Package at npm