Transcription
Automatic speech recognition (ASR) for speech-to-text — i.e., generate text transcriptions from audio input.
Overview
Transcription uses your choice of either qvac-fabric-speech.cpp or NVIDIA Parakeet (via the GGML-based parakeet-cpp engine) as inference engine. Load a model using modelType: "whisper" for qvac-fabric-speech.cpp, or modelType: "parakeet-transcription" for Parakeet. Parakeet supports multilingual transcription (TDT), English-only transcription (CTC), English transcription with a single model that supports both batch and duplex streaming (Unified), the TranscribePsy family (the full 25-language EU model and its EU-sub distillation for eight European languages), Indic-language transcription (Indic Conformer CTC), speaker diarization (Sortformer and Nemotron 3), and end-of-utterance detection (EOU) for duplex streaming.
Provide audio input as audioChunk, either as a file path (string) or an in-memory audio buffer.
transcribe() returns the full transcription as a single string. Its returned promise also exposes stats: await it after transcription to read terminal engine statistics. For Parakeet offline ASR, encoderOnCoreml reports whether a Core ML sidecar loaded, while encoderUsedCoreml reports whether this run encoded on Core ML. The stats promise resolves to undefined when the engine does not report statistics. If you need partial results as they become available, use transcribeStream() to receive text chunks in real-time. Both whisper and parakeet expose duplex transcribeStream() sessions; see "Streaming with transcribeStream()" below.
Functions
Use the following sequence of function calls:
For how to use each function, see SDK — API reference.
Models
qvac-fabric-speech.cpp
You should load two models:
- a
whisper.cpp-compatible model for transcription. Model file format:*.bin; and - a VAD model (e.g., Silero) converted to GGML. Model file format:
*.bin(optional, recommended).
Parakeet
Parakeet uses a GGUF checkpoint per variant, and the addon detects TDT / CTC / Unified / Sortformer / EOU from its GGUF metadata (TranscribePsy loads as a TDT model). Supply the GGUF via the top-level modelSrc:
await loadModel({
modelSrc: PARAKEET_TDT_0_6B_V3_Q8_0, // multilingual, ~750MB
modelType: "parakeet-transcription",
});
await loadModel({
modelSrc: PARAKEET_CTC_0_6B_Q8_0, // english-only, streaming-capable
modelType: "parakeet-transcription",
});
await loadModel({
modelSrc: PARAKEET_UNIFIED_0_6B_Q8_0, // english; batch + streaming, ~740MB
modelType: "parakeet-transcription",
});
await loadModel({
modelSrc: PARAKEET_TRANSCRIBEPSY_250M_Q8_0, // nl/en/fr/de/it/pl/pt/es, ~305MB
modelType: "parakeet-transcription",
});
await loadModel({
modelSrc: PARAKEET_TRANSCRIBEPSY_250M_EU_Q8_0, // 25 European languages
modelType: "parakeet-transcription",
modelConfig: { language: "auto" }, // "en", "other" or "auto" (default)
});
await loadModel({
modelSrc: PARAKEET_SORTFORMER_4SPK_V2_1_Q8_0, // 4-speaker diarization
modelType: "parakeet-transcription",
});
await loadModel({
modelSrc: PARAKEET_EOU_120M_V1_Q8_0, // end-of-utterance detection
modelType: "parakeet-transcription",
});
await loadModel({
modelSrc: PARAKEET_INDIC_CONFORMER_600M_Q8_0, // Indic languages
modelType: "parakeet-transcription",
modelConfig: { language: "hi" }, // required; e.g. "hi", "ta"
});On macOS and iOS, loading a supported Parakeet registry constant also downloads its complete Core ML encoder bundle and stages the .mlmodelc directory beside the GGUF. This applies to TDT 0.6B v3, Unified English 0.6B, EOU 120M v1, and streaming Sortformer v2.1. The sidecar weights increase the first download and disk use. Other platforms download the GGUF alone; if a sidecar is unavailable or cannot load, the GGUF encoder is used.
TranscribePsy EU-sub publishes a separate Core ML distribution. PARAKEET_TRANSCRIBEPSY_250M_COREML_Q8_0 is an encoder-less GGUF whose encoder and decoder bundles the SDK downloads and stages beside it, so both stages run on the Apple Neural Engine; stats.encoderUsedCoreml on the transcribe() call reports it. That constant loads on macOS and iOS only: elsewhere the load is refused up front, and PARAKEET_TRANSCRIBEPSY_250M_Q8_0 is the one to use.
TranscribePsy EU. The full TranscribePsy-250M-EU model (PARAKEET_TRANSCRIBEPSY_250M_EU_Q8_0) covers 25 European languages from one GGUF that embeds its CAFE language adapters and gate, and it runs on ggml on every platform: it has no Core ML distribution, and the EU-sub bundles do not contain its adapters. language in modelConfig picks the route: "en" decodes on the English trunk, "other" on the non-English adapters, and "auto" (the default when omitted) runs the gate on the first 20 seconds of the recording and routes it.
Backend selection. modelConfig.backend selects the native backend explicitly: auto (the default), cpu, cuda, vulkan, opencl, metal, or an exact ggml device name such as CUDA0 or MTL0. An explicit selection overrides useGPU, and the load fails if that backend is unavailable instead of falling back to the CPU. The same selector reaches assessModelFit, where it also overrides the GPU gate. Core ML sidecars keep their own routing; a ggml selector does not move them.
Long recordings. Parakeet encodes a long transcribe() input in overlapping windows, so memory stays flat however long the recording is and cancellation lands between windows. The defaults need nothing from you: TranscribePsy uses the 60 s windows with 25 % overlap it recommends, the other models the engine's automatic windows. To trade speed for a smaller footprint, for example on a phone with little memory, set longFormWindowFrames and longFormContextFrames in modelConfig (encoder frames of 80 ms: 375 is a 30 s window, and 47 frames of context on each side keeps the 25 % overlap); a negative window encodes the whole input in one pass.
For model artifacts available as constants, see SDK — Models.
Migrating from pre-0.6 Parakeet (ONNX multi-file): the legacy multi-file ONNX modelConfig shape (parakeetEncoderSrc / parakeetDecoderSrc / parakeetVocabSrc / parakeetPreprocessorSrc, plus parakeetCtcModelSrc / parakeetTokenizerSrc and parakeetSortformerSrc for the CTC/Sortformer variants) is no longer supported. Passing any of those fields raises a structured LegacyParakeetModelDeprecatedError with a migration message. The legacy ONNX constants (e.g. PARAKEET_TDT_ENCODER_INT8, PARAKEET_CTC_FP32, PARAKEET_SORTFORMER_FP32) remain exported for one minor cycle for codemod migrations only and will be removed in a future release.
On VAD: when using qvac-fabric-speech.cpp, you can optionally provide a separate model for voice activity detection (VAD); this is recommended. In turn, Parakeet handles VAD internally, so no additional model or configuration is required.
Speaker-aware MOSS transcription
MOSS-Transcribe-Diarize performs batch transcription with timestamps and speaker labels.
Load MOSS_TRANSCRIBE_DIARIZE_Q8_0 (or the F16/Q5_0 variant) with the
moss-transcribe model type. Registry constants select the type automatically.
import { loadModel, transcribe, MOSS_TRANSCRIBE_DIARIZE_Q8_0 } from '@qvac/sdk'
const modelId = await loadModel({ modelSrc: MOSS_TRANSCRIBE_DIARIZE_Q8_0 })
const segments = await transcribe({
modelId,
audioChunk: 'meeting.wav',
metadata: true,
hotwords: ['QVAC'],
maxNewTokens: 2048
})Segments include speaker labels such as S01, zero-based speakerId, and
startMs/endMs. Hotwords are request-specific: up to 64 nonempty terms, each
at most 64 UTF-8 bytes. A custom prompt cannot be combined with hotwords.
maxNewTokens: 0 uses the model default. MOSS supports whole-recording batch
inference; duplex streaming and configuration reload are unavailable.
Before downloading weights, call assessModelFit with the longest recording
you plan to transcribe and matching decoding options:
import { assessModelFit, MOSS_TRANSCRIBE_DIARIZE_Q8_0 } from "@qvac/sdk";
const fit = await assessModelFit({
models: [
{
modelSrc: MOSS_TRANSCRIBE_DIARIZE_Q8_0,
modelConfig: { useGPU: false },
transcriptionWorkload: {
audioSeconds: 300,
hotwords: ["QVAC"],
maxNewTokens: 0,
},
},
],
});transcriptionWorkload is fit-only and requires a positive duration in seconds.
It accepts the same prompt, hotwords, and token limit as MOSS transcription;
omit either prompt or hotwords. Without a declared workload, or without a
registry description or local model, the verdict is unknown.
Assessment does not load the model, reserve memory, or block transcription.
Nemotron 3 speaker diarization
Nemotron 3 Diarization runs through parakeet-transcription and identifies up to
eight speakers, including overlapping activity. It returns speaker turns rather
than recognized words. For local weights:
const modelId = await loadModel({
modelSrc: './Nemotron-3-Diarization.q8_0.gguf',
modelType: 'parakeet-transcription'
})
const segments = await transcribe({ modelId, audioChunk: 'meeting.wav', metadata: true })
const turns = segments.flatMap(segment => segment.speakerSegments ?? [])Each turn has zero-based speakerId and startMs/endMs. Configure
diarizationThreshold (0–1) and diarizationMinSegmentMs (a nonnegative integer)
at load or per stream in parakeetStreamingConfig; omit them to retain native
model defaults. Load with streaming: true before calling transcribeStream.
Enable streamingSpeakerVad at load or emitSpeakerVad per stream to receive
speaker-activity events. Streaming diarization metadata retains speaker IDs.
Speaker-attributed Parakeet transcription
A Parakeet ASR model can transcribe each speaker turn found by a separate
diarization model, so one transcribe() call returns text per speaker. Set
speakerAttribution in modelConfig with the diarization GGUF (Sortformer or
Nemotron 3 Diarization) as its modelSrc; the SDK resolves and downloads it like
any other model source and loads it beside the ASR model.
const modelId = await loadModel({
modelSrc: PARAKEET_UNIFIED_0_6B_Q8_0,
modelType: 'parakeet-transcription',
modelConfig: {
speakerAttribution: {
modelSrc: PARAKEET_SORTFORMER_4SPK_V2_1_Q8_0,
mergeSameSpeaker: true, // merge adjacent turns of the same speaker (default)
minSegmentMs: 200, // shortest padded turn sent to ASR; 0 keeps every turn
padSegmentMs: 0 // widen the audio slice around each turn
}
}
})
const segments = await transcribe({ modelId, audioChunk: 'meeting.wav', metadata: true })
for (const { speakerId, startMs, endMs, text } of segments) {
console.log(speakerId, startMs, endMs, text)
}Each segment carries a zero-based speakerId and the diarized turn's
startMs/endMs; padSegmentMs widens only the audio the ASR model hears.
Omitted options keep the addon's defaults, and diarizationThreshold and
diarizationMinSegmentMs still override the diarizer when set. backend
applies to both models. Speaker attribution is a batch mode: it cannot be
combined with streaming: true, and transcribeStream() is unavailable on such
a load. assessModelFit answers unknown for it, as the fitter projects one
model at a time; assess the ASR and diarization models as separate loads.
The CLI accepts MOSS hotwords and max_new_tokens in transcription requests.
Its verbose_json response includes MOSS speaker labels and Nemotron
speaker_segments with speaker_id, start, and end in seconds.
Streaming with transcribeStream()
transcribeStream() opens a duplex session for both engines — write audio chunks via session.write(...), iterate events with for await (const event of session) { ... }. Events are typed as a discriminated union { type }:
{ type: "text", text }— incremental transcript text.{ type: "segment", segment }— segment metadata whenmetadata: true, on both engines. Parakeet segments also carryisEndOfTurnandstartsWord, and in streaming the nativetokenIds,isFinal,chunkIndexandeotConfidence(see "Parakeet streaming metadata").{ type: "vad", speaking, probability, source }— voice-activity-detection state.sourceis"silero"for whisper,"energy"for Parakeet's energy detector, or"sortformer"for Parakeet speaker activity. Parakeet events also carrytimestamp(seconds from the session start) andchunkIndex(the native chunk,-1when unavailable).{ type: "endOfTurn", source: "whisper", silenceDurationMs }— turn boundary detected from a measured silence window (whisper).{ type: "endOfTurn", source: "parakeet", timestamp?, chunkIndex?, confidence? }— turn boundary detected from the EOU model's<EOU>token (parakeet; no silence window — the event is token-driven). The native boundary reports its time in seconds from the session start, its chunk index and the model's end-of-turn confidence; a boundary the addon derives from a segment carries none of them.
The source field on endOfTurn lets consumers narrow the union: whisper events always carry a numeric silenceDurationMs; parakeet events never do.
After the iterator completes, await session.stats to read terminal engine statistics. The promise resolves to undefined when the engine does not report stats.
for await (const event of session) {
// Handle text, segment, VAD, or end-of-turn events.
}
const stats = await session.stats;
console.log(stats?.audioDuration, stats?.realTimeFactor);Wire compatibility: post-0.6 servers emit source on every endOfTurn frame. SDK parsers still accept the legacy whisper wire shape { silenceDurationMs } (no source) and normalize it to source: "whisper". Upgrade client and server together when using parakeet source: "parakeet" events — older servers never emit that branch.
Parakeet duplex streaming
Pass parakeetStreamingConfig to transcribeStream() to override per-call streaming knobs (each falls back to its parakeetConfig.streaming* load-time counterpart):
const session = await transcribeStream({
modelId,
parakeetStreamingConfig: {
chunkMs: 1000, // encoder cadence
historyMs: 30000, // sortformer rolling-history window
leftContextMs: 500, // ASR encoder left-context window
rightLookaheadMs: 200, // ASR encoder right-lookahead window
emitPartials: true, // emit partial segments before chunk boundaries
emitEnergyVad: false, // CTC/TDT energy-based VAD hint (engine-internal)
},
});
for await (const event of session) {
switch (event.type) {
case "text":
process.stdout.write(event.text);
break;
case "endOfTurn":
// event.source: "whisper" | "parakeet"
console.log("\n[endOfTurn] turn boundary detected\n");
break;
}
}The { type: "endOfTurn", source: "parakeet" } event surfaces whenever the EOU model emits an <EOU> token, and is the parakeet equivalent of whisper's silence-window EOU. Pair it with the PARAKEET_EOU_120M_V1_Q8_0 checkpoint when you need explicit turn boundaries from parakeet. The boundary follows the transcript chunks of its turn, so a consumer can submit and clear the text it accumulated when the event arrives.
Parakeet streaming metadata
With metadata: true, every Parakeet streaming segment carries what the native
streaming session reports for its chunk, alongside text, startMs, endMs,
append, id, isEndOfTurn and startsWord:
tokenIds— the native token ids of the segment.isFinal— the engine's own finality flag. Empty final markers are delivered, so use it rather than non-empty text to detect a finalized chunk.chunkIndex— the native streaming chunk the segment belongs to;-1when unknown.eotConfidence— the EOU model's end-of-turn confidence for the segment; it may be0when the model reports none.
Diarization streaming segments carry isFinal and chunkIndex with their
speakerId. Batch transcribe() segments and whisper segments do not gain
these fields.
Examples
qvac-fabric-speech.cpp
The following script shows an example of qvac-fabric-speech.cpp transcription with prompt-guided decoding, VAD, and GPU acceleration. Pass the path of a 16 kHz mono WAV file as the first argument:
/**
* Whisper transcription with prompt example.
*
* Usage:
* npx tsx examples/asr/whispercpp-prompt.ts <wav-file>
*
* Pass the path of a 16 kHz mono WAV file. A sample is available in the QVAC
* source repository, but not included in the published npm package:
* https://github.com/tetherto/qvac/blob/main/packages/sdk/examples/audio/sample-16khz.wav
*/
import { loadModel, unloadModel, transcribe, WHISPER_TINY } from '@qvac/sdk';
const audioFilePath = process.argv[2];
if (!audioFilePath) {
console.error('Usage: npx tsx examples/asr/whispercpp-prompt.ts <wav-file>');
process.exit(1);
}
try {
console.log('▸ Starting Whisper transcription with prompt example...');
// Load the Whisper model
console.log('▸ Loading Whisper model...');
const modelId = await loadModel({
modelSrc: WHISPER_TINY,
modelConfig: {
audio_format: 'f32le',
// Sampling strategy
strategy: 'greedy',
n_threads: 4,
// Transcription options
language: 'en',
translate: false,
no_timestamps: false,
single_segment: false,
print_timestamps: true,
token_timestamps: true,
// Quality settings
temperature: 0.0,
suppress_blank: true,
suppress_nst: true,
// Advanced tuning
entropy_thold: 2.4,
logprob_thold: -1.0,
// VAD configuration
vad_params: {
threshold: 0.35,
min_speech_duration_ms: 200,
min_silence_duration_ms: 150,
max_speech_duration_s: 30.0,
speech_pad_ms: 600,
samples_overlap: 0.3
},
// Context parameters for GPU
contextParams: {
use_gpu: true,
flash_attn: true,
gpu_device: 0
}
},
onProgress: (p) => {
const mb = (n) => (n / 1e6).toFixed(1);
const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
if (p.percentage >= 100)
process.stderr.write('\n');
}
});
console.log(`▸ Whisper model loaded with ID: ${modelId}`);
// Perform transcription
console.log('▸ Transcribing audio...');
const text = await transcribe({
modelId,
audioChunk: audioFilePath,
prompt: 'This is a test recording with clear speech and proper punctuation.'
});
console.log('▸ Transcription result:');
console.log(text);
// Unload the model when done
console.log('▸ Unloading Whisper model...');
await unloadModel({ modelId });
console.log('▸ Whisper model unloaded successfully');
process.exit(0);
}
catch (error) {
console.error('✖', error);
process.exit(1);
}Parakeet TDT
The following script shows an example of multilingual transcription using the Parakeet TDT model from a WAV file. Pass the path of a 16 kHz mono WAV file as the first argument; an optional second argument overrides the default model:
/**
* Parakeet TDT transcription from a WAV file.
*
* Usage:
* npx tsx examples/asr/parakeet-tdt-filesystem.ts <wav-file> [parakeet-tdt-gguf]
*
* Loads a single GGUF checkpoint (`PARAKEET_TDT_0_6B_V3_Q8_0` by default) and
* transcribes the file with the batch `transcribe` API. Omit the model
* argument to use the registry constant.
*
* Audio should be 16 kHz mono PCM in a WAV container.
*/
import { loadModel, unloadModel, transcribe, PARAKEET_TDT_0_6B_V3_Q8_0 } from '@qvac/sdk';
const args = process.argv.slice(2);
if (!args[0]) {
console.error('Usage: npx tsx examples/asr/parakeet-tdt-filesystem.ts <wav-file-path> ' +
'[parakeet-tdt-gguf]');
console.error('\nIf the model path is omitted, defaults to the registry model.');
process.exit(1);
}
const audioFilePath = args[0];
const parakeetModelSrc = args[1] ?? PARAKEET_TDT_0_6B_V3_Q8_0;
try {
console.log('▸ Starting Parakeet transcription example...');
console.log('▸ Loading Parakeet model...');
const modelId = await loadModel({
modelSrc: parakeetModelSrc,
modelType: 'parakeet-transcription',
onProgress: (p) => {
const mb = (n) => (n / 1e6).toFixed(1);
const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
if (p.percentage >= 100)
process.stderr.write('\n');
}
});
console.log(`▸ Parakeet model loaded with ID: ${modelId}`);
console.log('▸ Transcribing audio...');
const text = await transcribe({ modelId, audioChunk: audioFilePath });
console.log(text);
console.log('▸ Unloading Parakeet model...');
await unloadModel({ modelId });
console.log('▸ Parakeet model unloaded successfully');
}
catch (error) {
console.error('✖', error);
process.exit(1);
}Parakeet CTC
The following script shows an example of English-only transcription using the Parakeet CTC model from a WAV file. Pass the path of a 16 kHz mono WAV file as the first argument; an optional second argument overrides the default model:
/**
* Parakeet CTC transcription from a WAV file.
*
* Usage:
* npx tsx examples/asr/parakeet-ctc-filesystem.ts <wav-file> [parakeet-ctc-gguf]
*
* Loads a single GGUF checkpoint (`PARAKEET_CTC_0_6B_Q8_0` by default) and
* transcribes the file with the batch `transcribe` API. Omit the model
* argument to use the registry constant.
*
* Audio should be 16 kHz mono PCM in a WAV container.
*/
import { loadModel, unloadModel, transcribe, PARAKEET_CTC_0_6B_Q8_0 } from '@qvac/sdk';
const args = process.argv.slice(2);
if (!args[0]) {
console.error('Usage: npx tsx examples/asr/parakeet-ctc-filesystem.ts <wav-file> ' + '[parakeet-ctc-gguf]');
console.error('\nIf the model path is omitted, defaults to the registry model.');
process.exit(1);
}
const audioFilePath = args[0];
const parakeetModelSrc = args[1] ?? PARAKEET_CTC_0_6B_Q8_0;
try {
console.log('▸ Loading Parakeet CTC model...');
const modelId = await loadModel({
modelSrc: parakeetModelSrc,
modelType: 'parakeet-transcription',
onProgress: (p) => {
const mb = (n) => (n / 1e6).toFixed(1);
const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
if (p.percentage >= 100)
process.stderr.write('\n');
}
});
console.log(`▸ Parakeet CTC model loaded with ID: ${modelId}`);
console.log('▸ Transcribing audio...');
const text = await transcribe({ modelId, audioChunk: audioFilePath });
console.log(text);
console.log('▸ Unloading model...');
await unloadModel({ modelId });
console.log('▸ Done');
}
catch (error) {
console.error('✖', error);
process.exit(1);
}Parakeet Unified
The following script shows an example of English transcription using the Parakeet Unified model from a WAV file. Load it once and use it with either transcribe() for batch or transcribeStream() for low-latency streaming. The script uses batch transcribe(). Pass the path of a 16 kHz mono WAV file as the first argument; an optional second argument overrides the default model:
/**
* Parakeet Unified transcription from a WAV file.
*
* Usage:
* npx tsx examples/asr/parakeet-unified-filesystem.ts <wav-file> [parakeet-unified-gguf]
*
* Loads a single GGUF checkpoint (`PARAKEET_UNIFIED_0_6B_Q8_0` by default) and
* transcribes the file with the batch `transcribe` API. The Unified RNN-T
* checkpoint is English-only and serves both batch and low-latency streaming
* from the same GGUF; the engine auto-detects the model type from the GGUF
* metadata. Omit the model argument to use the registry constant.
*
* Audio should be 16 kHz mono PCM in a WAV container.
*/
import { loadModel, unloadModel, transcribe, PARAKEET_UNIFIED_0_6B_Q8_0 } from '@qvac/sdk';
const args = process.argv.slice(2);
if (!args[0]) {
console.error('Usage: npx tsx examples/asr/parakeet-unified-filesystem.ts <wav-file-path> ' +
'[parakeet-unified-gguf]');
console.error('\nIf the model path is omitted, defaults to the registry model.');
process.exit(1);
}
const audioFilePath = args[0];
const parakeetModelSrc = args[1] ?? PARAKEET_UNIFIED_0_6B_Q8_0;
try {
console.log('▸ Starting Parakeet Unified transcription example...');
console.log('▸ Loading Parakeet Unified model...');
const modelId = await loadModel({
modelSrc: parakeetModelSrc,
modelType: 'parakeet-transcription',
onProgress: (p) => {
const mb = (n) => (n / 1e6).toFixed(1);
const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
if (p.percentage >= 100)
process.stderr.write('\n');
}
});
console.log(`▸ Parakeet Unified model loaded with ID: ${modelId}`);
console.log('▸ Transcribing audio...');
const text = await transcribe({ modelId, audioChunk: audioFilePath });
console.log(text);
console.log('▸ Unloading Parakeet Unified model...');
await unloadModel({ modelId });
console.log('▸ Parakeet Unified model unloaded successfully');
}
catch (error) {
console.error('✖', error);
process.exit(1);
}Parakeet Unified with Core ML
The following macOS script loads a regenerated Unified GGUF from disk, downloads its five Core ML encoder files through the registry, and transcribes a WAV file. Pass the path of the regenerated GGUF (parakeet-unified-en-0.6b.{f16,q4_0,q8_0}.gguf) as the first argument and the path of a 16 kHz mono WAV file as the second:
/**
* Transcribe a WAV file with Parakeet Unified and its Core ML encoder sidecar.
*
* From packages/sdk:
* npx tsx examples/asr/parakeet-unified-coreml.ts \
* /path/to/parakeet-unified-en-0.6b.q4_0.gguf <16-kHz-wav-file>
*
* The GGUF must be a regenerated Unified artifact. The five compiled Core ML
* files are downloaded through the SDK registry and staged beside the GGUF.
*/
import { downloadAsset, getModelInfo, loadModel, transcribe, unloadModel } from '@qvac/sdk';
import { copyFile, link, mkdir, mkdtemp, open, rm, symlink } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { basename, join, resolve } from 'node:path';
const MODEL_STEM = 'parakeet-unified-en-0.6b';
const COREML_FILES = [
['PARAKEET_UNIFIED_COREMLDATA', 'analytics/coremldata.bin'],
['PARAKEET_UNIFIED_COREMLDATA_1', 'coremldata.bin'],
['PARAKEET_UNIFIED_METADATA', 'metadata.json'],
['PARAKEET_UNIFIED_MODEL', 'model.mil'],
['PARAKEET_UNIFIED_WEIGHT', 'weights/weight.bin']
];
const modelPath = process.argv[2];
const audioPath = process.argv[3];
async function checkModel(path) {
const name = basename(path);
if (!/^parakeet-unified-en-0\.6b\.(?:f16|q4_0|q8_0)\.gguf$/.test(name)) {
throw new Error(`Expected a regenerated Unified GGUF, got: ${name}`);
}
const file = await open(path, 'r');
try {
const header = Buffer.alloc(1024 * 1024);
const { bytesRead } = await file.read(header, 0, header.length, 0);
if (!header.subarray(0, bytesRead).includes('parakeet.unified.left_context_frames')) {
throw new Error(`Unified GGUF lacks the new streaming metadata: ${path}`);
}
}
finally {
await file.close();
}
}
async function stageModel(path) {
if (process.platform !== 'darwin')
throw new Error('Core ML requires macOS');
await checkModel(path);
const dir = await mkdtemp(join(tmpdir(), 'qvac-unified-coreml-'));
try {
const modelSrc = join(dir, basename(path));
await symlink(resolve(path), modelSrc);
const bundleDir = join(dir, `${MODEL_STEM}-encoder.mlmodelc`);
for (const [name, relativePath] of COREML_FILES) {
const info = await getModelInfo({ name });
if (!info.isCached) {
if (!info.registrySource || !info.registryPath) {
throw new Error(`Registry location missing for ${name}`);
}
console.log(`▸ Downloading Core ML ${relativePath}...`);
await downloadAsset({ assetSrc: `registry://${info.registrySource}/${info.registryPath}` });
}
const cached = await getModelInfo({ name });
const sourcePath = cached.cacheFiles.find((file) => file.isCached)?.path;
if (!sourcePath)
throw new Error(`Core ML file is missing from the SDK cache: ${name}`);
const targetPath = join(bundleDir, relativePath);
await mkdir(join(targetPath, '..'), { recursive: true });
try {
await link(sourcePath, targetPath);
}
catch (error) {
// A hard link cannot cross filesystems, e.g. a QVAC_HOME_DIR on another drive.
if (error.code !== 'EXDEV')
throw error;
await copyFile(sourcePath, targetPath);
}
}
return { modelSrc, dir };
}
catch (error) {
await rm(dir, { recursive: true, force: true });
throw error;
}
}
async function main() {
if (!modelPath || !audioPath) {
throw new Error('Usage: npx tsx examples/asr/parakeet-unified-coreml.ts ' +
'<regenerated-unified-gguf> <16-kHz-wav-file>');
}
let staged;
let modelId;
try {
staged = await stageModel(modelPath);
console.log('▸ Loading Parakeet Unified with its Core ML encoder sidecar...');
modelId = await loadModel({
modelSrc: staged.modelSrc,
modelType: 'parakeet-transcription',
modelConfig: { useGPU: true }
});
console.log('▸ Transcribing audio...');
const text = await transcribe({ modelId, audioChunk: audioPath });
if (!text.trim())
throw new Error('Parakeet Unified returned an empty transcription');
console.log(text);
}
finally {
try {
if (modelId)
await unloadModel({ modelId });
}
finally {
if (staged)
await rm(staged.dir, { recursive: true, force: true });
}
}
}
main().catch((error) => {
console.error('✖', error);
process.exitCode = 1;
});TranscribePsy
The following script transcribes a file with TranscribePsy. By default it loads the ggml GGUF; on macOS, --coreml loads the Core ML distribution and reports whether the encoder ran on the Neural Engine, and --window-sec sets an explicit long-form window. Pass the path of the audio file as the first argument:
/**
* TranscribePsy transcription from an audio file.
*
* Usage:
* npx tsx examples/asr/parakeet-transcribepsy-filesystem.ts <audio-file> [--eu [--language en|other|auto]] [--coreml] [--window-sec <seconds>]
*
* TranscribePsy-250M-EU-sub is a 254M Parakeet-TDT distilled for Dutch,
* English, French, German, Italian, Polish, Portuguese and Spanish; the engine
* detects the language, so there is nothing to set. The default constant
* (`PARAKEET_TRANSCRIBEPSY_250M_Q8_0`) runs on ggml everywhere. On
* macOS and iOS, `--coreml` loads the Core ML distribution instead
* (`PARAKEET_TRANSCRIBEPSY_250M_COREML_Q8_0`): the SDK downloads its
* encoder and decoder bundles with the GGUF, and both stages run on the Neural
* Engine. Anywhere else that constant is refused at load time.
*
* `--eu` loads the full 25-language TranscribePsy-250M-EU instead
* (`PARAKEET_TRANSCRIBEPSY_250M_EU_Q8_0`): one GGUF with its CAFE language
* adapters and gate embedded, ggml on every platform, no Core ML distribution.
* `--language` picks its route: `en` for the English trunk, `other` for the
* non-English adapters, or `auto` (the default) to let the gate decide from
* the first seconds of the recording.
*
* Long recordings are encoded in overlapping windows, so memory stays flat
* however long the file is. Without `--window-sec` the engine uses the 60 s
* windows with 25 % overlap the model recommends; an explicit window is sent
* as encoder frames (12.5 per second) with a quarter of it as context, which
* is how a phone with little memory can trade speed for a smaller footprint.
*/
import { loadModel, unloadModel, transcribe, PARAKEET_TRANSCRIBEPSY_250M_Q8_0, PARAKEET_TRANSCRIBEPSY_250M_COREML_Q8_0, PARAKEET_TRANSCRIBEPSY_250M_EU_Q8_0 } from '@qvac/sdk';
const ENCODER_FRAMES_PER_SECOND = 12.5;
const RECOMMENDED_OVERLAP = 0.25;
const EU_LANGUAGES = ['en', 'other', 'auto'];
const USAGE = 'Usage: npx tsx examples/asr/parakeet-transcribepsy-filesystem.ts <audio-file> ' +
'[--eu [--language en|other|auto]] [--coreml] [--window-sec <seconds>]';
const args = process.argv.slice(2);
const valueFlags = new Set(['--window-sec', '--language']);
const audioFilePath = args.find((arg, index) => !arg.startsWith('--') && !valueFlags.has(args[index - 1] ?? ''));
const useCoreml = args.includes('--coreml');
const useEu = args.includes('--eu');
const windowIndex = args.indexOf('--window-sec');
const windowSeconds = windowIndex === -1 ? undefined : Number(args[windowIndex + 1]);
const languageIndex = args.indexOf('--language');
const language = languageIndex === -1 ? undefined : args[languageIndex + 1];
function isEuLanguage(value) {
return EU_LANGUAGES.includes(value ?? '');
}
if (!audioFilePath ||
(windowSeconds !== undefined && !(windowSeconds > 0)) ||
(useEu && useCoreml) ||
// `--language` needs a value, and only the full EU model has a route to pick.
(languageIndex !== -1 && (!useEu || !isEuLanguage(language)))) {
console.error(USAGE);
process.exit(1);
}
// An explicit window becomes the engine's frame counts; the overlap is shared
// half and half with each neighbouring window.
const longForm = windowSeconds === undefined
? {}
: {
longFormWindowFrames: Math.round(windowSeconds * ENCODER_FRAMES_PER_SECOND),
longFormContextFrames: Math.round((windowSeconds * ENCODER_FRAMES_PER_SECOND * RECOMMENDED_OVERLAP) / 2)
};
// The EU route only means something to the full model; EU-sub has no gate.
const route = useEu && language !== undefined ? { language } : {};
const modelSrc = useEu
? PARAKEET_TRANSCRIBEPSY_250M_EU_Q8_0
: useCoreml
? PARAKEET_TRANSCRIBEPSY_250M_COREML_Q8_0
: PARAKEET_TRANSCRIBEPSY_250M_Q8_0;
const variant = useEu
? `EU, 25 languages, route ${language ?? 'auto'}`
: useCoreml
? 'EU-sub, Core ML distribution'
: 'EU-sub, ggml';
try {
console.log(`▸ Loading TranscribePsy (${variant})...`);
const modelId = await loadModel({
modelSrc,
modelType: 'parakeet-transcription',
modelConfig: { ...longForm, ...route },
onProgress: (p) => {
const mb = (n) => (n / 1e6).toFixed(1);
const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
if (p.percentage >= 100)
process.stderr.write('\n');
}
});
console.log(windowSeconds === undefined
? '▸ Transcribing with the windows the model recommends (60 s, 25 % overlap)...'
: `▸ Transcribing in ${windowSeconds} s windows...`);
const call = transcribe({ modelId, audioChunk: audioFilePath });
const text = await call;
const stats = await call.stats;
console.log(text);
if (stats) {
console.log(`▸ ${stats.audioDuration?.toFixed(1) ?? '?'} s of audio, real-time factor ${stats.realTimeFactor?.toFixed(3) ?? '?'}` +
(useCoreml ? `, encoder on Core ML: ${stats.encoderUsedCoreml === 1 ? 'yes' : 'no'}` : ''));
}
await unloadModel({ modelId });
}
catch (error) {
console.error('✖', error);
process.exit(1);
}Parakeet Indic Conformer
The following script shows an example of Indic-language transcription using the Parakeet Indic Conformer CTC model from a WAV file. Pass the path of a 16 kHz mono WAV file as the first argument and the audio's language code (e.g., hi, ta) as the second; the language is required. An optional third argument overrides the default model:
/**
* Indic Conformer CTC transcription from a WAV file.
*
* Usage:
* npx tsx examples/asr/parakeet-indic-conformer-filesystem.ts <wav-file> <language> [gguf]
*
* Loads a single GGUF checkpoint (`PARAKEET_INDIC_CONFORMER_600M_Q8_0` by
* default) and transcribes with the batch `transcribe` API. `language` is
* required (e.g. `hi`, `ta`) because Indic Conformer CTC masks the vocab
* with `parakeet.ctc.lang_*` ranges. English Parakeet CTC ignores this field.
*
* Audio should be 16 kHz mono PCM in a WAV container.
*/
import { loadModel, unloadModel, transcribe, PARAKEET_INDIC_CONFORMER_600M_Q8_0 } from '@qvac/sdk';
const args = process.argv.slice(2);
if (!args[0] || !args[1]) {
console.error('Usage: npx tsx examples/asr/parakeet-indic-conformer-filesystem.ts ' +
'<wav-file> <language> [indic-conformer-gguf]');
console.error('\nExample: ... filesystem.ts speech.wav hi');
console.error('If the model path is omitted, defaults to the registry model.');
process.exit(1);
}
const audioFilePath = args[0];
const language = args[1];
const parakeetModelSrc = args[2] ?? PARAKEET_INDIC_CONFORMER_600M_Q8_0;
try {
console.log('▸ Loading Indic Conformer CTC model...');
const modelId = await loadModel({
modelSrc: parakeetModelSrc,
modelType: 'parakeet-transcription',
modelConfig: { language },
onProgress: (p) => {
const mb = (n) => (n / 1e6).toFixed(1);
const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
if (p.percentage >= 100)
process.stderr.write('\n');
}
});
console.log(`▸ Indic Conformer CTC model loaded with ID: ${modelId}`);
console.log(`▸ Language: ${language}`);
console.log('▸ Transcribing audio...');
const text = await transcribe({ modelId, audioChunk: audioFilePath });
console.log(text);
console.log('▸ Unloading model...');
await unloadModel({ modelId });
console.log('▸ Done');
}
catch (error) {
console.error('✖', error);
process.exit(1);
}Parakeet Sortformer
The following script shows an example of speaker diarization using the Parakeet Sortformer model, followed by per-segment transcription with the TDT model. Pass the path of a 16 kHz mono 16-bit WAV file as the first argument; an optional second argument overrides the default Sortformer model. To try it with a multi-speaker recording, download diarization-sample-16k.wav next to the example and pass its path:
/**
* Parakeet Sortformer diarization + TDT transcription pipeline.
*
* Usage:
* npx tsx examples/asr/parakeet-sortformer.ts <wav-file> [sortformer-gguf]
*
* Two-step flow: Sortformer v2.1 diarizes the audio, then TDT transcribes each
* speaker segment. Defaults to registry GGUFs. For live streaming + AOSC, see
* `parakeet-sortformer-streaming.ts`.
*
* Audio should be 16 kHz mono 16-bit PCM in a WAV container. A multi-speaker
* sample is in the QVAC source repo but not the published npm package:
* https://github.com/tetherto/qvac/blob/main/packages/sdk/examples/audio/diarization-sample-16k.wav
*/
import { loadModel, unloadModel, transcribe, PARAKEET_TDT_0_6B_V3_Q8_0, PARAKEET_SORTFORMER_4SPK_V2_1_Q8_0 } from '@qvac/sdk';
import { join } from 'path';
import { readFileSync, writeFileSync, mkdirSync } from 'fs';
import { tmpdir } from 'os';
const args = process.argv.slice(2);
if (!args[0]) {
console.error('Usage: npx tsx examples/asr/parakeet-sortformer.ts <wav-file> [sortformer-gguf]');
console.error('\nIf the model path is omitted, defaults to the registry model.');
process.exit(1);
}
const audioFilePath = args[0];
const sortformerSrc = args[1] ?? PARAKEET_SORTFORMER_4SPK_V2_1_Q8_0;
try {
// ── Step 1: Diarize with Sortformer ──
const sfModelId = await loadModel({
modelSrc: sortformerSrc,
modelType: 'parakeet-transcription'
});
const diarization = await transcribe({
modelId: sfModelId,
audioChunk: audioFilePath
});
await unloadModel({ modelId: sfModelId });
const segments = parseDiarization(diarization);
// ── Step 2: Transcribe each segment with TDT ──
const tdtModelId = await loadModel({
modelSrc: PARAKEET_TDT_0_6B_V3_Q8_0
});
const pcm = readPcm(audioFilePath);
const sliceDir = join(tmpdir(), `qvac-diarize-${Date.now()}`);
mkdirSync(sliceDir, { recursive: true });
const results = [];
for (let i = 0; i < segments.length; i++) {
const seg = segments[i];
const slicePath = join(sliceDir, `seg-${i}.wav`);
if (!writeWavSlice(pcm, seg.start, seg.end, slicePath)) {
results.push({ ...seg, text: '[No speech detected]' });
continue;
}
const text = await transcribe({
modelId: tdtModelId,
audioChunk: slicePath
});
results.push({ ...seg, text: text.trim() || '[No speech detected]' });
}
await unloadModel({ modelId: tdtModelId });
// ── Step 3: Merge consecutive same-speaker segments and print ──
const merged = mergeSpeakers(results);
console.log('\n▸ Diarized transcription');
for (const entry of merged) {
console.log(`Speaker ${entry.speaker} (${entry.start.toFixed(2)}s - ${entry.end.toFixed(2)}s):`);
console.log(` ${entry.text}\n`);
}
console.log('▸ Done');
}
catch (error) {
console.error('✖', error);
process.exit(1);
}
// ── Helpers ──
// Sortformer reports each segment as `Speaker N: HH:MM:SS.mmm - HH:MM:SS.mmm`
function parseDiarization(text) {
const segs = [];
for (const line of text.split('\n')) {
const m = line.match(/Speaker (\d+): (\d+):(\d+):([\d.]+) - (\d+):(\d+):([\d.]+)/);
if (m) {
segs.push({
speaker: +m[1],
start: seconds(m[2], m[3], m[4]),
end: seconds(m[5], m[6], m[7])
});
}
}
return segs.sort((a, b) => a.start - b.start);
}
function seconds(h, m, s) {
return +h * 3600 + +m * 60 + +s;
}
function readPcm(wavPath) {
const buf = readFileSync(wavPath);
const dataOffset = buf.indexOf('data') + 4;
return buf.subarray(dataOffset + 4, dataOffset + 4 + buf.readUInt32LE(dataOffset));
}
function writeWavSlice(pcm, startSec, endSec, outPath) {
const SR = 16000;
const BPS = 2;
const startByte = Math.floor(startSec * SR) * BPS;
const endByte = Math.min(Math.ceil(endSec * SR) * BPS, pcm.length);
if (startByte >= endByte)
return false;
const slice = pcm.subarray(startByte, endByte);
const hdr = Buffer.alloc(44);
hdr.write('RIFF', 0);
hdr.writeUInt32LE(36 + slice.length, 4);
hdr.write('WAVEfmt ', 8);
hdr.writeUInt32LE(16, 16);
hdr.writeUInt16LE(1, 20);
hdr.writeUInt16LE(1, 22);
hdr.writeUInt32LE(SR, 24);
hdr.writeUInt32LE(SR * BPS, 28);
hdr.writeUInt16LE(BPS, 32);
hdr.writeUInt16LE(16, 34);
hdr.write('data', 36);
hdr.writeUInt32LE(slice.length, 40);
writeFileSync(outPath, Buffer.concat([hdr, slice]));
return true;
}
function mergeSpeakers(entries) {
const out = [];
for (const e of entries) {
const last = out[out.length - 1];
if (last && last.speaker === e.speaker) {
last.text += ' ' + e.text;
last.end = e.end;
}
else {
out.push({ ...e });
}
}
return out;
}Tip: all examples throughout this documentation are self-contained and runnable. For instructions on how to run them, see the JS/TS quickstart, the Python quickstart, or the Kotlin quickstart.