New: TranslatePsy-AfriSLM translates directly between 19 African languages, offline.
QVAC Logo
SDKBCI
v0.21, the current release

BCI

Brain–computer interface (BCI) transcription — decode multi-channel neural signals into text.

Overview

Brain–computer interface (BCI) transcription uses a GGML engine (@qvac/bci-whispercpp, built on the qvac-fabric-speech.cpp fork) to decode multi-channel neural signals (e.g., 512-channel microelectrode-array recordings) into text. Load a model using modelType: "bci".

BCI consumes a neural-signal buffer. Provide the signal as neuralData, either as a file path (string, to a .bin file) or an in-memory Uint8Array buffer.

bciTranscribe() returns the complete transcript as a string, or — with metadata: true (as in the batch example below) — an array of segments, each with text and startMs/endMs timing. If you need partial results as they become available, use bciTranscribeStream() to open a duplex session that decodes a sliding window over the signal and yields transcript text as data arrives.

Functions

Use the following sequence of function calls:

  1. loadModel()
  2. bciTranscribe() or bciTranscribeStream()
  3. unloadModel()

For how to use each function, see SDK — API reference.

Neural signal format: the neuralData input is a binary .bin file (or equivalent Uint8Array) with an 8-byte header — [timesteps: uint32 LE, channels: uint32 LE] — followed by row-major float32 feature data (features[t * channels + c]). Each timestep represents a 20 ms bin of neural activity; channels correspond to individual electrodes in a microelectrode array (typically 512 channels). When streaming, this same header must lead the written bytes (timesteps is then ignored; channels must be non-zero).

Placing streamed segments on the timeline: with metadata: true and the default emit: 'delta', each streamed segment also carries windowStartTimestep, the absolute index of the 20 ms timestep where its decode window began. A segment's own startMs / endMs are window-local, so compute the stream position as startMs + windowStartTimestep * 20 (and the same for endMs). Batch segments do not carry the field: their timings are already absolute.

Where run stats live: stats and backend diagnostics ride the terminal frame of a batch bciTranscribe call, so RPC consumers (such as the Python client, which yields raw frames) can read them. The JavaScript bciTranscribe() resolves to text or segments only, and the streaming sessions expose neither, because the addon reports no stats for a stream.

On the day index: sessions recorded on different days use different day-specific projections. Set modelConfig.bciConfig.day_idx in loadModel() to match the recording session of the input signal — otherwise the decoded text will be misleading.

Models

BCI transcription loads a companion set of two files, both required for inference:

  • ggml-bci-windowed.bin — the GGML model: Whisper encoder/decoder (LoRA-merged), tokenizer, positional embedding, and windowed-attention header. Available constant: BCI_WINDOWED.
  • bci-embedder.bin — day-projection weights: per-recording-day low-rank matrices, month projections, and session-to-day mapping. Available constant: BCI_EMBEDDER.

For model artifacts available as constants, see SDK — Models.

Examples

Batch

The following script transcribes a full neural signal up-front and prints the decoded text:

bci-filesystem.js
/**
 * Batch BCI transcription from a neural-signal file.
 *
 * Reads a raw neural-signal `.bin` file, runs it through the BCI
 * (whisper.cpp) addon in one shot via `bciTranscribe`, and prints the
 * decoded transcript.
 *
 * Usage: bun run examples/bci/bci-filesystem.ts <neural-bin-file-path>
 */
import { loadModel, unloadModel, bciTranscribe, BCI_WINDOWED } from '@qvac/sdk';
const args = process.argv.slice(2);
if (!args[0]) {
    console.error('Usage: bun run examples/bci/bci-filesystem.ts <neural-bin-file-path>');
    process.exit(1);
}
const neuralFilePath = args[0];
try {
    console.log('▸ Starting BCI transcription example...');
    console.log('▸ Loading BCI model...');
    const modelId = await loadModel({
        modelSrc: BCI_WINDOWED,
        modelConfig: {
            whisperConfig: {
                language: 'en',
                n_threads: 4,
                temperature: 0.0
            },
            // Session day index selects the day-specific projection matrices.
            // Set this to match the recording session your neural file came from.
            bciConfig: {
                day_idx: 1
            }
        },
        onProgress: (p) => {
            const mb = (n) => (n / 1e6).toFixed(1);
            const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
            process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
            if (p.percentage >= 100)
                process.stderr.write('\n');
        }
    });
    console.log(`▸ BCI model loaded with ID: ${modelId}`);
    console.log('▸ Transcribing neural signal...');
    const segments = await bciTranscribe({
        modelId,
        neuralData: neuralFilePath,
        metadata: true
    });
    console.log('▸ Transcription result:');
    for (const segment of segments) {
        const start = (segment.startMs / 1000).toFixed(2);
        const end = (segment.endMs / 1000).toFixed(2);
        console.log(`  [${start}s → ${end}s] (id=${segment.id}, append=${segment.append}) ${segment.text}`);
    }
    console.log(segments
        .map((s) => s.text)
        .join('')
        .trim());
    console.log('▸ Unloading BCI model...');
    await unloadModel({ modelId });
    console.log('▸ BCI model unloaded successfully');
    process.exit(0);
}
catch (error) {
    console.error('✖', error);
    process.exit(1);
}

Streaming

The following script feeds neural-signal chunks into a duplex session and prints transcript text as it is decoded:

bci-filesystem-streaming.js
/**
 * Streaming BCI transcription from a neural-signal file.
 *
 * Reads a raw neural-signal `.bin` file and feeds it to the BCI
 * (whisper.cpp) addon chunk-by-chunk through a duplex `bciTranscribeStream`
 * session, printing transcript text as the sliding window decodes
 * successive windows.
 *
 * Usage: bun run examples/bci/bci-filesystem-streaming.ts <neural-bin-file-path>
 */
import { loadModel, unloadModel, bciTranscribeStream, BCI_WINDOWED } from '@qvac/sdk';
import { readFileSync } from 'fs';
const args = process.argv.slice(2);
if (!args[0]) {
    console.error('Usage: bun run examples/bci/bci-filesystem-streaming.ts <neural-bin-file-path>');
    process.exit(1);
}
const neuralFilePath = args[0];
// Feed the neural buffer in fixed-size chunks to simulate a live stream.
const CHUNK_SIZE = 64 * 1024;
try {
    console.log('▸ BCI transcribeStream file test');
    console.log(`▸ File: ${neuralFilePath}`);
    console.log(`▸ Chunk size: ${CHUNK_SIZE} bytes`);
    console.log('▸ Loading model...');
    const modelId = await loadModel({
        modelSrc: BCI_WINDOWED,
        modelConfig: {
            whisperConfig: {
                language: 'en',
                n_threads: 4,
                temperature: 0.0
            },
            // Session day index selects the day-specific projection matrices.
            // Set this to match the recording session your neural file came from.
            bciConfig: {
                day_idx: 1
            }
        }
    });
    console.log(`▸ Model loaded: ${modelId}`);
    console.log('▸ Opening live session...');
    const session = await bciTranscribeStream({ modelId, emit: 'delta' });
    console.log('▸ Session open. Streaming neural signal...');
    // Drain the session concurrently with writing so the sliding-window
    // decode can make progress as chunks arrive instead of stalling.
    const consume = (async () => {
        let transcript = '';
        for await (const text of session) {
            transcript += text;
            process.stdout.write(text);
        }
        return transcript;
    })();
    const data = readFileSync(neuralFilePath);
    let totalBytes = 0;
    for (let offset = 0; offset < data.length; offset += CHUNK_SIZE) {
        const chunk = data.subarray(offset, offset + CHUNK_SIZE);
        session.write(chunk);
        totalBytes += chunk.length;
        await new Promise((resolve) => setTimeout(resolve, 10));
    }
    console.log(`\n\n▸ Neural signal streamed: ${totalBytes} bytes`);
    console.log('▸ Waiting for transcription to finish...');
    session.end();
    const transcript = await consume;
    console.log('\n▸ Results');
    console.log(transcript.trim() || '(no text received)');
    console.log('▸ Unloading model...');
    await unloadModel({ modelId });
    console.log('▸ Done.');
    process.exit(0);
}
catch (error) {
    console.error('✖', error);
    process.exit(1);
}

The Python client supports this capability through the same worker. A dedicated Python example is not yet published — see the Python SDK for the API surface.

Tip: all examples throughout this documentation are self-contained and runnable. For instructions on how to run them, see the JS/TS quickstart or the Python quickstart.

On this page

Ask anything about QVAC.