QVAC Logo

BCI

Brain–computer interface (BCI) transcription — decode multi-channel neural signals into text.

Overview

Brain–computer interface (BCI) transcription uses a GGML engine (@qvac/bci-whispercpp, built on the qvac-ext-lib-whisper.cpp fork) to decode multi-channel neural signals (e.g., 512-channel microelectrode-array recordings) into text. Load a model using modelType: "bci".

BCI consumes a neural-signal buffer. Provide the signal as neuralData, either as a file path (string, to a .bin file) or an in-memory Uint8Array buffer.

bciTranscribe() returns the complete transcript as a string, or — with metadata: true (as in the batch example below) — an array of segments, each with text and startMs/endMs timing. If you need partial results as they become available, use bciTranscribeStream() to open a duplex session that decodes a sliding window over the signal and yields transcript text as data arrives.

Functions

Use the following sequence of function calls:

  1. loadModel()
  2. bciTranscribe() or bciTranscribeStream()
  3. unloadModel()

For how to use each function, see SDK — API reference.

Neural signal format: the neuralData input is a binary .bin file (or equivalent Uint8Array) with an 8-byte header — [timesteps: uint32 LE, channels: uint32 LE] — followed by row-major float32 feature data (features[t * channels + c]). Each timestep represents a 20 ms bin of neural activity; channels correspond to individual electrodes in a microelectrode array (typically 512 channels). When streaming, this same header must lead the written bytes (timesteps is then ignored; channels must be non-zero).

On the day index: sessions recorded on different days use different day-specific projections. Set modelConfig.bciConfig.day_idx in loadModel() to match the recording session of the input signal — otherwise the decoded text will be misleading.

Models

BCI transcription loads a companion set of two files, both required for inference:

  • ggml-bci-windowed.bin — the GGML model: Whisper encoder/decoder (LoRA-merged), tokenizer, positional embedding, and windowed-attention header. Available constant: BCI_WINDOWED.
  • bci-embedder.bin — day-projection weights: per-recording-day low-rank matrices, month projections, and session-to-day mapping. Available constant: BCI_EMBEDDER.

For model artifacts available as constants, see SDK — Models.

Examples

Batch

The following script transcribes a full neural signal up-front and prints the decoded text:

bci-filesystem.js
/**
 * Batch BCI transcription from a neural-signal file.
 *
 * Reads a raw neural-signal `.bin` file, runs it through the BCI
 * (whisper.cpp) addon in one shot via `bciTranscribe`, and prints the
 * decoded transcript.
 *
 * Usage: bun run examples/bci/bci-filesystem.ts <neural-bin-file-path>
 */
import { loadModel, unloadModel, bciTranscribe, BCI_WINDOWED } from '@qvac/sdk';
const args = process.argv.slice(2);
if (!args[0]) {
    console.error('Usage: bun run examples/bci/bci-filesystem.ts <neural-bin-file-path>');
    process.exit(1);
}
const neuralFilePath = args[0];
try {
    console.log('▸ Starting BCI transcription example...');
    console.log('▸ Loading BCI model...');
    const modelId = await loadModel({
        modelSrc: BCI_WINDOWED,
        modelConfig: {
            whisperConfig: {
                language: 'en',
                n_threads: 4,
                temperature: 0.0
            },
            // Session day index selects the day-specific projection matrices.
            // Set this to match the recording session your neural file came from.
            bciConfig: {
                day_idx: 1
            }
        },
        onProgress: (p) => {
            const mb = (n) => (n / 1e6).toFixed(1);
            const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
            process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
            if (p.percentage >= 100)
                process.stderr.write('\n');
        }
    });
    console.log(`▸ BCI model loaded with ID: ${modelId}`);
    console.log('▸ Transcribing neural signal...');
    const segments = await bciTranscribe({
        modelId,
        neuralData: neuralFilePath,
        metadata: true
    });
    console.log('▸ Transcription result:');
    for (const segment of segments) {
        const start = (segment.startMs / 1000).toFixed(2);
        const end = (segment.endMs / 1000).toFixed(2);
        console.log(`  [${start}s → ${end}s] (id=${segment.id}, append=${segment.append}) ${segment.text}`);
    }
    console.log(segments
        .map((s) => s.text)
        .join('')
        .trim());
    console.log('▸ Unloading BCI model...');
    await unloadModel({ modelId });
    console.log('▸ BCI model unloaded successfully');
    process.exit(0);
}
catch (error) {
    console.error('✖', error);
    process.exit(1);
}

Streaming

The following script feeds neural-signal chunks into a duplex session and prints transcript text as it is decoded:

bci-filesystem-streaming.js
/**
 * Streaming BCI transcription from a neural-signal file.
 *
 * Reads a raw neural-signal `.bin` file and feeds it to the BCI
 * (whisper.cpp) addon chunk-by-chunk through a duplex `bciTranscribeStream`
 * session, printing transcript text as the sliding window decodes
 * successive windows.
 *
 * Usage: bun run examples/bci/bci-filesystem-streaming.ts <neural-bin-file-path>
 */
import { loadModel, unloadModel, bciTranscribeStream, BCI_WINDOWED } from '@qvac/sdk';
import { readFileSync } from 'fs';
const args = process.argv.slice(2);
if (!args[0]) {
    console.error('Usage: bun run examples/bci/bci-filesystem-streaming.ts <neural-bin-file-path>');
    process.exit(1);
}
const neuralFilePath = args[0];
// Feed the neural buffer in fixed-size chunks to simulate a live stream.
const CHUNK_SIZE = 64 * 1024;
try {
    console.log('▸ BCI transcribeStream file test');
    console.log(`▸ File: ${neuralFilePath}`);
    console.log(`▸ Chunk size: ${CHUNK_SIZE} bytes`);
    console.log('▸ Loading model...');
    const modelId = await loadModel({
        modelSrc: BCI_WINDOWED,
        modelConfig: {
            whisperConfig: {
                language: 'en',
                n_threads: 4,
                temperature: 0.0
            },
            // Session day index selects the day-specific projection matrices.
            // Set this to match the recording session your neural file came from.
            bciConfig: {
                day_idx: 1
            }
        }
    });
    console.log(`▸ Model loaded: ${modelId}`);
    console.log('▸ Opening live session...');
    const session = await bciTranscribeStream({ modelId, emit: 'delta' });
    console.log('▸ Session open. Streaming neural signal...');
    // Drain the session concurrently with writing so the sliding-window
    // decode can make progress as chunks arrive instead of stalling.
    const consume = (async () => {
        let transcript = '';
        for await (const text of session) {
            transcript += text;
            process.stdout.write(text);
        }
        return transcript;
    })();
    const data = readFileSync(neuralFilePath);
    let totalBytes = 0;
    for (let offset = 0; offset < data.length; offset += CHUNK_SIZE) {
        const chunk = data.subarray(offset, offset + CHUNK_SIZE);
        session.write(chunk);
        totalBytes += chunk.length;
        await new Promise((resolve) => setTimeout(resolve, 10));
    }
    console.log(`\n\n▸ Neural signal streamed: ${totalBytes} bytes`);
    console.log('▸ Waiting for transcription to finish...');
    session.end();
    const transcript = await consume;
    console.log('\n▸ Results');
    console.log(transcript.trim() || '(no text received)');
    console.log('▸ Unloading model...');
    await unloadModel({ modelId });
    console.log('▸ Done.');
    process.exit(0);
}
catch (error) {
    console.error('✖', error);
    process.exit(1);
}

Tip: all examples throughout this documentation are self-contained and runnable. For instructions on how to run them, see SDK quickstart.

On this page

Ask anything about QVAC.