QVAC Logo

BCI

Brain–computer interface (BCI) transcription — decode multi-channel neural signals into text.

Overview

Brain–computer interface (BCI) transcription uses a GGML engine (@qvac/bci-whispercpp, built on the qvac-ext-lib-whisper.cpp fork) to decode multi-channel neural signals (e.g., 512-channel microelectrode-array recordings) into text. Load a model using modelType: "bci".

BCI consumes a neural-signal buffer. Provide the signal as neuralData, either as a file path (string, to a .bin file) or an in-memory Uint8Array buffer.

bciTranscribe() returns the complete transcript as a string, or — with metadata: true (as in the batch example below) — an array of segments, each with text and startMs/endMs timing. If you need partial results as they become available, use bciTranscribeStream() to open a duplex session that decodes a sliding window over the signal and yields transcript text as data arrives.

Functions

Use the following sequence of function calls:

  1. loadModel()
  2. bciTranscribe() or bciTranscribeStream()
  3. unloadModel()

For how to use each function, see SDK — API reference.

Neural signal format: the neuralData input is a binary .bin file (or equivalent Uint8Array) with an 8-byte header — [timesteps: uint32 LE, channels: uint32 LE] — followed by row-major float32 feature data (features[t * channels + c]). Each timestep represents a 20 ms bin of neural activity; channels correspond to individual electrodes in a microelectrode array (typically 512 channels). When streaming, this same header must lead the written bytes (timesteps is then ignored; channels must be non-zero).

On the day index: sessions recorded on different days use different day-specific projections. Set modelConfig.bciConfig.day_idx in loadModel() to match the recording session of the input signal — otherwise the decoded text will be misleading.

Models

BCI transcription loads a companion set of two files, both required for inference:

  • ggml-bci-windowed.bin — the GGML model: Whisper encoder/decoder (LoRA-merged), tokenizer, positional embedding, and windowed-attention header. Available constant: BCI_WINDOWED.
  • bci-embedder.bin — day-projection weights: per-recording-day low-rank matrices, month projections, and session-to-day mapping. Available constant: BCI_EMBEDDER.

For model artifacts available as constants, see SDK — Models.

Examples

Batch

The following script transcribes a full neural signal up-front and prints the decoded text:

bci-filesystem.js
/**
 * Batch BCI transcription from a neural-signal file.
 *
 * Reads a raw neural-signal `.bin` file, runs it through the BCI
 * (whisper.cpp) addon in one shot via `bciTranscribe`, and prints the
 * decoded transcript.
 *
 * Usage: bun run examples/bci/bci-filesystem.ts <neural-bin-file-path>
 */
import { loadModel, unloadModel, bciTranscribe, BCI_WINDOWED } from "@qvac/sdk";
const args = process.argv.slice(2);
if (!args[0]) {
    console.error("Usage: bun run examples/bci/bci-filesystem.ts <neural-bin-file-path>");
    process.exit(1);
}
const neuralFilePath = args[0];
try {
    console.log("🧠 Starting BCI transcription example...");
    console.log("📥 Loading BCI model...");
    const modelId = await loadModel({
        modelSrc: BCI_WINDOWED,
        modelConfig: {
            whisperConfig: {
                language: "en",
                n_threads: 4,
                temperature: 0.0,
            },
            // Session day index selects the day-specific projection matrices.
            // Set this to match the recording session your neural file came from.
            bciConfig: {
                day_idx: 1,
            },
        },
        onProgress: (progress) => {
            console.log(progress);
        },
    });
    console.log(`✅ BCI model loaded with ID: ${modelId}`);
    console.log("🧠 Transcribing neural signal...");
    const segments = await bciTranscribe({
        modelId,
        neuralData: neuralFilePath,
        metadata: true,
    });
    console.log("📝 Transcription result:");
    for (const segment of segments) {
        const start = (segment.startMs / 1000).toFixed(2);
        const end = (segment.endMs / 1000).toFixed(2);
        console.log(`  [${start}s → ${end}s] (id=${segment.id}, append=${segment.append}) ${segment.text}`);
    }
    console.log(`\nFull transcript: ${segments
        .map((s) => s.text)
        .join("")
        .trim()}`);
    console.log("🧹 Unloading BCI model...");
    await unloadModel({ modelId });
    console.log("✅ BCI model unloaded successfully");
    process.exit(0);
}
catch (error) {
    console.error("❌ Error:", error);
    process.exit(1);
}

Streaming

The following script feeds neural-signal chunks into a duplex session and prints transcript text as it is decoded:

bci-filesystem-streaming.js
/**
 * Streaming BCI transcription from a neural-signal file.
 *
 * Reads a raw neural-signal `.bin` file and feeds it to the BCI
 * (whisper.cpp) addon chunk-by-chunk through a duplex `bciTranscribeStream`
 * session, printing transcript text as the sliding window decodes
 * successive windows.
 *
 * Usage: bun run examples/bci/bci-filesystem-streaming.ts <neural-bin-file-path>
 */
import { loadModel, unloadModel, bciTranscribeStream, BCI_WINDOWED, } from "@qvac/sdk";
import { readFileSync } from "fs";
const args = process.argv.slice(2);
if (!args[0]) {
    console.error("Usage: bun run examples/bci/bci-filesystem-streaming.ts <neural-bin-file-path>");
    process.exit(1);
}
const neuralFilePath = args[0];
// Feed the neural buffer in fixed-size chunks to simulate a live stream.
const CHUNK_SIZE = 64 * 1024;
try {
    console.log("=== BCI transcribeStream file test ===");
    console.log(`File: ${neuralFilePath}`);
    console.log(`Chunk size: ${CHUNK_SIZE} bytes\n`);
    console.log("Loading model...");
    const modelId = await loadModel({
        modelSrc: BCI_WINDOWED,
        modelConfig: {
            whisperConfig: {
                language: "en",
                n_threads: 4,
                temperature: 0.0,
            },
            // Session day index selects the day-specific projection matrices.
            // Set this to match the recording session your neural file came from.
            bciConfig: {
                day_idx: 1,
            },
        },
    });
    console.log(`Model loaded: ${modelId}\n`);
    console.log("Opening live session...");
    const session = await bciTranscribeStream({ modelId, emit: "delta" });
    console.log("Session open. Streaming neural signal...\n");
    // Drain the session concurrently with writing so the sliding-window
    // decode can make progress as chunks arrive instead of stalling.
    const consume = (async () => {
        let transcript = "";
        for await (const text of session) {
            transcript += text;
            process.stdout.write(text);
        }
        return transcript;
    })();
    const data = readFileSync(neuralFilePath);
    let totalBytes = 0;
    for (let offset = 0; offset < data.length; offset += CHUNK_SIZE) {
        const chunk = data.subarray(offset, offset + CHUNK_SIZE);
        session.write(chunk);
        totalBytes += chunk.length;
        await new Promise((resolve) => setTimeout(resolve, 10));
    }
    console.log(`\n\nNeural signal streamed: ${totalBytes} bytes`);
    console.log("Waiting for transcription to finish...\n");
    session.end();
    const transcript = await consume;
    console.log("\n=== Results ===");
    console.log(`Transcript: ${transcript.trim() || "(no text received)"}`);
    console.log("\nUnloading model...");
    await unloadModel({ modelId });
    console.log("Done.");
    process.exit(0);
}
catch (error) {
    console.error("❌ Error:", error);
    process.exit(1);
}

Tip: all examples throughout this documentation are self-contained and runnable. For instructions on how to run them, see SDK quickstart.

On this page

Ask anything about QVAC.