--- title: "Plugin system" canonical: https://docs.qvac.tether.io/sdk/configuration/plugins/ collection: "SDK" package: "@qvac/sdk" line: v0.21 current_line: true --- # Plugin system (/sdk/configuration/plugins) ## Overview Each QVAC AI capability maps to a **built-in plugin** in the SDK. This lets you enable only what you need for your project and reduce your application's final bundle size. In addition, you can add **custom plugins** — both your own and community ones — that extend QVAC's capabilities. In both cases, you'll use the QVAC configuration file and then bundle either with [QVAC CLI](/cli) or [programmatically via `@qvac/sdk/commands`](#notes). ## Built-in plugins ### Catalog All the built-in plugins you can select, along with the AI tasks that depend on each one: | Plugin | Use in `qvac.config.*` | AI tasks that require it | | ---------------------------------- | ----------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | LLM | `@qvac/sdk/llamacpp-completion/plugin` | [Text generation](/sdk/ai-capabilities/text-generation); [multimodal](/sdk/ai-capabilities/multimodal); [RAG](/sdk/ai-capabilities/rag) ; [Fine-tuning](/sdk/ai-capabilities/fine-tuning) | | Embeddings | `@qvac/sdk/llamacpp-embedding/plugin` | [Text embeddings](/sdk/ai-capabilities/text-embeddings); [RAG](/sdk/ai-capabilities/rag) | | ASR with customized Whisper engine | `@qvac/sdk/whispercpp-transcription/plugin` | [Transcription](/sdk/ai-capabilities/transcription) | | ASR with Parakeet | `@qvac/sdk/parakeet-transcription/plugin` | [Transcription](/sdk/ai-capabilities/transcription) | | BCI transcription | `@qvac/sdk/bci-whispercpp-transcription/plugin` | [Transcription](/sdk/ai-capabilities/transcription#bci-neural-signal-transcription) | | BCI transcription | `@qvac/sdk/bci-whispercpp-transcription/plugin` | [Transcription](/sdk/ai-capabilities/transcription#bci-neural-signal-transcription) | | NMT | `@qvac/sdk/nmtcpp-translation/plugin` | [Translation](/sdk/ai-capabilities/translation) | | TTS | `@qvac/sdk/onnx-tts/plugin` | [Text-to-Speech](/sdk/ai-capabilities/text-to-speech) | | OCR | `@qvac/sdk/ggml-ocr/plugin` | [OCR](/sdk/ai-capabilities/ocr) | | Diffusion | `@qvac/sdk/sdcpp-generation/plugin` | [Image generation](/sdk/ai-capabilities/image-generation) | | AudioGen | `@qvac/sdk/audiogen-ggml/plugin` | [Music generation](/sdk/ai-capabilities/music-generation) | ### Enabling [In your `qvac.config.*`, add the built-in plugins you’ll need in your project](/sdk/configuration). For example: ```json title="qvac.config.json" { "plugins": [ "@qvac/sdk/llamacpp-completion/plugin", "@qvac/sdk/ggml-ocr/plugin" ] } ``` When developing for desktop environment, use the QVAC CLI to (re)bundle the SDK only with the selected plugins: ```bash qvac bundle sdk ``` [When developing for mobile environment, use the Expo CLI to prebuild your project only with the selected plugins:](/sdk/js-ts-sdk#expo) ```bash npx expo prebuild ``` Only the selected plugins are included in your bundle, significantly reducing your application size. If `plugins` is omitted or an empty array, it bundles all built-in plugins by default. **Tips:** * You **do not need** the CLI (`@qvac/cli`) to bundle the SDK. See [Notes](#notes) to learn how to do it programmatically via `@qvac/sdk/commands`. * See [QVAC CLI](/cli) to learn how to install and use it. * When developing an Electron app, you can bundle the SDK during packaging via Electron Forge. See [Tutorials → Electron → Step 4: package for distribution](/resources/tutorials/electron#step-4-package-for-distribution). ## Custom plugins Custom plugins are consumed as npm packages. Install the package, enable it in your `qvac.config.*` file like any built-in plugin, then import and use its API alongside the SDK's regular API. ### Enabling Install the plugin package in your project. For example: ```bash npm i qvac-echo-plugin ``` [In your `qvac.config.*`, add its `/plugin` specifier alongside any built-in plugins](/sdk/configuration). For example: ```json title="qvac.config.json" { "plugins": [ "@qvac/sdk/llamacpp-completion/plugin", "qvac-echo-plugin/plugin" // [!code highlight] ] } ``` When developing for desktop environment, use the SDK CLI to (re)bundle the SDK with all listed plugins: ```bash qvac bundle sdk ``` [When developing for mobile environment, use the Expo CLI to prebuild your project with all listed plugins:](/sdk/js-ts-sdk#expo) ```bash npx expo prebuild ``` **Tip:** when adding one or more custom plugins, you **must** also add **all** the built-in plugins you will need to use. ### Usage Custom plugins are consumed as npm packages. Install the package, enable it in your `qvac.config.*` file like any built-in plugin, then import and use its API alongside the SDK’s regular API. ```ts import { loadModel, unloadModel } from "@qvac/sdk"; // 1. Import the API you need from the custom plugin package: import { echo, echoStream } from "qvac-echo-plugin"; // 2. Load the model(s) required by the custom plugin: const modelId = await loadModel({ modelSrc: "/path/to/echo-model.bin", modelType: "echo", }); // 3. Call functions exposed by the custom plugin API: const result = await echo({ modelId, message: "Hello, plugin system!" }); console.log(result); for await (const char of echoStream({ modelId, message: "Streaming test!" })) { process.stdout.write(char); } await unloadModel({ modelId }); ``` The Python SDK exposes plugin invocation through the same worker contract: ```python file=/packages/sdk-python/examples/plugins.py title="plugins.py" lineNumbers """Python port of packages/sdk/examples/plugins.ts (plugin invocation). The low-level plugin API. A model's `modelType` selects a plugin; each plugin exposes named `handler`s with their own request/response schemas. `invoke_plugin(t, model_id, handler, params)` runs one handler and returns its result; `invoke_plugin_stream(...)` yields streamed chunks. This example uses the `custom-echo-plugin` fixture (modelType `echo`), whose handlers are `echo` ({message} -> {message}) and `echoStream` ({message} -> {chunk} chunks). To run it you need a worker bundled with that plugin; point the loader at it via argv, or adapt the handler/params to your own plugin. (A production model that exposes handlers is VLA — see vla.py, which drives the `vlaRun` / `vlaHparams` handlers through typed wrappers.) RUN: python examples/plugins.py [model-src] """ from __future__ import annotations import asyncio import sys from tetherto.qvac_sdk import ( Client, invoke_plugin, invoke_plugin_stream, load_model, unload_model, ) def print_progress(p) -> None: """Print model download progress; pass as `on_progress=` to `load_model`.""" line = ( f"▸ Downloading {p.percentage:.0f}% " f"({p.downloaded / 1e6:.1f}/{p.total / 1e6:.1f} MB)" ) print(line, end="\r" if sys.stderr.isatty() else "\n", file=sys.stderr) if p.percentage >= 100: print(file=sys.stderr) async def main() -> int: if len(sys.argv) < 2: print( "Usage: python examples/plugins.py \n" " must resolve to a model whose plugin exposes the " "'echo'/'echoStream' handlers (the custom-echo-plugin fixture).", file=sys.stderr, ) return 1 model_src = sys.argv[1] async with Client() as client: t = client.transport try: print("▸ Loading plugin-backed model...") model_id = await load_model( t, model_src=model_src, model_type="echo", on_progress=print_progress, ) print(f"▸ Model loaded: {model_id}") print("\n▸ 1. invoke_plugin (request/reply handler 'echo')") result = await invoke_plugin( t, model_id, "echo", params={"message": "hello from the python sdk"} ) print(f"▸ echo -> {result}") print("\n▸ 2. invoke_plugin_stream (streaming handler 'echoStream')") print("▸ chunks: ", end="") async for chunk in invoke_plugin_stream( t, model_id, "echoStream", params={"message": "streamed message"} ): sys.stdout.write(str(getattr(chunk, "chunk", chunk))) sys.stdout.flush() print() await unload_model(t, model_id) print("▸ Model unloaded") except Exception as error: print(f"✖ {error}", file=sys.stderr) return 1 return 0 if __name__ == "__main__": sys.exit(asyncio.run(main())) ``` ## Runtime registration on Bare The `plugins` array in `qvac.config.*` is **bundle-time** configuration — it controls which addons get packed into your worker. That is separate from **runtime registration**, which determines the plugins live in the worker process when an SDK call runs. On Node.js and Expo the SDK spawns a worker that auto-registers the full built-in set, so you never register manually. In-process Bare uses `@qvac/inference` with no spawned worker — register the plugins you use before the first call: ```js import { plugins } from "@qvac/inference"; import { llmPlugin } from "@qvac/inference/llamacpp-completion/plugin"; const sdk = plugins([llmPlugin]); // or registerPlugin(llmPlugin) from "@qvac/inference/plugins" ``` Calls made before any plugin is registered raise `PluginsNotRegisteredError`. In-process Bare uses [`@qvac/inference`](https://github.com/tetherto/qvac/tree/main/packages/inference). `@qvac/bare-sdk` is deprecated; last release is 0.18.2. ## Notes * In `qvac.config.*`, if `plugins` is omitted or set to an empty array, the SDK bundles all built-in plugins. If `plugins` is set, it bundles only the listed plugins. * [Each plugin maps to one QVAC addon](https://github.com/tetherto/qvac/tree/main/packages). * [Write a custom plugin](/sdk/configuration/plugins/write-custom-plugin). * **Programmatic bundling:** you can also prepare and validate the SDK bundle programmatically via API from a Node script. For example: ```ts import { bundleSdk, verifyBundle } from "@qvac/sdk/commands"; await bundleSdk({ projectRoot: process.cwd(), configPath: "./qvac.config.json", quiet: true, }); const result = await verifyBundle({ projectRoot: process.cwd(), addonsSource: "./qvac/worker.bundle.js", hosts: ["android-arm64", "ios-arm64"], }); ``` * **Addon platform packages:** QVAC ships part of its Android and iOS binaries in separate packages, for the speech addons (`tts-ggml`, `asr-ggml`, `audiogen-ggml`) and for `@qvac/fabric`, the runtime the other native addons share. They must be installed at an exact version. Pass `installMissingPrebuilds: true` to `bundleSdk` to add the missing ones to `package.json` with the project's package manager (npm 7+, pnpm, bun, or Yarn Berry) and bundle again. The package manager is read from the project's `packageManager` field, lockfile, or installed `node_modules`; a workspace member uses its workspace root's. If none of those identify it, nothing is installed and the error lists the dependencies to add. It is off by default, so a plain `bundleSdk` call never changes `package.json` or `node_modules`. `ensureHostPrebuilds({ projectRoot, hosts, addons })` runs the install step on its own.