Image classification

Overview

Image classification uses a GGML inference engine (@qvac/classification-ggml). Load a model using modelType: "classification". The addon ships with a bundled MobileNetV3-Small that classifies images into three labels — "food", "report", and "other" — so no modelSrc and no model download are required out of the box. Custom GGUF classifiers are supported by passing your own modelSrc.

Provide an image to classify() as a Uint8Array of either:

• an encoded JPEG or PNG buffer; or
• raw RGB bytes, alongside width, height, and channels: 3.

classify() returns ClassificationResult[] — an array of { label, confidence } entries sorted by confidence in descending order. Use topK to limit the number of results returned, either as a load-time default (modelConfig.topK) or as a per-call override.

Functions

Use the following sequence of function calls:

1. loadModel()
2. classify()
3. unloadModel()

For how to use each function, see SDK — API reference.

Models

Supported model families and their file layouts:

• MobileNetV3-Small: single all-in-one *.gguf file — the base model or any fine-tune of the same architecture (converted to GGUF). Fine-tunes may define their own classes and labels; the label set is sourced from the GGUF metadata.

For models available as constants, see SDK — Models.

Example

The following script classifies a JPEG image using the bundled MobileNetV3-Small model:

import fs from "fs";
import { startQVACProvider, stopQVACProvider, loadModel, classify, unloadModel } from "@qvac/sdk";
/**
 * Classify an image using the bundled MobileNetV3-Small model.
 *
 * The bundled model produces three classes: "food", "report", "other".
 * No modelSrc is needed — the model ships inside @qvac/classification-ggml.
 */
async function main() {
    await startQVACProvider({});
    const modelId = await loadModel({
        modelType: "ggml-classification",
    });
    const image = fs.readFileSync("image.jpg");
    const results = await classify({ modelId, image });
    console.log("Classification results:");
    for (const { label, confidence } of results) {
        console.log(`  ${label}: ${(confidence * 100).toFixed(1)}%`);
    }
    await unloadModel({ modelId });
    await stopQVACProvider();
}
main().catch(console.error);