---
title: "Image embeddings"
canonical: https://docs.qvac.tether.io/sdk/ai-capabilities/image-embeddings/
collection: "SDK"
package: "@qvac/sdk"
line: v0.24
current_line: true
---

# Image embeddings (/sdk/ai-capabilities/image-embeddings)



## Overview

Image embeddings use a GGML inference engine through [`@qvac/classification-ggml`](https://github.com/tetherto/qvac/tree/main/packages/classification-ggml). Load MegaLoc with `modelSrc: MEGALOC_F16` or `modelSrc: MEGALOC_MIXED`. For custom weights, use `modelType: "ggml-image-embedding"` with a GGUF URL or local file as `modelSrc`.

Pass one JPEG or PNG as `input: { image: Uint8Array }` to [`embed()`](/sdk/reference/api#embed). Use [`embedBatch()`](/sdk/reference/api#embedbatch) with `inputs: [{ image: firstBytes }, { image: secondBytes }]` for multiple images. For packed RGB bytes, also supply `width`, `height`, and `channels: 3`. Both dimensions must be integers from 1 to 16384, and the buffer must contain exactly `width * height * 3` bytes. Omit all three fields for encoded images. The addon converts the image to RGB and resizes it to 322 by 322 pixels without cropping or padding.

The result contains `embedding: number[]`, an L2-normalized descriptor with 8448 values, and optional `stats`. A dot product between two descriptors gives their cosine similarity. `embedBatch()` returns `embedding: number[][]` in input order, including batches with one image. Use these descriptors to compare images of places or build an image retrieval index. Text embeddings use the same `embed()` and `embedBatch()` functions with a text model. Class labels use `classify()`.

MegaLoc also accepts one encoded image media part per input, using a base64 source or a file path accessible to the worker. It rejects text, audio, and inputs with multiple media parts.

## Functions

Use the following sequence of function calls:

1. [`loadModel()`](/sdk/reference/api#loadmodel)
2. [`embed()`](/sdk/reference/api#embed) or [`embedBatch()`](/sdk/reference/api#embedbatch)
3. [`unloadModel()`](/sdk/reference/api#unloadmodel)

For how to use each function, see [SDK API reference](/sdk/reference/api/).

## Models

MegaLoc requires one GGUF file. Import `MEGALOC_F16` for F16 weights, or `MEGALOC_MIXED` for an F16 body with a Q6\_K final projection. Pass either constant as `modelSrc` to download the weights from the registry.

The original safetensors checkpoint cannot be loaded directly. See the addon's [MegaLoc conversion guide](https://github.com/tetherto/qvac/blob/main/packages/classification-ggml/docs/megaloc.md) for conversion and model provenance.

## Features

Choose a device with `modelConfig.backend`, using its exact GGML name, such as `CPU`, `CUDA0`, `Vulkan0`, or `MTL0`. `Metal` and `Metal0` are aliases. If omitted, the addon selects a GPU, then CPU. An explicitly requested device must be available. `modelConfig.threads` accepts integers from 1 to 1024. `modelConfig.backendsDir` optionally selects a directory of GGML backend modules.

`stats` exposes `totalTime` in milliseconds, `weightBytes`, `cpuFallbackNodes`, and `backendId`. Backend IDs are CPU=0, Metal=1, CUDA=2, Vulkan=3, OpenCL=4, and other=99. GPU support depends on the installed Fabric build. For a batch, `totalTime` sums the native times; the other fields describe the final image.

When bundling a plugin subset, include `@qvac/sdk/ggml-image-embedding/plugin` or `PLUGIN_IMAGE_EMBEDDING`. The default SDK worker includes it. The `ggml-classification` plugin continues to load MobileNet classifiers.

## Cancellation

Calls queue FIFO per model, with one active call and at most 64 waiting. Different models run independently. The returned promise exposes `requestId` immediately. Use [`cancel({ requestId })`](/sdk/reference/api#cancel) to cancel that call, or `cancel({ modelId, kind: "imageEmbedding" })` for all matching calls.

Cancelling a queued call prevents native inference from starting. Cancelling an active call discards its result after native inference finishes and rejects with `InferenceCancelledError`. The next call waits until that work finishes. MegaLoc does not expose a public method to interrupt native computation.

## Example

The following script compares two images using `MEGALOC_F16`. Pass the two JPEG or PNG paths, followed by an optional GGUF URL or local file path to use custom weights:

<Tabs>
  <Tab value="js" label="JavaScript" default>
    <WrapCode>
      ```js file=<rootDir>/packages/sdk/dist/examples/image/megaloc.js title="megaloc.js" lineNumbers
      /**
       * Compare two images using the MegaLoc F16 model.
       * Pass a GGUF URL or local file after the two JPEG/PNG paths to use custom weights.
       */
      import { readFile } from 'node:fs/promises';
      import { loadModel, embedBatch, unloadModel, close, ModelType, MEGALOC_F16 } from '@qvac/sdk';
      let modelId;
      try {
          const [firstPath, secondPath, modelSrc = MEGALOC_F16] = process.argv.slice(2);
          if (!firstPath || !secondPath) {
              throw new Error('Usage: megaloc <first.jpg> <second.jpg> [GGUF URL or path]');
          }
          console.log('▸ Loading MegaLoc');
          modelId = await loadModel({ modelSrc, modelType: ModelType.ggmlImageEmbedding });
          const { embedding, stats } = await embedBatch({
              modelId,
              inputs: [{ image: await readFile(firstPath) }, { image: await readFile(secondPath) }]
          });
          const [first, second] = embedding;
          if (!first || !second)
              throw new Error('Expected two image descriptors');
          const similarity = first.reduce((sum, value, index) => sum + value * second[index], 0);
          console.log(JSON.stringify({ similarity, dimensions: first.length, stats }, null, 2));
      }
      catch (error) {
          console.error('✖', error instanceof Error ? error.message : String(error));
          process.exitCode = 1;
      }
      finally {
          try {
              if (modelId)
                  await unloadModel({ modelId });
          }
          finally {
              await close();
          }
      }
      ```
    </WrapCode>
  </Tab>

  <Tab value="ts" label="TypeScript">
    <WrapCode>
      ```ts file=<rootDir>/packages/sdk/examples/image/megaloc.ts title="megaloc.ts" lineNumbers
      /**
       * Compare two images using the MegaLoc F16 model.
       * Pass a GGUF URL or local file after the two JPEG/PNG paths to use custom weights.
       */
      import { readFile } from 'node:fs/promises'
      import { loadModel, embedBatch, unloadModel, close, ModelType, MEGALOC_F16 } from '@qvac/sdk'

      let modelId: string | undefined
      try {
        const [firstPath, secondPath, modelSrc = MEGALOC_F16] = process.argv.slice(2)
        if (!firstPath || !secondPath) {
          throw new Error('Usage: megaloc <first.jpg> <second.jpg> [GGUF URL or path]')
        }
        console.log('▸ Loading MegaLoc')
        modelId = await loadModel({ modelSrc, modelType: ModelType.ggmlImageEmbedding })
        const { embedding, stats } = await embedBatch({
          modelId,
          inputs: [{ image: await readFile(firstPath) }, { image: await readFile(secondPath) }]
        })
        const [first, second] = embedding
        if (!first || !second) throw new Error('Expected two image descriptors')
        const similarity = first.reduce((sum, value, index) => sum + value * second[index]!, 0)
        console.log(JSON.stringify({ similarity, dimensions: first.length, stats }, null, 2))
      } catch (error) {
        console.error('✖', error instanceof Error ? error.message : String(error))
        process.exitCode = 1
      } finally {
        try {
          if (modelId) await unloadModel({ modelId })
        } finally {
          await close()
        }
      }
      ```
    </WrapCode>
  </Tab>
</Tabs>

<Callout type="info">
  The Python client supports this capability through the same worker, using `embed()` or `embed_batch()` with a request containing base64 image bytes. A dedicated Python example is not yet published. See the [Python SDK](/sdk/python-sdk) for the API.
</Callout>

<Callout type="success">
  All examples throughout this documentation are self-contained and runnable. For instructions on how to run them, see the [JS/TS quickstart](/sdk/js-ts-sdk#quickstart), the [Python quickstart](/sdk/python-sdk#quickstart), or the [Kotlin quickstart](/sdk/kotlin-sdk#quickstart).
</Callout>
