New: VisionPsy-Nano, a 460M vision model that outperforms models twice its size.
QVAC Logo
Hugging Face

RAG

Out-of-the-box retrieval-augmented generation workflow.

Overview

RAG (retrieval-augmented generation) uses text embeddings: you embed (vectorize) documents, persist them in a vector store, and later retrieve the most relevant chunks for a query using similarity search.

Compared to generating text embeddings only, the key differences are:

  • You must persist embeddings (vectors) alongside the original text (and optional metadata) in a vector store.
  • At query time, you embed the query and run top‑K vector search to fetch the most relevant documents/chunks.
  • You typically pass the retrieved text to completion() as context to ground the model's answer (this retrieval step is what makes it RAG).

Functions

  1. loadModel() (with modelType: "embeddings")
  2. embed() — generate vectors
  3. Use any combination of the RAG functions below as needed:
  4. unloadModel()

For how to use each function, see SDK — API reference.

Pipeline

Create your RAG pipeline using text embeddings, the functions above, and completion.

Regarding vector storage, you may use:

  • Built-in vector store: use the RAG workspace functions (ragIngest(), ragSearch(), etc.). QVAC persists the vectors for you, so this is the simplest path.
  • External vector DB: use the embedding primitives (ragChunk(), embed()) and store the vectors wherever you want (e.g., MongoDB, LanceDB, ChromaDB, SQLite-Vector). Choose this when you need custom persistence, filtering, or integration with an existing database.

Important: when using an external vector DB, make sure its schema matches the embedding dimensionality produced by your model (e.g., GTE Large embeddings used in the example are 1024‑dimensional).

Examples

Built-in vector store

The following script shows how to ingest documents into a built-in RAG workspace with ragIngest() and query them with ragSearch(), without setting up an external vector DB:

rag-managed.js
import { loadModel, unloadModel, GTE_LARGE_FP16, ragIngest, ragSearch, ragCloseWorkspace } from '@qvac/sdk';
try {
    // Get query from command line or use default
    const query = process.argv[2] || 'machine learning algorithms';
    const workspace = 'ingest-example';
    console.log(`▸ Query: "${query}"`);
    const modelId = await loadModel({
        modelSrc: GTE_LARGE_FP16,
        onProgress: (p) => {
            const mb = (n) => (n / 1e6).toFixed(1);
            const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
            process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
            if (p.percentage >= 100)
                process.stderr.write('\n');
        }
    });
    const samples = [
        'Machine learning is a subset of artificial intelligence that focuses on algorithms that can learn and make predictions from data without being explicitly programmed for every task.',
        'Deep learning uses neural networks with multiple layers to process and learn from complex data patterns, enabling breakthroughs in image recognition and natural language processing.',
        'Natural language processing combines computational linguistics with machine learning to help computers understand, interpret, and generate human language in a meaningful way.',
        'Computer vision enables machines to interpret and understand visual information from the world, using techniques like image classification, object detection, and facial recognition.',
        'Quantum computing leverages quantum mechanical phenomena to process information in fundamentally different ways than classical computers, potentially solving certain problems exponentially faster.',
        'Blockchain technology creates decentralized, immutable ledgers that enable secure peer-to-peer transactions without requiring a central authority or intermediary.',
        'Cloud computing delivers computing services over the internet, allowing users to access resources like storage, processing power, and applications on-demand from anywhere.',
        'Cybersecurity protects digital systems, networks, and data from malicious attacks, unauthorized access, and various forms of cyber threats through multiple layers of defense.'
    ];
    console.log('▸ Ingesting documents...');
    const result = await ragIngest({
        modelId,
        workspace,
        documents: samples,
        chunk: false
    });
    console.log(`▸ Ingested ${result.processed.length} documents`);
    console.log('▸ Searching for similar documents...');
    const results = await ragSearch({
        modelId,
        workspace,
        query,
        topK: 3
    });
    console.log('▸ Top 3 most similar documents:');
    results.forEach((result, index) => {
        console.log(`${index + 1}. (Score: ${result.score})`);
        console.log(`   ${result.content}`);
        console.log();
    });
    // Cleanup: close and delete workspace
    await ragCloseWorkspace({ workspace, deleteOnClose: true });
    console.log(`▸ Deleted '${workspace}' workspace`);
    await unloadModel({ modelId });
}
catch (error) {
    console.error('✖', error);
    process.exit(1);
}

External vector DB

MongoDB

The following script stores each document in MongoDB alongside its vector, builds a vector search index over it, and retrieves matches with a $vectorSearch aggregation that pre-filters by category:

rag-mongodb.js
import { embed, loadModel, unloadModel, GTE_LARGE_FP16 } from '@qvac/sdk';
import { MongoClient } from 'mongodb';
const INDEX_NAME = 'documents_vector_index';
const MONGODB_SETUP_INSTRUCTIONS = `
▸ This example needs a MongoDB deployment with Atlas Vector Search.

One way to get one is to run it in Docker:

   docker run -p 27017:27017 --name atlas-local mongodb/mongodb-atlas-local

For more details, visit: https://www.mongodb.com/docs/atlas/cli/current/atlas-cli-deploy-docker/
`;
async function initializeMongoClient() {
    // Replace with your own deployment's connection string if it is not the Docker one
    // https://www.mongodb.com/docs/manual/reference/connection-string/
    const client = new MongoClient('mongodb://localhost:27017/?directConnection=true');
    try {
        await client.connect();
        await client.db('admin').command({ ping: 1 });
        console.log('▸ Connected to MongoDB server');
        return client;
    }
    catch {
        console.error('✖ Failed to connect to MongoDB server');
        console.error('▸ Please ensure the server is running on localhost:27017');
        console.error(MONGODB_SETUP_INSTRUCTIONS);
        process.exit(1);
    }
}
function wait(ms) {
    return new Promise((resolve) => setTimeout(resolve, ms));
}
try {
    // Get query and category from command line or use defaults
    const query = process.argv[2] || 'machine learning algorithms';
    const category = process.argv[3] || 'ai';
    console.log(`▸ Query: "${query}" (category: "${category}")`);
    const client = await initializeMongoClient();
    const collection = client.db('qvac').collection('documents');
    const modelId = await loadModel({
        modelSrc: GTE_LARGE_FP16,
        onProgress: (p) => {
            const mb = (n) => (n / 1e6).toFixed(1);
            const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
            process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
            if (p.percentage >= 100)
                process.stderr.write('\n');
        }
    });
    // Sample corpus, each document tagged with a category to filter on
    const samples = [
        {
            id: 1,
            category: 'ai',
            text: 'Machine learning is a subset of artificial intelligence that focuses on algorithms that can learn and make predictions from data without being explicitly programmed for every task.'
        },
        {
            id: 2,
            category: 'ai',
            text: 'Deep learning uses neural networks with multiple layers to process and learn from complex data patterns, enabling breakthroughs in image recognition and natural language processing.'
        },
        {
            id: 3,
            category: 'ai',
            text: 'Natural language processing combines computational linguistics with machine learning to help computers understand, interpret, and generate human language in a meaningful way.'
        },
        {
            id: 4,
            category: 'ai',
            text: 'Computer vision enables machines to interpret and understand visual information from the world, using techniques like image classification, object detection, and facial recognition.'
        },
        {
            id: 5,
            category: 'computing',
            text: 'Quantum computing leverages quantum mechanical phenomena to process information in fundamentally different ways than classical computers, potentially solving certain problems exponentially faster.'
        },
        {
            id: 6,
            category: 'security',
            text: 'Blockchain technology creates decentralized, immutable ledgers that enable secure peer-to-peer transactions without requiring a central authority or intermediary.'
        },
        {
            id: 7,
            category: 'computing',
            text: 'Cloud computing delivers computing services over the internet, allowing users to access resources like storage, processing power, and applications on-demand from anywhere.'
        },
        {
            id: 8,
            category: 'security',
            text: 'Cybersecurity protects digital systems, networks, and data from malicious attacks, unauthorized access, and various forms of cyber threats through multiple layers of defense.'
        }
    ];
    // (Re)create the collection
    try {
        await collection.drop();
    }
    catch (e) {
        console.warn(`▸ Collection didn't exist, no need to drop: ${String(e)}`);
    }
    // Embed and store documents
    console.log('▸ Embedding documents...');
    const documents = [];
    for (const sample of samples) {
        const { embedding } = await embed({ modelId, text: sample.text });
        documents.push({
            id: sample.id,
            category: sample.category,
            text: sample.text,
            embedding
        });
    }
    await collection.insertMany(documents);
    // numDimensions is fixed at index creation and must match the model: GTE Large is 1024
    console.log('▸ Creating vector search index...');
    await collection.createSearchIndex({
        name: INDEX_NAME,
        type: 'vectorSearch',
        definition: {
            fields: [
                {
                    type: 'vector',
                    path: 'embedding',
                    numDimensions: 1024,
                    similarity: 'cosine'
                },
                // A field must be indexed as a filter to be usable in a $vectorSearch filter
                {
                    type: 'filter',
                    path: 'category'
                }
            ]
        }
    });
    // Index builds are asynchronous; querying too early returns no matches
    for (let attempt = 0; attempt < 60; attempt++) {
        const [index] = (await collection.listSearchIndexes(INDEX_NAME).toArray());
        if (index?.queryable)
            break;
        if (attempt === 59)
            throw new Error(`Index ${INDEX_NAME} did not become queryable`);
        await wait(1000);
    }
    console.log('▸ Searching for similar documents...');
    const { embedding: queryEmbedding } = await embed({ modelId, text: query });
    const results = await collection
        .aggregate([
        {
            $vectorSearch: {
                index: INDEX_NAME,
                path: 'embedding',
                queryVector: queryEmbedding,
                filter: { category: { $eq: category } },
                numCandidates: 100,
                limit: 3
            }
        },
        {
            $project: {
                _id: 0,
                id: 1,
                category: 1,
                text: 1,
                score: { $meta: 'vectorSearchScore' }
            }
        }
    ])
        .toArray();
    console.log('▸ Top 3 most similar documents:');
    results.forEach((result, index) => {
        console.log(`${index + 1}. (Score: ${result.score.toFixed(4)}, Category: ${result.category})`);
        console.log(`   ${result.text}`);
        console.log();
    });
    await unloadModel({ modelId });
    await client.close();
}
catch (error) {
    console.error('✖', error);
    process.exit(1);
}

SQLite

The following script shows the same workflow using embed() plus a SQLite vector index:

rag-sqlite.js
import { embed, loadModel, unloadModel, GTE_LARGE_FP16 } from '@qvac/sdk';
import sqlite3InitModule from '@sqliteai/sqlite-wasm';
try {
    // Get query from command line or use default
    const query = process.argv[2] || 'machine learning algorithms';
    console.log(`▸ Query: "${query}"`);
    // Initialize SQLite with vector extension
    const sqlite3 = await sqlite3InitModule();
    const db = new sqlite3.oo1.DB(':memory:', 'c');
    const modelId = await loadModel({
        modelSrc: GTE_LARGE_FP16,
        onProgress: (p) => {
            const mb = (n) => (n / 1e6).toFixed(1);
            const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
            process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
            if (p.percentage >= 100)
                process.stderr.write('\n');
        }
    });
    const samples = [
        {
            id: 1,
            text: 'Machine learning is a subset of artificial intelligence that focuses on algorithms that can learn and make predictions from data without being explicitly programmed for every task.'
        },
        {
            id: 2,
            text: 'Deep learning uses neural networks with multiple layers to process and learn from complex data patterns, enabling breakthroughs in image recognition and natural language processing.'
        },
        {
            id: 3,
            text: 'Natural language processing combines computational linguistics with machine learning to help computers understand, interpret, and generate human language in a meaningful way.'
        },
        {
            id: 4,
            text: 'Computer vision enables machines to interpret and understand visual information from the world, using techniques like image classification, object detection, and facial recognition.'
        },
        {
            id: 5,
            text: 'Quantum computing leverages quantum mechanical phenomena to process information in fundamentally different ways than classical computers, potentially solving certain problems exponentially faster.'
        },
        {
            id: 6,
            text: 'Blockchain technology creates decentralized, immutable ledgers that enable secure peer-to-peer transactions without requiring a central authority or intermediary.'
        },
        {
            id: 7,
            text: 'Cloud computing delivers computing services over the internet, allowing users to access resources like storage, processing power, and applications on-demand from anywhere.'
        },
        {
            id: 8,
            text: 'Cybersecurity protects digital systems, networks, and data from malicious attacks, unauthorized access, and various forms of cyber threats through multiple layers of defense.'
        }
    ];
    // Create table for documents with vector storage
    db.exec(`
  CREATE TABLE IF NOT EXISTS documents (
    id INTEGER PRIMARY KEY,
    text TEXT NOT NULL,
    embedding BLOB NOT NULL
  )
`);
    console.log('▸ Embedding documents...');
    for (const sample of samples) {
        const { embedding } = await embed({ modelId, text: sample.text });
        db.exec({
            sql: 'INSERT INTO documents VALUES (?, ?, vector_as_f32(?))',
            bind: [sample.id, sample.text, JSON.stringify(embedding)]
        });
    }
    // Initialize and optimize vector index
    db.exec(`SELECT vector_init('documents', 'embedding', 'type=FLOAT32,dimension=1024')`);
    // Quantize vectors
    db.exec(`SELECT vector_quantize('documents', 'embedding')`);
    // [Optional] Preload quantized vectors in memory for optimal performance
    db.exec(`SELECT vector_quantize_preload('documents', 'embedding')`);
    // Search for similar documents
    console.log('▸ Searching for similar documents...');
    const { embedding: queryEmbedding } = await embed({ modelId, text: query });
    const results = [];
    // Perform vector search
    db.exec({
        sql: `
    SELECT d.id, d.text, v.distance 
    FROM documents d
    JOIN vector_quantize_scan('documents', 'embedding', vector_as_f32(?), 3) v
    ON d.id = v.rowid
  `,
        bind: [JSON.stringify(queryEmbedding)],
        rowMode: 'object',
        callback: (row) => {
            const typedRow = row;
            results.push(typedRow);
        }
    });
    console.log('\n▸ Top 3 most similar documents:');
    results.forEach((result, index) => {
        console.log('='.repeat(50) + ' Top result:');
        console.log(`\n${index + 1}. [ID: ${result.id}] (Score: ${result.distance.toFixed(4)})`);
        console.log(`   ${result.text}`);
        console.log('='.repeat(100));
        console.log();
    });
    await unloadModel({ modelId });
    db.close();
}
catch (error) {
    console.error('✖', error);
    process.exit(1);
}

The Python client supports this capability through the same worker. A dedicated Python example is not yet published — see the Python SDK for the API surface.

Tip: all examples throughout this documentation are self-contained and runnable. For instructions on how to run them, see the JS/TS quickstart or the Python quickstart.

Text embeddings

Previous Page

Fine-tuning

Next Page

On this page

OverviewFunctionsPipelineExamplesBuilt-in vector storeExternal vector DBMongoDBSQLite

Ask anything about QVAC.