Assess model fit

Overview

assessModelFit() answers one question before anything is downloaded: is this model likely to fit in memory? It reads the model constant metadata together with a fresh memory sample, and returns a verdict for each candidate plus one for the set. For a single candidate it also fetches the registry's weightless description of the artifact — tens of KB, the tensor list with no data section — and runs the engine's own fitter against it.

It is advisory. It never downloads weights or loads a model, and it does not block loadModel(), reserve memory, pick a model for you, or make any claim about speed.

Functions

1. assessModelFit() — pass the candidate model constants with the workload you intend to run.
2. downloadAsset() or loadModel() — fetch whichever candidate you picked.

For how to use each function, see SDK — API reference.

Parameters

assessModelFit({
  models: [
    { model: QWEN3_8B_INST_Q4_K_M, workload: { kind: "llm", contextTokens: 8192 } },
  ],
  execution: "sequential",
});

models

The candidates, at least one. They are assessed together, against a single budget.

• model — a catalog model constant. Its name, checksum and registry coordinates are read; the byte totals and transformer facts come from the resource profile that checksum resolves to, and the coordinates locate the description the fitter reads.
• workload — what you intend to run on it: { kind: "llm", contextTokens } for completion and embedding models, { kind: "audio", windowMs, streaming } for speech-to-text, with an optional batch for concurrent windows per call.
• artifacts — optional. Extra catalog constants the same load also requires, such as a VAD or a pivot model. Their bytes are added to this candidate.

execution

Either sequential or concurrent:

• sequential: counts every model as resident but adds only the largest single working peak.
• concurrent: adds every peak.

The mode describes what you intend to do so the numbers match your plan — the SDK does not schedule, serialize, or reserve anything on the strength of it.

Defaults to sequential.

policy

How much memory to withhold from the budget as headroom, left for the rest of the system:

• interactive-v1: withholds 20% of the memory free at the time of the call, capped at 2 GiB on desktop and 1 GiB on mobile. Right now, it is the only accepted value and it applies by default, so passing it is optional.

Returns

The result answers for the whole set, and result.models repeats the same shape for each candidate. These are the fields to branch on.

verdict

evidence

What backs the verdict.

native-fit is the strongest: it is the answer the loader itself would give on this machine, so it outranks calibration where both exist. It carries no estimate, because the fitter returns a plan rather than a byte range, so branch on the verdict rather than on bounds. It is reachable only for a single llama.cpp candidate, without companion artifacts, whose registry record publishes a description; a set of candidates, a candidate with companions, a model with no description, and an offline caller keep the calibrated estimate.

Read evidence alongside the verdict, because the same unknown can mean different things:

• Under calibration, the numbers came out too close to call.
• Under computed-only, this platform has no measurements at all, so the answer says nothing about whether the model fits.

That difference matters once you act on the result. Hiding the likely-too-large candidates works the same way under all three, since that verdict is trustworthy either way. Telling a user that a model will fit needs native-fit or calibration; the computed floor never produces likely-fits.

basis

The memory the verdict was weighed against:

• system-memory — device RAM and system-wide use. Desktop, and Android, whose low-memory killer acts system-wide.
• process-memory — the app's own ceiling. iOS, where jetsam terminates an app on its own footprint against a limit well below device RAM.
• device-memory — a discrete GPU's own memory, when the model would execute there.
• device-budget — on Windows, the GPU memory the OS grants this process, since a card's readings there are per-process rather than device-wide.

Both device bases additionally require the system-memory budget to hold, because a GPU load is paid for in system RAM too.

reasons

Every unknown names its cause here — on the model when the cause is that model, on the result when it is the machine. The ones you are most likely to meet:

• No validated calibration for this platform, or for the GPU placement the model would use.
• The model is not a catalog constant, so there is no resource profile to read.
• The workload kind does not match the model's engine — an llm workload on a whisper model, or the reverse.

Where verdicts are available

calibration evidence — and with it any likely-fits — exists only where coefficients have been measured on real hardware and validated against a held-out model. Today that is LLM workloads (llamacpp-completion, llamacpp-embedding) on desktop, except win32-arm64. Where a GPU would run the model, the platform also needs a fixture measured on that placement.

Everywhere else the assessment falls back to the computed floor: likely-too-large when the weights alone exceed the budget, unknown otherwise. That covers Android, iOS, win32-arm64, audio workloads, and every engine with no estimator yet.

native-fit is independent of calibration. Wherever the engine's fitter runs — every platform with a llama.cpp addon, mobile included — a single llama.cpp candidate with a published description and no companion artifacts gets the fitter's verdict, including likely-fits, before any coefficients are consulted.

The per-platform calibration matrix, which changes with every release, lives in packages/sdk/docs/assess-model-fit.md.

Example

The following script asks which of five Qwen3 models this machine is likely to run at 8192 tokens of context, printing the budget basis, each verdict against the model's size on disk, and the reasons behind every unknown:

/**
 * Which models is this machine likely to run — asked before downloading any
 * weights. Nothing is fetched, loaded or reserved.
 *
 * The header prints the evidence the answer rests on: `system-memory` on a
 * CPU-only or integrated-GPU host, `device-memory` on a discrete card,
 * `device-budget` for the per-process allowance Windows grants on one. Which
 * of those applies depends on where the engine would put the model.
 *
 * `unknown` is a real answer, not an error: the evidence does not support a
 * call either way. Show it as "can't say", never as "no".
 */
import { assessModelFit, QWEN3_600M_INST_Q4, QWEN3_1_7B_INST_Q4, QWEN3_4B_INST_Q4_K_M, QWEN3_8B_INST_Q4_K_M, QWEN3_8_27B_MULTIMODAL_UD_Q8_K_XL } from '@qvac/sdk';
// `as const` so `kind` stays the `'llm'` literal the input type asks for.
const WORKLOAD = { kind: 'llm', contextTokens: 8192 };
// A ladder ending well past what a laptop has, so one screen shows every verdict.
const CANDIDATES = [
    QWEN3_600M_INST_Q4,
    QWEN3_1_7B_INST_Q4,
    QWEN3_4B_INST_Q4_K_M,
    QWEN3_8B_INST_Q4_K_M,
    QWEN3_8_27B_MULTIMODAL_UD_Q8_K_XL
];
const VERDICT_MARK = {
    'likely-fits': '✔',
    'likely-too-large': '✖',
    unknown: '?'
};
function gib(bytes) {
    return `${(bytes / 1024 ** 3).toFixed(2)} GiB`;
}
try {
    const result = await assessModelFit({
        models: CANDIDATES.map((model) => ({ model, workload: WORKLOAD })),
        // Declared for aggregation only, not a scheduling instruction: 'sequential'
        // counts the largest operation peak, 'concurrent' counts one per model.
        execution: 'sequential',
        policy: 'interactive-v1'
    });
    console.log(`▸ Budget basis: ${result.basis}`);
    if (result.budget) {
        console.log(`    ${gib(result.budget.availableAfterReserveBytes)} budget` +
            ` (${gib(result.budget.totalBytes)} total,` +
            ` ${gib(result.budget.usedBytes)} in use,` +
            ` ${gib(result.budget.availableBytes)} free,` +
            ` ${gib(result.budget.reservedBytes)} held back)`);
    }
    // Says which device the verdicts assume, when the model would run on a GPU.
    const placement = result.assumptions.find((line) => line.includes('assumed to execute on'));
    if (placement)
        console.log(`▸ ${placement}`);
    console.log(`\n▸ At ${WORKLOAD.contextTokens} tokens of context`);
    for (const [index, model] of result.models.entries()) {
        const mark = VERDICT_MARK[model.verdict];
        const size = gib(CANDIDATES[index].expectedSize).padStart(9);
        // A calibrated row has a two-sided estimate; a computed-only row has the floor.
        const needs = model.estimate
            ? `needs ${gib(model.estimate.upperBoundBytes)}`
            : model.evidence === 'computed-only' && model.floorBytes
                ? `at least ${gib(model.floorBytes)}`
                : '';
        console.log(`  ${mark} ${model.name.padEnd(46)} ${size} on disk  ${needs}`);
        // Present on every `unknown`, and worth surfacing: it names what is missing.
        if (model.verdict === 'unknown') {
            for (const reason of model.reasons)
                console.log(`      ${reason}`);
        }
    }
    console.log(`\n▸ All five together, sequentially: ${result.verdict}`);
    console.log('▸ Advisory only — this does not gate loadModel or reserve anything.');
    process.exit(0);
}
catch (error) {
    console.error('✖', error);
    process.exit(1);
}