HTTP server

Overview

To run the server, install the @qvac/cli npm package — it depends on @qvac/sdk directly, so the SDK is installed automatically. The server is provided by @qvac/cli and internally translates HTTP requests into SDK calls.

Surfaces

qvac serve runs the QVAC API. Additional surfaces are extensions, mounted by their own flag:

qvac serve --openai

Any system compatible with the OpenAI REST API can then point to http://localhost:11434/v1/ and work without changes.

Pass --no-default to leave the QVAC surface out. Introspection (/openapi.json, and /docs with --docs) is always served and describes exactly what you mounted.

AI capabilities

At the moment, the OpenAI-compatible extension supports the following QVAC AI capabilities:

• Text generation — via Chat (/v1/chat/completions), Responses (/v1/responses, modern), or Legacy completions (/v1/completions).
• Text embeddings — via /v1/embeddings.
• RAG — via Files (/v1/files) and Vector stores (/v1/vector_stores).
• Image generation — via /v1/images/generations and /v1/images/edits.
• Video generation — via /v1/videos.
• Transcription — via Audio (/v1/audio/transcriptions).
• Text-to-speech — via Audio (/v1/audio/speech).
• Translation (audio-to-English only) — via Audio (/v1/audio/translations, Whisper translate task).

The QVAC surface supports:

• Text translation — via /qvac/v1/translate, backed by an NMT model.

Running the server

npm install -g @qvac/cli

qvac configure           # interactive: pick models by capability or search
qvac configure --yes     # non-interactive: chat + transcription starter

{
  "serve": {
    "models": {
      "my-llm": {
        "model": "QWEN3_600M_INST_Q4",
        "default": true,
        "config": { "ctx_size": 8192 }
      }
    }
  }
}

qvac serve --openai

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-llm",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Configuration

Models are declared in qvac.config.* under the serve.models key. The server can only load models listed under this key — requests for unlisted models return 404. Each key in serve.models is a model alias — the name that HTTP clients use in the model field of their requests. For the full schema of serve.models, see Configuration — ServeConfig.

Example

{
  "serve": {
    "models": {
      "my-llm": {
        "model": "QWEN3_600M_INST_Q4",
        "default": true,
        "preload": true,
        "config": { "ctx_size": 8192, "tools": true }
      },
      "my-embed": {
        "model": "GTE_LARGE_FP16",
        "default": true
      },
      "whisper": {
        "model": "WHISPER_TINY",
        "default": true,
        "preload": true,
        "config": { "language": "en", "strategy": "greedy" }
      }
    }
  }
}

• model: SDK model constant name (e.g., QWEN3_600M_INST_Q4). The server resolves it to a download source and addon type automatically.
• default: when true, marks this model as the default for its endpoint category. This does not make the server auto-select the model for requests that omit model.
• preload: when true, the model is loaded into memory on server startup. When false, it is loaded lazily on the first request that names it (cold start — that request blocks while the model loads; concurrent first requests share a single load). Defaults to true for constant model entries and false for explicit { src, type } entries. Use --model <alias> to force a lazy model to warm at startup.
• config: model config overrides passed to the underlying addon. Same options as modelConfig in loadModel().

Lazy-load behavior (whether requests may load models on demand, load concurrency, per-load timeout, and cancel-on-disconnect) is tuned under serve.load, with matching qvac serve --openai flags. Set serve.load.lazy: false (or --no-lazy-load) to require preloaded models and return 503 model_not_loaded otherwise.

Integration

To create a client, you can use any OpenAI-compatible AI SDK provider, such as Vercel AI SDK. For a better developer experience, use our npm package @qvac/ai-sdk-provider.

CLI

Relevant options are summarized below; run qvac serve --help for the exact generated help output.

CORS

CORS is disabled unless trusted origins are supplied through repeatable --cors-origin flags or serve.cors.origins in qvac.config.*, or --docs adds its same-port loopback defaults. Origins must be exact HTTP(S) origins without credentials, paths, queries, or fragments. Wildcard (*) access is not supported.

qvac serve --openai \
  --cors-origin https://app.example.com \
  --cors-origin http://localhost:3000

The legacy --cors flag is retained as a compatibility validation switch, but it does not enable CORS by itself. It requires at least one explicit origin from --cors-origin or serve.cors.origins; otherwise startup fails. --cors --docs also fails without an explicit origin because the docs defaults are added only after this validation. Existing scripts can migrate as follows:

# Before
qvac serve --openai --cors

# After
qvac serve --openai --cors --cors-origin https://app.example.com

QVAC API

Served by the QVAC surface, mounted by default. Base path: /qvac/v1.

KV cache

Chat requests cache each conversation's prefix under ~/.qvac/kv-cache, so a follow-up turn only prefills the new tail. Chat is the only endpoint that caches — Responses and Legacy completions run uncached and leave nothing here to reclaim. The directory grows on the order of hundreds of megabytes per conversation, but it is bounded — you do not need to write a sweeper of your own.

Caches held by an in-flight request are never evicted.

DELETE /qvac/v1/kv_cache

Frees every automatic KV cache no in-flight request is using, immediately, rather than waiting for the quota or the TTL. Use it when reclaiming disk matters more than keeping conversations warm: deleting a cache costs the next turn a full prefill instead of a partial one, and loses no conversation content, since a KV cache is derived data.

curl -X DELETE http://localhost:11434/qvac/v1/kv_cache

Response:

{ "object": "kv_cache.reclaim", "deleted": true }

A failed delete returns 500 kv_cache_reclaim_failed.

This route needs the QVAC surface mounted. It is absent under --no-default, and therefore under the deprecated qvac serve openai, which implies it.

POST /qvac/v1/translate

Translate text with an NMT model, backed by the SDK's translate().

model names a serve.models alias whose endpoint category is translation. That alias configures the engine and the language direction, as shown at the end of this section.

translations comes back in the order the inputs were given, one entry per input:

{
  "object": "translation",
  "model": "ta-en",
  "translations": ["Hello, world."],
  "stats": { "totalTime": 41 }
}

stats is returned for a single input; a batch does not report stats.

Streaming a single input emits translation.chunk events carrying delta as the text is decoded. Streaming an array emits one translation.item per input, carrying index and the whole text, and these arrive together once the batch finishes. Both end with one translation.done event, then data: [DONE].

A client disconnect cancels the request and its result is dropped.

curl -sS http://localhost:11434/qvac/v1/translate \
  -H "Content-Type: application/json" \
  -d '{"model":"ta-en","text":"வணக்கம் உலகம்"}'

Configure a translation model with its engine and direction:

{
  "serve": {
    "models": {
      "ta-en": {
        "model": "BERGAMOT_TA_EN",
        "config": { "engine": "Bergamot", "from": "ta", "to": "en" }
      }
    }
  }
}

OpenAI-compatible API

Served by the --openai extension. All endpoints follow the OpenAI API request and response format. Base path: /v1.

Endpoints

Models

Inspect and unload models registered in serve.models. Every configured model is usable: one that is not preloaded loads on the first request that names it.

GET /v1/models

List all configured models (loaded or not — each loads on first request).

curl http://localhost:11434/v1/models

Response:

{
  "object": "list",
  "data": [
    { "id": "my-llm", "object": "model", "created": 1718000000, "owned_by": "qvac" }
  ]
}

GET /v1/models/:id

Get details of a configured model (whether or not it is currently loaded).

curl http://localhost:11434/v1/models/my-llm

DELETE /v1/models/:id

Unload a model, releasing its resources. The alias stays configured and reloadable — the next request that names it loads it again. There is no separate load endpoint; send a normal request (or set preload: true) to load a model.

curl -X DELETE http://localhost:11434/v1/models/my-llm

Response:

{ "id": "my-llm", "object": "model", "deleted": true }

Model catalog (browse)

GET /v1/models/catalog is a discovery endpoint (a QVAC extension, not part of OpenAI). It lists your configured models and every model the SDK ships with, so you can find one by capability. Browsing loads nothing — it's fully in-process.

Rows are catalog entries (object: "model_catalog_entry"), not usable model objects. A model that isn't configured shows state: "not_configured", usable: false and a hint — it cannot be called until you add it to serve.models. Configured models show configured: true with their live load state. (Only GET /v1/models lists callable models.)

Filter with query params (combined with AND): search (id substring), role (chat/embedding/transcription/speech/image/…), addon/type (llm/tts/whisper/diffusion/…), quantization, engine, configured, limit (omit for no limit), offset.

curl 'http://localhost:11434/v1/models/catalog?role=chat&limit=20'

Response: { "object": "list", "data": [ … ], "has_more": <bool> }, each entry:

{
  "object": "model_catalog_entry",
  "id": "QWEN3_600M_INST_Q4",
  "source": "builtin",
  "configured": false,
  "usable": false,
  "state": "not_configured",
  "role": "chat",
  "addon": "llm",
  "quantization": "q4",
  "params": "600M",
  "size": 382156480,
  "hint": "Not in serve.models — add it there to make it usable."
}

GET /v1/models/catalog/:id returns a single catalog entry. The live remote registry and on-disk download state are planned follow-ups.

Chat

OpenAI-compatible chat completions backed by any alias whose endpoint category is chat in serve.models.

POST /v1/chat/completions

Generate a chat completion. Supports both blocking and streaming (SSE) modes, tool/function calling, structured output, and per-request generation parameters.

Blocking request:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-llm",
    "messages": [{"role": "user", "content": "What is the capital of France?"}],
    "temperature": 0.7,
    "max_tokens": 256
  }'

Streaming request (server-sent events):

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-llm",
    "messages": [{"role": "user", "content": "What is the capital of France?"}],
    "stream": true
  }'

Tool calling:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-llm",
    "messages": [{"role": "user", "content": "What is the weather in London?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
          "type": "object",
          "properties": { "location": { "type": "string" } },
          "required": ["location"]
        }
      }
    }]
  }'

A tools request for a model loaded without config.tools: true returns 400 tools_not_enabled rather than answering in prose.

Forcing a tool call with tool_choice:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-llm",
    "messages": [{"role": "user", "content": "What is the weather in London?"}],
    "tools": [{ "type": "function", "function": { "name": "get_weather" } }],
    "tool_choice": "required"
  }'

tool_choice accepts "auto" (the default), "none", "required", or { "type": "function", "function": { "name": "get_weather" } } to force one specific tool. On llama.cpp-backed models required and a named tool constrain the sampler with the chat template's tool grammar, so the model calls a tool instead of answering in prose.

Both forms need a matching entry in tools: a demanding tool_choice with no tools, or a name that isn't declared, returns 400 invalid_tool_choice. A bare tool name in place of the object form is rejected the same way — use the object form to target one tool.

auto, none and required are reserved: a tool carrying one of those names cannot be targeted through tool_choice and the request returns 400 invalid_tool_choice. Rename the tool to target it.

A tool call the model emits that fails to parse or validate is dropped, so the response comes back with finish_reason: "stop" and no tool_calls. The server log records what happened (toolerrors=1 (PARSE_ERROR)) — the OpenAI response shape has no field for it.

Message content

messages[].content accepts both the plain string form and the OpenAI array-of-parts form ([{ "type": "text", "text": "…" }, …]) that modern clients such as Cline and Open WebUI send. Parts of type text are concatenated into a single string; non-text parts (image_url, input_audio, file) are silently dropped — the chat surface is text-only and vision is out of scope. Both shapes below are valid:

// string form
{ "role": "user", "content": "Describe a sunset." }

// array form (non-text parts ignored)
{ "role": "user", "content": [{ "type": "text", "text": "Describe a sunset." }] }

Generation parameters

The following OpenAI parameters are forwarded to the model on each request:

Reasoning (reasoning_content)

Models that emit <think>…</think> blocks (e.g. Qwen3.5) have their reasoning routed to the reasoning_content channel instead of leaking into content:

• Blocking — the reasoning is returned in choices[0].message.reasoning_content; content carries only the final answer.
• Streaming — reasoning arrives as choices[0].delta.reasoning_content chunks, interleaved with the regular delta.content chunks.

reasoning_content is present only when the model actually produced a reasoning block. Set reasoning_budget: false (see above) to disable reasoning entirely. Numeric 0 is also accepted; positive token caps are not.

Structured output (response_format)

response_format.type accepts text (default), json_object, and json_schema. When json_schema is used, the request must also carry json_schema.schema (a JSON Schema object) and may include json_schema.name and json_schema.strict.

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-llm",
    "messages": [{"role": "user", "content": "Pick a color."}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "color",
        "schema": {
          "type": "object",
          "properties": { "name": { "type": "string" } },
          "required": ["name"]
        }
      }
    }
  }'

Unsupported parameters

The following OpenAI parameters are accepted but ignored (a warning is logged): n, logprobs, stop, top_logprobs, logit_bias, parallel_tool_calls.

stream_options.include_usage is supported — see Response: finish_reason and token usage.

Response: finish_reason and token usage

Each choice carries a finish_reason that reflects how generation actually ended:

usage is populated from the SDK's CompletionStats:

• prompt_tokens — real prompt token count when the SDK reports it, else 0.
• completion_tokens — the SDK's generatedTokens when available, falling back to a whitespace word count of the output.
• total_tokens — the sum of the two.
• prompt_tokens_details.cached_tokens — present only when the SDK reports a KV-cache hit; it is the count of prompt tokens reused from a previous turn's cache, not the full prompt size.

Blocking and streaming share the same accounting, so counts do not drift between the two paths. By default a streaming response carries no usage object; set stream_options: { include_usage: true } to receive it as a trailing SSE chunk whose choices array is empty (it follows the finish_reason / tool_calls chunk), matching OpenAI's streaming shape.

Responses

OpenAI-compatible Responses API. Supports blocking, SSE streaming, retrieval by id, and previous_response_id chaining for multi-turn conversations. Backed by the same chat models registered under serve.models (any alias whose endpoint category is chat).

POST /v1/responses

Create a response.

Blocking request:

curl http://localhost:11434/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-llm",
    "input": "Say hello.",
    "store": true
  }'

Streaming request (SSE):

curl http://localhost:11434/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-llm",
    "input": "Say hello.",
    "stream": true
  }'

Multi-turn via previous_response_id:

curl http://localhost:11434/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-llm",
    "input": "and now?",
    "previous_response_id": "resp_..."
  }'

The same generation parameters (temperature, top_p, seed, max_output_tokens / max_tokens, frequency_penalty, presence_penalty, reasoning_budget) and the same response_format rules as /v1/chat/completions apply.

When generation is truncated because it hit max_output_tokens / max_tokens, the response is returned with status: "incomplete" and incomplete_details.reason: "max_output_tokens" — the Responses-API analogue of chat's finish_reason: "length". usage.output_tokens uses the same SDK-stats accounting as the other chat-category routes (input_tokens is 0).

The following Responses-API features are intentionally rejected with 400: conversation, background: true, and built-in tools (web_search, file_search, code_interpreter). function-typed tools work when the model was loaded with config.tools: true; otherwise the request returns 400 tools_not_enabled.

tool_choice is supported in the same shapes as chat completions, except that the object form is the Responses API's flattened { "type": "function", "name": "get_weather" }.

GET /v1/responses/:id

Retrieve a previously stored response by id.

curl http://localhost:11434/v1/responses/resp_abc123

DELETE /v1/responses/:id

Delete a stored response.

curl -X DELETE http://localhost:11434/v1/responses/resp_abc123

GET /v1/responses/:id/input_items

Paginate the original input items of a stored response. Accepts limit and after query parameters.

curl "http://localhost:11434/v1/responses/resp_abc123/input_items?limit=20"

Legacy completions

Legacy (pre-chat) OpenAI text-completions endpoint, kept for compatibility with older OpenAI clients and SDKs that have not migrated to /v1/chat/completions. Backed by the same chat-category models — any alias registered with endpoint category chat in serve.models serves both endpoints with no extra configuration.

POST /v1/completions

Generate a text completion from a raw prompt.

Blocking, single prompt:

curl http://localhost:11434/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"my-llm","prompt":"Say hello in one word.","max_tokens":16}'

Streaming (single prompt only):

curl http://localhost:11434/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"my-llm","prompt":"Say hello in one word.","stream":true}'

Multi-prompt fan-out (blocking only):

curl http://localhost:11434/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"my-llm","prompt":["Reply with alpha.","Reply with beta."],"max_tokens":8}'

Prompt input rules

• String or single-element string array — blocking JSON or SSE streaming. Response object is text_completion with cmpl- ids and choices[0].text.
• String array of length ≥ 2 (multi-prompt) — fanned out sequentially as N independent completions and returned in choices with matching index. Blocking only; combining with "stream": true returns 400 unsupported_streaming. If any single prompt fails, the whole request aborts (no partial results).
• Token-id prompts (number[], number[][]) and empty / missing prompts return 400 invalid_prompt.

The same generation parameters as /v1/chat/completions are accepted. The following OpenAI fields are accepted and ignored (warning logged): logprobs, echo, best_of, suffix, stop, logit_bias, stream_options, user, response_format, and n when greater than 1.

choices[].finish_reason follows the same rules as Chat: stop for a natural end, length when output is truncated by max_tokens. Token usage uses the same SDK-stats accounting; for multi-prompt requests, usage aggregates completion_tokens across every prompt.

Embeddings

Generate vector embeddings backed by any alias whose endpoint category is embedding.

POST /v1/embeddings

Generate text embeddings. Accepts a single string or a batch of strings.

Single input:

curl http://localhost:11434/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-embed",
    "input": "The quick brown fox"
  }'

Batch input:

curl http://localhost:11434/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-embed",
    "input": ["First sentence", "Second sentence"]
  }'

Response:

{
  "object": "list",
  "data": [
    { "object": "embedding", "index": 0, "embedding": [0.012, -0.034] }
  ],
  "model": "my-embed",
  "usage": { "prompt_tokens": 0, "total_tokens": 0 }
}

Audio

Transcription, translation, and text-to-speech endpoints. Transcription and translation use multipart/form-data; speech accepts JSON and returns binary audio.

POST /v1/audio/transcriptions

Transcribe audio using Whisper or Parakeet models. Uses multipart/form-data. Returns text in the source language.

JSON response (default):

curl http://localhost:11434/v1/audio/transcriptions \
  -F "file=@audio.wav" \
  -F "model=whisper" \
  -F "response_format=json"

Response: { "text": "transcribed text here" }

Plain text response:

curl http://localhost:11434/v1/audio/transcriptions \
  -F "file=@audio.wav" \
  -F "model=whisper" \
  -F "response_format=text"

With prompt (Whisper uses it as initial_prompt):

curl http://localhost:11434/v1/audio/transcriptions \
  -F "file=@audio.wav" \
  -F "model=whisper" \
  -F "prompt=President Kennedy speech about space exploration"

Parameters

Unsupported response_format values (srt, vtt, verbose_json) return a 400 error.

POST /v1/audio/translations

Translate audio into English text. Maps to Whisper's translate task (not "transcribe then run a text translator"). Uses multipart/form-data.

curl http://localhost:11434/v1/audio/translations \
  -F "file=@sample.wav" \
  -F "model=whisper-translate" \
  -F "response_format=json"

Response: { "text": "..." } for json; raw UTF-8 body for text.

Parameters

The language field is not supported — output is always English. Use /v1/audio/transcriptions if you need non-English text.

Registering a translation model

Use the virtual SDK type whispercpp-audio-translation in serve.models. The CLI resolves it to the whispercpp-transcription engine and forces translate: true on the load-time modelConfig. You can register the same Whisper weights twice — once for transcription, once for translation:

{
  "serve": {
    "models": {
      "whisper-transcribe": { "model": "WHISPER_EN_TINY_Q8_0", "preload": true },
      "whisper-translate": {
        "model": "WHISPER_EN_TINY_Q8_0",
        "type": "whispercpp-audio-translation",
        "preload": true
      }
    }
  }
}

POST /v1/audio/speech

OpenAI-compatible text-to-speech, backed by the SDK's textToSpeech capability (Chatterbox or Supertonic). Body is JSON, response body is binary audio.

curl http://localhost:11434/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model":"my-tts","voice":"alloy","input":"Hello from QVAC."}' \
  --output speech.wav

Loaded model

Register a TTS model in serve.models with type: "tts" (and typically preload: true to avoid cold-start latency):

{
  "serve": {
    "models": {
      "my-tts": {
        "src": "registry://hf/ResembleAI/chatterbox-turbo-ONNX/resolve/<sha>/tokenizer.json",
        "type": "tts",
        "preload": true,
        "config": {
          "ttsEngine": "chatterbox",
          "language": "en",
          "ttsTokenizerSrc": "registry://hf/ResembleAI/chatterbox-turbo-ONNX/resolve/<sha>/tokenizer.json",
          "ttsSpeechEncoderSrc": "registry://hf/ResembleAI/chatterbox-turbo-ONNX/resolve/<sha>/onnx/speech_encoder.onnx",
          "ttsEmbedTokensSrc": "registry://hf/ResembleAI/chatterbox-turbo-ONNX/resolve/<sha>/onnx/embed_tokens.onnx",
          "ttsConditionalDecoderSrc": "registry://hf/ResembleAI/chatterbox-turbo-ONNX/resolve/<sha>/onnx/conditional_decoder.onnx",
          "ttsLanguageModelSrc": "registry://hf/ResembleAI/chatterbox-turbo-ONNX/resolve/<sha>/onnx/language_model.onnx",
          "referenceAudioSrc": "./voices/alloy-ref.wav"
        }
      }
    }
  }
}

Voice → model alias

OpenAI clients select a voice via the voice field. QVAC TTS engines bind voice character to load-time config — Chatterbox uses referenceAudioSrc; Supertonic uses ttsVoiceStyleSrc. The route resolves the backing model in this order:

1. serve.openai.audio.speech.voices[voice] — explicit map from an OpenAI voice string to a serve.models alias (case-insensitive). When matched, the request's model field is not used for routing.
2. serve.models[model + "-" + voice] — hyphen alias (e.g. my-tts-alloy).
3. serve.models[model] — bare model alias.
4. None of the above — 404 model_not_found.

When voice is omitted, the configured serve.openai.audio.speech.defaultVoice is used (defaults to "alloy"). Set it to null to make voice strictly required.

{
  "serve": {
    "openai": {
      "audio": {
        "speech": {
          "defaultVoice": "alloy",
          "voices": {
            "alloy": "tts-chatter-alloy",
            "echo": "tts-chatter-echo"
          }
        }
      }
    }
  }
}

Request

The encoded formats (mp3, opus, aac, flac) are produced by transcoding the synthesized audio through ffmpeg, which must be on the server's PATH. When ffmpeg is absent they return 503 transcode_unavailable (use wav/pcm or install ffmpeg — see qvac doctor); unknown values return 400 invalid_response_format. The default stays wav so synthesis works on hosts without ffmpeg. speed, instructions, and stream_format are accepted but ignored — dropped fields are echoed back in the X-QVAC-Ignored-Params response header.

Response

The response body is binary audio. Headers always include:

The route always buffers the full audio before responding (chunked HTTP streaming is tracked as a follow-up).

GET /v1/audio/voices

Lists the configured TTS voices — the OpenAI voice names mapped under serve.openai.audio.speech.voices plus the configured defaultVoice. Used by clients such as Open WebUI's voice selector. QVAC enforces no fixed voice catalog, so callers may also send any voice string that resolves via a {model}-{voice} alias.

The response carries both a flat voices array (consumed by Open WebUI) and an OpenAI-style data array:

{
  "object": "list",
  "voices": ["alloy", "echo"],
  "data": [
    { "id": "alloy", "object": "audio.voice", "model": "tts-chatter-alloy" },
    { "id": "echo", "object": "audio.voice", "model": "tts-chatter-echo" }
  ]
}

GET /v1/audio/models

Lists loaded (READY) text-to-speech models — the speech-capable subset of /v1/models, filtered to models whose endpoint category is speech. Same { object: "list", data: [...] } shape, with each entry shaped like a /v1/models entry. Used by Open WebUI's TTS model selector.

{
  "object": "list",
  "data": [
    { "id": "tts-chatter-alloy", "object": "model", "created": 1718000000, "owned_by": "qvac" }
  ]
}

Images

Text-to-image and image-to-image endpoints backed by any alias whose endpoint category is image (built-in addons that resolve to this category are diffusion and sdcpp-generation).

POST /v1/images/generations

Text-to-image generation backed by the SDK's diffusion() primitive.

curl http://localhost:11434/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-diffusion",
    "prompt": "a watercolor cat at golden hour",
    "size": "1024x1024",
    "n": 1
  }'

Response:

{
  "created": 1718000000,
  "output_format": "png",
  "size": "1024x1024",
  "data": [{ "b64_json": "iVBORw0KGgoAAAANSUhEUgAA..." }]
}

Loaded model

Register an alias whose endpoint category is image (built-in addons that resolve to this category are diffusion and sdcpp-generation):

{
  "serve": {
    "models": {
      "my-diffusion": {
        "model": "SD_V2_1_1B_Q8_0",
        "preload": true,
        "config": { "prediction": "v" }
      }
    }
  }
}

response_format: b64_json (default) or url

• b64_json (default) — data[].b64_json carries the inline base64 PNG. No server-side state.
• url — requires --public-base-url <origin> (or serve.publicBaseUrl in the config). The image is stored in the in-memory ephemeral files store and data[].url resolves to ${publicBaseUrl}/v1/files/{id}/content. Each item also carries expires_at (Unix seconds) so clients know exactly when the URL stops working.

qvac serve --openai --public-base-url "https://api.example.com"

curl https://api.example.com/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{"model":"my-diffusion","prompt":"a watercolor cat","response_format":"url"}'

{
  "created": 1718000000,
  "output_format": "png",
  "data": [
    {
      "url": "https://api.example.com/v1/files/file-abcd/content",
      "expires_at": 1718003600
    }
  ]
}

Streaming (stream: true)

The response is text/event-stream and emits one image_generation.completed event per generated image (always carrying inline b64_json, regardless of the requested response_format), then [DONE].

The SDK does not surface intermediate image bytes (only step ticks via progressStream), so image_generation.partial_image events are not produced. This matches OpenAI's documented behavior for partial_images: 0.

Hard fails (400)

The server is intentionally loud about every OpenAI image-API field it cannot honor without producing the wrong bytes:

The following OpenAI fields are accepted and silently ignored (warning logged) because they are advisory: quality, style, moderation, partial_images, user, input_fidelity.

POST /v1/images/edits

Image-to-image (img2img) edits. Uses multipart/form-data. Shares the same validation, response shape, and response_format rules as /v1/images/generations.

curl http://localhost:11434/v1/images/edits \
  -F "image=@input.png" \
  -F "model=my-diffusion" \
  -F "prompt=oil painting style, warm lighting" \
  -F "strength=0.65"

Multipart fields

Files

The /v1/files endpoints expose an in-memory ephemeral file store used as the backing storage for image url responses and for vector-store ingestion. There is no disk or P2P persistence; entries are evicted by TTL or capacity.

POST /v1/files

Upload bytes (multipart).

curl http://localhost:11434/v1/files \
  -F "file=@notes.txt" \
  -F "purpose=assistants"

Response:

{
  "object": "file",
  "id": "file-abc123",
  "bytes": 4321,
  "created_at": 1718000000,
  "filename": "notes.txt",
  "purpose": "assistants",
  "status": "uploaded"
}

GET /v1/files

List files currently held in memory.

GET /v1/files/:id

Retrieve file metadata.

GET /v1/files/:id/content

Return the raw bytes with the stored Content-Type (used by image response_format=url).

Eviction

Defaults: 1 h TTL, 256 MB total cap, 256 files cap, oldest-first eviction. Every eviction logs a warn line with the reason (ttl / max_files / max_bytes). Files are also removed automatically when attached to a vector store via POST /v1/vector_stores/:id/files. GET /v1/files/:id/content sets Cache-Control: private, max-age=<seconds-until-eviction> so downstream proxies cannot serve bytes the store has dropped.

Vector stores

OpenAI-compatible vector-store endpoints backed by the SDK's RAG primitives. Each vector store maps 1:1 to a RAG workspace.

GET /v1/vector_stores

List all stores (merged with on-disk RAG workspaces).

POST /v1/vector_stores

Create a new store.

GET /v1/vector_stores/:id

Retrieve store metadata.

POST /v1/vector_stores/:id

Update name, expires_after, or metadata.

DELETE /v1/vector_stores/:id

Delete the store and the underlying RAG workspace.

POST /v1/vector_stores/:id/search

Embed query and run top-K similarity search.

POST /v1/vector_stores/:id/files

Attach a previously-uploaded /v1/files entry (UTF-8 text content).

End-to-end ingest + search:

curl http://localhost:11434/v1/vector_stores \
  -H "Content-Type: application/json" \
  -d '{"name":"my-docs"}'

curl http://localhost:11434/v1/files \
  -F "file=@notes.txt" \
  -F "purpose=assistants"

curl http://localhost:11434/v1/vector_stores/vs_my-docs/files \
  -H "Content-Type: application/json" \
  -d '{"file_id":"file-abc123"}'

curl http://localhost:11434/v1/vector_stores/vs_my-docs/search \
  -H "Content-Type: application/json" \
  -d '{"query":"what is in the notes?","max_num_results":4}'

Embedding model resolution

Search and ingest both pick an embedding model from serve.models:

1. If exactly one alias has default: true and endpoint category embedding, it is used.
2. If only one embedding alias is configured at all, it is used.
3. If multiple embedding aliases are configured and none is flagged as default, the request fails with 400 ambiguous_embedding_model.
4. If no embedding alias is configured, the request fails with 400 no_embedding_model_configured.

Once a vector store has been ingested with a particular embedding model, subsequent ingest or search calls must resolve to the same alias — otherwise the request fails with 400 embedding_model_mismatch. To switch embeddings, create a new vector store.

File ingest constraints

• Files attached via POST /v1/vector_stores/:id/files must be UTF-8 text (e.g. .txt, .md, .json). Binary uploads (PDF / PNG / DOCX) are rejected with 400 unsupported_file_type — no built-in document conversion is performed.
• Once attached, the file is removed from the in-memory file store. The chunks are persisted by the underlying RAG workspace; only the original file_id and filename are kept as attribution metadata so search hits can carry them.

Search results

Search returns OpenAI-shaped vector_store.search_results.page objects. Each chunk's attributes include the originating file_id and filename when they were attached through the file flow.

Videos

OpenAI-compatible async video generation backed by the SDK's video(). Creating a job returns immediately with status: "queued"; the generation runs in the background. Poll for status, then download the bytes.

Two modes are supported:

• txt2vid — JSON body with prompt only. No image needed.
• img2vid — include input_reference as a multipart file field (OpenAI SDK Uploadable), or JSON { image_url } (base64 data URI or HTTP(S) URL), or JSON { file_id } (file uploaded via POST /v1/files). Mode is inferred automatically.

The OpenAI sub-routes /edits, /remix, /extensions, and /characters are not implemented.

Loaded model

Register an alias whose endpoint category is video using the virtual SDK type sdcpp-video (it resolves to the sdcpp-generation addon with mode: "video"). Nested model-source fields (t5XxlModelSrc, vaeModelSrc, clipLModelSrc, …) accept SDK constant names, which the P2P registry resolves to downloadable weights:

{
  "serve": {
    "models": {
      "wan-t2v": {
        "src": "WAN2_1_T2V_1_3B_FP16",
        "type": "sdcpp-video",
        "preload": true,
        "config": {
          "t5XxlModelSrc": "UMT5_XXL_FP16",
          "vaeModelSrc": "WAN_2_1_COMFYUI_REPACKAGED_VAE",
          "offload_to_cpu": true
        }
      },
      "wan-i2v": {
        "src": "WAN2_1_I2V_14B_Q4_K_M",
        "type": "sdcpp-video",
        "preload": true,
        "config": {
          "t5XxlModelSrc": "UMT5_XXL_FP16",
          "vaeModelSrc": "WAN_2_1_COMFYUI_REPACKAGED_VAE",
          "clipVisionModelSrc": "CLIP_VISION_H",
          "offload_to_cpu": true
        }
      }
    }
  }
}

Clients select the model by passing the alias key (or its src string) in the request model field. There is no separate videos aliasing block — to be a drop-in for OpenAI SDK clients (client.videos.create(...), which defaults to model: "sora-2"), name the alias after the OpenAI model the client sends (e.g. "sora-2").

POST /v1/videos

Create a generation job. Accepts application/json (txt2vid or img2vid via { image_url } / { file_id }) or multipart/form-data (img2vid via a binary input_reference file field — this is what the OpenAI SDK sends when given a local File/Blob).

Returns 200 with the Video resource at status: "queued".

Text-to-video (txt2vid) — JSON body with prompt, no reference image:

curl http://localhost:11434/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "wan-t2v",
    "prompt": "a colorful bird flapping its wings in a sunny garden",
    "size": "480x832",
    "seconds": "2",
    "fps": 16,
    "steps": 30,
    "cfg_scale": 6.0,
    "flow_shift": 3.0,
    "negative_prompt": "blurry, low quality, static",
    "seed": 42
  }'

Image-to-video (img2vid) — animate a reference image. Supply input_reference in any of three forms (the job switches to img2vid mode automatically). Use a model whose weights include a vision encoder, e.g. WAN 2.1 I2V (clipVisionModelSrc).

Multipart file field (what the OpenAI SDK sends for a local File/Blob):

curl http://localhost:11434/v1/videos \
  -F "model=wan-i2v" \
  -F "prompt=the cat slowly turns its head and blinks" \
  -F "input_reference=@cat.png" \
  -F "strength=0.6" \
  -F "size=480x832" \
  -F "seconds=2"

JSON with a base64 data URI or HTTP(S) URL (≤ 100 MB, 30 s fetch timeout):

curl http://localhost:11434/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "wan-i2v",
    "prompt": "the cat slowly turns its head and blinks",
    "input_reference": { "image_url": "data:image/png;base64,iVBORw0KGgo..." },
    "strength": 0.6
  }'

JSON referencing a file previously uploaded via POST /v1/files:

curl http://localhost:11434/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "wan-i2v",
    "prompt": "the cat slowly turns its head and blinks",
    "input_reference": { "file_id": "file-abc123" }
  }'

Request fields

The Video resource returned by POST (and by GET /v1/videos/:id):

{
  "id": "video_8f3a…",
  "object": "video",
  "model": "wan-t2v",
  "status": "queued",
  "progress": 0,
  "created_at": 1748800000,
  "completed_at": null,
  "expires_at": 253402300799,
  "prompt": "a colorful bird flapping its wings in a sunny garden",
  "size": "480x832",
  "seconds": "2",
  "remixed_from_video_id": null,
  "error": null
}

progress is a monotonic 0–100 high-water mark. expires_at is a far-future sentinel — the resource itself has no TTL; the rendered bytes expire in the ephemeral file store (after which /content returns 410 video_expired).

GET /v1/videos/:id

Poll job status. status cycles queued → in_progress → completed / failed. Returns the same Video resource shape.

curl http://localhost:11434/v1/videos/video_abc123

GET /v1/videos/:id/content

Download the rendered bytes (only valid once status is completed).

curl http://localhost:11434/v1/videos/video_abc123/content --output out.mp4

• Default container is video/mp4 (fragmented MP4) when ffmpeg is on the server's PATH at startup; otherwise it falls back to video/avi (the SDK's native MJPG-AVI) and logs a warning once.
• ?format=mp4 forces MP4. With no ffmpeg available this returns 503 transcode_unavailable — omit ?format or use ?format=avi.
• ?format=avi forces the native MJPG-AVI and never transcodes.
• The MP4 transcode is lazy and cached: the first fetch after completion may take a few seconds; later fetches serve the cached bytes.
• ?variant other than video (e.g. thumbnail, spritesheet) returns 501 unsupported_variant — those assets are not rendered.

GET /v1/videos

List jobs, newest first by default. Cursor pagination via limit (default 20, max 100), order (asc / desc, default desc), and after. In-memory only — a restart clears the list, and old jobs are dropped once the 256-entry cap is reached.

{
  "object": "list",
  "data": [ { "id": "video_8f3a…", "object": "video", "status": "completed" } ],
  "first_id": "video_8f3a…",
  "last_id": "video_8f3a…",
  "has_more": false
}

DELETE /v1/videos/:id

Abort the job (if still queued / in_progress) and drop its rendered assets.

{ "id": "video_abc123", "object": "video.deleted", "deleted": true }

Errors

Request cancellation

When an HTTP client disconnects before a response finishes (closes the connection or aborts the request), the server cancels the in-flight inference for that request instead of letting it run to completion — freeing the model to serve the next call. This applies to both blocking and streaming requests across the inference routes (/v1/chat/completions, /v1/completions, /v1/responses, /v1/embeddings, /v1/audio/*).

Multiple HTTP requests to the same LLM alias run in parallel up to that model's serve.models.<alias>.config.parallel (see modelConfig.parallel), which defaults to 1 — set it explicitly to serve an alias concurrently. Additional requests wait in a first-come-first-serve queue. A client disconnect cancels only that one request — other requests still running on the same model keep going.

Video jobs are asynchronous and are not tied to the creating connection; cancel them explicitly with DELETE /v1/videos/{id} (see Videos).

Authentication

By default, the server accepts unauthenticated requests on 127.0.0.1. To require a Bearer token, run the server with the --api-key flag:

qvac serve --openai --api-key my-secret-token

Clients must then include the token in the Authorization header:

curl http://localhost:11434/v1/models \
  -H "Authorization: Bearer my-secret-token"

Requests without a valid token receive a 401 response.

Bearer authentication does not apply to the introspection routes: /openapi.json, /docs, and every path under /docs/ remain public even when --api-key is set. /docs and /docs/* exist only when --docs is enabled; /openapi.json is always available.

A non-loopback --host requires authentication: without --api-key or --api-key-file the server refuses to start. Pass --allow-unauthenticated to accept the risk and start anyway, which downgrades the refusal to a warning.

OpenAPI & Swagger UI

The server exposes a machine-readable OpenAPI 3.1.0 document derived from the same schemas it uses to validate requests, so the spec is always in sync with the running server.

GET /openapi.json

Always exposed (no flag required). Returns the full OpenAPI 3.1.0 document as JSON.

curl http://localhost:11434/openapi.json

Each operation in the document carries summary, tags, a full markdown description, the request body schema, and the response schema. Tags group endpoints by domain (Chat, Completions, Embeddings, Responses, Audio, Images, Files, Vector Stores, Models).

GET /docs

Swagger UI, opt-in via the --docs flag. Off by default to keep the production surface minimal.

qvac serve --openai --docs
open http://localhost:11434/docs

--docs adds same-port localhost, 127.0.0.1, and [::1] origins to CORS so a locally opened Swagger UI can use "Try it out." When the configured --host is itself loopback, its same-port origin is added too. It does not trust a bound non-loopback host, a port-forwarded origin, or arbitrary websites. Add each additional browser origin with --cors-origin or serve.cors.origins.

Because those origins are derived from the configured port, --docs requires a fixed port: --docs --port 0 fails startup with a Serve Options Error rather than binding an ephemeral port whose http://localhost:0 origins no browser can match. Pick a port, or drop --docs and read the always-available /openapi.json.

Emit the spec without starting the server

The CLI command qvac openai spec emits the same document without binding a port. Useful for piping into offline documentation generators or for shipping a stable spec file with your project.

qvac openai spec                       # JSON → stdout (pipe-safe)
qvac openai spec -o spec.json          # write JSON to file
qvac openai spec --yaml                # YAML → stdout
qvac openai spec --yaml -o spec.yaml   # write YAML to file

Pairs cleanly with offline doc generators:

qvac openai spec --yaml > openapi.yaml
npx @redocly/cli build-docs openapi.yaml -o api.html