---
title: "Batch processing"
canonical: https://docs.qvac.tether.io/sdk/v0.23/ai-capabilities/batch-processing/
collection: "SDK"
package: "@qvac/sdk"
line: v0.23
current_line: false
---

# Batch processing (/sdk/v0.23/ai-capabilities/batch-processing)



## Overview

**What it does:** `batchCompletion()` runs **multiple prompts through a single loaded model in one call**, instead of firing a separate [`completion()`](/sdk/v0.23/reference/api#completion) call per prompt.

**When to use it:** you have several prompts to run against the same model and want them handled together as one job.

**Why use it:**

* **Optimizes resources:** all prompts share one loaded model — you don't load the model multiple times to run prompts side by side.
* **Lower total run time:** the prompts run together rather than one after another, so the whole set finishes faster than the equivalent sequence of `completion()` calls.

To use it, load the model with `modelConfig.parallel >= 2` (this is what lets it handle more than one prompt at a time), then pass an array of `prompts` — each with its own `history` and, optionally, `generationParams`, `responseFormat`, `tools`, `mcp`, and multimodal `attachments`.

<Callout type="info">
  Batch processing is specific to **LLM text generation**. It does not apply to other capabilities such as embeddings or transcription.
</Callout>

<Callout type="info">
  **Related:** if instead of grouping many prompts into one `batchCompletion()` call you want to fire several independent `completion()` calls in parallel on the same model — one stream per call, cancellable independently — see [Concurrent completions](/sdk/v0.23/ai-capabilities/text-generation#concurrent-completions). Both features use the same `modelConfig.parallel` slots, so mixing them on one model does not add extra concurrency.
</Callout>

## Functions

Use the following sequence of function calls:

1. [`loadModel()`](/sdk/v0.23/reference/api#loadmodel) — load with `modelConfig.parallel >= 2`.
2. [`batchCompletion()`](/sdk/v0.23/reference/api#batchcompletion)
3. [`unloadModel()`](/sdk/v0.23/reference/api#unloadmodel)

For how to use each function, see [SDK — API reference](/sdk/v0.23/reference/api/).

## Models

You can load any [`llama.cpp`](https://github.com/ggml-org/llama.cpp)-compatible text-generation/chat model (same models as [Text generation](/sdk/v0.23/ai-capabilities/text-generation)). Model file format: `*.gguf`.

* Per-prompt `tools`/`mcp` require a tool-capable model loaded with `modelConfig.tools: true`.
* Per-prompt `attachments` require a multimodal-capable model plus its matching `projectionModelSrc` — see [Multimodal](/sdk/v0.23/ai-capabilities/multimodal).
* For models available as constants, see [SDK — Models](/sdk/v0.23/#models).

## Features

`batchCompletion()` returns a `BatchCompletionRun` that exposes several ways to consume the batch:

* **Merged event stream:** `run.events` is a single async iterable yielding `{ id, event }` interleaved across all prompts, where `event` is the same discriminated `CompletionEvent` (`contentDelta`, `thinkingDelta`, `toolCall`, `completionStats`, `completionDone`, etc.) used by `completion()`.
* **Per-prompt streams:** `run.byId(id)` returns `{ events, final }` scoped to one prompt — its own event iterable plus an aggregated `final` promise.
* **Assigned ids:** `run.ids` resolves to the addon-assigned ids in prompt order.
* **Ordered results:** `run.results` resolves to `BatchCompletionResult[]` in prompt order. It is **all-or-nothing for stream-level failures** — if the batch handler throws (e.g. a `ContextOverflowError`), the whole run rejects and no per-prompt finals can be recovered. Graceful terminal events such as cancellation still settle each `byId(id).final` according to that id's terminal state.
* **Batch-level stats:** `run.stats` resolves to a single `CompletionStats | undefined` aggregated across the whole batch (e.g. total tokens), rather than per prompt.
* **Per-prompt tools and MCP:** each prompt can carry its own `tools` and/or `mcp` clients, resolved client-side (including MCP tool discovery) before submission. Tool calls are routed back to the correct prompt.
* **Cancellable:** like any other request — see [Cancellation](/sdk/v0.23/runtime/cancellation).

<Callout type="info">
  Validation rules: prompt `id`s must be unique within a batch, and a prompt's `responseFormat` (`json_object` / `json_schema`) cannot be combined with `tools` or `mcp` (tools already constrain output via their parameter schema).
</Callout>

<Callout type="warn">
  The features above describe the JS/TS `BatchCompletionRun`. The Python and Kotlin clients expose the frame stream underneath it: each frame carries the interleaved `events`, one frame carries the assigned `ids`, and the final frame carries the batch-level `stats`. Group the events by prompt id yourself, as the Python and Kotlin examples do — there is no run object with `byId()` or ordered `results`.
</Callout>

## Example

The following script loads a model with `parallel: 4`, submits several prompts in one `batchCompletion()` call, consumes the merged event stream, and reads the ordered results and batch-level stats:

<Tabs>
  <Tab value="js" label="JavaScript" default>
    <WrapCode>
      ```js file=<rootDir>/packages/sdk/dist/examples/batch-completion.js title="batch-completion.js" lineNumbers
      /**
       * Batch completion — continuous batching over a single loaded LLM.
       *
       * `batchCompletion()` submits multiple prompts to one loaded model in a single
       * call so the llama.cpp addon can decode the active sequences together. Load the
       * model with `modelConfig.parallel >= 2` to make the concurrent slots available.
       *
       * The returned `BatchCompletionRun` exposes:
       *  - `events`     — a merged `AsyncIterable<{ id, event }>` across all prompts.
       *  - `byId(id)`   — per-prompt `events` and a `final` aggregation for one prompt.
       *  - `ids`        — a `Promise<string[]>` of addon-assigned ids in prompt order.
       *  - `results`    — a `Promise<BatchCompletionResult[]>` in prompt order. It is
       *                   ALL-OR-NOTHING for stream-level failures: if the batch
       *                   handler throws (e.g. a context overflow) the whole run
       *                   rejects and no per-prompt finals can be recovered. Graceful
       *                   terminal events such as cancellation still settle each
       *                   `byId(id).final` according to that id's terminal state.
       *  - `stats`      — a `Promise<CompletionStats | undefined>`. Stats are
       *                   BATCH-LEVEL: the addon aggregates decode metrics across the
       *                   whole batch (e.g. `avgConcurrentSeq`, total tokens), so they
       *                   surface once here rather than per prompt. Per-prompt
       *                   `final.stats` is intentionally left undefined.
       *
       * Each prompt may additionally carry (not shown here to keep the run fast/cheap):
       *  - `tools` / `mcp`  — per-prompt tools; handlers attach only to tool calls
       *                       produced by that prompt. Requires a tool-capable model
       *                       loaded with `modelConfig.tools: true` (e.g. Qwen). See
       *                       the `tools/` examples for the tool-calling surface.
       *  - `attachments`    — the same multimodal attachments `completion()` supports,
       *                       when the loaded model has a projection model.
       *  - `responseFormat` — `json_object` / `json_schema` (cannot be combined with
       *                       `tools` or `mcp`, which already constrain output).
       *
       * Run from packages/sdk:
       *   npx tsx examples/batch-completion.ts
       */
      import { batchCompletion, loadModel, unloadModel, LLAMA_3_2_1B_INST_Q4_0 } from '@qvac/sdk';
      try {
          // `parallel: 4` opens the concurrent decode slots continuous batching needs.
          const modelId = await loadModel({
              modelSrc: LLAMA_3_2_1B_INST_Q4_0,
              modelType: 'llm',
              modelConfig: { ctx_size: 4096, parallel: 4 },
              onProgress: (p) => {
                  const mb = (n) => (n / 1e6).toFixed(1);
                  const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
                  process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
                  if (p.percentage >= 100)
                      process.stderr.write('\n');
              }
          });
          console.log(`▸ Model loaded: ${modelId}`);
          const run = batchCompletion({
              modelId,
              prompts: [
                  {
                      id: 'cherry',
                      history: [{ role: 'user', content: 'Reply with only the word CHERRY.' }],
                      generationParams: { temp: 0, seed: 42, predict: 16 }
                  },
                  {
                      id: 'banana',
                      history: [{ role: 'user', content: 'Reply with only the word BANANA.' }],
                      generationParams: { temp: 0, seed: 42, predict: 16 }
                  },
                  {
                      id: 'grape',
                      history: [{ role: 'user', content: 'Reply with only the word GRAPE.' }],
                      generationParams: { temp: 0, seed: 42, predict: 16 }
                  }
              ]
          });
          console.log('\n▸ Merged event stream (interleaved across prompts):');
          for await (const { id, event } of run.events) {
              if (event.type === 'contentDelta') {
                  process.stdout.write(`[${id}] ${event.text}`);
              }
          }
          // `results` is ordered by prompt and all-or-nothing on stream-level failure.
          const results = await run.results;
          console.log('\n\n▸ Ordered results:');
          for (const { id, final } of results) {
              console.log(`  ▸ ${id}: ${final.contentText.replace(/\s+/g, ' ').trim()}`);
          }
          // Per-prompt aggregation is also available directly via `byId(id)`.
          const ids = await run.ids;
          const firstFinal = await run.byId(ids[0]).final;
          console.log(`\n▸ byId("${ids[0]}").final -> ${firstFinal.contentText.trim()}`);
          // Batch-level stats (a single CompletionStats for the whole run, or undefined).
          const stats = await run.stats;
          if (stats) {
              console.log(`\n▸ Batch stats: ${stats.tokensPerSecond?.toFixed(1)} tok/s across the batch`);
          }
          await unloadModel({ modelId, clearStorage: false });
          process.exit(0);
      }
      catch (error) {
          console.error('✖', error);
          process.exit(1);
      }
      ```
    </WrapCode>
  </Tab>

  <Tab value="ts" label="TypeScript">
    <WrapCode>
      ```ts file=<rootDir>/packages/sdk/examples/batch-completion.ts title="batch-completion.ts" lineNumbers
      /**
       * Batch completion — continuous batching over a single loaded LLM.
       *
       * `batchCompletion()` submits multiple prompts to one loaded model in a single
       * call so the llama.cpp addon can decode the active sequences together. Load the
       * model with `modelConfig.parallel >= 2` to make the concurrent slots available.
       *
       * The returned `BatchCompletionRun` exposes:
       *  - `events`     — a merged `AsyncIterable<{ id, event }>` across all prompts.
       *  - `byId(id)`   — per-prompt `events` and a `final` aggregation for one prompt.
       *  - `ids`        — a `Promise<string[]>` of addon-assigned ids in prompt order.
       *  - `results`    — a `Promise<BatchCompletionResult[]>` in prompt order. It is
       *                   ALL-OR-NOTHING for stream-level failures: if the batch
       *                   handler throws (e.g. a context overflow) the whole run
       *                   rejects and no per-prompt finals can be recovered. Graceful
       *                   terminal events such as cancellation still settle each
       *                   `byId(id).final` according to that id's terminal state.
       *  - `stats`      — a `Promise<CompletionStats | undefined>`. Stats are
       *                   BATCH-LEVEL: the addon aggregates decode metrics across the
       *                   whole batch (e.g. `avgConcurrentSeq`, total tokens), so they
       *                   surface once here rather than per prompt. Per-prompt
       *                   `final.stats` is intentionally left undefined.
       *
       * Each prompt may additionally carry (not shown here to keep the run fast/cheap):
       *  - `tools` / `mcp`  — per-prompt tools; handlers attach only to tool calls
       *                       produced by that prompt. Requires a tool-capable model
       *                       loaded with `modelConfig.tools: true` (e.g. Qwen). See
       *                       the `tools/` examples for the tool-calling surface.
       *  - `attachments`    — the same multimodal attachments `completion()` supports,
       *                       when the loaded model has a projection model.
       *  - `responseFormat` — `json_object` / `json_schema` (cannot be combined with
       *                       `tools` or `mcp`, which already constrain output).
       *
       * Run from packages/sdk:
       *   npx tsx examples/batch-completion.ts
       */

      import { batchCompletion, loadModel, unloadModel, LLAMA_3_2_1B_INST_Q4_0 } from '@qvac/sdk'

      try {
        // `parallel: 4` opens the concurrent decode slots continuous batching needs.
        const modelId = await loadModel({
          modelSrc: LLAMA_3_2_1B_INST_Q4_0,
          modelType: 'llm',
          modelConfig: { ctx_size: 4096, parallel: 4 },
          onProgress: (p) => {
            const mb = (n: number) => (n / 1e6).toFixed(1)
            const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`
            process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`)
            if (p.percentage >= 100) process.stderr.write('\n')
          }
        })
        console.log(`▸ Model loaded: ${modelId}`)

        const run = batchCompletion({
          modelId,
          prompts: [
            {
              id: 'cherry',
              history: [{ role: 'user', content: 'Reply with only the word CHERRY.' }],
              generationParams: { temp: 0, seed: 42, predict: 16 }
            },
            {
              id: 'banana',
              history: [{ role: 'user', content: 'Reply with only the word BANANA.' }],
              generationParams: { temp: 0, seed: 42, predict: 16 }
            },
            {
              id: 'grape',
              history: [{ role: 'user', content: 'Reply with only the word GRAPE.' }],
              generationParams: { temp: 0, seed: 42, predict: 16 }
            }
          ]
        })

        console.log('\n▸ Merged event stream (interleaved across prompts):')
        for await (const { id, event } of run.events) {
          if (event.type === 'contentDelta') {
            process.stdout.write(`[${id}] ${event.text}`)
          }
        }

        // `results` is ordered by prompt and all-or-nothing on stream-level failure.
        const results = await run.results
        console.log('\n\n▸ Ordered results:')
        for (const { id, final } of results) {
          console.log(`  ▸ ${id}: ${final.contentText.replace(/\s+/g, ' ').trim()}`)
        }

        // Per-prompt aggregation is also available directly via `byId(id)`.
        const ids = await run.ids
        const firstFinal = await run.byId(ids[0]!).final
        console.log(`\n▸ byId("${ids[0]}").final -> ${firstFinal.contentText.trim()}`)

        // Batch-level stats (a single CompletionStats for the whole run, or undefined).
        const stats = await run.stats
        if (stats) {
          console.log(`\n▸ Batch stats: ${stats.tokensPerSecond?.toFixed(1)} tok/s across the batch`)
        }

        await unloadModel({ modelId, clearStorage: false })
        process.exit(0)
      } catch (error) {
        console.error('✖', error)
        process.exit(1)
      }
      ```
    </WrapCode>
  </Tab>

  <Tab value="python" label="Python">
    <WrapCode>
      ```python file=<rootDir>/packages/sdk-python/examples/batch_completion.py title="batch_completion.py" lineNumbers
      """Batch completion — continuous batching over a single loaded LLM.

      Python port of packages/sdk/examples/batch-completion.ts.

      `batch_completion_stream` submits multiple prompts to one loaded model in a
      single call, so the llama.cpp addon decodes the active sequences together. Load
      the model with `parallel >= 2` to make the concurrent slots available.

      Shape difference from the JS SDK: there, `batchCompletion()` returns a
      `BatchCompletionRun` with a merged event stream, `byId(id)`, ordered `results`
      and batch-level `stats`. Python's public surface is the frame stream underneath
      that object, so this example does the grouping itself:

       - every frame carries `events`, a list of `{id, event}` records interleaved
         across prompts, where `event` is the union `completion` also streams;
       - the frame carrying `ids` gives the addon-assigned ids, in prompt order;
       - the final frame carries `stats`, one aggregate for the whole batch (e.g.
         `avg_concurrent_seq`), not one per prompt.

      Each prompt may also carry `tools`, `attachments` or `responseFormat`, as in
      the JS example; they are left out here to keep the run fast.

      RUN: python examples/batch_completion.py
      """

      from __future__ import annotations

      import asyncio
      import sys

      from tetherto.qvac_sdk import (
          BatchCompletionStreamRequest,
          Client,
          batch_completion_stream,
          load_model,
          unload_model,
      )
      from tetherto.qvac_sdk.models import LLAMA_3_2_1B_INST_Q4_0

      PROMPTS = [
          {
              "id": "cherry",
              "history": [{"role": "user", "content": "Reply with only the word CHERRY."}],
              "generationParams": {"temp": 0, "seed": 42, "predict": 16},
          },
          {
              "id": "banana",
              "history": [{"role": "user", "content": "Reply with only the word BANANA."}],
              "generationParams": {"temp": 0, "seed": 42, "predict": 16},
          },
          {
              "id": "grape",
              "history": [{"role": "user", "content": "Reply with only the word GRAPE."}],
              "generationParams": {"temp": 0, "seed": 42, "predict": 16},
          },
      ]


      def print_progress(p) -> None:
          """Print model download progress; pass as `on_progress=` to `load_model`."""
          line = (
              f"▸ Downloading {p.percentage:.0f}% "
              f"({p.downloaded / 1e6:.1f}/{p.total / 1e6:.1f} MB)"
          )
          print(line, end="\r" if sys.stderr.isatty() else "\n", file=sys.stderr)
          if p.percentage >= 100:
              print(file=sys.stderr)


      async def main() -> int:
          async with Client() as client:
              t = client.transport
              model_id = None
              try:
                  # `parallel: 4` opens the concurrent decode slots batching needs.
                  model_id = await load_model(
                      t,
                      model_src=LLAMA_3_2_1B_INST_Q4_0,
                      model_config={"ctx_size": 4096, "parallel": 4},
                      on_progress=print_progress,
                  )
                  print(f"▸ Model loaded: {model_id}")

                  request = BatchCompletionStreamRequest.model_validate(
                      {
                          "type": "batchCompletionStream",
                          "modelId": model_id,
                          "prompts": PROMPTS,
                      }
                  )

                  # Prompt order to start with; replaced by the addon-assigned ids
                  # once the frame that carries them arrives.
                  ids = [prompt["id"] for prompt in PROMPTS]
                  texts: dict[str, str] = {}
                  stats = None

                  print("\n▸ Merged event stream (interleaved across prompts):")
                  async for frame in batch_completion_stream(t, request):
                      if frame.ids:
                          ids = list(frame.ids)
                      for record in frame.events:
                          if record.event.type != "contentDelta":
                              continue
                          sys.stdout.write(f"[{record.id}] {record.event.text}")
                          sys.stdout.flush()
                          texts[record.id] = texts.get(record.id, "") + record.event.text
                      if frame.done:
                          stats = frame.stats

                  print("\n\n▸ Ordered results:")
                  for prompt_id in ids:
                      collapsed = " ".join(texts.get(prompt_id, "").split())
                      print(f"  ▸ {prompt_id}: {collapsed}")

                  # The per-prompt aggregation JS exposes as `byId(id).final` is this
                  # lookup: everything collected for that one id.
                  print(f'\n▸ texts["{ids[0]}"] -> {texts.get(ids[0], "").strip()}')

                  # Batch-level stats: one aggregate for the whole run.
                  if stats is not None and stats.tokens_per_second is not None:
                      print(
                          f"\n▸ Batch stats: {stats.tokens_per_second:.1f}"
                          " tok/s across the batch"
                      )
              except Exception as error:
                  print(f"✖ {error}", file=sys.stderr)
                  return 1
              finally:
                  if model_id is not None:
                      await unload_model(t, model_id)
          return 0


      if __name__ == "__main__":
          sys.exit(asyncio.run(main()))
      ```
    </WrapCode>
  </Tab>

  <Tab value="kotlin" label="Kotlin">
    <WrapCode>
      ```kotlin file=<rootDir>/packages/sdk-kotlin/examples/src/BatchCompletion.kt title="BatchCompletion.kt" lineNumbers
      // Kotlin port of packages/sdk/examples/batch-completion.ts.
      //
      // Batch completion — continuous batching over a single loaded LLM.
      //
      // `client.completion.batch()` submits multiple prompts to one loaded model in a
      // single call, so the llama.cpp addon decodes the active sequences together.
      // Load the model with `parallel >= 2` to make the concurrent slots available.
      //
      // As in Python, the Kotlin surface is the typed frame stream (a `Flow` of
      // `BatchCompletionStreamResponse`), so this example does the grouping itself:
      //
      //   - every frame carries `events`, a list of `{id, event}` records interleaved
      //     across prompts, where `event` is a sealed class (`ContentDelta`,
      //     `ToolCall`, `CompletionStats`, ...);
      //   - the frame carrying `ids` gives the addon-assigned ids, in prompt order;
      //   - the final frame carries `stats`, one aggregate for the whole batch (e.g.
      //     `avgConcurrentSeq`), not one per prompt.
      //
      // The prompts are built from the generated request types, so a misspelled field
      // or a wrong type is a compile error rather than a worker validation error.
      // Each prompt may also carry `tools`, `attachments` or `responseFormat`, as in
      // the JS example; they are left out here to keep the run fast.
      //
      // RUN: ./gradlew -p examples run -Pexample=BatchCompletion

      import io.tether.qvac.sdk.QvacClient
      import io.tether.qvac.sdk.QvacProgressEvent
      import io.tether.qvac.sdk.completion
      import io.tether.qvac.sdk.generated.ModelConstant
      import io.tether.qvac.sdk.generated.Models
      import io.tether.qvac.sdk.generated.schema.BatchCompletionStreamRequest
      import io.tether.qvac.sdk.generated.schema.BatchCompletionStreamRequestPromptsItem
      import io.tether.qvac.sdk.generated.schema.BatchCompletionStreamRequestPromptsItemGenerationParams
      import io.tether.qvac.sdk.generated.schema.BatchCompletionStreamRequestPromptsItemHistoryItem
      import io.tether.qvac.sdk.generated.schema.BatchCompletionStreamRequestPromptsItemHistoryItemContent
      import io.tether.qvac.sdk.generated.schema.BatchCompletionStreamResponseEventsItemEvent
      import io.tether.qvac.sdk.generated.schema.BatchCompletionStreamResponseStats
      import io.tether.qvac.sdk.generated.schema.LoadModelResponse
      import io.tether.qvac.sdk.generated.schema.ModelProgressResponse
      import io.tether.qvac.sdk.jvm.JvmBareKitTransport
      import io.tether.qvac.sdk.models
      import kotlinx.coroutines.runBlocking
      import kotlinx.serialization.json.JsonObject
      import kotlinx.serialization.json.buildJsonObject
      import kotlinx.serialization.json.put

      private val PROMPTS = listOf(
          "cherry" to "Reply with only the word CHERRY.",
          "banana" to "Reply with only the word BANANA.",
          "grape" to "Reply with only the word GRAPE.",
      ).map { (id, ask) ->
          BatchCompletionStreamRequestPromptsItem(
              id = id,
              history = listOf(
                  BatchCompletionStreamRequestPromptsItemHistoryItem(
                      role = "user",
                      content = BatchCompletionStreamRequestPromptsItemHistoryItemContent.Variant1(ask),
                  ),
              ),
              generationParams = BatchCompletionStreamRequestPromptsItemGenerationParams(
                  temp = 0.0,
                  seed = 42.0,
                  predict = 16.0,
              ),
          )
      }

      fun main(): Unit = runBlocking {
          val client = QvacClient(JvmBareKitTransport.connect())
          var modelId: String? = null
          try {
              // `parallel: 4` opens the concurrent decode slots batching needs.
              modelId = client.loadWithProgress(
                  Models.LLAMA_3_2_1B_INST_Q4_0,
                  modelConfig = buildJsonObject {
                      put("ctx_size", 4096)
                      put("parallel", 4)
                  },
              )
              println("▸ Model loaded: $modelId")

              val request = BatchCompletionStreamRequest(modelId = modelId, prompts = PROMPTS)

              // Prompt order to start with; replaced by the addon-assigned ids once
              // the frame that carries them arrives.
              var ids = PROMPTS.mapNotNull { it.id }
              val texts = mutableMapOf<String, String>()
              var stats: BatchCompletionStreamResponseStats? = null

              println("\n▸ Merged event stream (interleaved across prompts):")
              client.completion.batch(request).collect { frame ->
                  frame.ids?.takeIf { it.isNotEmpty() }?.let { ids = it }
                  for (record in frame.events) {
                      val event = record.event as? BatchCompletionStreamResponseEventsItemEvent.ContentDelta ?: continue
                      print("[${record.id}] ${event.value.text}")
                      System.out.flush()
                      texts[record.id] = texts.getOrDefault(record.id, "") + event.value.text
                  }
                  if (frame.done == true) stats = frame.stats
              }

              println("\n\n▸ Ordered results:")
              for (promptId in ids) {
                  println("  ▸ $promptId: ${texts.getOrDefault(promptId, "").collapseWhitespace()}")
              }

              // The per-prompt aggregation JS exposes as `byId(id).final` is this
              // lookup: everything collected for that one id.
              println("\n▸ texts[\"${ids[0]}\"] -> ${texts.getOrDefault(ids[0], "").trim()}")

              // Batch-level stats: one aggregate for the whole run.
              stats?.tokensPerSecond?.let { println("\n▸ Batch stats: %.1f tok/s across the batch".format(it)) }
          } finally {
              modelId?.let { client.models.unload(it) }
              client.close()
          }
      }

      private fun String.collapseWhitespace(): String = trim().split(Regex("\\s+")).joinToString(" ")

      /** Load [model], printing download progress, and return its model id. */
      private suspend fun QvacClient.loadWithProgress(model: ModelConstant, modelConfig: JsonObject? = null): String {
          var result: LoadModelResponse? = null
          models.loadWithProgress(model.src, model.engine, model.name, modelConfig = modelConfig).collect { event ->
              when (event) {
                  is QvacProgressEvent.Progress -> printProgress(event.value)
                  is QvacProgressEvent.Result -> result = event.value
              }
          }
          val loaded = checkNotNull(result) { "loadModel ended without a result" }
          return loaded.modelId ?: error(loaded.error ?: "loadModel failed")
      }

      private fun printProgress(p: ModelProgressResponse) {
          val line = "▸ Downloading %.0f%% (%.1f/%.1f MB)".format(p.percentage, p.downloaded / 1e6, p.total / 1e6)
          if (System.console() != null) System.err.print("\r$line") else System.err.println(line)
          if (p.percentage >= 100) System.err.println()
      }
      ```
    </WrapCode>
  </Tab>
</Tabs>

<Callout type="success">
  **Tip:** all examples throughout this documentation are self-contained and runnable. For instructions on how to run them, see the [JS/TS quickstart](/sdk/v0.23/js-ts-sdk#quickstart), the [Python quickstart](/sdk/v0.23/python-sdk#quickstart), or the [Kotlin quickstart](/sdk/v0.23/kotlin-sdk#quickstart).
</Callout>
