---
title: "World simulation"
canonical: https://docs.qvac.tether.io/sdk/v0.23/ai-capabilities/world-simulation/
collection: "SDK"
package: "@qvac/sdk"
line: v0.23
current_line: false
---

# World simulation (/sdk/v0.23/ai-capabilities/world-simulation)



## Overview

World simulation runs on a **customized Diffusion engine** ([`qvac-ext-stable-diffusion.cpp`](https://github.com/tetherto/qvac-ext-stable-diffusion.cpp)) exposed as an interactive session powered by **ABot-World**. Load a supported model using `modelType: "sdcpp-generation"` with `modelConfig.mode: "world"`. Then call `worldCreateScene()` once to build a world from a text prompt and a first-frame image, and `worldStep()` repeatedly to walk it. Each call generates one block of video streamed frame-by-frame.

The session runs on the machine hosting the worker; there is no remote route.

<Callout type="warn">
  World simulation is hardware-intensive. With weights held in video memory (the default), the **832×480** tier needs **≥20 GB** (a 24 GB card in practice). [Layer streaming](#fitting-the-walk-into-less-video-memory) changes where weights live, and a minimum for a streamed 832×480 walk has not been measured yet. Pass **448×256** to use the low-VRAM tier, which runs on \~6 GB cards well below interactive frame rates.
</Callout>

## Functions

Use the following sequence of function calls:

1. [`loadModel()`](/sdk/v0.23/reference/api#loadmodel)
2. [`worldCreateScene()`](/sdk/v0.23/reference/api) — once per world
3. [`worldStep()`](/sdk/v0.23/reference/api) — repeatedly, one call per generated block
4. [`unloadModel()`](/sdk/v0.23/reference/api#unloadmodel)

For how to use each function, see [SDK — API reference](/sdk/v0.23/reference/api/).

### `worldCreateScene()`

Builds the world from a text `prompt` and a first-frame `image` (`Uint8Array` of PNG or JPEG bytes, ≤ 3 MB). Optional `width` and `height` are positive multiples of **32**, at most **4096** each, and `width × height` must stay within **2,088,960** pixels (1920×1088). They default to **832×480**.

It returns `{ requestId, stats }`, plus `scene` (a `Promise<Uint8Array>` holding the scene pack) when you pass `returnPack: true`. Save that pack and load it back later with `modelConfig.sceneSrc` to walk the same world again without regenerating it. Calling `worldCreateScene()` twice on the same session replaces the world and restarts the walk.

### `worldStep()`

Generates the next block of the walk under the `keys` held for it. One call produces one block — a second call while one is in flight is rejected, so drive the next call off the previous one.

`keys` accepts three equivalent forms:

* an array of key names (`["W", "L"]`),
* a key-state object (`{ W: true, L: true }`), or
* a raw 8-bit mask.

Use **WASD** to move (forward, left, back, right) and **IJKL** to steer the camera (up, left, down, right). Pass an empty value for an idle block.

It returns `{ requestId, frameStream, progressStream, frames, stats }`:

* `frameStream` — an async iterable of decoded frames, PNG by default or JPEG when `modelConfig.world.frameJpegQuality` is set. Display each frame as it arrives.
* `progressStream` — emits one summary tick per block, **after** the frames.
* `frames` — a `Promise` resolving to the whole block once decoding is done.
* `stats` — timing and frame counts.

Cancelling with `cancel({ requestId })` is block-granular: the current block finishes internally, delivery stops, and the call rejects with `InferenceCancelledError`. A cancel that lands after the block already completed resolves normally — handle both outcomes.

## Models

ABot-World uses a split layout with four models:

* `ABOT_WORLD_0_5B_Q8_0` — the world-model transformer (DiT). Passed as `modelSrc`.
* `ABOT_WORLD_0_5B_LF_TAEHV_VAE` — the taehv streaming decoder used on every step. Passed as `modelConfig.taehvModelSrc`.
* `ABOT_WORLD_0_5B_LF_WAN_VAE` — the full-precision Wan2.2 VAE that encodes the first frame during `worldCreateScene`. Passed as `modelConfig.vaeModelSrc`.
* `UMT5_XXL_ENC_Q8_0` — the UMT5 text encoder for the prompt. Passed as `modelConfig.t5XxlModelSrc`.

```ts
import {
  ABOT_WORLD_0_5B_Q8_0,
  ABOT_WORLD_0_5B_LF_TAEHV_VAE,
  ABOT_WORLD_0_5B_LF_WAN_VAE,
  UMT5_XXL_ENC_Q8_0,
  loadModel,
} from "@qvac/sdk";

const modelId = await loadModel({
  modelSrc: ABOT_WORLD_0_5B_Q8_0,
  modelType: "sdcpp-generation",
  modelConfig: {
    mode: "world",
    taehvModelSrc: ABOT_WORLD_0_5B_LF_TAEHV_VAE,
    t5XxlModelSrc: UMT5_XXL_ENC_Q8_0,
    vaeModelSrc: ABOT_WORLD_0_5B_LF_WAN_VAE,
    world: { kvCache: true, frameJpegQuality: 85 },
  },
});
```

Set `world.numFramePerBlock` to control the block size; the SDK caps it at 64.

For models available as constants, see [SDK — Models](/sdk/v0.23/#models).

The weights are a GGML conversion of the [ABot-World project](https://github.com/amap-cvlab/ABot-World)'s [published models](https://huggingface.co/acvlab/ABot-World-0-5B-LF), used under the Apache-2.0 licence.

## Fitting the walk into less video memory

Four further `world` fields trade throughput for VRAM, so a session can run on a card that cannot hold the transformer outright:

```ts
world: {
  paramsBackend: "diffusion=cpu",
  maxVram: 2,
  streamLayers: true,
  kvCache: true,
  verbosity: 3,
},
```

* `paramsBackend` — where weights live, independently of where graphs run. `diffusion` is the walk transformer, `vae` the streaming decoder. `"diffusion=cpu"` stages transformer weights from system RAM; `"diffusion=disk"` reads them from the model file on demand. Disk is never chosen automatically, and `vae=disk` is refused because the decoder keeps its prepared weights between steps.
* `maxVram` — the video-memory budget for the transformer graph, in GiB or per device (`"cuda0=6,vulkan0=4"`). Negative values reserve that much headroom out of free memory; `0` turns budgeting off. This is **not** a cap on total video memory: the attention history, the decoder and the scene allocate on top of it.
* `streamLayers` — keeps the leading transformer segments inside the budget and transfers the rest from system RAM. It needs all three of CPU-backed transformer weights, GPU execution and a nonzero `maxVram`; with any one missing it does nothing.
* `verbosity` — the engine log level, `0` (errors) to `3` (debug). Use `3` to see the chosen budget and which layers are being streamed.

These live under `world` rather than at the top level. The flat `params_backend`, `max_vram`, `stream_layers` and `verbosity` accepted by image and video generation do not reach a walk session, so passing one in world mode fails at `loadModel()` and names the `world.*` field to use instead.

[`assessModelFit()`](/sdk/v0.23/reference/api#assessmodelfit) and the fit check in [`loadModel()`](/sdk/v0.23/reference/api#loadmodel) size a walk when the load names a pre-built `sceneSrc`: they count the transformer, the decoder, the attention caches and the history a walk accumulates. `world.fitSteps` (default `100`) sets how many walk steps that estimate covers; it does not limit the walk. Scene creation is not part of the estimate, so a load without `sceneSrc` gets no engine estimate.

## Example

The following script loads an ABot-World session, creates a world from a first-frame image, walks it for several blocks under a scripted key tape, and writes each decoded frame to disk:

<Tabs>
  <Tab value="js" label="JavaScript" default>
    <WrapCode>
      ```js file=<rootDir>/packages/sdk/dist/examples/abot-world.js title="abot-world.js" lineNumbers
      import { cancel, loadModel, unloadModel, worldCreateScene, worldStep, ABOT_WORLD_0_5B_Q8_0, ABOT_WORLD_0_5B_LF_TAEHV_VAE, ABOT_WORLD_0_5B_LF_WAN_VAE, UMT5_XXL_ENC_Q8_0 } from '@qvac/sdk';
      import fs from 'fs';
      import path from 'path';
      // ABot-World: build a world from a single image, then walk it. Each step
      // generates one block of video under the keys held for it and streams the
      // decoded frames.
      //
      // Runs on the machine hosting the worker; there is no remote route. The
      // default 832x480 tier needs >= 20 GB free VRAM on a dedicated GPU (24 GB card
      // in practice); pass 448 256 to use the low-VRAM tier, which runs on ~6 GB
      // cards well below interactive frame rates.
      const firstFramePath = process.argv[2];
      const prompt = process.argv[3] || '| unknown | A realistic outdoor world scene with a navigable path.';
      const outputDir = process.argv[4] || '.';
      const blocks = Number(process.argv[5] || 8);
      const width = Number(process.argv[6] || 832);
      const height = Number(process.argv[7] || 480);
      if (!firstFramePath) {
          console.error('✖ first frame image path is required');
          console.error('Usage: node dist/examples/abot-world.js ' +
              '<firstFrameImage> [prompt] [outputDir] [blocks] [width] [height]');
          process.exit(1);
      }
      // A scripted walk: forward, forward while looking right, forward, idle. `keys`
      // also accepts a key-state object ({ W: true }) or a raw 8-bit mask, so a live
      // keyboard handler can pass its state straight through.
      const TAPE = [['W'], ['W', 'L'], ['W'], []];
      try {
          console.log('▸ Loading ABot-World session (DiT + taehv + scene encoders)...');
          const modelId = await loadModel({
              modelSrc: ABOT_WORLD_0_5B_Q8_0,
              modelType: 'sdcpp-generation',
              modelConfig: {
                  mode: 'world',
                  // Two different VAEs, and the names are close enough to swap by accident:
                  //   taehvModelSrc — taew2_2, the tiny STREAMING decoder. Turns each block's
                  //     latents into pixels during the walk. Used by every worldStep.
                  //   vaeModelSrc   — the full-precision Wan2.2 VAE. ENCODES the first frame
                  //     during worldCreateScene, and is not touched again after that.
                  // Swapping them loads at first and fails once the walk starts.
                  taehvModelSrc: ABOT_WORLD_0_5B_LF_TAEHV_VAE,
                  t5XxlModelSrc: UMT5_XXL_ENC_Q8_0,
                  vaeModelSrc: ABOT_WORLD_0_5B_LF_WAN_VAE,
                  world: {
                      seed: 42,
                      // ~3.7x fewer frame-passes per block. Without it, block times ramp from
                      // ~1.8 s to ~7.5 s as the recompute window fills.
                      kvCache: true,
                      // JPEG rather than PNG: a block is ~14 MB of raw pixels, and this
                      // example writes every frame to disk.
                      frameJpegQuality: 85
                  }
              }
          });
          console.log('▸ Creating the world (umT5 prompt encode + Wan2.2 VAE first-frame encode)...');
          const creation = worldCreateScene({
              modelId,
              prompt,
              image: new Uint8Array(fs.readFileSync(firstFramePath)),
              width,
              height,
              // The world is live on the session without this; ask for the bytes only
              // because this example saves them to walk the same world again later.
              returnPack: true
          });
          const scene = await creation.scene;
          const sceneStats = await creation.stats;
          const scenePath = path.join(outputDir, 'world.safetensors');
          fs.writeFileSync(scenePath, scene);
          console.log(`  scene pack written to ${scenePath} (${scene.length} bytes, ${sceneStats?.sceneCreateMs ?? '?'} ms)`);
          console.log('  reuse it later with modelConfig.sceneSrc to walk this same world again');
          console.log(`▸ Walking ${blocks} blocks...`);
          let frameNumber = 0;
          for (let block = 0; block < blocks; block++) {
              const keys = TAPE[block % TAPE.length];
              // One block at a time: a second step while this one streams is rejected,
              // so drive the next call off this one rather than firing them in parallel.
              const run = worldStep({ modelId, keys });
              for await (const frame of run.frameStream) {
                  const framePath = path.join(outputDir, `frame-${String(frameNumber++).padStart(4, '0')}.jpg`);
                  fs.writeFileSync(framePath, frame);
              }
              const stats = await run.stats;
              console.log(`  block ${block + 1}/${blocks} [${keys.join('+') || 'idle'}] — ` +
                  `${stats?.frames ?? '?'} frames in ${stats?.stepMs ?? '?'} ms`);
          }
          console.log(`✔ Wrote ${frameNumber} frames to ${outputDir}`);
          // Targeted cancellation. Cancel is block-granular: the engine has no
          // mid-block abort, so the current block finishes internally, delivery stops,
          // and the step rejects rather than returning a truncated block. A cancel that
          // lands after the block already finished legitimately succeeds instead — both
          // outcomes are correct, so handle each.
          console.log('▸ Cancelling a block mid-flight to show the contract...');
          const cancelling = worldStep({ modelId, keys: ['W'] });
          // The timer can fire after the block already finished, and cancelling a
          // request that is no longer in flight rejects. Handle it here rather than
          // leaving a floating promise: an unhandled rejection from a fire-and-forget
          // cancel would take the whole example down for the race it is demonstrating.
          const cancelTimer = setTimeout(() => {
              cancel({ requestId: cancelling.requestId }).catch(() => {
                  // The block won the race; the await below reports that outcome.
              });
          }, 200);
          try {
              await cancelling.frames;
              console.log('  the block completed before the cancel landed (the tolerated race)');
          }
          catch (error) {
              console.log(`  cancelled as expected: ${error instanceof Error ? error.message : String(error)}`);
          }
          finally {
              clearTimeout(cancelTimer);
          }
          // A cancelled step is terminal for the native session, but the SDK drops it
          // for you: another worldStep on this same modelId would transparently rebuild
          // from the promoted pack, with no unloadModel/loadModel cycle. Unloading here
          // only because the example is finished.
          await unloadModel({ modelId });
      }
      catch (error) {
          console.error('✖ World walk failed:', error instanceof Error ? error.message : error);
          process.exit(1);
      }
      process.exit(0);
      ```
    </WrapCode>
  </Tab>

  <Tab value="ts" label="TypeScript">
    <WrapCode>
      ```ts file=<rootDir>/packages/sdk/examples/abot-world.ts title="abot-world.ts" lineNumbers
      import {
        cancel,
        loadModel,
        unloadModel,
        worldCreateScene,
        worldStep,
        ABOT_WORLD_0_5B_Q8_0,
        ABOT_WORLD_0_5B_LF_TAEHV_VAE,
        ABOT_WORLD_0_5B_LF_WAN_VAE,
        UMT5_XXL_ENC_Q8_0
      } from '@qvac/sdk'
      import fs from 'fs'
      import path from 'path'

      // ABot-World: build a world from a single image, then walk it. Each step
      // generates one block of video under the keys held for it and streams the
      // decoded frames.
      //
      // Runs on the machine hosting the worker; there is no remote route. The
      // default 832x480 tier needs >= 20 GB free VRAM on a dedicated GPU (24 GB card
      // in practice); pass 448 256 to use the low-VRAM tier, which runs on ~6 GB
      // cards well below interactive frame rates.
      const firstFramePath = process.argv[2]
      const prompt =
        process.argv[3] || '| unknown | A realistic outdoor world scene with a navigable path.'
      const outputDir = process.argv[4] || '.'
      const blocks = Number(process.argv[5] || 8)
      const width = Number(process.argv[6] || 832)
      const height = Number(process.argv[7] || 480)

      if (!firstFramePath) {
        console.error('✖ first frame image path is required')
        console.error(
          'Usage: node dist/examples/abot-world.js ' +
            '<firstFrameImage> [prompt] [outputDir] [blocks] [width] [height]'
        )
        process.exit(1)
      }

      // A scripted walk: forward, forward while looking right, forward, idle. `keys`
      // also accepts a key-state object ({ W: true }) or a raw 8-bit mask, so a live
      // keyboard handler can pass its state straight through.
      const TAPE: string[][] = [['W'], ['W', 'L'], ['W'], []]

      try {
        console.log('▸ Loading ABot-World session (DiT + taehv + scene encoders)...')
        const modelId = await loadModel({
          modelSrc: ABOT_WORLD_0_5B_Q8_0,
          modelType: 'sdcpp-generation',
          modelConfig: {
            mode: 'world',
            // Two different VAEs, and the names are close enough to swap by accident:
            //   taehvModelSrc — taew2_2, the tiny STREAMING decoder. Turns each block's
            //     latents into pixels during the walk. Used by every worldStep.
            //   vaeModelSrc   — the full-precision Wan2.2 VAE. ENCODES the first frame
            //     during worldCreateScene, and is not touched again after that.
            // Swapping them loads at first and fails once the walk starts.
            taehvModelSrc: ABOT_WORLD_0_5B_LF_TAEHV_VAE,
            t5XxlModelSrc: UMT5_XXL_ENC_Q8_0,
            vaeModelSrc: ABOT_WORLD_0_5B_LF_WAN_VAE,
            world: {
              seed: 42,
              // ~3.7x fewer frame-passes per block. Without it, block times ramp from
              // ~1.8 s to ~7.5 s as the recompute window fills.
              kvCache: true,
              // JPEG rather than PNG: a block is ~14 MB of raw pixels, and this
              // example writes every frame to disk.
              frameJpegQuality: 85
            }
          }
        })

        console.log('▸ Creating the world (umT5 prompt encode + Wan2.2 VAE first-frame encode)...')
        const creation = worldCreateScene({
          modelId,
          prompt,
          image: new Uint8Array(fs.readFileSync(firstFramePath)),
          width,
          height,
          // The world is live on the session without this; ask for the bytes only
          // because this example saves them to walk the same world again later.
          returnPack: true
        })
        const scene = await creation.scene
        const sceneStats = await creation.stats
        const scenePath = path.join(outputDir, 'world.safetensors')
        fs.writeFileSync(scenePath, scene)
        console.log(
          `  scene pack written to ${scenePath} (${scene.length} bytes, ${sceneStats?.sceneCreateMs ?? '?'} ms)`
        )
        console.log('  reuse it later with modelConfig.sceneSrc to walk this same world again')

        console.log(`▸ Walking ${blocks} blocks...`)
        let frameNumber = 0
        for (let block = 0; block < blocks; block++) {
          const keys = TAPE[block % TAPE.length]!
          // One block at a time: a second step while this one streams is rejected,
          // so drive the next call off this one rather than firing them in parallel.
          const run = worldStep({ modelId, keys })

          for await (const frame of run.frameStream) {
            const framePath = path.join(outputDir, `frame-${String(frameNumber++).padStart(4, '0')}.jpg`)
            fs.writeFileSync(framePath, frame)
          }

          const stats = await run.stats
          console.log(
            `  block ${block + 1}/${blocks} [${keys.join('+') || 'idle'}] — ` +
              `${stats?.frames ?? '?'} frames in ${stats?.stepMs ?? '?'} ms`
          )
        }

        console.log(`✔ Wrote ${frameNumber} frames to ${outputDir}`)

        // Targeted cancellation. Cancel is block-granular: the engine has no
        // mid-block abort, so the current block finishes internally, delivery stops,
        // and the step rejects rather than returning a truncated block. A cancel that
        // lands after the block already finished legitimately succeeds instead — both
        // outcomes are correct, so handle each.
        console.log('▸ Cancelling a block mid-flight to show the contract...')
        const cancelling = worldStep({ modelId, keys: ['W'] })
        // The timer can fire after the block already finished, and cancelling a
        // request that is no longer in flight rejects. Handle it here rather than
        // leaving a floating promise: an unhandled rejection from a fire-and-forget
        // cancel would take the whole example down for the race it is demonstrating.
        const cancelTimer = setTimeout(() => {
          cancel({ requestId: cancelling.requestId }).catch(() => {
            // The block won the race; the await below reports that outcome.
          })
        }, 200)
        try {
          await cancelling.frames
          console.log('  the block completed before the cancel landed (the tolerated race)')
        } catch (error) {
          console.log(
            `  cancelled as expected: ${error instanceof Error ? error.message : String(error)}`
          )
        } finally {
          clearTimeout(cancelTimer)
        }

        // A cancelled step is terminal for the native session, but the SDK drops it
        // for you: another worldStep on this same modelId would transparently rebuild
        // from the promoted pack, with no unloadModel/loadModel cycle. Unloading here
        // only because the example is finished.
        await unloadModel({ modelId })
      } catch (error) {
        console.error('✖ World walk failed:', error instanceof Error ? error.message : error)
        process.exit(1)
      }

      process.exit(0)
      ```
    </WrapCode>
  </Tab>

  <Tab value="python" label="Python">
    <WrapCode>
      ```python file=<rootDir>/packages/sdk-python/examples/abot_world.py title="abot_world.py" lineNumbers
      """Python port of packages/sdk/examples/abot-world.ts.

      ABot-World: build a world from a single image, then walk it. Each step
      generates one block of video under the keys held for it and streams the
      decoded frames.

      Runs on the machine hosting the worker; there is no remote route. The default
      832x480 tier needs >= 20 GB free VRAM on a dedicated GPU (24 GB card in
      practice); pass 448 256 to use the low-VRAM tier, which runs on ~6 GB cards
      well below interactive frame rates.

      Three things differ from TypeScript:

        - The functions are `world_scene_stream` and `world_step_stream`, each one
          async iterator, where TypeScript's `worldCreateScene` and `worldStep`
          hand back several channels. The scene pack arrives as base64 `data`
          followed by `done` with `stats`; a block arrives as one `data` per frame,
          then a tick carrying `step` and `total_steps`, then `done` with `stats`.
        - `image` is the base64 of the first frame, not its bytes.
        - `request_id` is generated here. TypeScript's `worldStep` returns one to
          cancel against; on this path the field is optional and the worker invents
          one when it is missing, which leaves nothing to name in `cancel`.

      RUN: python examples/abot_world.py <first-frame-image> [prompt] [output-dir] [blocks] [width] [height]
      """

      from __future__ import annotations

      import asyncio
      import base64
      import contextlib
      import sys
      import uuid
      from pathlib import Path

      from tetherto.qvac_sdk import (
          Client,
          InferenceCancelledError,
          WorldSceneStreamRequest,
          WorldStepStreamRequest,
          cancel,
          load_model,
          unload_model,
          world_scene_stream,
          world_step_stream,
      )
      from tetherto.qvac_sdk.models import (
          ABOT_WORLD_0_5B_LF_TAEHV_VAE,
          ABOT_WORLD_0_5B_LF_WAN_VAE,
          ABOT_WORLD_0_5B_Q8_0,
          UMT5_XXL_ENC_Q8_0,
      )

      # A scripted walk: forward, forward while looking right, forward, idle. `keys`
      # also accepts a key-state object ({ W: true }) or a raw 8-bit mask, so a live
      # keyboard handler can pass its state straight through.
      TAPE: list[list[str]] = [["W"], ["W", "L"], ["W"], []]


      async def main() -> int:
          first_frame_path = sys.argv[1] if len(sys.argv) > 1 else None
          prompt = (
              sys.argv[2]
              if len(sys.argv) > 2
              else "| unknown | A realistic outdoor world scene with a navigable path."
          )
          output_dir = Path(sys.argv[3]) if len(sys.argv) > 3 else Path(".")
          blocks = int(sys.argv[4]) if len(sys.argv) > 4 else 8
          width = int(sys.argv[5]) if len(sys.argv) > 5 else 832
          height = int(sys.argv[6]) if len(sys.argv) > 6 else 480

          if not first_frame_path:
              print("✖ first frame image path is required", file=sys.stderr)
              print(
                  "Usage: python examples/abot_world.py "
                  "<firstFrameImage> [prompt] [outputDir] [blocks] [width] [height]",
                  file=sys.stderr,
              )
              return 1

          async with Client() as client:
              t = client.transport
              try:
                  print("▸ Loading ABot-World session (DiT + taehv + scene encoders)...")
                  model_id = await load_model(
                      t,
                      model_src=ABOT_WORLD_0_5B_Q8_0,
                      model_type="sdcpp-generation",
                      model_config={
                          "mode": "world",
                          # Two different VAEs, and the names are close enough to
                          # swap by accident:
                          #   taehvModelSrc -- taew2_2, the tiny STREAMING decoder.
                          #     Turns each block's latents into pixels during the
                          #     walk. Used by every world_step_stream.
                          #   vaeModelSrc   -- the full-precision Wan2.2 VAE.
                          #     ENCODES the first frame during the scene stream, and
                          #     is not touched again after that.
                          # Swapping them loads at first and fails once the walk
                          # starts.
                          "taehvModelSrc": ABOT_WORLD_0_5B_LF_TAEHV_VAE,
                          "t5XxlModelSrc": UMT5_XXL_ENC_Q8_0,
                          "vaeModelSrc": ABOT_WORLD_0_5B_LF_WAN_VAE,
                          "world": {
                              "seed": 42,
                              # ~3.7x fewer frame-passes per block. Without it, block
                              # times ramp from ~1.8 s to ~7.5 s as the recompute
                              # window fills.
                              "kvCache": True,
                              # JPEG rather than PNG: a block is ~14 MB of raw
                              # pixels, and this example writes every frame to disk.
                              "frameJpegQuality": 85,
                          },
                      },
                  )

                  print(
                      "▸ Creating the world "
                      "(umT5 prompt encode + Wan2.2 VAE first-frame encode)..."
                  )
                  creation = WorldSceneStreamRequest.model_validate(
                      {
                          "type": "worldSceneStream",
                          "modelId": model_id,
                          "prompt": prompt,
                          "image": base64.b64encode(
                              Path(first_frame_path).read_bytes()
                          ).decode("ascii"),
                          "width": width,
                          "height": height,
                          # The world is live on the session without this; ask for
                          # the bytes only because this example saves them to walk
                          # the same world again later.
                          "returnPack": True,
                      }
                  )

                  scene = b""
                  scene_stats = None
                  async for response in world_scene_stream(t, creation):
                      if response.data is not None:
                          scene = base64.b64decode(response.data)
                      if response.done:
                          scene_stats = response.stats

                  scene_path = output_dir / "world.safetensors"
                  scene_path.write_bytes(scene)
                  scene_create_ms = (
                      scene_stats.scene_create_ms
                      if scene_stats and scene_stats.scene_create_ms is not None
                      else "?"
                  )
                  print(
                      f"  scene pack written to {scene_path} "
                      f"({len(scene)} bytes, {scene_create_ms} ms)"
                  )
                  print(
                      "  reuse it later with modelConfig.sceneSrc "
                      "to walk this same world again"
                  )

                  print(f"▸ Walking {blocks} blocks...")
                  frame_number = 0
                  for block in range(blocks):
                      keys = TAPE[block % len(TAPE)]
                      # One block at a time: a second step while this one streams is
                      # rejected, so drive the next call off this one rather than
                      # firing them in parallel.
                      step = WorldStepStreamRequest.model_validate(
                          {
                              "type": "worldStepStream",
                              "modelId": model_id,
                              "keys": keys,
                          }
                      )

                      stats = None
                      async for response in world_step_stream(t, step):
                          if response.data is not None:
                              frame_path = output_dir / f"frame-{frame_number:04d}.jpg"
                              frame_path.write_bytes(base64.b64decode(response.data))
                              frame_number += 1
                          if response.done:
                              stats = response.stats

                      frames = stats.frames if stats and stats.frames is not None else "?"
                      step_ms = stats.step_ms if stats and stats.step_ms is not None else "?"
                      print(
                          f"  block {block + 1}/{blocks} [{'+'.join(keys) or 'idle'}] — "
                          f"{frames} frames in {step_ms} ms"
                      )

                  print(f"✔ Wrote {frame_number} frames to {output_dir}")

                  # Targeted cancellation. Cancel is block-granular: the engine has
                  # no mid-block abort, so the current block finishes internally,
                  # delivery stops, and the step raises rather than returning a
                  # truncated block. A cancel that lands after the block already
                  # finished legitimately succeeds instead -- both outcomes are
                  # correct, so handle each.
                  print("▸ Cancelling a block mid-flight to show the contract...")
                  request_id = str(uuid.uuid4())
                  cancelling = WorldStepStreamRequest.model_validate(
                      {
                          "type": "worldStepStream",
                          "modelId": model_id,
                          "requestId": request_id,
                          "keys": ["W"],
                      }
                  )

                  async def cancel_soon() -> None:
                      await asyncio.sleep(0.2)
                      # The block can finish before this lands, and cancelling a
                      # request that is no longer in flight fails. The walk below
                      # reports whichever outcome won.
                      with contextlib.suppress(Exception):
                          await cancel(t, request_id=request_id)

                  canceller = asyncio.create_task(cancel_soon())
                  try:
                      async for _ in world_step_stream(t, cancelling):
                          pass
                      print(
                          "  the block completed before the cancel landed "
                          "(the tolerated race)"
                      )
                  except InferenceCancelledError as error:
                      print(f"  cancelled as expected: {error}")
                  finally:
                      canceller.cancel()
                      with contextlib.suppress(asyncio.CancelledError):
                          await canceller

                  # A cancelled step is terminal for the native session, but the SDK
                  # drops it for you: another world_step_stream on this same model_id
                  # would transparently rebuild from the promoted pack, with no
                  # unload/load cycle. Unloading here only because the example is
                  # finished.
                  await unload_model(t, model_id)
              except Exception as error:
                  print(f"✖ World walk failed: {error}", file=sys.stderr)
                  return 1

          return 0


      if __name__ == "__main__":
          sys.exit(asyncio.run(main()))
      ```
    </WrapCode>
  </Tab>

  <Tab value="kotlin" label="Kotlin">
    <WrapCode>
      ```kotlin file=<rootDir>/packages/sdk-kotlin/examples/src/AbotWorld.kt title="AbotWorld.kt" lineNumbers
      // Kotlin port of packages/sdk/examples/abot-world.ts.
      //
      // ABot-World: build a world from a single image, then walk it. Each step
      // generates one block of video under the keys held for it and streams the
      // decoded frames.
      //
      // Runs on the machine hosting the worker; there is no remote route. The default
      // 832x480 tier needs >= 20 GB free VRAM on a dedicated GPU (24 GB card in
      // practice); pass 448 256 to use the low-VRAM tier, which runs on ~6 GB cards
      // well below interactive frame rates.
      //
      // Kotlin specifics:
      //
      //   - `client.media.worldScene()` and `client.media.worldStep()` return a
      //     `QvacMediaRun` whose `events` Flow carries typed responses: the scene
      //     pack arrives as base64 `data` followed by `done` with `stats`; a block
      //     arrives as one `data` per frame, then a tick carrying `step` and
      //     `totalSteps`, then `done` with `stats`.
      //   - Held keys are the generated `WorldStepStreamRequestKeysItem` enum, so an
      //     invalid key fails at compile time.
      //   - `image` is the base64 of the first frame, not its bytes.
      //   - The cancel demo runs the canceller as a child coroutine (`launch`) of
      //     `runBlocking`, and a cancelled block surfaces as the typed
      //     `QvacKnownException.ServerInferenceCancelled`.
      //
      // RUN: ./gradlew -p examples run -Pexample=AbotWorld --args="<first-frame-image> [prompt] [output-dir] [blocks] [width] [height]"

      import io.tether.qvac.sdk.QvacClient
      import io.tether.qvac.sdk.cancel
      import io.tether.qvac.sdk.generated.Models
      import io.tether.qvac.sdk.generated.QvacKnownException
      import io.tether.qvac.sdk.generated.schema.CancelRequest
      import io.tether.qvac.sdk.generated.schema.CancelRequestRequest
      import io.tether.qvac.sdk.generated.schema.WorldSceneStreamRequest
      import io.tether.qvac.sdk.generated.schema.WorldSceneStreamResponseStats
      import io.tether.qvac.sdk.generated.schema.WorldStepStreamRequest
      import io.tether.qvac.sdk.generated.schema.WorldStepStreamRequestKeysItem
      import io.tether.qvac.sdk.generated.schema.WorldStepStreamResponseStats
      import io.tether.qvac.sdk.jvm.JvmBareKitTransport
      import io.tether.qvac.sdk.media
      import io.tether.qvac.sdk.models
      import io.tether.qvac.sdk.qvacRequestId
      import java.nio.file.Files
      import java.nio.file.Path
      import java.util.Base64
      import kotlin.system.exitProcess
      import kotlinx.coroutines.delay
      import kotlinx.coroutines.launch
      import kotlinx.coroutines.runBlocking
      import kotlinx.serialization.json.buildJsonObject
      import kotlinx.serialization.json.put
      import kotlinx.serialization.json.putJsonObject

      // A scripted walk: forward, forward while looking right, forward, idle.
      private val TAPE: List<List<WorldStepStreamRequestKeysItem>> = listOf(
          listOf(WorldStepStreamRequestKeysItem.W),
          listOf(WorldStepStreamRequestKeysItem.W, WorldStepStreamRequestKeysItem.L),
          listOf(WorldStepStreamRequestKeysItem.W),
          emptyList(),
      )

      fun main(args: Array<String>): Unit = runBlocking {
          val firstFramePath = args.getOrNull(0)
          val prompt = args.getOrNull(1) ?: "| unknown | A realistic outdoor world scene with a navigable path."
          val outputDir = Path.of(args.getOrNull(2) ?: ".")
          val blocks = args.getOrNull(3)?.toInt() ?: 8
          val width = args.getOrNull(4)?.toLong() ?: 832
          val height = args.getOrNull(5)?.toLong() ?: 480

          if (firstFramePath.isNullOrEmpty()) {
              System.err.println("✖ first frame image path is required")
              System.err.println(
                  "Usage: ./gradlew -p examples run -Pexample=AbotWorld " +
                      "--args=\"<firstFrameImage> [prompt] [outputDir] [blocks] [width] [height]\"",
              )
              exitProcess(1)
          }

          val client = QvacClient(JvmBareKitTransport.connect())
          try {
              println("▸ Loading ABot-World session (DiT + taehv + scene encoders)...")
              val loaded = client.models.load(
                  source = Models.ABOT_WORLD_0_5B_Q8_0.src,
                  modelType = "sdcpp-generation",
                  modelConfig = buildJsonObject {
                      put("mode", "world")
                      // Two different VAEs, and the names are close enough to swap
                      // by accident:
                      //   taehvModelSrc -- taew2_2, the tiny STREAMING decoder.
                      //     Turns each block's latents into pixels during the walk.
                      //     Used by every worldStep.
                      //   vaeModelSrc   -- the full-precision Wan2.2 VAE. ENCODES
                      //     the first frame during the scene stream, and is not
                      //     touched again after that.
                      // Swapping them loads at first and fails once the walk starts.
                      put("taehvModelSrc", Models.ABOT_WORLD_0_5B_LF_TAEHV_VAE.src)
                      put("t5XxlModelSrc", Models.UMT5_XXL_ENC_Q8_0.src)
                      put("vaeModelSrc", Models.ABOT_WORLD_0_5B_LF_WAN_VAE.src)
                      putJsonObject("world") {
                          put("seed", 42)
                          // ~3.7x fewer frame-passes per block. Without it, block
                          // times ramp from ~1.8 s to ~7.5 s as the recompute window
                          // fills.
                          put("kvCache", true)
                          // JPEG rather than PNG: a block is ~14 MB of raw pixels,
                          // and this example writes every frame to disk.
                          put("frameJpegQuality", 85)
                      }
                  },
              )
              val modelId = loaded.modelId ?: error(loaded.error ?: "loadModel failed")

              println("▸ Creating the world (umT5 prompt encode + Wan2.2 VAE first-frame encode)...")
              val creation = client.media.worldScene(
                  WorldSceneStreamRequest(
                      modelId = modelId,
                      prompt = prompt,
                      image = Base64.getEncoder().encodeToString(Files.readAllBytes(Path.of(firstFramePath))),
                      width = width,
                      height = height,
                      // The world is live on the session without this; ask for the
                      // bytes only because this example saves them to walk the same
                      // world again later.
                      returnPack = true,
                  ),
              )

              var scene = ByteArray(0)
              var sceneStats: WorldSceneStreamResponseStats? = null
              creation.events.collect { response ->
                  response.data?.let { scene = Base64.getDecoder().decode(it) }
                  if (response.done == true) sceneStats = response.stats
              }

              val scenePath = outputDir.resolve("world.safetensors")
              Files.write(scenePath, scene)
              val sceneCreateMs = sceneStats?.sceneCreateMs ?: "?"
              println("  scene pack written to $scenePath (${scene.size} bytes, $sceneCreateMs ms)")
              println("  reuse it later with modelConfig.sceneSrc to walk this same world again")

              println("▸ Walking $blocks blocks...")
              var frameNumber = 0
              for (block in 0 until blocks) {
                  val keys = TAPE[block % TAPE.size]
                  // One block at a time: a second step while this one streams is
                  // rejected, so drive the next call off this one rather than firing
                  // them in parallel.
                  val step = client.media.worldStep(WorldStepStreamRequest(modelId = modelId, keys = keys))

                  var stats: WorldStepStreamResponseStats? = null
                  step.events.collect { response ->
                      response.data?.let {
                          val framePath = outputDir.resolve("frame-%04d.jpg".format(frameNumber))
                          Files.write(framePath, Base64.getDecoder().decode(it))
                          frameNumber += 1
                      }
                      if (response.done == true) stats = response.stats
                  }

                  val frames = stats?.frames ?: "?"
                  val stepMs = stats?.stepMs ?: "?"
                  val held = keys.joinToString("+").ifEmpty { "idle" }
                  println("  block ${block + 1}/$blocks [$held] — $frames frames in $stepMs ms")
              }

              println("✔ Wrote $frameNumber frames to $outputDir")

              // Targeted cancellation. Cancel is block-granular: the engine has no
              // mid-block abort, so the current block finishes internally, delivery
              // stops, and the step fails rather than returning a truncated block. A
              // cancel that lands after the block already finished legitimately
              // succeeds instead -- both outcomes are correct, so handle each.
              println("▸ Cancelling a block mid-flight to show the contract...")
              val requestId = qvacRequestId()
              val cancelling = client.media.worldStep(
                  WorldStepStreamRequest(
                      modelId = modelId,
                      requestId = requestId,
                      keys = listOf(WorldStepStreamRequestKeysItem.W),
                  ),
              )

              val canceller = launch {
                  delay(200)
                  // The block can finish before this lands, and cancelling a request
                  // that is no longer in flight fails. The walk below reports
                  // whichever outcome won.
                  runCatching { client.cancel(CancelRequest.Request(CancelRequestRequest(requestId = requestId))) }
              }
              try {
                  cancelling.events.collect { }
                  println("  the block completed before the cancel landed (the tolerated race)")
              } catch (error: QvacKnownException.ServerInferenceCancelled) {
                  println("  cancelled as expected: ${error.message}")
              } finally {
                  canceller.cancel()
              }

              // A cancelled step is terminal for the native session, but the SDK
              // drops it for you: another worldStep on this same modelId would
              // transparently rebuild from the promoted pack, with no unload/load
              // cycle. Unloading here only because the example is finished.
              client.models.unload(modelId)
          } finally {
              client.close()
          }
      }
      ```
    </WrapCode>
  </Tab>
</Tabs>

<Callout type="success">
  **Tip:** all examples throughout this documentation are self-contained and runnable. For instructions on how to run them, see the [JS/TS quickstart](/sdk/v0.23/js-ts-sdk#quickstart), the [Python quickstart](/sdk/v0.23/python-sdk#quickstart), or the [Kotlin quickstart](/sdk/v0.23/kotlin-sdk#quickstart).
</Callout>
