---
title: "Caller sessions"
canonical: https://docs.qvac.tether.io/sdk/runtime/sessions/
collection: "SDK"
package: "@qvac/sdk"
line: v0.24
current_line: true
---

# Caller sessions (/sdk/runtime/sessions)



## Overview

One SDK runtime can serve several callers, for example an agent harness and the skill runner it starts. Without sessions, every caller shares one request namespace: any caller can cancel another caller's request, and any caller's [`unloadModel()`](/sdk/reference/api#unloadmodel) frees a model that another caller is still using.

`openSession()` returns a session handle with the same model-work methods as the module-level SDK. Every call made through the handle carries the session's id, so:

* [`cancel()`](/sdk/reference/api#cancel) reaches only the session's own requests.
* [`loadModel()`](/sdk/reference/api#loadmodel) takes a lease on the model. `unloadModel()` releases only the session's lease, and the model unloads when its last lease is released.
* `session.close()` cancels the session's requests, waits for them to end, disposes the vector indexes the session opened and releases its model leases.

Calls made without a session run as the **owner**. Existing single-caller code needs no change.

## Functions

1. `openSession({ name? })` — returns a session handle at once. The open round trip runs in the background; calls on the handle wait for it.
2. `session.close()` — ends the session. It does not stop the runtime.
3. `close()` — the owner's shutdown. It stops the runtime and ends every session.

For how to use each function, see [SDK — API reference](/sdk/reference/api/).

## Requests and cancellation

The worker keys every request by its session and its `requestId`, so two sessions may use the same `requestId` without conflict.

* `session.cancel({ requestId })` with another session's `requestId` cancels nothing, and the other session's request keeps running.
* `session.cancel({ modelId })` cancels only the session's requests on that model.
* The owner's `cancel({ modelId })` cancels the requests of every session on that model.

See [Cancellation](/sdk/runtime/cancellation) for the cancel forms themselves.

## Model leases

A loaded model records which sessions hold it.

* `loadModel()` for a configuration that is already loaded adds a lease and returns the same `modelId`. Concurrent loads of one configuration share one load.
* `unloadModel()` releases the caller's lease and resolves with `{ released, unloaded }`. `unloaded` is `true` only when that release was the last lease.
* [`getLoadedModelInfo()`](/sdk/reference/api#getloadedmodelinfo) reports the model's `leaseCount`.
* A reload of a loaded model's configuration, `loadModel({ modelId, modelConfig })`, fails with `ModelLeasedError` while another session holds a lease on the model.
* The `modelId` stays the hash of the configuration the model was first loaded with. After a reload, a session that holds no lease and loads that first configuration gets `ModelReconfiguredError` instead of the reconfigured model.

## Owner-only operations

[`suspend()`](/sdk/reference/api#suspend), [`resume()`](/sdk/reference/api#resume), [`deleteCache()`](/sdk/reference/api#deletecache), [`startRpcServer()`](/sdk/reference/api#startrpcserver) and [`stopRpcServer()`](/sdk/reference/api#stoprpcserver) change the runtime for every session, so only the owner may call them. A session handle has no method for them, and the worker refuses them from a session with `OwnerOnlyError`. A session cannot open another session.

## Runtime loss

When the worker exits or the owner calls `close()`, every session ends:

* Later calls on a session fail locally with `SessionInvalidatedError`.
* Calls that name a model the session had loaded fail with `ModelHandleInvalidatedError` until the model is loaded again.
* Calls in flight when the worker exits fail with `WorkerCrashedError`.

The SDK does not reconnect a session or restart the worker.

## Example

The following script opens two sessions that share one model by lease, shows that only the session that started a completion can cancel it, and releases the leases one session at a time:

<Tabs>
  <Tab value="js" label="JavaScript" default>
    <WrapCode>
      ```js file=<rootDir>/packages/sdk/dist/examples/sessions.js title="sessions.js" lineNumbers
      /**
       * Two caller sessions on one runtime.
       *
       * Each session gets its own requests and model leases:
       *
       *  - Both sessions load the same configuration and get the same `modelId`;
       *    the second load adds a lease instead of loading the model again.
       *  - A session's `cancel({ requestId })` reaches only its own requests, so
       *    the second session's cancel leaves the first session's completion
       *    running; the first session's own cancel stops it.
       *  - `unloadModel()` releases the caller's lease. The model unloads when its
       *    last lease is released, here when the second session closes.
       *
       * Only the module-level `close()` stops the runtime; it ends every session.
       */
      import { close, getLoadedModelInfo, openSession, QWEN3_600M_INST_Q4 } from '@qvac/sdk';
      try {
          const harness = openSession({ name: 'harness' });
          const skillRunner = openSession({ name: 'skill-runner' });
          const modelId = await harness.loadModel({
              modelSrc: QWEN3_600M_INST_Q4,
              modelConfig: { ctx_size: 4096 }
          });
          const sameModelId = await skillRunner.loadModel({
              modelSrc: QWEN3_600M_INST_Q4,
              modelConfig: { ctx_size: 4096 }
          });
          const { leaseCount } = await getLoadedModelInfo({ modelId });
          console.log(`▸ same model for both sessions: ${modelId === sameModelId}, leases: ${leaseCount}`);
          const run = harness.completion({
              modelId,
              history: [{ role: 'user', content: 'Write a long essay about the history of Rome.' }],
              stream: true
          });
          // The skill runner cannot reach the harness's request.
          await skillRunner.cancel({ requestId: run.requestId });
          let deltasAfterForeignCancel = 0;
          let stopReason;
          for await (const event of run.events) {
              if (event.type === 'contentDelta') {
                  deltasAfterForeignCancel++;
                  // The harness stops its own completion after a few more tokens.
                  if (deltasAfterForeignCancel === 5)
                      await harness.cancel({ requestId: run.requestId });
              }
              else if (event.type === 'completionDone') {
                  stopReason = event.stopReason;
              }
          }
          console.log(`▸ ${deltasAfterForeignCancel} tokens arrived after skill-runner's cancel; harness's cancel ended the run: stopReason=${stopReason}`);
          const result = await harness.unloadModel({ modelId });
          console.log(`▸ harness unload: released=${result.released}, unloaded=${result.unloaded}`);
          // Closing a session releases its leases; this was the model's last one.
          await skillRunner.close();
          console.log('▸ skill-runner closed and released the last lease');
          await harness.close();
          await close();
          process.exit(0);
      }
      catch (error) {
          console.error('✖', error);
          process.exit(1);
      }
      ```
    </WrapCode>
  </Tab>

  <Tab value="ts" label="TypeScript">
    <WrapCode>
      ```ts file=<rootDir>/packages/sdk/examples/sessions.ts title="sessions.ts" lineNumbers
      /**
       * Two caller sessions on one runtime.
       *
       * Each session gets its own requests and model leases:
       *
       *  - Both sessions load the same configuration and get the same `modelId`;
       *    the second load adds a lease instead of loading the model again.
       *  - A session's `cancel({ requestId })` reaches only its own requests, so
       *    the second session's cancel leaves the first session's completion
       *    running; the first session's own cancel stops it.
       *  - `unloadModel()` releases the caller's lease. The model unloads when its
       *    last lease is released, here when the second session closes.
       *
       * Only the module-level `close()` stops the runtime; it ends every session.
       */

      import { close, getLoadedModelInfo, openSession, QWEN3_600M_INST_Q4 } from '@qvac/sdk'

      try {
        const harness = openSession({ name: 'harness' })
        const skillRunner = openSession({ name: 'skill-runner' })

        const modelId = await harness.loadModel({
          modelSrc: QWEN3_600M_INST_Q4,
          modelConfig: { ctx_size: 4096 }
        })
        const sameModelId = await skillRunner.loadModel({
          modelSrc: QWEN3_600M_INST_Q4,
          modelConfig: { ctx_size: 4096 }
        })
        const { leaseCount } = await getLoadedModelInfo({ modelId })
        console.log(`▸ same model for both sessions: ${modelId === sameModelId}, leases: ${leaseCount}`)

        const run = harness.completion({
          modelId,
          history: [{ role: 'user', content: 'Write a long essay about the history of Rome.' }],
          stream: true
        })

        // The skill runner cannot reach the harness's request.
        await skillRunner.cancel({ requestId: run.requestId })

        let deltasAfterForeignCancel = 0
        let stopReason: string | undefined
        for await (const event of run.events) {
          if (event.type === 'contentDelta') {
            deltasAfterForeignCancel++
            // The harness stops its own completion after a few more tokens.
            if (deltasAfterForeignCancel === 5) await harness.cancel({ requestId: run.requestId })
          } else if (event.type === 'completionDone') {
            stopReason = event.stopReason
          }
        }
        console.log(
          `▸ ${deltasAfterForeignCancel} tokens arrived after skill-runner's cancel; harness's cancel ended the run: stopReason=${stopReason}`
        )

        const result = await harness.unloadModel({ modelId })
        console.log(`▸ harness unload: released=${result.released}, unloaded=${result.unloaded}`)

        // Closing a session releases its leases; this was the model's last one.
        await skillRunner.close()
        console.log('▸ skill-runner closed and released the last lease')

        await harness.close()
        await close()
        process.exit(0)
      } catch (error) {
        console.error('✖', error)
        process.exit(1)
      }
      ```
    </WrapCode>
  </Tab>

  <Tab value="python" label="Python">
    <WrapCode>
      ```python file=<rootDir>/packages/sdk-python/examples/sessions.py title="sessions.py" lineNumbers
      """Python port of packages/sdk/examples/sessions.ts.

      Two caller sessions on one runtime. Each session gets its own requests and
      model leases:

        - Both sessions load the same configuration and get the same `model_id`;
          the second load adds a lease instead of loading the model again.
        - A session's `cancel(request_id=...)` reaches only its own requests, so
          the second session's cancel leaves the first session's completion
          running; the first session's own cancel stops it.
        - `unload_model()` releases the caller's lease. The model unloads when its
          last lease is released, here when the second session closes.

      Pass `session.transport` wherever a function takes a transport. Only
      `Client`'s exit stops the runtime; it ends every session.

      RUN: python examples/sessions.py
      """

      from __future__ import annotations

      import asyncio
      import sys

      from tetherto.qvac_sdk import Client, cancel, completion, load_model, unload_model
      from tetherto.qvac_sdk.methods import get_loaded_model_info
      from tetherto.qvac_sdk.models import QWEN3_600M_INST_Q4
      from tetherto.qvac_sdk.schemas import GetLoadedModelInfoRequest


      async def lease_count(transport, model_id: str) -> int | None:
          response = await get_loaded_model_info(
              transport,
              GetLoadedModelInfoRequest.model_validate(
                  {"type": "getLoadedModelInfo", "modelId": model_id}
              ),
          )
          return response.info.lease_count


      async def main() -> int:
          async with Client() as client:
              try:
                  harness = await client.open_session(name="harness")
                  skill_runner = await client.open_session(name="skill-runner")

                  model_id = await load_model(
                      harness.transport,
                      model_src=QWEN3_600M_INST_Q4,
                      model_config={"ctx_size": 4096},
                  )
                  same_model_id = await load_model(
                      skill_runner.transport,
                      model_src=QWEN3_600M_INST_Q4,
                      model_config={"ctx_size": 4096},
                  )
                  print(
                      f"▸ same model for both sessions: {model_id == same_model_id}, "
                      f"leases: {await lease_count(client.transport, model_id)}"
                  )

                  run = completion(
                      harness.transport,
                      model_id=model_id,
                      history=[
                          {
                              "role": "user",
                              "content": "Write a long essay about the history of Rome.",
                          }
                      ],
                  )

                  # The skill runner cannot reach the harness's request.
                  await cancel(skill_runner.transport, request_id=run.request_id)

                  deltas_after_foreign_cancel = 0
                  stop_reason = None
                  async for event in run.events:
                      if event.type == "contentDelta":
                          deltas_after_foreign_cancel += 1
                          # The harness stops its own completion after a few more tokens.
                          if deltas_after_foreign_cancel == 5:
                              await cancel(harness.transport, request_id=run.request_id)
                      elif event.type == "completionDone":
                          reason = getattr(event, "stop_reason", None)
                          stop_reason = getattr(reason, "value", reason)
                  print(
                      f"▸ {deltas_after_foreign_cancel} tokens arrived after skill-runner's "
                      f"cancel; harness's cancel ended the run: stopReason={stop_reason}"
                  )

                  await unload_model(harness.transport, model_id)
                  print(
                      "▸ harness released its lease; "
                      f"leases left: {await lease_count(client.transport, model_id)}"
                  )

                  # Closing a session releases its leases; this was the model's last one.
                  await skill_runner.close()
                  print("▸ skill-runner closed and released the last lease")

                  await harness.close()
              except Exception as error:
                  print(f"✖ {error}", file=sys.stderr)
                  return 1
          return 0


      if __name__ == "__main__":
          sys.exit(asyncio.run(main()))
      ```
    </WrapCode>
  </Tab>

  <Tab value="kotlin" label="Kotlin">
    <WrapCode>
      ```kotlin file=<rootDir>/packages/sdk-kotlin/examples/src/Sessions.kt title="Sessions.kt" lineNumbers
      // Kotlin port of packages/sdk/examples/sessions.ts.
      //
      // Two caller sessions on one runtime. Each session gets its own requests and
      // model leases:
      //
      //   - Both sessions load the same configuration and get the same `modelId`;
      //     the second load adds a lease instead of loading the model again.
      //   - A session's cancel by request id reaches only its own requests, so the
      //     second session's cancel finds nothing and the first session's completion
      //     keeps running; the first session's own `run.cancel()` stops it. The
      //     second session sends its cancel with `raw.call`, since only a run's own
      //     handle has a typed `cancel()`.
      //   - `models.unload()` releases the caller's lease. The model unloads when its
      //     last lease is released, here when the second session closes.
      //
      // Only the owner's `client.close()` stops the runtime; it ends every session.
      //
      // RUN: ./gradlew -p examples run -Pexample=Sessions

      import io.tether.qvac.sdk.QvacClient
      import io.tether.qvac.sdk.QvacCompletionEvent
      import io.tether.qvac.sdk.QvacMessage
      import io.tether.qvac.sdk.completion
      import io.tether.qvac.sdk.generated.Models
      import io.tether.qvac.sdk.jvm.JvmBareKitTransport
      import io.tether.qvac.sdk.models
      import io.tether.qvac.sdk.openSession
      import io.tether.qvac.sdk.raw
      import io.tether.qvac.sdk.run
      import kotlinx.coroutines.launch
      import kotlinx.coroutines.runBlocking
      import kotlinx.serialization.json.buildJsonObject
      import kotlinx.serialization.json.put

      fun main(): Unit = runBlocking {
          val client = QvacClient(JvmBareKitTransport.connect())
          try {
              val harness = client.openSession(name = "harness")
              val skillRunner = client.openSession(name = "skill-runner")

              val model = Models.QWEN3_600M_INST_Q4
              val config = buildJsonObject { put("ctx_size", 4096) }
              val loaded = harness.models.load(source = model.src, modelType = model.engine, modelName = model.name, modelConfig = config)
              val modelId = loaded.modelId ?: error(loaded.error ?: "loadModel failed")
              val again = skillRunner.models.load(source = model.src, modelType = model.engine, modelName = model.name, modelConfig = config)
              val leases = client.models.loadedInfo(modelId).info.leaseCount
              println("▸ same model for both sessions: ${again.modelId == modelId}, leases: $leases")

              val run = harness.completion.run(
                  modelId = modelId,
                  history = listOf(QvacMessage.user("Write a long essay about the history of Rome.")),
              )

              // The skill runner cannot reach the harness's request. The cancel goes
              // through `raw`, since only a run's own handle has a typed `cancel()`.
              val foreign = skillRunner.raw.call(
                  buildJsonObject {
                      put("type", "cancel")
                      put("operation", "request")
                      put("requestId", run.requestId)
                  },
              )
              println("▸ skill-runner's cancel found ${foreign["cancelled"]} request(s)")

              var deltasAfterForeignCancel = 0
              var stopReason: String? = null
              run.events.collect { event ->
                  when (event) {
                      is QvacCompletionEvent.ContentDelta -> {
                          deltasAfterForeignCancel++
                          // The harness stops its own completion after a few more tokens.
                          if (deltasAfterForeignCancel == 5) launch { run.cancel() }
                      }
                      is QvacCompletionEvent.Done -> stopReason = event.stopReason
                      else -> Unit
                  }
              }
              println("▸ $deltasAfterForeignCancel tokens arrived after skill-runner's cancel; harness's cancel ended the run: stopReason=$stopReason")

              val unloaded = harness.models.unload(modelId)
              println("▸ harness unload: released=${unloaded.released}, unloaded=${unloaded.unloaded}")

              // Closing a session releases its leases; this was the model's last one.
              skillRunner.close()
              println("▸ skill-runner closed and released the last lease")

              harness.close()
          } finally {
              client.close()
          }
      }
      ```
    </WrapCode>
  </Tab>
</Tabs>

<Callout type="success">
  **Tip:** all examples throughout this documentation are self-contained and runnable. For instructions on how to run them, see the [JS/TS quickstart](/sdk/js-ts-sdk#quickstart), the [Python quickstart](/sdk/python-sdk#quickstart), or the [Kotlin quickstart](/sdk/kotlin-sdk#quickstart).
</Callout>

## Errors

* `SessionIdConflictError` — the worker already has a session with the id the client generated.
* `SessionNotOpenError` — the worker received a call for a session it does not have, for example one that already closed.
* `SessionInvalidatedError` — a call on a session handle after the session ended: its own `close()`, the owner's `close()` or the loss of the worker.
* `ModelHandleInvalidatedError` — a call naming a model the session loaded, after the session ended. Load the model again.
* `ModelLeasedError` — a reload of a model's configuration while another session holds a lease on it.
* `ModelReconfiguredError` — a load, from a session that holds no lease, of a configuration whose model was since reloaded with another configuration.
* `OwnerOnlyError` — a session called an owner-only operation.
