Generate music and other audio from text, lyrics, and musical controls using ACE-Step.

Overview

Music generation uses @qvac/audiogen-ggml, a GGML-backed ACE-Step 1.5 engine. Load the four model stages with modelType: "audiogen", then call audioGen() with a caption and optional lyrics or musical controls.

audioGen() returns immediately with:

requestId — a synchronous identifier for targeted cancellation.
progressStream — stage and step updates while generation runs.
audio — a promise resolving to interleaved PCM, its sample rate, channel count, and bitsPerSample.
stats — a promise resolving to timing statistics.

AudioGen is currently validated by the SDK end-to-end suite on desktop Node.js/Bare. Its four-model pipeline has a large memory and download footprint, so test your chosen model set and hardware before deploying to other runtimes.

Functions

Use the following sequence:

For complete signatures, see the SDK API reference.

Enable the plugin

When your project selects plugins explicitly, add the AudioGen plugin:

qvac.config.json

{
  "plugins": ["@qvac/sdk/audiogen-ggml/plugin"]
}

Then rebuild the SDK bundle:

qvac bundle sdk

If plugins is omitted or empty, all built-in plugins are included. See the plugin system for bundle and Bare runtime registration details.

Models

ACE-Step uses four independent GGUF files:

Stage	SDK model constant	Purpose
Text encoder	`AUDIOGEN_QWEN3_EMBEDDING_0_6B_Q8_0`	Encodes captions and lyrics
Language model	`AUDIOGEN_ACESTEP_5HZ_LM_0_6B_Q8_0`	Plans the song and musical structure
DiT	`AUDIOGEN_ACESTEP_V15_TURBO_Q4_K_M`	Fast, lower-memory waveform generation
DiT	`AUDIOGEN_ACESTEP_V15_TURBO_Q8_0`	Higher-precision Turbo generation
DiT	`AUDIOGEN_ACESTEP_V15_SFT_Q8_0`	Supervised fine-tuned generation
VAE	`AUDIOGEN_VAE_BF16`	Decodes the latent into PCM audio

The first AudioGen load downloads approximately 3.3 GB across these four files. The SDK downloads them sequentially to avoid splitting P2P bandwidth across four registry streams. On slow or unstable links, set registryStreamTimeoutMs: 600000 and registryDownloadMaxRetries: 10 in qvac.config.json, then point QVAC_CONFIG_PATH at that file before running the example.

Unlike single-file model families, AudioGen does not use a top-level modelSrc. Supply every stage in modelConfig:

import {
  AUDIOGEN_ACESTEP_5HZ_LM_0_6B_Q8_0,
  AUDIOGEN_ACESTEP_V15_TURBO_Q4_K_M,
  AUDIOGEN_QWEN3_EMBEDDING_0_6B_Q8_0,
  AUDIOGEN_VAE_BF16,
  loadModel,
} from "@qvac/sdk";

const modelId = await loadModel({
  modelType: "audiogen",
  modelConfig: {
    textEncModelSrc: AUDIOGEN_QWEN3_EMBEDDING_0_6B_Q8_0,
    lmModelSrc: AUDIOGEN_ACESTEP_5HZ_LM_0_6B_Q8_0,
    ditModelSrc: AUDIOGEN_ACESTEP_V15_TURBO_Q4_K_M,
    vaeModelSrc: AUDIOGEN_VAE_BF16,
    useGPU: true,
  },
});

Advanced load-time controls are inferenceSteps, shift, nGpuLayers, threads, and backendsDir. Turbo and SFT variants have different tuning requirements; omit inferenceSteps and shift unless you need to override the addon's defaults.

For more about model constants and sources, see SDK — Models.

Generate audio

caption is required and cannot be empty. Lyrics and musical controls are optional:

import { audioGen } from "@qvac/sdk";

const run = audioGen({
  modelId,
  caption: "Energetic cumbia with brass stabs and live percussion",
  lyrics: `[verse]
The city wakes beneath the moon

[chorus]
Dance until the morning comes`,
  vocalLanguage: "en",
  bpm: 98,
  keyscale: "A minor",
  timesignature: "4/4",
  duration: 30,
  seed: 42,
});

for await (const progress of run.progressStream) {
  console.log(`${progress.stage}: ${progress.step}/${progress.total}`);
}

const [{ pcm, sampleRate, channels, bitsPerSample }, stats] = await Promise.all([
  run.audio,
  run.stats,
]);

Omit lyrics or use "[Instrumental]" for instrumental output. Omitted musical controls are inferred from the caption.

duration is approximate. ACE-Step rounds the requested length to its latent frame grid, so the generated clip can be shorter or longer than requested. Use stats.audioDurationMs or calculate the duration from the returned PCM frame count and sampleRate; those values describe the actual output.

The SDK returns raw interleaved PCM rather than a WAV file. Use bitsPerSample when calculating sample counts or constructing a WAV header so consumers remain correct if the addon changes sample width. The complete example below includes a dependency-free WAV writer.

Cancellation

The request ID is available before generation starts, so a stop button can cancel exactly that run:

import { audioGen, cancel, InferenceCancelledError } from "@qvac/sdk";

const run = audioGen({ modelId, caption: "Ambient electronic music" });
stopButton.onclick = () => cancel({ requestId: run.requestId });

try {
  await run.audio;
} catch (error) {
  if (!(error instanceof InferenceCancelledError)) throw error;
}

AudioGen supports hard cancellation: the addon interrupts the active ACE-Step generation. The stream terminates with stopReason: "cancelled", and the audio and stats promises reject with InferenceCancelledError. You can also broad-cancel AudioGen work with cancel({ modelId, kind: "audiogen" }).

See Runtime — Cancellation for cancellation behavior shared across the SDK.

Example

This example loads the registry-hosted ACE-Step models, reports progress, generates an instrumental, and writes the PCM output as WAV.

The generated Python client exposes the contract-level audio_gen_stream() method. Unlike the TypeScript audioGen() wrapper, it yields progress, base64 PCM chunks, and the terminal stats frame directly; callers decode and concatenate the chunks themselves — see the Python tab.

generate-music.ts

import { writeFileSync } from 'node:fs'
import {
  AUDIOGEN_ACESTEP_5HZ_LM_0_6B_Q8_0,
  AUDIOGEN_ACESTEP_V15_TURBO_Q4_K_M,
  AUDIOGEN_QWEN3_EMBEDDING_0_6B_Q8_0,
  AUDIOGEN_VAE_BF16,
  audioGen,
  loadModel,
  unloadModel,
  type ModelProgressUpdate
} from '@qvac/sdk'

// Usage:
//   bun examples/audiogen/generate-music.ts "lo-fi hip hop, mellow piano" output.wav
const caption =
  process.argv[2] ?? 'Lo-fi hip hop with mellow piano, soft drums, and a warm bass line'
const outputPath = process.argv[3] ?? 'audiogen-output.wav'

let modelId: string | undefined
const lastLoggedPercentageByDownload = new Map<string, number>()
const completedDownloads = new Set<string>()

try {
  console.log('▸ Loading ACE-Step AudioGen models...')
  modelId = await loadModel({
    modelType: 'audiogen',
    modelConfig: {
      textEncModelSrc: AUDIOGEN_QWEN3_EMBEDDING_0_6B_Q8_0,
      lmModelSrc: AUDIOGEN_ACESTEP_5HZ_LM_0_6B_Q8_0,
      ditModelSrc: AUDIOGEN_ACESTEP_V15_TURBO_Q4_K_M,
      vaeModelSrc: AUDIOGEN_VAE_BF16,
      useGPU: true,
      inferenceSteps: 8
    },
    onProgress: (progress: ModelProgressUpdate) => {
      const mb = (bytes: number) => (bytes / 1e6).toFixed(1)
      const label = getDownloadLabel(progress.downloadKey)
      const line =
        `▸ Downloading ${label}: ${progress.percentage.toFixed(0)}% ` +
        `(${mb(progress.downloaded)}/${mb(progress.total)} MB)`

      if (process.stderr.isTTY) {
        process.stderr.write(`\r${line}`)
        if (progress.percentage >= 100 && !completedDownloads.has(progress.downloadKey)) {
          completedDownloads.add(progress.downloadKey)
          process.stderr.write('\n')
        }
        return
      }

      const percentageBucket = Math.floor(progress.percentage / 5) * 5
      const lastLogged = lastLoggedPercentageByDownload.get(progress.downloadKey)
      const isNewCompletion = progress.percentage >= 100 && lastLogged !== 100
      if (lastLogged === undefined || percentageBucket > lastLogged || isNewCompletion) {
        lastLoggedPercentageByDownload.set(progress.downloadKey, percentageBucket)
        process.stderr.write(`${line}\n`)
      }
    }
  })

  console.log(`▸ Model loaded: ${modelId}`)
  console.log(`▸ Generating: ${caption}`)

  const run = audioGen({
    modelId,
    caption,
    lyrics: '[Instrumental]',
    seed: 42,
    duration: 10
  })
  console.log(`▸ requestId: ${run.requestId}`)

  for await (const progress of run.progressStream) {
    console.log(`▸ ${progress.stage}: ${progress.step}/${progress.total}`)
  }

  const [audio, stats] = await Promise.all([run.audio, run.stats])
  const wav = createWav(audio.pcm, audio.sampleRate, audio.channels, audio.bitsPerSample)
  writeFileSync(outputPath, wav)

  const samplesPerChannel = audio.pcm.byteLength / (audio.bitsPerSample / 8) / audio.channels
  console.log(
    `▸ Generated ${samplesPerChannel} samples per channel at ` +
      `${audio.sampleRate} Hz (${audio.channels} channels)`
  )
  if (stats) console.log(`▸ Stats: ${JSON.stringify(stats)}`)
  console.log(`▸ Saved ${outputPath}`)

  await unloadModel({ modelId })
  modelId = undefined
  console.log('▸ Model unloaded')
  process.exit(0)
} catch (error) {
  if (modelId !== undefined) {
    try {
      await unloadModel({ modelId })
    } catch {
      // Preserve the generation error as the primary failure.
    }
  }
  console.error('✖', error)
  process.exit(1)
}

function getDownloadLabel(downloadKey: string) {
  return downloadKey.split('/').pop() ?? downloadKey
}

function createWav(pcm: Uint8Array, sampleRate: number, channels: number, bitsPerSample: number) {
  const header = new ArrayBuffer(44)
  const view = new DataView(header)
  const blockAlign = channels * (bitsPerSample / 8)

  writeAscii(view, 0, 'RIFF')
  view.setUint32(4, 36 + pcm.byteLength, true)
  writeAscii(view, 8, 'WAVE')
  writeAscii(view, 12, 'fmt ')
  view.setUint32(16, 16, true)
  view.setUint16(20, 1, true)
  view.setUint16(22, channels, true)
  view.setUint32(24, sampleRate, true)
  view.setUint32(28, sampleRate * blockAlign, true)
  view.setUint16(32, blockAlign, true)
  view.setUint16(34, bitsPerSample, true)
  writeAscii(view, 36, 'data')
  view.setUint32(40, pcm.byteLength, true)

  const wav = new Uint8Array(44 + pcm.byteLength)
  wav.set(new Uint8Array(header))
  wav.set(pcm, 44)
  return wav
}

function writeAscii(view: DataView, offset: number, value: string) {
  for (let index = 0; index < value.length; index++) {
    view.setUint8(offset + index, value.charCodeAt(index))
  }
}

Tip: see the JS/TS quickstart or the Python quickstart for setup instructions, and apply the first-run download configuration above on constrained connections.

Music generation