fish_drama3 is a community model port of
fishaudio/drama-3. Inference runs in
native audio.cpp with GGML; Python is used to prepare the checkpoint.
Built with Fish Audio.
The base model includes the 40-layer hybrid attention/MoE backbone, 256 experts with eight selected per token, four shared enrichment layers, nine residual codebook layers with persistent timeline caches, and the Fish DAC encoder/decoder. It supports offline and streaming TTS, voice references, speaker and event tags in text, long-form chunking, and continuation/edit history.
Install Visual Studio 2022 C++ build tools, a CUDA toolkit that supports your GPU, and Git. Use the Drama 3 branch and initialize its public frontend submodule:
git clone --branch drama-3-native https://github.com/JarodMica/audio.cpp.git
Set-Location audio.cpp
git -c submodule.external/audio.cpp-server-frontends.url=https://github.com/0xShug0/audio.cpp-server-frontends.git submodule update --init --recursiveThe command-local HTTPS override lets the public frontend submodule download
without GitHub SSH credentials. It does not modify .gitmodules or global Git
configuration.
Run the build helper from the repository root. It selects the required model dependencies and detects the CUDA architecture:
.\scripts\build_windows.ps1 -Preset windows-cuda-release -ModelSet custom -Models fish_drama3 -Target audiocpp_cli -Jobs 4
.\scripts\build_windows.ps1 -Preset windows-cuda-release -ModelSet custom -Models fish_drama3 -Target audiocpp_server -Jobs 4The executables are in build\windows-cuda-release\bin. A CPU runtime uses the
windows-cpu-release preset with the same model selection. To override automatic
CUDA detection, add -CudaArchitectures with the architecture for your GPU.
Install uv, then download one package from Jmica/Drama-3-GGUF. For the Q2 expert package:
uv run --no-project --python 3.12 --with huggingface_hub hf download `
Jmica/Drama-3-GGUF drama-3-backbone-8bit-experts-Q2_K.gguf `
--local-dir models\Drama-3-GGUFThe default files in Jmica/Drama-3-GGUF
use the October 29, 2026 upstream checkpoint,
revision 78e5ab7,
labeled V2 by the conversion repository. The original V1 packages, based on
upstream revision dd64a85, remain available at
revision original-2026-10-13.
V1 and V2 are GGUF release labels, not official Fish Audio model-version names.
Both releases retain the same filenames. Select the revision explicitly and use separate directories when keeping both. For example, to download V1:
uv run --no-project --python 3.12 --with huggingface_hub hf download `
Jmica/Drama-3-GGUF drama-3-backbone-8bit-experts-Q2_K.gguf `
--revision original-2026-10-13 --local-dir models\Drama-3-GGUF-V1The model manager lists the base and small packages:
uv run --no-project --python 3.12 tools\model_manager_v2.py list | Select-String 'fish_drama3'
uv run --no-project --python 3.12 tools\model_manager_v2.py install fish_drama3_q2_experts_q8_backbone --models-root models| File | Size | Expert banks | Other backbone matrices |
|---|---|---|---|
drama-3-backbone-8bit-experts-Q4_K_S.gguf |
22.91 GiB | Q4_K | Q8_0 |
drama-3-backbone-8bit-experts-Q3_K_S.gguf |
18.93 GiB | Q3_K | Q8_0 |
drama-3-backbone-8bit-experts-Q2_K.gguf |
15.88 GiB | Q2_K | Q8_0 |
drama-3-backbone-4bit-experts-Q4_K_S.gguf |
21.77 GiB | Q4_K | Q4_K, with Q4_0 text embedding |
drama-3-backbone-8bit-F32-routers-experts-Q4_K_S.gguf |
22.97 GiB | Q4_K | Q8_0, with FP32 routers/shared-expert gates |
drama-3-backbone-8bit-F32-routers-BF16-embed-head-experts-Q4_K_S.gguf |
23.87 GiB | Q4_K | Q8_0, with FP32 routers/shared-expert gates and BF16 text embedding/output head |
These packages use standard GGML K-quants. The Q3_K_S and Q4_K_S file profiles
use pure expert banks: all 120 routed expert projection banks have uniform
Q3_K or Q4_K storage. Their storage-type selection corresponds to a standard
--pure K profile with explicit precision overrides for the dense backbone,
text embedding and audio components. Architecture-specific higher-bit
promotion heuristics are not applied; an unmodified default llama.cpp preset
can choose different types for some tensors. See the standard converter's
pure and per-tensor options.
The native converter takes --expert-type q3_k or --expert-type q4_k.
backbone-8bit means Q8_0 backbone matrices; backbone-4bit means Q4_K matrices
with a Q4_0 text embedding. Q2 experts use uniform Q2_K storage.
All six packages retain FP16 audio-tower matrices and waveform codec weights. Backbone and audio-tower normalization vectors stay FP32; codec normalization vectors use FP16 with the rest of the codec. The packages embed their tokenizer, configuration and model contract. Stock audio.cpp releases without this port cannot run Drama 3.
Host sampling is used by default. Set
--session-option fish_drama3.device_sampling=true to enable tower decode and
codebook sampling in one cached backend graph, or select the cuda_perf preset.
Graph probability arithmetic is F32, so sampled outputs may differ from host sampling.
fish_drama3.perf_mode defaults to off. The opt-in cuda_perf preset sets
the following defaults; explicit session options override the preset:
| Session option | off |
cuda_perf |
Effect |
|---|---|---|---|
fish_drama3.fused_moe |
false |
true |
Cached-decode routing and SwiGLU fusion. |
fish_drama3.kv_cache_type |
f16 |
f16 |
KV storage precision, not weight precision. Set f32 to use F32 KV storage. |
fish_drama3.kv_cache_alignment |
1 |
256 |
Cached attention-prefix buckets; all history retained. |
fish_drama3.device_sampling |
false |
true |
Cached tower decode and device sampling. |
fish_drama3.codec_batch_frames |
4 |
4 |
Streaming codec batch ceiling; existing default retained. |
fish_drama3.codec_async_decode |
true |
true |
Streaming AR/codec overlap; existing default retained. |
fish_drama3.backbone_prefill_cache_mb |
256 |
256 |
Backbone prefill graph-cache budget in MiB; existing default retained. |
fish_drama3.tower_prefill_cache_mb |
256 |
256 |
Tower prefill graph-cache budget in MiB; existing default retained. |
off retains the defaults listed above. Explicit session options override either
preset, including fish_drama3.device_sampling=false with cuda_perf.
Fused arithmetic and attention shapes can change sampled audio. To enable the combined CUDA path, add:
--session-option fish_drama3.perf_mode=cuda_perf
Streaming defaults to a four-frame codec batch ceiling with AR/codec overlap. To use one-frame synchronous decoding:
--session-option fish_drama3.codec_batch_frames=1
--session-option fish_drama3.codec_async_decode=false
Batching starts with one frame, grows to the configured ceiling, and flushes the final partial batch. Larger batches retain more codec workspace and can introduce small waveform rounding differences without changing AR sampling. Overlap uses the dedicated codec backend and completes all audio before the request returns. Offline decoding is unchanged.
.\build\windows-cuda-release\bin\audiocpp_cli.exe `
--task tts --family fish_drama3 --backend cuda --mode streaming `
--model models\Drama-3-GGUF\drama-3-backbone-8bit-experts-Q2_K.gguf `
--text 'A quick test of Drama Three on Windows.' `
--max-tokens 500 --seed 123 --threads 8 `
--session-option moe_cpu_expert_layers=8 `
--session-option fish_drama3.context_length=2048 `
--out speech.wavFor the tested 16 GB offload configurations, use 8 CPU expert layers with Q2,
12 with Q3, 24 with Q4-experts, or 20 with Q4-backbone. For a 24 GB configuration,
use 4 with Q4-experts or 0 with the other packages. These are starting settings,
not enforced memory limits. They were measured on a 96 GB GPU; physical 16 GB
and 24 GB cards have not been tested. These counts control where expert weights and matmuls run.
The backbone's attention/dense weights and the audio tower use the selected
backend. fish_drama3.codec_backend=cpu moves codec computation to CPU as well.
If Vulkan voice cloning produces incomplete speech or reaches the token limit,
try --session-option fish_drama3.codec_encoder_f32_accumulation=true.
This opt-in control uses F32 accumulation in the Vulkan reference encoder while keeping F16 storage. It defaults to false and does not change codec decoding or other backends. It also works with existing GGUFs without replacing their specs.
On CUDA, BF16 reference-encoder arithmetic is enabled by default with F16/BF16
codec weights. Set --session-option fish_drama3.codec_encoder_bf16_compute=false
to restore the previous encoder arithmetic. This changes reference-conditioned
generation but does not change codec decoding or backbone inference.
To reduce reference-encoder convolution memory, set
--session-option fish_drama3.codec_encoder_chunk_samples=65536.
The value is measured in samples at the codec sample rate and must be zero
(the default, full encoding) or a positive multiple of 512. Chunking starts
when the padded reference exceeds the chunk size plus 16,384 context samples;
shorter references use the full graph. Overlapping convolution windows reuse
one cached graph; the transformer and quantizer still process the full feature
sequence, so total memory is not constant. Chunking can cost additional time
and produce small backend-dependent numerical differences. Existing GGUFs
accept the option without a spec override.
Sampling defaults to top_k=4097, which keeps every candidate for the semantic
token and each tower codebook, so only top_p=0.9 filters. This matches Fish's
official Drama 3 Space, which dropped the top-k cap because top_k=30 left takes
stuck in silence. With --request-option top_k=30, long chunked requests can
stall for about 9-21 s. The repetition-avoiding resample keeps its own
top_k=30, top_p=0.95.
Host memory is required for offloaded weights. Context capacity includes text, reference audio frames, prior takes, and the requested generation budget.
| Field | Value |
|---|---|
| Family | fish_drama3 |
| Tasks | tts, clon |
| Modes | offline, streaming |
| Languages | auto |
| Default package | fish_drama3_q4_experts_q8_backbone |
| Input | Text; voice conditioning as shown in the examples |
| Output | Generated speech audio |
“Model/session default” means no explicit override. The runtime uses the checkpoint or session setting, or leaves the optional value unset as described.
| Option | Meaning |
|---|---|
--model |
Model file or directory shown in the examples. |
--backend |
Select an available backend supported by this model. |
--threads |
CPU thread count. |
--log |
Enable runtime logging. |
--text |
Input text; see the examples for the expected prompt format. |
--voice-ref |
Reference audio for voice cloning. |
--out |
Output WAV file. |
| Option | Type / values | Default | Description |
|---|---|---|---|
encode_reference_only |
bool |
false |
Prepare reference VQ codes without generating speech. Use the task API to receive drama3_reference_v1 acoustic-token artifacts. |
reference_text |
string |
Model/session default | Transcript of inline reference audio. |
multi_reference_cond |
string |
Model/session default | Ordered references as JSON: [{text, audio_path or vq_codes}]. Speaker tags 0..15 select voices. |
references |
string |
Model/session default | Alias of multi_reference_cond. |
previous_takes |
string |
Model/session default | Continuation/edit history as a JSON array of {text, vq_codes or audio_path}; a string generates that earlier take first. |
previous_takes_file |
string |
Model/session default | JSON history file containing a take object or an ordered array of earlier takes. Avoids the Windows command-line length limit for VQ grids. |
max_tokens |
int |
1024 |
Maximum generated acoustic frames per text chunk (about 21.5 frames/second). Declared range: minimum 1. |
text_chunk_size |
int |
128 |
Maximum characters per long-form chunk. Declared range: minimum 1. |
skip_empty_chunks |
bool |
false |
Log and skip chunks that emit EOS before producing audio. The request fails if all chunks are empty. |
text_chunk_mode |
word_budget, tag_aware, japanese, endline |
tag_aware |
Framework text chunking mode. |
temperature |
float |
1.0 |
Main and residual sampling temperature. Zero makes the initial draws greedy; semantic repeat redraws retain the fixed Drama 3 RAS settings. Declared range: minimum 0. |
top_p |
float |
0.9 |
Inclusive nucleus cutoff before top-k filtering, matching Drama 3. Declared range: minimum 0, maximum 1. |
top_k |
int |
4097 |
Top-k candidate count. The default 4097 keeps every candidate (no cap), as Fish's Drama 3 Space does; 30 can stall in long silences. Declared range: minimum 1. |
seed |
int |
Model/session default | Reproducible native sampling seed. Declared range: minimum 0. |
MoE defaults: resident banks, pinning off, 32 cache slots, one read thread,
four CPU MoE layers; CPU share is zero.
Use the shared MoE controls via CLI
flags, --session-option, or the server model's session_options.
Legacy fish_drama3.disk_experts, .pin_experts, .expert_cache_size,
.disk_expert_read_threads, and .cpu_expert_layers aliases remain accepted;
supplying both a canonical key and its alias is an error, even with equal values.
CPU counts select the first N MoE ordinals, skipping dense-only backbone layers.
CPU-selected layers override moe_cpu_share and retain bounded CPU caches in
staged mode. Both moe_pin_experts=true and fish_drama3.pin_experts=true
are rejected without staging, preserving alias equivalence.
| Option | Type / values | Default | Description |
|---|---|---|---|
fish_drama3.perf_mode |
off, cuda_perf |
off |
AR decode preset. cuda_perf enables fused MoE, 256-row attention buckets and device sampling. F16 KV storage, codec batching/overlap and prefill-cache defaults remain unchanged in both modes. Explicit session options override the preset. |
fish_drama3.fused_moe |
bool |
false |
Fuse cached backbone MoE routing and SwiGLU activations. Fused arithmetic may change sampled output; prompt prefill is unchanged. |
fish_drama3.kv_cache_alignment |
int |
1 |
Cached decode attention prefix bucket size. Values above 1 retain all history while skipping unused KV capacity; execution shapes may change numerical results. Declared range: minimum 1. |
fish_drama3.kv_cache_type |
f32, f16 |
f16 |
Storage precision for full-attention backbone KV caches. |
fish_drama3.backbone_prefill_cache_mb |
int |
256 |
Bounded cache of backbone prefill graphs, in MiB. Reuses exact shapes and their CUDA captures. One active graph is retained even if larger than this budget. Zero retains only the active graph. Declared range: minimum 0, maximum 4096. |
fish_drama3.tower_prefill_cache_mb |
int |
256 |
Bounded cache of audio-tower prefill graphs, in MiB. Reuses exact shapes and their CUDA captures. Zero disables it. Declared range: minimum 0, maximum 4096. |
fish_drama3.codec_batch_frames |
int |
4 |
Streaming codec batch ceiling. Starts at one frame and grows to this size; partial final batches are flushed. Offline decoding is unchanged. Declared range: minimum 1. |
fish_drama3.codec_async_decode |
bool |
true |
Overlap streaming codec decoding with AR generation on the dedicated codec backend. Uses a bounded queue and waits for all audio before completing the request. |
fish_drama3.device_sampling |
bool |
false |
Enable tower decode and codebook sampling in one cached backend graph. Disabled by default; enabled by cuda_perf unless explicitly overridden. Uses F32 probabilities; sampled outputs may differ from the host sampler. |
fish_drama3.profile_first_audio |
bool |
false |
Emit first-audio stage timings to stderr for performance diagnosis. Disabled by default. |
fish_drama3.context_length |
int |
2048 |
Cache capacity for prompt plus generated frames; includes references and history. Declared range: minimum 1. |
moe_cpu_expert_layers |
int |
4 |
Number of the 40 expert banks kept and evaluated on CPU; use 4 for a 24 GB profile or 24 for a 16 GB profile. Dense layers and the tower use the selected backend. Declared range: minimum 0, maximum 40. |
moe_cpu_share |
float |
0 |
CPU share of eligible staged GPU misses, 0–1; requires moe_disk_experts=true. CPU-selected layers take precedence. |
moe_disk_experts |
bool |
false |
Read routed expert weights from GGUF on demand into a bounded cache instead of keeping full expert banks resident. Trades speed for memory; default false. |
moe_pin_experts |
bool |
false |
Keep full expert banks in pinned host RAM for direct cache-miss uploads. Requires staged mode and pinned host-buffer support. Both canonical and legacy spellings reject true without staging. |
moe_expert_cache_size |
int |
32 |
Resident expert slots per layer in disk_experts mode; 8 to 256. Larger caches trade memory for fewer disk reads; ignored in resident mode. Declared range: minimum 8, maximum 256. |
moe_disk_expert_read_threads |
int |
1 |
Host workers for copying selected GGUF experts into staging when disk_experts is enabled and pin_experts is false. Default 1 preserves serial reads; values above 1 use OpenMP or Apple dispatch. Declared range: minimum 1, maximum 64. |
fish_drama3.codec_encoder_f32_accumulation |
bool |
false |
Use F32 accumulation for F16 reference-encoder GEMMs on Vulkan. Try this if voice cloning produces incomplete speech or reaches the token limit. Other backends and codec decoding are unchanged. |
fish_drama3.codec_encoder_bf16_compute |
bool |
true |
Use BF16 reference-encoder arithmetic on CUDA with F16/BF16 codec weights; default true. Decoding is unchanged. |
fish_drama3.codec_encoder_chunk_samples |
int |
0 |
Reference-encoder convolution chunk size in samples at the codec sample rate. Use 0 for full encoding or a positive multiple of 512; default 0. Declared range: minimum 0. |
fish_drama3.codec_backend |
same, cpu |
same |
Run the codec on the selected backend or on CPU. |
Both variants use --family fish_drama3 and the same CLI and server routes.
The loader reads layer-specific expert counts and the output-head layout from
the checkpoint, so Drama3-small does not need a separate family.
| Variant | GGUF packages | Source |
|---|---|---|
| Drama3 base | Q2_K, Q3_K, and Q4_K expert packages | Jmica/Drama-3-GGUF |
| Drama3-small | fish_drama3_small_iq3_xxs: Drama3-small-GGUF/drama3-small-IQ3_XXS.gguf |
pentacoxian-dev/drama3-small-GGUF |
Install the small variant with:
python tools/model_manager_v2.py install fish_drama3_small_iq3_xxs --models-root modelsUse --model models/Drama3-small-GGUF/drama3-small-IQ3_XXS.gguf with the
generation examples below. Base-model weight sizes and memory measurements
in this guide do not describe the small checkpoint.
Conversion is optional when using a downloaded package. Build the native converter if you want to prepare a different precision:
.\scripts\build_windows.ps1 -Preset windows-cpu-release -ModelSet custom -Models fish_drama3 -Target audiocpp_gguf -Jobs 4Download the complete official checkpoint into models\drama-3-original.
The example below pins the original V1 checkpoint used for the earlier port
checks. To prepare V2, select revision
78e5ab79e49aa9a25abe8799eff2c4d73a1c3216 and use separate checkpoint and output
paths so the original files remain available:
uv run --no-project --python 3.12 --with huggingface_hub hf download fishaudio/drama-3 --revision dd64a85cd82e9521f7202610ede97b367fc83946 --local-dir models\drama-3-originalIts original per-expert FP8 tensors have 128-by-128 inverse scales. The preparation script applies those scales and packs each projection into an expert bank.
uv run --python 3.12 tools\community_models\convert_fish_drama3.py `
--checkpoint models\drama-3-original `
--output models\drama3-f16-local.gguf `
--quantized-output models\drama-3-backbone-8bit-experts-Q4_K_S.gguf `
--gguf-tool build\windows-cpu-release\bin\audiocpp_gguf.exeThe intermediate tensor archive is approximately 68.2 GiB. The runnable package is approximately 22.9 GiB, with Q4_K routed experts, Q8_0 backbone matrices, FP16 audio tower and codec weights, and embedded configuration, tokenizer and license files. Codec weight normalization is folded during packaging. Allow space for both files during conversion. The intermediate archive is retained.
An existing intermediate can be reused with --input-gguf in place of --output.
--expert-type selects the routed expert format; --backbone-type selects other
backbone matrices; --embedding-type selects the text embedding format. Keep the
embedding at a CUDA-supported lookup type such as Q8_0 or Q4_0. Conversion uses
CPU PyTorch dependencies managed by uv and does not load weights onto a GPU.
Set moe_disk_experts=true to keep the full routed expert banks in the
GGUF file and read only selected experts into a bounded native-format cache.
moe_expert_cache_size sets resident slots per layer (8 to 256; default
32). Dense backbone, tower, and codec weights remain resident. The existing
moe_cpu_expert_layers setting still selects CPU execution for those layers, but
their expert caches are bounded too. No GGUF conversion is required.
moe_disk_expert_read_threads selects 1 to 64 host workers for copying
selected experts from the memory-mapped GGUF into staging. The default is 1
(serial); try 4 to enable parallel copies using OpenMP or Apple dispatch. This applies only
with moe_disk_experts=true and moe_pin_experts=false. Backend uploads and inference
are unchanged. The host reader is backend-independent; Apple builds do not
require OpenMP for parallel reads.
This is a memory/speed trade-off, not a faster MoE kernel. With
moe_cpu_expert_layers=0, prompt prefill groups tokens by expert within each layer,
so one upload serves multiple tokens. Attention and shared/router projections
retain the resident batched-prefill path; cached experts retain their vector
projection path. Session-owned graphs share one activation workspace and are
reused for the same exact prompt length while retained. A different length
replaces that graph set rather than reusing a rounded capacity for stateful
attention. Offline disk mode releases backbone and tower prefill graphs before
waveform decoding, without clearing continuation state. Later full prefills
rebuild those workspaces; resident execution keeps its existing cache policy.
Mixed CPU/backend expert execution retains token-wise prefill. Storage speed
still matters, and resident mode remains the default. For the minimum cache:
--session-option moe_disk_experts=true
--session-option moe_expert_cache_size=8
--session-option moe_cpu_expert_layers=0
When sufficient host RAM is available, add
moe_pin_experts=true. This loads full expert banks into pinned host memory
once, then stages cache misses directly without an intermediate host copy.
Expert matmuls retain their configured placement: CPU-selected layers use
bounded CPU caches; the remaining layers use the selected backend. Set
moe_cpu_expert_layers=0 to run all expert layers on that backend. Cache slot
counts are unchanged. This option is off by
default and requires pinned host-buffer support; allocation failure is an error,
not a fallback to a different memory policy.
Logs report disk_experts.bytes_read, disk_experts.expert_loads,
disk_experts.cache_hits, disk_experts.buffer_mb, and
disk_experts.host_buffer_mb when the session is destroyed. These describe
expert I/O and cache allocation, not total process peak VRAM. With pinned
experts, bytes read include the initial host-bank load instead of per-request
cache-miss reads.
Save the following as drama3-server.json in the repository root:
{
"host": "127.0.0.1",
"port": 8035,
"backend": "cuda",
"threads": 8,
"lazy_load": true,
"max_loaded_models": 1,
"idle_unload_ms": 300000,
"ui": true,
"models": [{
"id": "drama-3",
"family": "fish_drama3",
"path": "models/Drama-3-GGUF/drama-3-backbone-8bit-experts-Q2_K.gguf",
"task": "tts",
"mode": "streaming",
"session_options": {
"moe_cpu_expert_layers": 8,
"fish_drama3.context_length": 2048
},
"default_request_options": {"max_tokens": 500}
}]
}.\build\windows-cuda-release\bin\audiocpp_server.exe --config drama3-server.jsonThe server exposes POST /v1/audio/speech and the task APIs, including streaming
events and continuation artifacts. See the server guide.
For cloning, add --voice-ref reference.wav and
--request-option 'reference_text=What the reference says.'. Use --task clon
to require a reference. Ordered references can also be supplied through
multi_reference_cond or references as a JSON string:
[{"text":"<|speaker:0|> First reference.","audio_path":"first.wav"},
{"text":"<|speaker:1|> Second reference.","audio_path":"second.wav"}]Descriptors accept vq_codes instead of audio_path: a nonempty integer grid
with ten rows, one per codebook. Text accepts <|speaker:0|> through
<|speaker:15|> and event tags such as [sighs] and [laughs].
previous_takes is a JSON string containing ordered earlier takes. Each object
has text and either vq_codes or audio_path. Earlier text strings can also
be given; the runtime generates their codes before the requested take.
previous_takes_file accepts the same data from a JSON file, including a single
take object. Use it for VQ grids that exceed the Windows command-line limit.
--request-option 'previous_takes=["The report is due on Wednesday."]'
--text 'The report is due on Thursday.'The library returns an AcousticTokens artifact named drama3_take, with
family=fish_drama3 and schema=drama3_take_v1. Its UTF-8 JSON payload contains
the text and VQ grid. Keep this artifact and pass it in the next request's
input_artifacts to reuse the original codes for an edit. Multiple artifacts
carry chained history. The current take's audio is returned separately.
Streaming sessions use IStreamingVoiceTaskSession. start_stream() runs
synchronously and calls the event sink while audio is generated; finalize()
returns the complete audio and the reusable take artifact. An application can
invoke generation on its worker thread to keep its UI responsive.
Long-form text uses the framework chunker, with text_chunk_size=128 and
text_chunk_mode=tag_aware by default. Each new chunk includes the first generated
chunk as an anchor and the latest chunk as recent context, without duplicating
the first chunk. If that prompt does not fit, automatic history is reduced to the first chunk only, then removed.
References and caller-supplied history remain ordered and are never dropped.
skip_empty_chunks=false preserves the error when a chunk emits EOS before any
audio frame. Set --request-option skip_empty_chunks=true to skip such chunks
and continue in offline or streaming mode. Skipped chunks do not enter generated
voice history, matching upstream. If all chunks are empty, the request fails with
Drama 3 produced no audio: all text chunks were empty. Other errors are not
suppressed.
max_tokens is an upper bound: generation is capped to the rows remaining after
the selected prompt within fish_drama3.context_length. Reductions are logged,
and reached_limit reports reaching this effective budget. A required prompt
that leaves no generation room is still rejected. These rules apply to both
offline and streaming generation.
If speech repeats words or phrases near chunk boundaries, increasing
text_chunk_size may help by keeping more of a sentence in one generation pass.
Increase it gradually and check the audio and peak VRAM usage. Larger chunks
can increase peak memory substantially, especially during offline decoding,
which processes each chunk's generated audio codes together.
Each chunk is limited by max_tokens and the context space remaining after the
text, references and history. The default max_tokens=1024 permits up to about
48 seconds at roughly 21.5 acoustic frames per second; context limits can shorten
this allowance. Increasing context can restore a shortened allowance up to
max_tokens. Increasing max_tokens requires enough remaining context to use it.
Chunk size and max_tokens are request options set with --request-option.
Context capacity is set with --session-option fish_drama3.context_length=N
when creating the session. Larger context capacity also increases memory use.
Check custom_schema_output.data.reached_limit in task JSON or CLI output, or
look for the runtime warning generation budget reached; audio may be incomplete.
The flag indicates that at least one generated output chunk reached its effective
budget. A max_tokens reduced log message identifies a context-limited budget.
If passages are cut off, reduce chunk size or increase the limiting budget
within available memory. Compare several seeds when checking repetition.
A community evaluation replayed the same 5,000 requests across English, Japanese, Chinese, French and Spanish, with 1,000 requests per language. Seeds, reference voices, runtime and generation settings stayed unchanged between checkpoints.
The comparison used Q4_K experts, FP32 routers and BF16 text embedding/full
output head, with all experts on GPU. The native runtime was pinned to
4faef969850e2dd485e717fe2159875b91cfa3dc. Settings were temperature 1.0,
top_p=0.9, top_k=4097, text_chunk_size=128, context 8192 and a maximum of
1024 acoustic frames per text chunk. These results describe this precision
profile, not every quantization or Drama3-small.
| Automatic check | V1 | V2 |
|---|---|---|
| Trailing quiet >=2 seconds | 25 | 0 |
| Internal quiet >=2 seconds | 35 | 0 |
| Clips reaching a chunk's frame cap before EOS | 9 | 14 |
The updated checkpoint improved silent-continuation behavior in this test. However, repetitive speech and prolonged nonverbal reactions remain possible, sometimes continuing until the generation limit. An ASR-assisted pass over the V2 outputs flagged 29 potential speech-repetition cases. All five selected cases checked by listening contained repetition. Review generated audio for looping speech even when no long quiet sections are detected.
These results describe a fixed benchmark with 5,000 text strings representing 1,000 translated meaning families, not a general failure rate or a guarantee that trailing silence is eliminated. The full V1 corpus was not evaluated with the same repetition screen, so the repetition results do not establish a V1/V2 regression. ASR discrepancies are review flags, not a full assessment of spoken content, voice similarity or acting quality.
Reproducible codec, tokenizer, quantization and HTTP smoke commands are in the Drama 3 test guide.
The port was exercised on native Windows with an RTX PRO 6000 Blackwell 96 GB, CUDA 12.8 and eight CPU worker threads. English speech, voice references, streaming audio and continuation/edit artifacts were checked. Streaming emits 44.1 kHz audio, with streamed chunks matching the complete output in the tests.
Independent checks cover hybrid backbone prefill and cached decode, actual Q2_K, Q3_K and Q4_K expert bank types and layouts against dequantized PyTorch weights, CPU/GPU expert splits, audio tower prefill/cached decode, and exact prompt rows against the official builder. Repeated seeded generation and offline/streaming generation produced identical codes within each profile. English transcript checks recovered the requested speech, including Wednesday-to-Thursday and noon-to-five edits.
This is an experimental port. Some emotion and nonverbal prompts produce prolonged reactions, omit dialogue or reach the generation cap. This occurs with both the original Q4-expert package and smaller formats. A heavier Q2 CPU offload trial also reached the long-form cap; the documented eight-layer configuration completed that case. These checks do not establish full perceptual parity with the original checkpoint. Multilingual pronunciation, speaker similarity and event delivery have not received a broad perceptual evaluation. A native seed reproduces a take within the same execution configuration; changing offload placement can change floating-point results and the sampled take.
The weights are subject to the Fish Audio Community License. See the model repository's LICENSE and NOTICE.