Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,7 @@ GPUs report compute capability 12.0 (`sm_120`) and should use the CPU package or
build from source with `-DCMAKE_CUDA_ARCHITECTURES=120` until a dedicated CUDA
asset is published.

**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.0](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.0) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.0/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.0/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.0/funasr-llamacpp-windows-x64-cuda.zip) · **Download & quickstart:** [funasr.com/deploy/llama-cpp](https://www.funasr.com/en/deploy/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/)
**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.1](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.1) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.1/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.1/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.1/funasr-llamacpp-windows-x64-cuda.zip) · **Download & quickstart:** [funasr.com/deploy/llama-cpp](https://www.funasr.com/en/deploy/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/)

[OpenAI API example →](./examples/openai_api/) · [Gradio demo →](./examples/openai_api/GRADIO.md) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript recipes →](./examples/openai_api/JAVASCRIPT.md) · [Kubernetes template →](./examples/openai_api/kubernetes/) · [Workflow recipes →](./examples/openai_api/WORKFLOWS.md) · [Postman collection →](./examples/openai_api/POSTMAN.md) · [OpenAPI spec →](./examples/openai_api/OPENAPI.md) · [Security guide →](./examples/openai_api/SECURITY.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)

Expand All @@ -330,6 +330,7 @@ asset is published.

## What's new

- 2026/08/27: **llama.cpp runtime v0.2.1** — Vulkan device selection now accepts matching integrated GPUs, prefers a matching discrete GPU when both are available, and otherwise falls back to the iGPU. Nine Linux, macOS, and Windows archives were rebuilt and verified with published SHA-256 values. Radeon 780M still needs reporter hardware confirmation; the separate RX 9070 XT `0xC0000005` initialization crash is not claimed fixed. [Download matrix & quickstart ->](https://www.funasr.com/en/deploy/llama-cpp.html) · [Release ->](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.1)
- 2026/08/26: **v1.4.4 on PyPI** — realtime WebSocket decoding now batches compatible sessions instead of serializing every connection behind one process-wide lock. On the H100 regression workload, 12-client STOP p95 fell from 19.8 s to 0.4 s and 16-client aggregate throughput rose from 8.6x to 13.2x, with no client errors. This hotfix also restores compatibility with PyTorch releases that do not expose `torch.amp` and raises real runtime-binding exceptions. Upgrade with `python -m pip install -U "funasr==1.4.4"`. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.4.4)
- 2026/08/21: **v1.4.3 on PyPI** — `AutoModel(vad_model="silero-vad")` now offers an optional Silero VAD adapter with millisecond segments, configurable thresholds, 8/16 kHz input, ONNX mode, and bounded segment lengths. Upgrade with `python -m pip install -U "funasr==1.4.3"`; enable the adapter with `python -m pip install -U "funasr[silero]==1.4.3"`. Speaker diarization with a known speaker count now routes large embedding sets through fixed-K clustering instead of memory-heavy dense spectral clustering. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.4.3)
- 2026/08/14: **v1.4.2 on PyPI** — punctuation-aware sentence alignment now preserves subtitle segmentation when a punctuation token boundary splits a timestamped ASR word. Distributed training now synchronizes DDP/FSDP gradients on the final microbatch of each accumulation window and initializes DeepSpeed/FSDP mode from the resolved configuration. The tagged source also includes llama.cpp SRT output and the v0.2.0 AMD Vulkan submission updates. Install with `python -m pip install -U "funasr==1.4.2"`. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.4.2)
Expand Down
1 change: 1 addition & 0 deletions README_ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ Whisper は単一モデルですが、**FunASR はツールキット**です—

## 最新情報

- 2026/08/27:**llama.cpp runtime v0.2.1** — Vulkan のデバイス選択が一致する統合 GPU を受け入れ、一致する discrete GPU があればそちらを優先し、なければ iGPU にフォールバックするようになりました。SHA-256 を公開した Linux、macOS、Windows 向け 9 archive を再構築・検証済みです。Radeon 780M は報告者の実機確認待ちで、別件の RX 9070 XT `0xC0000005` 初期化 crash は修正済みとはしていません。[ダウンロード一覧と quickstart →](https://www.funasr.com/en/deploy/llama-cpp.html) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.1)
- 2026/08/26:**v1.4.4 を PyPI に公開** — realtime WebSocket decode は、全 connection を 1 つの process-wide lock で直列化せず、互換性のある session を batch 処理するようになりました。H100 regression workload では、12 client の STOP p95 が 19.8 秒から 0.4 秒に短縮され、16 client の aggregate throughput は 8.6x から 13.2x に向上し、client error は 0 件でした。この hotfix は `torch.amp` を持たない PyTorch release との互換性も復元し、runtime binding から実際の例外を送出します。更新:`python -m pip install -U "funasr==1.4.4"`。[Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.4)
- 2026/08/21:**v1.4.3 を PyPI に公開** — `AutoModel(vad_model="silero-vad")` で任意の Silero VAD アダプターを利用でき、ミリ秒単位の区間、しきい値、8/16 kHz 入力、ONNX mode、最大区間長を設定できます。更新:`python -m pip install -U "funasr==1.4.3"`。アダプターの有効化:`python -m pip install -U "funasr[silero]==1.4.3"`。話者数が既知の話者ダイアライゼーションは、大規模 embedding をメモリ負荷の高い密なスペクトラルクラスタリングではなく fixed-K クラスタリングで処理します。[Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.3)
- 2026/08/14:**v1.4.2 を PyPI に公開** — 句読点モデルの token 境界がタイムスタンプ付き ASR 単語の内部にある場合でも、文アラインメントが字幕分割を正しく保持するようになりました。分散学習では、各勾配累積 window の最後の microbatch で DDP/FSDP 勾配を同期し、解決済み設定から DeepSpeed/FSDP mode を初期化します。対応する GitHub ソース tag には llama.cpp SRT 出力と v0.2.0 の AMD Vulkan submission 更新も含まれます。インストール:`python -m pip install -U "funasr==1.4.2"`。[Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.2)
Expand Down
1 change: 1 addition & 0 deletions README_ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ Whisper는 단일 모델이지만, **FunASR는 툴킷**입니다. 용도에 맞

## 최신 소식

- 2026/08/27: **llama.cpp runtime v0.2.1** — Vulkan device 선택이 일치하는 integrated GPU를 허용하며, 일치하는 discrete GPU가 함께 있으면 이를 우선하고 없으면 iGPU로 fallback합니다. SHA-256을 공개한 Linux, macOS, Windows용 archive 9개를 다시 빌드하고 검증했습니다. Radeon 780M은 제보자의 실제 하드웨어 확인이 필요하며, 별도 RX 9070 XT `0xC0000005` 초기화 crash는 수정됐다고 주장하지 않습니다. [다운로드 목록 및 quickstart →](https://www.funasr.com/en/deploy/llama-cpp.html) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.1)
- 2026/08/26: **v1.4.4 PyPI 공개** — realtime WebSocket decode가 모든 connection을 하나의 process-wide lock으로 직렬화하지 않고 호환되는 session을 batch 처리합니다. H100 regression workload에서 12-client STOP p95는 19.8초에서 0.4초로 줄었고, 16-client aggregate throughput은 8.6x에서 13.2x로 향상됐으며 client error는 0건이었습니다. 이 hotfix는 `torch.amp`가 없는 PyTorch release와의 호환성도 복원하고 runtime binding에서 실제 예외를 발생시킵니다. 업데이트: `python -m pip install -U "funasr==1.4.4"`. [Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.4)
- 2026/08/21: **v1.4.3 PyPI 공개** — `AutoModel(vad_model="silero-vad")`에서 선택형 Silero VAD adapter를 사용할 수 있으며, 밀리초 단위 구간, threshold, 8/16 kHz 입력, ONNX mode, 최대 구간 길이를 지원합니다. 업데이트: `python -m pip install -U "funasr==1.4.3"`. Adapter 활성화: `python -m pip install -U "funasr[silero]==1.4.3"`. 화자 수를 알고 있는 speaker diarization은 대규모 embedding을 메모리 사용량이 큰 dense spectral clustering 대신 fixed-K clustering으로 처리합니다. [Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.3)
- 2026/08/14: **v1.4.2 PyPI 공개** — 문장부호 모델의 token 경계가 타임스탬프가 있는 ASR 단어 내부에 놓여도 문장 정렬이 올바른 자막 분할을 유지합니다. 분산 학습은 각 gradient accumulation window의 마지막 microbatch에서 DDP/FSDP gradient를 동기화하고, 해석된 설정에서 DeepSpeed/FSDP mode를 초기화합니다. 해당 GitHub 소스 tag에는 llama.cpp SRT 출력과 v0.2.0 AMD Vulkan submission 업데이트도 포함됩니다. 설치: `python -m pip install -U "funasr==1.4.2"`. [Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.2)
Expand Down
3 changes: 2 additions & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ Whisper 是单个模型,**FunASR 是一个工具箱**——按场景挑模型

## 最新动态

- 2026/08/27:**llama.cpp runtime v0.2.1** — Vulkan 设备选择现在接受匹配的集成 GPU;同时存在匹配独显时优先独显,否则回退到核显。九个 Linux、macOS 与 Windows 压缩包已重新构建并通过公开 SHA-256 复核。Radeon 780M 仍需报告者实机确认;单独的 RX 9070 XT `0xC0000005` 初始化崩溃不在本次修复声明内。[下载矩阵与快速开始 →](https://www.funasr.com/deploy/llama-cpp.html) · [发布页 →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.1)
- 2026/08/26:**v1.4.4 已发布到 PyPI** — 实时 WebSocket 解码会对兼容会话进行批处理,不再让所有连接排队经过同一个进程级锁。在 H100 回归负载下,12 路 STOP p95 从 19.8 秒降至 0.4 秒,16 路聚合吞吐从 8.6x 提升到 13.2x,且客户端零错误。本热修复同时兼容不提供 `torch.amp` 的 PyTorch 版本,并让运行时绑定抛出真实异常。升级命令:`python -m pip install -U "funasr==1.4.4"`。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/v1.4.4)
- 2026/08/21:**v1.4.3 已发布到 PyPI** — `AutoModel(vad_model="silero-vad")` 现可选用 Silero VAD 适配器,返回毫秒级片段,并支持阈值、8/16 kHz 输入、ONNX 模式和最长片段限制。基础升级命令:`python -m pip install -U "funasr==1.4.3"`;启用该适配器:`python -m pip install -U "funasr[silero]==1.4.3"`。已知说话人数的说话人分离在大规模 embedding 输入下改用固定 K 聚类,避免内存开销较高的稠密谱聚类。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/v1.4.3)
- 2026/08/14:**v1.4.2 已发布到 PyPI** — 标点模型的 token 边界落在带时间戳的 ASR 词内部时,句子对齐现在仍能保留正确的字幕分段。分布式训练会在每个梯度累积窗口的最后一个 microbatch 同步 DDP/FSDP 梯度,并从解析后的配置正确初始化 DeepSpeed/FSDP 模式。对应 GitHub 源码 tag 同时包含 llama.cpp SRT 输出和 v0.2.0 AMD Vulkan submission 更新。安装命令:`python -m pip install -U "funasr==1.4.2"`。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/v1.4.2)
Expand Down Expand Up @@ -359,7 +360,7 @@ Windows Vulkan ZIP 使用显卡驱动提供的系统 Vulkan loader,不需要
compute capability 12.0(`sm_120`),在专用 CUDA 产物发布前,请使用 CPU 包,或从
源码构建并设置 `-DCMAKE_CUDA_ARCHITECTURES=120`。

**预编译二进制:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.0](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.0) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.0/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.0/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.0/funasr-llamacpp-windows-x64-cuda.zip) · **下载与快速开始:** [funasr.com/deploy/llama-cpp](https://www.funasr.com/deploy/llama-cpp.html) · **GGUF 模型:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测:** [runtime/llama.cpp/](./runtime/llama.cpp/)
**预编译二进制:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.1](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.1) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.1/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.1/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.1/funasr-llamacpp-windows-x64-cuda.zip) · **下载与快速开始:** [funasr.com/deploy/llama-cpp](https://www.funasr.com/deploy/llama-cpp.html) · **GGUF 模型:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测:** [runtime/llama.cpp/](./runtime/llama.cpp/)

[OpenAI API 示例 →](./examples/openai_api/README_zh.md) · [Gradio Demo →](./examples/openai_api/GRADIO_zh.md) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript 配方 →](./examples/openai_api/JAVASCRIPT_zh.md) · [Kubernetes 模板 →](./examples/openai_api/kubernetes/README_zh.md) · [工作流配方 →](./examples/openai_api/WORKFLOWS_zh.md) · [Postman 集合 →](./examples/openai_api/POSTMAN_zh.md) · [OpenAPI 规范 →](./examples/openai_api/OPENAPI_zh.md) · [安全指南 →](./examples/openai_api/SECURITY_zh.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)

Expand Down
4 changes: 2 additions & 2 deletions runtime/llama.cpp/tests/test_download_funasr_model.sh
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@ assert_readme_quickstart() {
printf 'non-portable Releases link in %s\n' "$readme" >&2
exit 1
fi
if ! grep -F 'runtime-llamacpp-v0.2.0' <<<"$section" >/dev/null; then
printf 'missing current runtime v0.2.0 link in %s\n' "$readme" >&2
if ! grep -F 'runtime-llamacpp-v0.2.1' <<<"$section" >/dev/null; then
printf 'missing current runtime v0.2.1 link in %s\n' "$readme" >&2
exit 1
fi
if ! grep -F 'funasr-llamacpp-linux-x64-vulkan' <<<"$section" >/dev/null; then
Expand Down
4 changes: 2 additions & 2 deletions tests/test_docs_funasr_install_commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ def test_top_level_readmes_surface_current_release_and_edge_runtime():
for name, text in readmes.items():
assert 'python -m pip install -U "funasr==1.4.1"' in text, name
assert "https://github.com/modelscope/FunASR/releases/tag/v1.4.1" in text, name
assert "runtime-llamacpp-v0.2.0" in text, name
assert "runtime-llamacpp-v0.2.1" in text, name

assert "https://www.funasr.com/en/deploy/llama-cpp.html" in readmes["README.md"]
assert "https://www.funasr.com/deploy/llama-cpp.html" in readmes["README_zh.md"]
Expand All @@ -358,7 +358,7 @@ def test_top_level_readmes_surface_current_release_and_edge_runtime():
"funasr-llamacpp-windows-x64-cuda.zip",
):
assert (
f"releases/download/runtime-llamacpp-v0.2.0/{asset}" in text
f"releases/download/runtime-llamacpp-v0.2.1/{asset}" in text
), name
assert "releases/download/runtime-llamacpp-v0.1.9/" not in text, name

Expand Down
Loading
Loading