Skip to content

new ns-benches - #3009

Merged
kali merged 1 commit into
mainfrom
ns-benches
Oct 13, 2026
Merged

kali merged 1 commit into
mainfrom
ns-benches

Conversation

@kali

@kali kali commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

@kali
kali marked this pull request as draft October 12, 2026 16:45
@kali

kali commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator Author

@czoli1976 let's see how long it runs...

@kali

kali commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator Author

The jetson is crashed, i think it needs a reboot, it should happen soon.

@kali

kali commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator Author

(and I did it, it's not the bench)

@czoli1976

czoli1976 commented Oct 13, 2026 via email

Copy link
Copy Markdown
Contributor

@kali

kali commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator Author

the jetson is online again, but i will have some catching up to do...

@czoli1976

czoli1976 commented Oct 13, 2026 via email

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown

⚠️ Bench vs main — no speed regressions · 4 secondary regression(s)

Reference: 2026-10-13 morning nightly run (0d old) · full report → run

Speed — evaltime · prefill · decode

no inference-speed regressions

Improvements

Δ metric device main → PR
🟢 -6.2% en_tdnn_15M
evaltime · 2600ms
cortex-a55 627 ms → 588 ms
⚠️ 4 secondary regression(s)
Δ metric device main → PR
⚠️ +22.4% trunet
load · pulse1_f32
i9-9900k 125 ms → 153 ms
⚠️ +21.1% parakeet_tdt_600m_v3_f32f32_joint_pass
load+optimize · cpu
i9-11900kb_rtx-4060 38 ms → 46 ms
⚠️ +20.9% nemotron_3_5_asr_streaming_0_6b_f32f32_preprocessor_1s
load · cuda
jetson-orin-nx 43 ms → 52 ms
⚠️ +19.6% trunet
load+optimize · pulse1_f32
i9-9900k 138 ms → 165 ms

Bench numbers come from shared CI hardware and do produce false positives: a flagged
row is not on its own evidence of a real regression, and a clean report is not proof
there is none. Contributors do not need to chase what is reported here — leave the
reading of it to the maintainers, who will say if something needs action.

@kali
kali marked this pull request as ready for review October 13, 2026 09:08
@kali

kali commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator Author

ok. running these looks very affordable if i'm interpreting correctly the logs.

@kali
kali merged commit deea2bf into main Oct 13, 2026
100 of 101 checks passed
@kali

kali commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator Author

And I'll have a look at re-creating historical data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants