Skip to content

docs(benchmarks): gate-971 section 8 points the embedding stall at the shared slot (#975) #1781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MakazhanAlpamys merged 1 commit into from
Oct 12, 2026
Merged

docs(benchmarks): gate-971 section 8 points the embedding stall at the shared slot (#975) #1781

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions benchmarks/gate-971-async-nvme-source.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -458,6 +458,12 @@ its **absolute** step times (49.2, 39.4, 39.3 s) are not throughput evidence;
the ratio is what it measures. JSON:
`layer0_wait_cold_synth70b_nf4.json`.

**Correction, 2026-10-12.** For the embedding entry, #975 supersedes this
section's opening sentence: `embed_tokens` and `lm_head` share one device slot,
so each step's `_prime()` re-copies the whole matrix host to device with nothing
to overlap it (a copy, not a read; fixed by #1178). The record's other findings,
including the 0.07–0.12% materiality ruling, stand.

| step | layer 0, forward / backward | other layers, mean / median / max | vocabulary loads | brackets |
|---|---|---|---|---|
| 0 | **17** / 271 ms | 252 / 283 / 420 ms | 662, 386 ms | 40.6 s of 49.2 s = 82.6% |
Expand Down Expand Up @@ -525,6 +531,11 @@ and not an inference about it. Step 0 is the exception in the useful direction:
the reader has the build phase to start that 662 ms first touch, so only
32.66 ms of it reaches the compute thread.

**Correction, 2026-10-12.** For the embedding entry, the cause in this finding's
first sentence and the "single-member-group behaviour" above are superseded by
the shared-slot copy in the note under this section's first paragraph (#975).
The stall figures and the ruling below stand.

**It is still not material, so Finding 8's ruling stands.** The step's entire
stall — the embedding, `lm_head`, layer 0 and all 79 other decoder layers
together, every one of the 162 entries — is 33/48/48 ms, which is
Expand Down
Loading