Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/models/muse-glimmer/runtime/runners/solo.cpp
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
/*
* Copyright (c) Meta Platforms, Inc. and affiliates.
* All rights reserved.
Expand Down Expand Up @@ -252,7 +252,7 @@
return Error::Ok;
}

// Muse Glimmer image special-token ids (from the tokenizer / OnyxConfig).
// Muse Glimmer image special-token ids (from the tokenizer / MuseGlimmerConfig).

// Preprocess one image file, run the exported vision_encoder, and copy the
// resulting soft-token embeddings ([num_soft_tokens, hidden] bf16) to host.
Expand Down
23 changes: 0 additions & 23 deletions examples/models/muse-glimmer/tests/gen_prompt_golden.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,29 +11,6 @@

Only lengths, the special-token skeleton, and a digest of the full id sequence
are written. The text ids themselves stay out of the repo.

Re-validating after a regeneration
----------------------------------
``tokenizer.json`` and the golden are pinned to each other by sha256, so drift is
caught automatically and the checks below are not worth running on a schedule.
Run them once whenever you bump the pinned revision, to confirm the new
tokenizer still agrees with the independent sources.

1. tiktoken over ``l4_200k_base`` from the quantized repo. A different BPE
implementation reading a different vocab file. Build the Encoding as
``meta_reference_implementation/standalone_inference.py`` does, registering
``tokenizer_config.json``'s ``extra_special_tokens`` at ``200000 + index``,
then compare ``encode(prompt, allowed_special="all")`` against this
tokenizer for every case prompt. Last run: identical ids, all 7 cases.

2. The vocab embedded in ``onyx-rl_v2-q4km-gs128.gguf``. Compare all 202048
id-to-string pairs against ``tokenizer.json``. Last run: zero mismatches.

3. ``transformers`` with the Onyx wheel from the transformers_onyx repo, to
check assembly rather than the BPE. Pass ``current_date`` explicitly: the
template calls ``strftime_now``, so a rendering left to default is not
reproducible tomorrow. Last run: ``apply_chat_template`` output matched a
raw encode of the same text, 64 ids.
"""

import json
Expand Down
4 changes: 2 additions & 2 deletions examples/models/muse-glimmer/tests/prompt_cases.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@
IMAGE_MARKER = "<img>"

# Tokens the image span costs beyond the patches themselves. Zero: the canonical
# format is a bare <|patch|> run, which is what OnyxProcessor.replace_image_token
# emits and what meta_reference_implementation splices on.
# format is a bare <|patch|> run, which is what the Muse Glimmer processor's
# replace_image_token method emits and what meta_reference_implementation splices on.
Comment on lines 24 to +26
IMAGE_WRAPPER_TOKENS = 0

HF_DIR_ENV = "MUSE_GLIMMER_HF_DIR"
Expand Down
4 changes: 2 additions & 2 deletions examples/models/muse-glimmer/tests/test_vision_precompute.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

Verifies the host-side precomputed tensors (patchify, positional-embedding
interpolation, 2D-RoPE, sparse permutation, block-diagonal masks, pixel-shuffle
permutation) match the eager ``OnyxVisionEncoder`` math. Runs on CPU.
permutation) match the eager ``MuseGlimmerVisionEncoder`` math. Runs on CPU.
"""

import unittest
Expand Down Expand Up @@ -88,7 +88,7 @@ def test_matches_eager_complex_form(self):
grid_h = grid_w = 5
cos, sin = make_2d_rope(grid_h, grid_w, cfg)

# Eager reference (OnyxVisionEncoder._make_2d_rope), inline.
# Eager reference (MuseGlimmerVisionEncoder._make_2d_rope), inline.
head_dim = cfg.head_dim
half_dim = head_dim // 2
quarter = half_dim // 2
Expand Down
8 changes: 4 additions & 4 deletions examples/models/muse-glimmer/tests/test_vision_tower.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
"""Unit tests for the export-friendly Muse Glimmer vision encoder (vision_tower.py).

Checks (CPU):
* ``MuseGlimmerVisionEncoder`` reproduces an inline eager reference (the onyx vision
math) to bf16 tolerance, given shared random weights.
* ``MuseGlimmerVisionEncoder`` reproduces an inline eager reference (the Muse
Glimmer vision math) to bf16 tolerance, given shared random weights.
* The forward runs on a single-tile image (identity sparse perm) and a
multi-tile image (non-trivial sparse perm), producing the right shapes.
* ``torch.export(strict=True)`` traces the encoder with a dynamic num_patches.
Expand Down Expand Up @@ -60,11 +60,11 @@ def _init_random(model: torch.nn.Module, seed: int = 0) -> None:


# ---------------------------------------------------------------------------
# Inline eager reference (mirrors OnyxVisionEncoder math on float32).
# Inline eager reference (mirrors MuseGlimmerVisionEncoder math on float32).


def _rotate_interleaved_complex(x, cos, sin):
"""Adjacent-pair rotation via complex mul (eager onyx formulation)."""
"""Adjacent-pair rotation via complex mul (eager Muse Glimmer formulation)."""
freqs = torch.complex(cos, sin) # [P, d/2]
xc = torch.view_as_complex(x.float().reshape(*x.shape[:-1], -1, 2))
out = torch.view_as_real(xc * freqs.unsqueeze(0).unsqueeze(2)).flatten(-2)
Expand Down
Loading