Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/models/muse-glimmer/runtime/runners/solo.cpp
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
/*
* Copyright (c) Meta Platforms, Inc. and affiliates.
* All rights reserved.
Expand Down Expand Up @@ -252,7 +252,7 @@
return Error::Ok;
}

// Muse Glimmer image special-token ids (from the tokenizer / OnyxConfig).
// Muse Glimmer image special-token ids (from the tokenizer / MuseGlimmerConfig).

// Preprocess one image file, run the exported vision_encoder, and copy the
// resulting soft-token embeddings ([num_soft_tokens, hidden] bf16) to host.
Expand Down
23 changes: 0 additions & 23 deletions examples/models/muse-glimmer/tests/gen_prompt_golden.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,29 +11,6 @@

Only lengths, the special-token skeleton, and a digest of the full id sequence
are written. The text ids themselves stay out of the repo.

Re-validating after a regeneration
----------------------------------
``tokenizer.json`` and the golden are pinned to each other by sha256, so drift is
caught automatically and the checks below are not worth running on a schedule.
Run them once whenever you bump the pinned revision, to confirm the new
tokenizer still agrees with the independent sources.

1. tiktoken over ``l4_200k_base`` from the quantized repo. A different BPE
implementation reading a different vocab file. Build the Encoding as
``meta_reference_implementation/standalone_inference.py`` does, registering
``tokenizer_config.json``'s ``extra_special_tokens`` at ``200000 + index``,
then compare ``encode(prompt, allowed_special="all")`` against this
tokenizer for every case prompt. Last run: identical ids, all 7 cases.

2. The vocab embedded in ``onyx-rl_v2-q4km-gs128.gguf``. Compare all 202048
id-to-string pairs against ``tokenizer.json``. Last run: zero mismatches.

3. ``transformers`` with the Onyx wheel from the transformers_onyx repo, to
check assembly rather than the BPE. Pass ``current_date`` explicitly: the
template calls ``strftime_now``, so a rendering left to default is not
reproducible tomorrow. Last run: ``apply_chat_template`` output matched a
raw encode of the same text, 64 ids.
"""

import json
Expand Down
4 changes: 2 additions & 2 deletions examples/models/muse-glimmer/tests/prompt_cases.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@
IMAGE_MARKER = "<img>"

# Tokens the image span costs beyond the patches themselves. Zero: the canonical
# format is a bare <|patch|> run, which is what OnyxProcessor.replace_image_token
# emits and what meta_reference_implementation splices on.
# format is a bare <|patch|> run, which is what the Muse Glimmer processor's
# replace_image_token method emits and what meta_reference_implementation splices on.
Comment on lines 24 to +26
IMAGE_WRAPPER_TOKENS = 0

HF_DIR_ENV = "MUSE_GLIMMER_HF_DIR"
Expand Down
4 changes: 2 additions & 2 deletions examples/models/muse-glimmer/tests/test_vision_precompute.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

Verifies the host-side precomputed tensors (patchify, positional-embedding
interpolation, 2D-RoPE, sparse permutation, block-diagonal masks, pixel-shuffle
permutation) match the eager ``OnyxVisionEncoder`` math. Runs on CPU.
permutation) match the eager ``MuseGlimmerVisionEncoder`` math. Runs on CPU.
"""

import unittest
Expand Down Expand Up @@ -88,7 +88,7 @@ def test_matches_eager_complex_form(self):
grid_h = grid_w = 5
cos, sin = make_2d_rope(grid_h, grid_w, cfg)

# Eager reference (OnyxVisionEncoder._make_2d_rope), inline.
# Eager reference (MuseGlimmerVisionEncoder._make_2d_rope), inline.
head_dim = cfg.head_dim
half_dim = head_dim // 2
quarter = half_dim // 2
Expand Down
8 changes: 4 additions & 4 deletions examples/models/muse-glimmer/tests/test_vision_tower.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
"""Unit tests for the export-friendly Muse Glimmer vision encoder (vision_tower.py).

Checks (CPU):
* ``MuseGlimmerVisionEncoder`` reproduces an inline eager reference (the onyx vision
math) to bf16 tolerance, given shared random weights.
* ``MuseGlimmerVisionEncoder`` reproduces an inline eager reference (the Muse
Glimmer vision math) to bf16 tolerance, given shared random weights.
* The forward runs on a single-tile image (identity sparse perm) and a
multi-tile image (non-trivial sparse perm), producing the right shapes.
* ``torch.export(strict=True)`` traces the encoder with a dynamic num_patches.
Expand Down Expand Up @@ -60,11 +60,11 @@ def _init_random(model: torch.nn.Module, seed: int = 0) -> None:


# ---------------------------------------------------------------------------
# Inline eager reference (mirrors OnyxVisionEncoder math on float32).
# Inline eager reference (mirrors MuseGlimmerVisionEncoder math on float32).


def _rotate_interleaved_complex(x, cos, sin):
"""Adjacent-pair rotation via complex mul (eager onyx formulation)."""
"""Adjacent-pair rotation via complex mul (eager Muse Glimmer formulation)."""
freqs = torch.complex(cos, sin) # [P, d/2]
xc = torch.view_as_complex(x.float().reshape(*x.shape[:-1], -1, 2))
out = torch.view_as_real(xc * freqs.unsqueeze(0).unsqueeze(2)).flatten(-2)
Expand Down
Loading