Skip to content

[RWKV] Align module names with Transformers conventions - #11

Merged
Cai-z-us merged 1 commit into
mainfrom
fix/rwkv-community-integration-contracts
Aug 27, 2026
Merged

[RWKV] Align module names with Transformers conventions#11
Cai-z-us merged 1 commit into
mainfrom
fix/rwkv-community-integration-contracts

Conversation

@Cai-z-us

@Cai-z-us Cai-z-us commented Aug 27, 2026

Copy link
Copy Markdown

CPU CI GPU run-slow

What does this PR do?

Follows up on #10 by making the RWKV-7 model layout native to the current Transformers and FLA ecosystem. Canonical BlinkDL names now exist only at the .pth conversion boundary; the Transformers model and serialized state dict use standard decoder/module names without duplicate aliases.

  • rename the model layout to embed_tokens, embedding_norm, layers, norm, and lm_head
  • rename the decoder components to RwkvDecoderLayer, linear_attn, and mlp
  • rename dense projections to r_proj, k_proj, v_proj, o_proj, up_proj, and down_proj
  • rename the linear-attention GroupNorm to g_norm
  • rename public stateful-training fields to attention_shift, recurrent_state, and mlp_shift
  • translate BlinkDL checkpoint keys once in temp/rwkv_pth2st.py, including blocks.0.ln0.* -> model.embedding_norm.*
  • remove the obsolete RWKV-4 converter, which emitted a third incompatible naming scheme
  • update LoRA targets, documentation, structure tests, conversion round trips, and inference-layout tests

The RWKV math, raw formula parameters, canonical head_size=64, FlashRWKV2 provider calls, Git-pinned tokenizer dependency, and support boundaries are unchanged. Qwen2Rwkv remains out of scope. External consumers such as vLLM will be updated separately to consume the new state-dict contract.

Coordination

No repository issue owns this follow-up, and the repository owner requested the rename directly. Duplicate-work checks found no overlapping open PR in this repository. Upstream already has huggingface#47780 and issue huggingface#47787, so this PR targets only the rwkv-rs integration repository and does not create competing upstream work.

Validation

  • local RWKV modeling/GPU suite: 50 passed, 1 skipped
  • local RWKV tokenizer suite: 11 passed
  • focused shared linear-attention cache test: 1 passed
  • rwkv-sha-pro6000x8, exact commit 38fd0b212a1e, official pinned vocabulary: 63 passed, 9 subtests passed
  • make typing
  • RWKV modeling structure check with all TRF rules enabled and cache disabled
  • make check-repository-consistency
  • wheel metadata: unconditional exact Git tokenizer commit; rwkv extra adds only FlashRWKV2==0.1.0a8
  • make style
  • git diff --check

Before submitting

  • Read the contributor and PR checks
  • Updated the public naming and conversion documentation
  • Added/updated regression tests for every renamed public path

@Cai-z-us
Cai-z-us force-pushed the fix/rwkv-community-integration-contracts branch from 49b17d4 to 38fd0b2 Compare August 27, 2026 09:08
@github-actions

Copy link
Copy Markdown

Thank you for your contribution 🤗!

CI Security Gate — automatic approval blocked

This PR was not automatically approved for CI because the security gate failed.

Possible reasons:

  • The PR touches 50 or more files — only PRs with fewer than 50 changed files are automatically approved
  • A changed file is outside the allowed directories (src/, tests/, docs/, utils/), has a disallowed extension (only .py, .txt, .md permitted outside tests/ and docs/), or is not .md/.yml inside docs/ — this covers files the PR deletes or renames, not only the ones it edits
  • A new high-severity security issue was detected in the changed Python files (Bandit check)
  • The PR touches a path this repository protects from untrusted PRs, such as the file that decides who reviews it — a maintainer must make that change in a separate PR

See the workflow run for the exact violations.

A maintainer can review and manually approve CI if a finding is a false positive.

@github-actions

Copy link
Copy Markdown

[For maintainers] Suggested jobs to run (before merge)

run-slow: rwkv

@github-actions

Copy link
Copy Markdown

CI recap

Dashboard: View test results in Grafana
Latest run: 33057174854
Result: failure | Grafana metrics are not available yet.

Code quality check failed: test jobs were skipped. Fix the code quality issues and push again to run tests.

@Cai-z-us
Cai-z-us merged commit a57893d into main Aug 27, 2026
42 of 51 checks passed
@Cai-z-us
Cai-z-us deleted the fix/rwkv-community-integration-contracts branch August 27, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant