feat(rwkv): replace RWKV-4 with RWKV-7 - #12
Merged
Merged
Conversation
|
Thank you for your contribution 🤗! CI Security Gate — automatic approval blockedThis PR was not automatically approved for CI because the security gate failed. Possible reasons:
See the workflow run for the exact violations. A maintainer can review and manually approve CI if a finding is a false positive. |
CI recapDashboard: View test results in Grafana
|
|
[For maintainers] Suggested jobs to run (before merge) run-slow: auto, rwkv |
Cai-z-us
added a commit
that referenced
this pull request
Aug 28, 2026
Replace the legacy RWKV-4 integration with RWKV-7, FlashRWKV2 a9, tokenizer, converter, tests, and documentation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Replaces the legacy RWKV-4 implementation under the existing
rwkvmodel identity with a clean RWKV-7 integration based on the current upstream Transformersmain.Model and runtime
FlashRWKV2==0.1.0a9Tokenizer, conversion, and docs
RwkvTokenizerbacked bytokenizers-rwkv.pthto sharded Safetensors converterThis intentionally does not preserve the former RWKV-4 Python/state/checkpoint API and does not add CPU or FLA fallbacks.
An upstream RWKV-7 PR already exists at huggingface#47780, so this PR targets the project integration branch only and does not create duplicate upstream work.
Validation
make stylepython -m pytest -q tests/models/rwkv/test_modeling_rwkv_mock_provider.py tests/models/rwkv/test_tokenization_rwkv.py— 13 passedgit diff --checkCode Agent Policy
This PR is opened in the project fork with explicit maintainer authorization. It is not submitted to
huggingface/transformers.Before submitting