Fix RWKV framework integration contracts - #8
Merged
Conversation
CI recapDashboard: View test results in Grafana
|
|
[For maintainers] Suggested jobs to run (before merge) run-slow: rwkv |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GenerationMixin, retaining only RWKV's cached-decode attention-mask rule, and mark the recurrent cache as non-compilabletrust_remote_code=Falsefrom native RWKV tokenizer and conversion loading testsThis intentionally excludes the proposed Qwen2/RWKV composition example and does not add padding/ragged batching, gradient checkpointing, tensor parallelism, compile/export, provider fallbacks, or kernel/backend changes.
Validation
Local
./.venvloadedFlashRWKV2==0.1.0a8from the installed public package.UV_CACHE_DIR=/tmp/transformers-rwkv-uv-cache uv run pytest -q tests/models/rwkv/test_tokenization_rwkv.py tests/models/rwkv/test_modeling_rwkv.py—53 passed, 2 skipped; the available CUDA cases covered training forward/backward, stateful training, prefill/decode continuation, hidden-state capture, tuple output, greedy generation, stream-scoped metadata, and CUDA Graph captureUV_CACHE_DIR=/tmp/transformers-rwkv-uv-cache uv run make typing— passed both type annotations and modeling structure checks with the branch-pinnedtransformers-mlinter==0.1.2UV_CACHE_DIR=/tmp/transformers-rwkv-uv-cache uv run make style— passedgit diff --check origin/main...HEAD— passedThe planned independent run on
rwkv-sha-pro6000x8was stopped before syncing because its existingrwkvworktree has overlapping uncommitted RWKV test changes. Nothing on that remote worktree was modified. A read-only environment check confirmed that it loadsFlashRWKV2==0.1.0a8.Initial fork CI was checked after opening this PR.
Check code quality,Check repository consistency, andTest preparation (fetcher)each failed before executing any step with the GitHub annotationRequired runner group 'aws-m8i-l-cache' not found; their dependent test jobs were therefore skipped. The independentanti-slopandsecurity-gate / bandit-checkchecks passed. This is a fork runner-configuration blocker, not a source-test failure.Upstream relationship
Hugging Face Transformers already has overlapping RWKV-7 work in #47780 and #47787. This PR maintains the FlashRWKV2-specific implementation in the
rwkv-rs/transformers-rwkvfork only; it does not claim to replace or supersede that upstream work.AI assistance
This implementation was AI-assisted. The resulting four-file diff was reviewed path-by-path and validated with the commands and results above.