Skip to content

feat(framework): allow overriding the vLLM repository to target upstream vllm - #1029

Closed
akii96 wants to merge 1 commit into
AMD-AGI:mainfrom
akii96:fix/use-upstream-vllm-repo
Closed

feat(framework): allow overriding the vLLM repository to target upstream vllm#1029
akii96 wants to merge 1 commit into
AMD-AGI:mainfrom
akii96:fix/use-upstream-vllm-repo

Conversation

@akii96

@akii96 akii96 commented Jul 24, 2026

Copy link
Copy Markdown

Summary

Allow operators to override Hyperloom’s vLLM repository while preserving the existing ROCm-fork default.

Motivation

Most vLLM development happens in vllm-project/vllm, including new model support, quantization formats, scheduler changes, serving features, and general kernel integration. Deployments built from upstream vLLM should be able to search that larger and more current PR stream.

Hyperloom currently searches ROCm/vllm unconditionally. That fork may be valuable for AMD-specific fixes that may have not yet reached upstream, so this change preserves it as the default while allowing operators to select the repository that matches their installed runtime!

Changes

  • Add HYPERLOOM_VLLM_REPO_URL.
  • Keep ROCm/vllm as the default.
  • Apply the override consistently to framework discovery and PR Monitor.
  • Preserve the ROCm fork as a recognized origin for existing candidates.
  • Canonicalize GitHub HTTPS, SSH, and owner/name forms.
  • Reject invalid or non-GitHub overrides clearly.
  • Document the setting and upstream PR-KB namespace.

Testing

  • Focused framework, discovery, prompt, registry, and PR-KB suite: 221 passed, 1 skipped

@akii96
akii96 requested a review from a team as a code owner July 24, 2026 20:37
Comment thread src/hyperloom/orchestrator/framework/client.py Fixed
Keep ROCm/vllm as the default while allowing operators to target an upstream vLLM checkout and PR stream.

Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
@tsrikris

tsrikris commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

@chaojhou not sure if we should open up to custom containers without some manual sanitization. I'm concerned aspects of the recommended container will be missing if a user adds in their own container/setup script and lead to secondary bugs. What do you think?

The challenge I can see for egs. is that interfaces haven't been tested for different vLLM/SGLang versions and the right approach might be for us to plan how to enable all vLLM/SGLang versions

@devalshahamd

Copy link
Copy Markdown
Collaborator

Hi @akii96 ROCm/vllm has been deprecated (branches are used for local development AFAIK) and we use vllm-project/vllm for all practical usage. Can we just change the ROCm/vllm to vllm-project/vllm rather than adding new variables and code?

@akii96

akii96 commented Jul 29, 2026

Copy link
Copy Markdown
Author

@devalshahamd Yeah, we can! But after the discussion in meeting today I am just pondering now about what is expected of framework agent?

For instance if I were to take a docker image with the latest upstream vllm image installed and then install hyperloom + the dependencies on it and run the optimize loop

the framework agent then in most cases only rediscovers what is already merged and available in the installed vllm, is that correct? 😅 It might surface an env var or serving arg at best but then starts to overlap with what the hyperloom's grid sweeper does

Did I get this correctly ?

@tsrikris tsrikris closed this Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants