Skip to content

GPT OSS Integration Code#771

Closed
hlahkar wants to merge 25 commits intovllm-project:mainfrom
hlahkar:gpt_oss_latest
Closed

GPT OSS Integration Code#771
hlahkar wants to merge 25 commits intovllm-project:mainfrom
hlahkar:gpt_oss_latest

Conversation

@hlahkar
Copy link
Copy Markdown
Contributor

@hlahkar hlahkar commented Jan 2, 2026

No description provided.

GPT OSS Code
7d63e51 
Signed-off-by: Himangshu Lahkar <hlahkar@habana.ai>
Copilot AI review requested due to automatic review settings January 2, 2026 01:42
Copilot AI reviewed Jan 2, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR integrates support for the GPT OSS model type, including additions for handling model-specific routing logic, bias support in MoE layers, and attention sink mechanisms for improved inference.

  • Adds GPT OSS-specific expert routing and softmax handling in the MoE forward pass
  • Implements bias support throughout the MoE pipeline
  • Introduces attention sink functionality across attention backends and operations

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
vllm_gaudi/v1/worker/hpu_model_runner.py Increases sliding window block size calculation by 1
vllm_gaudi/ops/hpu_fused_moe.py Adds GPT OSS model type detection, bias handling in MoE operations, and model-specific expert routing
vllm_gaudi/extension/utils.py Adds sinks parameter support to forward pass
vllm_gaudi/extension/ops.py Implements sink attention mechanism in pipelined and naive attention functions, adds bias support to MoE operations
vllm_gaudi/attention/backends/hpu_attn.py Adds sinks parameter and dtype consistency checks in attention implementation

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread vllm_gaudi/extension/ops.py
@hlahkar hlahkar mentioned this pull request Jan 2, 2026
Closed
Himangshu Lahkar added 3 commits January 2, 2026 04:36
Update MOE
0b49b97 
Signed-off-by: Himangshu Lahkar <hlahkar@habana.ai>
Update Pipelined PA
9cdf3f3 
Signed-off-by: Himangshu Lahkar <hlahkar@habana.ai>
Format MOE
7c8e4eb 
Signed-off-by: Himangshu Lahkar <hlahkar@habana.ai>
@github-actions github-actions Bot mentioned this pull request Jan 2, 2026
Open
Himangshu Lahkar and others added 5 commits January 2, 2026 06:59
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 19, 2026

🚧 CI Blocked

The main CI workflow was not started for the following reason:

Your branch is behind the base branch. Please merge or rebase to get the latest changes.

@wpyszka
Copy link
Copy Markdown
Collaborator

wpyszka commented Jan 19, 2026

/run-gaudi-tests

wpyszka and others added 3 commits January 19, 2026 13:29
@wpyszka
Merge branch 'main' into gpt_oss_latest
9a37437
@hlahkar
Merge branch 'main' into gpt_oss_latest
87edb67 
Signed-off-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
qkv dtype cast, needed for GPT-OSS
69f4178 
Signed-off-by: Himangshu Lahkar <hlahkar@habana.ai>
@hlahkar
Copy link
Copy Markdown
Contributor Author

hlahkar commented Jan 22, 2026

/run-gaudi-tests

@sys-hab-pt-service
Copy link
Copy Markdown
Collaborator

sys-hab-pt-service commented Jan 22, 2026

Only codeowners and testowners can request to run Gaudi tests. Contact list: kzawora-intel, xuechendi, adobrzyn, mgawarkiewicz-intel, afierka-intel, michalkuligowski, iboiko-habana, kamil-kaczor, ksmusz, PatrykWo, kamil-kaczor, kfojcik-intel, ksmusz, wuxun-zhang, xuechendi, attafosu, ulivne, Kacper-Pietkun, iboiko-habana, jkaniecki, jbyczkow, wpyszka

hlahkar and others added 3 commits January 23, 2026 06:07
@hlahkar
Merge branch 'main' into gpt_oss_latest
f2c3d2d
Put a check for hf_config
bc26332 
Signed-off-by: Himangshu Lahkar <hlahkar@habana.ai>
Make bias the last arg
a63ceb2 
Signed-off-by: Himangshu Lahkar <hlahkar@habana.ai>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 23, 2026

🚧 CI Blocked

The main CI workflow was not started for the following reason:

Your branch is behind the base branch. Please merge or rebase to get the latest changes.

@wpyszka
Copy link
Copy Markdown
Collaborator

wpyszka commented Jan 23, 2026

/run-gaudi-tests

Himangshu Lahkar and others added 5 commits January 27, 2026 08:07
check for bias not None
0f46ae2 
Signed-off-by: Himangshu Lahkar <hlahkar@habana.ai>
@afierka-intel
Fix Llama4 shape mismatch for 32k+ context window (vllm-project#842) (v…
8b4bd00 
…llm-project#855)

Llama4 for `max_model_len > 32k` enable temperature adjustment
https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/llama4.py#L719.
Enabled adjustment causes tensor `q` shape modification from 2D to 3D:
https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/llama4.py#L307.
This tensor is passing to `UnqnatizedFusedMoEMetod -> forward`:
https://github.com/vllm-project/vllm-gaudi/blob/main/vllm_gaudi/ops/hpu_fused_moe.py#L163
causing invalid reshaping - we trying to return a 3D `output.view` based
on 2D output tensor.

Found that following PR introduced the bug: vllm-project#680 and vllm-project#684

Cherry-picked from `releases/v0.13.0`

---------

Signed-off-by: Artur Fierka <artur.fierka@intel.com>
@dudilester @kamil-kaczor
Fix HPU model runner profile_run to work with dynamic kv-cache scales (…
ba3e55a 
…vllm-project#852)

Signed-off-by: Dudi Lester <dlester@habana.ai>
Co-authored-by: Kamil Kaczor <kamil.kaczor@intel.com>
@adobrzyn @xuechendi
Revert "skip HPU graphs for long prefills" (vllm-project#850)
8a68c7d 
Reverts vllm-project#780

---------

Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
fix pre-commit error
f3a4560 
Signed-off-by: Himangshu Lahkar <hlahkar@habana.ai>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 27, 2026

🚧 CI Blocked

The main CI workflow was not started for the following reason:

Your branch is behind the base branch. Please merge or rebase to get the latest changes.

@wpyszka
Copy link
Copy Markdown
Collaborator

wpyszka commented Jan 27, 2026

/run-gaudi-tests

@hlahkar
Copy link
Copy Markdown
Contributor Author

hlahkar commented Feb 5, 2026

PR is taken care through #887

@hlahkar hlahkar closed this Feb 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@kzawora-intel kzawora-intel Awaiting requested review from kzawora-intel

@xuechendi xuechendi Awaiting requested review from xuechendi xuechendi is a code owner

@adobrzyn adobrzyn Awaiting requested review from adobrzyn adobrzyn is a code owner

@mgawarkiewicz-intel mgawarkiewicz-intel Awaiting requested review from mgawarkiewicz-intel mgawarkiewicz-intel is a code owner

@afierka-intel afierka-intel Awaiting requested review from afierka-intel afierka-intel is a code owner

@michalkuligowski michalkuligowski Awaiting requested review from michalkuligowski michalkuligowski is a code owner

@iboiko-habana iboiko-habana Awaiting requested review from iboiko-habana iboiko-habana is a code owner

@kamil-kaczor kamil-kaczor Awaiting requested review from kamil-kaczor kamil-kaczor is a code owner

@ksmusz ksmusz Awaiting requested review from ksmusz ksmusz is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@hlahkar @wpyszka @sys-hab-pt-service @afierka-intel @dudilester @adobrzyn