Skip to content

[fix] adding temperature only when fused kernels are used#1688

Closed
mingruimingrui wants to merge 1 commit intoverl-project:mainfrom
mingruimingrui:fix/extra-forward-kwargs
Closed

[fix] adding temperature only when fused kernels are used#1688
mingruimingrui wants to merge 1 commit intoverl-project:mainfrom
mingruimingrui:fix/extra-forward-kwargs

Conversation

@mingruimingrui
Copy link
Contributor

Checklist Before Starting

  • Search for similar PR(s).

What does this PR do?

Temperature was automatically added to dp_actor model.forward, however this breaks under some transformers version.
This PR only adds temperature to model.forward when fused_kernels patch is used.

High-Level Design

Specific Changes

API

Usage Example

Test

Additional Info.

  • Issue Number: Fixes issue # or discussion # if any.
  • Training: [Note which backend this PR will affect: FSDP, Megatron, both, or none]
  • Inference: [Note which backend this PR will affect: vLLM, SGLang, both, or none]

Checklist Before Submitting

  • Read the Contribute Guide.
  • Apply pre-commit checks.
  • Add [BREAKING] to the PR title if it breaks any API.
  • Update the documentation about your changes in the docs.
  • Add CI test(s) if necessary.

@hiyouga
Copy link
Collaborator

hiyouga commented May 26, 2025

duplicated of #1687

@hiyouga hiyouga closed this May 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants