Skip to content

Conversation

@bryce13950
Copy link
Collaborator

  • added new functions for determining the best available device

  • added device test and integrated new function into device selection

  • fixed reference

  • made sure embed are still on the same device

  • added log point

  • added more log points

  • fixed typo

  • set W_E to be on the same device

  • set rms norm to correct device

  • set device to grouped query attention

  • Revert "set device to grouped query attention"

This reverts commit 788d355.

  • Revert "set rms norm to correct device"

This reverts commit e7018c4.

  • Revert "set W_E to be on the same device"

This reverts commit 33f4436.

  • added debug points

  • reverted most calls to new function

  • reverted device list

  • reverted block loop

  • reverted cache call

  • updated move model function to use calculations

  • fixed remaining device identification issues

  • restored if

  • made sure rms norm or on the same device before calculations

  • added device check before linear attn

  • checked b_

  • moved device selection

  • moved rotary to device

  • changed device move

  • rassigned prop

  • made sure all abstract attention tensors are on the same device

  • remvoed assignment

  • updated prop setting

  • put resid pre on device

  • removed some log points

  • ran format

  • resolved test

  • removed reassignment

  • changed device selection point

  • esnure gated mlp items are on the same device

  • chagned device direction

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

* added new functions for determining the best available device

* added device test and integrated new function into device selection

* fixed reference

* made sure embed are still on the same device

* added log point

* added more log points

* fixed typo

* set W_E to be on the same device

* set rms norm to correct device

* set device to grouped query attention

* Revert "set device to grouped query attention"

This reverts commit 788d355.

* Revert "set rms norm to correct device"

This reverts commit e7018c4.

* Revert "set W_E to be on the same device"

This reverts commit 33f4436.

* added debug points

* reverted most calls to new function

* reverted device list

* reverted block loop

* reverted cache call

* updated move model function to use calculations

* fixed remaining device identification issues

* restored if

* made sure rms norm or on the same device before calculations

* added device check before linear attn

* checked b_

* moved device selection

* moved rotary to device

* changed device move

* rassigned prop

* made sure all abstract attention tensors are on the same device

* remvoed assignment

* updated prop setting

* put resid pre on device

* removed some log points

* ran format

* resolved test

* removed reassignment

* changed device selection point

* esnure gated mlp items are on the same device

* chagned device direction
@bryce13950 bryce13950 merged commit d7f9eb1 into main Feb 13, 2025
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants