[Spyre-Next] 🎨 Fix docstring inaccuracies and typos by yannicks1 · Pull Request #880 · torch-spyre/sendnn-inference

yannicks1 · 2026-03-31T07:25:16Z

Description

Fix docstring inaccuracies, typos and typing.

Changes:

cleans up docstrings after [Spyre-Next] Reworked rms_norm #873
cleans up comment after ✨ Add boh's custom RMS norm #754
typos and typing

Test Plan

Documentation-only changes, no functional impact.

Checklist

I have read the contributing guidelines
My code follows the project's code style (run bash format.sh)
I have added tests for my changes (if applicable)
I have updated the documentation (if applicable)
My commits include a Signed-off-by: line (DCO compliance)

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

yannicks1 · 2026-03-31T07:29:38Z


        Key differences from upstream:
-            - Uses transpose(-1, -2) for computation efficiency on Spyre
            - Creates epsilon tensor via torch.ops.spyre.full() instead of scalar


@bohnstingl I was wondering: is this still true? Is full a custom spyre ops or do we use torch.full?

No, this is not true anymore. We can now just use torch.full with torch-spyre.

I changed that with my latest PR

yannicks1 · 2026-03-31T07:31:01Z

@@ -18,7 +18,6 @@
    - Minimum batch size: 64 (due to spyre constraint, automatically padded)
    - Device dtype: float16 (converted for CPU)
    - Output dtype: bfloat16 (converted on CPU)


@bohnstingl Is this always bfloat16 or is it just matching the input data type?

Actually, maybe we can rephrase these dtypes a bit in general?
The Input dtype is defined by the model, respectively by the user. The computation in our wrappings are then always carried out in torch.float16.

do you have a suggestion?

sth like

Suggested change

- Output dtype: bfloat16 (converted on CPU)

- Output dtype: model data type/ user defined (converted on CPU)

what about the bfloat mentions in line 158 and 168?

Corrected in my PR.

github-actions · 2026-03-31T07:39:19Z

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, run ./format.sh.
Now you are good to go 🚀.

We also recommend installing prek and configuring it to check your code before every local commit.

bohnstingl

Thank you very much @yannicks1 for opening this PR. I think it is very valuable to do these kind of refactors every once in a while 😊

bohnstingl · 2026-03-31T18:50:43Z

@@ -18,7 +18,6 @@
    - Minimum batch size: 64 (due to spyre constraint, automatically padded)
    - Device dtype: float16 (converted for CPU)
    - Output dtype: bfloat16 (converted on CPU)


Actually, maybe we can rephrase these dtypes a bit in general?
The Input dtype is defined by the model, respectively by the user. The computation in our wrappings are then always carried out in torch.float16.

bohnstingl · 2026-03-31T18:52:39Z


        Key differences from upstream:
-            - Uses transpose(-1, -2) for computation efficiency on Spyre
            - Creates epsilon tensor via torch.ops.spyre.full() instead of scalar


No, this is not true anymore. We can now just use torch.full with torch-spyre.

Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>

bohnstingl · 2026-03-31T19:50:57Z

@yannicks1, could you please rebase to latest main?
I also made some additional, small docstring changes. Please check and feel free to either merge in or ignore. yannicks1#1

[Docstrings] Some additional docstring updates

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

yannicks1 · 2026-03-31T23:00:08Z

@@ -18,7 +18,6 @@
    - Minimum batch size: 64 (due to spyre constraint, automatically padded)
    - Device dtype: float16 (converted for CPU)
    - Output dtype: bfloat16 (converted on CPU)


do you have a suggestion?

sth like

Suggested change

- Output dtype: bfloat16 (converted on CPU)

- Output dtype: model data type/ user defined (converted on CPU)

yannicks1 · 2026-03-31T23:02:28Z

@@ -18,7 +18,6 @@
    - Minimum batch size: 64 (due to spyre constraint, automatically padded)
    - Device dtype: float16 (converted for CPU)
    - Output dtype: bfloat16 (converted on CPU)


what about the bfloat mentions in line 158 and 168?

Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>

…ocstrings Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>

bohnstingl

@yannicks1 I created a PR with small changes again

bohnstingl · 2026-04-01T13:11:01Z

@@ -18,7 +18,6 @@
    - Minimum batch size: 64 (due to spyre constraint, automatically padded)
    - Device dtype: float16 (converted for CPU)
    - Output dtype: bfloat16 (converted on CPU)


Corrected in my PR.

bohnstingl · 2026-04-01T13:11:03Z


        Key differences from upstream:
-            - Uses transpose(-1, -2) for computation efficiency on Spyre
            - Creates epsilon tensor via torch.ops.spyre.full() instead of scalar


I changed that with my latest PR

Docstrings

…ings

Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>

…ocstrings

Added debug info about CustomOps

bohnstingl

LGTM!

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

yannicks1 requested review from joerunde, nikolaospapandreou, prashantgupta24, sducouedic and tdoublep as code owners March 31, 2026 07:25

yannicks1 added 4 commits March 31, 2026 09:26

remove redundant comment

dae43ac

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

remove old comments about transpose

9a14e51

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

fix typo

c07bf2a

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

fix typing

40dd656

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

yannicks1 force-pushed the small_things branch from 2069208 to 40dd656 Compare March 31, 2026 07:27

github-actions bot changed the title ~~🎨 Fix docstring inaccuracies and typos~~ [Spyre-Next] 🎨 Fix docstring inaccuracies and typos Mar 31, 2026

yannicks1 commented Mar 31, 2026

View reviewed changes

bohnstingl reviewed Mar 31, 2026

View reviewed changes

Minor docstring updates

a0257c7

Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>

yannicks1 added 3 commits April 1, 2026 00:47

Merge pull request #1 from bohnstingl/docstrings

f7d33ef

[Docstrings] Some additional docstring updates

remove torch.ops.spyre.full() reference

bc69351

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

remove unused argument

1c5204a

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

yannicks1 commented Mar 31, 2026

View reviewed changes

bohnstingl added 2 commits April 1, 2026 13:05

Various docstring corrections

7e62590

Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>

Merge branch 'small_things' of github.com:yannicks1/vllm-spyre into d…

6de1a6d

…ocstrings Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>

bohnstingl requested changes Apr 1, 2026

View reviewed changes

Merge pull request #2 from bohnstingl/docstrings

b7ea825

Docstrings

yannicks1 requested a review from bohnstingl April 1, 2026 13:58

bohnstingl mentioned this pull request Apr 8, 2026

[Spyre-Next] E2E test with optional offloading to spyre layers #900

Merged

5 tasks

bohnstingl and others added 4 commits April 8, 2026 19:18

Merge branch 'main' of github.com:vllm-project/vllm-spyre into docstr…

94e1eae

…ings

Added debug statement to silu and embedding

bff1d4a

Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>

Merge branch 'main' into small_things

adc4334

Merge branch 'small_things' of github.com:yannicks1/vllm-spyre into d…

f4da8cd

…ocstrings

Merge pull request #3 from bohnstingl/docstrings

e87f586

Added debug info about CustomOps

yannicks1 enabled auto-merge (squash) April 9, 2026 08:15

bohnstingl approved these changes Apr 9, 2026

View reviewed changes

github-actions bot added the ready Runs the full CI test suite. Only add to PRs once ready to merge to limit public GHA usage label Apr 9, 2026

yannicks1 requested a review from bohnstingl April 9, 2026 08:16

fix ruff

c43484a

Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

yannicks1 merged commit 95f6a44 into torch-spyre:main Apr 9, 2026
14 checks passed

	- Output dtype: bfloat16 (converted on CPU)
	- Output dtype: model data type/ user defined (converted on CPU)

Conversation

yannicks1 commented Mar 31, 2026

Description

Test Plan

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 31, 2026

Uh oh!

bohnstingl left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bohnstingl commented Mar 31, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bohnstingl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bohnstingl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants