[kv_offload+HMA][6/N]: Split offloading_connector.py#37405
[kv_offload+HMA][6/N]: Split offloading_connector.py#37405NickLucche merged 1 commit intovllm-project:mainfrom
Conversation
offloading_connector.py is quite long and contains multiple logical independent classes. This commit splits it to 3 files: scheduler-side connector, worker-side connector, and metrics. Signed-off-by: Or Ozeri <oro@il.ibm.com>
orozery
force-pushed
the
offloading-connector-split
branch
from
ed2160f to
51ffdc3
Compare
orozery
changed the title
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
The pull request successfully refactors the offloading_connector.py file by splitting its functionality into dedicated common, metrics, scheduler, and worker modules. This significantly improves modularity and maintainability, making the codebase easier to understand and extend. The changes align well with the stated goal of preparing for additional code. However, a critical bug was identified in the metrics.py file where OffloadingOperationMetrics dataclass instances are incorrectly accessed as dictionaries, which will lead to runtime errors.
False detection. |
2 participants
offloading_connector.py is quite long and contains multiple logical independent classes.
This PR splits it to 3 files: scheduler-side connector, worker-side connector, and metrics.
This is in preparation for additional code that will be added to offloading connector which will increase its number of lines.