Skip to content

Conversation

@chus-chus
Copy link
Contributor

@chus-chus chus-chus commented May 5, 2025

Motivation

Fix issue #6005 (AttributeError: 'MHATokenToKVPoolHost' object has no attribute 'start_layer' when using --enable-hierarchical-cache).

Modifications

Replaced instances of self.start_layer with self.device_pool.start_layer in both MLA and MHA TokenToKVPoolHost classes (memory_controller.py).

Checklist

@chus-chus chus-chus changed the title Fix "AttributeError: 'MHATokenToKVPoolHost' object has no attribute 'start_layer'" fix: Fix "AttributeError: 'MHATokenToKVPoolHost' object has no attribute 'start_layer'" May 5, 2025
@xiezhq-hermann xiezhq-hermann self-assigned this May 5, 2025
@xiezhq-hermann
Copy link
Collaborator

To merge with #6016.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants