Repository navigation
feat(hipblaslt): Introduce Solution Unique ID generation foundation #12185
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 10 commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
71e0690
feat(hipblaslt): add SolutionUID generation foundation
dsaffars 0d613df
ci(hipblaslt): gate SolutionUID uniqueness
dsaffars 679724c
test(hipblaslt): shrink concurrent SolutionUID generation coverage
dsaffars 11d8701
test(hipblaslt): cover SolutionUID generator error paths
dsaffars 0c247bc
feat(hipblaslt): encode SolutionUID as base62
dsaffars 3a9a8de
fix(tensilelite) Loading unit test for CI component without rocisa
dsaffars e7a1914
test(hipblaslt): skip UID uniqueness scan when logic corpus is absent
dsaffars b94afb8
Merge branch 'develop' into users/dsaffars/universal_unique_id_step_1
dsaffars 9335d0c
Merge branch 'develop' into users/dsaffars/universal_unique_id_step_1
dsaffars 25b6d49
chore(hipblaslt): place SolutionUID with the leading logic YAML keys
dsaffars 1074f03
Merge branch 'develop' into users/dsaffars/universal_unique_id_step_1
dsaffars a225888
Merge branch 'develop' into users/dsaffars/universal_unique_id_step_1
JeniferC99 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| # Copyright (c) Advanced Micro Devices, Inc., or its affiliates. | ||
| # SPDX-License-Identifier: MIT | ||
|
|
||
| name: "Component CI: Library Uniqueness" | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| docker_image: | ||
| type: string | ||
| required: true | ||
| linux_runner: | ||
| type: string | ||
| required: true | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| linux: | ||
| name: Linux | ||
| runs-on: ${{ inputs.linux_runner }} | ||
| container: | ||
| image: ${{ inputs.docker_image }} | ||
| defaults: | ||
| run: | ||
| shell: bash | ||
| working-directory: projects/hipblaslt/tensilelite | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
|
|
||
| - name: Install tox | ||
| run: python -m pip install tox | ||
|
|
||
| - name: Run Library Uniqueness gate | ||
| run: python -m tox -e library-uniqueness |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| # Solution UID foundation | ||
|
|
||
| `SolutionUID` is a stable 64-bit identifier for a hipBLASLt/TensileLite | ||
| solution. This first migration step provides the identifier format and | ||
| maintenance tooling, but does not assign or consume UIDs in library generation | ||
| or at runtime. | ||
|
|
||
| ## Identifier format | ||
|
|
||
| ```text | ||
| | 40 bits: milliseconds since 2026-01-01 UTC | 24 bits: OS randomness | | ||
| ``` | ||
|
|
||
| UIDs are generated by | ||
| `tensilelite/Tensile/Common/SolutionIdGen.py`. Internally, a UID remains an | ||
| unsigned 64-bit integer. Human-readable files and command-line output use an | ||
| unpadded base62 representation: | ||
|
|
||
| - the frozen alphabet is `0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz`; | ||
| - writers use the `0u` prefix; readers also accept `0U`; | ||
| - leading zero digits are forbidden, except that zero is written as `0u0`; | ||
| - values larger than `2^64 - 1` are rejected. | ||
|
|
||
| For example, decimal UID `200` is represented as `0u3E`. A UID is stored as | ||
| the optional, unquoted `SolutionUID` field immediately after the local | ||
| `SolutionIndex` in dict-format logic YAML: | ||
|
|
||
| ```yaml | ||
| - SolutionIndex: 0 | ||
| SolutionUID: 0u3E | ||
| ``` | ||
|
|
||
| YAML readers decode the text immediately and all identity, comparison, and | ||
| mapping operations use the unsigned 64-bit value. | ||
|
|
||
| ## Regenerate a UID | ||
|
|
||
| To replace one solution's UID after a duplicate is reported: | ||
|
|
||
| ```bash | ||
| cd projects/hipblaslt/tensilelite | ||
| python3 -m Tensile.TensileGenerateUID path/to/logic.yaml --index 5 --inplace | ||
| ``` | ||
|
|
||
| `--index` is the solution's local `SolutionIndex` in that YAML file. Without | ||
| `--inplace`, the command prints the new canonical `0u` UID without modifying | ||
| the file. | ||
|
|
||
| Repository-wide assignment, runtime selection, and mandatory uniqueness are | ||
| introduced by later migration steps. |
192 changes: 192 additions & 0 deletions
192
projects/hipblaslt/tensilelite/Tensile/Common/SolutionIdGen.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,192 @@ | ||
| # Copyright Advanced Micro Devices, Inc., or its affiliates. | ||
| # SPDX-License-Identifier: MIT | ||
|
|
||
| """Generate stable 64-bit solution IDs for hipBLASLt.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import secrets | ||
| import time | ||
|
|
||
| """ | ||
| Solution ID is a 64-bit integer that is used to uniquely identify a solution. | ||
| It is a combination of the current time in milliseconds since the SolutionUID | ||
| epoch and an OS-backed random 24-bit integer. | ||
|
|
||
| | 40 bits | 24 bits | | ||
| |---------|---------| | ||
| | Time | Random | | ||
| """ | ||
|
|
||
| EPOCH_MS = 1_767_225_600_000 # 2026-01-01 00:00:00 UTC | ||
| MS_MASK = (1 << 40) - 1 | ||
| RANDOM_BITS = 24 | ||
| RANDOM_MASK = (1 << RANDOM_BITS) - 1 | ||
| RANDOM_SHIFT = RANDOM_BITS | ||
| BASE62_ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" | ||
| BASE62_LOOKUP = {character: index for index, character in enumerate(BASE62_ALPHABET)} | ||
| BASE62_PREFIX = "0u" | ||
| UINT64_MAX = (1 << 64) - 1 | ||
|
|
||
|
|
||
| def encode_solution_uid(solution_uid: int) -> str: | ||
| """Encode a 64-bit solution UID in canonical ``0u`` base62 form. | ||
|
|
||
| Args: | ||
| solution_uid: Unsigned 64-bit solution identifier. | ||
|
|
||
| Returns: | ||
| Canonical unpadded base62 representation prefixed with ``0u``. | ||
|
|
||
| Raises: | ||
| ValueError: If ``solution_uid`` is outside the unsigned 64-bit range. | ||
| """ | ||
| if solution_uid < 0 or solution_uid > UINT64_MAX: | ||
| raise ValueError( | ||
| f"SolutionUID ({solution_uid}) is out of range for uint64 " | ||
| f"(0..{UINT64_MAX})" | ||
| ) | ||
|
|
||
| if solution_uid == 0: | ||
| return f"{BASE62_PREFIX}0" | ||
|
|
||
| digits = [] | ||
| value = solution_uid | ||
| while value: | ||
| value, remainder = divmod(value, len(BASE62_ALPHABET)) | ||
| digits.append(BASE62_ALPHABET[remainder]) | ||
| return BASE62_PREFIX + "".join(reversed(digits)) | ||
|
|
||
|
|
||
| def decode_solution_uid(encoded_uid: str) -> int: | ||
| """Decode a canonical ``0u`` or ``0U`` base62 solution UID. | ||
|
|
||
| Args: | ||
| encoded_uid: Canonical prefixed, unpadded base62 representation. | ||
|
|
||
| Returns: | ||
| Unsigned 64-bit solution identifier. | ||
|
|
||
| Raises: | ||
| TypeError: If ``encoded_uid`` is not a string. | ||
| ValueError: If the representation is malformed, non-canonical, or | ||
| outside the unsigned 64-bit range. | ||
| """ | ||
| if not isinstance(encoded_uid, str): | ||
| raise TypeError("SolutionUID representation must be a string") | ||
| if len(encoded_uid) < 3 or encoded_uid[:2] not in {"0u", "0U"}: | ||
| raise ValueError("SolutionUID must start with 0u or 0U") | ||
|
|
||
| payload = encoded_uid[2:] | ||
| if len(payload) > 1 and payload[0] == "0": | ||
| raise ValueError("SolutionUID base62 payload must not contain leading zeros") | ||
|
|
||
| value = 0 | ||
| for character in payload: | ||
| digit = BASE62_LOOKUP.get(character) | ||
| if digit is None: | ||
| raise ValueError(f"Invalid SolutionUID base62 character: {character!r}") | ||
| if value > (UINT64_MAX - digit) // len(BASE62_ALPHABET): | ||
| raise ValueError("SolutionUID is out of range for uint64") | ||
| value = value * len(BASE62_ALPHABET) + digit | ||
| return value | ||
|
|
||
|
|
||
| def _ms_since_epoch(now_ms: int) -> int: | ||
| """Convert Unix milliseconds to ms since :data:`EPOCH_MS`. | ||
|
|
||
| Args: | ||
| now_ms: Unix timestamp in milliseconds. | ||
|
|
||
| Returns: | ||
| Milliseconds since the configured epoch. | ||
|
|
||
| Raises: | ||
| ValueError: If the value does not fit in 40 bits. | ||
| """ | ||
| ms = now_ms - EPOCH_MS | ||
| if ms < 0 or ms > MS_MASK: | ||
| raise ValueError( | ||
| f"ms since epoch ({ms}) is out of range for 40 bits (0..{MS_MASK})" | ||
| ) | ||
| return ms | ||
|
|
||
|
|
||
| def generate_solution_id(*, now_ms: int | None = None) -> int: | ||
|
dsaffars marked this conversation as resolved.
Resolved
|
||
| """Generate a 64-bit solution ID. | ||
|
|
||
| Layout: ``(ms_since_epoch << 24) | random24`` where ``ms_since_epoch`` uses | ||
| 40 bits and ``random24`` uses ``secrets.randbits(24)``. | ||
|
|
||
| Args: | ||
| now_ms: Optional fixed Unix timestamp in milliseconds for testing. | ||
| When omitted, ``time.time()`` is used. | ||
|
|
||
| Returns: | ||
| A 64-bit unsigned solution identifier. | ||
|
|
||
| Raises: | ||
| ValueError: If the ms-since-epoch value does not fit in 40 bits. | ||
| """ | ||
| if now_ms is None: | ||
| now_ms = int(time.time() * 1000) | ||
|
|
||
| ms = _ms_since_epoch(now_ms) | ||
| random24 = secrets.randbits(RANDOM_BITS) | ||
| return (ms << RANDOM_SHIFT) | random24 | ||
|
|
||
|
|
||
| def decode_solution_id(solution_id: int) -> tuple[int, int]: | ||
| """Split a solution ID into its timestamp and random components. | ||
|
|
||
| Args: | ||
| solution_id: 64-bit solution identifier from :func:`generate_solution_id`. | ||
|
|
||
| Returns: | ||
| Tuple of ``(ms_since_epoch, random24)``. | ||
| """ | ||
| return solution_id >> RANDOM_SHIFT, solution_id & RANDOM_MASK | ||
|
|
||
|
|
||
| def read_solution_uid(solution: dict) -> int: | ||
| """Read a solution UID from YAML state, generating one if absent. | ||
|
|
||
| Args: | ||
| solution: Solution dictionary from logic YAML. | ||
|
|
||
| Returns: | ||
| 64-bit solution UID. | ||
| """ | ||
| if "SolutionUID" in solution: | ||
| return decode_solution_uid(solution["SolutionUID"]) | ||
| return generate_solution_id() | ||
|
|
||
|
|
||
| def ensure_solution_uid(solution: dict) -> None: | ||
| """Assign ``SolutionUID`` when the solution has no UID field yet. | ||
|
|
||
| Args: | ||
| solution: Solution dictionary to update in place. | ||
|
|
||
| Raises: | ||
| ValueError: Propagated from :func:`generate_solution_id` on time overflow. | ||
| """ | ||
| if "SolutionUID" not in solution: | ||
| solution["SolutionUID"] = encode_solution_uid(generate_solution_id()) | ||
|
|
||
|
|
||
| def regenerate_solution_uid(solution: dict) -> int: | ||
| """Replace ``SolutionUID`` with a newly generated value. | ||
|
|
||
| Args: | ||
| solution: Solution dictionary to update in place. | ||
|
|
||
| Returns: | ||
| The newly assigned 64-bit solution UID. | ||
|
|
||
| Raises: | ||
| ValueError: Propagated from :func:`generate_solution_id` on time overflow. | ||
| """ | ||
| uid = generate_solution_id() | ||
| solution["SolutionUID"] = encode_solution_uid(uid) | ||
| return uid | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please use smaller runners for these tests (or move them into an existing workflow instead of adding a new workflow). 96 core machines are expensive and in short supply.
https://github.com/ROCm/rocm-libraries/actions/runs/37805607146/job/113409341093 took 3 minutes total for just 7 seconds of tests. 2m53 was just overhead checking out the repository.
cc @kailash-khalasi