Skip to content
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/scripts/component_ci.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
- geko=true/false
- miopen=true/false
- tensilelite_coverage=true/false
- hipblaslt_library_uniqueness=true/false

Each component defines a set of path patterns. If any changed file matches,
that component is marked as triggered.
Expand Down Expand Up @@ -44,6 +45,13 @@
"projects/hipblaslt/tensilelite/**",
".github/workflows/component-ci-tensilelite-coverage.yml",
],
"hipblaslt_library_uniqueness": [
"projects/hipblaslt/library/src/amd_detail/rocblaslt/src/Tensile/Logic/**",
"projects/hipblaslt/tensilelite/Tensile/Tests/unit/"
"test_solution_uid_uniqueness.py",
"projects/hipblaslt/tensilelite/tox.ini",
".github/workflows/component-ci-library-uniqueness.yml",
],
}

# Paths that affect every component's CI run: the workflow, this script,
Expand Down
37 changes: 37 additions & 0 deletions .github/workflows/component-ci-library-uniqueness.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Copyright (c) Advanced Micro Devices, Inc., or its affiliates.
# SPDX-License-Identifier: MIT

name: "Component CI: Library Uniqueness"

on:
workflow_call:
inputs:
docker_image:
type: string
required: true
linux_runner:
type: string
required: true

permissions:
contents: read

jobs:
linux:
name: Linux
runs-on: ${{ inputs.linux_runner }}
container:
image: ${{ inputs.docker_image }}
defaults:
run:
shell: bash
working-directory: projects/hipblaslt/tensilelite
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

- name: Install tox
run: python -m pip install tox

- name: Run Library Uniqueness gate
run: python -m tox -e library-uniqueness
11 changes: 11 additions & 0 deletions .github/workflows/component-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ jobs:
geko: ${{ steps.configure.outputs.geko }}
miopen: ${{ steps.configure.outputs.miopen }}
tensilelite-coverage: ${{ steps.configure.outputs.tensilelite_coverage }}
library-uniqueness: ${{ steps.configure.outputs.hipblaslt_library_uniqueness }}
docker-image: ${{ steps.env.outputs.docker-image }}
linux-runner: ${{ steps.env.outputs.linux-runner }}
windows-runner: ${{ steps.env.outputs.windows-runner }}
Expand Down Expand Up @@ -88,6 +89,15 @@ jobs:
docker_image: ${{ needs.changes.outputs.docker-image }}
linux_runner: ${{ needs.changes.outputs.linux-runner }}

library-uniqueness:
name: "hipBLASLt library uniqueness"
needs: changes
if: needs.changes.outputs.library-uniqueness == 'true'
uses: ./.github/workflows/component-ci-library-uniqueness.yml
with:
docker_image: ${{ needs.changes.outputs.docker-image }}
linux_runner: ${{ needs.changes.outputs.linux-runner }}
Comment on lines +92 to +99

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use smaller runners for these tests (or move them into an existing workflow instead of adding a new workflow). 96 core machines are expensive and in short supply.

https://github.com/ROCm/rocm-libraries/actions/runs/37805607146/job/113409341093 took 3 minutes total for just 7 seconds of tests. 2m53 was just overhead checking out the repository.

cc @kailash-khalasi


component-ci-summary:
name: "Component CI Summary"
if: always()
Expand All @@ -97,6 +107,7 @@ jobs:
- geko
- miopen
- tensilelite-coverage
- library-uniqueness
runs-on: ubuntu-24.04
steps:
- name: Checkout TheRock repository
Expand Down
50 changes: 50 additions & 0 deletions projects/hipblaslt/docs/solution-uid.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Solution UID foundation

`SolutionUID` is a stable 64-bit identifier for a hipBLASLt/TensileLite
solution. This first migration step provides the identifier format and
maintenance tooling, but does not assign or consume UIDs in library generation
or at runtime.

## Identifier format

```text
| 40 bits: milliseconds since 2026-01-01 UTC | 24 bits: OS randomness |
```

UIDs are generated by
`tensilelite/Tensile/Common/SolutionIdGen.py`. Internally, a UID remains an
unsigned 64-bit integer. Human-readable files and command-line output use an
unpadded base62 representation:

- the frozen alphabet is `0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz`;
- writers use the `0u` prefix; readers also accept `0U`;
- leading zero digits are forbidden, except that zero is written as `0u0`;
- values larger than `2^64 - 1` are rejected.

For example, decimal UID `200` is represented as `0u3E`. A UID is stored as
the optional, unquoted `SolutionUID` field immediately after the local
`SolutionIndex` in dict-format logic YAML:

```yaml
- SolutionIndex: 0
SolutionUID: 0u3E
```

YAML readers decode the text immediately and all identity, comparison, and
mapping operations use the unsigned 64-bit value.

## Regenerate a UID

To replace one solution's UID after a duplicate is reported:

```bash
cd projects/hipblaslt/tensilelite
python3 -m Tensile.TensileGenerateUID path/to/logic.yaml --index 5 --inplace
```

`--index` is the solution's local `SolutionIndex` in that YAML file. Without
`--inplace`, the command prints the new canonical `0u` UID without modifying
the file.

Repository-wide assignment, runtime selection, and mandatory uniqueness are
introduced by later migration steps.
192 changes: 192 additions & 0 deletions projects/hipblaslt/tensilelite/Tensile/Common/SolutionIdGen.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
# Copyright Advanced Micro Devices, Inc., or its affiliates.
# SPDX-License-Identifier: MIT

"""Generate stable 64-bit solution IDs for hipBLASLt."""

from __future__ import annotations

import secrets
import time

"""
Solution ID is a 64-bit integer that is used to uniquely identify a solution.
It is a combination of the current time in milliseconds since the SolutionUID
epoch and an OS-backed random 24-bit integer.

| 40 bits | 24 bits |
|---------|---------|
| Time | Random |
"""

EPOCH_MS = 1_767_225_600_000 # 2026-01-01 00:00:00 UTC
MS_MASK = (1 << 40) - 1
RANDOM_BITS = 24
RANDOM_MASK = (1 << RANDOM_BITS) - 1
RANDOM_SHIFT = RANDOM_BITS
BASE62_ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
BASE62_LOOKUP = {character: index for index, character in enumerate(BASE62_ALPHABET)}
BASE62_PREFIX = "0u"
UINT64_MAX = (1 << 64) - 1


def encode_solution_uid(solution_uid: int) -> str:
"""Encode a 64-bit solution UID in canonical ``0u`` base62 form.

Args:
solution_uid: Unsigned 64-bit solution identifier.

Returns:
Canonical unpadded base62 representation prefixed with ``0u``.

Raises:
ValueError: If ``solution_uid`` is outside the unsigned 64-bit range.
"""
if solution_uid < 0 or solution_uid > UINT64_MAX:
raise ValueError(
f"SolutionUID ({solution_uid}) is out of range for uint64 "
f"(0..{UINT64_MAX})"
)

if solution_uid == 0:
return f"{BASE62_PREFIX}0"

digits = []
value = solution_uid
while value:
value, remainder = divmod(value, len(BASE62_ALPHABET))
digits.append(BASE62_ALPHABET[remainder])
return BASE62_PREFIX + "".join(reversed(digits))


def decode_solution_uid(encoded_uid: str) -> int:
"""Decode a canonical ``0u`` or ``0U`` base62 solution UID.

Args:
encoded_uid: Canonical prefixed, unpadded base62 representation.

Returns:
Unsigned 64-bit solution identifier.

Raises:
TypeError: If ``encoded_uid`` is not a string.
ValueError: If the representation is malformed, non-canonical, or
outside the unsigned 64-bit range.
"""
if not isinstance(encoded_uid, str):
raise TypeError("SolutionUID representation must be a string")
if len(encoded_uid) < 3 or encoded_uid[:2] not in {"0u", "0U"}:
raise ValueError("SolutionUID must start with 0u or 0U")

payload = encoded_uid[2:]
if len(payload) > 1 and payload[0] == "0":
raise ValueError("SolutionUID base62 payload must not contain leading zeros")

value = 0
for character in payload:
digit = BASE62_LOOKUP.get(character)
if digit is None:
raise ValueError(f"Invalid SolutionUID base62 character: {character!r}")
if value > (UINT64_MAX - digit) // len(BASE62_ALPHABET):
raise ValueError("SolutionUID is out of range for uint64")
value = value * len(BASE62_ALPHABET) + digit
return value


def _ms_since_epoch(now_ms: int) -> int:
"""Convert Unix milliseconds to ms since :data:`EPOCH_MS`.

Args:
now_ms: Unix timestamp in milliseconds.

Returns:
Milliseconds since the configured epoch.

Raises:
ValueError: If the value does not fit in 40 bits.
"""
ms = now_ms - EPOCH_MS
if ms < 0 or ms > MS_MASK:
raise ValueError(
f"ms since epoch ({ms}) is out of range for 40 bits (0..{MS_MASK})"
)
return ms


def generate_solution_id(*, now_ms: int | None = None) -> int:
Comment thread
dsaffars marked this conversation as resolved.
Resolved
"""Generate a 64-bit solution ID.

Layout: ``(ms_since_epoch << 24) | random24`` where ``ms_since_epoch`` uses
40 bits and ``random24`` uses ``secrets.randbits(24)``.

Args:
now_ms: Optional fixed Unix timestamp in milliseconds for testing.
When omitted, ``time.time()`` is used.

Returns:
A 64-bit unsigned solution identifier.

Raises:
ValueError: If the ms-since-epoch value does not fit in 40 bits.
"""
if now_ms is None:
now_ms = int(time.time() * 1000)

ms = _ms_since_epoch(now_ms)
random24 = secrets.randbits(RANDOM_BITS)
return (ms << RANDOM_SHIFT) | random24


def decode_solution_id(solution_id: int) -> tuple[int, int]:
"""Split a solution ID into its timestamp and random components.

Args:
solution_id: 64-bit solution identifier from :func:`generate_solution_id`.

Returns:
Tuple of ``(ms_since_epoch, random24)``.
"""
return solution_id >> RANDOM_SHIFT, solution_id & RANDOM_MASK


def read_solution_uid(solution: dict) -> int:
"""Read a solution UID from YAML state, generating one if absent.

Args:
solution: Solution dictionary from logic YAML.

Returns:
64-bit solution UID.
"""
if "SolutionUID" in solution:
return decode_solution_uid(solution["SolutionUID"])
return generate_solution_id()


def ensure_solution_uid(solution: dict) -> None:
"""Assign ``SolutionUID`` when the solution has no UID field yet.

Args:
solution: Solution dictionary to update in place.

Raises:
ValueError: Propagated from :func:`generate_solution_id` on time overflow.
"""
if "SolutionUID" not in solution:
solution["SolutionUID"] = encode_solution_uid(generate_solution_id())


def regenerate_solution_uid(solution: dict) -> int:
"""Replace ``SolutionUID`` with a newly generated value.

Args:
solution: Solution dictionary to update in place.

Returns:
The newly assigned 64-bit solution UID.

Raises:
ValueError: Propagated from :func:`generate_solution_id` on time overflow.
"""
uid = generate_solution_id()
solution["SolutionUID"] = encode_solution_uid(uid)
return uid
23 changes: 15 additions & 8 deletions projects/hipblaslt/tensilelite/Tensile/LibraryIO.py
Original file line number Diff line number Diff line change
Expand Up @@ -604,9 +604,11 @@ def prepareLibraryLogicDict(data: dict[str, Any]) -> None:
def reorderSolutionsParams(data: Dict[str, Any]) -> None:
"""Reorder solution dict keys after list-to-dict conversion.

Moves ``SolutionIndex``, ``KernelNameMin``, and ``SolutionNameMin`` to the
top of each entry in ``data["Solutions"]``. Used when migrating legacy
list-format logic to dict format and after :func:`reorderSolutionDictForDictMerge`.
Moves ``SolutionIndex``, ``SolutionUID``, ``KernelNameMin``, and
``SolutionNameMin`` to the top of each entry in ``data["Solutions"]``.
``SolutionUID`` is dropped from that lead list when the solution has no
such field. Used when migrating legacy list-format logic to dict format
and after :func:`reorderSolutionDictForDictMerge`.

Args:
data: Dict-format library logic data (mutated in place). Must contain
Expand All @@ -618,16 +620,19 @@ def reorderSolutionsParams(data: Dict[str, Any]) -> None:
Raises:
None.
"""
keys = ["SolutionIndex", "KernelNameMin", "SolutionNameMin"]
sols = data.get("Solutions")
if not sols:
return
for solIdx in range(len(sols)):
sol = sols[solIdx]
keys = ["SolutionIndex", "SolutionUID", "KernelNameMin", "SolutionNameMin"]
if "SolutionUID" not in sol:
keys.remove("SolutionUID")
vals: Dict[str, Any] = {}
for key in keys:
if key in sols[solIdx]:
vals[key] = sols[solIdx].pop(key)
sols[solIdx] = {**vals, **sols[solIdx]}
if key in sol:
vals[key] = sol.pop(key)
sols[solIdx] = {**vals, **sol}


def reorderSolutionDictForDictMerge(state: Dict[str, Any]) -> Dict[str, Any]:
Expand Down Expand Up @@ -971,7 +976,9 @@ def createLibraryLogic(

Emits the root mapping written for YAML (``ProblemType``, ``Solutions``,
``ExactLogic``, etc.) including sorted ``ProblemType`` keys and merge-aligned
solution key layout. ``DefaultSolution`` is a sorted snapshot of
solution key layout (``SolutionIndex``, ``SolutionUID``,
``KernelNameMin``, ``SolutionNameMin``, then remaining keys).
``DefaultSolution`` is a sorted snapshot of
``defaultSolution`` at write time; any solution field equal to that snapshot
is omitted from per-solution dicts so defaults live only under
``DefaultSolution``.
Expand Down
Loading
Loading