[hipDNN] Heuristic Policy Framework + Default Plugins (RFC 0007 - Part 2/3)

[cderb]

PR Overview

This PR implements Part 2 of RFC 0007 (Engine Selection Heuristics), building on the plugin infrastructure from Part 1 (#6467). It delivers the policy orchestration framework via EngineHeuristicDescriptor, plus two default plugins (Config and StaticOrdering) that replace the legacy sortEngineIds() behavior.

Key Changes:

• Policy orchestration logic in EngineHeuristicDescriptor::finalize() (iterates over policies until one succeeds)
• SelectionHeuristic wrapper providing typed C++ interface over policy descriptors
• Two default heuristic plugins: Config (placeholder for user preferences) and StaticOrdering (legacy behavior)
• Query APIs for introspecting registered policies (hipdnnGetHeuristicPolicyCount_ext, hipdnnGetHeuristicPolicyInfo_ext)
• Shared EngineOrdering.hpp utility in data_sdk for code reuse

Statistics:

• 29 files changed, 3,488 insertions(+), 55 deletions(-) (relative to PR1)
• 45 new tests across 3 test files (policy framework, plugins, static ordering)
• 2 production plugins shipping in lib/hipdnn_plugins/heuristics/

---

Core Components

1. EngineHeuristicDescriptor - Policy Orchestration

Purpose: Orchestrates multiple heuristic policies, invoking each in priority order until one succeeds.

Implementation (backend/src/descriptors/EngineHeuristicDescriptor.cpp:201-254):

bool success = false;
for (size_t i = 0; i < _policySlots.size(); ++i) {
    auto& selection = _policySlots[i];
    if (selection == nullptr) continue;  // Plugin not loaded

    try {
        selection->setEngineIds(candidates);
        selection->setSerializedGraph(&serializedGraph);

        if (!selection->finalize()) {
            continue;  // Policy declined
        }

        candidates = selection->getSortedEngineIds();
        success = true;
        break;  // First policy wins
    } catch (const HipdnnException& e) {
        continue;  // Policy failed, try next
    }
}

if (!success) {
    throw HipdnnException(..., "No heuristic policy succeeded.");
}

Policy Slot Management:

• _orderedPolicyIds: Policy IDs in priority order (lower priority number = higher precedence)
• _policySlots: Corresponding SelectionHeuristic wrappers (one per policy)
• Default ordering: Config (priority 100) before StaticOrdering (priority 1000)

---

2. SelectionHeuristic - Policy Wrapper

Purpose: C++ facade for one policy slot, wrapping hipdnnHeuristicPolicyDescriptor_t.

Key Methods (backend/src/heuristics/SelectionHeuristic.cpp):

• setEngineIds(): Sets candidate engines on policy descriptor
• setSerializedGraph(): Passes serialized OperationGraph to policy
• finalize(): Invokes policy's logic via hipdnnHeuristicPolicyDescriptorFinalize()
• getSortedEngineIds(): Retrieves result from policy

Lifecycle: Owned by EngineHeuristicDescriptor, created when policy list is established.

---

3. Config Plugin

Purpose: Reserved for user-specified engine preferences (e.g., environment variables).

Current Implementation (plugins/heuristics/config/ConfigPlugin.cpp):

• No-op placeholder: Policy finalize returns false (declines all requests)
• Priority 100: First in default ordering
• Future roadmap: Will implement HIPDNN_PREFERRED_ENGINE env var support

Rationale for shipping empty:

• Validates plugin loading infrastructure
• Reserves priority slot for future implementation

---

4. StaticOrdering Plugin

Purpose: Preserves legacy sortEngineIds() behavior.

Implementation (plugins/heuristics/static_ordering/StaticOrderingPlugin.cpp):

• Direct port of old backend sorting logic
• Engine ordering: MIOPEN_ENGINE first, MIOPEN_ENGINE_DETERMINISTIC last, others middle
• Stable sort: Preserves relative order of engines with same priority
• Priority 1000: Last in default ordering (deterministic fallback)
• Always succeeds: Returns sorted list, finalize() always returns true

Algorithm (delegates to shared EngineOrdering.hpp):

auto getPriority = [](int64_t engineId) -> int {
    if (engineId == MIOPEN_ENGINE_ID) return 0;
    if (engineId == MIOPEN_ENGINE_DETERMINISTIC_ID) return 2;
    return 1; // Other engines
};

---

5. Query APIs

Purpose: Allow users to introspect registered heuristic policies.

New Functions (backend/include/hipdnn_backend.h, backend/src/HipdnnBackend.cpp):

// Get number of registered policies
hipdnnStatus_t hipdnnGetHeuristicPolicyCount_ext(
    hipdnnHandle_t handle,
    int64_t* count);

// Get metadata for specific policy (name, ID, priority)
hipdnnStatus_t hipdnnGetHeuristicPolicyInfo_ext(
    hipdnnHandle_t handle,
    int64_t policyIndex,
    hipdnnHeuristicPolicyInfo_t* policyInfo);

---

Architecture Highlights

Shared EngineOrdering.hpp

What: Header-only sorting logic in data_sdk/include/hipdnn_data_sdk/utilities/EngineOrdering.hpp

Why:

• Both backend and StaticOrdering plugin need sortEngineIds()
• Plugins cannot link against backend (circular dependency)
• Header-only = compile-time inclusion, zero runtime dependencies

Backend Delegation (backend/src/utilities/EngineOrdering.cpp):

void sortEngineIds(std::vector<int64_t>& engineIds) {
    hipdnn_data_sdk::utilities::sortEngineIds(engineIds);
}

Plugin Usage (plugins/heuristics/static_ordering/StaticOrderingPlugin.cpp):

#include <hipdnn_data_sdk/utilities/EngineOrdering.hpp>
// ...
hipdnn_data_sdk::utilities::sortEngineIds(d->sortedEngineIds);

---

Testing & Validation

Test Files Added

TestHeuristicPolicyFramework.cpp (18 tests):

• Multi-policy orchestration and fallback behavior
• Policy slot ordering (priority-based)
• EngineHeuristicDescriptor lifecycle

TestHeuristicPolicyPlugins.cpp (14 tests):

• Config and StaticOrdering plugin loading
• Plugin discovery from lib/hipdnn_plugins/heuristics/
• Policy behavior validation

TestStaticOrderingPolicy.cpp (13 tests):

• Engine ordering permutations (MIOPEN/DETERMINISTIC/OTHER)
• Stable sort validation
• Edge cases (empty lists, single engine)

---

Files Modified

Backend Framework

backend/include/
  └── hipdnn_backend.h                      (+2 query APIs)

backend/src/
  ├── HipdnnBackend.cpp                     (+96: query API impl)
  ├── descriptors/
  │   ├── EngineHeuristicDescriptor.cpp    (+347: policy orchestration)
  │   └── EngineHeuristicDescriptor.hpp    (+27: policy slots)
  ├── heuristics/
  │   ├── SelectionHeuristic.cpp           (+121: policy wrapper)
  │   └── SelectionHeuristic.hpp           (+137: wrapper interface)
  └── utilities/EngineOrdering.cpp         (-36: delegate to data_sdk)

Data SDK

data_sdk/include/hipdnn_data_sdk/
  └── utilities/
      └── EngineOrdering.hpp                (+66: header-only sortEngineIds)

Plugins

plugins/
  ├── CMakeLists.txt                        (new: plugin build root)
  └── heuristics/
      ├── CMakeLists.txt                    (new: heuristics plugin build)
      ├── config/
      │   ├── ConfigPlugin.cpp              (+379: no-op policy)
      │   ├── README.md                     (plugin documentation)
      │   └── CMakeLists.txt                (build config)
      └── static_ordering/
          ├── StaticOrderingPlugin.cpp      (+410: legacy sorting)
          ├── README.md                     (plugin documentation)
          └── CMakeLists.txt                (build config)

Tests

backend/tests/
  ├── TestHeuristicPolicyFramework.cpp      (+316: 18 tests)
  ├── TestHeuristicPolicyPlugins.cpp        (+368: 14 tests)
  └── TestStaticOrderingPolicy.cpp          (+264: 13 tests)

---

Notable Design Decisions

1. Policy Orchestration in Descriptor finalize()

Decision: Implement policy loop in EngineHeuristicDescriptor::finalize() rather than a separate orchestrator class.

Rationale:

• Descriptor already owns policy slots and ordering
• Keeps orchestration logic with state
• Simpler API (no additional orchestrator object)

---

2. Empty Config Plugin Ships in Production

Decision: Ship Config plugin with no-op implementation (finalize returns false).

Rationale:

• Validates plugin loading infrastructure
• Reserves priority slot (100) in default ordering
• Future PR adds env var logic without framework changes

---

3. EngineOrdering.hpp Restoration

Decision: Restore sortEngineIds() as header-only in data_sdk.

Context: Revert commit (6a24652) removed it from Part 1, breaking StaticOrdering plugin compilation.

Rationale:

• Plugins need sorting but cannot link against backend
• Header-only = zero runtime dependencies
• Backend delegates to avoid duplication

---

Follow-Up Work

Part 3: Frontend Integration (Next PR)

Scope: Add frontend C++ APIs for heuristic usage.

Expected Features:

• Graph::getEngineHeuristic() for creating heuristic descriptors
• Policy registration from user code
• High-level engine selection workflows

---

Key Files to Review

1. backend/src/descriptors/EngineHeuristicDescriptor.cpp - Policy orchestration loop
2. backend/src/heuristics/SelectionHeuristic.cpp - Policy wrapper/facade
3. plugins/heuristics/static_ordering/StaticOrderingPlugin.cpp - Legacy behavior port
4. backend/src/HipdnnBackend.cpp - Query API implementation

---

Resolves: RFC 0007 Part 2/3
Depends On: #6467 (Plugin SDK Infrastructure)
Enables: Part 3 frontend integration, custom heuristic plugins

[BrianHarrisonAMD]

Approving, I think its mostly good with a couple minor things to address before merge.

Make sure to do a final update from latest develop, get a passing CI run, and do a manual ASAN test to verify no new leaks exist from this change before merging.

Thanks for all the work on this!