Issue: New way on lemonade cli to select which Rocm architecture to use

[GabrielReusRodriguez]

Installing lemonade on ubuntu, I saw that when you have iGPU and eGPU and you install the llamacpp:rocm backend , it always installs the gfx1150 version of therock. I tryed with HSA_OVERRIDE_GFX_VERSION environment var but it was impossible. I think it is because lemonade loops on devices and uses first GPU found, usually the igpu.

I added a new option on lemonade cli that lists all the rocm architectures avaible (gfxXXXX) with lemonade rocm-arch and so you can set your prefered archituecture with lemonade rocm-arch set gfx1201 .

You can also add "auto" or empty string to execute as always.

I tested on my machines and I think it works! I think it could be usefull .

Examples:

you can also set "auto"

It is a config.json var so you can also set via lemonade config set instruction.

:)

[superm1]

As a general comment; I think this actually is a concept that should be up-leveled. We should allow "GPU" selection for individual devices. For example if I have two 9700's they're an identical arch, how else would I pick between them?

[superm1]

there is already very similar code elsewhere, can you please instead split that to a helper?

[GabrielReusRodriguez]

Hello,
I see that there is some code in system_info.cpp file that does the same:

My proposal would be to create 2 helper functions on system_info.cpp:

std::string transform_isakfd_to_gfx(const std::string& isa) {
if (!isa.empty() &&
std::all_of(isa.begin(), isa.end(), ::isdigit)) {
int v;
try {
v = std::stoi(isa);
} catch (const std::exception& e) {
throw std::runtime_error(
"Failed to parse gfx_target_version '" + isa + "': " + e.what());
}
int major = v / 10000;
int minor = (v / 100) % 100;
int step = v % 100;

    char buf[16];
    std::snprintf(buf, sizeof(buf), "gfx%d%x%x", major, minor, step);
    return std::string(buf);
} else {
    return "";
}

}

and

std::string get_gfx_from_device_name(const std::string device_name) {
std::string device_lower = device_name;
std::transform(device_lower.begin(), device_lower.end(), device_lower.begin(), ::tolower);

std::smatch gfx_match;
// Match 3- or 4-digit gfx tokens; the trailing nibble can be hex (e.g. gfx90a).
if (std::regex_search(device_lower, gfx_match, std::regex(R"((gfx[0-9a-f]{3,4}))"))) {
    return gfx_match[1].str();
}
return "";

}

So I can reuse that functions and apply in the original std::string identify_rocm_arch_from_name(const std::string& device_name)
.

Are the function names correct? is the driver version of rocm all numbers an isa ? example : 120001

I think it is better to do wrapper funtions in ROCmArchUtils calling the previous ones to avoid the #include "lemon/system_info.h" in lemonade_client.cpp.

I try it... :)

[superm1]

If you stick to ROCm ISA selection play with function names and see what feels appropriate. But yes we're always detecting ISA from the kfd sysfs.

Make sure you look at my other comment about upleveling this though.

[superm1]

missing newline

[GabrielReusRodriguez]

Change done

[superm1]

missing newline

[GabrielReusRodriguez]

Change done

[GabrielReusRodriguez]

As a general comment; I think this actually is a concept that should be up-leveled. We should allow "GPU" selection for individual devices. For example if I have two 9700's they're an identical arch, how else would I pick between them?

That's a good challenge :). I just started getting into the lemonade code but I'll see if I can think of anything that might be useful.