Wrong locale picked for System Language

[timothyqiu]

Pixelorama version: 0.8.1 stable

OS/device including version: Not graphics related

Issue description:
My macOS has its language set to Simplified Chinese. If Pixelorama's language is set to System Language, it's actually Traditional Chinese that's picked.

System Language is achieved like this:

Pixelorama/src/Preferences/HandleLanguages.gd

Lines 76 to 77 in d85efce

	if index == 0:
	TranslationServer.set_locale(OS.get_locale())

As per godotengine/godot#40708, OS.get_locale is not stable across platforms. It returns zh-Hans-CN in my case, and TranslationServer.set_locale treats the string like zh, dropping the "Hans-CN" (Simplified Chinese) part.

Steps to reproduce:
Start Pixelorama on macOS with Simplified Chinese locale.

Or manually try TranslationServer.set_locale("zh-Hans-CN").

[dasimonde]

On linux (PopOS), I get the same behavior.

TranslationServer.set_locale("zh-Hans-CN")
print(TranslationServer.get_locale())

My output is zh

[dasimonde]

In godotengine/godot/blob/master/core/translation.cpp are following localizations for chinese

"zh", //  Chinese
"zh_CN", //  Chinese (China)
"zh_HK", //  Chinese (Hong Kong)
"zh_SG", //  Chinese (Singapore)
"zh_TW", //  Chinese (Taiwan)

[OverloadedOrama]

I suppose this issue will get fixed once godotengine/godot#40708 is merged into Godot, right?

[timothyqiu]

Yes and no I think.

godotengine/godot#40708 is mainly documenting the format of locale returned by OS.get_locale. It may fix a similar issue on Windows (returning zh on Simplified Chinese). But on macOS, the optional -Hans / _Hans part still makes TranslationServer's locale zh.

[dasimonde]

I would say it is a issue with godot.

static const char *locale_list[] = {

	...
	
	"zh", //  Chinese
	"zh_CN", //  Chinese (China)
	"zh_HK", //  Chinese (Hong Kong)
	"zh_SG", //  Chinese (Singapore)
	"zh_TW", //  Chinese (Taiwan)
	
	...
	
	nullptr
};

bool TranslationServer::is_locale_valid(const String &p_locale) {
	const char **ptr = locale_list;

	while (*ptr) {
		if (*ptr == p_locale) {
			return true;
		}
		ptr++;
	}

	return false;
}

String TranslationServer::standardize_locale(const String &p_locale) {
	String univ_locale = p_locale.replace("-", "_");

	int idx = 0;
	while (locale_renames[idx][0] != nullptr) {
		if (locale_renames[idx][0] == univ_locale) {
			univ_locale = locale_renames[idx][1];
			break;
		}
		idx++;
	}

	return univ_locale;
}

String TranslationServer::get_language_code(const String &p_locale) {
	ERR_FAIL_COND_V_MSG(p_locale.length() < 2, p_locale, "Invalid locale '" + p_locale + "'.");

// 3. if p_locale == "zh_hans_CN" then
//     split is 2
	int split = p_locale.find("_");         

// This block is skipped                                                   
	if (split == -1) {
		split = p_locale.find("-");
	}
// This block is skipped
	if (split == -1) {
		return p_locale;
	}

// 4. p_locale.left(split) is "zh"
	return p_locale.left(split);
}

void TranslationServer::set_locale(const String &p_locale) {

 // 1. After standardize_locale, univ_locale is "zh_Hans_CN"
	String univ_locale = standardize_locale(p_locale);

// 2. "zh_Hans_CN" isn't in p_local_list
//     that's why !is_locale_valid(univ_locale) is equal true
	if (!is_locale_valid(univ_locale)) {

		String trimmed_locale = get_language_code(univ_locale);
		print_verbose(vformat("Unsupported locale '%s', falling back to '%s'.", p_locale, trimmed_locale));

		if (!is_locale_valid(trimmed_locale)) {
			ERR_PRINT(vformat("Unsupported locale '%s', falling back to 'en'.", trimmed_locale));
			locale = "en";
		} else {
			locale = trimmed_locale;
		}
	} else {
		locale = univ_locale;
	}

	if (OS::get_singleton()->get_main_loop()) {
		OS::get_singleton()->get_main_loop()->notification(MainLoop::NOTIFICATION_TRANSLATION_CHANGED);
	}

	ResourceLoader::reload_translation_remaps();
}

String TranslationServer::get_locale() const {
	return locale;
}

String String::replace(const String &p_key, const String &p_with) const {
	String new_string;
	int search_from = 0;
	int result = 0;

	while ((result = find(p_key, search_from)) >= 0) {
		new_string += substr(search_from, result - search_from);
		new_string += p_with;
		search_from = result + p_key.length();
	}

	if (search_from == 0) {
		return *this;
	}

	new_string += substr(search_from, length() - search_from);

	return new_string;
}

[Variable-ind]

does anyone still experience this issue?

[OverloadedOrama]

I tested using zh-Hans-CN as the input locale in Pixelorama 1.0 and it seems to be correctly returning zh_CN, so I believe this issue should be fixed now.