expose whisper_lang_id() function

[yuroller]

expose a whisper.cpp function to get the id of a language or to check for an existing language.
I use it in my branch, but if you do not find useful, then reject the pull request.

[sandrohanea]

Hello @yuroller ,
I can merge it, but this does not seem to be exposed to the public consumer of the library (the INativeWhisper is internal by design).

Maybe we can expose it on the WhisperFactory, similar with GetSupportedLanguages(), othewise only those that are using the code directly and not the nuget package can benefit from this addition.

The actual exposed method can be something like:

public static string GetLanguageId(string language)
{
       CheckLibraryLoaded();

        var stackByteCount = Encoding.UTF8.GetByteCount(language);
        var stackBytes = stackalloc byte[stackByteCount + 1];
        var stackLanguage = (IntPtr)stackBytes;
        MarshalUtils.CopyStringToPtr(language, stackLanguage, stackByteCount);
        return LibraryLoaded.Value.NativeWhisper!.Whisper_Lang_Id(stackLanguage);
        
}

What do you think?

[yuroller]

I use it as a part of another modification to the whispernet library in order to restrict the language autodetect to a bunch of languages and not all those supported by whisper: as you rigthly noted, you can only call that function inside the library.
I did not make a pull request for restricted language autodetection, because it changes signatures of public exposed methods, so it will break compatibility.

Basically I changed this signature, adding the array of enabled languages for autodetect:
public unsafe (string? language, float probability) DetectLanguageWithProbability(ReadOnlySpan<float> samples, string[] enabledLanguages)

I then get the id of the enabled languages and return that with highest value in prob array.

[sandrohanea]

Interesting use-case indeed,
We should be able to expose this in the public api as well using some different overload for that DetectLanguageWithProbability method.
Maybe named "DetectLanguageWithProbability(ReadOnlySpan samples, IEnumerable candidateLanguages).

If you want and have time, feel free to add it, otherwise, I'll just merge the current PR as it is and will add the overload myself when I'll have some time.

[yuroller]

If you want and have time, feel free to add it, otherwise, I'll just merge the current PR as it is and will add the overload myself when I'll have some time.

Thanks for merging: one less patch to maintain in my branch :-)
When I touch my code next time, I try to refactor it in order to use the signature you suggested and make a pull request.

[sandrohanea]

If you want and have time, feel free to add it, otherwise, I'll just merge the current PR as it is and will add the overload myself when I'll have some time.

Thanks for merging: one less patch to maintain in my branch :-) When I touch my code next time, I try to refactor it in order to use the signature you suggested and make a pull request.

No worries, thank you for the idea and fixes.
I already implemented the functionality under #474 (feel free to use it and please, let me know If this makes sense)