[Question] Google ML Kit Packages

[vhugogarcia]

Hello @AdamEssenmacher ,

While reviewing the latest code from the repository, I noticed that you also updated the packages to the new Google ML Kit by using the new namespace: AdamE.MLKit.iOS.BarcodeScanning

I'm wondering if there is a way I can access an alpha version of the ML Kit NuGet package using the latest SDK version https://developers.google.com/ml-kit/vision/barcode-scanning/ios from Google. I would be happy to help to test them out.

Thanks in advance,
Victor

[AdamEssenmacher]

@vhugogarcia the old Xamarin.* namespaces were just replaced wholesale--that wasn't necessarily an indication that I have any immediate intent to upgrade all of these packages myself--though I'd be happy to support more contributions from the community. It sounds like you're willing to help out here.. so let's talk about MLKit.BarcodeScanning :)

Upgrading any of the packages involves two major tasks. First, we need to update the dependency chain so that the right .xcframeworks make it into the nugets with all the right linker flags set. This can be a relatively straightforward task if it's just a matter of bumping version numbers and updating static library URLs. It gets a little more complicated if dependencies have changed.

The second major task is actually updating the actual binding definitions (i.e. ApiDefinition.cs and Enums.cs). This may or may not actually be necessary. If later versions of a native library removed or renamed APIs, for example, the bindings absolutely need to updated to reflect that. If new APIs were added, the published binding library will be usable--just without the newer APIs until the bindings are added. If the API surface hasn't changed at all, then the bindings shouldn't need to change.

In the case of MLKit.BarcodeScanning, I might have some good news for Task 1. It looks like there hasn't been any major shakeup in the dependency chain, so it should be no big deal for me to bump version numbers and publish some -alpha nugets.

For Task 2 though... The SDK version Microsoft left us with is 1.6. Current is 5.0. Without digging into the native SDK release history, there's no way to tell how much work might be needed here. It could be nothing. If the API surface has changed radically, it could be a lot. Either way, I can't put any effort into it right now, so this work would be up to you.

[vhugogarcia]

I would be happy to help @AdamEssenmacher, yes. If it is possible for you just to bump the numbers and create an alpha package that would be very helpful, so I can jump in and take a look at the Task 2.

I also have a few questions:

• When I try to test locally on a demo app, I keep getting an error when trying to reference the csproj files, I keep getting that the Xamarin.Sdk.Legacy does not exist anymore. So, I'm wondering if it is safe to migrate the project files to the new Microsoft SDK, What do you think?
  From: <Project Sdk="Xamarin.Legacy.Sdk">
  To: <Project Sdk="Microsoft.NET.Sdk">
• Should I take advantage of this and replace the <TargetFrameworks>xamarin.ios10;net6.0-ios</TargetFrameworks> for <TargetFramework>net8.0-ios</TargetFramework>
• How do you get the IDs from the targets of the packages? So, I can know to which Ids to target the ML Kit.

[AdamEssenmacher]

Regarding Xamarin.Legacy.Sdk and net6.0, they will eventually need to go away. However, there are still a number of Xamarin apps in the wild (mine included 😬), so I'd rather continue to support these apps for a while to give devs some more breathing room. MAUI still isn't production-ready.

This said, it sounds like directly referencing the .csproj binding projects from a MAUI app isn't working. If the goal of your demo app is to help you develop binding definitions, I can think of three possible solutions:

1. Use a Xamarin app instead.
2. Don't reference the .csproj directly. Pack the nuget and add it to a local feed. This is clunky, but probably the path of least resistance if you only need to make a few minor changes.
3. Create a branch to migrate to Microsoft.NET.Sdk and net8.0-ios. This is probably the most work, but would be most helpful to this project in the long run. This would let us develop API bindings using the current tooling and merge or cherry pick binding definition updates into or from main. Eventually we can just merge the branch in when we're ready to cut Xamarin support.

[AdamEssenmacher]

Regarding the URLs for the static libraries, they're in the podspec and I left comments linking where to find them.

An easy way to find podspecs is to look on cocoapods.org:

[AdamEssenmacher]

@vhugogarcia I made a branch with updated dependency versions. The nugets build and might 'just work'--give it a try and let me know.

[vhugogarcia]

Hello @AdamEssenmacher, thank you so much! I will take a look.

[vhugogarcia]

Excuse me @AdamEssenmacher , do you have the externals\_SSZipArchive.xcframework near to you, could you zip it and share it so I can download it? if not do not worry, I can try to build and generate it on XCode.

[AdamEssenmacher]

Looking at this again, it seems that Microsoft never actually published these packages, so they were a WiP?

If that's right then this effort isn't a matter of upgrading previously working bindings (which I incorrectly assumed was the case) and instead is a matter of finishing what Microsoft started.

Some of what I saw when I was upgrading the dependencies (i.e. missing dependencies) makes more sense now.

I'm afraid this is probably a little beyond what I'm able to spend time on myself.

[vhugogarcia]

yeah! I was also thinking the same! OK, let me look for alternatives to try to get the MLKit frameworks independently like this repository which posts the XCFrameworks on each Firebase release, so you don't have to run the CAKE Tool to download the externals and compiling and all that stuff: https://github.com/akaffenberger/firebase-ios-sdk-xcframeworks

By the help of that repository, what we also can do is just execute these 3 commands and we will have the XCFrameworks from a specific version in our externals folder:

gh release list -R akaffenberger/firebase-ios-sdk-xcframeworks
gh release download 10.24.0 -R akaffenberger/firebase-ios-sdk-xcframeworks
open *.zip

Thanks @AdamEssenmacher for looking into.

[AdamEssenmacher]

Oooo I'll have to look more into that project!

Microsoft left the original components project in a really poor state. Managing the dependencies is a painful, manual process that could definitely be improved. The Android side was built much better--which I'd guess is a big reason the team hasn't had an issue keeping it maintained.

I haven't put any effort into improving the structure / approach from where Microsoft left it. I was holding out hope that the Microsoft team might reverse course and want to merge this fork back in. Seeing as they finally archived the original project a few weeks ago, I don't think that's going to happen.

So, I guess we're clear to start making improvements where we can!

[bengavin]

@vhugogarcia @AdamEssenmacher - I'd also like to see the MLKit stuff [Barcode Scanning specifically] working here, my initial attempts to update dependencies is continuing to fail when building the actual consuming application with various clang++ errors referencing missing GUL* functions in arm64. Getting things working on an M2 MacBook was a non-starter until I switched over to net8.0-ios, even though I have all the Xamarin/net6/etc stuff locally on this machine.

My current plan was to try to get Google's MLKitVisionSample working locally and then try to back into what's missing that's preventing things from working on this side of things. I will agree that this structure of this repo is breaking my brain :) Really feels like there must be an easier way to maintain this stuff, but I'm at a loss as to what that might be :)

[AdamEssenmacher]

Make sure you clear your externals folder before each time you run dotnet cake --target=externals --names=MLKit.BarcodeScanning

[bengavin]

Good point, it didn't change anything at the moment, but I'll circle back on this later. I'm well outside the timebox I initially started with, so need to switch over to other items for a bit. May circle back and look to take a new branch from the baseline and see if making more targeted changes, knowing what I do now, makes a difference.

[AdamEssenmacher]

If and when you do get back to it, here's the approach I'd recommend.

1. Start from a fresh state
2. Update the MLKit.Core and MLKit.BarcodeScanning dependencies (*.csproj, components.cake, and *.targets files). Leave the dependency on Firebase.Core alone
3. Temporarily strip out implementations in MLKit ApiDefinitions.cs and Enums.cs files
4. Build nuget for MLKit.Barcode scanning artifact (dotnet cake externals, libs, then nuget targets)
5. Get an iOS.NET app successfully building and running that references the generated nugets
   --
   Once you get this far, all that should remain is restoring and fixing the binding definitions.

[bengavin]

@AdamEssenmacher - #25

The above gets things working in the 'BarcodeScanner.Mobile.Maui' project and I'm able to successfully capture a barcode from the camera. The issue I'm currently running into with our application is that the new MLKit iOS components [MLImage specifically] does not support the iOS simulator on M-series processors, which leads to linking errors during the build process. Could be in my re-packaging of that library though, so I'll put this up for PR since it does appear to work on the MAUI side. I have not yet tested the Xamarin side of things though.

[bengavin]

Hrm... nevermind, was referencing some old binding libraries in the main project [from previous versions of the scanner]. It's working there too now.

[AdamEssenmacher]

The intention was not to move anything out of Firebase.Core, but rather to remove MLKit's dependency on it.

Given the way Microsoft originally structured this repo, this shouldn't be done.

Firebase.Core really should have been named something like Google.Core. It serves as a sort of 'dumping ground' for shared dependencies across the entire project (e.g. GTMSessionFetcher).

[bengavin]

I suppose that is true, but it -also- includes a lot of Firebase specific libraries that, in general, MLKit users do not need. I guess I'm confused as to why we need to continue to carry forward the link between the two packages when they are independent now [they were not originally, which may be why things are this way now]. What is the harm in having those dependencies tracked separately, beyond what is likely a need to ensure that when the shared versions change, they are changed for both 'pods' for the case where someone needs both Firebase and MLKit in the same solution?

[AdamEssenmacher]

I guess I'm confused as to why we need to continue to carry forward the link between the two packages when they are independent now

The problem isn't unique to MLKit -> Firebase.Core. Even with the Firebase packages, Firebase.Core contains extra dependencies that may not actually be needed. Again, it's real purpose is just to be a dumping ground for all shared dependences of all Google packages.

This was never a 'good' approach. Ideally, there would be one Nuget package per pod dependency. This is how things are managed on the Android side (which Microsoft is still thankfully maintaining).

My first goal with this fork was to get us to a place where we were on modern native SDK versions incrementally with the minimum required code changes. I had hoped that Microsoft might pull these updates back into their official project and didn't want to go off re-architecting their solution at the same time. Now that it's clear Microsoft is done with the original repo for good (they've archived it), we can consider putting effort in to changing the structure of the project since we 'own' it now.

This sort of improvement would affect all the artifacts in this project and should be done in isolation from other efforts.

[bengavin]

I like the idea of splitting things out into package-per-pod, and yeah, definitely not something you'd likely want to start from this repo structure :)

[bengavin]

@AdamEssenmacher - looks like the latest alpha1 packages published to nuget are missing the .xcframework files. For instance, my local package build is 815KB vs. the 50KB of the current nuget, which contains only the MLKit.Core.dll and none of the resources. Thoughts?

[AdamEssenmacher]

That doesn't sound good.... I'll investigate.

[AdamEssenmacher]

@bengavin fixed it. Rider does this really obnoxious thing where if you delete the externals directory from within rider it sweeps through all the projects and removes references without asking.