Alternatives to Spotlight

[JScott]

Every time we check installed Xcode versions we use the Spotlight system (mdfind "kMDItemCFBundleIdentifier == 'com.apple.dt.Xcode'"). This creates a dependency (#36), blackbox problems (#203), and fragility (#128). I ran into problems myself with this tool in an automated pipeline and had to introduce an arbitrary wait time to recognize a new installation. It's not trivial to find a replacement but after playing around with it myself I have a couple suggestions that I'd like more opinions on.

Also, don't worry about the ugliness of these examples. The text manipulation can easily be done in Ruby which would look a lot nicer.

---

system_profiler has exactly the information needed but is terrible at displaying it. I ran this for the same results as Spotlight:

system_profiler -detailLevel mini SPApplicationsDataType |
grep -A8 Xcode: |
sed -n "s/.*Location: \(.*\).*/\1/p"

Unfortunately, system_profiler has a custom, human-readable data format. This makes it fragile if you think that Apple will change that format.

---

pkgutil can find installed versions, which isn't exactly the same as what we have with Spotlight. Nonetheless, I could find all formally installed versions with:

pkgutil  --only-dirs --files com.apple.pkg.Xcode |
grep -E "Xcode[\d\.-]*\.app$"

I like system_profiler better but pkgutil is an interesting fallback.

---

Either has potential to be a less problematic replacement for Spotlight. In fact, we could even use all 3 as fallbacks for each other when the others aren't available. What are your thoughts? I didn't want to waste my time making a PR before talking it through.

[skeenan947]

system_profiler also supports output in xml, which ends up actually being a funky plist. Using this would reduce the worry about parsing human-readable data.
This command pulls an array of all apps out, and could be parsed down further with a decent xml parser:

system_profiler -detailLevel full SPApplicationsDataType -xml|plutil -extract '0._items' xml1 -o - -|xmllint --xpath '/plist/array' -

[timsutton]

Using system_profiler's Applications DB is also what Munki uses for a local applications database:

https://github.com/munki/munki/blob/4053a0dbe77d900308260ba8a39ca1df4814d33c/code/client/munkilib/info.py#L442-L464

And today I learned about plutil -extract. Amazing!

[KrauseFx]

Thanks for investigating this, feel free to submit a PR, I can push a new release with this change 👍

[JScott]

Alright, I'll make some time to put something together next week. It seems that system_profiler is a fine replacement.

Is Nokogiri a reasonable dependency? I haven't used it in a while but I remember it caused a lot of trouble with its native extensions during installation. Maybe it's gotten better. I don't strictly need it but it's probably nicer than regex and text manipulation.

[KrauseFx]

Ha, no sorry, we can't introduce nokogiri as a dependency for multiple reasons. One is that it requires Xcode Developer Tools to be installed. Secondly it's still very hard for users to install

[JScott]

Yeah, that all sounds about right. Not a problem, glad I asked :)

[JScott]

According to the Internet, it turns out that system_profiler uses Spotlight under the hood:

• https://discussions.apple.com/thread/6510972?start=0&tstart=0
• https://www.lansweeper.com/kb/159/apple-mac-scanning-requirements.html

pkgutil can work and I don't think is linked to Spotlight but is taking me ~7 seconds to run. It also doesn't remove the need for Spotlight, just shores up some gaps it has before indexing finishes.

I'll submit a PR later for some other unrelated fixes that I found along the way but unless @KrauseFx says otherwise I'm going to guess that 7 seconds for each run of installed_versions is too long. It could be a backup instead of failing for Spotlight but that isn't really solving the problem I set out to fix here.

gg Apple

[Ashton-W]

I've done a quick patch that looks for any Xcode*.app under /Applications/ and checks them too.
Looks a bit like this:

Dir.glob("/Applications/Xcode*.app") do |xcode_path|
            installed_xcode = XcodeInstall::InstalledXcode.new(xcode_path)
            # add to list / select this version
end

It's not perfect, we could validate the bundleID of the app at that path too.

[francois-codes]

@KrauseFx @JScott are you guys still looking into this ?
This is causing failures on CI because the spotlight index is not always up to date when fastlane invokes this method.

I understand options are :

• system_profiler
• pkg_util
• @Ashton-W suggestion ?

[JScott]

Nothing on my end. Everything I know is in here :)