Discussion: Adding support for new features to MSIX manifests

[wjk]

Discussion: Adding support for new features to MSIX manifests

Currently, only Microsoft can add new features to the MSIX manifest schema, through extending the code and data inside AppxPackaging.dll. However, some users have asked for MSIX to be able to install more esoteric payloads, such as MMC snapins or Windows credential providers. Currently, the only way one could try to do this is to use customInstallActions and pack an MSI into the MSIX package. While this would work (assuming App Installer permits installation of apps that declare that rescap, which I am unsure of), IMO it goes against the design of MSIX. Furthermore, waiting for Microsoft to add new features into Windows core goes against the spirit of Project Reunion. 😄

One way new features in MSIX could be implemented in a backwards-compatible way would be to to invent a namespace not natively supported by Windows, and put custom features into that. Then users would add its prefix into IgnorableNamespaces so that Windows doesn’t complain about not recognizing it. Finally, the app would bundle a small component run using customInstallActions that parses the manifest file and processes the elements during the installation process. This small component should be extensible, so that third parties can add support for their specific needs by writing a plugin DLL, instead of having to fork and vendor the entire tool.

I have a good enough grasp of this process to start on a preliminary implementation in my own repo, but I don’t yet have enough detail for a formal Feature Proposal just yet.

[jonwis]

Check out https://github.com/microsoft/msix-packaging - that's a great repo in which to try out some of these ideas before integrating into the overall product package manager.

What would you want to see in a Reunion-style package manager?

Your observation is correct - MSIX ensures that no (or very little, tightly controlled) app code is run during install and uninstall. Those are tricky times and it's challenging to write code that runs correctly on half-staged / broken-needs-repair / being-uninstalled, along with the testing on all the various variants of systems.

MSI's custom actions ended up being very attractive and people built entire "sub installers" based on them.

What would be some rules for the kinds of code that apps can run during install & uninstall?

[wjk]

I’m not proposing that I add arbitrary code to run during install/uninstall; the use of customInstallActions is simply a necessary reality given how today’s system MSIX support is written. All the logic would be put in the AppxManifest.xml file, just as we do today for MS-supported installation features. I actually really like the design of MSIX as it is currently designed, including the declarative installation system; I just know that IT admins have been saying they want to make it able to install more kinds of software than it can today. (There was a post in the MSIX techcommunity outlining this issue, but I cannot seem to find it right now. I do remember it specified MMC and credential providers specifically, which is why I mentioned those in the OP.)

I understand that msix-packaging includes support for packing and unpacking MSIX files; it also includes MsixCore for use on downlevel OSes. However, MsixCore does not work on Windows 10 (it is programmed to always delegate to the OS if running on a recent enough build), so adding features there would seem to be a non-starter. How do you suggest I work on this, beyond simply opening design issues there? If that means forking MsixCore to make it work on Windows 10 and then support a superset of what AppxPackaging does, that’s fine by me. Thanks!

[aclinick-zz]

as the pm who put custom actions in MSI i can tell you it didn't work out how the 29 year old me expected :)

IT admins can run scripts with MSIX using the Package Support Framework (PSF) https://github.com/microsoft/MSIX-PackageSupportFramework/tree/master/samples/PSFScriptingSample allows you to run a Powershell onFirstRun, Run and exit

I would look at the PSFbut also feel free to provide feedback/code to MSIX Core so that we can determine how best to undock MSIX for you to use with Reunion

[stevewri]

What are the next steps here? @wjk - were you planning to try out some ideas with the packaging repo @jonwis pointed to and submit a formal proposal, or work with MSIX Core per @aclinick's suggestion?

[wjk]

@stevewri Unfortunately, I don't quite understand what you’re asking. As I said above, the MSIX Core repo doesn't have the ability to change the OS implementation of the MSIX packager AFAIK. The package-support framework won’t work for my needs either. While I can run code on first-run to modify the system as needed, there is no way to ensure that the changes are then cleaned up on uninstall, as PSF only supports code running when the application exits. Dangling references to package files will still be a problem, as described in #97.

However, the implementation I described in the OP probably won’t work, as I remember reading somewhere that the customInstallActions rescap is protected by the system. Only apps specially blessed by Microsoft can successfully use it; random sideloaded packages will not install if they declare this capability. If I am mistaken here, I’d be glad to hear it. Thanks!

[jonwis]

You're correct, customInstallActions requires a specific capability and approval as it explicitly breaks the "MSIX install/uninstall are programmatically driven" proposition.

The packaging system can trigger a background task when your app has been updated. Maybe we could define another background task - "you're being uninstalled" - that would let you do your cleanup? The main issue we have is preventing unbounded/uncontrolled work in servicing operations - Contoso shouldn't be able to camp in their background task for 20m and use that time to beg the user to cancel/reinstall.

For now, can you enumerate the features you'd want to see? The packaging team (hi @jvintzel and @aclinick !) monitors these threads and we're always looking for ways to make MSIX more useful, even if we can't deliver that as part of Project Reunion directly (yet.) :)

[wjk]

@jonwis An on-uninstallation background task would work quite well here. The specific scenarios I am interested in are as follows:

• Unregister ETW manifests (Question: How best to handle ETW (un)registration under MSIX? #97). This may be better handled via a new, dedicated AppX manifest element rather than programmatic install/removal, but I don't know how difficult that is/how long it would take to implement. (Please see also Remove/deprecate serverpreview AppX elements? MicrosoftDocs/winrt-related#201, as while the serverpreview namespace has supported this task since it was introduced, Windows 10 did not appear to honor <serverpreview:EventProvider> elements in the AppX manifest when I tested it.)
• Remove licensing information from shared user cache (cf Add per-machine storage support to MSIX #13), which may involve sending a "deactivation" signal to a license server so I can decrement the used-license count and allow installation elsewhere.

In terms of preventing abuse of this feature, I would recommend that on-uninstall background tasks be strictly forbidden to show UI of any kind to the user. (Enforcing this requirement when the task is running in full-trust, as would be required for ETW support, would be difficult.) To prevent apps from simply stalling indefinitely in their uninstall routines without showing UI, I would recommend that Windows display a dialog box stating "Please wait while Contoso is uninstalling..." if the background task doesn't return within x seconds. I don't think this dialog should allow for the uninstall process to be interrupted, as that may leave the system in a bad state that is difficult to recover from.

Finally, given that I am planning on storing licensing information in the Shared Local folder, I am wondering if it would be possible to add an element in the AppX manifest that would instruct Windows to automatically enable the Shared Local policy during install, obviating the need for the workaround described in #13 (comment). Thanks!

[DrusTheAxe]

@wjk An on-uninstallation background task would work quite well here
Do you have additional thoughts about such an uninstall task?

Execute it before uninstall?

• what if it takes a while or hangs? Should it be time boxed? How big a box? Terminated after timeout and proceed with the uninstall?
• What if it crashes (or otherwise 'unexpected prematurely terminates')? Just one bite at the apple or retry N times? Can it fail and block/stop the uninstall?

Execute it after uninstall?

• should it be time boxed?
• what if the package is reinstalled while the uninstall task is running?
• where do we find the bits if we uninstalled the package? Do we need to delay removing the disk files until the uninstall task completes?
• does the uninstall task need to be an EXE linked with /SWAPRUN so it can be run just before we delete the files w/o being impacted (or impacting) those deletes?

No-UI is a good policy though it can be challenging to enforce, and there's plenty of non-UI potentially lengthy operations e.g. network operations.

Asynchronous is ideal but hard to see how Windows could be entirely fire-and-forget. Maybe fire-and-be-patient? -but-not-too-patient :P

Best-effort vs must-succeed/can stop the uninstall? The latter poses obvious concerns. What would happen in your scenario if the uninstall task failed to complete? Under what conditions could you craft it to be OK if it doesn't complete successfully? How much latitude would you need to ensure it did its important work?

if it would be possible to add an element in the AppX manifest that would instruct Windows to automatically enable the Shared Local policy during install
An interesting idea. Or something equivalent e.g. the policy only applies to packages without <custom:Capability Name="sharedLocalFolderRegardlessOfPolicy">. +@jvintzel for follow up

[jonwis]

There is no current UX for MSIX uninstall and we should not add it - at least not this way. The only visible behavior to the user is either Remove-AppxPackage take a while (there's lots of files and content to delete, processes to tear down) or the start menu item lingers around a while longer. So no, app UX during uninstall should be prevented; I suspect our friends in lifecycle management can help with that.

The uninstall task should happen before final unregistration, as the task might need content from the Desktop Bridge registry & filesystem redirection in order to complete its work. I suggest it be boxed, and we work with our friends in lifetime management to effectively enforce this boxing. A failure to run the task is ignored and the uninstall proceeds. An app should not be allowed to hang around by crashing in its uninstall handler.

[stevewri]

@jonwis @DrusTheAxe @wjk - nothing new here for several months. Let me try to sum up where we are so we can convert this into a proposal; let me know if I've missed anything:

1. We should support an on-uninstallation background task triggered by the packaging system (@jvintzel @aclinick)
2. Need help from lifecycle team (@andreww-msft) to timebox the task and to prevent it from creating UI
3. I'm unclear whether we need a similar on-installation background task as well?

[DrusTheAxe]

@stevewri

1. We should support an on-uninstallation background task triggered by the packaging system (@jvintzel @aclinick)
2. Need help from lifecycle team (@andreww-msft) to timebox the task and to prevent it from creating UI
3. I'm unclear whether we need a similar on-installation background task as well?

#3 it's come up though unclear the priority - and restrictions.

We currently have ` but that's only available to preinstalled (and inbox) packages, it's not a general "on install" trigger. The windows.updateTask only fires on package update e.g. when registering foo-v2 when you have foo-v1 registered. It doesn't fire if you register foo-v2 when you have no foo package registered.

A generic windows.installTask poses questions regarding login performance. Is the perf impact and potential for abuse not the concern they once were? Can use be (reasonably) limited via custom capabilities?

[wjk]

@DrusTheAxe Your message is slightly garbled. As to perf impact, can't these be run asynchronously in the background and not block login? If the user tries to open an app that hasn't completed its on-install code yet, the user will get a progress dialog until then.

Also, since opening this issue I have found alternatives to on-install/uninstall tasks that serve the same function without needing inbox support. This issue is now personally almost zero priority. Thanks nevertheless!

[DrusTheAxe]

@wjk asynchronously - yes but you still have the ThunderingHerd(TM) problem. Login is a very busy period of time so we continually strive to shrink cpu/io/memory hit at login. The trick is to do so w/o limiting functionality. A tough act to juggle.