[RFC] Remove support for unmanaged store locations

[msfjarvis]

Summary

This RFC aims to eliminate the functionality from APS that allows usage of password stores from device and/or external storage.

Motivation

Android's storage access story has progressively moved towards a privacy-first mindset over the years, with newly introduced restrictions preventing free access to public storage for apps without upfront intimation to users and only through the Storage Access Framework. For APS, this introduces a multi-fold challenge. The app has internally never distinguished between hidden and public storage repositories in its code, and always uses java.io.File which is no longer allowed for public storage. This introduces massive implementation complexities that we've attempted to address multiple¹ times² with very little success.

Google has taken a hard stance on enforcing that these privacy improvements have real world impact, and enforces that app updates submitted to the Google Play Store target at least API 30 (Android 11) which takes away the escape hatch we've used so far to avoid the lockdown on storage access. This has effectively disabled our ability to make releases to Google Play, which means the bulk of nearly 18 months of development since October 2020 has not reached most of our users.

While we do have a maintainer team of 4 people, I have been the most consistent contributor over the nearly 3 years I've worked on the project. As noted in this discussion, as of late I am the only person driving development on the project and I need to admit that there are limits to my ability and availability. External storage is a massive endeavour which like the tickets closed as part of the cleanup, offers me no personal value and thus I have absolutely no motivation to spend effort on it. I am pushing this RFC forth as admission of selfishness: I do not wish for Password Store's future to be held up by a singular feature.

Drawbacks

The removal will prevent users who utilise this feature from being upgrade to newer versions of Password Store. Since being a FOSS app has so far meant that I can not attempt to gather any analytics on how the app is used, lest I be made an example of, I can only make an educated guess that the percentage of users affected by this is diminishingly low. I am willing to accept that tradeoff.

Rationale and alternatives

I've spent over 6 months on and off trying to tackle this problem and have come up short each time. If we do not make the decision now to get rid of this roadblock, we'll simply never be able to make a release again. I consider F-Droid a secondary mirror, so the option of only ever releasing to their platform is not something I can accept.

Unresolved questions

• How to prevent existing Play Store users from being upgraded? Maybe we can offer a prompt post-upgrade if they were using external storage and link them to the 1.13.5 release on GitHub?

[alanthehat]

Hi, I see your problem, thanks for explaining.

Google seem to have gone mad with security for Android 11 & it seems to be causing many developers problems, some have resorted to abandoning Play Store till the situation resolves, it looks (to a non-developer like me) to be a failure (in Google's documentation) to explain how to adapt.

There is a way through this though I don't know it, some apps are managing to break through whatever Google has done, please don't give up.

I haven't yet spotted an OpenSource app that's figured it out so I can't point you to source code but maybe closed-source developers might be friendly enough to give you pointers?

CubeACR (a call recording app which also has other, now resolved, issues with API30) is using /external/Documents/[appname]/ rather than /sdcard/[appname]/ or /storage/[whatever]/[appname]/ I have no idea if this will help.

Lonely Cat Software's X-plore file manager is working now.

Also, I was wondering if you could just hand off the problem completely? X-plore, ES File Explorer, the built-in file manager & several others seem to have an intent or other mechanism whereby they accept a data stream & ask the user where to store it.
I don't know if they feed back the user's selection (which you could store) or accept a pre-loaded intent which you could fill for the next save & for syncing.

[msfjarvis]

Hi, I see your problem, thanks for explaining.

Google seem to have gone mad with security for Android 11 & it seems to be causing many developers problems, some have resorted to abandoning Play Store till the situation resolves, it looks (to a non-developer like me) to be a failure (in Google's documentation) to explain how to adapt.

There is a way through this though I don't know it, some apps are managing to break through whatever Google has done, please don't give up.

I haven't yet spotted an OpenSource app that's figured it out so I can't point you to source code but maybe closed-source developers might be friendly enough to give you pointers?

I've seen open source apps that have dealt with this. It's not really a matter of "I do not know how to do this", more so "I do not know how to do this for Password Store".

CubeACR (a call recording app which also has other, now resolved, issues with API30) is using /external/Documents/[appname]/ rather than /sdcard/[appname]/ or /storage/[whatever]/[appname]/ I have no idea if this will help.

It's fairly straightforward to implement one-off exports (which I have done in the past), but it does not benefit Password Store since we require persistent access to a selected directory.

Lonely Cat Software's X-plore file manager is working now.

Certain classes of apps as described here are allowed to continue having free access to the filesystem, file managers fall into that group.

Also, I was wondering if you could just hand off the problem completely? X-plore, ES File Explorer, the built-in file manager & several others seem to have an intent or other mechanism whereby they accept a data stream & ask the user where to store it. I don't know if they feed back the user's selection (which you could store) or accept a pre-loaded intent which you could fill for the next save & for syncing.

Unfortunately that will not work, because the app still needs the ability to list and read files.

[msfjarvis]

Adding another note onto the main RFC: similar to the issue tracker cleanup, removing support now does not mean I am completely closing the door on this functionality.

Thanks to help from the community we've been able to reach a mutually beneficial middle ground back when I had to remove OTP support for similar reasons, and I'd be willing to do it again and assist any developer who wishes to implement this functionality for their use-case.

[alanthehat]

Certain classes of apps as described here are allowed to continue having free access to the filesystem, file managers fall into that group.

Silicon Valley Californian I often see as a foreign language & much is lost in translation to English but even so, in Google's list they present several categories which I think you could argue are a reasonable fit for APS.

• File managers
• Backup and restore apps -- this one
• Anti-virus apps
• Document management apps -- this one
• On-device file search
• Disk and file encryption -- this one
• Device-to-device data migration

1. APS does backups -- of passwords
2. The password files are documents which need managing
3. These documents are encrypted files
4. You could possibly also argue "data migration" because the whole purpose of this exercise is to allow the data to be "migrated" (via 3rd party apps such as Syncthing).

Though I expect you've already had this discussion with Google, I'm just expressing my frustration.

[alanthehat]

CubeACR (a call recording app which also has other, now resolved, issues with API30) is using /external/Documents/[appname]/ rather than /sdcard/[appname]/ or /storage/[whatever]/[appname]/ I have no idea if this will help.

It's fairly straightforward to implement one-off exports (which I have done in the past), but it does not benefit Password Store since we require persistent access to a selected directory.

This definitely looks plausible if you can get Password Store to use it, Guardian Project's Proofmode seems to be doing the same thing, /external/Documents/ seems to be shared space accessible to other apps as /sdcard/Documents/ so it looks like Syncthing et al would work as expected & /external/Documents/AndroidPasswordStore/ would have persistent access for Password Store.
Proofmode offers no way of changing the storage location so it's possible this is the only place Guardian Project has found to be working.

[msfjarvis]

RFC accepted, implementation is being tracked at #1862

[benrhughes]

I'm not particularly across Android storage terminology. Does this mean that there will no longer be the option to choose the repo location at all? If so, won't that seriously impact anyone using 3rd party sync options like syncthing?

[msfjarvis]

I'm not particularly across Android storage terminology. Does this mean that there will no longer be the option to choose the repo location at all? If so, won't that seriously impact anyone using 3rd party sync options like syncthing?

That is correct and it will, but I prefer offering a feature set I can actually support versus never making a release again. Users who rely on Syncthing can continue using the old versions that will stay available on GitHub Releases and F-Droid.

[benrhughes]

That is correct and it will, but I prefer offering a feature set I can actually support versus never making a release again. Users who rely on Syncthing can continue using the old versions that will stay available on GitHub Releases and F-Droid.

That's a real shame. One of the most attractive features of Android was its relative openness and flexibility.