Persistent directory access API

[mgiuca]

I'm filing an issue here for something we might want to explore (not necessarily part of Writable Files, but this seemed the best fit). This is spun off from w3c/manifest#/626 discussion.

The model would be similar to the Chrome Apps chrome.fileSystem API, part of the proprietary Chrome apps platform. This lets developers prompt the user to choose a directory, and grant persistent read/write access to that directory and subdirectories to the application.

Obviously this is a huge security red flag, but it could be potentially useful for applications like Git clients, and generally other software that has to access all the files in a user-supplied directory (like maybe the Play Music uploader if given access to your entire Music directory).

We'd have to be careful about:

• Users granting access to their entire drive or maybe entire documents folder, and have platform-specific restrictions on what you can't grant access to. (On Chrome Apps, we prevented top-level drives, system folders like C:\Windows, users' documents folder, etc.)
• Developers asking for this permission when they really only need a sandboxed file system (i.e., don't need access to "real files", just need a place to write and read app-specific files).

It's possible that this permission is so scary that it would only be grantable to "installed" apps (Web App Manifest style).

[mgiuca]

@AshleyScirra and @skddc from the other thread.

@ewilligers for spec.

See Ashley's comment on that thread about Windows.

Hosted web apps in the Windows Store can already do this, and it seems to have passed Microsoft's security review, so it might be a model to consider for the wider web.

This is quite a different security scenario. Web apps hosted in Microsoft's store are given the full security model of a native application. While this is far more constrained than Win32 apps of old, it is still a fundamentally different security model to the web, which is supposed to be safe for drive-by browsing.

Having said that, I think this is something we could explore for the web if we take a very careful approach to security.

[kenchris]

I also think this would be quite useful for something like VSCode, even for Chromebooks now that Chrome OS seems to be getting Linux VMs in form of "Terminal for Chrome OS"

As a user, I would to totally fine with this requiring installation and seeing one huge scary prompt, even if that included something like a captcha for "please ensure that you know what you are doing".

What I mean with that, is that the security prompt can be different from that of the existing ones, as users will most likely know what they are doing especially if this also requires installation (a2hs).

I was on desktop PWAs on Chrome OS that they show in the title bar when they are using geolocation, we might want to do something similar for such file access.

[raucao]

I think it would make sense to allow developers to choose between sandboxed mode (give me a directory for my app) and user files mode (ask user to pick a directory from their home folder).

Note: The former was already possible (and working quite well for us), with FileSystem API in Chrome. Which wasn't implemented in any other web browser, but it is still implemented in Cordova. Security-wise, it's much easier to handle, of course, and you only need to ask users permission to store a certain amount of data, similar to localStorage for example.

[AshleyScirra]

I'd also be fine with this kind of API only being available to installed web apps (e.g. A2HS). In a broader sense, I think an Electron/NW.js style set of APIs available to installed web apps would make it unnecessary to use these wrappers (and bring some nice benefits like not having to ship around entire copies of the Chromium engine with the app).

There might be an opportunity for a hybrid approach: if there's a sandboxed filesystem, but the user can choose which folder that is on their device, then they could do something like create a new folder for a web app, then move any files they want the web app to see there. This provides at least some way for users to integrate with the OS file system, while preventing any wider access. IIRC, the filesystem API that Chrome had didn't directly expose the actual files anywhere that normal users might find them.

[mgiuca]

@AshleyScirra:

if there's a sandboxed filesystem, but the user can choose which folder that is on their device

How is that different from an unsandboxed file system (that happens to be empty at the time it is chosen)? Are you merely proposing that the file system API be limited to choosing (or creating) an empty directory to begin with? I'd say that has a few major downsides and doesn't really increase security. I'd like to focus on the unsandboxed file system.

It's a shame that FileSystem API went away, but let's just focus on unsandboxed file system access for now. Perhaps it works out that we bring back the FileSystem API (and it's perhaps more developer-friendly now that we have promises and async/await).

[inexorabletash]

+1 to sorting out persistent access. (And I can't believe this isn't captured well here anyway beyond the jumping-to-the-solution #7 and #6). Four persistent access scenarios we link to think about are:

• Photo editor - I want to grant it persistent access to my Photos collection.
• Git / IDE - I want to open a specific project directory
• File > Recent menu - if I open MY.DOC in a cloud based editor today, I should still be able to edit it after a reload, or when I come back next week
• Open/Re-Save - if a document editor site lets you save a file, it should let you keep editing and re-save even if you hit reload in the middle. Bonus points if you can open and then re-save. (The web today really only has Save As)

(This issue is about directories, but does splitting it make sense? maybe?)

There are challenging permission issues to work out here, since the permission grants would be very subtle. The one that I find terrifying is the "evil template" site that appears to let you download e.g. a resume template, but is actually saving a file with persistent access. If you visit the site again in the future, does the site get to read the contents of the file? A naive solution is that if the file is changed by something other than the web site (e.g. native editor, other site) the permission is lost. But that impacts collaboration scenarios, and the persistent directory access could be used to mimic it (e.g. the site create seemingly innocently creates a "my templates" folder but then gets to snoop on the contents).

[mgiuca]

Thanks. #6 looks like almost the same as this issue but I'll keep these separate because here I'm asking for persistent directory access, not just one-time.

[AshleyScirra]

How is that different from an unsandboxed file system

I imagined an unsandboxed file system would allow you access to any part of the file system, if the user grants permission. If there is essentially a sandboxed file system, but limited to one folder of the user's choosing, then it could cover some use cases while ruling out security problems like gaining access to the user's entire documents folder.

[inexorabletash]

If there is essentially a sandboxed file system, but limited to one folder of the user's choosing...

Ah, terminology confusion. We usually use "sandboxed" within web platform storage discussions to mean storage scoped to an origin, like Indexed DB or Cache API (or legacy attempts like WebSQL, FileSystem, etc), entirely managed by the user agent.

Access to a subset of the local file system will need a different term to disambiguate. "Scoped" or some such.

[kenchris]

Scoped is what Android uses as terminology: https://developer.android.com/training/articles/scoped-directory-access.html

[AshleyScirra]

Yeah, that's probably a less confusing term :) Yes, if the browser creates a folder for the user somewhere, and all file system access is scoped to just that folder, then that gives you a basic level of file system interop without ever letting web apps see anything else on the file system. Existing files can only be exposed if the user explicitly moves files to that folder.

[mkruisselbrink]

I think most of the proposals in this thread are addressed by the current proposal, except perhaps the scoped directory access thing. I'm not quite sure what use cases would be for that though, that aren't already solved by the various other forms of filesystem access?

[inexorabletash]

I think scoped access is implicit - we don't allow navigation "up" past the chosen folder, do we?

[mkruisselbrink]

I think scoped access is implicit - we don't allow navigation "up" past the chosen folder, do we?

Sure, although that doesn't seem to quite match what the previous comment described as scoped access? I.e. one particular folder on the filesystem created by the browser for this particular origin (and only this origin) to access. Requiring the user to move files into/out of that folder to achieve interop with other apps.

[inexorabletash]

Ah, thanks for clarifying.

[mgiuca]

The current proposal doesn't mention persistence at all (for either files or directories).

I'm not quite sure what use cases would be for that though, that aren't already solved by the various other forms of filesystem access?

A Git explorer or other application you want to grant ongoing access to a directory. You don't want to have to manually choose the directory every time you open up the app.

I know it's "scary" to grant this, so we'll have to think very carefully about permissions and UI. But I think there are definitely use cases for it.

[mkruisselbrink]

The current proposal does mention that you can store references in IndexedDB. It doesn't currently mention how that would work with permissions and UI etc (although UI shouldn't be in the spec anyway), but it is certainly the intention that you can use that for persistent access.

[mgiuca]

OK I missed that. In that case, assuming both files and directories can be stored in IndexedDB and retrieved later, I think that closes this feature request.

I guess this ("The browser can choose when to allow or not allow this open.") leaves it open to the browser whether to add increased permissions or security requirements around this. (e.g., we could make it that only installed apps can use persistent access, or that a prompt is shown to re-open a file from IndexedDB.) Correct?

[jordanaustin]

The current proposal does mention that you can store references in IndexedDB

Just so I can make sure I understand this correctly, storing the references in IndexedDB would allow you to return to the app at a later point and use that reference to continue to either read or write back to the previously chosen file or directory?

[kenchris]

I guess we could even set a time limit on how long you have access, so if you return to access a directory you haven't in 3 months, then it could show a simplified dialog just reconfirming access

[inexorabletash]

I forked the "scoped" suggestion into a separate issue, since it seems different than the "persistence" discussion here. #21

[mkruisselbrink]

Just so I can make sure I understand this correctly, storing the references in IndexedDB would allow you to return to the app at a later point and use that reference to continue to either read or write back to the previously chosen file or directory?

Yep, that's the idea. Of course figuring out how to do this in a way that doesn't violate the users expectations is going to be a big challenge, but hopefully we can design the API in such a way to give web browsers lots of freedom of coming up with sensible UI and permission treatments.

[bitnom]

Persistence would be so awesome. Imagine having a permanent (Until revoked) handle to a remote user's storage. Developers could cook up a nice file syncing solution, quickly. It could be done similar to how we currently have Push notifications. I'm not sure exactly how the browser handles push. Will look into it more. This issue intersects with #9 because if it's too easy to accidentally revoke access, the handle would be lost.

For this reason, I propose that clearing private data does not clear storage API permissions by default. This new storage API should be a very explicit permission with clear consequences prompted to the user; therefore held in separate regard to that of cookies and cache. Of course it would have an entry under clear private data sections. As stated however, simply not checked by default.

[ddumont]

Adding my $0.02:
I've been working on a PWA for an application that uses torrents to distribute (often large) user-created content. The current limits on max quota for other storage solutions are getting in my way for sandboxed storage. This might provide me with the ability to have a user designate a folder I have access to. This would work even better if it could be an off-device share for really large files.

I work with web security fairly often in my day-job... I understand the crazed fanaticism the authors here are employing to be secure.. but you gotta let people make apps that do what the users want.
A user is not going to want to be prompted for each file contained in the torrent when they paste a magnet url or specify a torrent url and expect the torrent to download.

I certainly don't mind having to have the user install the PWA, or jumping through a few hoops to make sure the user knows what's going on... perhaps reminders about filesystem access ?

User access the site after a certain amount of time... and they must respond to a prompt from the browser: Reminder, xyz.com is using 20GB of storage at <link to open in explorer/finder>. Do you want to continue to allow this access? And give them an option to choose stop asking me.

[Eloque]

I would very much be in favor of adding persistence. It's the only thing stopping my current use case from being viable and user friendly.

My use case is an app that monitors a directory on the local disk, recognizes when a file has been added. And based on that, copies that file to another directory. This is done by having the user select a source and a target folder, obtaining full permissions on both of those and then monitoring every 5 seconds if a file is added to source. I've implemented such already and it works perfectly, once started.

There is just one issue, in usability; It is not possible to save permissions between reloads. If the user closes the tab or reloads the page or does any action equivalent to that. Ie, click on an internal link, both permissions and the location of Source and Target are lost. Requiring them to select again. Perhaps a sort of semi-persistence can be achieved, but allowing the handle or another reference to be saved and if the page is reloaded, asking for the permissions in those, lets call them "stale" handles again.

I am trying to circumvent the issue currently by having the app open up a second tab, request all the needed permissions there and having very simple Ajax calls on that tab to communicate to the backend. The main app, the one that the user spends most his time in, is then less likely to result in a reload or such, requiring new permissions.

In the blog by @petele it's stated that only PWA's would be able to be persistent. That would be workable, but I'd prefer not to have that limitation.

[mkruisselbrink]

As of M83 (and the new origin trial) chrome supports the IndexedDB persistence as described here. Currently no special power for installed PWAs, everybody can persist handles in IDB. Details around persistence of permissions are still to be worked out (but imho also out of scope of the spec; how long a permission remains granted for is really up to implementations). Chrome currently revokes all permissions when the last tab for an origin is closed (so when loading a handle from IndexedDB you'll need to re-request permission to read and/or write from/to it).

[EricSimons]

@mkruisselbrink thanks for the update- regarding this line:

so when loading a handle from IndexedDB you'll need to re-request permission to read and/or write from/to it

Is the plan to remove this restriction in M86 after OT finishes? (I hope so :)

[mkruisselbrink]

There is definitely some hope to be able to loosen that restriction. Not entirely clear yet how. Perhaps installed PWAs can retain permissions, or perhaps it'll be time-bound (i.e. we won't re-prompt the user if it's been less than X days since granting permission), or only if for session restore... Unclear at this point what we'll end up doing, but definitely one of the main areas where we hope to explore alternatives in permission behavior.

[connorjclark]

@mkruisselbrink is there a crbug for retaining permissions in PWAs, or is the way forward still unclear?

[tomayac]

crbug/1011533 is the bug to track.

[ddumont]

@tomayac can't see that. Is it private?

[tomayac]

Updated #14 (comment) to point at a bug that's world-visible. Sorry.

[danqdinh]

@mkruisselbrink

everybody can persist handles in IDB

those are the handles only, possible to persist permission after tab close?

[a-sully]

Not yet, but this is coming soon. Updates will be posted to #297