fs: remove experimental language from rmdir recursive

[bcoe]

I would like to make a case for removing the experimental flag from rmdir's recursive option:

1. the feature landed over a year ago.
2. the functionality is quite useful to tooling authors (we use it in Node.js' own test suite, see: test: work around ENOTEMPTY when cleaning tmp dir #30849),
3. users are hesitant to use the recursive option, due to it being experimental.
4. I feel that this feature aligns closely with the value of "Developer Experience", described in the technical values document @mhdawson's is working on.

Regarding #34278, "Rethink recursive flag".

As a compromise, I initially looked at raising an ENOENT exception, if the initial path provided to rmdir did not exist ... I was thinking perhaps we could get away with deviating a bit from rimraf's behavior, in the name of reaching consensus.

This immediately broke our test test suite, because we rely on the current recursive behavior in our tmpdir.js helper ... It also drilled home for me that deviating from rimraf might not be a great idea...

The current implementation of rmdir + recursive provides behavior the community is accustomed to, due to its similarity to rimraf.

An alternative compromise

What if we explicitly call out in documentation the fact that setting recursive to true gives you behavior that matches the community module rimraf?

It might be the case that, due to some of the oddities in this API surface, a module like fs-extra avoids setting the recursive option. But, as our own use case in tmdir.js demonstrates, the recursive option, as it exists today, is great for a variety of tooling needs.

CC: @CxRes, @nodejs/tooling, @RyanZim, @iansu

Refs: #34278

Checklist

• make -j4 test (UNIX), or vcbuild test (Windows) passes
• tests and/or benchmarks are included
• documentation is changed or added
• commit message follows commit guidelines

[cjihrig]

The Promise-based rmdir() should be updated as well.

[cjihrig]

I think it would be OK to drop the reference to rimraf and just explain this function's behavior.

[bcoe]

My concern here is that rimraf implies a lot of behavior, perhaps we can work together and figure out the most important behavioral things to call out? I chose two things that stood out for me, from what @RyanZim and @CxRes mentioned.

[cjihrig]

My personal point of view: I don't think I've ever directly used the rimraf module in my own code, so referencing it here would force me to go read about implied behavior of an npm module in order to understand how core works. That feels a little backward.

I don't feel strongly about it though. If you think it's important to note, then it's fine with me.

[bcoe]

@cjihrig I looked at the rimraf docs, they point out that rimraf has behavior similar to rm -rf. Why don't we use this language instead of calling out the module name?

[cjihrig]

That sounds good to me 👍

[bcoe]

Playing a bit with GitHub's search, it's also clear that quite a few folks have been using the recursive option.

This perhaps speaks to the fact that we should have had a warning when the recursive setting was added? It feels like, going forward, we should do this for experimental options?

[cjihrig]

It feels like, going forward, we should do this for experimental options?

I believe that is already the policy, it was just missed here.

[Trott]

+1 to going stable with it.

Maybe the ENOENT behavior can be made available with an option? (Not in this PR, though.)

[bcoe]

Maybe the ENOENT behavior can be made available with an option? (Not in this PR, though.)

@Trott I really like this idea, perhaps we introduce an option like recursiveStrict and work with the fs-extra to make the behavior closer to what folks would expect building a library on top of it.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/33138/

[mcollina]

lgtm

[CxRes]

Although I can sympathize with the intent of this PR, it seems like a band-aid to cover up what is already an oversight of design. The problem of going ahead with just the language change is that deeper issues that have been raised, will in all likelihood remain unaddressed, which I think is the worst possible outcome.

At least, I would urge the community to make a good faith attempt to find solutions from which we might find the compromise. I, for example, would not mind the introduction of new flags as a compromise, as suggested above (though my preference would be to deprecate the recursive flag but not removing it for the foreseeable future).

Also, I agree that the documentation should refer to rm -rf and not rimraf.

@bcoe It seems you are trying to argue your case both ways:

users are hesitant to use the recursive option, due to it being experimental.

and

Playing a bit with GitHub's search, it's also clear that quite a few folks have been using the recursive option.

I point this out, not to criticize, but to bring to attention to the deeper questions on how new features are to be onboarded in Node! These are more social issues such as what should be honored - documentation or popularity. Should we change experimental feature to achieve a more consistent API or will it be the case that experimental features will not be changed once they become reasonably popular (or when warnings have not been added)? This question is above my pay grade (though I hold strong opinions); what is certain is that either answer will lead to considerable developer pain.

[boneskull]

• To be clear: I am -1 on deprecating any part of the recursive flag in any manner
• The current behavior is useful and what most consumers want.
• While it works for most, the recursive flag does not work for all, as noted
• Removing the "experimental" status is not mutually exclusive with adding an option for changing ENOENT behavior

[mhdawson]

LGTM

[RyanZim]

Removing the "experimental" status is not mutually exclusive with adding an option for changing ENOENT behavior

@boneskull to be clear, are you flatly opposed to changing the way that recursive works by default in any way, including fixing the fact that it deletes both files and directories?

[boneskull]

It should not remove a file if given a file (unsure if that's still a bug). But it should remove all files within a directory if given a directory.

[bcoe]

@bcoe It seems you are trying to argue your case both ways:

There are two different groups of people 😛

• many users are using the functionality as it is implemented, and we would break them with significant changes to the API.
• the fact that the feature is experimental limits another large group of users from adopting it.

At least, I would urge the community to make a good faith attempt to find solutions from which we might find the compromise. I, for example, would not mind the introduction of new flags as a compromise, as suggested above (though my preference would be to deprecate the recursive flag but not removing it for the foreseeable future).

I am an advocate for an additional flag (perhaps recursiveStrict?). We could work with you to make sure is closer to the expectations from a POSIX perspective.

I am also -1 on deprecating the existing feature. Wearing the hat of someone writing small command line tools, behavior like rm -rf is frequently what I want.

I point this out, not to criticize, but to bring to attention to the deeper questions on how new features are to be onboarded in Node!

I care deeply about listening to people and finding great compromises whenever possible. I apologize that this wasn't how I came across in my initial reaction to #34278.

I think what's difficult here is that we have two significantly different perspectives:

• quite a few folks in this thread are quite happy with behavior like rm -rf, they like the magic one liner.
• you bring an alternate perspective, that you would rather the functionality is less magical, leaving more error handling and configuration to the user (but allowing tools built on the functionality to surface more nuance to the user).

[Trott]

@nodejs/fs

[RyanZim]

It should not remove a file if given a file (unsure if that's still a bug). But it should remove all files within a directory if given a directory.

As of Node v14.10.1, fs.rmdir(file, {recursive: true}) still deletes the file with no error. This should be fixed before this is considered stable.

As far as the ENOENT stuff goes, it's still a bit odd, but if it's clearly documented (which it currently is), I'm not going to make a big fight about it. In a perfect world, recursive would be its own method, but unfortunately, it looks like we're past that point.

[bcoe]

@RyanZim

As of Node v14.10.1, fs.rmdir(file, {recursive: true}) still deletes the file with no error. This should be fixed before this is considered stable.

My concern about changing this behavior is just that it moves us away from the behavior of rm -rf, and makes the feature slightly harder to describe/puts a bit more burden on the user.

I'm personally -0 on this change; but would be supportive of a consensus in either direction.

One thought, it might be good to look at some of the other platforms that have added rmdir recursive:

• .NET: https://docs.microsoft.com/en-us/dotnet/api/system.io.directory.delete?view=netcore-3.1:
  • my colleagues believe system.io.directory.delete also throws if given a file path (have not confirmed).
• Deno: https://doc.deno.land/builtin/stable#Deno.remove
  • throws on unfound directory Uncaught NotFound: No such file or directory.
  • deletes both directories and files.
• Python's shutil.rmtree throws on file: NotADirectoryError: [Errno 20]
• Ruby's FileUtils.remove_dir follows the behavior of rm -rf, it does not throw.

I'd be interested to see what decision these other platforms have taken, when given a file path rather than directory.

[RyanZim]

My concern about changing this behavior is just that it moves us away from the behavior of rm -rf, and makes the feature slightly harder to describe/puts a bit more burden on the user.

It does move away from rm -rf, but this is recursive rmdir, not rm. I don't see how this really makes it harder to describe, it's doing exactly what it says, recursively removing a directory, with a note that it won't throw if the path doesn't exist.

[bcoe]

@RyanZim having done a bit of a review of other languages, I've honestly come around to either wanting us to stick with the very liberal implementation (how things sit today).

Or moving wholesale to a stricter implementation where both non directory paths and missing paths throw. I couldn't find a precedent that throws on a non-directory, but allows a missing path (in the languages I looked at).

[CxRes]

Let me propose a part of my original solution as alternative that might satisfy both the groups of users and cause minimum disruption (but at the cost of some redundancy):

Lets leave rmdir -r as it is (To @bcoe's point) except that it will throw when deleting a file only (To @RyanZim's point) and maybe on a missing path (I wouldn't bother too much either way due to what comes next). Lets then have another function rm (or you might want to call it something else) with a strict implementation matching (at least similar to) POSIX rm that gives users more fine-grained control.

[andersk]

This missed the Stability: 1 line under fsPromises.rmdir. Fix submitted as #36131.