Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doesn't work #25

Closed
Petah opened this issue Jul 9, 2013 · 31 comments
Closed

Doesn't work #25

Petah opened this issue Jul 9, 2013 · 31 comments

Comments

@Petah
Copy link

Petah commented Jul 9, 2013

fs.js:456
  return binding.rmdir(pathModule._makeLong(path));
                 ^
Error: ENOTEMPTY, directory not empty 'c:\work\modules\raptor-gold\build\temp'
    at Object.fs.rmdirSync (fs.js:456:18)
    at Function.rimrafSync [as sync] (c:\work\modules\raptor-gold\node_modules\rimraf\rimraf.js:128:12)
    at Object.build (c:\work\modules\raptor-gold\build\raptor-builder.js:77:12)
    at Object.<anonymous> (c:\work\modules\raptor-gold\build\build.js:5:9)
    at Module._compile (module.js:449:26)
    at Object.Module._extensions..js (module.js:467:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.runMain (module.js:492:10)
    at process.startup.processNextTick.process._tickCallback (node.js:244:9)
@isaacs
Copy link
Owner

isaacs commented Jul 11, 2013

Can you provide more details? There's something in there that it's failing to delete. What files are in there? How are you creating this dir?

@Petah
Copy link
Author

Petah commented Jul 11, 2013

Seems that it was creating files while rimraf was trying to delete at the same time

@mlarcher
Copy link

mlarcher commented Nov 8, 2013

@isaacs: could you please check out the log pasted in the phantomjs ticket and tell me if you need more info in order to investigate this issue ?

@isaacs
Copy link
Owner

isaacs commented Nov 8, 2013

If you're creating files while trying to rimraf the dir, then yes, that will definitely not work. That will also confuse the builtin rm -rf command line utility. Don't do that!

@isaacs
Copy link
Owner

isaacs commented Nov 8, 2013

@mlarcher What's happening in the phantomjs ticket's npm log looks like the phantomjs install script fails (exits non-zero), but is still doing stuff (perhaps because it shelled out to windows unzip, which does not stop writing files immediately when the process exit event happens.)

Do not fetch stuff at install time. Fetch it when you need it.

The correct behavior in this case is for rimraf to surface the error. Rimraf is not, and cannot ever be, an atomic FS operation.

@isaacs isaacs closed this as completed Nov 8, 2013
@Petah
Copy link
Author

Petah commented Nov 9, 2013

While I agree that rimraf should not work in this case maybe a more meaningful error would be beneficial.

@nullivex
Copy link

In case other people are having this issue its most likely a virus scanner scanning the folder where the file operations are happening. This will make the files in use during the deletion.

In this specific case it was caused by ESET Anti-Virus.

SOLUTION: Ignore the following paths
image

Make sure to replace the username with yours obviously. I have a feeling this is why the issue is so intermittent for most people and unconfirmed.

This should also help fix Medium/phantomjs#108

@isaacs
Copy link
Owner

isaacs commented Nov 10, 2013

What error code gets raised with the anti-virus program thwarts rimraf? There's some special handling for EBUSY for exactly that purpose.

@nullivex
Copy link

This is the error code that gets thrown while the anti-virus is scanning the files in the directory. I assume what is happening here is that the phantomjs install is trying to remove the directory while the Anti-Virus still has file descriptors open in the folder. Not sure how you would formulate a workaround for that? Maybe some sort of blocking?

Errorlog:
Phantom installation failed { [Error: ENOTEMPTY, directory not empty 'C:\tmp\phantomjs\phantomjs-1.9.2-windows.zip-extract-1380823312359\phantomjs-1.9.2-windows']
errno: 53,
code: 'ENOTEMPTY',
path: 'C:\tmp\phantomjs\phantomjs-1.9.2-windows.zip-extract-1380823312359\phantomjs-1.9.2-windows',
syscall: 'rmdir' } Error: ENOTEMPTY, directory not empty 'C:\tmp\phantomjs\phantomjs-1.9.2-windows.zip-extract-1380823312359\phantomjs-1.9.2-windows'
at Object.fs.rmdirSync (fs.js:456:18)
at rimrafSync (C:\homedir\Agora\node_modules\phantomjs\node_modules\rimraf\rimraf.js:160:6)
at C:\homedir\Agora\node_modules\phantomjs\node_modules\rimraf\rimraf.js:158:5
at Array.forEach (native)
at rimrafSync (C:\homedir\Agora\node_modules\phantomjs\node_modules\rimraf\rimraf.js:157:21)
at Promise.successFn (C:\homedir\Agora\nodemodules\phantomjs\install.js:291:12)
at Promise.withInput (C:\homedir\Agora\nodemodules\phantomjs\node_modules\kew\kew.js:204:25)
at Promise.resolve (C:\homedir\Agora\node_modules\phantomjs\node_modules\kew\kew.js:92:25)
at resolver (C:\homedir\Agora\node_modules\phantomjs\node_modules\kew\kew.js:264:17)
at cb (C:\homedir\Agora\node_modules\phantomjs\node_modules\ncp\lib\ncp.js:217:38)
npm ERR! [email protected] install: node install.js
npm ERR! cmd "/c" "node install.js" failed with 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is most likely a problem with the phantomjs package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node install.js
npm ERR! You can get their info via:
npm ERR! npm owner ls phantomjs
npm ERR! There is likely additional logging output above.

npm ERR! System Windows_NT 5.1.2600
npm ERR! command "C:\Programme\nodejs\\node.exe" "C:\Programme\nodejs\node_modules\npm\bin\npm-cli.js" "install" "phantomjs"
npm ERR! cwd C:\homedir\Agora
npm ERR! node -v v0.8.9
npm ERR! npm -v 1.1.61
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! C:\homedir\Agora\npm-debug.log
npm ERR! not ok code 0

@isaacs
Copy link
Owner

isaacs commented Nov 10, 2013

ENOTEMPTY would happen when rmdir'ing the directory. That would indicate that either:

  1. A file unlink failed.
  2. A file was written into the directory after doing the readdir step.

In the first case, it would fail with the unlink error, so it's not that.

If the directory was locked (because, as you suggest, a file descriptor was open) then we'd get an EBUSY error, and rimraf would try 3 more times with increasingly slower backoff.

If it IS an EBUSY error, then node isn't exposing it properly, and that's a bug in Node. I notice that you are using a very old version of Node and npm. Please update to v0.10 (or at least, the latest v0.8) and see if that helps the situation.

If there is some other process writing to the dir while we're trying to delete it, then that is a valid error, and rimraf is doing the correct thing by failing in this way.

@nullivex
Copy link

@isaacs

Actually I just grabbed that trace off the other thread. I can try to get this error reproducible we just got around it by doing this last night. So I figured it might help some of the other people that have been having the issue intermittently.

I was going to note that this feels a lot like a race condition which may be why we are not seeing the right error codes. However I have not had a look at any of the underlying code so I am unsure how the process works.

I should mention that when I got the error I am on the latest version of node 0.10.21, I just didnt have a stack trace handy to show you.

@isaacs
Copy link
Owner

isaacs commented Nov 10, 2013

It seems like this error only happens in connection to PhantomJS, is that correct?

I'd look through their build script, and see if it's trying to write to directors it shouldn't, or continuing to do things after exiting with a non-zero status.

@nullivex
Copy link

I dont believe so I have followed a couple other threads into this one not all related to phantom. I am currently spinning up a VM to make this reproduce-able.

The references above for say grunt-contrib-clean seems to be running into it as well with the same intermittent symptoms so I believe they're related.

@nullivex
Copy link

@isaacs Okay, got this in a nice reproducible environment.

Setup

  • Windows 7 64-bit
  • NodeJS v0.10.21 32-bit
  • Python 2.7.5 32-bit
  • Visual Studio Express 2012 32-bit
  • Git 32-bit latest
  • ESET 7 64-bit

Instructions

  • Open command prompt
  • Type npm -g install karma

Output


> [email protected] install C:\Users\Tandy7\AppData\Roaming\npm\node_modules\kar
ma-phantomjs-launcher\node_modules\phantomjs
> node install.js

Downloading http://phantomjs.googlecode.com/files/phantomjs-1.9.2-windows.zip
Saving to \tmp\phantomjs\phantomjs-1.9.2-windows.zip
Receiving...
Received 782K...
Received 1569K...
Received 2354K...
Received 3136K...
Received 3922K...
Received 4705K...
Received 5495K...
Received 6280K...
Received 6937K total.
Extracting zip contents
Renaming extracted folder \tmp\phantomjs\phantomjs-1.9.2-windows.zip-extract-138
4116124440\phantomjs-1.9.2-windows -> C:\Users\Tandy7\AppData\Roaming\npm\node_m
odules\karma-phantomjs-launcher\node_modules\phantomjs\lib\phantom
Phantom installation failed { [Error: ENOTEMPTY, directory not empty 'C:\tmp\pha
ntomjs\phantomjs-1.9.2-windows.zip-extract-1384116124440\phantomjs-1.9.2-windows
']
  errno: 53,
  code: 'ENOTEMPTY',
  path: 'C:\\tmp\\phantomjs\\phantomjs-1.9.2-windows.zip-extract-1384116124440\\
phantomjs-1.9.2-windows',
  syscall: 'rmdir' } Error: ENOTEMPTY, directory not empty 'C:\tmp\phantomjs\pha
ntomjs-1.9.2-windows.zip-extract-1384116124440\phantomjs-1.9.2-windows'
    at Object.fs.rmdirSync (fs.js:612:18)
    at rimrafSync (C:\Users\Tandy7\AppData\Roaming\npm\node_modules\karma-phanto
mjs-launcher\node_modules\phantomjs\node_modules\rimraf\rimraf.js:160:6)
    at C:\Users\Tandy7\AppData\Roaming\npm\node_modules\karma-phantomjs-launcher
\node_modules\phantomjs\node_modules\rimraf\rimraf.js:158:5
    at Array.forEach (native)
    at rimrafSync (C:\Users\Tandy7\AppData\Roaming\npm\node_modules\karma-phanto
mjs-launcher\node_modules\phantomjs\node_modules\rimraf\rimraf.js:157:21)
    at Promise._successFn (C:\Users\Tandy7\AppData\Roaming\npm\node_modules\karm
a-phantomjs-launcher\node_modules\phantomjs\install.js:291:12)
    at Promise._withInput (C:\Users\Tandy7\AppData\Roaming\npm\node_modules\karm
a-phantomjs-launcher\node_modules\phantomjs\node_modules\kew\kew.js:204:25)
    at Promise.resolve (C:\Users\Tandy7\AppData\Roaming\npm\node_modules\karma-p
hantomjs-launcher\node_modules\phantomjs\node_modules\kew\kew.js:92:25)
    at resolver (C:\Users\Tandy7\AppData\Roaming\npm\node_modules\karma-phantomj
s-launcher\node_modules\phantomjs\node_modules\kew\kew.js:264:17)
    at ReadStream.cb (C:\Users\Tandy7\AppData\Roaming\npm\node_modules\karma-pha
ntomjs-launcher\node_modules\phantomjs\node_modules\ncp\lib\ncp.js:217:38)
npm ERR! error rolling back Error: ENOTEMPTY, rmdir 'C:\Users\Tandy7\AppData\Roa
ming\npm\node_modules\karma-phantomjs-launcher\node_modules\phantomjs\lib\phanto
m'
npm ERR! error rolling back  [email protected] { [Error: ENOTEMPTY, rmdir 'C:\Us
ers\Tandy7\AppData\Roaming\npm\node_modules\karma-phantomjs-launcher\node_module
s\phantomjs\lib\phantom']
npm ERR! error rolling back   errno: 53,
npm ERR! error rolling back   code: 'ENOTEMPTY',
npm ERR! error rolling back   path: 'C:\\Users\\Tandy7\\AppData\\Roaming\\npm\\n
ode_modules\\karma-phantomjs-launcher\\node_modules\\phantomjs\\lib\\phantom' }
npm ERR! error rolling back Error: EPERM, unlink 'C:\Users\Tandy7\AppData\Roamin
g\npm\node_modules\karma-phantomjs-launcher\node_modules\phantomjs\lib\phantom\p
hantomjs.exe'
npm ERR! error rolling back  [email protected] { [Error: EPERM, unl
ink 'C:\Users\Tandy7\AppData\Roaming\npm\node_modules\karma-phantomjs-launcher\n
ode_modules\phantomjs\lib\phantom\phantomjs.exe']
npm ERR! error rolling back   errno: 50,
npm ERR! error rolling back   code: 'EPERM',
npm ERR! error rolling back   path: 'C:\\Users\\Tandy7\\AppData\\Roaming\\npm\\n
ode_modules\\karma-phantomjs-launcher\\node_modules\\phantomjs\\lib\\phantom\\ph
antomjs.exe' }
npm ERR! weird error 1
npm ERR! not ok code 0

C:\Users\Tandy7>

This is in a fresh Windows 7 VM and I followed this guide for setting up my environment: https://gist.github.com/nullivex/7115612

If you would like access to the VM message me off list.

@nullivex
Copy link

Here is a screenshot

image

@isaacs
Copy link
Owner

isaacs commented Nov 13, 2013

@nullivex Take a look at that stack trace. The problem is in the karma/phantomjs install script. I'll take a look at this when I get a chance, but my best guess is that it's trying to delete a dir that's currently being written to.

@nullivex
Copy link

Yeah that's what we figured as well. Again I'm not sure where the fix would
go just trying to centralize this problem as a couple issues point to it.

I believe what is happening is that when phantomjs or any other package
comes with an executable the AV starts scanning it which most likely leaves
the directory unclean causing the error.

I'd be happy to try to work out a pull request against phantomjs if we can
narrow down how it's being used incorrectly. Certainly the developers
assumed there wouldn't be anyone else in the directory they were working in.

However since AV programs are really common for windows users it is worth
fixing in my opinion.

On Wednesday, November 13, 2013, Isaac Z. Schlueter wrote:

@nullivex https://github.com/nullivex Take a look at that stack trace.
The problem is in the karma/phantomjs install script. I'll take a look at
this when I get a chance, but my best guess is that it's trying to delete a
dir that's currently being written to.


Reply to this email directly or view it on GitHubhttps://github.com//issues/25#issuecomment-28417969
.

eSited LLC
(701) 390-9638

@jbondc
Copy link

jbondc commented Nov 14, 2013

There's a similar issue here:
https://issues.apache.org/jira/browse/CB-5330

To reproduce:

wget https://github.com/apache/cordova-cli/archive/master.zip
unzip master.zip
cd cordova-cli-master
npm update
npm test

...................................................................................rm: could not remove directory (code ENOTEMPTY): Z:\cordo
va-cli-master\temp/.cordova/hooks/before_build

rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp/.cordova/hooks/before_build
rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp/.cordova/hooks

rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp/.cordova/hooks/before_build
rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp/.cordova/hooks
rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp/.cordova

rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp/.cordova/hooks/before_build
rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp/.cordova/hooks
rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp/.cordova
rm: could not remove directory (code ENOTEMPTY): Z:\cordova-cli-master\temp

.{ [Error: EPERM, operation not permitted 'Z:\cordova-cli-master\temp.cordova\hooks\before_build\fail.bat']
errno: 50,
code: 'EPERM',
path: 'Z:\cordova-cli-master\temp.cordova\hooks\before_build\fail.bat',
syscall: 'stat' }

The tests work fine on linux, I traced it down to code that tries to recursively remove a directory in 'shelljs'.

What happens is:

a) File is deleted (before_build\fail.bat)
https://github.com/arturadib/shelljs/blob/master/src/rm.js#L126

b) Removal of directory fails with ENOTEMPTY
https://github.com/arturadib/shelljs/blob/master/src/rm.js#L141

At the last failure:
https://github.com/arturadib/shelljs/blob/master/src/rm.js#L15

fs.readdirSync(dir) reports the file as existing! so fs.lstatSync() throws [Error: EPERM, operation not permitted

This points to node on Windows either:
a) fs.unlinkSync() is successful but directory entry still exists
b) fs.readdirSync(dir) is using some cached / old entry of directory info?

I think it's more (b) since I tried adding sleep(5) and doesn't change the fact the directory entry still thinks the file exists.

@jbondc
Copy link

jbondc commented Nov 14, 2013

I give up, best next guess is a bug in libuv:
https://github.com/joyent/libuv/blob/master/src/win/fs.c#L742

I'm not sure what FindFirstFileW and FindNextFileW are doing, from what I can tell, the win32 api is FindFirstFile, FindNextFile?

@nullivex
Copy link

@jbondc Do you have any AV installed? Or any other program that actively monitors certain file types like executables (.bat, .exe, etc). I have only seen this occur when someone has an AV that scans executable's.

@jbondc
Copy link

jbondc commented Nov 14, 2013

On Windows 8.1, if I disable:

  • Windows Defender (no spyware, no virus protection)
  • Windows Search

I get the same result. Here's what the order of operations looks like:
http://sdrv.ms/HW49Re

@nullivex
Copy link

Strange everything for me seems to work fine on windows 8 as long as I dont have an AV enabled. And the fact that its happening with a .bat leads me to believe that maybe that bat file hadnt finished executing or something is scanning it not sure.

But overall @isaacs I guess we really need some way to block until all the file handles have free'd up for things to work as expected? Just brainstorming.

@jbondc
Copy link

jbondc commented Nov 15, 2013

I get the same result in Windows 7, really odd.

Maybe some answers here:
http://blogs.msdn.com/b/oldnewthing/archive/2011/12/26/10251026.aspx

But using a FAT filesystem, I get the same result.
Signs point to weird stuff node is doing to make the 'async' appear synchronous.

unlink on UNIX:
https://github.com/joyent/libuv/blob/master/src/unix/fs.c#L921

unlink for windows:
https://github.com/joyent/libuv/blob/master/src/win/fs.c#L1608
https://github.com/joyent/libuv/blob/master/src/win/fs.c#L635

The unlink for windows is more complex, maybe a timing/thread issue there. I'm just guessing.

@jbondc
Copy link

jbondc commented Nov 15, 2013

Maybe relevant for 'FindFileFirst':
"In rare cases, file attribute information on NTFS file systems may not be current at the time you call this function. To be assured of getting the current NTFS file system file attributes, call the GetFileInformationByHandle function."

http://msdn.microsoft.com/en-us/library/windows/desktop/aa364952%28v=vs.85%29.aspx

@piscisaureus
Copy link

None of the above.

ENOTEMPTY is in fact a condition similar to EBUSY that can happen on directories.

When a file is deleted while another handle or another process still has the file open, the file just gets 'marked for deletion'. The directory entry remains until the file is actually deleted, which happens when the last handle to it is closed.

This is unlike unix where the inode has the same semantics of a file on windows (it isn't actually deleted until all FDs are closed), but the directory entry can be removed immediately.

Note that this case is also distinct from the EBUSY case - that happens when deletion itself is prevented by another process that did not set the FILE_SHARE_DELETE flag when opening the file. But it's likely that virus scanners set the flag when opening files to be scanned, so antivirus software might indeed trigger this.

(and yes, it is dumb)

@jbondc
Copy link

jbondc commented Nov 15, 2013

But which handle or process still has the file open? It appears that only node is accessing those files.
http://sdrv.ms/HW49Re

Should CloseHandle() be before SET_REQ_SUCCESS() ?

https://github.com/joyent/libuv/blob/master/src/win/fs.c#L693

@piscisaureus
Copy link

But which handle or process still has the file open? It appears that only node is accessing those files.
http://sdrv.ms/HW49Re

The virus scanner, that starts scanning the file in response to it being created.

http://sdrv.ms/HW49Re

It probably won't show up so near to the node action. But I can't look through that file and have only your screen shot to go by.

Should CloseHandle() be before SET_REQ_SUCCESS() ?
https://github.com/joyent/libuv/blob/master/src/win/fs.c#L693

no. throwing shit at the wall doesn't really help.

@isaacs
Copy link
Owner

isaacs commented Nov 15, 2013

Thanks for the feedback, @piscisaureus.

ENOTEMPTY is in fact a condition similar to EBUSY that can happen on directories.

So, for rimraf's purposes, much as it pains me, should we just keep trying for a while like we do with EBUSY?

Of course, I wouldn't ever want to do such a thing on Unix, but if it would work around a windows-specific issue, then that's probably fine.

@piscisaureus
Copy link

So, for rimraf's purposes, much as it pains me, should we just keep trying for a while like we do with EBUSY?

Right on. Unfortunately there is no good way to tell this case apart of a real ENOTEMPTY condition, and it's hard to tell what is an appropriate time to retry. Not anything that isn't very complex atleast.

So yeah, just keep trying for a while...

Of course, I wouldn't ever want to do such a thing on Unix, but if it would work around a windows-specific issue, then that's probably fine.

:trollface:

@jbondc
Copy link

jbondc commented Nov 15, 2013

Fair enough, but I'm pretty sure this case has nothing to do with the virus scanner:
https://skydrive.live.com/?cid=295047487FC4F731&id=295047487FC4F731%213386&v=3

I put up the log files here:
http://sdrv.ms/17wmVob

Can also export in csv or xml.

@piscisaureus
Copy link

@jbondc It seems that in this case node has the file open when it gets deleted. So you get the same effect.

@isaacs isaacs reopened this Nov 28, 2013
@isaacs isaacs closed this as completed in d819b12 Nov 29, 2013
lars-hakansson pushed a commit to lars-hakansson/ng-openapi-gen that referenced this issue Apr 27, 2020
luisfpg pushed a commit to cyclosproject/ng-openapi-gen that referenced this issue Apr 27, 2020
* Delete folder that works on windows (isaacs/rimraf#25)

* .

Co-authored-by: Håkansson Lars    IT-konsulter <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants