-
-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
755 folder permissions stomped on my lambda... #64
Comments
It was just changed to correctly preserve directory permissions: 864a8fd |
Edit: I see what's happening. Before this update, the bad permissions just happened to work for our project. After the update, the permissions were being set correctly and those correct permissions were breaking our app. |
@pe8ter so, to you, this issue may be closed? |
Yes, but it's not my Issue to say so. Thanks for timely responses! |
@pe8ter you're welcome! Sure, the issue is not yours, but yet, 1 of 2 issuers are OK with the module's behavior! |
maybe it wasn't the permissions on ours that was causing the problem - all the issues we had with our deployment were coming from windows - they all work correctly with 3.0.2 but failed with 3.1.0... we aren't setting any special permissions on the folders, its a newly created folder that gets zipped. and its hard to tell exactly what the problem is from the AWS lambda side, we have re-downloaded the zip from amazon and not had any problems opening it or running the node scripts inside of them. |
@jjmartin We were building on a Windows box too. Time to update! |
time to update what tho - i am not sure what i need to be doing differently... i can accept that we are doing something wrong but its pretty default behavior... gulp.task('zip', ['npm', 'subfolders'], function () {
return gulp.src([distributionFolder + '**/*'])
.pipe(zip(butils.getZipName()))
.pipe(gulp.dest(distributionFolder));
}); |
This update means it's time for us to not build on a Windows box and deploy to a Linix machine. Hard to say much else without details of your CI. |
well that is unacceptable... one of the whole points of my migrating my deployment to gulp was so that it would be cross platform. (was previously using a powershell script). My build server actually is linux and i believe i still get the issue. |
@jjmartin After a few new tests, I could reproduce your problem. I assume your variable Try to add a slash after the folder name, like this one |
ok thanks - i will give that a try ... |
actually after getting a chance to look back at my code that distribution folder does have a trailing slash so it looks somthing like '../myproject-dist/' and the final glob looks like '../myproject-dist/*/' which i think is about what you are saying it should look like, but we did get the issue of Lambda not being able to read the node_modules folder. is there some windows setting i can put on the folders as i create them that would make the gulp-zip 3.1.0 make the folders 775 again? (or appear a such to the linux container that lambda runs in? ) |
ok after watching a debug and reading a bunch - i think i understand the underlying problem. Windows is ALWAYS going to return 16822 for its stat.mode on a directory. when translated to octal we get 40666 - (directory)read/write for owner, group and others (no execute) I'm assuming that the node_modules of my deployment should have execute permissions (otherwise i wouldn't be having this problem) would you accept a merge request that included an option to force set a mode on the zipped files/folders? (or left them unset) i think thats the only solution that i would be able to continue to use gulp-zip to do this - i think i'd rather move to my own use of yazl or another library rather than remained pinned to a specific version. |
@jjmartin |
ok - i'll look into the gulp-chown - that is totally a fine solution - yazl did do 775 by default (this is what was set in 3.0.2), but since you added your own mode setting - you aren't using the default anymore - you are using what is coming off of the node file.stat.mode which in this case of folders is the 16822. |
just in case anyone else is finding this - i think what i actually want is your other project https://github.com/sindresorhus/gulp-chmod |
We ran into a related issue, which I believe might be the underlying cause of some of the symptoms described in this thread. I just wanted to document it here. We're creating zip files on Windows and unzipping them on Linux. (Actually, AWS Elastic Beanstalk is doing the latter for us.) The code that creates the zip files used to be fairly trivial: return gulp.src('build/**/*')
.pipe(zip('package.zip'))
.pipe(gulp.dest('dist')) The thing is that the glob also matches directories, which makes gulp-zip create explicit directory entries in the zip file, copying their permissions. Unlike Linux, Windows doesn't require that directories be executable in order to allow access, meaning that the directory entries in the zip file won't be marked as executable. Consequently, unzipping The easiest fix we could think of is passing return gulp.src('dist/**/*', {nodir: true}) This will only pass files to gulp-zip, keeping it from creating explicit directory entries, and instead creating the directories implicitly upon unzipping files contained inside them. Of course, this means that directory permissions won't be preserved, which may not be what you're after. (I didn't check, but I'm pretty sure the default directory permissions from Of course, as Sindre already mentioned, gulp-chmod and/or gulp-chown will work as well, but you'll need to combined it with gulp-filter to only modify directories, which could become tedious. Alternatively, I guess you could also pipe the files through an object stream yourself, check if they're directories, and change their permissions. |
Open an issue/PR on https://github.com/gulpjs/vinyl-fs. This plugin is just passing the permissions it get's from there. |
@sindresorhus Do you think that's relevant though? I don't think vinyl-fs is doing something wrong per se. It's just that the permissions on Windows and Linux don't align, so it kinda makes sense that you have to do some additional work if you really want to preserve them. |
By which I don't mean that gulp-zip should take care of that for you. An option like |
…pacting Windows users
just as a followup for people finding this... this was my workaround: using the gulp-tap:
|
As another follow-up for those finding this: Thank you, @timdp! Your idea to use the |
@jjmartin, your tap workaround saved my day! |
@jjmartin 3.5 hours of struggling with this today and BOOM your gulp-tap solution works. I REALLY need to get a linux machine so I don't have to deal with windows-related issues... |
This was a great suggestion @jjmartin !! Many thanks for the workaround. |
@jjmartin not sure what your tap workaround does but it saved me from hours of figuring it out! thanks |
@timdp suggestion to add |
The suggestion from @timdp to add {nodir: true} worked for me as well. To prevent future complaints/"me toos" would you accept a PR to note this in the documentation? |
There's a known issue with gulp-zip that means that failing to use the nodir: true parameter means that the zip file doesn't extract correctly once uploaded to AWS Lambda. The change ensures this works as expected. See the thread here: sindresorhus/gulp-zip#64
See wprig#594. gulp `src` passes directories by default. However, on Windows and Linux the permissions are different so if the prod bundle is created on a Windows machine and deployed to a Linux server there are issues. Setting `nodir: true` in gulp `src` is a workaround suggested [here](sindresorhus/gulp-zip#64 (comment)).
#62
This fix (i think) broke my AWS lambda function. With 3.0.2 it works fine but with 3.1.0, the AWS lambda isn't able to load external modules.
(it could be something else but we have explicitly tested with the only change being the version number of gulp-zip)
I'm not saying this is a bug in gulp-zip - but i would like to know if there are options to set it back to the permissions i think were working. if you think it was some other change between 3.0.2 and 3.1.0, i'd be glad to explore that too.
Thanks for your help.
The text was updated successfully, but these errors were encountered: