Delay between a release (minor release with security fixes) and Docker images available on hub #168
Comments
|
I'm not aware of the delay times but I agree if this is indeed the issue. |
|
I will. Leave it with me.
…On 20 Mar 2018 22:08, "Liran Tal" ***@***.***> wrote:
I'm not aware of the delay times but I agree if this is indeed the issue.
I believe this falls under the scope of the core nodejs/security team as
this relates to job setup and such. Should we ping nodejs code security for
more info on this?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#168 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAHkOsljOMPFdrd3qrKp0a_zJGdd_3qCks5tgX3vgaJpZM4SyQap>
.
|
|
I think it will also involve the @nodejs/docker team as they do the updates which trigger new docker images to be built. It may be that they need to be part of the security release process so that the commits can be landed as soon as possible after the releases are available. |
|
New versions for Official Docker Images needs to go through approval with Docker Inc. hence the 12-24h delay. |
|
@Starefossen is right, most of the update processes in the docker-node repository go fast, but we'll need the approval with Docker in the official-images repository, so it'll be a delay. Another issue is that the members of @nodejs/docker have no control of our Travis CI, thus we can't stop the invalid or useless builds, and it takes a while to finish the all the CI tasks, that's also the part maybe we could improve. |
|
Sorta related: nodejs/node#14190 Getting a new version into master of docker-node as quickly as possible (and PRing docker hub), maybe even coordinating with the docker hub folks to have them ready to start the merging process beforehand, would be really good, I think. |
|
So, what about for security releases holding the main release for another
12-24 hours till docker is ready? If there is no disclosure of the vuln it
is ok-ish. The world is moving towards Docker and we need to think of an
scenario where 90% of users will be Docker based.
…On 22 March 2018 at 08:00, Simen Bekkhus ***@***.***> wrote:
Sorta related: nodejs/node#14190
<nodejs/node#14190>
Getting a new version into master of docker-node as quickly as possible,
maybe even coordinating with the docker hub folks to have them ready to
start the merging process beforehabd, would be really good, I think.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#168 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAHkOl7-dQi5H2pEr70jkvDkKC3lh-Blks5tg1oOgaJpZM4SyQap>
.
|
|
@dgonzalez Holding the main release won't work currently, because official Docker images are built from public sources by design. So anyone can see which release tarball is used and access it. Delaying disclosure of the actual vulnerabilities is a good idea though. |
|
Having a binary for alpine would also help speed up the updating of docker images but there isn't a consensus on this ATM. |
|
@pesho and what about prepare the build on a private repository and at the moment of release PR into the repo and build it? I am also ok on the delaying disclosure. I am a big fan on disclosing as early as possible but 12-24 hours won't make a huge difference. Said that, the commit history will surely show what was fixed and an avid hacker can use it as an attack vector. |
This will help a bit, but not enough. The Travis build will still take about 40 minutes (due to having to build for Alpine), and then it still must be submitted to Docker Inc. and wait for their separate review and approval. |
|
There was feedback in the Enterprise user-feedback session today that this does cause anxiety for them. The feedback is that when we say you need up update quickly that activates processes in the and then they are under pressure while they have to wait a day for the release. @Starefossen do we have any contacts at docker that we would talk to about the review delay for security releases? I can understand that for normal releases but I wonder if there is some way we can expedite by giving them advance notice of when a release will be coming. |
|
We're also dealing with timezone differences. |
|
Note that we are working hard to reduce build times and implement automation to speed all of this up. |
|
I find myself wishing we'd been looped in on this thread sooner. 😅 Would it be possible for us to get an advance (private) heads up of incoming security releases so we can adjust our schedule to process them in a more timely manner? It's really just @yosifkit and I, and we are in Pacific Time doing most of our review during normal working hours Mon-Fri, but we're happy to make adjustments/arrangements if we know a couple days in advance that a security release is coming. It would definitely help to have pre-built Alpine releases instead of having the image build from source, without question. As part of our review we do both a diff and build test, so the faster the images build, the faster we can get that out of the way (the So to summarize, my (official) recommendations for a timely merge of a security release to an image in the official-images program are:
(Going to go work on some language for the |
|
@ahmadnassri-Telus adding you as an FYI since you were interested in this topic. |
|
@tianon for 1 and 2 when you say PR, is that the PR to the docker repo? |
|
Yes |
|
@tianon we generally announce publicly ~ 1 week in advance. We could add to our process to directly notify you as well. What would be the best way to do that? I'd probably see if we can get an agreement for a private notification as well in case there is an instance were for some reason we can't do the public announcement far enough in advance (although this should be rare or never hopefully). |
|
If https://groups.google.com/group/nodejs-sec is the appropriate place to subscribe to see those (which it appears to be), we can just subscribe (no need to make us a special part of the official notification process). It would still be useful to have whichever https://github.com/nodejs/docker-node maintainer is planning to do the actual bump PR to us give us a poke on IRC or via email (https://github.com/docker-library/official-images/blob/master/MAINTAINERS) to let us know their expected timeframe, but not strictly necessary. (Perhaps adding something like "fyi @tianon @yosifkit" to the security bump PR in that repo so we know it's in progress?) Getting something down for private notification is probably also a good idea, but if that's as rare as you hope, we could wait and do that when it becomes necessary. |
|
@tianon https://groups.google.com/group/nodejs-sec is the right place to subscribe to. @SimenB is the poke through IRC something you can add to your process. We should also talk about getting the docker-node team more closely tied to the security release process so you have a better insight into when the releases are going to be shipped, with at least some notifications when we publish the release (unless you already get that effectively through the nodejs-sec mailing list). |
|
With a security release coming up, do we have a process ready? https://nodejs.org/en/blog/vulnerability/june-2018-security-releases/ /cc @tianon as a heads up |
|
@SimenB I dont think we made progress here. I don't know much about how we publish the docker images and how this process goes. What is the status and how could we help improve it? |
|
See the messages above from @tianon. It also involves pinging the Docker people beforehand so they can be ready to run their tests. Not sure what happened about holding back disclosure until the new docker images are available? Or if it's needed (or even feasible). |
|
@SimenB I've sent you an email to connect you to the individual who will be being doing this release. I suggest we use this release to try and experiment with what helps and the capture what works into the process. |
|
In terms of holding back disclosure, I think we still need to understand what that means. Does that mean holding the final announcement with the details for a shortish period of time (ex 1 hour) or something more complicated than that which requires being able to build before the regular binaries are available? |
|
I'm not sure what holding back disclosure would entail either, I just mentioned it since it was brought up in #168 (comment). Just holding back the blog post about the exact details for a short time was my interpretation, though. Again, not sure if it's helpful or feasible. I agree we should use this opportunity to learn what we can about the optimal process for this. |
|
CI is still a big issue to us we, as I mentioned, @nodejs/docker has no permission to stop, restart any certain CI tasks, also, Travis CI sometimes hangs, reason unknown. |
|
Oh boy, this is the first day of DockerCon, this'll be fun 😅 We'll make it work, the most helpful would be more details on the rough timing we'll see so we can be on the lookout for it (especially since we'll be in conference mode). 👍 |
|
@mhdawson I think that went reasonably well? Only hitch was waiting on approval for merge in nodejs/docker-node#783. I think an admin in the organisation would be able to just merge it, but it's maybe not a big deal? |
|
I get a lot of notifications on GitHub. Maybe in the future it would be nice to get a direct email? |
|
I don't think security releases should need approval in the first place, but I think only org admins can ignore stuff like that |
|
I agree, they should be able to to approve and merge when they need to |
|
As a TSC member I think I'm an org admin, and whatever is in place to prevent a push without a review blocked me as well. Both in the UI and also from pushing manually from the git command line. |
|
Hmm, OK. Not sure how to get around it, then. Maybe the bottom box is checked in this form (screenshot from another repo)?
|
|
Yup,
|
|
So in theory removing the "include administrators" would help? |
|
@SimenB I think so. |
|
There was some discussion of improvements to make to the process, and also some delays that are unavoidable. I know the sec release process includes a heads up to the docker team so they know a release is coming. If the concerns that are addressable have been addressed, can this be closed now, @mhdawson @dgonzalez @nodejs/docker ? |
|
I think this can be closed. |
This delay can allow attackers to exploit a vulnerability in node core on Docker based systems. Last time I remember there was a delay of 1 day (which is not a lot) but still has to be considered as an attack vector.
The text was updated successfully, but these errors were encountered: