Support yarn?

[kentcdodds]

This was just a thought that came to my head. But would ya'll consider supporting yarn? Seems like it'd be really useful here because the yarn.lock locks down the package versions and maintainers would probably like to keep that up to date. I envision seeing pull requests that update the yarn.lock and potentially the package.json if the version range is out of date.

[rkuykendall]

As far as I can tell, the package.json is exactly the same in yarn as it is in NPM, so greenkeeper as it is would work just fine with yarn. However, there is a proposed feature update npm-shrinkwrap.json if exists, which would present a feature difference in NPM vs Yarn. You may want to leave a comment there, since this seems more like a variant on that feature than a change to anything existing.

[kentcdodds]

Yeah, maybe these could be merged, but I was thinking more specifically:

I envision seeing pull requests that update the yarn.lock and potentially the package.json if the version range is out of date.

Greenkeeper doesn't support this yet, and that's why I filed this issue. Happy to merge them though. But I think the work required will be pretty different.

[boennemann]

We would love to send updates to the yarn.lock file along with our pull requests. Especially because that would be a deterministic operation, that doesn't require a file system, like shrinkwrap does.

We are operating Greenkeeper without any outside investment into the company. We want to build a business that must only serve the community we are building this for, which of course includes ourselves, but not any investors.

With the very limited resources we're having because of this, we decided to focus our engineering efforts on migrating Greenkeeper over to GitHub integrations as fast as possible. It doesn't sound like that big of a change, but it's changing a lot behind the curtains.

I think that Yarn and Greenkeeper could be an excellent match, but unfortunately they did not reach out to us prior to their public launch.

They pride themselves as an open community project, and that their ideas were discussed and validated by a lot of people. I'm sad to see that we weren't part of that group.

All I'm left with now is the very sad task to tell you that this isn't going to land anytime soon. It could have been different a few weeks ago when we set our current roadmap.

We're trying our very best to get this to you as soon as possible, and hope you understand our current priorities.

Thank you,
Stephan

[kentcdodds]

Thanks for the explanation @boennemann! Best of luck on your endeavors! I appreciate all that you do for the community. Looking forward to when you can get around to this!

[jasonLaster]

@boennemann I completely see where you're coming from.

I'd be happy to help implement this feature as it could be a really nice win for yarn projects. Do you think you could provide some tips on how to do this? I understand if it is out of scope and that's fine too.

Also as a sidenote I took a quick glance through the code and found evilhackerdude and some other random files. :P what's the story there?

[danielkcz]

I am actually wondering if this could be solved on the CI side, eg. Travis. When greenkeeper makes a pull request, Travis could spot a difference in package.json, run yarn and commit updated yarn.lock to the PR. It might be a long shot and I don't have that much experience with Travis customization just yet. Opinions?

[kentcdodds]

Thanks for the idea, but the PR should update the lock file too

[kentcdodds]

Oh, misunderstood your comment. Yes, this would still require work on greenkeepers part

[danielkcz]

Really? How so? Once there is a PR, the greenkeeper doesn't care about it anymore, right? So if Travis can just commit to that PR with updated yarn.lock, everything should be just fine.

Thinking about it, there actually no need to check on change in package.json, because yarn takes care of it. After successful build, the Travis would just need to commit yarn.lock if it has changed during run.

[danielkcz]

There is a slight issue when build is actually executed for a multiple environments (node versions). It would be necessary to specify which version is used for this update I guess.

[danielkcz]

I guess this is no longer that hot idea after recent discoveries: https://medium.com/@boennemann/avoid-yarn-for-packages-and-fully-enjoy-its-benefits-for-application-development-8bdd4deb33cf#.vnapi4xvi

For open source apps it can still find it's use, but there is a probably much lower number than modules.

[steebchen]

It would be cool to know whether this will be implemented in Greenkeeper since that would be one of a few reasons to switch from private Bitbucket to private Github.

[ghost]

@FredyC you may be interested in a recent Yarn blog post about this subject - https://yarnpkg.com/blog/2016/11/24/lockfiles-for-all

The blog post was made shortly after a discussion here sindresorhus/meta#4

[travi]

i see lots of value in what yarn provides, but greenkeeper is more important to my workflow at this point. because of this, i'm holding off adopting yarn until greenkeeper supports it, but would love to be able to use it for some of my projects.

[boennemann]

A quick update: For the time being there is a community built workaround for this: https://gist.github.com/maxrimue/16e272c4e0b8a8ac9dd5570e180e08c0

We published our roadmap: https://git.io/greenkeeper-roadmap
Yarn support still isn't in it, but we're planning to open up Greenkeeper soon, so that the community can contribute features, just like yarn support.

[erbridge]

Possibly useful: https://yarnpkg.com/en/docs/cli/generate-lock-entry

[unional]

https://yarnpkg.com/blog/2016/11/24/lockfiles-for-all

@destroyerofbuilds, I read it but don't buy the idea.

The problem is when the user discovers the problem, they report the issue to the contributors.
The way we report the issue is typically by the symptom.
At this point, no one, users or contributors, knows about what went wrong.

With the lock file in the library, contributors have a hard time to figure out what actually breaks because when they run yarn and run tests in the library, everything still works as expect.

It is a burden on the contributor needing to aware that, "ah, maybe there is a break in the dependency".
Which defeat the purpose of lock file in the first place.

[clarkbw]

BTW I created this project to help with this: https://github.com/clarkbw/circleci-update-yarn-lock definitely not perfect but at least automates the process of creating an updated yarn.lock file.

[diegohaz]

We should consider prioritizing this as Travis will soon run yarn with --frozen-lockfile causing greenkeeper builds to fail with the following error:

If I got it correctly, it's because greenkeeper doesn't update yarn.lock.

Reference on Travis repo: travis-ci/travis-ci#7395 (comment)

[apertureless]

@diegohaz well travis already run yarn with --frozen-lockfile so every greenkeeper branch is failing because if this.

mass greenkeeper issues inc 🙈

[p0wl]

Any updates on this? Circleci is not using --frozen-lockfile, but since the yarn.lock is not updated on greenkeeper branches, it is worthless.

If you need any help, please let us know how.

(the workaround from @clarkbw does not work for us, because it uses yarn generate-lock-entry to regenerate yarn.lock which is different from what yarn creates when running yarn install)

[clarkbw]

(the workaround from @clarkbw does not work for us, because it uses yarn generate-lock-entry to regenerate yarn.lock which is different from what yarn creates when running yarn install)

BTW, I'm updating the fix right now. Happy to look into this as well. An important note for anyone working on this is that the yarn.lock file needs to be updated before tests are run.

[audiolion]

Well with the May 2nd release of Greenkeeper 2.0 and a lot of features put out on the roadmap I hope this gets added!

[jrjohnson]

I've added yarn check to my CI setup. Ensures that the yarn.lock and package.json file are in sync. It isn't ideal, but it keeps tests from passing in cases where the lock file is out of sync.

[jdalton]

Ya'll might dig that npm5, which in many cases is as fast as yarn, will have lock files too.
So lock files will need to be tackled sooner or later for either.

[SimenB]

Will greenkeeper support updating that lockfile?

[keimlink]

We're also using yarn check to ensure files are in sync like @jrjohnson.

@jdalton Will npm5 have a command comparable to yarn check? Unfortunately I couldn't find a changelog for npm5.

[zcei]

@keimlink I think your safest bet might be npm/npm#16244
What is still left you can find in v5.0.0 (project)

[keimlink]

@zcei Thanks for the links! The performance comparisons between npm and yarn in npm/npm#16244 were very interesting.

[ghost]

I've added yarn check to my CI setup. Ensures that the yarn.lock and package.json file are in sync. It isn't ideal, but it keeps tests from passing in cases where the lock file is out of sync.

@jrjohnson, @keimlink you may wan to run yarn install with --frozen-lockfile. It's a flag that causes yarn to check that the lockfile agrees with your package.json file. If they are out of sync --frozen-lockfile will fail the install.

--frozen-lockfile is not yet documented on the website, but there is an issue file to do that - yarnpkg/website#378

Y'all might dig that npm5, due out ~May 5th, is as fast as yarn and will have lock files too.

@jdalton please keep in mind that npm has had shrinkwrap files (their lockfiles) for years. A core missing feature from npm is the lack of support for automatically generating a lockfile the first time your run npm. Instead, you must call npm shrinkwrap to generate a shrinkwrap file for the first time. This makes adopting the best practice of using lockfiles difficult to organically grow within a community.

npm@5 is not quite as fast as yarn according to this comment.

yarn also just accepted the following change, which will introduce a three fold decrease in installation times - yarnpkg/yarn#3234 (comment)

Will greenkeeper support updating that lockfile?

@SimenB GreenKeeper 2.0 supports npm's npm-shrinkwrap.json lockfile. GreenKeeper does not support Yarn.

Will npm5 have a command comparable to yarn check?

@keimlink @jdalton I haven't seen any support within npm for ensuring the lockfile doesn't fall out of sync.

---

In general, if anyone wants to maintain their dependencies, and use Yarn, please look into renovate. I came across it several month ago, and it's been a really nice tool to throw into a Jenkins setup, and keep all my package dependencies updated, and occasionally update my project's lockfiles.

[danielkcz]

@jdalton I am just curious, what makes that benchmark you have linked more relevant than the one posted later (by non-owner)? That shows kinda opposite opinion...

[jdalton]

@destroyerofbuilds
npm5 will automatically generate lock files on install and perf can beat yarn in cases too. Pretty neat!

@FredyC Others have reported wins too. It's been said that speeds of either vary from project to project so you'll likely see cases of both winning out. It's pretty rad though that they're that close to each other. It puts more pressure to support lock files for sure. Everyone wins 🎉

[ghost]

At this point, no one, users or contributors, knows about what went wrong.

@unional I assume users refers to application developers. Let's assume these users are following engineering best practices and committing lock files to their projects. Under this assumption, then we can assume that at some point, before their application broke, they ran yarn upgrade, npm test, and their unit tests failed (or, if they don't have unit tests, then they tested their application in a browser).

Upon encountering the failure, is it to much to ask them to run git diff, and notice the transitive dependencies that changed, and report the issue to those dependencies?

npm5 will automatically generate lock files on install

@jdalton thank you for bringing that to my attention. I hadn't realized that feature made it into the npm@5 roadmap. Glad to see npm@5 adopting a feature that will lead to a greater adoption of lockfiles within our community.

@unional the lockfile behavior of npm@5 will now, by default, be similar to yarn, though you will be able to still use npm-shrinkwrap.json files if you wish (just requires the extra npm shrinkwrap step).

@FredyC Others have reported wins too. It's been said that speeds of either vary from project to project so you'll likely see cases of both winning out.

I do agree with you @jdalton that speeds with vary between projects, but also operating systems, network environments, etc.

If you find any benchmarks that come with their test methodology, I'd appreciate references. I'm always trying to keep an eye out for how people benchmark the performance of these package managers (I have a slightly modified benchmark script myself). I'm also trying to keep an eye out for what conditions allow one package manager to excel over another.

[voxsim]

Hi everyone, I am a yarn contributor and I am not here to tell to use yarn instead of npm, but I need your help to understand better the situation :)

I saw greenkeeper the last week and I think it does an awesome work; @finnp @boennemann are there any plans to support yarn in some ways and specifically to support the generation of yarn.lock file? We want to adopt it in the yarn project itself without doing things like that https://gist.github.com/maxrimue/16e272c4e0b8a8ac9dd5570e180e08c0 if we can.

I tagged you guys because you seem the most active developers on the codebase, sorry if I am wrong O.o and thanks for your time!

[ghost]

@voxsim please see the Greenkeeper roadmap - https://github.com/greenkeeperio/greenkeeper/projects?sort=oldest

As noted by @boennemann, and listed under the title Open up Greenkepper on the Roadmap, the Greenkeeper core team will be looking at opening their core to community maintained modules that implement features, including Yarn support.

Therefore, my assumption is that Yarn support is not imminent, but the core team of Greenkeeper is aware of the community's interest, and we can look forward to seeing Yarn support in the future.

In the meantime, please consider employing an alternative such as renovate. (It comes with Yarn and yarn.lock support. I use renovate because it meets my specific needs, such as keeping my Yarn projects updated. I have also contributed to renovate to improve on that experience.)

[voxsim]

@jdalton ? could you please explain me? who is competitor of who? Maybe I lost some pieces O.o btw thanks @destroyerofbuilds! :D

[boennemann]

Alrighty, yarn support, what's taking us so long?

Generating lockfiles on the server side isn't the most trivial problem to solve, but with npm5 adopting package-lock.json by default it's something we will hopefully find a nice solution for soon.

In the meantime: We renamed greenkeeper-shrinkwrap to greenkeeper-lockfile.

We added

• npm5 support
• yarn support
• made it simple to add more CI services

Please let us know how it works for y'all.

I'm going to close this issue while we're trying to figure out how to best add server-side lockfile support, especially with private and git dependencies.

Think you can beat us to it? Great news: This very repo now contains our core worker job implementation. It's not documented yet, but we made the first step of open sourcing it to progress with our promise of opening up Greenkeeper for community contributions.

Peace and love,
Stephan

P.S.: Thanks a lot @maxrimue for providing initial yarn support with this script.