Migrate away from yarn 1/classic

[jagthedrummer]

We're running on yarn 1.22.22 by default and the 1.x line has been deprecated since 2020.

This has started to cause problems due to depfu trying to make some updates that cause the tests to fail.

The failures look like this:

yarn exec v1.22.22
/home/runner/work/bullet_train/bullet_train/tmp/starter/node_modules/cliui/build/index.cjs:291
const stringWidth = require('string-width');
                    ^

Error [ERR_REQUIRE_ESM]: require() of ES Module /home/runner/work/bullet_train/bullet_train/tmp/starter/node_modules/string-width/index.js from /home/runner/work/bullet_train/bullet_train/tmp/starter/node_modules/cliui/build/index.cjs not supported.
Instead change the require of index.js in /home/runner/work/bullet_train/bullet_train/tmp/starter/node_modules/cliui/build/index.cjs to a dynamic import() which is available in all CommonJS modules.
    at Object.<anonymous> (/home/runner/work/bullet_train/bullet_train/tmp/starter/node_modules/cliui/build/index.cjs:291:21) {
  code: 'ERR_REQUIRE_ESM'
}

Node.js v20.12.2
error Command failed.
Exit code: 1

Other folks have run in to this and it seems like the best solution is to upgrade off of yarn 1.

[gazayas]

I've been curious about this for a while, and we temporarily had things working with v3.5.1 back in #754, but I haven't looked at it in a bit. That PR might have some hints though.

[jagthedrummer]

@gazayas that's interesting, and I'd forgotten about that PR. I don't find anything in the contents of that PR that actually pins yarn to a version. So maybe you were just using 3.5.1 locally? And we were still running 1.22.22 in CI?

[gazayas]

@jagthedrummer I don't quite remember the CI setup at that point, but yes, I was running v3.5.1 locally. It created all these files in .yarn/ which I put in .gitignore, but it might be a different setup in v4.

It seems like you got v4 to pass on CI though? I'll see if I can test out the branch locally soon.

[jagthedrummer]

@gazayas Looks like we probably want to tweak .gitignore to not ignore quite so much. https://yarnpkg.com/getting-started/qa#which-files-should-be-gitignored

I wonder if version 3.5.1 might have put some kind of version identifier somewhere in .yarn/ to keep track of which version to use.

I'll add the .gitignore tweaks to #1445 tomorrow.