Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CircleCI node images distributing outdated, unpackaged yarn 0.27.5 #4407

Closed
mikemaccana opened this issue Sep 11, 2017 · 12 comments
Closed

CircleCI node images distributing outdated, unpackaged yarn 0.27.5 #4407

mikemaccana opened this issue Sep 11, 2017 · 12 comments
Labels
os-linux

Comments

@mikemaccana
Copy link
Contributor

From https://yarnpkg.com/en/docs/install#linux-tab

echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add

Running this on Debian 8:

lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:  Debian GNU/Linux 8.9 (jessie)
Release:  8.9
Codename: jessie

Results in yarn 0.27.5 being installed:

yarn --version
0.27.5
@mikemaccana mikemaccana mentioned this issue Sep 11, 2017
Closed
@Daniel15
Copy link
Member

That's strange because 1.0.1 is definitely there:

curl -s https://dl.yarnpkg.com/debian/dists/stable/main/binary-amd64/Packages | grep 'Version: '
Version: 1.0.1-1
Version: 0.27.5-1
Version: 0.27.4-1
Version: 0.27.3-1
Version: 0.27.2-1
Version: 0.24.6-1
...

Can you confirm that you've ran apt update to update your package cache?

@Daniel15
Copy link
Member

I just confirmed that I get 1.0.1 from the repo, on Ubuntu on Windows (Windows Subsystem for Linux):

daniel@DANLO--155:~$ sudo apt-get update
[sudo] password for daniel:
Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Hit:4 https://dl.yarnpkg.com/debian stable InRelease
Hit:5 http://ppa.launchpad.net/ondrej/php/ubuntu xenial InRelease
Fetched 204 kB in 4s (41.8 kB/s)
Reading package lists... Done

daniel@DANLO--155:~$ sudo apt-get install yarn
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  fontconfig-config fonts-dejavu-core libfontconfig1 libfreetype6 libgd3 libjbig0 libjpeg-turbo8 libjpeg8 libssl1.0.2
  libtiff5 libvpx3 libxpm4 libxslt1.1 php-common php5.6-cli php5.6-common php5.6-json php5.6-opcache php5.6-readline
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  libuv1 nodejs
The following NEW packages will be installed:
  libuv1 nodejs yarn
0 to upgrade, 3 to newly install, 0 to remove and 121 not to upgrade.
Need to get 3,863 kB of archives.
After this operation, 17.3 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ppa.launchpad.net/ondrej/php/ubuntu xenial/main amd64 libuv1 amd64 1.9.1-3+deb.sury.org~xenial+1 [60.3 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 nodejs amd64 4.2.6~dfsg-1ubuntu4.1 [3,161 kB]
Get:3 https://dl.yarnpkg.com/debian stable/main amd64 yarn all 1.0.1-1 [641 kB]
Fetched 3,863 kB in 3s (1,287 kB/s)
Selecting previously unselected package libuv1:amd64.
(Reading database ... 26478 files and directories currently installed.)
Preparing to unpack .../libuv1_1.9.1-3+deb.sury.org~xenial+1_amd64.deb ...
Unpacking libuv1:amd64 (1.9.1-3+deb.sury.org~xenial+1) ...
Selecting previously unselected package yarn.
Preparing to unpack .../archives/yarn_1.0.1-1_all.deb ...
Unpacking yarn (1.0.1-1) ...
Selecting previously unselected package nodejs.
Preparing to unpack .../nodejs_4.2.6~dfsg-1ubuntu4.1_amd64.deb ...
Unpacking nodejs (4.2.6~dfsg-1ubuntu4.1) ...
Processing triggers for libc-bin (2.23-0ubuntu7) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up libuv1:amd64 (1.9.1-3+deb.sury.org~xenial+1) ...
Setting up yarn (1.0.1-1) ...
Setting up nodejs (4.2.6~dfsg-1ubuntu4.1) ...
update-alternatives: using /usr/bin/nodejs to provide /usr/bin/js (js) in auto mode
Processing triggers for libc-bin (2.23-0ubuntu7) ...

daniel@DANLO--155:~$ yarn --version
1.0.1

@mikemaccana
Copy link
Contributor Author

Yep I'm running apt-get update. I can reliably reproduce the behavior as it runs on every build as part of CircleCI - here's my .circleci/config.yml:

# See https://circleci.com/docs/2.0/configuration-reference (badly written)
# And https://circleci.com/docs/2.0/configuration-reference/#full-example (better)
version: 2
jobs:
  build:
    docker:
      # Circle does not currently have a node 8 image
      # https://circleci.com/docs/2.0/circleci-images/#nodejs
      - image: circleci/node:latest

    # Note this needs a trailing slash or Circle will fail.
    # Cannot be '~/' - see https://discuss.circleci.com/t/directory-tmp-you-are-trying-to-checkout-to-is-not-empty-and-not-git-repository/11370
    working_directory: '~/myapp'
    branches:
      only:
        - master # list of branches to build

    steps:
      - checkout

      # Debian doesn't consider LSB essential.
      # Debian doesn't securely download packages by default.
      # Sigh.
      - run: sudo apt-get install lsb-release apt-transport-https

      # RethinkDB repo. From https://rethinkdb.com/docs/install/debian/
      - run: echo "deb http://download.rethinkdb.com/apt `lsb_release -cs` main" | sudo tee /etc/apt/sources.list.d/rethinkdb.list
      - run: wget -qO- https://download.rethinkdb.com/apt/pubkey.gpg | sudo apt-key add -

      # Yarn repo. From https://yarnpkg.com/en/docs/install#linux-tab
      - run: echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
      - run: curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -

      # Add required OS packages
      - run: sudo apt-get update
      - run: sudo apt-get install rethinkdb yarn

      # From https://www.rethinkdb.com/docs/start-on-startup/
      # (systemctl currently fails with 'Failed to get D-Bus connection: Unknown error -1')
      - run: sudo cp /etc/rethinkdb/default.conf.sample /etc/rethinkdb/instances.d/instance1.conf
      - run: sudo /etc/init.d/rethinkdb start

      # Configure yarn and install required packages
      - run: yarn --version
      - run: cp .yarnrc ~
      - run: sudo yarn global add mocha
      - run: yarn

      # Debugging info (just in case someone asks)
      - run: lsb_release -a
      - run: uname -a
      - run: node --version
      - run: yarn --version
      - run: mocha --version
      - run: rethinkdb --version
      - run: pwd

      # Run the tests
      - run: ./run-tests.sh

@mikemaccana
Copy link
Contributor Author

OK here's something SUPER weird: 1.0.1 is being installed but it looks like it contains an old version of yarn:

#!/bin/bash -eo pipefail
sudo apt-get install rethinkdb yarn
Reading package lists... Done


Building dependency tree       


Reading state information... Done

The following extra packages will be installed:
  libprotobuf9
Recommended packages:
  nodejs
The following NEW packages will be installed:
  libprotobuf9 rethinkdb yarn
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 12.0 MB of archives.
After this operation, 45.9 MB of additional disk space will be used.


Get:1 http://download.rethinkdb.com/apt/ jessie/main rethinkdb amd64 2.3.6~0jessie [11.0 MB]

Get:2 https://dl.yarnpkg.com/debian/ stable/main yarn all 1.0.1-1 [641 kB]

Get:3 http://deb.debian.org/debian/ jessie/main libprotobuf9 amd64 2.6.1-1 [361 kB]

Fetched 12.0 MB in 0s (23.8 MB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libprotobuf9:amd64.
(Reading database ... 23624 files and directories currently installed.)
Preparing to unpack .../libprotobuf9_2.6.1-1_amd64.deb ...
Unpacking libprotobuf9:amd64 (2.6.1-1) ...
Selecting previously unselected package rethinkdb.
Preparing to unpack .../rethinkdb_2.3.6~0jessie_amd64.deb ...
Unpacking rethinkdb (2.3.6~0jessie) ...
Selecting previously unselected package yarn.
Preparing to unpack .../archives/yarn_1.0.1-1_all.deb ...
Unpacking yarn (1.0.1-1) ...
Processing triggers for systemd (215-17+deb8u7) ...
Setting up libprotobuf9:amd64 (2.6.1-1) ...
Setting up rethinkdb (2.3.6~0jessie) ...
invoke-rc.d: policy-rc.d denied execution of start.
The RethinkDB startup service is installed but disabled. To enable it,
follow the instructions in the guide located at http://www.rethinkdb.com/docs/guides/startup/
Setting up yarn (1.0.1-1) ...
Processing triggers for libc-bin (2.19-18+deb8u10) ...
Processing triggers for systemd (215-17+deb8u7) ...

A moment later:

#!/bin/bash -eo pipefail
yarn --version
0.27.5

@BYK
Copy link
Member

BYK commented Sep 12, 2017

@mikemaccana I think you have an older yarn installed somewhere and that is included in your path earlier than apt packages.

@mikemaccana mikemaccana changed the title Debian packages out of date (0.27.5 rather than 1.0.1) CircleCI images distributing outdated, unpackaged yarn 0.27.5 Sep 12, 2017
@mikemaccana
Copy link
Contributor Author

mikemaccana commented Sep 12, 2017 •
edited
Loading

Found it. CircleCI image: circleci/node:latest includes an unpackaged version of yarn installed to /usr/local/bin. This breaks newer versions (since the latter path takes prominence) and has wasted a whole bunch of my (and your) time.

I wonder if it's worth talking to CircleCI and getting then to unfu... ahem, fix their images.

@mikemaccana mikemaccana changed the title CircleCI images distributing outdated, unpackaged yarn 0.27.5 CircleCI node images distributing outdated, unpackaged yarn 0.27.5 Sep 12, 2017
@mikemaccana
Copy link
Contributor Author

CC @FelicianoTech who might be able to help.

@BYK
Copy link
Member

BYK commented Sep 12, 2017

@mikemaccana thanks a lot for the digging! Shall we close this since we can't do much about it?

@FelicianoTech
Copy link

I think it should be fine to close. If anyone wants to pick up where this left off, I opened an Issue in our circleci-images repo. It's referenced above ^.

@BYK
Copy link
Member

BYK commented Sep 12, 2017

@FelicianoTech thanks a lot!

@BYK BYK closed this as completed Sep 12, 2017
@mikemaccana
Copy link
Contributor Author

mikemaccana commented Sep 12, 2017 •
edited
Loading

Thanks @FelicianoTech and @BYK ! Yes please close and your help and patience has been much appreciated!

@Daniel15
Copy link
Member

Oh, good point, I guess CircleCI installs Yarn either through the tarball or through the standalone .js file.

@mikemaccana - As an aside, the best way of using Docker is to install all your dependencies into your own Docker image via a Dockerfile. That way, they don't need to be reinstalled on every single build. That's the primary use case for Docker on CircleCI. 😃

@mikemaccana mikemaccana mentioned this issue Sep 12, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
os-linux
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Daniel15 @BYK @mikemaccana @FelicianoTech