Skip to content

Conversation

smortex
Copy link
Contributor

@smortex smortex commented Dec 16, 2022

CI is currently broken. This PR is an attempt to fix it.

Fix #3121

Before orchestrator_client-0.5.4, the verion of faraday was not
constrained.  When resolving dependencies, bundler install
orchestrator_client-0.5.3 in order to avoid the constraint of an old
version of faraday and install the latest faraday-2.7.2.

The latest version of orchestrator_client has a constraint for faraday
to use version `~> 1.4`.

Update the minimal version of orchestrator_client to fix some CI
failures.
@smortex smortex requested a review from a team as a code owner December 16, 2022 18:36
OpenSSL 3.0 broke older versions of net-ssh which raise an exception:
> pkeys are immutable on OpenSSL 3.0

Bump the maximim net-ssh version to allow version 7.0 which fixed this
by adding support for OpenSSL 3.0.

More info: net-ssh/net-ssh#843
@nicklewis
Copy link
Contributor

I think at least one of the problems is that we're now testing against OpenSSL 3, but we need net-ssh 7 to support it. But net-ssh 7 doesn't support Ruby 2.5, which is our oldest supported Ruby version.

...and as I typed that, you added a commit to allow net-ssh 7! That should be mostly fine. We only test against 2.7 and 3.1, and we ship 2.7 in our packages. I think it will prevent you from doing gem install bolt on Ruby 2.5 though. Maybe if you explicitly install net-ssh 6 first it will work? I will test that out a bit.

@nicklewis
Copy link
Contributor

nicklewis commented Dec 16, 2022

It looks like we already have an issue like that with a couple of other gems (public_suffix and faraday-net_http), so this is probably fine.

#3121 mentions some breaking changes to supported algorithms, hmm...

edit: actually, faraday seems to have no compatible versions for 2.5

@smortex
Copy link
Contributor Author

smortex commented Dec 16, 2022

According to net-ssh 7.0 gemspec:

  spec.required_ruby_version = Gem::Requirement.new(">= 2.6")

So with older versions of Ruby, bundler should resolve to version 6.x which is still allowed. I would expect it work seamlessly. That being said, Ruby 2.5 has reached EOL 20 months ago and does not receive security updates anymore. Support for it might be less important than support for modern setups.

@nicklewis
Copy link
Contributor

Bundler may do the right thing, but I believe gem install doesn't actually use Ruby version requirements to decide which gem version to install. It just fails if the version it resolves to doesn't support the Ruby version you're using. It's kind of a pain...

Puppet 8 is planned for February and will be shipping with Ruby 3.2 and dropping support for Ruby 2.5 and 2.6, along with upgrading to OpenSSL 3.0. Sounds like it's time for Bolt 4!

@smortex
Copy link
Contributor Author

smortex commented Dec 16, 2022

Bundler may do the right thing, but I believe gem install doesn't actually use Ruby version requirements to decide which gem version to install. It just fails if the version it resolves to doesn't support the Ruby version you're using. It's kind of a pain...

Ah I see what you mean… Indeed, installing out of bundler its a PITA, nobody wants to do that 😁

romain@buster:~/bolt$ gem build bolt.gemspec 
WARNING:  description and summary are identical
WARNING:  open-ended dependency on puppet (>= 6.18.0) is not recommended
  if puppet is semantically versioned, use:
    add_runtime_dependency 'puppet', '~> 6.18', '>= 6.18.0'
WARNING:  open-ended dependency on puppet-resource_api (>= 1.8.1) is not recommended
  if puppet-resource_api is semantically versioned, use:
    add_runtime_dependency 'puppet-resource_api', '~> 1.8', '>= 1.8.1'
WARNING:  open-ended dependency on bundler (>= 1.14, development) is not recommended
  if bundler is semantically versioned, use:
    add_development_dependency 'bundler', '~> 1.14'
WARNING:  See http://guides.rubygems.org/specification-reference/ for help
  Successfully built RubyGem
  Name: bolt
  Version: 3.26.2
  File: bolt-3.26.2.gem
root@buster:/home/romain/bolt# gem install bolt-3.26.2.gem 
ERROR:  Error installing bolt-3.26.2.gem:
	The last version of faraday-net_http (< 3.1, >= 2.0) to support your Ruby & RubyGems was 2.1.0. Try installing it with `gem install faraday-net_http -v 2.1.0` and then running the current command again
	faraday-net_http requires Ruby version >= 2.6.0. The current ruby version is 2.5.0.
root@buster:/home/romain/bolt# gem install faraday-net_http -v 2.1.0
Fetching: faraday-net_http-2.1.0.gem (100%)
Successfully installed faraday-net_http-2.1.0
Parsing documentation for faraday-net_http-2.1.0
Installing ri documentation for faraday-net_http-2.1.0
Done installing documentation for faraday-net_http after 0 seconds
1 gem installed
root@buster:/home/romain/bolt# gem install bolt-3.26.2.gem 
Fetching: faraday-2.7.2.gem (100%)
ERROR:  Error installing bolt-3.26.2.gem:
	There are no versions of faraday (~> 2.0) compatible with your Ruby & RubyGems. Maybe try installing an older version of the gem you're looking for?
	faraday requires Ruby version >= 2.6. The current ruby version is 2.5.0.
root@buster:/home/romain/bolt# gem install faraday -v 1.10.2
root@buster:/home/romain/bolt# gem install bolt-3.26.2.gem 
root@buster:/home/romain/bolt# gem install bolt-3.26.2.gem --ignore-dependencies
Successfully installed bolt-3.26.2
Parsing documentation for bolt-3.26.2
Installing ri documentation for bolt-3.26.2
Done installing documentation for bolt after 16 seconds
1 gem installed
root@buster:/home/romain/bolt# bolt
Traceback (most recent call last):
	8: from /usr/local/bin/bolt:23:in `<main>'
	7: from /usr/lib/ruby/2.5.0/rubygems.rb:304:in `activate_bin_path'
	6: from /usr/lib/ruby/2.5.0/rubygems.rb:304:in `synchronize'
	5: from /usr/lib/ruby/2.5.0/rubygems.rb:305:in `block in activate_bin_path'
	4: from /usr/lib/ruby/2.5.0/rubygems/specification.rb:1440:in `activate'
	3: from /usr/lib/ruby/2.5.0/rubygems/specification.rb:1458:in `activate_dependencies'
	2: from /usr/lib/ruby/2.5.0/rubygems/specification.rb:1458:in `each'
	1: from /usr/lib/ruby/2.5.0/rubygems/specification.rb:1469:in `block in activate_dependencies'
/usr/lib/ruby/2.5.0/rubygems/dependency.rb:310:in `to_specs': Could not find 'addressable' (~> 2.5) among 71 total gem(s) (Gem::MissingSpecError)
Checked in 'GEM_PATH=/root/.gem/ruby/2.5.0:/var/lib/gems/2.5.0:/usr/lib/x86_64-linux-gnu/rubygems-integration/2.5.0:/usr/share/rubygems-integration/2.5.0:/usr/share/rubygems-integration/all', execute `gem env` for more information
root@buster:/home/romain/bolt# gem install addressable
Fetching: public_suffix-5.0.1.gem (100%)
ERROR:  Error installing addressable:
	The last version of public_suffix (< 6.0, >= 2.0.2) to support your Ruby & RubyGems was 4.0.7. Try installing it with `gem install public_suffix -v 4.0.7` and then running the current command again
	public_suffix requires Ruby version >= 2.6. The current ruby version is 2.5.0.
root@buster:/home/romain/bolt# gem install public_suffix -v 4.0.7
Fetching: public_suffix-4.0.7.gem (100%)
Successfully installed public_suffix-4.0.7
Parsing documentation for public_suffix-4.0.7
Installing ri documentation for public_suffix-4.0.7
Done installing documentation for public_suffix after 0 seconds
1 gem installed
root@buster:/home/romain/bolt# gem install addressable
Fetching: addressable-2.8.1.gem (100%)
Successfully installed addressable-2.8.1
Parsing documentation for addressable-2.8.1
Installing ri documentation for addressable-2.8.1
Done installing documentation for addressable after 3 seconds
1 gem installed
root@buster:/home/romain/bolt# bolt
Traceback (most recent call last):
	8: from /usr/local/bin/bolt:23:in `<main>'
	7: from /usr/lib/ruby/2.5.0/rubygems.rb:304:in `activate_bin_path'
	6: from /usr/lib/ruby/2.5.0/rubygems.rb:304:in `synchronize'
	5: from /usr/lib/ruby/2.5.0/rubygems.rb:305:in `block in activate_bin_path'
	4: from /usr/lib/ruby/2.5.0/rubygems/specification.rb:1440:in `activate'
	3: from /usr/lib/ruby/2.5.0/rubygems/specification.rb:1458:in `activate_dependencies'
	2: from /usr/lib/ruby/2.5.0/rubygems/specification.rb:1458:in `each'
	1: from /usr/lib/ruby/2.5.0/rubygems/specification.rb:1469:in `block in activate_dependencies'
/usr/lib/ruby/2.5.0/rubygems/dependency.rb:310:in `to_specs': Could not find 'aws-sdk-ec2' (~> 1) among 73 total gem(s) (Gem::MissingSpecError)
Checked in 'GEM_PATH=/root/.gem/ruby/2.5.0:/var/lib/gems/2.5.0:/usr/lib/x86_64-linux-gnu/rubygems-integration/2.5.0:/usr/share/rubygems-integration/2.5.0:/usr/share/rubygems-integration/all', execute `gem env` for more information
root@buster:/home/romain/bolt# gem install aws-sdk-ec2
root@buster:/home/romain/bolt# gem install CFPropertyList -v 2.3.6
root@buster:/home/romain/bolt# gem install concurrent-ruby
root@buster:/home/romain/bolt# gem install hiera-eyaml
root@buster:/home/romain/bolt# gem install jwt
root@buster:/home/romain/bolt# gem install minitar
root@buster:/home/romain/bolt# gem install net-scp
ERROR:  Error installing net-scp:
	The last version of net-ssh (< 8.0.0, >= 2.6.5) to support your Ruby & RubyGems was 6.1.0. Try installing it with `gem install net-ssh -v 6.1.0` and then running the current command again
	net-ssh requires Ruby version >= 2.6. The current ruby version is 2.5.0.
root@buster:/home/romain/bolt# gem install net-scp -v 6.1.0
ERROR:  Could not find a valid gem 'net-scp' (= 6.1.0) in any repository
ERROR:  Possible alternatives: net-scp
root@buster:/home/romain/bolt# gem install net-ssh -v 6.1.0
root@buster:/home/romain/bolt# gem install net-scp -v 1.2.1
root@buster:/home/romain/bolt# gem install net-ssh-krb
root@buster:/home/romain/bolt# gem install orchestrator_client
root@buster:/home/romain/bolt# gem install puppet
root@buster:/home/romain/bolt# gem install puppetfile-resolver
root@buster:/home/romain/bolt# gem install puppet-strings -v 2.9.0
root@buster:/home/romain/bolt# gem install faraday_middleware -v 0.14.0
root@buster:/home/romain/bolt# gem install r10k
root@buster:/home/romain/bolt# bolt 
             `.::-`
        `.-:///////-.`
     `-:////:.  `-:///:-  /ooo.                        .ooo/
 `.-:///::///:-`   `-//:  ymmm-                        :mmmy  .---.
:///:-.   `.:////.  -//:  ymmm-                        :mmmy  +mmm+
://.          ///.  -//:  ymmm--/++/-       `-/++/:`   :mmmy-:smmms::-
://.          ://. .://:  ymmmdmmmmmmdo`  .smmmmmmmmh: :mmmysmmmmmmmms
://.          ://:///:-.  ymmmh/--/hmmmy -mmmd/-.:hmmm+:mmmy.-smmms--.
://:.`      .-////:-`     ymmm-     ymmm:hmmm-    `dmmm/mmmy  +mmm+
`-:///:-..:///:-.`        ymmm-     ommm/dmmm`     hmmm+mmmy  +mmm+
   `.-:////:-`            ymmm+    /mmmm.ommms`   /mmmh:mmmy  +mmmo
       `-.`               ymmmmmhhmmmmd:  ommmmhydmmmy`:mmmy  -mmmmdhd
                          oyyy+shddhs/`    .+shddhy+-  -yyyo   .ohddhs


🎉 Welcome to Bolt 3.26.2
😌 We're here to help bring order to the chaos
📖 Find our documentation at https://bolt.guide
🙋 Ask a question in #bolt on https://slack.puppet.com/
🔩 Contribute at https://github.com/puppetlabs/bolt/
💡 Not sure where to start? Try "bolt command run 'hostname' --target localhost"

We only print this message once. Run "bolt" again for help text.

Puppet 8 is planned for February and will be shipping with Ruby 3.2 and dropping support for Ruby 2.5 and 2.6, along with upgrading to OpenSSL 3.0. Sounds like it's time for Bolt 4!

Has the End-Of-Life date of older components been announced yet?

@smortex smortex changed the title Attempt to fix CI Fix CI Dec 17, 2022
@bastelfreak
Copy link
Contributor

Given that Ruby 2.5 and 2.6 are EoL, I think Bolt should drop support for it.

@smortex
Copy link
Contributor Author

smortex commented Jan 6, 2023

Given that Ruby 2.5 and 2.6 are EoL, I think Bolt should drop support for it.

Also, 2.7 is in end-of-support phase and only receive security updates until the end of march this year, so it probably makes sense to also drop support for a Ruby version that will be dead 1 month later [source].

@donoghuc
Copy link
Contributor

Thanks, I relaxed net-ssh in #3179 not quite sure why requiring the newer orchestrator-client is required?

@smortex
Copy link
Contributor Author

smortex commented Jan 23, 2023

@donoghuc I had to update both to fix CI, see for example:
https://github.com/puppetlabs/bolt/actions/runs/3715476307/jobs/6300691846

Will try to rebase #3170 on top of main to see if the situation improved.

@donoghuc
Copy link
Contributor

I had some green runs when I did my work. I think there is a new issue with puppet and concurrent ruby 1.2.0 that may pop up in our CI today 😭

@donoghuc
Copy link
Contributor

Sorry, I was vague in my last comment... I was not seeing the same failures you were regarding the orchestrator client. Thanks for the link to the test that was failing.

Unfortunately the https://tickets.puppetlabs.com/browse/PUP-11722 ticket (which I see you actually filed 😅 ) is going to be breaking bolt CI today. I suppose we can restrict concurrent ruby here until we get a puppet release out.

@bastelfreak
Copy link
Contributor

The concurrent-ruby problem got fixed \o/

@smortex
Copy link
Contributor Author

smortex commented Jan 31, 2023

Looks like main is now fixed and no need to tweak the conf for orchestrator-client anymore according to #3170. Closing.

@smortex smortex closed this Jan 31, 2023
@smortex smortex deleted the ci branch January 31, 2023 04:18
@donoghuc
Copy link
Contributor

Thanks for helping with this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bump to net-ssh 7.x

4 participants