(PE-36348) Enable legacy openssl algos in PE Installer runtime to support Bolt's WinRM transport #699

justinstoller · 2023-07-12T17:40:40Z

Implements the solution described in https://tickets.puppetlabs.com/browse/PE-36078

Still needs building & testing but putting this up so I can verify I understand the basics correctly.

justinstoller · 2023-07-12T17:43:46Z

configs/components/openssl-3.0.rb

+  if settings[:use_legacy_openssl_algos]
+    pkg.apply_patch 'resources/patches/openssl/openssl-3-activate-legacy-algos.patch'
+  else
+    configure_flags << 'no-legacy' << 'no-md4' << 'no-des'


This ticket (https://tickets.puppetlabs.com/browse/PE-36078) enumerates the changes need, and it says "no-des" should be included when legacy algos are not enabled, but.... it wasn't included before.

I added it thinking it might have been an oversight in the original Puppet 8 work, but would love to know if this is overkill and/or actively not wanted in the puppet-runtime. /cc @joshcooper

It like you read my mind 😁 We had plans on adding no-des in https://tickets.puppetlabs.com/browse/PA-5587, but put it on hold when PE started failing due to no-md4. But now that you've detangled that, I'm 👍 to this PR

justinstoller · 2023-07-12T17:56:26Z

Note: there's a settings[:openssl_extra_configure_flags] array that it seems has been copy pasted into a lot of different projects, but it seemed incredibly hard to reason about. I found adding a toggle for a discrete set of configure flags much less of a cognitive load.

justinstoller · 2023-07-12T18:47:41Z

this builds without error and the resulting tarball has the openssl.cnf changes specified in the ticket:

$ nvim opt/puppetlabs/installer/ssl/openssl.cnf
...snip....
[openssl_init]
providers = provider_sect

# List of providers to load
[provider_sect]
default = default_sect
legacy = legacy_sect
# The fips section name should match the section name inside the
# included fipsmodule.cnf.
# fips = fips_sect

# If no providers are activated explicitly, the default one is activated implicitly.
# See man 7 OSSL_PROVIDER-default for more details.
#
# If you add a section explicitly activating any other provider(s), you most
# probably need to explicitly activate the default provider, otherwise it
# becomes unavailable in openssl.  As a consequence applications depending on
# OpenSSL may not work correctly which could lead to significant system
# problems including inability to remotely access the system.
[default_sect]
activate = 1

[legacy_sect]
activate = 1

...snip...

Still figuring out how to work that into a PE build.

justinstoller · 2023-07-14T19:04:31Z

Using a pe-installer built with this runtime on Redhat 8 and a Windows 2022 agent I think I've verified this works as intended:

[root@icy-protection check]# puppet infrastructure run regenerate_agent_certificate  --use-winrm --username Administrator --password '******' --no-ssl-verify

...snip...

Agent_cert_regen: Verifying certs on 1 node(s)
Starting: command 'C:\Program` Files\Puppet` Labs\Puppet\bin/puppet.bat ssl verify' on huge-numerology.delivery.puppetlabs.net
Finished: command 'C:\Program` Files\Puppet` Labs\Puppet\bin/puppet.bat ssl verify' with 0 failures in 3.6 sec
Agent_cert_regen: Restarting pxp-agent on 1 node(s)
Starting: task service on huge-numerology.delivery.puppetlabs.net
Finished: task service with 0 failures in 7.43 sec
Starting: wait until available on huge-numerology.delivery.puppetlabs.net
Finished: wait until available with 0 failures in 0.43 sec
Agent_cert_regen: Restarting puppet service on 1 node(s) that had it running before the plan was run
Starting: task service on huge-numerology.delivery.puppetlabs.net
Finished: task service with 0 failures in 11.56 sec
Starting: task enterprise_tasks::enable_agent on localhost, huge-numerology.delivery.puppetlabs.net
Finished: task enterprise_tasks::enable_agent with 0 failures in 9.17 sec
Finished: plan regenerate_agent_certificate in 1 min, 51 sec
Plan completed successfully with no result

justinstoller · 2023-07-14T19:10:03Z

Weeelllll, I guess that all might be because I'm still shipping the forked gem that vendors the a ruby implementation of md4. I'll put up a PR with that pulled out and then redo this testing.

justinstoller · 2023-07-14T21:52:19Z

Okay the newest package installs what I believe is the proper rubyntlm gem, one that does not have the md4.rb file in it:

[root@helpful-wallop ~]# ls /opt/puppetlabs/installer/lib/ruby/gems/3.2.0/gems/rubyntlm-0.6.3/lib/net/ntlm/
blob.rb		   client     encode_util.rb  field.rb      int16_le.rb  int64_le.rb  message.rb	  string.rb	  version.rb
channel_binding.rb  client.rb  exceptions.rb   field_set.rb  int32_le.rb  message      security_buffer.rb  target_info.rb

And can execute the puppet infrastructure run correctly over both ssh and winrm with commands like:

[root@helpful-wallop ~]# puppet infrastructure run regenerate_agent_certificate --use-ssh --username root --no-ssl-verify
....
[root@helpful-wallop ~]# puppet infrastructure run regenerate_agent_certificate --use-winrm --username Administrator --password 'Qu@lity!' --no-ssl-verify

(One caveat being I could not get it to pass in the list of agents I wanted to so I manually edited the $agents variable in /opt/puppetlabs/installer/share/Boltdir/site-modules/enterprise_tasks/plans/agent_cert_regen.pp)

I'll set the lifetime of these nodes through next Monday so others can inspect the boxes if they want. (console login is admin:password)

…to support Bolt/WinRM

This reverts commit 0ebee8c.

donoghuc

Looks great! I'm not sure about the des algorithm. I thought it was one of the algos in question to be able to continue to support building or running postrgres. But i cant seem to find that info. We will see what Josh has to say on that but I dont think we need to block on that. I'm going to go ahead and get this merged so we can get tests/promotions going.

joshcooper · 2023-07-17T20:04:53Z

Not sure if you've seen this, but Gabi wrote a handy tool to check what projects and/or platforms will be affected. For this PR, I see the following project changes, which I think is ok/expected, but wanted to double check?

$ bundle exec  rake vanagon:component_diff -- -m -P all -p el-7-x86_64 --from 5172806 --to HEAD

Here is what your code changes would affect:

Project `pe-installer-runtime-main`

Platform name: `el-7-x86_64`

Component 'rubygem-rubyntlm-fork' was removed, not showing diff for it ❌

Component 'openssl-3.0'

Field: configure[0]

- [" ./Configure --prefix=/opt/puppetlabs/installer --libdir=lib --openssldir=/opt/puppetlabs/installer/ssl shared no-gost linux-x86_64  no-camellia no-md2 no-ssl3 no-ssl3-method no-dtls1-method no-dtls1_2-method no-aria no-rc5 no-mdc2 no-whirlpool no-legacy no-dtls no-dtls1 no-idea no-seed no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS"]
+ [" ./Configure --prefix=/opt/puppetlabs/installer --libdir=lib --openssldir=/opt/puppetlabs/installer/ssl shared no-gost linux-x86_64  no-camellia no-md2 no-ssl3 no-ssl3-method no-dtls1-method no-dtls1_2-method no-aria no-rc5 no-mdc2 no-whirlpool no-dtls no-dtls1 no-idea no-seed no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS"]

Field: patches[0]

+ {"origin_path"=>"resources/patches/openssl/openssl-3-activate-legacy-algos.patch", "namespace"=>"openssl-3.0", "assembly_path"=>"patches/openssl-3.0/openssl-3-activate-legacy-algos.patch", "strip"=>1, "fuzz"=>0, "after"=>"unpack", "destination"=>nil}

Project `pe-bolt-server-runtime-main`

Nothing is affected 😊

Project `agent-runtime-7.x`

Nothing is affected 😊

Project `pe-bolt-server-runtime-2021.7.x`

Nothing is affected 😊

Project `pe-bolt-server-runtime-2019.8.x`

Nothing is affected 😊

Project `pe-installer-runtime-2019.8.x`

Nothing is affected 😊

Project `pe-installer-runtime-2021.7.x`

Nothing is affected 😊

Project `agent-runtime-6.x`

Nothing is affected 😊

Project `bolt-runtime`

Nothing is affected 😊

Project `pdk-runtime`

Platform name: `el-7-x86_64`

Component 'openssl-3.0'

Field: configure[0]

- [" ./Configure --prefix=/opt/puppetlabs/pdk --libdir=lib --openssldir=/opt/puppetlabs/pdk/ssl shared no-gost linux-x86_64  no-camellia no-md2 no-ssl3 no-ssl3-method no-dtls1-method no-dtls1_2-method no-aria no-rc5 no-mdc2 no-whirlpool no-legacy no-dtls no-dtls1 no-idea no-seed no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS"]
+ [" ./Configure --prefix=/opt/puppetlabs/pdk --libdir=lib --openssldir=/opt/puppetlabs/pdk/ssl shared no-gost linux-x86_64  no-camellia no-md2 no-ssl3 no-ssl3-method no-dtls1-method no-dtls1_2-method no-aria no-rc5 no-mdc2 no-whirlpool no-legacy no-md4 no-des no-dtls no-dtls1 no-idea no-seed no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS"]

Project `client-tools-runtime-main`

Nothing is affected 😊

Project `client-tools-runtime-2019.8.x`

Nothing is affected 😊

Project `client-tools-runtime-2021.7.x`

Nothing is affected 😊

Project `agent-runtime-main`

Platform name: `el-7-x86_64`

Component 'openssl-3.0'

Field: configure[0]

- [" ./Configure --prefix=/opt/puppetlabs/puppet --libdir=lib --openssldir=/opt/puppetlabs/puppet/ssl shared no-gost linux-x86_64  no-camellia no-md2 no-ssl3 no-ssl3-method no-dtls1-method no-dtls1_2-method no-aria no-rc5 no-mdc2 no-whirlpool no-legacy no-dtls no-dtls1 no-idea no-seed no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS"]
+ [" ./Configure --prefix=/opt/puppetlabs/puppet --libdir=lib --openssldir=/opt/puppetlabs/puppet/ssl shared no-gost linux-x86_64  no-camellia no-md2 no-ssl3 no-ssl3-method no-dtls1-method no-dtls1_2-method no-aria no-rc5 no-mdc2 no-whirlpool no-legacy no-md4 no-des no-dtls no-dtls1 no-idea no-seed no-weak-ssl-ciphers -DOPENSSL_NO_HEARTBEATS"]

joshcooper · 2023-07-18T03:54:09Z

Unfortunately seeing errors when building curl on most of the puppet8 agent runtimes:

https://jenkins-platform.delivery.puppetlabs.net/view/puppet-agent/view/Acceptance%20Suites/view/CI%20Goalie/job/platform_agent-runtime_runtime-vanagon-packaging_main/447/

18:18:31 curl_ntlm_core.c:115:4: error: #error "Can't compile NTLM support without a crypto library with DES."
18:18:31  #  error "Can't compile NTLM support without a crypto library with DES."
18:18:31     ^~~~~
18:18:31   CC       libcurl_la-curl_path.lo

Specifying -DCURL_DISABLE_NTLM for the curl component should resolve that https://github.com/curl/curl/blob/master/docs/CURL-DISABLE.md#curl_disable_ntlm

justinstoller · 2023-07-18T13:48:17Z

Okay, I'll open a PR to fix.

justinstoller · 2023-07-18T15:44:17Z

Should be fixed in #702

justinstoller requested review from a team as code owners July 12, 2023 17:40

justinstoller commented Jul 12, 2023

View reviewed changes

justinstoller mentioned this pull request Jul 12, 2023

(PE-36345) Add ruby/openssl to bolt-server main #697

Merged

justinstoller changed the title ~~[wip] (PE-36348) Enable legacy openssl algos in PE Installer runtime to support Bolt's WinRM transport~~ (PE-36348) Enable legacy openssl algos in PE Installer runtime to support Bolt's WinRM transport Jul 14, 2023

justinstoller added 4 commits July 14, 2023 14:54

(PE-36348) Set use_legacy_openssl_algos in pe-installer-runtime-main …

039c2ed

…to support Bolt/WinRM

(PE-36348) Honor use_legacy_openssl_algos in openssl 3 configure flags

b85bf0f

(PE-36348) Activate legacy section in openssl.cnf if needed

c7ddc67

Revert "(PE-36078) Use fork of ntlm with md4 ruby algo"

d993409

This reverts commit 0ebee8c.

justinstoller force-pushed the PE-36348 branch from 50acc29 to d993409 Compare July 14, 2023 21:54

justinstoller mentioned this pull request Jul 14, 2023

(PE-36346) Use legacy openssl algos for pe-bolt-server's openssl mcdonaldseanp/puppet-runtime#8

Closed

donoghuc approved these changes Jul 17, 2023

View reviewed changes

donoghuc merged commit d31b127 into puppetlabs:master Jul 17, 2023
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(PE-36348) Enable legacy openssl algos in PE Installer runtime to support Bolt's WinRM transport #699

(PE-36348) Enable legacy openssl algos in PE Installer runtime to support Bolt's WinRM transport #699

justinstoller commented Jul 12, 2023 •

edited

Loading

justinstoller Jul 12, 2023

joshcooper Jul 17, 2023

justinstoller commented Jul 12, 2023

justinstoller commented Jul 12, 2023

justinstoller commented Jul 14, 2023

justinstoller commented Jul 14, 2023

justinstoller commented Jul 14, 2023

donoghuc left a comment

joshcooper commented Jul 17, 2023

joshcooper commented Jul 18, 2023

justinstoller commented Jul 18, 2023

justinstoller commented Jul 18, 2023

(PE-36348) Enable legacy openssl algos in PE Installer runtime to support Bolt's WinRM transport #699

(PE-36348) Enable legacy openssl algos in PE Installer runtime to support Bolt's WinRM transport #699

Conversation

justinstoller commented Jul 12, 2023 • edited Loading

justinstoller Jul 12, 2023

Choose a reason for hiding this comment

joshcooper Jul 17, 2023

Choose a reason for hiding this comment

justinstoller commented Jul 12, 2023

justinstoller commented Jul 12, 2023

justinstoller commented Jul 14, 2023

justinstoller commented Jul 14, 2023

justinstoller commented Jul 14, 2023

donoghuc left a comment

Choose a reason for hiding this comment

joshcooper commented Jul 17, 2023

Here is what your code changes would affect:

Project pe-installer-runtime-main

Platform name: el-7-x86_64

Project pe-bolt-server-runtime-main

Project agent-runtime-7.x

Project pe-bolt-server-runtime-2021.7.x

Project pe-bolt-server-runtime-2019.8.x

Project pe-installer-runtime-2019.8.x

Project pe-installer-runtime-2021.7.x

Project agent-runtime-6.x

Project bolt-runtime

Project pdk-runtime

Platform name: el-7-x86_64

Project client-tools-runtime-main

Project client-tools-runtime-2019.8.x

Project client-tools-runtime-2021.7.x

Project agent-runtime-main

Platform name: el-7-x86_64

joshcooper commented Jul 18, 2023

justinstoller commented Jul 18, 2023

justinstoller commented Jul 18, 2023

justinstoller commented Jul 12, 2023 •

edited

Loading

Project `pe-installer-runtime-main`

Platform name: `el-7-x86_64`

Project `pe-bolt-server-runtime-main`

Project `agent-runtime-7.x`

Project `pe-bolt-server-runtime-2021.7.x`

Project `pe-bolt-server-runtime-2019.8.x`

Project `pe-installer-runtime-2019.8.x`

Project `pe-installer-runtime-2021.7.x`

Project `agent-runtime-6.x`

Project `bolt-runtime`

Project `pdk-runtime`

Platform name: `el-7-x86_64`

Project `client-tools-runtime-main`

Project `client-tools-runtime-2019.8.x`

Project `client-tools-runtime-2021.7.x`

Project `agent-runtime-main`

Platform name: `el-7-x86_64`