Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename [database].ssl* options to [database].tls* to support pymongo 4 #6250

Merged
merged 15 commits into from
Sep 24, 2024

Conversation

cognifloyd
Copy link
Member

@cognifloyd cognifloyd commented Sep 24, 2024

pymongo 4 removes deprecated names for database connection options that were renamed some time ago. We even have a TODO about updating our usage of the deprecated names in the code:

# NOTE: In pymongo 3.9.0 some of the ssl related arguments have been renamed -
# https://api.mongodb.com/python/current/changelog.html#changes-in-version-3-9-0
# Old names still work, but we should eventually update to new argument names.

One change in particular could not be supported without breaking backwards compatibility: The files for ssl_keyfile and ssl_certfile must be concatenated and passed as one file in the new tls_certificate_key_file option.

Why update to pymongo 4 (in a follow-up PR)? We need to update pymongo to ensure we're using a version that tests with and officially supports our target MongoDB version(s) (we are planning on using MongoDB 7; see #6246 and #6236). The pymongo4 upgrade guide has details on the option naming migration.

Each commit touches one option or aspect of this migration, so it will be useful to review each commit.

Since we're using newer oslo.config now, we can also be more explicit about deprecations when defining the options. So, a few of the commits make use of those newer features to improve our st2.conf.sample file.

In summary, these options were migrated:

  • 3622868 ssl -> tls
  • dc62d11 ssl_keyfile + ssl_certfile -> tls_certificate_key_file (files must be concatenated)
    • Also add tls_certificate_key_file_password
  • 2094ff4 ssl_cert_reqs -> tls_allow_invalid_certificates (from a string opt to a bool opt)
  • e610bdc ssl_ca_certs -> tls_ca_file
  • c962d3c ssl_match_hostnames -> tls_allow_invalid_hostnames (inverted meaning)

Note: In #6246, I initially developed this using mongo's camelCase naming convention in st2.conf. After discussing with @nzlosh I went back to using snake_case to be consistent with the rest of the st2.conf options.

…l_certfile

pymongo 4 will ignore the ssl_keyfile/ssl_certfile options.

For consistency in st2.conf, this uses snake_case not the mongo camelCase option name.

This also adds tls_certificate_key_file_password.
We did not support ssl_pem_passphrase before, so there was nothing to migrate.
This needed to be a different option (instead of just renaming) because
the option type is changing from str+choices to a bool.

For consistency in st2.conf, this uses snake_case not the mongo
camelCase option name.
For consistency in st2.conf, this uses snake_case not the mongo
camelCase option name.
…tname

For consistency in st2.conf, this uses snake_case not the mongo
camelCase option name.
Not sure if this wasn't available before, or why it wasn't used. Try and see.
And use fix the sample default for python_binary to use python3.
@cognifloyd cognifloyd added this to the 3.9.0 milestone Sep 24, 2024
@cognifloyd cognifloyd self-assigned this Sep 24, 2024
@pull-request-size pull-request-size bot added the size/XL PR that changes 500-999 lines. Consider splitting work into several ones that easier to review. label Sep 24, 2024
@cognifloyd cognifloyd marked this pull request as ready for review September 24, 2024 03:37
@cognifloyd cognifloyd requested review from amanda11 and a team September 24, 2024 03:37
@@ -64,7 +64,7 @@ async def generate_sample_conf_via_fmt(
pex = await Get(VenvPex, PexFromTargetsRequest, subsystem.pex_request())

result = await Get(
FallibleProcessResult,
ProcessResult,
Copy link
Member Author

@cognifloyd cognifloyd Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change prevents the conf/st2.conf.sample from being replaced if tools/config_gen.py exits with an error. It also makes pants report the error by printing stdout from the process, which includes the traceback.

Copy link
Contributor

@nzlosh nzlosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job!

@cognifloyd cognifloyd requested a review from a team September 24, 2024 06:44
@cognifloyd cognifloyd merged commit 828e49f into master Sep 24, 2024
36 checks passed
@cognifloyd cognifloyd deleted the rename-db-tls-opts branch September 24, 2024 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change enhancement external dependency migrations mongodb size/XL PR that changes 500-999 lines. Consider splitting work into several ones that easier to review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants