pantsbuild: Generate PEX lockfile for st2

[cognifloyd]

Background

This is another part of introducing pants, as discussed in various TSC meetings.

Related PRs can be found in:

• pantsbuild milestone (which includes PRs since Initialize pantsbuild #5713)
• pantsbuild label (which also includes preparatory PRs that came before adding pantsbuild)

Overview of this PR

This introduces a lockfile for our 3rd party dependencies (based on dependencies/constraints from #5789 and python constraints from #5795).

To create this lockfile, we need to register a "resolve" (called "st2") in pants, which represents the wold of direct and transitive dependencies required by our code. We also register the dependency constraints file added in #5789.

Finally, we create the lockfile by running ./pants generate-lockfiles --resolve=st2.

Note that the lockfile has 4840 lines. Beyond that this PR only:

• adds 11 lines to pants.toml
• updates 1 line in CHANGELOG.rst

Relevant Pants documentation

• Python 3rd Party Dependencies
  • Python Lockfiles
  • Python dependency constraints
  • Manually Generating Lockfiles
• Python Interpreter Compatibility
• Goals:
  • generate-lockfiles
• pants.toml options:
  • [python].enable_resolves = <bool>
  • [python.resolves].<resolve name> = "<lockfile path>"
  • [python].default_resolve = "<resolve name>"
  • [python.resolves_to_constraints_file].<resolve name> = "<constraints file path>"

Resolves and Lockfiles

Pants has multiple resolves, where a resolve represents the world of dependencies for some tool or code. Each resolve can have a lockfile associated with it. All tools (like black, flake8, etc) have lockfiles by default, but first-party code only gets a lockfile if we enable resolves and add a lockfile for it. So, we:

• enable resolves by setting [python].enable_resolves = true in pants.toml;
• name the resolve for our code "st2" instead of defaulting to the name "python-default" by setting [python].default-resolve = "st2";
• register the st2 resolve's lockfile by setting [python.resolves].st2 = "lockfiles/st2.lock"; and
• register the st2 resolve's constraints file by setting [python.resolves_to_constraints_file].st2 = "lockfiles/st2-constraints.txt".

Finally, generate the lockfile with ./pants generate-lockfiles --resolve=st2.

We can also regenerate all lockfiles (st2 and tools) with ./pants generate-lockfiles.

[cognifloyd]

Our current infra relies on the legacy resolver and old versions of pip. Old versions of pants also supported the older pip + legacy resolver, but the version we're using now does not.

So, I left this comment here for future reference and to call out this departure from how we used to do things.

[nzlosh]

Nice job!

[Eric-Arellano]

Woohoo!

[rush-skills]

LGTM