Indicate the PyPi package doesn't have a plugin #358
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
justinvp
force-pushed
the
justin/pulumiplugin
branch
from
80ffee8 to
0839564
Compare
This was referenced Jul 20, 2024
Frassle
approved these changes
Jul 20, 2024
github-merge-queue bot
pushed a commit
to pulumi/pulumi
that referenced
this pull request
Jul 22, 2024
In the Python language host, we hardcode that `pulumi-policy` doesn't have an associated resource (provider) plugin, because we know it doesn't have one. However, this hardcode no longer works with the latest version of `pulumi-policy` (v1.11.0) because it was built with a newer version of `setuptools` which has a behavior change where the package name in the metadata will now allow underscores, instead of having underscores replaced with hyphens (pypa/setuptools#4159). This means that the package name reported from `pip list` is now `pulumi_policy` instead of `pulumi-policy`, which doesn't match the hardcoded list. Note that this change is really only to help with `pulumi-policy` v1.11.0. Future versions of `pulumi-policy` will have a `pulumi-plugin.json` file in the package, which properly indicates that it doesn't have an associated plugin. Related: pulumi/pulumi-policy#358 Part of addressing: pulumi/pulumi-policy#356
github-merge-queue bot
pushed a commit
to pulumi/pulumi
that referenced
this pull request
Jul 22, 2024
In the Python language host, we hardcode that `pulumi-policy` doesn't have an associated resource (provider) plugin, because we know it doesn't have one. However, this hardcode no longer works with the latest version of `pulumi-policy` (v1.11.0) because it was built with a newer version of `setuptools` which has a behavior change where the package name in the metadata will now allow underscores, instead of having underscores replaced with hyphens (pypa/setuptools#4159). This means that the package name reported from `pip list` is now `pulumi_policy` instead of `pulumi-policy`, which doesn't match the hardcoded list. Note that this change is really only to help with `pulumi-policy` v1.11.0. Future versions of `pulumi-policy` will have a `pulumi-plugin.json` file in the package, which properly indicates that it doesn't have an associated plugin. Related: pulumi/pulumi-policy#358 Part of addressing: pulumi/pulumi-policy#356
github-merge-queue bot
pushed a commit
to pulumi/pulumi
that referenced
this pull request
Jul 22, 2024
In the Python language host, we hardcode that `pulumi-policy` doesn't have an associated resource (provider) plugin, because we know it doesn't have one. However, this hardcode no longer works with the latest version of `pulumi-policy` (v1.11.0) because it was built with a newer version of `setuptools` which has a behavior change where the package name in the metadata will now allow underscores, instead of having underscores replaced with hyphens (pypa/setuptools#4159). This means that the package name reported from `pip list` is now `pulumi_policy` instead of `pulumi-policy`, which doesn't match the hardcoded list. Note that this change is really only to help with `pulumi-policy` v1.11.0. Future versions of `pulumi-policy` will have a `pulumi-plugin.json` file in the package, which properly indicates that it doesn't have an associated plugin. Related: pulumi/pulumi-policy#358 Part of addressing: pulumi/pulumi-policy#356
Pulumi automatically determines required plugins for a program. For Python programs, it essentially does this by running `python -m pip list --format json`, and any packages prefixed with `pulumi-` or `pulumi_` are assumed to have associated plugins, unless the package includes a `pulumi-plugin.json` file that has indicated there is no plugin via `{ "resource": false }`.
When the `GetRequiredPlugins` support was originally added to the Python language host, it [hardcoded](https://github.com/pulumi/pulumi/blob/e6d20d26f7f64a624238f86204d862642ff27e16/sdk/python/cmd/pulumi-language-python/main.go#L428-L430) that `pulumi-policy` did not have an associated plugin. The hardcode mainly for older versions of `pulumi-policy` that did not contain a `pulumi-plugin.json` file.
`pulumi-plugin.json` was actually originally named `pulumiplugin.json` (no dash). This file wasn't used anywhere, aside from in `pulumi-policy`, but the hardcod prevented it from ever being loaded in that case. It turned out that there was a bug parsing `pulumiplugin.json` that caused the CLI to error when that file existed in other packages. Since we were planning to expand the use of the file to other languages, and make it generated by default by SDKgen, we changed the name from `pulumiplugin.json` to `pulumi-plugin.json` to avoid breaking older CLIs with the bug (see pulumi/pulumi#8593).
After making that change, we never followed up to rename the `pulumiplugin.json` file in `pulumi-policy` to `pulumi-plugin.json`, largely because it didn't matter since we had the hardcode.
However, this hardcode no longer works with the latest version of `pulumi-policy` (v1.11.0). This version was built with a newer version of `setuptools` which has a behavior change where the package name in the metadata will now allow underscores, instead of having underscores replaced with dashes (pypa/setuptools#4159). This means that the package name reported from `pip list` is now `pulumi_policy` instead of `pulumi-policy`, which doesn't match the hardcoded list. And since there is no `pulumi-plugin.json` file in the package (it's still the old `pulumiplugin.json` name), the Pulumi Python language host thinks this package needs a plugin and tries to retrieve one that doesn't exist.
This change addresses the issue by renaming `pulumiplugin.json` to `pulumi-plugin.json` in the package.
justinvp
force-pushed
the
justin/pulumiplugin
branch
from
0839564 to
639b4f7
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is stacked on top of #357, which should be merged first1.
Pulumi automatically determines required plugins for a program. For Python programs, it essentially does this by running
python -m pip list --format json, and any packages prefixed withpulumi-orpulumi_are assumed to have associated plugins, unless the package includes apulumi-plugin.jsonfile that has indicated there is no plugin via{ "resource": false }.When the
GetRequiredPluginssupport was originally added to the Python language host, it hardcoded thatpulumi-policydid not have an associated plugin. The hardcode mainly for older versions ofpulumi-policythat did not contain apulumi-plugin.jsonfile.pulumi-plugin.jsonwas actually originally namedpulumiplugin.json(no dash). This file wasn't used anywhere, aside from inpulumi-policy, but the hardcod prevented it from ever being loaded in that case. It turned out that there was a bug parsingpulumiplugin.jsonthat caused the CLI to error when that file existed in other packages. Since we were planning to expand the use of the file to other languages, and make it generated by default by SDKgen, we changed the name frompulumiplugin.jsontopulumi-plugin.jsonto avoid breaking older CLIs with the bug (see pulumi/pulumi#8593).After making that change, we never followed up to rename the
pulumiplugin.jsonfile inpulumi-policytopulumi-plugin.json, largely because it didn't matter since we had the hardcode.However, this hardcode no longer works with the latest version of
pulumi-policy(v1.11.0). This version was built with a newer version ofsetuptoolswhich has a behavior change where the package name in the metadata will now allow underscores, instead of having underscores replaced with dashes (pypa/setuptools#4159). This means that the package name reported frompip listis nowpulumi_policyinstead ofpulumi-policy, which doesn't match the hardcoded list. And since there is nopulumi-plugin.jsonfile in the package (it's still the oldpulumiplugin.jsonname), the Pulumi Python language host thinks this package needs a plugin and tries to retrieve one that doesn't exist.This change addresses the issue by renaming
pulumiplugin.jsontopulumi-plugin.jsonin the package.Fixes #356
Footnotes
[sdk/python] Allow editable installs without build step #357 makes it possible to actually test this, because we need a simple version like
1.0.0rather than1.12.0a1721429785for the locally installed package in the test, in order for the Python language host'sGetRequiredPluginsnot to skip the package. ↩