Include API Version in input/state

[EronWright]

Overview

Closes #3766

This PR is aimed at improving the experience around api versions, e.g. when upgrading a resource to a new api version. Note that major version upgrades of this provider tend to change the 'default' versions of many resources.

1. Show some diff output and and actually update the resource state when the api version of a resource is being changed.
2. Record the actual api version that was used in create/update/read, to inform subsequent operations.

In 2.x mode, the provider simply records the actual azureApiVersion as an output property, but doesn't expose it at the SDK level. The rationale is to have the information available in "old state" +before+ the major upgrade is performed; this allows the provider to avoid a spurious diff and to make fewer update calls during the major upgrade.

Note: these screenshots are outdated; apiVersion is now azureApiVersion.

In 3.x mode, the provider introduces a new azureApiVersion input property, with a constant value. The input property is NOT added to the resource args; it is always defaulted in Check. In the future, a true input property might be added to support overrides. The purpose of having a checked input is to produce a diff when the API version has changed (for whatever reason). This leads to an update call against the new API version of the resource, to validate the inputs and refresh the state.

The output property that was added to state in 2.x is fully exposed in 3.x at the SDK level.

Migration

In a migration case, the old azureApiVersion isn't yet known, and this may lead to a spurious (though harmless) update because the value goes from empty to known. We mitigate this by reading azureApiVersion from old state during Diff. The recommended migration flow is:

1. User upgrades to the latest 2.x with pulumi up (to update the provider version) followed by pulumi refresh (to update the resource's state. Now there's an azureApiVersion in state for each 2.x resource.
2. User upgrades to 3.0 with pulumi up, and the provider compares old state to new input to see whether the api version is actually changing, e.g. due to a change to the "default version" of a resource.

For example, here's the impact to the state file after completing the migration as described. We see two affected resources: a resource group and a storage account, with the latter undergoing a change to the 'default' API version.

Invokes

Another aspect to this PR is the azureApiVersion output property and supporting the getFoo functions. The code generator must be careful to apply the output property to the resource type (as represented by isResourceGetter and isTopLevel), not to functions. There is, however, a special "get" function where it does make sense to return the azureApiVersion.

Provider Version

This PR ensures that the provider's version field is used by its methods, rather than the static version.

[github-actions]

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

[codecov]

Codecov Report

Attention: Patch coverage is 81.57895% with 14 lines in your changes missing coverage. Please review.

Project coverage is 57.74%. Comparing base (3d36516) to head (632c3b1).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
provider/pkg/gen/properties.go	0.00%	11 Missing and 1 partial ⚠️
provider/pkg/provider/provider.go	95.45%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4015      +/-   ##
==========================================
+ Coverage   57.53%   57.74%   +0.21%     
==========================================
  Files          82       82              
  Lines       13099    13144      +45     
==========================================
+ Hits         7536     7590      +54     
+ Misses       4987     4976      -11     
- Partials      576      578       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[EronWright]

Some results using this test program.

In upgrade case, the user sees no changes on their next up (--expect-no-changes), aside from some new live state from the existing resource (because it is an external resource). Notably, we do not see new outputs for the other resources, because they are not (yet) refreshed.

❯ pulumi up --expect-no-changes
Previewing update (v2)

View in Browser (Ctrl+O): https://app.pulumi.com/eron-pulumi-corp/issue-3766/v2/previews/e84e78bb-30db-497c-94c9-b92c617bd61a

     Type                 Name                 Plan     Info
     pulumi:pulumi:Stack  issue-3766-v2           3 warnings

Resources:
    5 unchanged

Do you want to perform this update? details
  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:v2::issue-3766::pulumi:pulumi:Stack::issue-3766-v2]
    > azure-native:storage/v20230501:StorageAccount: (read)
        [id=/subscriptions/0282681f-7a9e-424b-80b2-96babd57a8a1/resourceGroups/pulumi-dev-shared/providers/Microsoft.Storage/storageAccounts/pulumitesting]
        [urn=urn:pulumi:v2::issue-3766::azure-native:storage/v20230501:StorageAccount::existing]
        [provider=urn:pulumi:v2::issue-3766::pulumi:providers:azure-native::default_2_84_0::6d9de670-b436-4084-84a1-22e3a859c3e0]
        --outputs:--
      + azureApiVersion             : "2023-05-01"

Do you want to perform this update? yes
Updating (v2)

View in Browser (Ctrl+O): https://app.pulumi.com/eron-pulumi-corp/issue-3766/v2/updates/2

     Type                 Name                 Status     Info
     pulumi:pulumi:Stack  issue-3766-v2            3 warnings

Outputs:
    existingId            : "/subscriptions/0282681f-7a9e-424b-80b2-96babd57a8a1/resourceGroups/pulumi-dev-shared/providers/Microsoft.Storage/storageAccounts/pulumitesting"
    storageAccountLocation: "westus2"

Now we refresh and see no change:

Refreshing (v2-final)

View in Browser (Ctrl+O): https://app.pulumi.com/eron-pulumi-corp/issue-3766/v2-final/updates/4

     Type                                              Name                 Status     Info
     pulumi:pulumi:Stack                               issue-3766-v2-final             3 warnings
     ├─ azure-native:resources:ResourceGroup           resourceGroup                   
     ├─ azure-native:storage:Blob                      blob                            
     ├─ azure-native:storage/v20230501:BlobContainer   container                       
     ├─ azure-native:storage/v20230501:StorageAccount  sa                              
     └─ azure-native:storage/v20230501:StorageAccount  existing                        

Outputs:
    existingId            : "/subscriptions/0282681f-7a9e-424b-80b2-96babd57a8a1/resourceGroups/pulumi-dev-shared/providers/Microsoft.Storage/storageAccounts/pulumitesting"
    storageAccountLocation: "westus2"

Resources:
    6 unchanged

Looking at the state file, we see azureApiVersion as an output property across all resources.

[pulumi-bot]

This PR has been shipped in release v2.89.2.