Refactor versioning process by danielrbradley · Pull Request #3714 · pulumi/pulumi-azure-native

danielrbradley · 2024-11-18T18:06:01Z

Copy squeeze code from schema-tools.
Combine gen raw-schema + schema-tools squeeze into gen squeeze.
Output removed invokes from squeeze & remove from BuildSchema.
Tidy and simplify BuildSchema internal methods.
Add unit tests as documentation/spec.
Fix bug to get tests to pass.

Precursor to implementing #2661

github-actions · 2024-11-18T18:11:28Z

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

codecov · 2024-11-18T20:32:45Z

Codecov Report

Attention: Patch coverage is 18.66029% with 170 lines in your changes missing coverage. Please review.

Please upload report for BASE (master@112225d). Learn more about missing BASE report.
Report is 12 commits behind head on master.

Files with missing lines	Patch %	Lines
provider/pkg/squeeze/squeeze.go	17.51%	143 Missing and 3 partials ⚠️
provider/pkg/versioning/gen.go	27.77%	13 Missing ⚠️
provider/cmd/pulumi-gen-azure-native/main.go	0.00%	10 Missing ⚠️
provider/pkg/versioning/build_schema.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##             master    #3714   +/-   ##
=========================================
  Coverage          ?   56.31%           
=========================================
  Files             ?       74           
  Lines             ?    11735           
  Branches          ?        0           
=========================================
  Hits              ?     6609           
  Misses            ?     4635           
  Partials          ?      491

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests
JS Bundle Analysis - Avoid shipping oversized bundles

- Remove cobra entry point. - Keep only CompareAll function. - Copy isVersionedName and versionlessName from schema-tools/pkg. - Pass schema and result via objects instead of via disk. - Copy existing tests.

- Generate raw-schema in-memory then immediately squeeze and output the result. - Derive filename from current major version in makefile. - Remove schema-tools from makefile.

- Expose OpenApi providers from the BuildSchema function to avoid loading twice. - Remove obsolete makefile dependency on the full-schema - we introduced the raw-schema to avoid this dependency.

This is no longer calculated during the BuildSchema process - only read in as it's generated at the same time as the removed resources.

- Pass openapi.AzureProviders in as a "source". - Add documentation comments for some nested data structures.

thomas11

LGTM with nits. Great start to a proper test suite! The readability of nested subtests is a matter of taste, I guess.

danielrbradley · 2024-11-19T10:36:08Z

The readability of nested subtests is a matter of taste, I guess

Would you think a single level of nesting with longer test names is better instead?

Edit: flattened to a single level of nesting - just grouped by the function under test.

We incorrectly assumed that extracted keys were in order when Go actually randomises the order of keys while iterating. - Require keys to be of an ordered type. - Use the built-in slices.Max generic function.

Describe basic behaviour of version selection process via structured test cases.

pulumi-bot · 2024-11-20T14:09:23Z

This PR has been shipped in release v2.73.0.

danielrbradley self-assigned this Nov 18, 2024

danielrbradley temporarily deployed to env-ci November 18, 2024 18:28 — with GitHub Actions Inactive

danielrbradley temporarily deployed to env-ci November 18, 2024 20:40 — with GitHub Actions Inactive

danielrbradley added 9 commits November 19, 2024 09:49

Copy squeeze code from schema-tools

651e144

- Remove cobra entry point. - Keep only CompareAll function. - Copy isVersionedName and versionlessName from schema-tools/pkg. - Pass schema and result via objects instead of via disk. - Copy existing tests.

Replace raw-schema with squeeze command

29c8375

- Generate raw-schema in-memory then immediately squeeze and output the result. - Derive filename from current major version in makefile. - Remove schema-tools from makefile.

Don't print every squeeze entry

dfd1907

Emit removed invokes during squeeze

70e34e6

- Expose OpenApi providers from the BuildSchema function to avoid loading twice. - Remove obsolete makefile dependency on the full-schema - we introduced the raw-schema to avoid this dependency.

Move removed invokes to sources

f7a5472

This is no longer calculated during the BuildSchema process - only read in as it's generated at the same time as the removed resources.

Remove unused majorVersion argument

5908c05

Move AllResourcesByVersion to VersionSources

81460f7

- Pass openapi.AzureProviders in as a "source". - Add documentation comments for some nested data structures.

Inline data calculations not used by other steps

2b72124

Remove unused providers argument

0223c16

danielrbradley force-pushed the vendor-squeeze branch from a6697f3 to b75dcde Compare November 19, 2024 09:49

danielrbradley marked this pull request as ready for review November 19, 2024 09:49

danielrbradley requested a review from thomas11 November 19, 2024 09:50

danielrbradley temporarily deployed to env-ci November 19, 2024 10:10 — with GitHub Actions Inactive

danielrbradley temporarily deployed to env-ci November 19, 2024 10:11 — with GitHub Actions Inactive

thomas11 approved these changes Nov 19, 2024

View reviewed changes

Comment thread provider/pkg/versioning/defaultVersion.go Outdated

Comment thread provider/pkg/versioning/gen_calculateVersionMetadata_test.go

danielrbradley added 2 commits November 19, 2024 10:40

Fix unreliable selection of default version

2fac7d3

We incorrectly assumed that extracted keys were in order when Go actually randomises the order of keys while iterating. - Require keys to be of an ordered type. - Use the built-in slices.Max generic function.

Build initial test suite

1faedac

Describe basic behaviour of version selection process via structured test cases.

danielrbradley force-pushed the vendor-squeeze branch from b75dcde to 1faedac Compare November 19, 2024 10:44

danielrbradley enabled auto-merge November 19, 2024 10:46

danielrbradley temporarily deployed to env-ci November 19, 2024 11:05 — with GitHub Actions Inactive

danielrbradley temporarily deployed to env-ci November 19, 2024 11:06 — with GitHub Actions Inactive

danielrbradley merged commit 1d3d099 into master Nov 19, 2024

danielrbradley deleted the vendor-squeeze branch November 19, 2024 11:15

dependabot Bot mentioned this pull request Aug 18, 2025

Bump Pulumi and Pulumi.AzureNative merca/data-and-stuff#118

Closed

dependabot Bot mentioned this pull request Aug 25, 2025

Bump Pulumi and Pulumi.AzureNative merca/data-and-stuff#123

Closed

This was referenced Sep 1, 2025

Bump Pulumi and Pulumi.AzureNative merca/data-and-stuff#130

Closed

Bump Pulumi and Pulumi.AzureNative merca/data-and-stuff#138

Closed

dependabot Bot mentioned this pull request Sep 15, 2025

Bump Pulumi and Pulumi.AzureNative merca/data-and-stuff#145

Closed

dependabot Bot mentioned this pull request Sep 22, 2025

Bump Pulumi and Pulumi.AzureNative merca/data-and-stuff#152

Closed

dependabot Bot mentioned this pull request Feb 2, 2026

Bump Pulumi and Pulumi.AzureNative merca/data-and-stuff#213

Closed

dependabot Bot mentioned this pull request Mar 2, 2026

Bump Pulumi and Pulumi.AzureNative merca/data-and-stuff#220

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor versioning process#3714

Refactor versioning process#3714
danielrbradley merged 11 commits into
masterfrom
vendor-squeeze

danielrbradley commented Nov 18, 2024 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 18, 2024

Uh oh!

codecov Bot commented Nov 18, 2024 •

edited

Loading

Uh oh!

thomas11 left a comment

Uh oh!

Uh oh!

Uh oh!

danielrbradley commented Nov 19, 2024 •

edited

Loading

Uh oh!

pulumi-bot commented Nov 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

danielrbradley commented Nov 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Nov 18, 2024

Does the PR have any schema changes?

Uh oh!

codecov Bot commented Nov 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

thomas11 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

danielrbradley commented Nov 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pulumi-bot commented Nov 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

danielrbradley commented Nov 18, 2024 •

edited

Loading

codecov Bot commented Nov 18, 2024 •

edited

Loading

danielrbradley commented Nov 19, 2024 •

edited

Loading