First draft of breaking change policy #2
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # Breaking Change Policy | ||
|
|
||
| Customers and MVPs have told us many times in emphatic terms that breaking changes are a continuing problem in Azure PowerShell releases. These changes cause an undue burden on users maintaining scripts to manage their Azure assets, and this causes reluctance in picking up the later versions of Azure PowerShell. It is clear that continuing, unmanaged breaking changes are unnacceptable to customers. | ||
|
There was a problem hiding this comment. Can we get a definition of 'undue', what is the end-user facing SLA for script updating? Something like: A brand new script should work for XX months.
Owner
Author
There was a problem hiding this comment. @brendanburns Agreed the extra adjectives are unhelpful. This is based on user's expectations as expressed - currently there is no stated policy, so no official SLA. |
||
|
|
||
| **To address customer concerns, Azure PowerShell is committing to limit breaking changes to one release per year and to treat any unplanned breaking change that ships as | ||
|
There was a problem hiding this comment. I have concerns about this, it's going to significantly limit module authors flexibility and it doesn't dramatically improve the end user experience. Instead I would suggest that every breaking change requires a 6 month deprecation period (with warnings, etc) but that the deprecation period can start at any time.
Owner
Author
There was a problem hiding this comment. @brendanburns I think this is a valid concern - for some kinds of breaks (breaks for which customer scripts can be immediately changed to accomodate), notifications should suffice, for other types of breaks (breaks which require the break to ship b=efore scripts can be changed), it makes sense for us to coordinate, if possible. |
||
| a showstopper bug requiring a hotfix**. | ||
|
|
||
| At the same time, we are introducing a new mechanism for [Pre-release AzureRM modules](#preview-modules). This is detailed in the section [here](#preview-modules). | ||
|
|
||
| This proposed policy is meant to apply to all stable Azure modules, starting with the 2.0 release. Details of the policy are as follows. | ||
| - Each module in Azure PowerShell is versioned using [semantic versioning](http://semver.org). | ||
| - Non-breaking changes and associated version updates are handled independently by module owners. [Breaking changes](#breaking-change-definition) are scheduled for once a year and coordinated with the Azure SDK Team. | ||
|
There was a problem hiding this comment. This coordination is going to be extremely expensive for everyone involved, and (as above) I don't think it will material improve the user experience over other options.
Owner
Author
There was a problem hiding this comment. @brendanburns. This is codifying coordination that already happens. For example, we coordinated changes in Tags and Confirmation to occur at the same time to avoid customers' scripts breaking twice. The Confirmation change effectively happened a couple of months before, but we didn't remove deprecated parameters until ther next breaking change came along. This kind of coordination would be desirable to customers, if it is possible. |
||
| - Each breaking change release will be preceded by 3-months notice of the break to customers. The SDK Team will coordinate the notice of breaking changes. | ||
|
There was a problem hiding this comment. I think 3-months is actually pretty short for most users. I'd advocate for 6 months of warning.
Owner
Author
There was a problem hiding this comment. @brendanburns The current standard is 1 month, I agree that 6 months is better. We will discuss the actual interval with MVPs for feedback |
||
| - It is the responsibility of the service teams to introduce no breaking changes in their cmdlet external interfaces. A definition of breaking changes is provided [here](#breaking-change-definition). It is the responsibility of the SDK Team to introduce no breaking changes in SDK cmdlets, and in the public types in common code used as parameters or returned by cmdlets. | ||
| - The SDK Team will provide tools in the October release to automatically detect and prevent breaking changes | ||
| - Until breaking change detection tools are available, the feature teams and SDK team will use code reviews to prevent breaking changes | ||
|
There was a problem hiding this comment. You should require command line integration tests for all new features.
Owner
Author
There was a problem hiding this comment. @brendanburns We try to. It is difficult to catch all breaking changes this way, however, which is why tooling is necessary. |
||
| - In the event that a breaking change is released outside of a planned breaking change release, the cmdlet owner must coordinate with the SDK team to create a hotfix that mitigates the impact of the breaking changes on existing scripts. | ||
| - The release notes for any breaking change version will contain a complete description of the changes and a migration guide, explaining how scripts should be updated to accomodate the breaking changes. | ||
| - Service Teams must provide a list of breaking changes and the migration guide at the start of the sprint containing the breaking changes | ||
| - The SDK Team will coordinate the breaking change documentation and assemble a migration guide from the feature team submissions | ||
| - A sample migration guide for the 2.0 release is provided [here](https://github.com/Azure/azure-powershell/blob/dev/documentation/release-notes/migration-guide.2.0.0.md) | ||
| - The breaking change list and migration guide will be distributed to customers in the github release, in an azure.com blog post, and in powershell help topics | ||
|
|
||
| ### Exceptions | ||
| The majority of cmdlet enhancement and fixes should not require breaking changes. However, to accomodate extraordinary circumstances, exceptions to the breaking change policy can be made for the following reasons: | ||
| - High impact security bugs, data loss, or other severe functionality issues that can only be resolved through a breaking change | ||
| - Important fixes and new features that require a breaking change, for which usage data or extensive customer outreach shows extremely low impact on customer scripts | ||
|
|
||
| ## Preview Modules | ||
|
|
||
| The breaking change policy should not prevent providing cmdlet support for non-GA services or experimenting with new cmdlets for existing services. To satisfy these two needs, Azure PowerShell will begin supporting Preview modules in the October release. | ||
|
|
||
| Note that, participation in Preview modules is entirely volunatry for feature teams. If cmdlets for an Azure service remain stable, the service team may decide not to support a preview module. | ||
|
|
||
| - Preview modules use the suffix Preview in their module name, as in 'AzureRM.Compute.Preview' | ||
| - Preview modules will not be included in the AzureRM module in the PowerShell Gallery, or in the released PowerShell MSI. | ||
| - A new roll-up module, AzureRM.Preview will reference the existing set of preview modules, allowing uses to download all of them at once. | ||
| - Non-GA services will automatically begin as Preview modules. However, a full set of non-preview cmdlets must be available in the PowerShell Gallery and PowerShell msi within 3 months of service GA. | ||
| - GA Services may *also* have a preview module to use when experimenting with new cmdlets | ||
| - Preview modules will *not* use semantic versioning | ||
|
|
||
| ## Breaking Change Definition | ||
|
|
||
| Breaking changes in cmdlets are defined as follows: | ||
|
There was a problem hiding this comment. I don't see anything in here about command behavior being a breaking change. It's important that there aren't syntactic breaking changes, but its equally important that there aren't semantic breaking changes (e.g. 'create vm' suddenly creates networks instead of VMs (obviously an extreme case))
Owner
Author
There was a problem hiding this comment. @brendanburns We are specifically limiting this to breaking the API surface for powershell scripts. We have left the behavior breaks out of the equation, because they are difficult to quantify. |
||
|
|
||
| - Cmdlets: | ||
| - Removing a cmdlet | ||
| - Changing a cmdlet name without an alias to the original name | ||
| - Removing or changing a cmdlet alias | ||
| - Removing a Cmdlet attribute option (SupportShouldProcess, SupportsPaging) | ||
| - Breaking change in OutputType or removal of OutputType attribute | ||
| - Parameters | ||
|
There was a problem hiding this comment. What about changing the default value for a parameter? |
||
| - Removing a parameter | ||
| - Changing the name of a parameter without an alias to the original parameter name | ||
| - Breaking change in parameter type | ||
| - Adding a required parameter to an existing parameter set (adding new parameter sets, or adding additional optional parameters is not breaking) | ||
| - Changing parameter order for parameter sets with ordered parameters | ||
| - Removing or changing a parameter alias | ||
| - Removing or changing existing parameter attribute values | ||
| - Making parameter validation more exclusive (for example: removing values from a ValidateSet) | ||
| - Output and Parameter Types | ||
| - Changing property names without an accompanying alias to the original name | ||
| - Removing properties | ||
| - Adding additional required properties | ||
| - Adding required parameters, changing parameter names, or parameter types for methods or constructors | ||
| - Changing return types of methods | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think in general, customers ignore deprecation warnings, and only notice when things stop working, and thus the flow really ought to be:
- Deprecation warnings for deprecated commands/features
2) Defaults change, things are semi-removed, they stop working by default _but_ there is a `--activate-deprecated` flag that restores things. This way customers notice, but they have a rollback path. 3) Things are removed for realz.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@brendanburns How to notify customers is a serious problem, and we are soliciting customer feedback . Having two API states would be ideal, but difficult to implement without architectural changes. Architectural changes that might support this are planned, but not soon.