Skip to content

Mark deprecated pipeline fields #2646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kanterov merged 6 commits into from
Apr 8, 2025
+16 −2
Merged

Mark deprecated pipeline fields #2646

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
75b7155
Mark deprecated pipeline fields
kanterov Apr 3, 2025
c79d642
Update deprecation for 'target' field
kanterov Apr 7, 2025
284951e
Update schema
kanterov Apr 7, 2025
cf9560d
Merge branch 'main' into pipeline-deprecated
kanterov Apr 7, 2025
475d663
Address CR feedback
kanterov Apr 8, 2025
f596fa7
Address CR feedback
kanterov Apr 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions NEXT_CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,6 @@
* Added support for model serving endpoints in deployment bind/unbind commands ([#2634](https://github.com/databricks/cli/pull/2634))
* Added include/exclude flags support to bundle sync command ([#2650](https://github.com/databricks/cli/pull/2650))
* Removed pipeline 'deployment' field from jsonschema ([#2653](https://github.com/databricks/cli/pull/2653))
* Updated json schema for deprecated pipeline fields ([#2646](https://github.com/databricks/cli/pull/2646))
Comment thread
kanterov marked this conversation as resolved.
Outdated

### API Changes
3 changes: 3 additions & 0 deletions bundle/docsgen/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,9 @@ func assignAnnotation(s *jsonschema.Schema, a annotation.Descriptor) {
s.Deprecated = true
s.DeprecationMessage = a.DeprecationMessage
}
if a.Deprecated {
s.Deprecated = true
}

@pietern pietern Apr 8, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please define what happens if Deprecated == false and DeprecationMessage != "".

This seems confusing. Alternatively, we can exclusively use the message field for this.

cc @ilyakuz-db

}

func fillTemplateVariables(s string) string {
Expand Down
1 change: 1 addition & 0 deletions bundle/internal/annotation/descriptor.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ type Descriptor struct {
Enum []any `json:"enum,omitempty"`
MarkdownExamples string `json:"markdown_examples,omitempty"`
DeprecationMessage string `json:"deprecation_message,omitempty"`
Deprecated bool `json:"deprecated,omitempty"`

@pietern pietern Apr 8, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is duplicated in #2652. What's the sequence of merging you're thinking of?

@kanterov kanterov Apr 8, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't matter much, because we can rebase one PR on another. This PR is more important, I feel it will take time to review all deprecations introduced in #2652

}

const Placeholder = "PLACEHOLDER"
4 changes: 4 additions & 0 deletions bundle/internal/schema/annotations.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,10 @@ func assignAnnotation(s *jsonschema.Schema, a annotation.Descriptor) {
s.DeprecationMessage = a.DeprecationMessage
}

if a.Deprecated {
s.Deprecated = true
}

s.MarkdownDescription = convertLinksToAbsoluteUrl(a.MarkdownDescription)
s.Title = a.Title
s.Enum = a.Enum
Expand Down
9 changes: 9 additions & 0 deletions bundle/internal/schema/annotations_openapi_overrides.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,12 @@ github.com/databricks/cli/bundle/config/resources.Pipeline:
"run_as":
"description": |-
PLACEHOLDER
"target":
# should not be marked as deprecated until its possible to switch
# existing pipelines to use 'schema' instead
"deprecated": false

@pietern pietern Apr 8, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this set as deprecated in the annotations file already?

If not, absence of an entry means it won't be set.

@kanterov kanterov Apr 8, 2025

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's primarily for the case when parent file gets deprecated annotation from OpenAPI spec

"trigger":
"deprecation_message": Use continuous instead
Comment thread
kanterov marked this conversation as resolved.
github.com/databricks/cli/bundle/config/resources.QualityMonitor:
"_":
"markdown_description": |-
Expand Down Expand Up @@ -567,6 +573,9 @@ github.com/databricks/databricks-sdk-go/service/pipelines.PipelineTrigger:
"manual":
"description": |-
PLACEHOLDER
github.com/databricks/databricks-sdk-go/service/pipelines.PipelineLibrary:
"whl":
"deprecated": true
github.com/databricks/databricks-sdk-go/service/serving.ServedEntityInput:
"entity_version":
"description": |-
Expand Down
7 changes: 5 additions & 2 deletions bundle/schema/jsonschema.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.