HealthDataAIServices Deidentificiation Data Plane - GA Stable API Changes by GrahamMThomas · Pull Request #30518 · Azure/azure-rest-api-specs

GrahamMThomas · 2024-09-11T18:38:03Z

Data Plane API Specification Update Pull Request

Tip

Overwhelmed by all this guidance? See the Getting help section at the bottom of this PR description.

Goal of this PR is to add a few features we are looking to add in GA. Going from a preview version to GA. Want to make a few breaking changes to the schema before we release our first stable version.

Created surrogateLocale to allow customer's to request their output in a certain locale
Moved dataType into SourceStorageLocation model
Introduced an overwrite flag in TargetStorageLocation model
Moved redactionFormat, operation into DeidentificationCustomizationOption models
Changed outputPrefix behavior from including jobName to prefix replacement method.
Removed Path and Etag from TaggerResult output files in favor of DeidentificationDocumentDetails.

PR review workflow diagram

Please understand this diagram before proceeding. It explains how to get your PR approved & merged.

API Info: The Basics

Most of the information about your service should be captured in the issue that serves as your API Spec engagement record.

Link to API Spec engagement record issue:

Is this review for (select one):

a private preview
a public preview
GA release

Change Scope

^{This section will help us focus on the specific parts of your API that are new or have been modified.
Please share a link to the design document for the new APIs, a link to the previous API Spec document (if applicable), and the root paths that have been updated.}

Design Document:
Previous API Spec Doc:
Updated paths:

Viewing API changes

For convenient view of the API changes made by this PR, refer to the URLs provided in the table
in the Generated ApiView comment added to this PR. You can use ApiView to show API versions diff.

Suppressing failures

If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the
Swagger-Suppression-Process
to get approval.

❔Got questions? Need additional info?? We are here to help!

Contact us!

The Azure API Review Board is dedicated to helping you create amazing APIs. You can read about our mission and learn more about our process on our wiki.

💬 Teams Channel
💌 email

Click here for links to tools, specs, guidelines & other good stuff

Tooling

Open API validation tools were run on this PR. Go here to see how to fix errors
Spectral Linting

Guidelines & Specifications

Helpful Links

Schedule a data plane REST API spec review

Getting help

First, please carefully read through this PR description, from top to bottom.
If you don't have permissions to remove or add labels to the PR, request write access per aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories
To understand what you must do next to merge this PR, see the Next Steps to Merge comment. It will appear within few minutes of submitting this PR and will continue to be up-to-date with current PR state.
For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure
and https://aka.ms/ci-fix.
If the PR CI checks appear to be stuck in queued state, please add a comment with contents /azp run.
This should result in a new comment denoting a PR validation pipeline has started and the checks should be updated after few minutes.
If the help provided by the previous points is not enough, post to https://aka.ms/azsdk/support/specreview-channel and link to this PR.

openapi-pipeline-app · 2024-09-11T18:38:07Z

Next Steps to Merge

Next steps that must be taken to merge this PR:

❌ The required check named Swagger Avocado has failed. Refer to the check in the PR's 'Checks' tab for details on how to fix it and consult the aka.ms/ci-fix guide

openapi-pipeline-app · 2024-09-11T18:38:07Z

Generated ApiView

Language	Package Name	ApiView Link
JavaScript	@azure-rest/health-deidentification	https://apiview.dev/Assemblies/Review/6f21d1340b1240a0a98433243eafbe0f?revisionId=3f54e97f2bcc4db489b1cb9df442348e
.Net	Azure.Health.Deidentification	https://apiview.dev/Assemblies/Review/a2851822fa7045a192ad02d92d71214b?revisionId=d396363acaff4d799e8957d2fdbd46c5
Python	azure-health-deidentification	https://apiview.dev/Assemblies/Review/22a62e8622be49bbbd53bc45583d2afa?revisionId=2c386bfdba0a40149d3427ed3c32f3f1
TypeSpec	HealthDataAIServices.DeidServices	https://apiview.dev/Assemblies/Review/8c9761c1db034600bf425aa490e86fba?revisionId=130794a513da485dade302c705e096a3

…github.com/GrahamMThomas/azure-rest-api-specs into healthdataaiservices/customization-options

weidongxu-microsoft

I wasn't in the review meeting. If any of my feedback is already resolved there, feel free to "resolve" that comment.

I didn't see response to review meeting feedback of #30515 (comment)
I'd expect a simple response to it (e.g. which item is resolved, which is not).

specification/healthdataaiservices/HealthDataAIServices.DeidServices/models.tsp

specification/healthdataaiservices/HealthDataAIServices.DeidServices/routes.tsp

specification/healthdataaiservices/HealthDataAIServices.DeidServices/models_document_flow.tsp

azure-sdk · 2024-11-26T18:07:47Z

API change check

APIView has identified API level changes in this PR and created following API reviews.

HealthDataAIServices.DeidServices-HealthDataAIServices.DeidServices

GrahamMThomas · 2024-11-26T18:22:00Z

I wasn't in the review meeting. If any of my feedback is already resolved there, feel free to "resolve" that comment.

I didn't see response to review meeting feedback of #30515 (comment) I'd expect a simple response to it (e.g. which item is resolved, which is not).

I've added TODOs for the customizationOptions doc link as we don't have the docs built yet.

Regarding the "Recommend looking at Document Translator for naming consistency", do you remember what specifically which models we were trying to sync?

weidongxu-microsoft · 2024-11-27T02:19:35Z

I've added TODOs for the customizationOptions doc link as we don't have the docs built yet.

Regarding the "Recommend looking at Document Translator for naming consistency", do you remember what specifically which models we were trying to sync?

I went a recap of the meeting. I think the "naming consistency" is about the name of locale property. From what I get, the decision is to use industrial standard for the name.

PS: ideally, during SDK review, the changes would be limited to the customization within client.tsp (rename, restructure the clients, etc.), which only need SDK dev approval.

weidongxu-microsoft

LGTM.

A few minor comments, but no major problem to me.

specification/healthdataaiservices/HealthDataAIServices.DeidServices/models_document_flow.tsp

XiaofeiCao · 2024-11-28T06:29:58Z

/azp run

azure-pipelines · 2024-11-28T06:30:20Z

Azure Pipelines successfully started running 3 pipeline(s).

weidongxu-microsoft · 2024-11-28T06:34:33Z

SDK automation for Java is expected to fail, as the customization is not yet in Java SDK repo

azure-rest-api-specs/specification/healthdataaiservices/HealthDataAIServices.DeidServices/tspconfig.yaml

Line 41 in 4698f7c

customization-class: "customization/src/main/java/ListJobsCustomization.java"

…om:GrahamMThomas/azure-rest-api-specs into healthdataaiservices/customization-options

…er prefix

mikekistler

Looks good! 👍

mikekistler · 2024-12-06T21:02:54Z

...e/HealthDataAIServices.DeidServices/stable/2024-11-15/HealthDataAIServices.DeidServices.json

+            "schema": {
+              "$ref": "#/definitions/Azure.Core.Foundations.ErrorResponse"
+            },
+            "headers": {


You probably want the x-ms-client-request-id defined for error responses as well -- that's where it is really needed. This is a bug in TypeSpec and there is already an open issue for it:

https://github.com/Azure/typespec-azure-pr/issues/3427

cc: @allenjzhang @mario-guerra

…nges (Azure#30518) * Adds CustomizationOptions * Regenerates Examples * Model tweaks * Updates examples * Updates after team meeting * Renames customization options * Introduces new version * Revert old version * Fixes pipeline failures * Recompile * Location is no longer a sas uri * Fixes models prettier * Rename variables in csharp only * tsp format * Latest updates from SDK Review * Add OperationType model for realtime * Change Java service name from DeidServices (default) to Deidentification * get*Count() for Java * Rename DeidentificationClient in JS * WIP * Attempt switching to raw operation * Round 2 * Round 3 * Round 4 * Initial examples * Rename to CustomizationOptions * Mark listJobs and listJobDocuments as internal * csharp-specific changes * Add customization for java in tspconfig * npm update * Changes for python * No longer need python method to be internal * Re-ran e2e script * Revert "Merge branch 'main' into healthdataaiservices/customization-options" This reverts commit a355b96, reversing changes made to 3803319. * tsp format * Delete unnecessary files * Reapply "Merge branch 'main' into healthdataaiservices/customization-options" This reverts commit db141b4. * Restore package-lock.json * Remove HTTPS prefix * Rename body to content in C# * Change endpoint from str to url * Typespec validation * Deprecate 2024-07-12 examples; Updates examples * Validation fixes 1 * Prefixes customizationoptions with deidentification * Adds back in old spec * Adds back in old spec to readme.md * REVERT - Adds back in old spec to readme.md * Moving Azure.ClientGenerator.Core to client.tsp * Use clientName in routes.tsp for renaming "body" * Delete specification/healthdataaiservices/HealthDataAIServices.DeidServices/.gitignore * Use ResourceList and alias * e2e with name instead of jobName * Deprecates Partial Failed * Adds todos for doc links * Adds doc link to redaction format * Recompiles * Tsp format * Removes minimum length restriction from prefix to allow whole container prefix --------- Co-authored-by: Graham Thomas <gthomas@microsoft.com> Co-authored-by: Alexa Thomases <athomases@microsoft.com> Co-authored-by: alexathomases <55257302+alexathomases@users.noreply.github.com> Co-authored-by: Mike Soennichsen <msoennichsen@microsoft.com>

Graham Thomas added 2 commits September 11, 2024 11:30

Adds CustomizationOptions

a12b4c5

Regenerates Examples

e6bfe16

AzureRestAPISpecReview added data-plane TypeSpec Authored with TypeSpec VersioningReviewRequired <valid label in PR review process>add this label when versioning review is required labels Sep 11, 2024

GrahamMThomas mentioned this pull request Sep 17, 2024

[Health Data DeIdentification - Azure Health AI Deidentification service ] API Review #30515

Closed

Graham Thomas added 3 commits October 17, 2024 09:45

Model tweaks

8b2e2b1

Merge branch 'healthdataaiservices/customization-options' of https://…

4e7822d

…github.com/GrahamMThomas/azure-rest-api-specs into healthdataaiservices/customization-options

Updates examples

000f77a

azure-sdk added the BreakingChange-Python-Sdk label Oct 17, 2024

Graham Thomas added 2 commits October 18, 2024 10:58

Updates after team meeting

0d8ef41

Merge branch 'main' into healthdataaiservices/customization-options

bfb8897

GrahamMThomas marked this pull request as ready for review October 18, 2024 18:01

GrahamMThomas requested a review from a team as a code owner October 18, 2024 18:01

GrahamMThomas requested review from marclerwick and tg-msft and removed request for a team October 18, 2024 18:01

Graham Thomas added 4 commits October 21, 2024 10:18

Renames customization options

79c771f

Merge branch 'main' into healthdataaiservices/customization-options

dfc754e

Introduces new version

84636d2

Revert old version

680314a

AzureRestAPISpecReview added new-api-version and removed VersioningReviewRequired <valid label in PR review process>add this label when versioning review is required labels Oct 21, 2024

azure-sdk removed the BreakingChange-Python-Sdk label Oct 21, 2024

Alexa Thomases added 2 commits November 22, 2024 14:05

Merge branch 'healthdataaiservices/customization-options' of https://…

116ccef

…github.com/GrahamMThomas/azure-rest-api-specs into healthdataaiservices/customization-options

e2e with name instead of jobName

ebe6111

alexathomases requested a review from weidongxu-microsoft November 23, 2024 01:35

weidongxu-microsoft reviewed Nov 25, 2024

View reviewed changes

Deprecates Partial Failed

ed09a5b

Adds todos for doc links

4698f7c

weidongxu-microsoft approved these changes Nov 27, 2024

View reviewed changes

specification/healthdataaiservices/HealthDataAIServices.DeidServices/models_document_flow.tsp Show resolved Hide resolved

Graham Thomas and others added 5 commits December 6, 2024 10:19

Adds doc link to redaction format

9a1215b

Merge branch 'main' into healthdataaiservices/customization-options

9489ede

Recompiles

8bab247

Merge branch 'healthdataaiservices/customization-options' of github.c…

6562b04

…om:GrahamMThomas/azure-rest-api-specs into healthdataaiservices/customization-options

Tsp format

25863ea

mikekistler linked an issue Dec 6, 2024 that may be closed by this pull request

[Health Data DeIdentification - Azure Health AI Deidentification service ] API Review #30515

Closed

Removes minimum length restriction from prefix to allow whole contain…

9189606

…er prefix

mikekistler approved these changes Dec 6, 2024

View reviewed changes

mikekistler added the APIStewardshipBoard-SignedOff The Azure API Stewardship team has reviewed and approved the changes. label Dec 6, 2024

GrahamMThomas merged commit 45baf6e into Azure:main Dec 6, 2024

jovinson-ms mentioned this pull request May 1, 2025

Deid GA release Azure/azure-sdk-for-python#40850

Merged

6 tasks

jovinson-ms added the Health Deidentification label May 1, 2025

Comments

Conversation

GrahamMThomas commented Sep 11, 2024 • edited by alexathomases Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Data Plane API Specification Update Pull Request

PR review workflow diagram

API Info: The Basics

Change Scope

Viewing API changes

Suppressing failures

❔Got questions? Need additional info?? We are here to help!

Tooling

Guidelines & Specifications

Helpful Links

Getting help

Uh oh!

openapi-pipeline-app bot commented Sep 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Next Steps to Merge

Uh oh!

openapi-pipeline-app bot commented Sep 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Generated ApiView

Uh oh!

weidongxu-microsoft left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

azure-sdk commented Nov 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GrahamMThomas commented Nov 26, 2024

Uh oh!

weidongxu-microsoft commented Nov 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

weidongxu-microsoft left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

XiaofeiCao commented Nov 28, 2024

Uh oh!

azure-pipelines bot commented Nov 28, 2024

Uh oh!

weidongxu-microsoft commented Nov 28, 2024

Uh oh!

mikekistler left a comment

Choose a reason for hiding this comment

Uh oh!

mikekistler Dec 6, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

GrahamMThomas commented Sep 11, 2024 •

edited by alexathomases

Loading

openapi-pipeline-app bot commented Sep 11, 2024 •

edited

Loading

openapi-pipeline-app bot commented Sep 11, 2024 •

edited

Loading

weidongxu-microsoft left a comment •

edited

Loading

azure-sdk commented Nov 26, 2024 •

edited

Loading

weidongxu-microsoft commented Nov 27, 2024 •

edited

Loading