-
Notifications
You must be signed in to change notification settings - Fork 5.1k
[TA] Added RecognizeCustomEntities Functionality #24245
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
maririos
merged 7 commits into
Azure:feature/textanalytics/custom
from
ZulaMostafa:TA-CustomNER
Oct 19, 2021
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
dded44f
[TA] Added RecognizeCustomEntities Functionality (#24245)
9993deb
Changes based on review
2886506
minor changes based on review
a405ac8
minor changes
2335135
minor changes
75c450b
undid non-needed records
1d6666d
deleted JobManifestsTasks.cs
ZulaMostafa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
112 changes: 112 additions & 0 deletions
112
...textanalytics/Azure.AI.TextAnalytics/samples/Sample9_RecognizeCustomEntities.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| # Recognizing Custom Entities from Documents | ||
| This sample demonstrates how to recognize custom entities in one or more documents. To get started you'll need a Text Analytics endpoint and credentials. See [README][README] for links and instructions. | ||
|
|
||
| ## Creating a `TextAnalyticsClient` | ||
|
|
||
| To create a new `TextAnalyticsClient` to recognize custom entities in a document, you need a Text Analytics endpoint and credentials. You can use the [DefaultAzureCredential][DefaultAzureCredential] to try a number of common authentication methods optimized for both running as a service and development. In the sample below, however, you'll use a Text Analytics API key credential by creating an `AzureKeyCredential` object, that if needed, will allow you to update the API key without creating a new client. | ||
ZulaMostafa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| You can set `endpoint` and `apiKey` based on an environment variable, a configuration setting, or any way that works for your application. | ||
|
|
||
| ```C# Snippet:CreateTextAnalyticsClient | ||
| string endpoint = "<endpoint>"; | ||
| string apiKey = "<apiKey>"; | ||
| var client = new TextAnalyticsClient(new Uri(endpoint), new AzureKeyCredential(apiKey)); | ||
| ``` | ||
|
|
||
| ## Recognizing custom entities in a single or multiple documents | ||
|
|
||
| To recognize custom entities in documents, set up a `RecognizeCustomEntitiesAction` and call `StartAnalyzeActionsAsync` on the documents. The result is a Long Running operation of type `AnalyzeActionsOperation` which polls for the results from the API. You can use [Azure language studio][azure_language_studio] to train custom models. | ||
|
|
||
| ```C# Snippet:RecognizeCustomEntitiesActionAsync | ||
| // Create input documents. | ||
| string documentA = @"We love this trail and make the trip every year. The views are breathtaking and well | ||
| worth the hike! Yesterday was foggy though, so we missed the spectacular views. | ||
| We tried again today and it was amazing. Everyone in my family liked the trail although | ||
| it was too challenging for the less athletic among us."; | ||
|
|
||
| string documentB = @"Last week we stayed at Hotel Foo to celebrate our anniversary. The staff knew about | ||
| our anniversary so they helped me organize a little surprise for my partner. | ||
| The room was clean and with the decoration I requested. It was perfect!"; | ||
|
|
||
| var batchDocuments = new List<TextDocumentInput> | ||
| { | ||
| new TextDocumentInput("1", documentA) | ||
| { | ||
| Language = "en", | ||
ZulaMostafa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| }, | ||
| new TextDocumentInput("2", documentB) | ||
| { | ||
| Language = "en", | ||
| } | ||
| }; | ||
|
|
||
| // prepare actions. | ||
| string projectName = "<projectName>"; | ||
| string deploymentName = "<deploymentName>"; | ||
| var actions = new TextAnalyticsActions() | ||
| { | ||
| RecognizeCustomEntitiesActions = new List<RecognizeCustomEntitiesAction>() | ||
| { | ||
| new RecognizeCustomEntitiesAction(projectName, deploymentName); | ||
| } | ||
| }; | ||
|
|
||
| AnalyzeActionsOperation operation = await client.StartAnalyzeActionsAsync(batchDocuments, actions); | ||
|
|
||
| await operation.WaitForCompletionAsync(); | ||
| ``` | ||
|
|
||
| The returned `AnalyzeActionsOperation` contains general information about the status of the operation. It can be requested while the operation is running or when it has completed. For example: | ||
|
|
||
| ```C# Snippet:RecognizeCustomEntitiesActionOperationStatus | ||
| // View operation status. | ||
| Console.WriteLine($"AnalyzeActions operation has completed"); | ||
| Console.WriteLine(); | ||
| Console.WriteLine($"Created On : {operation.CreatedOn}"); | ||
| Console.WriteLine($"Expires On : {operation.ExpiresOn}"); | ||
| Console.WriteLine($"Id : {operation.Id}"); | ||
| Console.WriteLine($"Status : {operation.Status}"); | ||
| Console.WriteLine($"Last Modified: {operation.LastModified}"); | ||
| Console.WriteLine(); | ||
| ``` | ||
|
|
||
| To view the final results of the long-running operation: | ||
|
|
||
| ```C# Snippet:RecognizeCustomEntitiesActionAsyncViewResults | ||
| await foreach (AnalyzeActionsResult documentsInPage in operation.Value) | ||
| { | ||
| IReadOnlyCollection<RecognizeCustomEntitiesActionResult> customEntitiesActionResults = documentsInPage.RecognizeCustomEntitiesResults; | ||
| foreach (RecognizeCustomEntitiesActionResult customEntitiesActionResult in customEntitiesActionResults) | ||
| { | ||
| int docNumber = 1; | ||
| foreach (RecognizeEntitiesResult documentResults in customEntitiesActionResult.DocumentsResults) | ||
| { | ||
| Console.WriteLine($" Document #{docNumber++}"); | ||
| Console.WriteLine($" Recognized the following {documentResults.Entities.Count} entities:"); | ||
|
|
||
| foreach (CategorizedEntity entity in documentResults.Entities) | ||
| { | ||
| Console.WriteLine($" Entity: {entity.Text}"); | ||
| Console.WriteLine($" Category: {entity.Category}"); | ||
| Console.WriteLine($" Offset: {entity.Offset}"); | ||
| Console.WriteLine($" Length: {entity.Length}"); | ||
| Console.WriteLine($" ConfidenceScore: {entity.ConfidenceScore}"); | ||
| Console.WriteLine($" SubCategory: {entity.SubCategory}"); | ||
| } | ||
| Console.WriteLine(""); | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
ZulaMostafa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| To see the full example source files, see: | ||
|
|
||
| * [Synchronously RecognizeCustomEntities](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample9_RecognizeCustomEntities.cs) | ||
| * [Asynchronously RecognizeCustomEntities](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples//Sample9_RecognizeCustomEntitiesAsync.cs) | ||
| * [Synchronously RecognizeCustomEntities Convenience](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample9_RecognizeCustomEntitiesConvenience.cs) | ||
| * [Asynchronously RecognizeCustomEntities Convenience](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample9_RecognizeCustomEntitiesConvenienceAsync.cs) | ||
|
|
||
| <!-- LINKS --> | ||
| [azure_language_studio]: https://language.azure.com/ | ||
| [README]: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/README.md | ||
| [DefaultAzureCredential]: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/identity/Azure.Identity/README.md | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.