Skip to content
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions sdk/parents/azure-client-sdk-parent/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -1086,6 +1086,7 @@
--add-opens com.azure.core/com.azure.core.implementation.serializer=ALL-UNNAMED
--add-opens com.azure.ai.formrecognizer/com.azure.ai.formrecognizer=ALL-UNNAMED
--add-opens com.azure.ai.textanalytics/com.azure.ai.textanalytics=ALL-UNNAMED
Comment thread
samvaity marked this conversation as resolved.
--add-opens com.azure.ai.textanalytics/com.azure.ai.textanalytics.implementation.models=com.azure.core
Comment thread
mssfang marked this conversation as resolved.
--add-opens com.azure.data.appconfiguration/com.azure.data.appconfiguration=ALL-UNNAMED
--add-opens com.azure.security.keyvault.certificates/com.azure.security.keyvault.certificates=ALL-UNNAMED
--add-opens com.azure.security.keyvault.secrets/com.azure.security.keyvault.secrets=ALL-UNNAMED
Expand Down
16 changes: 8 additions & 8 deletions sdk/textanalytics/azure-ai-textanalytics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,17 +229,17 @@ documentSentiment.getSentences().forEach(sentenceSentiment ->
```java
String document = "Bonjour tout le monde";
DetectedLanguage detectedLanguage = textAnalyticsClient.detectLanguage(document);
System.out.printf("Detected language name: %s, ISO 6391 name: %s, score: %f.%n",
detectedLanguage.getName(), detectedLanguage.getIso6391Name(), detectedLanguage.getScore());
System.out.printf("Detected language name: %s, ISO 6391 name: %s, confidence score: %f.%n",
detectedLanguage.getName(), detectedLanguage.getIso6391Name(), detectedLanguage.getConfidenceScore());
```

### Recognize entity
<!-- embedme ./src/samples/java/com/azure/ai/textanalytics/ReadmeSamples.java#L123-L126 -->
```java
String document = "Satya Nadella is the CEO of Microsoft";
textAnalyticsClient.recognizeEntities(document).forEach(entity ->
System.out.printf("Recognized entity: %s, category: %s, subCategory: %s, score: %f.%n",
entity.getText(), entity.getCategory(), entity.getSubCategory(), entity.getConfidenceScore()));
System.out.printf("Recognized entity: %s, category: %s, subcategory: %s, confidence score: %f.%n",
entity.getText(), entity.getCategory(), entity.getSubcategory(), entity.getConfidenceScore()));
```

### Recognize linked entity
Expand All @@ -251,8 +251,8 @@ textAnalyticsClient.recognizeLinkedEntities(document).forEach(linkedEntity -> {
System.out.println("Linked Entities:");
System.out.printf("Name: %s, entity ID in data source: %s, URL: %s, data source: %s.%n",
linkedEntity.getName(), linkedEntity.getDataSourceEntityId(), linkedEntity.getUrl(), linkedEntity.getDataSource());
linkedEntity.getLinkedEntityMatches().forEach(linkedEntityMatch ->
System.out.printf("Text: %s, score: %f.%n", linkedEntityMatch.getText(), linkedEntityMatch.getConfidenceScore()));
linkedEntity.getMatches().forEach(match ->
System.out.printf("Text: %s, confidence score: %f.%n", match.getText(), match.getConfidenceScore()));
});
```
### Extract key phrases
Expand Down Expand Up @@ -314,13 +314,12 @@ When you submit a pull request, a CLA-bot will automatically determine whether y
This project has adopted the [Microsoft Open Source Code of Conduct][coc]. For more information see the [Code of Conduct FAQ][coc_faq] or contact [opencode@microsoft.com][coc_contact] with any additional questions or comments.

<!-- LINKS -->
[azure_key_credential]: https://github.com/Azure/azure-sdk-for-java/blob/master/sdk/core/azure-core/src/main/java/com/azure/core/credential/AzureKeyCredential.java
[aad_credential]: https://docs.microsoft.com/azure/cognitive-services/authentication#authenticate-with-azure-active-directory
[key]: https://docs.microsoft.com/azure/cognitive-services/cognitive-services-apis-create-account?tabs=multiservice%2Cwindows#get-the-keys-for-your-resource
[api_reference_doc]: https://aka.ms/azsdk-java-textanalytics-ref-docs
[authentication]: https://docs.microsoft.com/azure/cognitive-services/authentication
[azure_cli]: https://docs.microsoft.com/azure/cognitive-services/cognitive-services-apis-create-account-cli?tabs=windows
[azure_identity]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/identity/azure-identity#credentials
[azure_key_credential]: https://github.com/Azure/azure-sdk-for-java/blob/master/sdk/core/azure-core/src/main/java/com/azure/core/credential/AzureKeyCredential.java
[azure_portal]: https://ms.portal.azure.com
[azure_subscription]: https://azure.microsoft.com/free
[cla]: https://cla.microsoft.com
Expand All @@ -333,6 +332,7 @@ This project has adopted the [Microsoft Open Source Code of Conduct][coc]. For m
[default_azure_credential]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/identity/azure-identity#defaultazurecredential
[grant_access]: https://docs.microsoft.com/azure/cognitive-services/authentication#assign-a-role-to-a-service-principal
[install_azure_identity]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/identity/azure-identity#install-the-package
[key]: https://docs.microsoft.com/azure/cognitive-services/cognitive-services-apis-create-account?tabs=multiservice%2Cwindows#get-the-keys-for-your-resource
[key_phrase_extraction]: https://docs.microsoft.com/azure/cognitive-services/text-analytics/how-tos/text-analytics-how-to-keyword-extraction
[language_detection]: https://docs.microsoft.com/azure/cognitive-services/text-analytics/how-tos/text-analytics-how-to-language-detection
[language_regional_support]: https://docs.microsoft.com/azure/cognitive-services/text-analytics/language-support
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,27 +13,31 @@
import com.azure.ai.textanalytics.models.SentenceSentiment;
import com.azure.ai.textanalytics.models.SentimentConfidenceScores;
import com.azure.ai.textanalytics.models.TextAnalyticsRequestOptions;
import com.azure.ai.textanalytics.models.TextAnalyticsWarning;
import com.azure.ai.textanalytics.models.TextDocumentInput;
import com.azure.ai.textanalytics.models.TextSentiment;
import com.azure.ai.textanalytics.models.WarningCode;
import com.azure.ai.textanalytics.util.TextAnalyticsPagedFlux;
import com.azure.ai.textanalytics.util.TextAnalyticsPagedResponse;
import com.azure.core.exception.HttpResponseException;
import com.azure.core.http.rest.SimpleResponse;
import com.azure.core.util.Context;
import com.azure.core.util.IterableStream;
import com.azure.core.util.logging.ClientLogger;
import reactor.core.publisher.Mono;

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Locale;
import java.util.Objects;
import java.util.stream.Collectors;

import static com.azure.ai.textanalytics.TextAnalyticsAsyncClient.COGNITIVE_TRACING_NAMESPACE_VALUE;
import static com.azure.ai.textanalytics.Transforms.toBatchStatistics;
import static com.azure.ai.textanalytics.Transforms.toTextAnalyticsError;
import static com.azure.ai.textanalytics.Transforms.toTextDocumentStatistics;
import static com.azure.ai.textanalytics.implementation.Utility.getMockHttpResponse;
import static com.azure.ai.textanalytics.implementation.Utility.inputDocumentsValidation;
import static com.azure.ai.textanalytics.implementation.Utility.mapToHttpResponseExceptionIfExist;
import static com.azure.core.util.FluxUtil.fluxError;
import static com.azure.core.util.FluxUtil.withContext;
import static com.azure.core.util.tracing.Tracer.AZ_TRACING_NAMESPACE_KEY;
Expand Down Expand Up @@ -66,18 +70,12 @@ class AnalyzeSentimentAsyncClient {
*/
TextAnalyticsPagedFlux<AnalyzeSentimentResult> analyzeSentimentBatch(Iterable<TextDocumentInput> documents,
TextAnalyticsRequestOptions options) {
Objects.requireNonNull(documents, "'documents' cannot be null.");
final Iterator<TextDocumentInput> iterator = documents.iterator();
if (!iterator.hasNext()) {
throw logger.logExceptionAsError(new IllegalArgumentException("'documents' cannot be empty."));
}

try {
inputDocumentsValidation(documents);
return new TextAnalyticsPagedFlux<>(() -> (continuationToken, pageSize) -> withContext(context ->
getAnalyzedSentimentResponseInPage(documents, options, context)).flux());
} catch (RuntimeException ex) {
return new TextAnalyticsPagedFlux<>(() ->
(continuationToken, pageSize) -> fluxError(logger, ex));
return new TextAnalyticsPagedFlux<>(() -> (continuationToken, pageSize) -> fluxError(logger, ex));
}
}

Expand All @@ -93,14 +91,13 @@ TextAnalyticsPagedFlux<AnalyzeSentimentResult> analyzeSentimentBatch(Iterable<Te
*/
TextAnalyticsPagedFlux<AnalyzeSentimentResult> analyzeSentimentBatchWithContext(
Iterable<TextDocumentInput> documents, TextAnalyticsRequestOptions options, Context context) {
Objects.requireNonNull(documents, "'documents' cannot be null.");
final Iterator<TextDocumentInput> iterator = documents.iterator();
if (!iterator.hasNext()) {
throw logger.logExceptionAsError(new IllegalArgumentException("'documents' cannot be empty."));
try {
inputDocumentsValidation(documents);
return new TextAnalyticsPagedFlux<>(() -> (continuationToken, pageSize) ->
getAnalyzedSentimentResponseInPage(documents, options, context).flux());
} catch (RuntimeException ex) {
return new TextAnalyticsPagedFlux<>(() -> (continuationToken, pageSize) -> fluxError(logger, ex));
}

return new TextAnalyticsPagedFlux<>(() -> (continuationToken, pageSize) ->
getAnalyzedSentimentResponseInPage(documents, options, context).flux());
}

/**
Expand All @@ -119,6 +116,14 @@ private TextAnalyticsPagedResponse<AnalyzeSentimentResult> toTextAnalyticsPagedR
analyzeSentimentResults.add(convertToAnalyzeSentimentResult(documentSentiment));
}
for (DocumentError documentError : sentimentResponse.getErrors()) {
/*
* TODO: Remove this after service update to throw exception.
Comment thread
mssfang marked this conversation as resolved.
* Currently, service sets max limit of document size to 5, if the input documents size > 5, it will
* have an id = "", empty id. In the future, they will remove this and throw HttpResponseException.
*/
if (documentError.getId().isEmpty()) {
throw logger.logExceptionAsError(new HttpResponseException(getMockHttpResponse(response)));
}
analyzeSentimentResults.add(new AnalyzeSentimentResult(documentError.getId(), null,
toTextAnalyticsError(documentError.getError()), null));
}
Expand Down Expand Up @@ -148,7 +153,7 @@ private AnalyzeSentimentResult convertToAnalyzeSentimentResult(DocumentSentiment
documentSentiment.getSentiment())));
}

final SentimentConfidenceScorePerLabel confidenceScorePerLabel = documentSentiment.getDocumentScores();
final SentimentConfidenceScorePerLabel confidenceScorePerLabel = documentSentiment.getConfidenceScores();

// Sentence text sentiment
final List<SentenceSentiment> sentenceSentiments = documentSentiment.getSentences().stream()
Expand All @@ -163,16 +168,19 @@ private AnalyzeSentimentResult convertToAnalyzeSentimentResult(DocumentSentiment
sentenceSentiment.getSentiment())));
}
final SentimentConfidenceScorePerLabel confidenceScorePerSentence =
sentenceSentiment.getSentenceScores();
sentenceSentiment.getConfidenceScores();

return new SentenceSentiment(
return new SentenceSentiment(sentenceSentiment.getText(),
sentenceSentimentLabel,
new SentimentConfidenceScores(confidenceScorePerSentence.getNegative(),
confidenceScorePerSentence.getNeutral(), confidenceScorePerSentence.getPositive()),
sentenceSentiment.getLength(),
sentenceSentiment.getOffset());
confidenceScorePerSentence.getNeutral(), confidenceScorePerSentence.getPositive()));
}).collect(Collectors.toList());

// Warnings
final List<TextAnalyticsWarning> warnings = documentSentiment.getWarnings().stream().map(
warning -> new TextAnalyticsWarning(WarningCode.fromString(warning.getCode().toString()),
warning.getMessage())).collect(Collectors.toList());

return new AnalyzeSentimentResult(
documentSentiment.getId(),
documentSentiment.getStatistics() == null
Expand All @@ -184,7 +192,8 @@ private AnalyzeSentimentResult convertToAnalyzeSentimentResult(DocumentSentiment
confidenceScorePerLabel.getNegative(),
confidenceScorePerLabel.getNeutral(),
confidenceScorePerLabel.getPositive()),
new IterableStream<>(sentenceSentiments)));
new IterableStream<>(sentenceSentiments),
Comment thread
mssfang marked this conversation as resolved.
new IterableStream<>(warnings)));
}

/**
Expand All @@ -199,14 +208,15 @@ private AnalyzeSentimentResult convertToAnalyzeSentimentResult(DocumentSentiment
*/
private Mono<TextAnalyticsPagedResponse<AnalyzeSentimentResult>> getAnalyzedSentimentResponseInPage(
Iterable<TextDocumentInput> documents, TextAnalyticsRequestOptions options, Context context) {
return service.sentimentWithRestResponseAsync(
return service.sentimentWithResponseAsync(
new MultiLanguageBatchInput().setDocuments(Transforms.toMultiLanguageInput(documents)),
context.addData(AZ_TRACING_NAMESPACE_KEY, COGNITIVE_TRACING_NAMESPACE_VALUE),
options == null ? null : options.getModelVersion(),
options == null ? null : options.isIncludeStatistics(),
context.addData(AZ_TRACING_NAMESPACE_KEY, COGNITIVE_TRACING_NAMESPACE_VALUE))
options == null ? null : options.isIncludeStatistics())
.doOnSubscribe(ignoredValue -> logger.info("A batch of documents - {}", documents.toString()))
.doOnSuccess(response -> logger.info("Analyzed sentiment for a batch of documents - {}", response))
.doOnError(error -> logger.warning("Failed to analyze sentiment - {}", error))
.map(this::toTextAnalyticsPagedResponse);
.map(this::toTextAnalyticsPagedResponse)
.onErrorMap(throwable -> mapToHttpResponseExceptionIfExist(throwable));
}
}
Loading