Skip to content

[text analytics] change return type of analyze actions to list of list #18994

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kristapratico merged 5 commits into from
Jun 14, 2021
Merged

[text analytics] change return type of analyze actions to list of list #18994

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d4f55d4
change return type of analyze actions to list of list
iscai-msft May 27, 2021
0307a4a
address johan's and krista's comments
iscai-msft Jun 1, 2021
86f8294
limit # of action calls in readme example
iscai-msft Jun 1, 2021
f192e29
Merge branch 'master' of https://github.com/Azure/azure-sdk-for-pytho…
iscai-msft Jun 3, 2021
917305d
Merge branch 'master' into update_analyze_output
kristapratico Jun 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions sdk/textanalytics/azure-ai-textanalytics/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

## 5.1.0b8 (Unreleased)

**Breaking Changes**
- Changed the response structure of `being_analyze_actions`. Now, we return a list of list of an action result of documents
- Removed `AnalyzeActionsType`
- Removed `AnalyzeActionsResult`
- Removed `AnalyzeActionsError`

## 5.1.0b7 (2021-05-18)

Expand Down
114 changes: 49 additions & 65 deletions sdk/textanalytics/azure-ai-textanalytics/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -532,72 +532,56 @@ poller = text_analytics_client.begin_analyze_actions(
)

# returns multiple actions results in the same order as the inputted actions
result = poller.result()

first_action_result = next(result)
print("Results of Entities Recognition action:")
docs = [doc for doc in first_action_result.document_results if not doc.is_error]

for idx, doc in enumerate(docs):
print("\nDocument text: {}".format(documents[idx]))
for entity in doc.entities:
print("Entity: {}".format(entity.text))
print("...Category: {}".format(entity.category))
print("...Confidence Score: {}".format(entity.confidence_score))
print("...Offset: {}".format(entity.offset))
print("------------------------------------------")

second_action_result = next(result)
print("Results of PII Entities Recognition action:")
docs = [doc for doc in second_action_result.document_results if not doc.is_error]

for idx, doc in enumerate(docs):
print("Document text: {}".format(documents[idx]))
for entity in doc.entities:
print("Entity: {}".format(entity.text))
print("Category: {}".format(entity.category))
print("Confidence Score: {}\n".format(entity.confidence_score))
print("------------------------------------------")

third_action_result = next(result)
print("Results of Key Phrase Extraction action:")
docs = [doc for doc in third_action_result.document_results if not doc.is_error]

for idx, doc in enumerate(docs):
print("Document text: {}\n".format(documents[idx]))
print("Key Phrases: {}\n".format(doc.key_phrases))
print("------------------------------------------")

fourth_action_result = next(result)
print("Results of Linked Entities Recognition action:")
docs = [doc for doc in fourth_action_result.document_results if not doc.is_error]

for idx, doc in enumerate(docs):
print("Document text: {}\n".format(documents[idx]))
for linked_entity in doc.entities:
print("Entity name: {}".format(linked_entity.name))
print("...Data source: {}".format(linked_entity.data_source))
print("...Data source language: {}".format(linked_entity.language))
print("...Data source entity ID: {}".format(linked_entity.data_source_entity_id))
print("...Data source URL: {}".format(linked_entity.url))
print("...Document matches:")
pages = poller.result()

for doc, document_results in zip(documents, pages):
print("\nDocument text: {}".format(doc))
recognize_entities_result = document_results[0]
assert not recognize_entities_result.is_error
Copy link
Member

@johanste johanste Jun 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably not a good use of an assert. Any document could have failed for a transient reason, so you are not really in a position to say that this didn't fail. I would print a "failed" message instead...

Copy link
Member

@johanste johanste Jun 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, scratch that, this is in the readme.md. It is a bit long for the readme already. I don't think we need to show all types of actions in the readme example inline. It definitely won't scale as new action types comes along. Can we limit this to two?

print("...Results of Recognize Entities Action:")
for entity in recognize_entities_result.entities:
print("......Entity: {}".format(entity.text))
print(".........Category: {}".format(entity.category))
print(".........Confidence Score: {}".format(entity.confidence_score))
print(".........Offset: {}".format(entity.offset))

recognize_pii_entities_result = document_results[1]
assert not recognize_pii_entities_result.is_error
print("...Results of Recognize PII Entities action:")
for entity in recognize_pii_entities_result.entities:
print("......Entity: {}".format(entity.text))
print(".........Category: {}".format(entity.category))
print(".........Confidence Score: {}".format(entity.confidence_score))

extract_key_phrases_result = document_results[2]
assert not extract_key_phrases_result.is_error
print("...Results of Extract Key Phrases action:")
print("......Key Phrases: {}".format(extract_key_phrases_result.key_phrases))

recognize_linked_entities_result = document_results[3]
assert not recognize_linked_entities_result.is_error
print("...Results of Recognize Linked Entities action:")
for linked_entity in recognize_linked_entities_result.entities:
print("......Entity name: {}".format(linked_entity.name))
print(".........Data source: {}".format(linked_entity.data_source))
print(".........Data source language: {}".format(linked_entity.language))
print(".........Data source entity ID: {}".format(linked_entity.data_source_entity_id))
print(".........Data source URL: {}".format(linked_entity.url))
print(".........Document matches:")
for match in linked_entity.matches:
print("......Match text: {}".format(match.text))
print(".........Confidence Score: {}".format(match.confidence_score))
print(".........Offset: {}".format(match.offset))
print(".........Length: {}".format(match.length))
print("------------------------------------------")

fifth_action_result = next(result)
print("Results of Sentiment Analysis action:")
docs = [doc for doc in fifth_action_result.document_results if not doc.is_error]

for doc in docs:
print("Overall sentiment: {}".format(doc.sentiment))
print("Scores: positive={}; neutral={}; negative={} \n".format(
doc.confidence_scores.positive,
doc.confidence_scores.neutral,
doc.confidence_scores.negative,
print("............Match text: {}".format(match.text))
print("............Confidence Score: {}".format(match.confidence_score))
print("............Offset: {}".format(match.offset))
print("............Length: {}".format(match.length))

analyze_sentiment_result = document_results[4]
assert not analyze_sentiment_result.is_error
print("...Results of Analyze Sentiment action:")
print("......Overall sentiment: {}".format(analyze_sentiment_result.sentiment))
print("......Scores: positive={}; neutral={}; negative={} \n".format(
analyze_sentiment_result.confidence_scores.positive,
analyze_sentiment_result.confidence_scores.neutral,
analyze_sentiment_result.confidence_scores.negative,
))
print("------------------------------------------")
```
Expand Down
10 changes: 2 additions & 8 deletions sdk/textanalytics/azure-ai-textanalytics/azure/ai/textanalytics/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,7 @@
RecognizeLinkedEntitiesAction,
RecognizePiiEntitiesAction,
ExtractKeyPhrasesAction,
AnalyzeActionsResult,
RequestStatistics,
AnalyzeActionsType,
AnalyzeActionsError,
_AnalyzeActionsType,
HealthcareEntityRelationRoleType,
HealthcareRelation,
HealthcareRelationRole,
Expand Down Expand Up @@ -93,10 +90,7 @@
'RecognizeLinkedEntitiesAction',
'RecognizePiiEntitiesAction',
'ExtractKeyPhrasesAction',
'AnalyzeActionsResult',
'RequestStatistics',
'AnalyzeActionsType',
"AnalyzeActionsError",
'_AnalyzeActionsType',
"PiiEntityCategoryType",
"HealthcareEntityRelationType",
"HealthcareEntityRelationRoleType",
Expand Down
64 changes: 1 addition & 63 deletions sdk/textanalytics/azure-ai-textanalytics/azure/ai/textanalytics/_models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1356,7 +1356,7 @@ def __repr__(self):
.format(self.positive, self.neutral, self.negative)[:1024]


class AnalyzeActionsType(str, Enum):
class _AnalyzeActionsType(str, Enum):
"""The type of action that was applied to the documents
"""
RECOGNIZE_ENTITIES = "recognize_entities" #: Entities Recognition action.
Expand All @@ -1365,67 +1365,6 @@ class AnalyzeActionsType(str, Enum):
RECOGNIZE_LINKED_ENTITIES = "recognize_linked_entities" #: Linked Entities Recognition action.
ANALYZE_SENTIMENT = "analyze_sentiment" #: Sentiment Analysis action.


class AnalyzeActionsResult(DictMixin):
"""AnalyzeActionsResult contains the results of a recognize entities action
on a list of documents. Returned by `begin_analyze_actions`

:ivar document_results: A list of objects containing results for all Entity Recognition actions
included in the analysis.
:vartype document_results: list[~azure.ai.textanalytics.RecognizeEntitiesResult]
:ivar bool is_error: Boolean check for error item when iterating over list of
actions. Always False for an instance of a AnalyzeActionsResult.
:ivar action_type: The type of action this class is a result of.
:vartype action_type: str or ~azure.ai.textanalytics.AnalyzeActionsType
:ivar ~datetime.datetime completed_on: Date and time (UTC) when the result completed
on the service.
:ivar statistics: Overall statistics for the action result.
:vartype statistics: ~azure.ai.RequestStatistics
"""
def __init__(self, **kwargs):
self.document_results = kwargs.get("document_results")
self.is_error = False
self.action_type = kwargs.get("action_type")
self.completed_on = kwargs.get("completed_on")
self.statistics = kwargs.get("statistics")

def __repr__(self):
return "AnalyzeActionsResult(document_results={}, is_error={}, action_type={}, completed_on={}, " \
"statistics={})".format(
repr(self.document_results),
self.is_error,
self.action_type,
self.completed_on,
repr(self.statistics)
)[:1024]


class AnalyzeActionsError(DictMixin):
"""AnalyzeActionsError is an error object which represents an an
error response for an action.

:ivar error: The action result error.
:vartype error: ~azure.ai.textanalytics.TextAnalyticsError
:ivar bool is_error: Boolean check for error item when iterating over list of
results. Always True for an instance of a DocumentError.
"""

def __init__(self, **kwargs):
self.error = kwargs.get("error")
self.is_error = True

def __repr__(self):
return "AnalyzeActionsError(error={}, is_error={}".format(
repr(self.error), self.is_error
)

@classmethod
def _from_generated(cls, error):
return cls(
error=TextAnalyticsError(code=error.code, message=error.message, target=error.target)
)


class RecognizeEntitiesAction(DictMixin):
"""RecognizeEntitiesAction encapsulates the parameters for starting a long-running Entities Recognition operation.

Expand Down Expand Up @@ -1719,7 +1658,6 @@ def to_generated(self):
)
)


class RequestStatistics(DictMixin):
def __init__(self, **kwargs):
self.documents_count = kwargs.get("documents_count")
Expand Down
12 changes: 6 additions & 6 deletions sdk/textanalytics/azure-ai-textanalytics/azure/ai/textanalytics/_request_handlers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
RecognizePiiEntitiesAction,
RecognizeLinkedEntitiesAction,
AnalyzeSentimentAction,
AnalyzeActionsType,
_AnalyzeActionsType,
)

def _validate_input(documents, hint, whole_input_hint):
Expand Down Expand Up @@ -71,14 +71,14 @@ def _validate_input(documents, hint, whole_input_hint):

def _determine_action_type(action):
if isinstance(action, RecognizeEntitiesAction):
return AnalyzeActionsType.RECOGNIZE_ENTITIES
return _AnalyzeActionsType.RECOGNIZE_ENTITIES
if isinstance(action, RecognizePiiEntitiesAction):
return AnalyzeActionsType.RECOGNIZE_PII_ENTITIES
return _AnalyzeActionsType.RECOGNIZE_PII_ENTITIES
if isinstance(action, RecognizeLinkedEntitiesAction):
return AnalyzeActionsType.RECOGNIZE_LINKED_ENTITIES
return _AnalyzeActionsType.RECOGNIZE_LINKED_ENTITIES
if isinstance(action, AnalyzeSentimentAction):
return AnalyzeActionsType.ANALYZE_SENTIMENT
return AnalyzeActionsType.EXTRACT_KEY_PHRASES
return _AnalyzeActionsType.ANALYZE_SENTIMENT
return _AnalyzeActionsType.EXTRACT_KEY_PHRASES

def _check_string_index_type_arg(string_index_type_arg, api_version, string_index_type_default="UnicodeCodePoint"):
string_index_type = None
Expand Down
Loading