Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions sdk/core/azure-core/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

# Release History

## 1.6.0 (Unreleased)
## 1.6.0 (2020-06-03)

### Bug fixes

Expand All @@ -13,7 +13,7 @@
- Added support for changesets as part of multipart message support #10485
- Add AsyncLROPoller in azure.core.polling #10801
- Add get_continuation_token/from_continuation_token/polling_method methods in pollers (sync and async) #10801
- HttpResponse objects are now pickable #10801
- HttpResponse and PipelineContext objects are now pickable #10801

## 1.5.0 (2020-05-04)

Expand Down
18 changes: 18 additions & 0 deletions sdk/core/azure-core/azure/core/pipeline/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ class PipelineContext(dict):
:param transport: The HTTP transport type.
:param kwargs: Developer-defined keyword arguments.
"""
_PICKLE_CONTEXT = {
'deserialized_data'
}

def __init__(self, transport, **kwargs): # pylint: disable=super-init-not-called
self.transport = transport
Expand All @@ -75,6 +78,21 @@ def __getstate__(self):
del state['transport']
return state

def __reduce__(self):
reduced = super(PipelineContext, self).__reduce__()
saved_context = {}
for key, value in self.items():
if key in self._PICKLE_CONTEXT:
saved_context[key] = value
# 1 is for from __reduce__ spec of pickle (generic args for recreation)
# 2 is how dict is implementing __reduce__ (dict specific)
# tuple are read-only, we use a list in the meantime
reduced = list(reduced)
dict_reduced_result = list(reduced[1])
dict_reduced_result[2] = saved_context
reduced[1] = tuple(dict_reduced_result)
return tuple(reduced)

def __setstate__(self, state):
self.__dict__.update(state)
# Re-create the unpickable entries
Expand Down
32 changes: 32 additions & 0 deletions sdk/core/azure-core/tests/test_universal_pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
#
#--------------------------------------------------------------------------
import logging
import pickle
try:
from unittest import mock
except ImportError:
Expand Down Expand Up @@ -56,6 +57,37 @@
HTTPPolicy,
)

def test_pipeline_context():
kwargs={
'stream':True,
'cont_token':"bla"
}
context = PipelineContext('transport', **kwargs)
context['foo'] = 'bar'
context['xyz'] = '123'
context['deserialized_data'] = 'marvelous'

assert context['foo'] == 'bar'
assert context.options == kwargs

with pytest.raises(TypeError):
context.clear()

with pytest.raises(TypeError):
context.update({})

assert context.pop('foo') == 'bar'
assert 'foo' not in context

serialized = pickle.dumps(context)

revived_context = pickle.loads(serialized)
assert revived_context.options == kwargs
assert revived_context.transport is None
assert 'deserialized_data' in revived_context
assert len(revived_context) == 1


def test_request_history():
class Non_deep_copiable(object):
def __deepcopy__(self, memodict={}):
Expand Down
37 changes: 27 additions & 10 deletions sdk/formrecognizer/azure-ai-formrecognizer/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,31 +4,48 @@

**Breaking Changes**

- `training_files` parameter of `begin_train_model` is renamed to `training_files_url`
- `use_labels` parameter of `begin_train_model` is renamed to `use_training_labels`
- All asynchronous long running operation methods now return an instance of an `AsyncLROPoller` from `azure-core`
- All asynchronous long running operation methods are renamed with the `begin_` prefix to indicate that an `AsyncLROPoller` is returned:
- `train_model` is renamed to `begin_training`
- `recognize_receipts` is renamed to `begin_recognize_receipts`
- `recognize_receipts_from_url` is renamed to `begin_recognize_receipts_from_url`
- `recognize_content` is renamed to `begin_recognize_content`
- `recognize_content_from_url` is renamed to `begin_recognize_content_from_url`
- `recognize_custom_forms` is renamed to `begin_recognize_custom_forms`
- `recognize_custom_forms_from_url` is renamed to `begin_recognize_custom_forms_from_url`
- Sync method `begin_train_model` renamed to `begin_training`
- `training_files` parameter of `begin_training` is renamed to `training_files_url`
- `use_labels` parameter of `begin_training` is renamed to `use_training_labels`
- `list_model_infos` method has been renamed to `list_custom_models`
- Removed `get_form_training_client` from `FormRecognizerClient`
- Added `get_form_recognizer_client` to `FormTrainingClient`
- A `HttpResponseError` is now raised if a model with `status=="invalid"` is returned from the `begin_train_model()` or `train_model()` methods
- A `HttpResponseError` is now raised if a model with `status=="invalid"` is returned from the `begin_training` methods
- `PageRange` is renamed to `FormPageRange`
- `first_page` and `last_page` renamed to `first_page_number` and `last_page_number`, respectively on `FormPageRange`
- `FormField` does not have a page_number.
- `begin_recognize_receipts` APIs now return `RecognizedReceipt` instead of `USReceipt`
- `USReceiptType` is renamed to `ReceiptType`
- `use_training_labels` is now a required positional param in the `begin_training` APIs.
- `stream` and `url` parameters found on methods for `FormRecognizerClient` have been renamed to `form` and `form_url`, respectively.
- For recognize receipt methods, parameters have been renamed to `receipt` and `receipt_url`.
- `FormField` does not have a page_number
- `use_training_labels` is now a required positional param in the `begin_training` APIs
- `stream` and `url` parameters found on methods for `FormRecognizerClient` have been renamed to `form` and `form_url`, respectively
- For `begin_recognize_receipt` methods, parameters have been renamed to `receipt` and `receipt_url`
- `created_on` and `last_modified` are renamed to `requested_on` and `completed_on` in the
`CustomFormModel` and `CustomFormModelInfo` models.
`CustomFormModel` and `CustomFormModelInfo` models
- `models` property of `CustomFormModel` is renamed to `submodels`
- `CustomFormSubModel` is renamed to `CustomFormSubmodel`
- `begin_recognize_receipts` APIs now return `RecognizedReceipt` instead of `USReceipt`
- Removed `USReceipt`. To see how to deal with the return value of `begin_recognize_receipts`, see the recognize receipt samples in the [samples directory](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/formrecognizer/azure-ai-formrecognizer/samples) for details.
- Removed `USReceiptItem`. To see how to access the individual items on a receipt, see the recognize receipt samples in the [samples directory](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/formrecognizer/azure-ai-formrecognizer/samples) for details.
- Removed `USReceiptType` and the `receipt_type` property from `RecognizedReceipt`. See the recognize receipt samples in the [samples directory](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/formrecognizer/azure-ai-formrecognizer/samples) for details.

**New features**

- Support to copy a custom model from one Form Recognizer resource to another
- Authentication using `azure-identity` credentials now supported
- see the [Azure Identity documentation](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/identity/azure-identity/README.md) for more information
- `page_number` attribute has been added to `FormTable`
- All long running operation methods now accept the keyword argument `continuation_token` to restart the poller from a saved state

**Dependency updates**

- Adopted [azure-core](https://pypi.org/project/azure-core/) version 1.6.0 or greater

## 1.0.0b2 (2020-05-06)

Expand Down
38 changes: 17 additions & 21 deletions sdk/formrecognizer/azure-ai-formrecognizer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,10 +140,10 @@ Long-running operations are operations which consist of an initial request sent
followed by polling the service at intervals to determine whether the operation has completed or failed, and if it has
succeeded, to get the result.

Methods that train models or recognize values from forms are modeled as long-running operations. The client exposes
a `begin_<method-name>` method that returns an `LROPoller`. Callers should wait for the operation to complete by
calling `result()` on the operation returned from the `begin_<method-name>` method. Sample code snippets are provided
to illustrate using long-running operations [below](#examples "Examples").
Methods that train models, recognize values from forms, or copy models are modeled as long-running operations.
The client exposes a `begin_<method-name>` method that returns an `LROPoller` or `AsyncLROPoller`. Callers should wait
for the operation to complete by calling `result()` on the operation returned from the `begin_<method-name>` method.
Sample code snippets are provided to illustrate using long-running operations [below](#examples "Examples").


## Examples
Expand Down Expand Up @@ -210,7 +210,7 @@ for cell in table.cells:
```

### Recognize Receipts
Recognize data from USA sales receipts using a prebuilt model.
Recognize data from USA sales receipts using a prebuilt model. [Here][service_recognize_receipt] are the fields the service returns for a recognized receipt.

```python
from azure.ai.formrecognizer import FormRecognizerClient
Expand All @@ -227,21 +227,16 @@ with open("<path to your receipt>", "rb") as fd:
poller = form_recognizer_client.begin_recognize_receipts(receipt)
result = poller.result()

r = result[0]
print("Receipt contained the following values with confidences: ")
print("Receipt Type: {} has confidence: {}".format(r.receipt_type.type, r.receipt_type.confidence))
print("Merchant Name: {} has confidence: {}".format(r.merchant_name.value, r.merchant_name.confidence))
print("Transaction Date: {} has confidence: {}".format(r.transaction_date.value, r.transaction_date.confidence))
print("Receipt items:")
for item in r.receipt_items:
print("...Item Name: {} has confidence: {}".format(item.name.value, item.name.confidence))
print("...Item Quantity: {} has confidence: {}".format(item.quantity.value, item.quantity.confidence))
print("...Individual Item Price: {} has confidence: {}".format(item.price.value, item.price.confidence))
print("...Total Item Price: {} has confidence: {}".format(item.total_price.value, item.total_price.confidence))
print("Subtotal: {} has confidence: {}".format(r.subtotal.value, r.subtotal.confidence))
print("Tax: {} has confidence: {}".format(r.tax.value, r.tax.confidence))
print("Tip: {} has confidence: {}".format(r.tip.value, r.tip.confidence))
print("Total: {} has confidence: {}".format(r.total.value, r.total.confidence))
for receipt in result:
for name, field in receipt.fields.items():
if name == "Items":
print("Receipt Items:")
for idx, items in enumerate(field.value):
print("...Item #{}".format(idx))
for item_name, item in items.value.items():
print("......{}: {} has confidence {}".format(item_name, item.value, item.confidence))
else:
print("{}: {} has confidence {}".format(name, field.value, field.confidence))
```

### Train a model
Expand All @@ -259,7 +254,7 @@ credential = AzureKeyCredential("<api_key>")
form_training_client = FormTrainingClient(endpoint, credential)

container_sas_url = "xxx" # training documents uploaded to blob storage
poller = form_training_client.begin_train_model(container_sas_url)
poller = form_training_client.begin_training(container_sas_url)
model = poller.result()

# Custom model information
Expand Down Expand Up @@ -439,6 +434,7 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con
[cognitive_authentication_aad]: https://docs.microsoft.com/azure/cognitive-services/authentication#authenticate-with-azure-active-directory
[azure_identity_credentials]: ../../identity/azure-identity#credentials
[default_azure_credential]: ../../identity/azure-identity#defaultazurecredential
[service_recognize_receipt]: https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-preview/operations/GetAnalyzeReceiptResult

[cla]: https://cla.microsoft.com
[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,6 @@
TrainingStatus,
CustomFormModelStatus,
FormContentType,
USReceipt,
ReceiptType,
USReceiptItem,
FormTable,
FormTableCell,
TrainingDocumentInfo,
Expand Down Expand Up @@ -45,9 +42,6 @@
'CustomFormModelStatus',
'FormContentType',
'FormContent',
'USReceipt',
'ReceiptType',
'USReceiptItem',
'FormTable',
'FormTableCell',
'TrainingDocumentInfo',
Expand Down
Loading