Unable to invoke SageMaker API endpoint

[damoresa]

Hello!

The last few days I've been trying to deploy my trained model to SageMaker so I can query it from an AWS lambda using this SDK.
So far I've been able to train my POC model and deploy and endpoint, but I've not been able to query it after several tries. In the other hand, I've been able to query the generated SavedModel on my own computer.

The API endpoint has been giving me errors I've not been able to decypher, would you be so kind to give me a hand with this?

All my code can be found on this repository.
I'll break down the sources for you:

• aws_dnnreg.py: Script used to fit the model at SageMaker
• aws_dnnreg_cli.py: First client attempt, based on the iris classifier example.
• aws_dnnreg_cli_v2.py: Second client attempt, based on a issue I saw on this Github.
• aws_ecommerce_poc_dnn.py: Training script.
• local/ecommerce-poc-dnn.py: Script I've been using to test the training script at local.
• local/ecommerce-poc-predictor.py: Script I've been using to query the SavedModel locally.

The datasets are as follows:

• Train: AWS-Ecommerce-Train.csv
• Test: AWS-Ecommerce-Test.csv

Thank you for your time.
Yours,
Daniel.

** EDIT ** : This is the issue I based my second client on.

Also adding the log trace I retrieved from CloudWatch:

[2018-03-15 20:33:55,234] ERROR in serving: u'tensorflow/serving/regress' Traceback (most recent call last): File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 165, in _invoke self.transformer.transform(content, input_content_type, requested_output_content_type) File "/opt/amazon/lib/python2.7/site-packages/tf_container/serve.py", line 254, in transform return self.transform_fn(data, content_type, accepts), accepts File "/opt/amazon/lib/python2.7/site-packages/tf_container/serve.py", line 178, in f input = input_fn(serialized_data, content_type) File "/opt/amazon/lib/python2.7/site-packages/tf_container/serve.py", line 211, in _default_input_fn data = self.proxy_client.parse_request(serialized_data) File "/opt/amazon/lib/python2.7/site-packages/tf_container/proxy_client.py", line 47, in parse_request request = request_fn_map[self.prediction_type]() KeyError: u'tensorflow/serving/regress' 2018-03-15 20:33:55,234 ERROR - model server - u'tensorflow/serving/regress' Traceback (most recent call last): File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 165, in _invoke self.transformer.transform(content, input_content_type, requested_output_content_type) File "/opt/amazon/lib/python2.7/site-packages/tf_container/serve.py", line 254, in transform return self.transform_fn(data, content_type, accepts), accepts File "/opt/amazon/lib/python2.7/site-packages/tf_container/serve.py", line 178, in f input = input_fn(serialized_data, content_type) File "/opt/amazon/lib/python2.7/site-packages/tf_container/serve.py", line 211, in _default_input_fn data = self.proxy_client.parse_request(serialized_data) File "/opt/amazon/lib/python2.7/site-packages/tf_container/proxy_client.py", line 47, in parse_request request = request_fn_map[self.prediction_type]() KeyError: u'tensorflow/serving/regress' [2018-03-15 20:33:55,234] ERROR in serving: u'tensorflow/serving/regress' 2018-03-15 20:33:55,234 ERROR - model server - u'tensorflow/serving/regress'

[winstonaws]

Thanks for the very detailed report!

Your example seems very similar to the iris example you mentioned, so it should work the same way. From just eyeballing your code, the potential issue I see is here:
https://github.com/damoresa/ecommerce-poc/blob/master/aws_dnnreg_cli.py#L9

You're attempting to send a list of lists, when the iris example is set up to just receive a list. Can you try changing that and seeing if it fixes it?

I'll set up now to reproduce your model and endpoint from your source files, but that will take some time. I'll update once I have that working.

[winstonaws]

Hey, I've reproduced the issue, and determined that it's a bug in our code. We aren't properly supporting tensorflow serving for regression right now (which is the difference between your code and the iris example - you're using a Regressor and the Iris example uses a Classifier).

I've added a task in our backlog to fix this issue. I'll also work on providing a workaround in the meantime - expect another update on Monday.

[damoresa]

Hello there!

Thanks for taking the time to check this out.
Let me know if I can be of any help.

[winstonaws]

@damoresa after looking at this further, unfortunately, I can't see a fast workaround that will work for DNNRegressor without making a code change on our side. We'll look at the priority for the fix and provide an update when we have more information on when you can expect it.

[damoresa]

Do not worry, take your time.
As a workaround we're deploying the trained SavedModel into an EC2 instance and using Flask to create a REST API we can query from our frontend.

[jesterhazy]

This issue has been closed due to inactivity. If you still have questions, please re-open it.

[fernandoamat]

Hi, has this bug been fixed? I am following the Iris example but with a tf.estimator.LinearRegressor and I hit the same error described above ERROR in serving tensorflow/serving/regress.
If it has not been fixed, is there a workaround?
Thanks!
Fernando