[MXNet example] how to replace the existing endpoint from the notebook

[richardxin]

when I ran the sample mxnet notebook, every time it created a new endpoint (with timestamps), i.e. "sagemaker-mxnet-py2-cpu-2018-01-25-18-04-31-174"

Let's say I have a use case that the inference end point will be consumed by individual application outside sagemaker. how do i specify the model name and replace if exist in the notebook when creating model and endpoint?

[ChoiByungWook]

Hello,

Thanks for trying out SageMaker!

From my understanding, your use case is to update a model behind a static endpoint that will be consumed by an individual application outside of SageMaker. Is that correct?

This can be done by using the update_endpoint API within the boto3 client: http://boto3.readthedocs.io/en/latest/reference/services/sagemaker.html#SageMaker.Client.update_endpoint.

Creating the endpoint_config can be done through the Session's create_endpoint_config(): https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/session.py#L309.

Unfortunately, it seems that it would require both the Session class and boto3 client for executing your SageMaker job instead of the MXNet's Estimator's functions, which is not the ideal experience. We added this as a feature into our backlog.

Here is some sample code for accomplishing this:

import boto3

from sagemaker.mxnet import MXNet


# MXNet Estimator
# Your estimator parameters here
estimator = MXNet()

# Training Job
estimator.fit()

# Create model within SageMaker
model = estimator.create_model()
session = model.sagemaker_session

container_def = model.prepare_container_def(instance_type='ml.m4.xlarge')
model_name = model.name
role = 'your-role'

session.create_model(model_name, role, container_def)

# Create endpoint config
endpoint_config_name = session.create_endpoint_config(name=model_name,
                                                      model_name=model_name,
                                                      initial_instance_count=1,
                                                      instance_type='ml.m4.xlarge')

# Update desired endpoint with new Endpoint Config
client = boto3.client('sagemaker')
client.update_endpoint(EndpointName='endpoint-name',
                       EndpointConfigName=endpoint_config_name)

If you already have models within SageMaker, then just replace the model_name with the name of your model and a endpoint config should be created, which can be used to update the endpoint.

Please let me know if there is anything else I can assist with.

[haseleyi]

A couple of comments here:

1. The last line sets EndpointConfigName=endpoint_config_name, but endpoint_config_name is the endpoint config itself. For its name, use model_name instead.

2. When I adapted this code, I received "ClientError: An error occurred (ValidationException) when calling the UpdateEndpoint operation: Cannot update in-progress endpoint". My understanding must be flawed here, because I thought that's what the purpose of update_endpoint is. Can anyone help me understand?

[haseleyi]

Update: Got this working! I'm not sure what precluded my above error, but here's the adaptation of the code that worked. Follow-up question: I'm planning to train on a daily or weekly basis. As such, I used an RNG to make model names and config names that likely wouldn't conflict. What's the smarter way to do this?

import random

# Create model within SageMaker
model = estimator.create_model()
session = model.sagemaker_session

container_def = model.prepare_container_def(instance_type='ml.m4.xlarge')
model_name = str(random.random())[2:]
session.create_model(model_name, role, container_def)

# Create endpoint config
config_name = str(random.random())[2:]
session.create_endpoint_config(name=config_name,
                              model_name=model_name,
                              initial_instance_count=1,
                              instance_type='ml.m4.xlarge')

# Update desired endpoint with new Endpoint Config
client = boto3.client('sagemaker')
client.update_endpoint(EndpointName='<name_here>',
                       EndpointConfigName=config_name)

[haseleyi]

Answering my own follow-up with code I found here: Use datetime.

str(datetime.datetime.today()).replace(' ', '-').replace(':', '-').rsplit('.')[0]

[chuyang-deng]

This feature was added within this PR: #606. Updating the endpoint can be done by specifying update_endpoint to be True within the deploy method, usage case can be found here : example