Sagemaker Localization Timeout

[kannappan]

Sagemaker times out during long running inference jobs.

Whether this would solve by changing the instance type or instance count

Instance Type: ml.m4.4xlarge

20:53:12
10.32.0.2 - - [03/Jan/2019:20:53:11 +0000] "POST /invocations HTTP/1.1" 200 231 "-" "AHC/2.0"

20:53:15
2019/01/03 20:53:14 [error] 55#55: *754210 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.32.0.2, server: , request: "POST /invocations HTTP/1.1", upstream: "http://unix:/tmp/gunicorn.sock/invocations", host: "model.aws.local:8080"

20:53:15
10.32.0.2 - - [03/Jan/2019:20:53:14 +0000] "POST /invocations HTTP/1.1" 504 192 "-" "AHC/2.0"

20:53:15
[2019-01-03 20:53:14 +0000] [45] [CRITICAL] WORKER TIMEOUT (pid:3906)


[laosunhust]

what's your use case?
Sagemaker has both real-time inference and batch transform. If you do not need immediate response for your inference job, consider batch transform job:
https://docs.aws.amazon.com/sagemaker/latest/dg/ex1-model-deployment.html

[laosunhust]

sagemaker real-time inference has timeout set as 60 seconds:
https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.html#your-algorithms-inference-code-container-response

If your use case requires real-time inference, consider using instances with more CPU power, such as c5 family instances.