[Feature Request] Scikit-learn local mode support (or, a refactor to allow it for all frameworks?)

[zmjjmz]

System Information

• Framework (e.g. TensorFlow) / Algorithm (e.g. KMeans): Scikit-Learn
• Framework Version: 0.20.0
• Python Version: 2.7
• CPU or GPU: n/a
• Python SDK Version: 1.18.13
• Are you using a custom image: no

Describe the problem

The scikit learn framework doesn't seem to support the 'local' instance type for either training jobs or the prediction endpoint.

Any call like sagemaker.sklearn.estimator.SKLearn(..., train_instance_type='local') or sklearn_estimator.deploy(..., instance_type='local') results in botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the CreateTrainingJob operation: 1 validation error detected: Value 'local' at 'resourceConfig.instanceType' failed to satisfy constraint.

[chuyang-deng]

Hi @zmjjmz, thank you for using SageMaker, can you provide with the code you are running with the job? And please make sure you are not using sagemaker.session with 'local' instance type.

[zmjjmz]

Hey there,

I can't really easily create a reproduction of the code I'm trying to use, but I'm able to reproduce the ClientError by changing the call here to use train_instance_type='local'.

I can get the behavior I want (i.e. running locally) by using a sagemaker.LocalSession instead of a standard sagemaker.Session -- however the API for using this differs from the API for the TensorFlow estimator (where you simply give the instance type as 'local' but use a normal sagemaker.Session), which is a bit frustrating is all.

[laurenyu]

hi @zmjjmz, thanks for the detail. That's interesting that you were able to use Local Mode + TF with a normal sagemaker.Session - that shouldn't work either.

If you don't specify anything for sagemaker_session, the estimator (Scikit-learn, TF, or otherwise) will determine behind-the-scenes which type of session object to use.

[zmjjmz]

Ah, that makes sense -- in the code I have that was using the TF Estimator I wasn't specifying the sagemaker_session argument. Thanks for the clarification! I'll close this.

[leolorenzoluis]

@laurenyu If I pass in local with sagemaker_session set, it fails. But if it doesn't it works fine locally? So if I don't pass in a sagemaker_session then it should load local?

[laurenyu]

@leolorenzoluis there's a different session object for handling local jobs: sagemaker.LocalSession. You must use this if you are using Local Mode. If sagemaker_session is unset, the estimator will figure out which type of session object to use based on the instance type.