Skip to content

Updated OnnxScoringEstimator's documentation #4966

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
antoniovs1029 merged 8 commits into from
Mar 25, 2020
Merged

Updated OnnxScoringEstimator's documentation #4966

Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e833580
Updated onnx estimator documentation
antoniovs1029 Mar 24, 2020
2f38c07
Added CUDA and CuDNN versions
antoniovs1029 Mar 24, 2020
cca0d7b
Make it explicit that the GPU nuget should replace the CPU one if GPU…
antoniovs1029 Mar 24, 2020
3e412d1
Update with Luis suggestion
antoniovs1029 Mar 24, 2020
3b5743f
Adding link to supported GPU list
antoniovs1029 Mar 25, 2020
57dbcdc
Update src/Microsoft.ML.OnnxTransformer/OnnxTransform.cs
antoniovs1029 Mar 25, 2020
36e0c49
Update src/Microsoft.ML.OnnxTransformer/OnnxTransform.cs
antoniovs1029 Mar 25, 2020
2b57310
Update src/Microsoft.ML.OnnxTransformer/OnnxTransform.cs
antoniovs1029 Mar 25, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions src/Microsoft.ML.OnnxTransformer/OnnxTransform.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -702,17 +702,17 @@ public NamedOnnxValue GetNamedOnnxValue()
/// | Does this estimator need to look at the data to train its parameters? | No |
/// | Input column data type | Known-sized vector of <xref:System.Single> or <xref:System.Double> types |
/// | Output column data type | As specified by the ONNX model |
/// | Required NuGet in addition to Microsoft.ML | Microsoft.ML.OnnxTransformer (always), Microsoft.ML.OnnxRuntime.Gpu (only if GPU processing is used) |
/// | Required NuGet in addition to Microsoft.ML | Microsoft.ML.OnnxTransformer (always), either Microsoft.ML.OnnxRuntime 1.2.0 (for CPU processing) or Microsoft.ML.OnnxRuntime.Gpu 1.2.0 (only if GPU processing is desired) |

@kere-nel kere-nel Mar 24, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for changing from "used" to "desired"?

@antoniovs1029 antoniovs1029 Mar 24, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just found it confusing because you can't use GPU processing without the nuget, so you should install it before it is "used".

Do you think it's clearer with 'used'?

@kere-nel kere-nel Mar 24, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm would "available" be a better word? Doesn't ml.net fail if there isn't a GPU, but you try to use the nuget?

@luisquintanilla luisquintanilla Mar 24, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// | Required NuGet in addition to Microsoft.ML | Microsoft.ML.OnnxTransformer (always), either Microsoft.ML.OnnxRuntime 1.2.0 (for CPU processing) or Microsoft.ML.OnnxRuntime.Gpu 1.2.0 (only if GPU processing is desired) |
/// | Required NuGet in addition to Microsoft.ML | Microsoft.ML.OnnxTransformer (always), either Microsoft.ML.OnnxRuntime 1.2.0 (for CPU processing) or Microsoft.ML.OnnxRuntime.Gpu 1.2.0 (for GPU processing if GPU is available) |

@natke natke Mar 24, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Luis' suggestion is good. Can I clarify what happens if:
a) you install both
b) you install the GPU nuget and you don't don't run with a GPU?

@antoniovs1029 antoniovs1029 Mar 24, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I've accepted Luis' suggestion 😄

Regarding your questions, @natke
a) I hadn't tested the case of installing both, because I was told by @harishsk that users should install only one. I have just made a quick test installing both on my laptop (without GPU), and it doesn't throw any errors when running, so I guess it is simply using the non-GPU nuget. I am having some troubles using my remote GPU computer, so I haven't had the chance to test it there.
b) If I install the GPU nuget on a computer without GPU, the following error is thrown when running the code:

    System.TypeInitializationException : The type initializer for 'Microsoft.ML.OnnxRuntime.NativeMethods' threw an exception.
    ---- System.DllNotFoundException : Unable to load DLL 'onnxruntime' or one of its dependencies: The specified module could not be found. (Exception from HRESULT: 0x8007007E)

Reason for this is that the onnxruntime DLL has a dependency on CUDA and CuDNN, so if those aren't installed, it can't load onnxruntime either.

@antoniovs1029 antoniovs1029 Mar 24, 2020 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested this now on my remote GPU. And when installing both nugets, the GPU nuget gets ignored when running a sample code, it throws the exception that is expected when trying to use the GPU feature without the GPU nuget:
Unable to find an entry point named 'OrtSessionOptionsAppendExecutionProvider_CUDA' in DLL 'onnxruntime'

So in general it seems that when installing both nugets, the GPU one gets completely ignored.

/// | Exportable to ONNX | No |
///
/// Supports inferencing of models in ONNX 1.2 and 1.3 format (opset 7, 8 and 9), using the
/// [Microsoft.ML.OnnxRuntime](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime/) library.
/// Supports inferencing of models in ONNX 1.6 format (opset 11), using the
/// [Microsoft.ML.OnnxRuntime](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime/) library (version 1.2.0).
/// Models are scored on CPU by default. If GPU execution is needed (optional), use the
/// NuGet package available at [Microsoft.ML.OnnxRuntime.Gpu](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.Gpu/)
/// and download [CUDA 9.1 Toolkit](https://developer.nvidia.com/cuda-downloads) and [cuDNN](https://developer.nvidia.com/cudnn).
/// NuGet package available at [Microsoft.ML.OnnxRuntime.Gpu](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.Gpu/) (version 1.2.0) instead of the Microsoft.ML.OnnxRuntime nuget (which is for CPU processing)
Comment thread
antoniovs1029 marked this conversation as resolved.
Outdated
/// and download [CUDA 10.1 Toolkit](https://developer.nvidia.com/cuda-downloads) and [cuDNN 7.6.5](https://developer.nvidia.com/cudnn) (as indicated on [Onnxruntime's documentation](https://github.com/Microsoft/onnxruntime#default-gpu-cuda)).
/// Set parameter 'gpuDeviceId' to a valid non-negative integer. Typical device ID values are 0 or 1.
/// The inputs and outputs of the ONNX models must be Tensor type. Sequence and Maps are not yet supported.
/// OnnxRuntime currently works on Windows and Ubuntu 16.04 Linux 64-bit platforms. Mac OS to be supported soon.
/// OnnxRuntime works on Windows, MacOS and Ubuntu 16.04 Linux 64-bit platforms.
/// Visit [ONNX Models](https://github.com/onnx/models) to see a list of readily available models to get started with.
/// Refer to [ONNX](http://onnx.ai) for more information.
///
Expand Down