Skip to content
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions docs/code/MlNetCookBook.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ Please feel free to search this page and use any code that suits your needs.
- [How do I train using cross-validation?](#how-do-i-train-using-cross-validation)
- [Can I mix and match static and dynamic pipelines?](#can-i-mix-and-match-static-and-dynamic-pipelines)
- [How can I define my own transformation of data?](#how-can-i-define-my-own-transformation-of-data)
- [How can I read and write binary data?](#how-can-i-read-and-write-binary-data)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this make sense to be next to the "How do I load data from a text file?" question?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I agree! :)


In reply to: 263541837 [](ancestors = 263541837)


### General questions about the samples

Expand Down Expand Up @@ -1022,3 +1023,46 @@ ITransformer loadedModel;
using (var fs = File.OpenRead(modelPath))
loadedModel = newContext.Model.Load(fs);
```

## How can I read and write binary data?

@rogancarr rogancarr Mar 4, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cookbook examples all have corresponding tests in Tests/Scenarios/Api/CookbookSamples/. Could you please add one for this too? #Resolved

@jwood803 jwood803 Mar 5, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I must be missing something when I try to add the example. It's not finding the CreateDataView method on the ML context. It may be missing a reference for it, but I'm not sure which one it needs. #Resolved

@jwood803 jwood803 Mar 5, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rogancarr I switched to LoadFromEnumerable which is what the sample is already using. Hopefully the test looks good now. #Resolved

Other than using text files ML.NET will allow you to read and write binary data.
Comment thread
codemzs marked this conversation as resolved.
Outdated

@rogancarr rogancarr Mar 4, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

read and write binary data. [](start = 53, length = 28)

Maybe a sentence on why we would do that: Smaller files, faster reading time, no need to specify the schema when you read it, etc. #Resolved


To write binary data you need some data to be able to save. Specifically you need an instance of an `IDavaView`. Below is a code snippet that uses the iris data as an example.

```csharp
// Data model for the iris data
public class IrisData
{
public float Label;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

public float Label; [](start = 4, length = 19)

{ get; set; } for all of these.

public float SepalLength;
public float SepalWidth;
public float PetalLength;
public float PetalWidth;
}

// An array of iris data points
var dataArray = new[] {
new IrisData{Label=1, PetalLength=1, SepalLength=1, PetalWidth=1, SepalWidth=1},
new IrisData{Label=0, PetalLength=2, SepalLength=2, PetalWidth=2, SepalWidth=2}
};

// Create the ML.NET context.
var context = new MLContext();

// Create the data view.
// This method will use the definition of IrisData to understand what columns there are in the
// data view.
var data = context.CreateDataView(dataArray);

@rogancarr rogancarr Mar 4, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the language of Schema Comprehension here, as done above in "How do I look at the intermediate data?". Also can link to this doc: https://github.com/dotnet/machinelearning/blob/master/docs/code/SchemaComprehension.md #Resolved


// Use a FileStream to create a file. Use the stream and the data view in the "SaveAsBinary" method.
using(var stream = new FileStream("./iris.idv", FileMode.Create))
{
context.Data.SaveAsBinary(data, stream);
}
```

To read a binary file, simply use the `context.Data.ReadFromBinary` method and pass in the path of the binary file to read in.

@rogancarr rogancarr Mar 4, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

of the binary file to read in. [](start = 95, length = 31)

Note that the schema does not need to be defined, as the binary file contains the schema! #Resolved


```csharp
var data = context.Data.ReadFromBinary("./iris.idv");
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
using System.Collections.Generic;
using System.IO;
using Microsoft.Data.DataView;
using Microsoft.ML.SamplesUtils;

namespace Microsoft.ML.Samples.Dynamic.DataOperations
{
public class SaveAndLoadFromBinary
{
public static void Example()
{
var mlContext = new MLContext();

// Get a small dataset as an IEnumerable.
IEnumerable<DatasetUtils.SampleTemperatureData> enumerableOfData = DatasetUtils.GetSampleTemperatureData(5);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IEnumerable<DatasetUtils.SampleTemperatureData> [](start = 12, length = 47)

var?
I'm never sure if we should use var or be explicit in samples....

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a var fan, myself, but I'll be more explicit in these examples :)


// Load dataset into an IDataView.
IDataView data = mlContext.Data.LoadFromEnumerable(enumerableOfData);

// Creating a FileStream object to create a file and use
// the stream to create a binary file.
using (var stream = new FileStream("./sample-temp-data.idv", FileMode.Create))
{
mlContext.Data.SaveAsBinary(data, stream);
}

// Load a binary file by file path.
var binaryData = mlContext.Data.LoadFromBinary("./sample-temp-data.idv");
}
}
}