Skip to content

CountFeatureSelectingEstimator no selection support #5000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kere-nel merged 6 commits into from
Apr 10, 2020
Merged

CountFeatureSelectingEstimator no selection support #5000

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
045d26e
updating baseline
kere-nel Mar 25, 2020
e0332a5
slotdroppingtransformer fix
kere-nel Apr 3, 2020
08a22be
resolving conflicts
kere-nel Apr 3, 2020
d677453
updates
kere-nel Apr 6, 2020
29f9bc9
updating docs
kere-nel Apr 6, 2020
b96e73c
resolving comments
kere-nel Apr 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 9 additions & 11 deletions src/Microsoft.ML.Data/Transforms/SlotsDroppingTransformer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -896,27 +896,25 @@ public void SaveAsOnnx(OnnxContext ctx)
public bool SaveAsOnnxCore(OnnxContext ctx, int iinfo, string srcVariableName, string dstVariableName)
{
string opType;
if (_srcTypes[iinfo] is VectorDataViewType)
var slots = _slotDropper[iinfo].GetPreservedSlots();
if (_srcTypes[iinfo] is VectorDataViewType && slots.Count() > 0)

@frank-dong-ms-zz frank-dong-ms-zz Apr 6, 2020 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_srcTypes[iinfo] is VectorDataViewType && slots.Count() > 0 [](start = 20, length = 59)

what if _srcTypes[iinfo] is VectorDataViewType && slots.Count() == 0? should we return true or false?
in current implementation seems it will goes into the "else" condition in line 917 and which should handling double numeric data type #Resolved

@kere-nel kere-nel Apr 6, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If slots.Count == 0, then the column get's "suppressed". It's not handled well by the GatherElements operator, so we create an empty vector in the else clause. Does that make sense?

@frank-dong-ms-zz frank-dong-ms-zz Apr 9, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, but we better to make the logic clear like below, don't let different cases fall into same condition which will make code harder to read and will cause potential issue in future:

if (_srcTypes[iinfo] is VectorDataViewType)
{
if (count > 0) { ... }
else { ... }
}
else { ... }

In reply to: 404404988 [](ancestors = 404404988)

@kere-nel kere-nel Apr 10, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

{
opType = "GatherElements";
IEnumerable<long> slots = _slotDropper[iinfo].GetPreservedSlots();
var slotsVar = ctx.AddInitializer(slots, new long[] { 1, slots.Count() }, "PreservedSlots");
var node = ctx.CreateNode(opType, new[] { srcVariableName, slotsVar }, new[] { dstVariableName }, ctx.GetNodeName(opType), "");
node.AddAttribute("axis", 1);

}
else
{
string constVal;
long[] dims = { 1, 1 };
float[] floatVals = { 0.0f };
long[] keyVals = { 0 };
string[] stringVals = { "" };
if (_rawTypes[iinfo] is TextDataViewType)
Comment thread
frank-dong-ms-zz marked this conversation as resolved.
constVal = ctx.AddInitializer(stringVals, dims);
else if (_rawTypes[iinfo] is KeyDataViewType)
constVal = ctx.AddInitializer(keyVals, dims);
var type = _srcTypes[iinfo].GetItemType();
if (type == TextDataViewType.Instance)
return false;
else if (type == NumberDataViewType.Single)
constVal = ctx.AddInitializer(new float[] { 0 }, new long[] { 1, 1 });
else
constVal = ctx.AddInitializer(floatVals, dims);
constVal = ctx.AddInitializer(new double[] { 0 }, new long[] { 1, 1 });

opType = "Identity";
ctx.CreateNode(opType, constVal, dstVariableName, ctx.GetNodeName(opType), "");
Expand Down
75 changes: 40 additions & 35 deletions test/Microsoft.ML.Tests/OnnxConversionTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1648,57 +1648,62 @@ public void UseKeyDataViewTypeAsUInt32InOnnxInput()
Done();
}

[Fact]
public void FeatureSelectionOnnxTest()
[Theory]
[InlineData(DataKind.String)]
[InlineData(DataKind.Single)]
[InlineData(DataKind.Double)]
public void FeatureSelectionOnnxTest(DataKind dataKind)
{
var mlContext = new MLContext(seed: 1);

string dataPath = GetDataPath("breast-cancer.txt");

var dataView = ML.Data.LoadFromTextFile(dataPath, new[] {
new TextLoader.Column("ScalarFloat", DataKind.Single, 6),
new TextLoader.Column("VectorFloat", DataKind.Single, 1, 4),
new TextLoader.Column("VectorDouble", DataKind.Double, 4, 8),
var dataView = mlContext.Data.LoadFromTextFile(dataPath, new[] {
new TextLoader.Column("Scalar", dataKind, 6),
new TextLoader.Column("Vector", dataKind, 1, 6),
new TextLoader.Column("Label", DataKind.Boolean, 0)
});

var columns = new[] {
new CountFeatureSelectingEstimator.ColumnOptions("FeatureSelectDouble", "VectorDouble", count: 1),
new CountFeatureSelectingEstimator.ColumnOptions("ScalFeatureSelectMissing690", "ScalarFloat", count: 690),
new CountFeatureSelectingEstimator.ColumnOptions("ScalFeatureSelectMissing100", "ScalarFloat", count: 100),
new CountFeatureSelectingEstimator.ColumnOptions("VecFeatureSelectMissing690", "VectorDouble", count: 690),
new CountFeatureSelectingEstimator.ColumnOptions("VecFeatureSelectMissing100", "VectorDouble", count: 100)
};
var pipeline = ML.Transforms.FeatureSelection.SelectFeaturesBasedOnCount("FeatureSelect", "VectorFloat", count: 1)
.Append(ML.Transforms.FeatureSelection.SelectFeaturesBasedOnCount(columns))
.Append(ML.Transforms.FeatureSelection.SelectFeaturesBasedOnMutualInformation("FeatureSelectMIScalarFloat", "ScalarFloat"))
.Append(ML.Transforms.FeatureSelection.SelectFeaturesBasedOnMutualInformation("FeatureSelectMIVectorFloat", "VectorFloat"));
IEstimator<ITransformer>[] pipelines =
{
// one or more features selected
mlContext.Transforms.FeatureSelection.SelectFeaturesBasedOnCount("VectorOutput", "Vector", count: 690).
Append(mlContext.Transforms.FeatureSelection.SelectFeaturesBasedOnCount("ScalarOutput", "Scalar", count: 100)),

var model = pipeline.Fit(dataView);
var transformedData = model.Transform(dataView);
var onnxModel = mlContext.Model.ConvertToOnnxProtobuf(model, dataView);
// no feature selected => column suppressed
mlContext.Transforms.FeatureSelection.SelectFeaturesBasedOnCount("VectorOutput", "Vector", count: 800).
Append(mlContext.Transforms.FeatureSelection.SelectFeaturesBasedOnCount("ScalarOutput", "Scalar", count: 800)),

var onnxFileName = "countfeatures.onnx";
var onnxModelPath = GetOutputPath(onnxFileName);
mlContext.Transforms.FeatureSelection.SelectFeaturesBasedOnMutualInformation("VectorOutput", "Vector").
Append(mlContext.Transforms.FeatureSelection.SelectFeaturesBasedOnMutualInformation("ScalarOutput", "Scalar"))
};
for (int i = 0; i < pipelines.Length; i++)
{
//There's currently no support for suppressed string columns, since onnx string variable initiation is not supported
if (dataKind == DataKind.String && i > 0)
break;
var model = pipelines[i].Fit(dataView);
var transformedData = model.Transform(dataView);
var onnxModel = mlContext.Model.ConvertToOnnxProtobuf(model, dataView);

SaveOnnxModel(onnxModel, onnxModelPath, null);
var onnxFileName = "countfeatures.onnx";
var onnxModelPath = GetOutputPath(onnxFileName);

if (IsOnnxRuntimeSupported())
{
// Evaluate the saved ONNX model using the data used to train the ML.NET pipeline.
var onnxEstimator = mlContext.Transforms.ApplyOnnxModel(onnxModelPath);
var onnxTransformer = onnxEstimator.Fit(dataView);
var onnxResult = onnxTransformer.Transform(dataView);
CompareSelectedColumns<float>("FeatureSelectMIScalarFloat", "FeatureSelectMIScalarFloat", transformedData, onnxResult);
CompareSelectedColumns<float>("FeatureSelectMIVectorFloat", "FeatureSelectMIVectorFloat", transformedData, onnxResult);
CompareSelectedColumns<float>("ScalFeatureSelectMissing690", "ScalFeatureSelectMissing690", transformedData, onnxResult);
CompareSelectedColumns<double>("VecFeatureSelectMissing690", "VecFeatureSelectMissing690", transformedData, onnxResult);
SaveOnnxModel(onnxModel, onnxModelPath, null);

if (IsOnnxRuntimeSupported())
{
// Evaluate the saved ONNX model using the data used to train the ML.NET pipeline.
var onnxEstimator = mlContext.Transforms.ApplyOnnxModel(onnxModelPath);
var onnxTransformer = onnxEstimator.Fit(dataView);
var onnxResult = onnxTransformer.Transform(dataView);
CompareResults("VectorOutput", "VectorOutput", transformedData, onnxResult);
CompareResults("ScalarOutput", "ScalarOutput", transformedData, onnxResult);
}
}
Done();
}



[Fact]
public void SelectColumnsOnnxTest()
{
Expand Down