Cross Validation and TrainTest #212
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -469,6 +469,35 @@ | |
| "ILearningPipelineLoader" | ||
| ] | ||
| }, | ||
| { | ||
| "Name": "Data.TransformModelArrayConverter", | ||
| "Desc": "Create and array variable", | ||
| "FriendlyName": null, | ||
| "ShortName": null, | ||
| "Inputs": [ | ||
| { | ||
| "Name": "TransformModel", | ||
| "Type": { | ||
| "Kind": "Array", | ||
| "ItemType": "TransformModel" | ||
| }, | ||
| "Desc": "The models", | ||
| "Required": true, | ||
| "SortOrder": 1.0, | ||
| "IsNullable": false | ||
| } | ||
| ], | ||
| "Outputs": [ | ||
| { | ||
| "Name": "OutputModel", | ||
| "Type": { | ||
| "Kind": "Array", | ||
| "ItemType": "TransformModel" | ||
| }, | ||
| "Desc": "The model array" | ||
| } | ||
| ] | ||
| }, | ||
| { | ||
| "Name": "Models.AnomalyDetectionEvaluator", | ||
| "Desc": "Evaluates an anomaly detection scored dataset.", | ||
|
|
@@ -1238,6 +1267,116 @@ | |
| "IEvaluatorOutput" | ||
| ] | ||
| }, | ||
| { | ||
| "Name": "Models.CrossValidationResultsCombiner", | ||
| "Desc": "Combine the metric data views returned from cross validation.", | ||
| "FriendlyName": null, | ||
| "ShortName": null, | ||
| "Inputs": [ | ||
| { | ||
| "Name": "Kind", | ||
| "Type": { | ||
| "Kind": "Enum", | ||
| "Values": [ | ||
| "SignatureBinaryClassifierTrainer", | ||
| "SignatureMultiClassClassifierTrainer", | ||
| "SignatureRankerTrainer", | ||
| "SignatureRegressorTrainer", | ||
| "SignatureMultiOutputRegressorTrainer", | ||
| "SignatureAnomalyDetectorTrainer", | ||
| "SignatureClusteringTrainer" | ||
| ] | ||
| }, | ||
| "Desc": "Specifies the trainer kind, which determines the evaluator to be used.", | ||
| "Required": true, | ||
| "SortOrder": 0.0, | ||
| "IsNullable": false, | ||
| "Default": "SignatureBinaryClassifierTrainer" | ||
| }, | ||
| { | ||
| "Name": "OverallMetrics", | ||
| "Type": { | ||
| "Kind": "Array", | ||
| "ItemType": "DataView" | ||
| }, | ||
| "Desc": "Overall metrics datasets", | ||
| "Required": false, | ||
| "SortOrder": 1.0, | ||
| "IsNullable": false, | ||
| "Default": null | ||
| }, | ||
| { | ||
| "Name": "PerInstanceMetrics", | ||
| "Type": { | ||
| "Kind": "Array", | ||
| "ItemType": "DataView" | ||
| }, | ||
| "Desc": "Per instance metrics datasets", | ||
| "Required": false, | ||
| "SortOrder": 2.0, | ||
| "IsNullable": false, | ||
| "Default": null | ||
| }, | ||
| { | ||
| "Name": "ConfusionMatrix", | ||
| "Type": { | ||
| "Kind": "Array", | ||
| "ItemType": "DataView" | ||
| }, | ||
| "Desc": "Confusion matrix datasets", | ||
| "Required": false, | ||
| "SortOrder": 3.0, | ||
| "IsNullable": false, | ||
| "Default": null | ||
| }, | ||
| { | ||
| "Name": "Warnings", | ||
| "Type": { | ||
| "Kind": "Array", | ||
| "ItemType": "DataView" | ||
| }, | ||
| "Desc": "Warning datasets", | ||
| "Required": false, | ||
| "SortOrder": 4.0, | ||
| "IsNullable": false, | ||
| "Default": null | ||
| }, | ||
| { | ||
| "Name": "LabelColumn", | ||
| "Type": "String", | ||
| "Desc": "The label column name", | ||
| "Aliases": [ | ||
| "Label" | ||
| ], | ||
| "Required": false, | ||
| "SortOrder": 4.0, | ||
| "IsNullable": false, | ||
| "Default": "Label" | ||
| } | ||
| ], | ||
| "Outputs": [ | ||
| { | ||
| "Name": "Warnings", | ||
| "Type": "DataView", | ||
| "Desc": "Warning dataset" | ||
| }, | ||
| { | ||
| "Name": "OverallMetrics", | ||
| "Type": "DataView", | ||
| "Desc": "Overall metrics dataset" | ||
| }, | ||
| { | ||
| "Name": "PerInstanceMetrics", | ||
| "Type": "DataView", | ||
| "Desc": "Per instance metrics dataset" | ||
| }, | ||
| { | ||
| "Name": "ConfusionMatrix", | ||
| "Type": "DataView", | ||
| "Desc": "Confusion matrix dataset" | ||
| } | ||
| ] | ||
| }, | ||
| { | ||
| "Name": "Models.CrossValidator", | ||
| "Desc": "Cross validation for general learning", | ||
|
|
@@ -1301,9 +1440,28 @@ | |
| "Name": "Model", | ||
| "Type": "PredictorModel", | ||
| "Desc": "The model", | ||
| "Required": false, | ||
| "SortOrder": 1.0, | ||
| "IsNullable": false, | ||
| "Default": null | ||
| }, | ||
| { | ||
| "Name": "TransformModel", | ||
| "Type": "TransformModel", | ||
| "Desc": "The transform model", | ||
| "Required": false, | ||
| "SortOrder": 2.0, | ||
| "IsNullable": false, | ||
| "Default": null | ||
| }, | ||
| { | ||
| "Name": "UseTransformModel", | ||
| "Type": "Bool", | ||
| "Desc": "Indicates to use transform model instead of predictor model.", | ||
| "Required": false, | ||
| "SortOrder": 3.0, | ||
| "IsNullable": false, | ||
| "Default": false | ||
| } | ||
| ] | ||
| }, | ||
|
|
@@ -1367,35 +1525,31 @@ | |
| "Desc": "The final model including the trained predictor model and the model from the transforms, provided as the Input.TransformModel." | ||
| }, | ||
| { | ||
| "Name": "TransformModel", | ||
| "Type": { | ||
| "Kind": "Array", | ||
| "ItemType": "TransformModel" | ||
| }, | ||
| "Desc": "The final model including the trained predictor model and the model from the transforms, provided as the Input.TransformModel." | ||
|
Member
There was a problem hiding this comment. Is the description correct? #Resolved
Member
Author
There was a problem hiding this comment. Yeah, so we introduced a second option in CV and TrainTest where the final model is of ITransformModel. In reply to: 191033404 [](ancestors = 191033404)
Member
There was a problem hiding this comment. Update the part of the comment that talks about bundling the predictor model in it.
Member
Author
There was a problem hiding this comment. Can you please clarify? The comment already talks about predictor being included in the final model. In reply to: 191049219 [](ancestors = 191049219) |
||
| }, | ||
| { | ||
| "Name": "Warnings", | ||
| "Type": "DataView", | ||
| "Desc": "Warning dataset" | ||
| }, | ||
| { | ||
| "Name": "OverallMetrics", | ||
| "Type": "DataView", | ||
| "Desc": "Overall metrics dataset" | ||
| }, | ||
| { | ||
| "Name": "PerInstanceMetrics", | ||
| "Type": "DataView", | ||
| "Desc": "Per instance metrics dataset" | ||
| }, | ||
| { | ||
| "Name": "ConfusionMatrix", | ||
| "Type": "DataView", | ||
| "Desc": "Confusion matrix dataset" | ||
| } | ||
| ] | ||
|
|
@@ -2904,9 +3058,28 @@ | |
| "Name": "Model", | ||
| "Type": "PredictorModel", | ||
| "Desc": "The model", | ||
| "Required": false, | ||
| "SortOrder": 1.0, | ||
| "IsNullable": false, | ||
| "Default": null | ||
| }, | ||
| { | ||
| "Name": "TransformModel", | ||
| "Type": "TransformModel", | ||
| "Desc": "Transform model", | ||
| "Required": false, | ||
| "SortOrder": 2.0, | ||
| "IsNullable": false, | ||
| "Default": null | ||
| }, | ||
| { | ||
| "Name": "UseTransformModel", | ||
| "Type": "Bool", | ||
| "Desc": "Indicates to use transform model instead of predictor model.", | ||
| "Required": false, | ||
| "SortOrder": 3.0, | ||
| "IsNullable": false, | ||
| "Default": false | ||
| } | ||
| ] | ||
| }, | ||
|
|
@@ -2960,6 +3133,11 @@ | |
| "Type": "PredictorModel", | ||
| "Desc": "The final model including the trained predictor model and the model from the transforms, provided as the Input.TransformModel." | ||
| }, | ||
| { | ||
| "Name": "TransformModel", | ||
| "Type": "TransformModel", | ||
| "Desc": "The final model including the trained predictor model and the model from the transforms, provided as the Input.TransformModel." | ||
| }, | ||
| { | ||
| "Name": "Warnings", | ||
| "Type": "DataView", | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So what's the difference between this one and Data.PredictorModelArrayConverter and Data.IDataViewArrayConverter. All of them "Create and array variable" #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
they create array variables of different types, like ITranformModel, IPredictorModel, IDV etc.
In reply to: 191003203 [](ancestors = 191003203)