Debugging tests disabled on Linux and MacOS builds

[mstfbl]

Debugging PR for diagnosing tests disabled on Linux and MacOS builds:

Disabled tests - Disabled build(s):

• BinaryClassifierSymSgdTest() - Linux, MacOS
• TextNormalizingOnnxConversionTest() - Linux
• MatrixFactorizationSimpleTrainAndPredict() - MacOS

Report:

• BinaryClassifierSymSgdTest()
  • Fixed, added new baselines for Linux and MacOS builds with PR Re-enabled BinaryClassifierSymSgdTest with platform-specific baselines #5094 .
  • Test re-disabled on Linux due to varying baselines.
• MatrixFactorizationSimpleTrainAndPredict()
  • The baseline on MacOS varies greatly. In 10 consecutive tests, the calculated MSE ranged between 0.586040127051849 to 0.625339146273452. With these results, a tolerance of Math.Pow(10, -1) is required.
  • I have been able to reproduce this issue on my own MacOS computer.
  • Issue stems from here. This random_shuffle libmf function shuffle values with a rand(). With no given seed, it behaved non-deterministically. By commenting out this shuffling, the MSEs on MacOS and Windows are now very close, within 2 decimal digits, and these are consistent.
  • Fixed with PR Updated libmf and corresponding MatrixFactorizationSimpleTrainAndPredict() baselines per build #5121
• TextNormalizingOnnxConversionTest()

  • Will be fixed when Issue Ubuntu ML .NET Docker Image has no configured locales #5093 is fixed.

  • This test fails on the Linux Net CoreApp 2.1 builds on Linux with the following error:

  • System.InvalidOperationException : Error initializing model :Microsoft.ML.OnnxRuntime.OnnxRuntimeException: [ErrorCode:RuntimeException] Exception during initialization: /onnxruntime_src/onnxruntime/core/providers/cpu/nn/string_normalizer.cc:87 onnxruntime::string_normalizer::Locale::Locale(const string&) Failed to construct locale with name:en_US.UTF-8:locale::facet::_S_create_c_locale name not valid:Please, install necessary language-pack-XX and configure locales

  • The issue being faced here may be different than before, as the error being faced is related to locales. As this test passes on MacOS Net Core App 2.1 and Linux Net Core App 3.1, this may be a Linux Net Core App 2.1 or more specifically Ubuntu Net Core App 2.1 issue.

  • The ONNX team faced the exact same issue as this, and they have fixed it with this PR, which installs the appropiate locales in their docker instance.

[codecov]

Codecov Report

Merging #5087 into master will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #5087   +/-   ##
=======================================
  Coverage   75.66%   75.66%           
=======================================
  Files         993      993           
  Lines      178157   178154    -3     
  Branches    19125    19123    -2     
=======================================
+ Hits       134800   134801    +1     
+ Misses      38136    38129    -7     
- Partials     5221     5224    +3     

Flag	Coverage Δ
#Debug	75.66% <100.00%> (+<0.01%)	⬆️
#production	71.63% <ø> (-0.01%)	⬇️
#test	88.70% <100.00%> (+0.02%)	⬆️

Impacted Files	Coverage Δ
...est/Microsoft.ML.Predictor.Tests/TestPredictors.cs	70.18% <100.00%> (+0.10%)	⬆️
...icrosoft.ML.AutoML/Experiment/SuggestedPipeline.cs	88.65% <0.00%> (-4.13%)	⬇️
...L.AutoML/TrainerExtensions/TrainerExtensionUtil.cs	85.15% <0.00%> (-1.75%)	⬇️
src/Microsoft.ML.Maml/MAML.cs	24.75% <0.00%> (-1.46%)	⬇️
....ML.AutoML/PipelineSuggesters/PipelineSuggester.cs	86.55% <0.00%> (-0.85%)	⬇️
...rosoft.ML.AutoML/ColumnInference/TextFileSample.cs	62.25% <0.00%> (+2.64%)	⬆️
...ameworkCommon/Attributes/IterationDataAttribute.cs	90.90% <0.00%> (+90.90%)	⬆️

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@a2406f6). Click here to learn what that means.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##             master    #5087   +/-   ##
=========================================
  Coverage          ?   75.59%           
=========================================
  Files             ?      990           
  Lines             ?   179205           
  Branches          ?    19287           
=========================================
  Hits              ?   135476           
  Misses            ?    38464           
  Partials          ?     5265           

Flag	Coverage Δ
#Debug	75.59% <100.00%> (?)
#production	71.49% <ø> (?)
#test	88.88% <100.00%> (?)

Impacted Files	Coverage Δ
test/Microsoft.ML.Tests/OnnxConversionTest.cs	99.43% <100.00%> (ø)

[mstfbl]

/azp run

[azure-pipelines]

Pull request contains merge conflicts.

[mstfbl]

/azp run

[azure-pipelines]

Pull request contains merge conflicts.