Add DataView Type Register (supporting loading in-memory image and other in-memory custom types)

[wschin]

Fix #3162 by considering image type in our bridge code. Also fix #3723, fix #3369, fix #3274, fix #445, fix #3460, fix #2121, fix #2495, fix #3784.

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@51b10fc). Click here to learn what that means.
The diff coverage is 84.84%.

@@            Coverage Diff            @@
##             master    #3263   +/-   ##
=========================================
  Coverage          ?   72.64%           
=========================================
  Files             ?      807           
  Lines             ?   145126           
  Branches          ?    16220           
=========================================
  Hits              ?   105424           
  Misses            ?    35281           
  Partials          ?     4421

Flag	Coverage Δ
#Debug	72.64% <84.84%> (?)
#production	68.19% <72.72%> (?)
#test	88.93% <96.96%> (?)

Impacted Files	Coverage Δ
src/Microsoft.ML.DataView/ImageDataViewType.cs	70.96% <ø> (ø)
...osoft.ML.Data/DataView/InternalSchemaDefinition.cs	57.53% <100%> (ø)
src/Microsoft.ML.Data/Utils/ApiUtils.cs	91.74% <100%> (ø)
src/Microsoft.ML.Data/Data/SchemaDefinition.cs	70.93% <68%> (ø)
src/Microsoft.ML.Data/DataView/TypedCursor.cs	83.38% <75%> (ø)
test/Microsoft.ML.Tests/ImagesTests.cs	98.6% <96.96%> (ø)

[TomFinley]

Hi @wschin, thanks for working on this!

I do not consider it acceptable that our answer to the problem of what prediction engine does with its reflection based mechanism is that this core assembly has to have a direct, actual dependence on the type -- quite apart from the already excellent objections raised by @Ivanidzo4ka and @eerhardt, this is incompatible with IDataView's own principle as having an extensible type system.

I will post more details on what I'd consider at least a more correct alternative in the existing thread.

[eerhardt]

[artidoro]

The PR looks great, but we will need a sample for this work. It's fine if it comes in another PR. Could you adapt a test and make it a sample? #Resolved

[artidoro]

Also would it be useful to surface the DataViewTypeManager in MLContext? Or some of it's methods? #Pending

[wschin]

For this advanced functionality, I'd keep it as is for now. If we add an alias for it, we will have to support it for years.

---

In reply to: 496640481 [](ancestors = 496640481)

[wschin]

Added two samples.

---

In reply to: 496639086 [](ancestors = 496639086)

[artidoro]

[ADD-eNavarro]

Hi!
Is there a date for this to make it into ML.NET? I really could use this feature, but it seems a little stuck.
Thanks!

[johncalvowtg]

Hello!
We have actually the same problem here at Wisetech. I am trying to consume a TensorFlow model with multiple multi-dimensional inputs . Same message and no workaround on this.

I've tried some of your examples, but none of them worked for me.

Please let us know if you manage to fully support TensorFlow integration with ML.NET.

Regards,
John Calvo-Martinez Ph.D.

[jamsoft]

Phew, it's not just me going mad. I've been trying to use in memory images all day and have so far failed to get any of this to work. I have SO question open as well in case anyone wants to follow.

[Webreaper]

@jamsoft Did you remove the SO question? I'm also looking for an ML.Net sample for image classification that takes a bitmap directly rather than a folder of images. Seems sad that so many people are asking for this and MSFT's ML.Net team still haven't put together a useful sample.

[michaelgsharp]

@Webreaper I thought we had a sample or documentation about it, but I just went and looked and couldn't find anything myself. I'll make sure we have something. The basic idea is that you have 2 pipelines. 1 pipeline loads the images and stops when it has the images loaded as bytes, and the second pipeline starts from where you would use these image bytes already.

I can' upload a zip file here since its not a github issue, but I do have a sample interactive notebook that shows it. If you can create a new issue I can get attach it for you there.

[Webreaper]

Thanks @michaelgsharp. Hopefully this should be sufficient? #5980