Wrapper to annotate the MIME type of some data

[Keno]

I frequently find myself in the situation, where I have some data (say image/png), that I just want to have show up properly in the REPL without having to create a whole extra type for it. I don't think we currently have something like that. I would propose something like this (name to be bikeshedded), unless there already exists something better. cc @stevengj

Example usage:

[stevengj]

vs. display("image/png", data)?

[yuyichao]

OT. I thought you've got julia running on a oscilloscope...

[Keno]

The point is that I want it to be passed around automatically. Ideally the MIMEdata would just be inside a function (or say the result of a manipulate cough).

[stevengj]

Fair enough. MIMEData("image/png", data) would be easier to type.

[stevengj]

I thought we normally put spaces after , in argument lists, but not around <:

[Keno]

Yeah, fair enough. This was quick and dirty, because I have to get something done. Once the name/API is agreed on I'll clean this up, export, add tests/docs, etc.

[shashi]

this definition seems redundant given the one on the next line.

[Keno]

The one on the next line is ambigous with (io::IO,::MIME"text/plain", ::Any).

[shashi]

Ah, alright

[vtjnash]

the ambiguity warning caused by the conflict between this design and the ability to write an arbitrary object as a specific mime type seems concerning to me

[Keno]

text/plain is a bit of a special case, because it falls back to show.

[vtjnash]

I thought the whole goal of the display function was to decouple the display of the data from the raw bytes. That is, for this example, the data would be wrapped in an Images.Image wrapper type that understood how to meaningfully output the data in any context (e.g. a text description or an image or a graphical popup or a clipboard, etc), not just dumb forwarding of the the input bytes.

[ivarne]

+1

Good Julia code should not need this, but when you just have one format availible and want to pass it through the display system a shortcut like this seems reasonable.

[shashi]

+1 MIMEData sounds OK to me.

[ivarne]

I'd like something like MIMEWrapper or MIMERaw. MIMEData doesn't seem rough enough to describe how the type work.

A constructor that reinterprets anything to Array{UInt8} might also be reasonable. I'd like to be able to use MIMERaw("text/html"."This is some <blink>blinking</blink> <big>html</big>")

[Keno]

There's already Base.Text/Base.HTML. It would be good to figure out how this fits together. I'm not entirely sure what the purpose of those is (right not it seems to coerce the display, which is slightly different from what this does).

[vtjnash]

I came across this looking back over old PRs, and I'm pretty sure this is one of the missing pieces that should fit into #13825 somehow. That part still remains to be figured out, however.

[vtjnash]

fwiw, i think IPython effectively calls this same concept a DisplayObject (https://github.com/ipython/ipython/blob/657391d313d5105700a79d82a66468dd7c6ea6f7/IPython/core/display.py#L347)

[StefanKarpinski]

I don't love the name. I do wonder how much we should be tying all of this to MIME types, per se. @fperez and @ellisonbg, any thoughts on the naming of DisplayObject, having lived with it for a while?

[ellisonbg]

@StefanKarpinski

If I am understanding the discussion, I think it may make sense to have at least two levels of APIs.

• A high-level API should be super simple and mostly decoupled from the MIME type details. When we talk about the display function in ipython, we say "it is just like print, but for richer representations." This level of API should automatically know how to compute any and all representations (MIME types) and send them back. For this API, I really would stick with the display naming convention.
• A lower level API that the high-level one sits on top of of display. This API can and should know about MIME types, etc., but I wouldn't expect most users to rely on this.

Does this make sense?

[StefanKarpinski]

Thanks for the feedback, @ellisonbg. I think that given our nomenclature, since this type is used to wrap an object that has effectively already had show(mime, x) called on it, and in light of the planned changes in #14052, I wonder if this shouldn't be called Shown{mime}(x). I.e. an object that is already displayed in a particular mime type. Then again, when I spell that sentence out, calling it DisplayObject{mime}(x) also seems pretty reasonable.

[Keno]

I just wanted this again, so bump, I guess ;)?

[StefanKarpinski]

Data point on naming: whenever I've thought about this, MIMEData is the name that's stuck in my mind.

[vtjnash]

There's a number of deprecations to fix, but I think there's strong consensus now that this should be merged.

[StefanKarpinski]

@vtjnash: would it be more effective if you just fixed this PR up yourself?

[clarkevans]

@vtjnash Yes. This seems very related to #38540 -- I'm looking for a package-independent way to annotate that a particular chunk of data is text/html so that it can be treated in an interoperable manner. So, with writemime deprecated, equivalent changes to display and cousins would be included?

Is there a way to get this in the Julia 1.7 LTS? It would improve interoperability of otherwise completely independent packages which work with hypertext. Perhaps for Julia 2.0, one could explore breaking changes such as deprecating HTML and Text in favor of using this approach.

[clarkevans]

@StefanKarpinski - is this PR still acceptable if updated? There are many cases where otherwise independent applications could use a way to share text/html and other labeled without having to know about each other. In particular, I can see applications consisting of libraries that each use different HTML generation libraries. It would be convenient if the output of one of these libraries could be directly used inside another. A way to signal that a string is already valid text/html would aid this sort of interoperability. I was thinking of using HTML for this purpose, but its constructor has surprising behavior -- something even more minimalist might be appropriate.

[StefanKarpinski]

Dunno, maybe @Keno or @vtjnash have an opinion, five years later?

[clarkevans]

I think this feature could be useful for sharing multimedia content between collaborating libraries that do not otherwise know about each other -- MIMEData pairs an arbitrary data object with a preferred mime type you would use to show it. Perhaps it is a useful generalization of the rather underdeveloped HTML datatype. As a side note, @vtjnash's comment that data should be untyped seems correct; forcing the object to be rendered as UInt8 is premature (it may be a proxy object, for example).

Even so, I was barking up the wrong tree. I've since learned that showable("text/html", obj) along with show(io, MIME("text/html"), obj) are sufficient for my purposes. That said, I can imagine future cases where a set of attachments might be passed along a chain of applications, and an wrapper object such as MIMEData might let a content producer better coordinate with a visual display without prior knowledge and without intermediate middleware having to be involved.

[vtjnash]

We approved this years ago, but still needs to be rebased as it is likely stale, and docs and tests before we can merge

[DilumAluthge]

@Keno and/or @clarkevans - can you add tests and docs?

[KristofferC]

There is also https://github.com/tkf/DisplayAs.jl.

[DilumAluthge]

Also, CI is failing.

[fingolfin]

Still any plans to update this, or can it be closed?

[stevengj]

I rebased this and modernized it, incorporating comments by @vtjnash above, in case we ever want to merge it.

Still needs tests if we still want it. (But there doesn't seem to have been a burning need for it?)

[vtjnash]

Also needs an export and to be added to docs, but seems still useful even if rare (how much do packages actually notice MIME-type customizations they could be doing anyways?)