A case for DOM-less rendering

[simonihmig]

Based on a lively discussion on Twitter with @pzuraq et al, I am trying to summarize our (that's @nickschot and me) experience and the shortcomings of using Ember to build a declarative layer for DOM-less rendering.
In our specific case rendering a 3D scene, using an additional layer of abstraction by using an Entity-Component-System to improve composability, to large parts inspired by the popular A-Frame framework.

Tell me more...

Rendering a template essentially creates a side-effect, by calling (imperative) APIs that you would
normally not use by yourself. The "render target" is predominantly DOM, and the APIs are DOM-APIs like document.createElement() or el.appendChild(). And as such DOM as the rendering target is built into GlimmerVM itself (unlike AFAIK react which needs react-dom for DOM-bindings, but also allows for completely DOM-less environments like react-native).

In the following I want to bring up shortcomings of our current Ember/Glimmer APIs when trying to
render to a different render target (a 3D scene, managed by some 3D library like three.js
or babylon.js in our case, but that could be also something like a leaflet map etc.)

Managing hierarchies

Just as DOM represents a hierarchy of elements, a 3D scene is also organized in a hierarchy of nodes. For example you could parent a light node to a mesh node, so when you move the mesh (say a car) the light would follow.

In DOM-land this is easy, as Glimmer will manage the hierarchy automatically:

<Sidebar>
  <Button>
</Sidebar>

By nesting the button component inside of the sidebar component, the rendering side-effect of the button component (a <button> DOM element) will automatically be added as a child to the sidebar (basically using sidebarEl.appendChild(buttonEl)).

It would be essential that this is just as easy in a no-DOM world, like:

<Scene>
  <Box>
    <Light>
  </Box>
</Scene>

Unfortunately it is not. Here the Light component, which would create a light instance using the 3D library's API like new PointLight(), would need to e.g. set its parent to the instance created by the Box component, to match the hierarchy in the template. But as we now have to manage the rendering in a 3D context by ourselves, instead of relying on GlimmerVM's built-in DOM-rendering support, we need to know the light's parent (component).

Legacy Ember.Component classes do have a parentView property with which this would be possible, but Glimmer components do not have this, and it's currently not possible to provide that functionality using a custom component manager (which we already use).

We could use contextual components or other yielded context to pass the hierarchy along:

<Scene as |scene|>
  <Box @parent={{scene}} as |box|>
    <Light @parent={{box}}/>
  </Box>
</Scene>

or

<Scene as |s|>
  <s.box as |b|>
    <b.light/>
  </s.box>
</Scene>

But I see a number of serious problems with that:

I would call this "implementation-driven design" rather than "design-driven implementation": it
considerably falls short of the easy of use and readability of DOM-based rendering, or other declarative systems like A-Frame or react-three-fiber which do not have to make these kind of DX sacrifices.

The second example in particular has another big drawback, in that it requires every 3D component to yield any other component, although they don't really have to know of each other. This introduces serious difficulties to enable extensibility, like having components from a library, which now would also have to yield user-provided custom components, just for the purpose of managing the hierarchy.

And it provides a severe possibility to shoot yourself in the foot, as the visible hierarchy - provided by the nesting and indentation of component, as we know it from DOM - is not really relevant for the actual hierarchy of the rendering side-effect. Can you spot the problem here:

<Scene as |s|>
  <s.box as |b|>
    <s.light/>
  </s.box>
</Scene>

The light will actually not be a child of the box! 🤯

tl;dr

We need a way for a custom component manager to supply its managed components their parent component, not for our regular DOM-based components (where this is managed automatically for us in GlimmerVM), but for other render targets were we have to manage the hierarchy of the rendering side-effect by ourselves.

DOM-less modifiers

Modifiers provide an extraordinarily elegant way to compose functionality, splitting previously big fat monolithic classes to small, reusable functionality that do just one thing, but do it well (aka the "Unix philosophy").

But they have one caveat: the API is inherently coupled to DOM. While there are use cases for things that very much match modifiers in a DOM-less world.

Let's take the most common modifier as an example: {{on}} to handle events. It takes the rendering side-effect as its "context", which in DOM-world is a DOM node, and attaches a listener.

In a 3D scene, we have just the same use case: we want something to happen when the user clicks on a 3D object. So continuing with our primitive example from above, something like this:

<Scene>
  <Box {{three-on "click" this.doSomething}}>
    <Light>
  </Box>
</Scene>

Here the rendering side-effect would not be a DOM node, but a node in our 3D scene (e.g. a mesh), that needs to be passed as the context of the modifier to do its event handling work (which is very different btw to DOM events, as it requires raycasting calculations in 3D space to find the affected 3D object "under the mouse")

From a user's point of view, the above example of a 3D-world modifier would IMO pretty much match with how we think of modifiers in a DOM-world. But again, this is not possible as the modifier manager only knows of DOM elements as the modifiers "context".

You could certainly implement event handling functionality inside of the component, like <Box @onClick={{this.doSomething}}>. But that feels like a big step backwards, similar to how Ember.Component classes had all these event handler methods, instead of the simpler composability over inheritance that we got with modifiers.

This falls apart even more if you want to add special behavior that is not possible to bake into every component that might need this, like some physics behaviour (using e.g. cannon.js):

<Scene {{physics gravity=(vector3 0 -9.81 0)}}>
  <Box {{physics mass=1 restitution=0.9}}>
    <Light>
  </Box>
</Scene>

Outlook

Currently we have found a way to somehow make our 3D library work with user-facing APIs similar to the examples above, i.e. managing hierarchy implicitly and enabling DOM-less modifiers. However these are just "dirty workarounds" by doing custom AST transforms, which are quite ugly and probably error prone. But at least it enabled us to design our APIs the way we wanted them to be, so prioritizing ease of use over ease of implementation.

But it showed that though Ember is in principle able to support DOM-less rendering, there are still considerable shortcomings when digging deeper into the space and focusing on DX-friendly APIs.

So I hope this helps to drive the conversation, to make Ember even more awesome, by fully embracing DOM-less rendering!

Note: our current work is publicly available as ember-ecsy-babylon and ecsy-babylon, however it's still very much WIP and not ready for general use

[NullVoxPopuli]

cc @karimbeyrouti

[NullVoxPopuli]

@simonihmig there is a lot of good stuff in here!

Part of this is why I made: https://github.com/NullVoxPopuli/ember-lifecycle-component, to help out with creating 3d scenes. Demos and perf trade-offs here: https://nullvoxpopuli.github.io/ember-three-boxes-demo/
(@pzuraq is aware of some of the perf costs in the GlimmerVM and the demo is set to auto-publish as ember-source is released, so as the GlimmerVM improves, so will the demo)

---

Anywho, overall, I think we should def find a way to support these use cases. In doing so, I think we'd also support a way to have "Contexts" (https://reactjs.org/docs/context.html), which would allow for a "Scene" to be a context, and protect app-devs from passing the scene object around, which would vastly improve DX, imo. Your "Managing Hierarchies" section covers this nicely 💯

Re: DOM-less modifiers
So, we'd need some API for declaring what attributes (including modifiers) bind to?

maybe inside the component manager (and maybe providing an API to the component instance itself):

setAttributes(this.boxMesh);

idk.

[pzuraq]

I just want to put out there that long term, I really disagree with the general direction of ember-lifecycle-component. The perf benefits in that benchmark come primarily from the fact that Glimmer VM is not properly optimizing tags, and the fact that we don’t have a way to side-effect while rendering other than via modifiers. The former is being worked on as we continue to streamline the VM, and the later is being solved by the work on @use and resources.

I think it would be very unfortunate if the community started using ember-lifecycle-component commonly to address these problems. It would undo a lot of the work that went into the design of Glimmer components.

That said, I really do appreciate the work you put into it @NullVoxPopuli, I think it shows us exactly where our weaknesses are and some things we can do to address them 😄

Re: the larger discussion, I think this is an amazing summary of the pain points and issues @simonihmig! Thank you for taking this on, it’s very much appreciated.

[karimbeyrouti]

Thank you for tagging me @NullVoxPopuli . One of the issues I ran against when working on this: https://github.com/karimbeyrouti/ember-three-ui was not having each component aware of scene hierarchy, and having to pass down the parent felt less than ideal:

<Scene as |scene|>
  <Box @parent={{scene}} as |box|>
    <Light @parent={{box}}/>
  </Box>
</Scene>

Also yielding all the entities in a template to manage the hierarchy like the example below was not ideal either and added unnecessary complexity to the templates:

<Scene as |s|>
  <s.box as |b|>
    <b.light/>
  </s.box>
</Scene>

It would be great to have each component be aware of its parentage (or context?)
i really think that would provide a nice api for these systems. It would be
a lot more intuitive to just have something like that:

<Scene>
  <Box >
    <Light/>
  </Box>
</Scene>

as scene hierarchy is implied in the markup.
While these little demos are maintainable:

https://github.com/karimbeyrouti/ember-three-ui/tree/master/tests/dummy/app/components/ember-three-demo/demos

I could see manually managing parentage in larger applications
quickly becoming tedious.

[kellyselden]

Slight aside: if anyone wants to join https://github.com/ember-vr as a place to store all these ideas and experiments, let me know. If you have no intent of jumping into VR and staying on screen, that's fine too and I'll shove off :)

[simonihmig]

Part of this is why I made: https://github.com/NullVoxPopuli/ember-lifecycle-component, to help out with creating 3d scenes.

Yep, aware of your work! 🙂 We also used a custom component manager to introduce a didUpdate component hook. I did not mention this, as this was already properly supported by Ember's (component manager) APIs.

I think we'd also support a way to have "Contexts" (https://reactjs.org/docs/context.html), which would allow for a "Scene" to be a context

Yes, we also pass the scene as a "context", which is implemented by using the existing parent/child relationships to find the first ancestor that owns a context. But the issue still applies that managing the parent relationship is quite ugly, as described in the issue.

[wagenet]

I'm closing this due to inactivity. This doesn't mean that the idea presented here is invalid, but that, unfortunately, nobody has taken the effort to spearhead it and bring it to completion. Please feel free to advocate for it if you believe that this is still worth pursuing. Thanks!

[simonihmig]

I still feel very strongly about this! At least for the "Managing hierarchies" part mentioned above, which is the main pain point.

I have used declarative 3D rendering in the past, and if you have ever had to deal with it with the usual imperative APIs, you can definitely feel the power and joy of this approach! I did this by learning how to deal with the shortcomings of Ember in this regard, but that really s*cks, and shows that there is a real gap with what you can do with Ember when not rendering to DOM. Which definitely should be closed IMO! Other ecosystems (React, Vue, Svelte etc.) don't have that limitation, and they show what incredible things you can do. Look at the popularity of https://github.com/pmndrs/react-three-fiber for example, and its own sub-ecosystem.

I am willing to invest the time it requires to work on an RFC. However I won't be able to do this without active guidance from someone else. Especially someone who is well aware of the inner workings of GlimmerVM, as I think whatever we come up with will have to be balanced against the implications it has on the GlimmerVM.

And I would need some guidance on the direction of the RFC. There are two main questions to be answered:

• what is the main new feature we need to expose?
• on what level (high/low) do we want to do that?

With regards to the first question, there are basically two ways to go for:

• a) we could expose the parent component to the child component. This is what I mentioned in the initial issue description
• b) alternatively, we could provide something like "context". See this (also closed 😉) issue, and my related comment there: Add context-like solution to avoid props drilling #775. People have proposed this for the use case of "props drilling", but it also would enable the use case I brought up here about managing the component hierarchy. Basically each parent component would then spawn its own context, yielding itself (or whatever the child actually needs from the parent) to its (direct) children. This is what Svelte Cubed does for example.

And with regard to the second question, we would need to decide if the new API we expose is something that is read to use, or more a low-level (see our "manager" APIs) API that ships the minimum feature to implement this on top of this. We probably can agree that we wouldn't want a high level API for a) (like a "parent component" readily available for all Glimmer components). That's why I was already thinking about exposing this "parent component" only at the component manager level, to not make this a general purpose feature. With regards to b), we could make a real user-friendly context API, or again just at a low level (also component manager?) something that enables building that.

So again, I am willing to work on that, and bring the perspective and experience of declarative 3D (or just DOM-less) rendering, but need a sparring partner to decide on the questions mentioned above, especially with regard to what this means for GlimmerVM. @wagenet if you could help to get the ball rolling here, this would be awesome!

[lolmaus]

a) we could expose the parent component to the child component. This is what I mentioned in the initial issue description

Note that this has been solved in userland by @miguelcobain in https://github.com/miguelcobain/ember-composability-tools.

But this addon is not trivial to use, it is likely to cause huge overhead (compared to a hypothetical native, low-level implementation) and I don't know how Octane-friendly it is.

We do need a native solution. @wagenet, please reopen.

[simonihmig]

Note that this has been solved in userland by @miguelcobain in https://github.com/miguelcobain/ember-composability-tools.

Well, yes and no. Given this example from its Readme...

<LeafletMap @lat={{51.505}} @lng={{-0.09}} @zoom={{13}} as |layers|>
  <layers.tile @url="http://sometiles.com/{z}/{x}/{y}.png"/>
</LeafletMap>

... it seems to force you to yield the things you can use as children (here a "tile"). This is basically the possible workaround I mentioned in the issue description:

<Scene as |s|>
  <s.box as |b|>
    <b.light/>
  </s.box>
</Scene>

It might work ok if the tree you are managing has a limited depth (here 2), like a map and all the things you can add directly to it. But not so much with deeply nested nodes (like in a 3D scene). All the drawbacks I mentioned before (composability issues, manual error prone hierarchy management) still apply...

We do need a native solution

So yes, tackling this with a truly beautiful DX does require upstream changes!

[wagenet]

@simonihmig would you be interested in turning this into an RFC?

[simonihmig]

@wagenet yes, absolutely. As I stated above I am totally willing to do this, but as I stated also I'd need some help:

I am willing to invest the time it requires to work on an RFC. However I won't be able to do this without active guidance from someone else

So basically I need someone qualified to guide me through the questions I raised in #597 (comment) above. Or at least having someone would increase the chances of getting this accepted by orders of magnitude.

[wagenet]

@simonihmig I’ve reopened this and assigned @wycats. He should be able to help you figure out a path forwards.