tracked-built-ins built-in

[NullVoxPopuli]

Propose making tracked-built-ins built-in, which unblocks the implementations of #1000 (array) and #999 (hash)

Rendered

Summary

This pull request is proposing a new RFC.

To succeed, it will need to pass into the Exploring Stage, followed by the Accepted Stage.

A Proposed or Exploring RFC may also move to the Closed Stage if it is withdrawn by the author or if it is rejected by the Ember team. This requires an "FCP to Close" period.

An FCP is required before merging this PR to advance to Accepted.

Upon merging this PR, automation will open a draft PR for this RFC to move to the Ready for Released Stage.

Exploring Stage Description

This stage is entered when the Ember team believes the concept described in the RFC should be pursued, but the RFC may still need some more work, discussion, answers to open questions, and/or a champion before it can move to the next stage.

An RFC is moved into Exploring with consensus of the relevant teams. The relevant team expects to spend time helping to refine the proposal. The RFC remains a PR and will have an Exploring label applied.

An Exploring RFC that is successfully completed can move to Accepted with an FCP is required as in the existing process. It may also be moved to Closed with an FCP.

Accepted Stage Description

To move into the "accepted stage" the RFC must have complete prose and have successfully passed through an "FCP to Accept" period in which the community has weighed in and consensus has been achieved on the direction. The relevant teams believe that the proposal is well-specified and ready for implementation. The RFC has a champion within one of the relevant teams.

If there are unanswered questions, we have outlined them and expect that they will be answered before Ready for Release.

When the RFC is accepted, the PR will be merged, and automation will open a new PR to move the RFC to the Ready for Release stage. That PR should be used to track implementation progress and gain consensus to move to the next stage.

Checklist to move to Exploring

• The team believes the concepts described in the RFC should be pursued.
• The label S-Proposed is removed from the PR and the label S-Exploring is added.
• The Ember team is willing to work on the proposal to get it to Accepted

Checklist to move to Accepted

• This PR has had the Final Comment Period label has been added to start the FCP
• The RFC is announced in #news-and-announcements in the Ember Discord.
• The RFC has complete prose, is well-specified and ready for implementation.
  • All sections of the RFC are filled out.
  • Any unanswered questions are outlined and expected to be answered before Ready for Release.
  • "How we teach this?" is sufficiently filled out.
• The RFC has a champion within one of the relevant teams.
• The RFC has consensus after the FCP period.

[ef4]

Notes from first discussing of this proposal in RFC review:

• regardless of whether we decide to accept this change, it would be good to replace the tracked storage polyfill with a real implementation first.
• it does seem implied by the built-in array and hash helper RFCs that tracked implementations of those things would exist at a fairly low level (assuming they get implemented as glimmer keywords, glimmer itself would necessarily know about implementation of TrackedArray and TrackedObject).

[NullVoxPopuli]

tracked-storage-primitive in implementation here: emberjs/ember.js#20814

[NullVoxPopuli]

related to non-reactive entries for array and hash: #1070 (Array and Object)

[josemarluedke]

Great work on this RFC! I’d like to propose a utility function, such as reactive, that could create the appropriate tracked data structure based on the input type. This could simplify the process of creating tracked values and make the API more flexible.

For example, if you pass an object to the reactive function, it would return a TrackedObject. If you pass an array, it would return a TrackedArray. This would allow developers to more easily create tracked data structures without needing to manually specify the type each time.

import { reactive } from '@ember/reactive';

const trackedObj = reactive({ key: 'value' }); // returns TrackedObject
const trackedArr = reactive([1, 2, 3]); // returns TrackedArray

Additionally, I’d suggest reconsidering the naming of Tracked*. To better align with the language used in other frameworks and make the terms more intuitive, perhaps these could be renamed to Reactive*. This would emphasize the reactive nature of these structures and help make the API more consistent with other reactive libraries.

[NullVoxPopuli]

realized that i don't need tracked-storage-primitives or cells to implement this.

PR here, work in progress: glimmerjs/glimmer-vm#1713.

By skipping that intermediary user-land abstraction, the TrackedArray implementation would likely be even faster.

(implementation happening in glimmer-vm, so that glimmer-vm can implement keywords that build on these structures)

[NullVoxPopuli]

All tests are ported over to glimmer-vm now -- tho I (re)learned that glimmer-vm doesn't have each-in, so I skipped those tests for now.

Now that the infra is ported for quickly testing collection reactivity, porting over Object, Map, Set, WeakMap, WeakSet will be trivial.

[Eredrim]

Hello!
I love the idea and I think these new reactive structure would be really useful for many purposes. However, it's not clear for me how TrackedArray is different of EmberArray (not the mutable one). EmberArray is currently what I use to track arrays. I fear it might be complicated to maintain both structures and suggest to use one in place of another one.

[kategengler]

Hello! I love the idea and I think these new reactive structure would be really useful for many purposes. However, it's not clear for me how TrackedArray is different of EmberArray (not the mutable one). EmberArray is currently what I use to track arrays. I fear it might be complicated to maintain both structures and suggest to use one in place of another one.

EmberArray should not be used. TrackedArray is preferred and is the way forward. This was addressed in the original RFC for tracked-built-ins https://github.com/emberjs/rfcs/blob/master/text/0812-tracked-built-ins.md?plain=1#L40 This RFC is about making the external implementation internal.

[Eredrim]

EmberArray should not be used. TrackedArray is preferred and is the way forward. This was addressed in the original RFC for tracked-built-ins https://github.com/emberjs/rfcs/blob/master/text/0812-tracked-built-ins.md?plain=1#L40 This RFC is about making the external implementation internal.

Thanks for your answer and for clarification. Considering that, maybe you should consider deprecating EmberArray usage or at least update API documentation to signify TrackedArray should be used instead.

[kategengler]

Thanks for your answer and for clarification. Considering that, maybe you should consider deprecating EmberArray usage or at least update API documentation to signify TrackedArray should be used instead.

We are certainly working towards that.

[gossi]

Great work on this RFC! I’d like to propose a utility function, such as reactive, that could create the appropriate tracked data structure based on the input type. This could simplify the process of creating tracked values and make the API more flexible.

For example, if you pass an object to the reactive function, it would return a TrackedObject. If you pass an array, it would return a TrackedArray. This would allow developers to more easily create tracked data structures without needing to manually specify the type each time.

import { reactive } from '@ember/reactive';

const trackedObj = reactive({ key: 'value' }); // returns TrackedObject
const trackedArr = reactive([1, 2, 3]); // returns TrackedArray

Additionally, I’d suggest reconsidering the naming of Tracked*. To better align with the language used in other frameworks and make the terms more intuitive, perhaps these could be renamed to Reactive*. This would emphasize the reactive nature of these structures and help make the API more consistent with other reactive libraries.

I second that. Compared to other frameworks, which use "one" function to do that.

• Svelte:
  • $state()
• Vue:
  • ref()
  • reactive()
• Angular:
  • signal()
• React:
  • useState() (I guess, according to their official website - I'm so not into react)

And then ember:

• tracked()
• trackedObject()
• trackedArray()
• trackedMap()
• trackedWeakMap()
• trackedSet()
• trackedWeakSet()

where one might say, that ember is really trying hard not wanting to be used... and I wouldn't disagree.

My preference would be with one reactive() here (similar to $state()) - but potentially the idea from vue where in unsupported cases a second function with a proxy might be the better option (I don't have deep glimmer VM knowledge to judge the edge cases).

[NullVoxPopuli]

implementing a reactive() (or tracked()) isn't technically a big deal -- it's more an issue of specification -- how to we define what it adapts?
Do we say:

• primitives
• objects
• arrays
• maps / sets
• weak maps / sets

and that's it?
and by it, I mean, the absolute minimal app someone can make would end up shipping all 7 reactive data structures.

IF we say only primitives, objects, and arrays, what is the justification for omitting maps/sets and their weak variants? less used?

What do we say about the future planned reactive data structures? (which are planned under separate imports as well)

What do folks think?

[gossi]

my hunch was to accept anything "native" to js. Then I looked what svelte is accepting, which comes close I'd say: https://github.com/sveltejs/svelte/blob/main/packages/svelte/src/ambient.d.ts - I think that's what I'd probably expect (more like gut feeling here), dunno if svelte has an RFC for that.

My questions:

• On an API surface level, does it compete with cell() in A new reactive primitive: cell #1071 ?
  ➡️ see ref() vs reactive() in vue
• Will it be deeply reactive?
  ➡️ see $state() in svelte

[NullVoxPopuli]

Thanks for providing those links! that was helpful 💪

On an API surface level, does it compete with

I think what you're proposing does, but the main thing that this RFC is concerned about is public (not quite low-level) API -- what you're proposing can totally wrap these built in tracked-built-ins as well as cell -- that could even (and this is my preference) be a separate subsequent RFC -- because there are more reactive utilities to implement beyond just the 6 that this RFC is concerned with.
What you want in a general "I don't care what I'm using, nor do I care for any nuance in perf / memory-pressure / updating, I just want it to be reactive" utility is probably even more stuff, and probably some way to describe how to more generically handle reactive wrapping as svelte as done. These would be implementation details of said subsequent RFC.

Will it be deeply reactive?

no, that is out of scope for this RFC.

---

For a deep-do-everything utility, a new RFC could be written that builds on all this that is maybe exported from

import { tracked } from '@ember/reactive/all'; 
//                       or something short like
//                      '@ember/x';
//                      if, during implementation of tracked-built-ins,
//                      we figure out how to more generally implement all these utilities
//                      without shipping so many bytes

[gossi]

I think what you're proposing does, but the main thing that this RFC is concerned about is public (not quite low-level) API

yes, I'm speaking exactly that. What will we make public and thus define the experience for people.

Currently we have this narrative: "Use JS as you are used to. Use @tracked to make your code reactive and ember will take care of the rest..." but we are about to continue the story with "... but you failed making your code reactive bcz from the 87 reactive exports we provide you, you have chosen the wrong one".

I see this parallel to react's shouldComponentUpdate() and to put ember devs in a similar situation to actively think about updates again.

Which is why I consider this question important:

Will it be deeply reactive?

no, that is out of scope for this RFC.

Let's put this together with an example and two approaches to deal with it.

Example:

const customer = reactive({
  givenName: 'Luke',
  familyName: 'Skywalker',
  address: {
    planet: 'Tatooine',
    location: 'At my uncles farm'
  }
});

function moveToHooth(customer) {
  customer.address.planet = 'Hooth';
  customer.address.location = 'Rebell base';
}

... and somewhere in the template we render the address.

This example would work in svelte, because the API was designed to work towards the expectations of consumers. It wouldn't in ember right now as deep reactivity is explicitely excluded, though this is unknown for consumers at first and breaks the narrative of "just use JS".

We find ourself somewhere between correctness (given implementation details) and consumer expectation, for which I have two options:

1. "Use JS"

In this case deep reactivity is a given and works with the idea of "performance optimization comes from measuring at first, then apply a treatment" and gives you two functions/parameters:

• reactive(data) - makes everything deeply reactive
• reactive(data, proxy) - provide a second param for fine tuning your reactive approach

this idea is a bit with the assumption that most reactive data is ok to be deeply reactive (this is very much subjective by me), but provides an escape hatch for those working on data-heavy apps

2. Separate deep and non-deep reactivity

Make it explicit in the API what you get:

• reactive(data) - makes everything reactive (non-deeply)
• deepReactive(data) - makes everything deeply reactive

The learning curve is a bit higher and comes with a couple rounds of frustrations and negative experiences at first.

Additional Comments:

• It might seem like I shifted a little to deep reactivity, but this is what collections are about. Whether this is a list or a dict.
• "Correctness" about the API is knowing the internals, which consumers do not need to know
• Keep the API surface as minimal as possible
• Design a good narrative

Happy to hop on a call to discuss.

[NullVoxPopuli]

I think you've convinced me to write an RFC for the everything-reactive (even deeply) util -- I am curious what others think tho. Our learning store would then become:

• use reactive(/* this is deep */) or @reactive (also deep), for when you have unknown-shaped or deep data, and use the fine-grained utils directly if you know ahead of time what optimization you want to make -- and the thing we'd maybe yak shave is the import path of this util -- unless .. "it's probably fine" to have on @ember/reactive, because we're only talking about 2KB (gzip) or so anyway

thanks for the detailed write up!

[gossi]

Based on the discussion from yesterday, I think it was Ed who dropped the idea for the API I'm playing around with. Here are my blurbs:

import { reactive } from 'ember/reactive';

// casual usage
const user = reactive({
  givenName: 'Luke',
  familyName: 'Skywalker',
  address: {
    planet: 'Tatooine',
    location: 'At my uncles farm'
  }
});

// advanced usage
const heavySet = reactive.set(new Set());

which comes down to a statement above: "measure your own system and improve upon that" for which explicit/narrowed "sub-functions" exist, that actually carry on this narrative through its syntax.

Going into deep nesting, the two options from above persist, but let me precise them.

1) Opt-out: "Just" Use JS

deeply reactive by default.

• reactive(data) - makes everything deeply reactive (casual mode)
• reactive(data, function) - provide a second param for fine tuning your reactive approach (advanced mode)

providing a function as a second parameter gives authors fine control over where they want to make improvements and more importantly how - so they can tailor this to their situation. I dunno how that function would look like and it is rather complex (I guess?), but we can yield supporting utils into that function for people to use, eg.

const data = { /* ... */ };

const optimized = reactive(data, (data, key, utils) => {
  utils.something();
});

ofc, this would work in "advanced mode" too:

• reactive.set(data)
• reactive.set(data, function)

2) Opt-in: Ember optimizes your data

Make it explicit in the API what you get:

• reactive(data) - makes everything reactive (non-deeply)
• reactive.date(data) - makes everything reactive (non-deeply)

for deep reactivity this can come in two forms:

a) two functions

• deepReactive(data) - makes everything deeply reactive
• deepReactive.date(data) - makes everything deeply reactive

b) with param

• reactive(data, true) - makes everything deeply reactive
• reactive.date(data, true) - makes everything deeply reactive

now, you will receive embers deep reactivity, no control over it though.

Thoughts

From these two I'd actually consider the opt-out version to have advantages over the opt-in version:

• It has a good narrative: "Just use JS"
• Optimization can happen according to your data
• Optimization function can become quite a challenge to design and to use - but if done right and people love to use it, then it is :chefs-kiss:
• Ember's deep reactivity (as in opt-in) might not be the best optimization for consumers
• Less error-prone, but more performance-prone
  • what's the assumption we see more often here: "casual" usage or data-heavy apps

Questions

• How does reactive() compete with other reactive exports?

  • Can they be reactive.something() - ok, now it sounds like I found the hammer and make every problem look like a nail, but its a question worth asking

• Is reactive.something() tree-shakeable?

• Would it work like so:

   function moveToHooth(user) {
     user.address.planet = 'Hooth';
     user.address.location = 'Rebell base';
   }
   
   function calculateDistanceTo(user, anotherPlanet) {
     const planet = user.adress.planet;
   }

  or is it more like a cell then:

   function moveToHooth(user) {
     user.set({ 
       ...user.current,
       adress: {
         ...user.current.address,
         planet: 'Hooth',
         location: 'Rebell base'
       }
     })
   }
   
   function calculateDistanceTo(user, anotherPlanet) {
     const planet = user.value.adress.planet;
   }

blurbing end.

[scottmessinger]

I'm very in favor of deep reactivity being the default (with an escape hatch). I can't think of many times where shallow reactivity would be assumed or wanted (outside of performance reasons).

I like the casual/advanced usage @gossi spec'd out:

import { reactive } from 'ember/reactive';

// casual usage
const user = reactive({});

// advanced usage
const heavySet = reactive.set(new Set());

I would think shallow tracking would be:

const shallow = reactive.shallow({...})
const shallowSet = reactive.shallow.set(new Set());

I like the idea of passing a function as the second argument for specifying the equality function or other tweaks.

[NullVoxPopuli]

Starting writing it up here:

• Propose new reactivity utility: reactive from @ember/reactive #1079

[NullVoxPopuli]

If anyone wants this:

• Propose new reactivity utility: reactive from @ember/reactive #1079

please comment over there -- there are a number of tradeoffs, problems that aren't worked out, and questions about the very feasibility of coherent deep tracking.

[MrChocolatine]

For my own knowledge, was it intentional to replace @ember/< ... > with ember/< ... > ?
I am asking just in case because I can also read:

(...) potentially by eventually reclaiming the 'ember' package (...)

[NullVoxPopuli]

Nay

Tho, i do have an rfc upcoming that reintroduces 'ember' so it's possible i got confused

[vlascik]

Maybe don't shuffle boxes pointlessly around for a while?

The insane amount of zero value busywork that would create, omg.

[NullVoxPopuli]

bruh wat

[NullVoxPopuli]

@MrChocolatine I went to go look at this to fix it, but in context, this is actually correct.

It's about the note above about the plan to reclaim the ember package.

which, for broader context is about reducing the amount of magic in our build system, and making this real packages.

all the @ember/* packages are fake, and it would be a lot to juggle all of them in your package.json -- which is why I plan to soon propose repurposing 'ember'

[scottmessinger]

@vlascik What are you even talking about? He made something you for you and you were unkind. Your entire tone is unproductive. Speak to the very real issues Ember has -- that's great! (Ember has many). You're being personally unkind in the process. Your anger and aggression are unproductive and unhelpful.

I have deep, deep, deep frustration about Ember. I agree with all the points you've raised (though not the tone). Bring them up kindly. Don't sugarcoat. Also, don't personally attack.

@NullVoxPopuli Can he be blocked? No one needs this.

[vlascik]

@scottmessinger If you'd read the discussion, you'd know that deprecation was not the issue, just an example. I didn't ask for nor need a babel plugin to undo that bad decision either, I already dealt with it long time ago. I used it just as an example of pointless deprecation that creates busywork and adds no value.

Plus points for the inovative deflection though.

Still doesn't answer the question of who's going to deal with the other 764 addons that will stop working in 6 - 9 months. Will everyone have to install that babel plugin to undo that deprecation just keep their apps working? That sounds like a good solution here? Kind of a thing that needs to be done again, but on a bigger scale? How many manhours burned is acceptable just to move a box 2 feet away and back?

Also. The context what you're missing is that I've had these same discussions for years now. And it always goes the same way:
"I have a problem."
... crickets
or
"It must be something you did, goodbye."
or, best case, it's a full on PR chatbot
"We appreciate your concern and work on a solution <3." level of non-answer. And I've heard them all already.

Then of course nothing happens, problems pile up, apps blow up, and in the end, 7 years down the line, I have to complain about the same things I did in 2018.

So as to the tone, this is the only tone that gets through. Same in this case.

Also, I don't think nvp is going to collapse due to some salt, he's perfectly capable of dishing it out as well. Most adults should be able to have uneasy conversations about unpleasant topics too.

Or, is ignoring the problems more productive? Will "blocking" everyone with an issue make Ember problems go away?

Who knew it was this easy, "block" away then.

[NullVoxPopuli]

if a project's chosen addons don't want to update, then like all other js ecosystems, folks decision tree is:

• move off the addon
• don't upgrade

These are valid options, and nothing actively stops your app from working -- non-working states can only happen with a change to your codebase.

The Angular (stuck on v9),

React (stuck pre strict mode),

and Vue (actually doing pretty healthy)

ecosystems all have this problem.

where ember-source is at

We know we messed up around this.
We are not blind to ember's faults.
We keep at it to try to do better.

We're about to do a series of another set of deprecations (getting rid of computed, classic classes, etc) -- because keeping that code around has done more harm than good.

Now, can we do another set of babel plugins to keep folks in the past? sure, maybe. But in order for the framework to move forward and recover from our mistakes, from the very problems brought up in this thread, we must get rid of our cruft.

<3

764

I didn't know it was possible to have this many addons.
My giant project at work only has 90 something, so it feels like 764 is all the addons, which is impressive in its own way!

[vlascik]

Want to see a fun graph? Here's one:

React is at 45 million downloads weekly, Angular is at 4 million, Vue is at 7.7 million,. Ember is at 190K.

Or, Ember is at 0.4% of React's downloads, 4% of Angular's, and 2% of Vue's downloads. Ember is now a rounding error.

And no, the reason for that is not that the code don't look pretty, it's that you consistently do not provide what your users need and ask you to do.

What I need from a frontend framework is 4 things - fast rendering, fast builds, a way to query/mutate complex data, and good dev tools. And for the love of god, mostly bug free experience. That's it. And everything on this list is still broken in one way or another, and has been for entirety of Ember's existence.

Yet instead of fixing it, you burn the whole ecosystem's efforts on shuffling boxes around.

The only thing keeping Ember afloat it's that the number of addons was enough to be somewhat productive in it. So, if instead of fixing the basics, the grand plan to save Ember is to blow up the entire addon ecosystem, that kills the last remaining piece of Ember's value and that will be pretty much it.

And if you keep digging this hole, you'll kill it for sure - it took ember-intl 9 months to fix a single line of code deprecation, and that's a probably a top 10 used addon by a premier ember consultancy or something. If you think everyone is going to spring to action whenever you come up with a great deprecation idea, you're delusional. It's been 6 years and most addons aren't even Octane yet.

Arguing by how many people are stuck in other frameworks is pointless - first of all, even probably the worst offender, the update checklist for Angular to update across 5 last majors is 2 pages long and can be done in a week/mostly automated. If people don't do it, it's because they don't care enough, not because they are terminally stuck. Also, it's one thing to find an alternative in 45 million download framework, and a 190K download framework. There are no alternatives in Ember. There are barely enough addons to do the basics, half decently at that. In fact, I now have to port Vue addons just to stay in the game.

I'm glad your very important billion dollar company will be able to swing rewriting 90 addons, but have you really told your managers about the cool million-ending sum on an invoice they are going to have to pay for that, just to end up in the same place they were before? They might have a different opinion on the subject.

I sure can't rewrite all the addons in my apps. Is being a billion dollar company a barrier to entry to Ember now? Because if even companies like LinkedIn and Square with maybe $100 billion value combined can't and don't want to be paying for it, you sure the other companies will be able to? What's the plan here, who's going to do all the updates? Or, do you plan on EmberConf 2030 being just 5 guys around a camp fire talking about how beautiful their code is?

Anyway. You keep digging that hole, it's going to go great. As it has so far.

[kategengler]

I think it is important to remember that nearly all work on Ember is a volunteer effort. We're here because we believe in Ember and we think that it is worthy of our time.

We are not at all unaware of the problems you have mentioned or frustrations. They are frequently discussed in our many open weekly meetings and most of our efforts are to address them.

The particular deprecation that has caused you trouble: import { inject } from '@ember/service' was introduced in Ember 6.3, released in March of this year, so your claim that addons took over a year to fix the deprecation is confusing. The replacement, import { service } from '@ember/service' was added in version 4.1, way back in 2021, but without a deprecation users saw no signal that they should move to the new thing. Deprecations are introduced at least 3 minors before their 'until' version to give packages time to update -- in this case the deprecation won't error until 7.0 which is over 36 weeks and two LTS versions in the future and until then it is a harmless warning.

Since the 3.x error we have learned to be more cautious with deprecations and we do evaluate RFCs for whether they will create "busy" work. For example, we held off deprecating features of EmberObject or @ember/component until we were ready to deprecate the entire feature to avoid having users make updates when they were better off moving away from the feature entirely. We also consider addon usage and how addons can support versions before and after a deprecated feature.

I'm going to lock this conversation now as it is going in a circle and was initiated by something that is not even proposed yet.