Derived arrays in MobX

[nardi]

What is the best way to define a derived array in MobX? By that I mean, if I have an observable array
var a = observable(['foo', 'bar']);
and I then perform an operation on it such as

function foo(a) {
  return a.filter(s => s.indexOf('foo') !== -1);
}
var b = foo(a);

how do I do this in a way so that:

1. b is an ObservableArray itself
2. adding the string 'foobar' to a also updates b to ['foo', 'foobar']

It seems as if I either have to construct a new observable array with the contents of a.filter(...) to have 1 but not 2, or I have to use a computed observable to have 2, but I can then only get the contents out as a regular array by doing b.get() and so I don't have 1.

I would maybe expect calling something like .filter(...) on an observable array would return a read-only observable array whose contents contain everything in the original array (subject to changes) that passes the filter function. So adding or deleting an element in the original array is propagated to the filtered array. Is this something already possible or is it worth implementing using array.observe?

[mweststrate]

Hi @nardi,

Currently approach 2 is the idiomatic way: var filteredArray = computed(() => a.filter(...)). As noticed this does indeed not result in a new observable array as value from the computation. But the computation itself is cached and will yield the same result as long as the original array doesn't change.

I have planned to add special filter, map, slice etc operations that returns observable arrays as that would potentially result in huge performance benefits, but so far nobody really needed it yet as the computed arrays are usually already fast enough. The reason for that is probably that most people use observable arrays to power their react components, and React would not benefit directly from smartly updating observable arrays in all cases, as it will always map the complete list to re-create child components (see for details this SO question).

That being said, I did some successful experimenting in the past with smart map and filter functions that intelligently update the result array only for the changed items and I plan to add this feature to MobX sooner or later. If you need it, sooner ;-).

A bit elaborate so I hope this answers your question, otherwise just let me now.

[capaj]

@mweststrate I would love to see these derived arrays in mobx. It would help with awfully big arrays, which are cumbersome to reiterate on each small change.

[mweststrate]

Give me a few weeks, but I'll definitely plan on adding this. Which
operations would you need (first)? map, filter, slice? Maybe sort?

Op wo 30 mrt. 2016 om 12:34 schreef Jiri Spac [email protected]:

@mweststrate https://github.com/mweststrate I would love to see these
derived arrays in mobx. It would help with awfully big arrays, which are
cumbersome to reiterate on each small change.

—
You are receiving this because you were mentioned.

Reply to this email directly or view it on GitHub
#166 (comment)

[capaj]

@mweststrate thanks, order of importance for me:
1.filter
2.sort
3. slice
4. map

[nardi]

For me it would be filter, map, sort. I don't really see slice as important, nor am I sure what the semantics would be - I my case I wouldn't care about the actual numerical index of elements, as long as they are stored in unmodified order (except for sort) and enumerable.

[danieldunderfelt]

1. Filter
2. Sort (and a new sortBy à la lodash would be EXCELLENT)
3. Map

[AriaFallah]

I have planned to add special filter, map, slice etc operations that returns observable arrays as that would potentially result in huge performance benefits.

Just wondering how would it improve perf? Is it similar to view vs materialized view in SQL?

[mweststrate]

@AriaFallah yeah it is very similar to that indeed. Instead of always remapping the array, the splices can be observed and applied (after mapping) to the target array. Etc. I made something similar in the past and it was quite doable and very efficient. Only nasty one is sort, but still doable I think with binary search / insertion.

[benjamingr]

If you base MobX observables over xstream or even rx all these problems go away, just saying.

[AriaFallah]

@benjamingr

That sounds interesting. Could you elaborate?

[mweststrate]

@benjamingr no it doesn't relate, streams map events in time, this is about mapping values in space.

Simply put:

const todos = ["fetch coffee", "fetch tea"]
const upperCaseTodos =  todos.map(s => s.toUpperCase())

MobX already has the means to already reflect the changes in todos automatically in upperCaseTodos. This is about optimizing that changing one of the specific todo's results in only that value being remapped, and not all of them. createTransformer can already do that, this story is about generalizing that concept for all array operations.

[akagomez]

That being said, I did some successful experimenting in the past with smart map and filter functions that intelligently update the result array only for the changed items and I plan to add this feature to MobX sooner or later. If you need it, sooner ;-).

@mweststrate I've done some work like this using CanJS's computes and red-black binary trees in the canjs/can-derive project. Specifically on the .filter() transformation.

Unfortunately, I didn't achieve the performance gains I was hoping for: Be proactive, not reactive – Faster DOM updates via change propagation

I'm stoked to learn how your approach compares.

[aravantv]

I think my issue is similar: I would like that a computed function returning an array can be observed like observable arrays, i.e., that I can observe if some elements are added/removed from the computed array.

[aravantv]

Mmmh, just saw that the observe method of ObservableArrays actually just conform to the (former) ES7 proposal but does not allow to observe what was added/removed, so that would actually be my first wish...

[mweststrate]

@aravantv nope that knowledge is already available, additions and removals are just expressed in the form of 'splices'.

You could achieve roughly what you want by doing something like:

var source = observable([])
var derived = observable([])

source.observe(change => {
   // inspect the splice / update and update the relevant parts of the derived array
})

Depending on your computation that might either be trivial or a very complicated thing to do :)

[ckwong90]

I'm incredibly excited for these new smart map, filter, slice etc. array functions. What's the status those features?

Seems like it would make changing and optimizing big data sets without doing much work. Also will be quite powerful optimizing createTransformation even further

[mweststrate]

I'm still planning on this, but not in the near future (next month) as there don't seem to be much need for these kind of optimizations so far. If in dire need, a case specific solution using splices will often not be that hard (a generic solution is harder, as there need to be clear semantics on the influence of transactions, arbitrary observables used in mapping functions etc, but I already build something similar in a predecessor of MobX)

[mavericken]

I created a gist for a linked filtered observable array function:
https://gist.github.com/mavericken/f0eb2cc5699c3a406aabe7582d87fa54
I haven't tested it sufficiently yet, but I wanted to add it to this case before I forgot about it.

[francesco-carrella]

Hello, any news about this topic?
I'm implementing a 'model' like structure based on MobX and I cannot find a way to easily filter and sort (in chain) an observable array.

[dzhng]

Just going to add my 2 cents to this as well, array.filter would be awesome if optimized, @computed doesn't work too well for us when we start to deal with thousands of items.

[cellvia]

is there a best practice workaround for this while we continue to wait and wish?

[evelant]

I am also curious about this. Just getting started with mobx and I'm not sure of the best way to get observabledata -> filtered observabledata -> my react component.

[laurensdijkstra]

@AndrewMorsillo it is ok to use the filtered ObservableArray, which is just a normal array, as props input to a React component; it will use the completely new array reference that is returned when the ObservableArray updates, to rerender the view (which does diffing on the virtual DOM, so if your array doesn't contain thousands of entries it should be OK)

[evelant]

@laurensdijkstra Yes, but doesn't that mean that the component will re-render if any property of any item in the parent array is updated? Maybe I'm not fully understanding the mobx observation model yet, but it would be preferable to only have a component update if one of the referenced properties of the items in the filtered array is updated.

[kalmi]

@AndrewMorsillo I am not laurensdijkstra, but I map the (problematic) array to a list of observer components during render, and these observer components only get a reference to a single item. This way these observer components do not rerender when not needed as their props didn't change.

Make sure to pick keys based on the item, and not on the item's index.

Still not ideal, but combined with virtualization, it seems to be good enough for now for me.

[mweststrate]

@laurensdijkstra / @kalmi, when using mobx in the standard way, mobx already will totally track changes to properties, assuming the object is observable and the child component is ready. For such scenarios the feature
described in this issue is not needed

[Jurcik84]

I have got similar problem when I was trying map array to mutate some properties inside callback like example show :

it looks like the @computed is not listening when you mutate array in service callback
But this helper me :)
I store the array local into storage variable and then simply swap.
I will make re-render !!! but as far deadline is closet it helped me.

@action
cm_action_fetch_all_tableRows = type => {
const ob_loadRows = {
type: type,
filter: {
type: type
}
};

 const storage = this.arr_store_forLoadedTableEntities;
 this.arr_store_forLoadedTableEntities = [];

 http.service_fetch_filteredEntities(ob_loadRows, response => {
     runInAction(() => {
         this.arr_store_forLoadedTableEntities = storage.map(
             (item, itemIndex) => ({
                 ...item,
                 results: item.type === type ? response.results : item.results
             })
         );

         console.log(
             "arr_store_forLoadedTableEntitiesTest",
             toJS(this.arr_store_forLoadedTableEntities)
         );
     });
 });

};

[janat08]

Couldn't you use smart mobx function to map jsx children from the computed which would then rip the benefits of granular reactivity?

[pelotom]

I'd like to be able to create synthetic "view models" which are computed by drawing on information from different parts of a normalized model. For a very simplified example,

export class ListModel {
  @observable items = [];
  @observable lowerBound = 0;

  @computed
  get viewItems() {
    return this.items.map(item => observable.object({
      get id() { return item.id },
      get faded() { return item.id < this.lowerBound },
    }));
  }
}

export class ItemModel {
  constructor(id) {
    this.id = id;
  }
}

I would like to render an array of react components based on viewItems, but if I do so they will all rerender whenever the items array is modified. The problem is that viewItems isn't observable, but computed. This is necessary because of the faded property, which is synthesized from information at the list level as well as the item level.

The only workarounds I know of to this problem currently are

• Pass both the ListModel and the ItemModel to each item component instance, and let it be in charge of synthesizing the faded property internally. This works, but it requires my components to become smarter, which I'd prefer to avoid, and makes it so I can't write tests against just a pure view model (only against the model, or the component).
• Give the ItemModel a reference to its parent, or by some other means inject access to the information it needs to compute the faded property on ItemModel itself. This is messy, and doesn't account for more complicated scenarios where there isn't a 1-to-1 correspondence between the models and view models (think for example of a UI that coalesces groups of items into a single unit, perhaps because they overlap spatially or in time).

Having map, filter, etc. produce projections which efficiently recompute only as necessary would address this problem neatly I think.

Here's a live example.

[jraoult]

@pelotom I went through the same journey recently. Because I didn't want to introduce any sort of circular dependencies in my stores, I tried a couple of things. I used createTransformer, but I ran out of time and I decided to not care until I hit the wall in terms of performances.

Have you measured the actual impact of re rendering and see any benefit in using a transformer? As far as I can understand, the collection should still need to be re-render but not necessarily each item since they are "cached".

[pelotom]

@jraoult thanks for the link, I wasn’t aware of createTransformer but it looks promising, will give it a look!

[pelotom]

Ok, using createTransformer I think I've solved my problem. I now have a nice clean separation:

model (normalized) --> view model (denormalized) --> view (pure presentational components)

And only absolutely necessary rerenders are being performed 🎉 See the updated example at https://codesandbox.io/s/x357o6pmjz if interested.

Thanks again for the tip, @jraoult!

[sjmeverett]

@pelotom this looks like a nice solution, thanks for sharing. One thing I don't understand though - why does get items on line 10 of viewModel.js not need to be marked computed?

[mweststrate]

@stewartml computed is the default decorator for getters when using observable(plain object)

[cellvia]

@pelotom i was interested in what you accomplished w/ create transformer and dug into your codepen, but something wasnt making sense to me and i couldnt put my finger on it. well i adjusted the code a bit, removed the viewmodel layer completely and removed createtransformer and im getting the hoped for result all the same, only rerenders individual components as needed:

https://codesandbox.io/s/jj3rn4xzq9

im not doing this just to nitpick, im genuinely trying to understand how to work with derived arrays, and possible performance benefits. in this particular case im seeing a level of abstraction that i often use (creating an intermediary viewmodel) but im not seeing how this is needed or illustrating any benefits i this case... or am i missing something here?

[jumika]

@cellvia Play around with these two (based on yours, and pelotom's codepens) and watch the console output.
https://codesandbox.io/s/silly-booth-ft1fr
https://codesandbox.io/s/goofy-jones-j7h2w

Search for calcluations in view.js of yours and viewModel.js in pelotom's codepen. I've added this calculations variable witch increments and logs to console every time the derivations required by the item are calculated. You can see that pelotom's implementation runs the calculations much less times.

[cellvia]

ok @jumika less calculations of the computed, got it. i was only looking at number of re-renders which i imagine as the "heavier" thing, but perhaps not always.

[dslmeinte]

Sorry to resurrect this thread, but using createTransformer i.c.w. wrapping everything in observable reeks like an Observable monad trying through break through ;)
I could see the Observable API gaining fmap and subscribe functions which do exactly that.

[danielkcz]

Whoa, thread from 2016 and it's still alive :) I will close this for now and unless there some useful conclusion or solution, it will get locked eventually. It would be probably best to start a new discussion if there is such a need.

[PEZO19]

+1 for this one as needed. :) At least a handful, simple utils for this reason:

smart map and filter functions that intelligently update the result array only for the changed items

For offline first apps this can be quite important I think.