Compare with redux performance

[avkonst]

https://codesandbox.io/s/large-redux-form-perf-example-inmlv

[dai-shi]

Or, would you consider trying https://github.com/krausest/js-framework-benchmark?
I was interested in doing it but haven't had time. (I mean my focus is not benchmark lately.)

[avkonst]

This performance demo equivalent was written by Mark, redux maintainer. I am going to measure how fast it is and compare with the same demo for hookstate. I could create the performance benchmark implementation for hookstate by cloning existing one for react-mobx if there is one. I can not find one. Do you know where react mobx implementation of the benchmark source code is located?

…

On Thu, 21 Nov 2019, 17:47 Daishi Kato, ***@***.***> wrote: Or, would you consider trying https://github.com/krausest/js-framework-benchmark? I was interested in doing it but haven't had time. (I mean my focus is not benchmark lately.) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#14?email_source=notifications&email_token=AA6JSVLEVRCXX6QBGJLLN4LQUYHHVA5CNFSM4JP4O7NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEY6MII#issuecomment-556918305>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA6JSVLDWYPWNI365HHLB4DQUYHHVANCNFSM4JP4O7NA> .

[dai-shi]

https://github.com/krausest/js-framework-benchmark/tree/master/frameworks/keyed/react-mobX
This is only the one I know.

[avkonst]

Preliminary results. Partial update test shows strange results. hookstate rerenders only every 10th row, but 2 others rerender the whole table and still faster. Not sure why it is not that fast.

[avkonst]

select row test is OK, but still strange. Hookstate rerenders only 2 rows - one unselected and one selected. Other 2 frameworks rerender the whole table. Hookstate should be significantly faster here as it is faster in swap rows. Is it not representative scenario?

[avkonst]

Some results.

FYI @dai-shi

Hookstate is many times faster than competitors on per field update performance (eg. set a field value of large form). Eg. 20x times faster for 1000 fields list. It is about the same when the whole data set is updated. Note: hookstate benchmark is implemented without React.memo. Partial update (10% of the data set is updated) is updated as the whole, but rerenders every field individually, which requires 100 dom changes by React (other frameworks rerender the whole data set at once).

About the same load performance.

A bit better on RAM.

[avkonst]

FYI @ppwfx @praisethemoon

[avkonst]

@praisethemoon, look at the swap rows benchmark - this is similar to your app use case where an individual element in a large dataset is updated frequently.

[dai-shi]

swap rows

Looks very nice. Is it because each row subscribes to the store?

[avkonst]

Any individual row update will be many times faster. Swap is an example of it. It uses scoped state technique (see docs about scoped state). We can tell that each row subscribes to the store, but it is not quite like this. On update there is NO loop through all rows to find out which row to update. Instead affected rows are effectively identified by indexes within hookstate links. These indexes are automatically built when you use the state and place scoped state hooks. This will scale really well over thousands of rows and may be even more.

[dai-shi]

I think I understand the basics if it hasn't changed drastically since before. My question is if it's 20x faster than react-redux, is it faster than vanillajs-keyed? That sounds unlikely.
https://krausest.github.io/js-framework-benchmark/current.html

[avkonst]

React+Hookstate is a bit faster than Preact+Hookstate.

But Preact + Hookstate loads faster:

[avkonst]

@dai-shi this is how it compares against vanilla react and vanillajs. It is faster than vanillajs a bit!!!

[dai-shi]

It looks way too fast to me. But, I'd admit I didn't read either of benchmark code, so I might be wrong.

[avkonst]

Benchmark source code for Hookstate:

import React from 'react';
import ReactDOM from 'react-dom';
import { useStateLink, createStateLink, None, Downgraded } from '@hookstate/core';

function random(max) { return Math.round(Math.random() * 1000) % max; }

const A = ["pretty", "large", "big", "small", "tall", "short", "long", "handsome", "plain", "quaint", "clean",
  "elegant", "easy", "angry", "crazy", "helpful", "mushy", "odd", "unsightly", "adorable", "important", "inexpensive",
  "cheap", "expensive", "fancy"];
const C = ["red", "yellow", "blue", "green", "pink", "brown", "purple", "brown", "white", "black", "orange"];
const N = ["table", "chair", "house", "bbq", "desk", "car", "pony", "cookie", "sandwich", "burger", "pizza", "mouse",
  "keyboard"];

let nextId = 1;

function buildData(count) {
  const data = {};
  for (let i = 0; i < count; i++) {
    data[nextId] = {
      id: nextId,
      label: `${A[random(A.length)]} ${C[random(C.length)]} ${N[random(N.length)]}`,
    };
    nextId += 1;
  }
  return data;
}

const globalState = createStateLink({});

const GlyphIcon = <span className="glyphicon glyphicon-remove" aria-hidden="true"></span>;

const Row = ({ itemState, selectedRef }) => {
  const state = useStateLink(itemState).with(Downgraded);
  const item = state.value;
  const select = () => {
    if (selectedRef.current && selectedRef.current.value) {
      selectedRef.current.set(p => {
        p.selected = false;
        return p;
      })
    }
    selectedRef.current = state
    
    state.set(p => {
      p.selected = true;
      return p;
    })
  };
  const remove = () => state.set(None);

  return (<tr className={item.selected ? "danger" : ""}>
    <td className="col-md-1">{item.id}</td>
    <td className="col-md-4"><a onClick={select}>{item.label}</a></td>
    <td className="col-md-1"><a onClick={remove}>{GlyphIcon}</a></td>
    <td className="col-md-6"></td>
  </tr>);
}

const Button = ({ id, cb, title }) => (
  <div className="col-sm-6 smallpad">
    <button type="button" className="btn btn-primary btn-block" id={id} onClick={cb}>{title}</button>
  </div>
);

const Jumbotron = () => {
  const dataState = globalState
  
  return (<div className="jumbotron">
    <div className="row">
      <div className="col-md-6">
        <h1>React Hookstate keyed</h1>
      </div>
      <div className="col-md-6">
        <div className="row">
          <Button id="run" title="Create 1,000 rows" cb={() => {
            dataState.set(buildData(1000))
          }} />
          <Button id="runlots" title="Create 10,000 rows" cb={() => {
            dataState.set(buildData(10000))
          }} />
          <Button id="add" title="Append 1,000 rows" cb={() => {
            dataState.merge(buildData(1000))
          }} />
          <Button id="update" title="Update every 10th row" cb={() => {
            dataState.merge(p => {
              const mergee = {}
              const keys = Object.keys(p);
              for (let i = 0; i < keys.length; i += 10) {
                const itemId = keys[i];
                const itemState = p[itemId];
                itemState.label = itemState.label + " !!!";
                mergee[itemId] = itemState
              }
              return mergee;
            })
          }} />
          <Button id="clear" title="Clear" cb={() => {
            dataState.set({})
          }} />
          <Button id="swaprows" title="Swap Rows" cb={() => {
            dataState.merge(p => {
              const mergee = {}
              const keys = Object.keys(p);
              if (keys.length > 2) {
                mergee[keys[1]] = p[keys[keys.length - 2]]
                mergee[keys[keys.length - 2]] = p[keys[1]]
              }
              return mergee;
            })
          }} />
        </div>
      </div>
    </div>
  </div>)
}

const Rows = () => {
  const dataState = useStateLink(globalState);
  const selectedRef = React.useRef();

  return (<table className="table table-hover table-striped test-data"><tbody>
      {dataState.keys.map(itemKey => {
          const itemState = dataState.nested[itemKey];
          return <Row key={itemKey} itemState={itemState} selectedRef={selectedRef} />
      })}
  </tbody></table>)
}

const Main = () => {
  return (<div className="container">
    <Jumbotron />
    <Rows />
    <span className="preloadicon glyphicon glyphicon-remove" aria-hidden="true"></span>
  </div>);
}

ReactDOM.render(<Main />, document.getElementById('main'));

[avkonst]

Nothing strange here. I believe vanillajs forces the browser to rerender the entire list. Hookstate causes rerender for only 2 rows out of 1000.

[dai-shi]

Yeah, so it would be possible to apply your approach to vanillajs, maybe?
I'm not sure but the benchmark is meant to test performance of rendering the entire list. Don't know...

[avkonst]

In this case it is only for frameworks without smart state management.

…

On Wed, 12 Feb 2020, 18:52 Daishi Kato, ***@***.***> wrote: Yeah, so it would be possible to apply your approach to vanillajs, maybe? I'm not sure but the benchmark is meant to test performance of rendering the entire list. Don't know... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#14?email_source=notifications&email_token=AA6JSVJJGETKXOUO6BTF5XTRCOFBDA5CNFSM4JP4O7NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELPQTRI#issuecomment-585042373>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA6JSVLXTSKQQ3QD2LAR6FTRCOFBDANCNFSM4JP4O7NA> .

[avkonst]

@dai-shi I need your help and knowledge. I have noticed that many state tracking libraries use the following hook:

const useIsomorphicLayoutEffect = typeof window !== 'undefined' ? React.useLayoutEffect : React.useEffect;

for subscribing to store updates.

I noticed that useEffect is a bit more responsive than useLayoutEffect

I have read the docs about layout effect and conclude that the subscription can be just well done in the useEffect hook. My question is: what would be the side effect, if I remove useIsomorphicLayoutEffect and useEffect in Hookstate ?

[avkonst]

@dai-shi BTW, I have fixed the benchmark for swap rows to please isKeyed check of the benchmark. As a result, Hookstate is only 10x times faster than Redux, not 20x as before. Vanilla JS is about 50% faster than Hookstate, in both variants keyed and non-keyed. But the real code will not put additional keys to table rows, so rows can be updated in place (non-keyed) instead of replaced (keyed). New results are published here krausest/js-framework-benchmark#693

[dai-shi]

As a result, Hookstate is only 10x times faster than Redux, not 20x as before.

Sounds reasonable! I would like to take a deeper look at it later.

useLayoutEffect

The rule of thumb is a) useEffect for subscription as long as you check the possible update right after subscribing, and b) useLayoutEffect to update refs.
There's a technique to avoid b). Check out use-subscription package if you are interested. It doesn't mean this technique is always applicable (it requires immutability).

useIsomorphicLayoutEffect is just a hack to avoid React warning message. You don't need it at all, if you are not supporting Server Side Rendering, just useLayoutEffect.
It's very tricky, and the implementation in react-redux is changed from the original one.
I recently came up with my own solution: https://twitter.com/dai_shi/status/1227237850707447809

[ryansolid]

If you guys don't mind my interjection I might be able to help in terms of the benchmark. I haven't looked exactly at what you are doing but I understand exactly how the benchmark works/should work. I have written half a dozen implementations including my library Solid, one of the Vanilla JS implementations, and the React Hooks implementation. Those implementations are correct.

Nothing strange here. I believe vanillajs forces the browser to rerender the entire list.
Hookstate causes rerender for only 2 rows out of 1000.

That is incorrect VanillaJS does the minimal amount of work. For keyed implementations, they move no more than they have to. In the case of Swap only those 2 TRs swap. All the vanilla code does is grab the TR and insert before the node after the other node and vice versa. 2 DOM insertBefore calls. There is very little to no code to optimize here. No diffing or reconciliation, no state management.

It is likely your keyed implementation is still non-keyed if you are getting the numbers you posted.
Like bypassing Reacts reconciler like keeping the key in the indexed position somehow. The reason React is so terrible at Swap rows is that when it uses a simple bubble algorithm to sort. But if the VDOM output of the library uses keys and moves rows React will run this naive algorithm no matter how efficient your code is. The only way to avoid this is to update the data associated with the key and not move the row at which point it isn't keyed.

============
EDIT: I should mention the reason there are less non-keyed implementations is it considered an anti-pattern by most library authors, a novel hack for benchmarks. So that's why keyed is used for most comparisons and why there are less React variants there.

[avkonst]

I understood the difference between keyed and non-keyed when checked diff between keyed/react and non-keyed/react. The keyed/react-hookstate benchmark satisfies the criteria now - 2 table rows are deleted and inserted by React, but only these 2 rows, thanks to Hookstate's state usage tracking. keyed/react-hooks rerenders the whole list, no matter how many rows are affected by the state change.

I still wanted to preserve the initial fast implementation of the react-hookstate, which did not pass isKeyed check, and moved it to non-keyed in the referenced PR.

[avkonst]

You don't need it at all, if you are not supporting Server Side Rendering, just useLayoutEffect.

Well, I actually wanted to use useEffect everywhere and drop useLayoutEffect. It seems it makes the benchmark faster. wondered how "safe" it would be for Hookstate, and how much it is side-effect free? The reason I used it initially, is because I ported state tracking from react-tracked, which used useLayoutEffect at the time.

[avkonst]

For keyed implementations, they move no more than they have to. In the case of Swap only those 2 TRs swap. All the vanilla code does is grab the TR and insert before the node after the other node and vice versa. 2 DOM insertBefore calls.

keyed/react-hookstate does effectively the following: 'delete TR at 1st and 998th position' and 'insert new TR at 1st and 998th position'.

non-keyed/react-hookstate does the following: 'set TD text for the label and id and TR class for 'selected' for the 1st row copying state from the 998th' and vice versa.

there is no surprise react-hookstate is fast on per field update - I have got the benchmark with a table with 10000 table cells, where it updates a cell per every single millisecond: https://hookstate.netlify.com/performance-demo-large-table

[avkonst]

@ryansolid and do not worry, vanillajs is faster after I fixed keyed implementation :)

[avkonst]

Here how it looks now. I guess this makes more sense:

see comment here as well: krausest/js-framework-benchmark#693 (comment)

[ryansolid]

@avkonst Ok in the same post you mentioned fixing it the numbers(krausest/js-framework-benchmark#693 (comment)) still looked strange. This last one seems plausible. Although you've definitely piqued my interest. I wasn't aware that it was possible with a state management library to bypass Reacts reconciler in a keyed operation. Interesting stuff.

[avkonst]

@ryansolid check this video of a real app powered by hookstate: https://youtu.be/GnS-OUwGyaw
somewhere in the middle of the video (from 2:20 for example). Each piano key state is in the single state array. Each individual key is rerendered independently from the rest of the keyboard.

[avkonst]

Need to redo code samples with Hookstate 2.0 which is simpler and even faster!

[avkonst]

Performance is also documented here: https://hookstate.js.org/docs/performance-intro

[silouanwright]

@markerikson Any thoughts here on the performance metrics for redux? Just interested if we should concentrate on getting more performance out of redux, or if this is a better tool in some circumstances.

[markerikson]

Sorry, not sure what the question or the context is here.

[ryansolid]

@markerikson and @reywright and whoever else.

I wouldn't be too concerned about this. The performance shown here in HookState is cool but it's not an equivalent comparison. See krausest/js-framework-benchmark#693. Basically it doesn't actually do keyed row swapping in the demo. So it more or less bypasses the tests so don't use that JS Framework comparison to base any decision.

The only delta you could be concerned with is the gap between React implementation and Redux and if that is acceptable (latest results: https://krausest.github.io/js-framework-benchmark/current.html). React Redux Hooks and React Hooks implementations are more or less equivalent performance so I don't think there is a general concern. Peoples mileage may vary depending on how they create their state shape etc but that deserves specific questions.

I am yet to see any state implementation make any meaningful positive difference (and usually a slightly negative) over React's core performance in focused benchmarks. That's because we write the most optimal code there. So the only value of such benchmarks is to show when a state library is substantially slow. Which in this case neither of these libraries are, so this should have absolutely zero impact on your decision.

[avkonst]

Hey @ryansolid and @reywright
Hookstate makes deep nested state updates a magnitude faster than React useState, because React updates starting from the component there useState is used and Hookstate updates only deeply nested components.

I have not progressed this issue because the benchmark has got the criteria for rows swapping, and Hookstate does meet the criteria of the benchmark tool, because it updates in place not giving React any chance to slowdown the process, hence it is way faster than vanila useState or Redux. Once you give React the control what to update, the benchmark essentially tests React + plus any overhead of the state management library. No point in doing this theoretical exercise as it shows nothing.

I might comeback to this issue sometime later and submit the compliant implementation to the benchmark, but I sort of lost the interest. I do not need to prove anybody that Hookstate makes React driven updates faster. I know it is faster. So, I focus on other more important activities. But I keep it open in case I or somebody else decides to come back to the issue.

[ryansolid]

No point in doing this theoretical exercise as it shows nothing.

Agreed, if the library is sufficiently fast enough, which most are.

Hookstate makes deep nested state updates a magnitude faster than React useState, because React updates starting from the component there useState is used and Hookstate updates only deeply nested components.

Which is the challenge with benchmarks as no performance benchmark ever is going to use Context API or not nest the state change if possible. But real people do. Benchmarks will write things in a more awkward way to get absolute performance out of React which when given the same behavior criteria will always be the fastest. After all, any library written on top React uses React in its internals and someone could always write the same code without the library if it is allowed and beneficial. However not being a library can be tailored specifically to the example allowing for shortcut optimizations a general library could never account for. In the same way, no framework can be faster than Vanilla JS no state library can ultimately be faster than the underlying framework.

But that's not why we use these. As no one wants to write that painstakingly optimized code themselves. And through performance exploration, we can learn how to better optimize our underlying React code. It's win-win.

EDIT: And I don't mean anything negative by this. Just commenting about the value of the benchmark and the perspective of Vanilla React vs any state library. I think HookState's approach is generally great. Granular updates have huge benefits and are also at the core of my reactive work. I made similar experiments which lead me to React always being the bottleneck hence moving on, but I love to see how close state libraries can get to the metal to offer a much nicer experience without introducing the overhead that is added in other solutions or simply from the way most people structure their React code.

[silouanwright]

I do not need to prove anybody that Hookstate makes React driven updates faster. I know it is faster.

Didn't mean to offend :) You don't have to do or prove anything. I do think benchmarks (and accurate ones) would help show people who don't know that it is faster for their use case. I just wanted to see if the redux benchmarks were appropriate.

https://praisethemoon.org/hookstate-how-one-small-react-library-saved-moonpiano/

This article I think is great but the only thing that makes it ambiguous is that the redux maintainer in the comments said that the previous redux implementation didn't seem entirely optimized.

The purpose of being 100% sure the redux implementation in benchmarks is optimized, and making sure these success stories where redux is slow, are optimized, is that it's a clear smoking gun for uninitiated developers. As it stands.... most developers would need to give this a whirl themselves, and even if it's faster than their redux implementations, there's no guarantee that their redux implementations were optimal as far as the maintainer of that library is concerned. So it's just a clump of ambiguity.

Cool looking library though, looking forward to giving it a whirl.

[avkonst]

@reywright you might be interested in this: https://hookstate.js.org/docs/scoped-state#it-is-unique and the story above it. It explains where Redux stops scaling and where Hookstate really shines.