Discussion: declarative side effects and data fetching approaches

[markerikson]

Currently, Redux Toolkit adds thunks as the default supported approach for any kind of async / side effect behavior, largely because it's the simplest possible approach that works, and because it's also the most widely used Redux async middleware.

That's a large part of why I'm considering adding some kind of a createAsyncThunk API to abstract common data fetching and action dispatching logic. It matches what we already have built in to RTK, and what many Redux users are already doing.

@davidkpiano has strongly argued that side effects should be declarative, instead, possibly along the lines of the redux-loop store enhancer.

I'm open to discussing possible APIs and approaches here. My initial point of concern is that this is a very different kind of API than most Redux users are used to, and it would add an extra level of teaching and documentation to explain.

Discuss!

[davidkpiano]

My initial point of concern is that this is a very different kind of API than most Redux users are used to

If you had asked developers what they wanted, they would have said they want to easily mutate state and call side-effects whenever and wherever they want.

“If I had asked people what they wanted, they would have said faster horses.”

EDIT: I'm looking over this and will provide my thoughts soon. Here's another source of inspiration: https://redux-saga.js.org/docs/basics/DeclarativeEffects.html

[oleksii-demedetskyi]

I have tried Declarative Side-Effects many times and always fail to scale them. Whole system is become too fragile to accept changes.

I have found that Data-Driven Side-Effects works instead. For the short:

1. Reducers update the state based on actions.
2. Side effect interpreter perform side effects based on the state.
3. That's it.

In some sense UI is just another side effect.

[markerikson]

@AlexeyDemedetskiy : I think that's literally what David means by "declarative" here:

// reducer:
case 'FETCH':
  return {
    ...state,

    // finite state
    status: 'loading',

    // actions (effects) to execute
    actions: [
      { type: 'fetchUser', id: 42 }
    ]
  }

// UI:
const { actions } = state;

useEffect(() => {
  actions.forEach(action => {
    if (action.type === 'fetchUser') {
      fetch(`/api/user/${action.id}`)
        .then(res => res.json())
        .then(data => {
           dispatch({ type: 'RESOLVE', user: data });
        })
    }
    // ... etc. for other action implementations
  });
}, [actions]);

[davidkpiano]

@AlexeyDemedetskiy This is insufficient. There are use-cases when a side-effect needs to be executed without the state changing.

[oleksii-demedetskyi]

@davidkpiano I never met these use-cases, so your experiences can be different.
@markerikson I have misread then. My bad, sorry. However, usually I just have separated reducers for storing required actions per domain. Usually these domains are - db access, network, gps, routing.

[phryneas]

@davidkpiano I'm a bit tired so I've just skimmed this (will give it a proper reading tomorrow), but I'm already seeing some problems with the approach you are suggesting, maybe you can already correct me on them if I read something wrong? :)

• This seems like it only works with a local reducer state. If this is one global state and multiple instances of the same component are listening to it, all those components will start the fetch. If it were only one central component that handled all async stuff I wouldn't really see the difference to one central middleware.
• I don't see how this approach allows for cancelling, which you are (rightfully) critiqueing with thunks
• Adding additional actions will re-trigger the previous ones if they haven't been removed yet. Only always returning one action might lead to missed actions, if multiple dispatches are batched
• If the component that would execute the fetch would be dismounted before the fetch is executed, you would end up with half-baked asynchronous state somehow.

I am sure that some of these things can be addressed by additional safeguards in an actual implementation, but that would get quite complicated and potentially very long.
A lot more complicated than for example writing a saga, which many people already consider overkill for many use cases.

I think the idea of declarative side effects is quite appealing, but from the examples in the blog posts, I see more problems than solutions right now. Especially binding the execution of these async actions into component lifecycle seems a bit weird to me. And I wouldn't really know how we could teach this without completely confusing people.

Could you provide one example implementation that does not have the problems I described so that we can see if this is feasible?

[neurosnap]

I've spent a lot of time thinking about and writing code using concepts like side-effects as data, so I'm very excited to see redux-toolkit contributors are discussing it!

If you want a battle-tested example of writing declarative side-effects in the redux ecosystem, redux-saga should be the primary source of inspiration. re-frame is also another great resource that emphasizes side-effects as data.

@markerikson and I have already had plenty of discussions about redux-saga and why it's hard to nominate it for official support. Personally, generators are so incredibly useful for async flow control that it's worth the struggle to onboard developers to the concept. There are definitely downsides but I've built many large scale web applications using redux-saga and I would need a very compelling reason to use something else for managing side-effects, even for moderately-sized web apps.

Having said that, the arguments against it are still valid and I have spent some time trying to figure out how to solve those shortcomings.

Here are some libraries that I've built to try to make adoption easier. I lend these to you all as possible inspiration.

• https://erock.io/simplify-testing-async-io-javascript/
• https://github.com/neurosnap/cofx
• https://github.com/neurosnap/redux-cofx
• https://github.com/neurosnap/use-cofx

cofx is detached from redux but shares a similar API to redux-saga. Building on top of that, redux-cofx is a hybrid between redux-thunk and redux-saga. The benefit is there are no watchers, they are used like redux-thunk but instead of the middleware calling a simple function, it will also properly handle executing generators. Because cofx is separated from redux it's also easy to create a hook using a very similar API, which is not straightforward with redux-saga.

Please forgive me for citing my own projects, but I do feel like they are relevant and possible sources of inspiration to discuss.

[phryneas]

This unfortunately doesn't solve the main gripe I have with saga (which I - in theory - love).
It's not typesafe. You cannot have TS infer the return type of yield based on what you yielded.
And until TS gets better support (and I have no idea how that would look) for generator function typings, that won't change.

It's unfortunate, but as it currently stands, I think the best way to go is something promise/async-based.

[neurosnap]

Totally agree, typescript is the worst part about using generators for async flow control, no argument there. I find the compromise acceptable, but others might not.

[bdwain]

@phryneas @neurosnap redux-loop provides declarative side effects (without generators) and we've been improving the type safety a lot lately. We have an upcoming major release that should fix the remaining issues with it. While I am a biased maintainer, I have used it for a large application for 4 years now and find it works very well.

[martpie]

Maybe I can share a little bit a real-world example I am trying to tackle right now for some inspiration (you are all thinking this problem much better than me).

• I have a list of products in a cart
• I have a list of discounts, depending on multiple factors, let's say:
  • the number of products
  • the combination of some products
  • the birth date of the user
  • the zip code of the user

Any time one of this parameter changes, I need to re-fetch the discounts (from a specific API endpoint)

Solution 1: call refreshDiscounts from components (componentDidUpdate, events handlers...)

• dev error prone
• not centralized
• some edgecases may be missed
• false positives

Solution 2: call refreshDiscounts from thunks

• it works, and is (kind of) centralized
• some advanced use-cases may be hard to tackle (data aggregation from multiple slices)

Solution 3: subscribe to store changes via redux-on-state-change

this is my current solution, but it is hard to test, and a bit dirty:

• you can do advanced comparisons/aggregate data from multiple stores to see if you need to refresh the discounts
• performance impact

Others

• redux-saga (heavy, steep learning curve)
• redux-loop (a good in between, but same issues as thunks)
• etc

---

I think this is a particularly common scenario, does anyone have suggestions/best practices/implementations to share?

[phryneas]

@martpie I'd dispatch a thunk from a component - the thunk itself would decide if there is actually a need of refreshing some things, depending on when the last refresh was etc. (probably overridable from a thunk argument, so that lifecycle refresh could be handled differently from "the user pressed a refresh button" refresh)

I don't really see why your "aggregation of data" there would be more complicated than in any other solution (given that thunks have access to the full state), but I believe that those values might actually be derived values - and those should never be persisted in the store, but rather be calculated in a (possibly memoized) selector.
The store should probably contain discount percentages, but not the finalized prices.

[thirafidide]

@martpie In case of redux-loop, you can make an action of REFRESH_DISCOUNT which only run API and doesn't change state

case 'REFRESH_DISCOUNT':
  return loop(state, Cmd.run(refreshDiscount, {
    args: [getArgs(state, action)]
  });

then whichever action that would trigger the refresh should call the action via Cmd.action

case 'ADD_PRODUCT':
  const newState = addProduct(state, action.product);
  return loop(newState, Cmd.action({ type: 'REFRESH_DISCOUNT'; additionalParams: ... }))

So you can refresh the discount from any slices. If you put the handler of REFRESH_DISCOUNT on reducer high enough so it has access to any necessary state, I think there is no problem with data aggregation

[bdwain]

You can also pass getState (and dispatch) to the refreshDiscount method if your reducer does not have access to the data that the function needs on its own.

case 'REFRESH_DISCOUNT':
  return loop(state, Cmd.run(refreshDiscount, {
    args: [whatever, whatever2, Cmd.getState, Cmd.dispatch]
  });

//refresh-discount.js
function refreshDiscount(whatever, whatever2, getState, dispatch){
  const myPieceOfState = getState().my.piece.of.state;
  dispatch({type: 'hello'});
}