Derived atoms of ✨potentially asynchronous✨ atoms.

[iwoplaza]

The problem

Atoms for query/data management libraries (e.g. jotai-apollo) usually have a signature that looks something like this:

const userAtom: Atom<User | Promise<User>> = ...;

If no data is cached yet, the atom holds a Promise, but if the data is already there and an optimistic update was performed, the atom's data is updated in sync and the atom can hold a concrete value instead of a Promise. This dual nature is nice (concrete value or a promise) because derived atoms could process the value in sync if it is available, and fallback to awaiting and returning a Promise if it is not.

However, handling those cases looks to be more trouble that it is worth, so the easiest is to just always await.

const userAtom: Atom<User | Promise<User>> = ...;

// typeof uppercaseNameAtom is Atom<Promise<String>>
const uppercaseNameAtom = atom(async (get) => {
  const user = await get(userAtom);
  return user.name.toUppercase();
});

That is what I have done in my code as well, but because of that, the parts of the app that use any atom that derives userAtom will suspend on synchronous updates to the userAtom, causing tiny loading states and unnecessary remounts. I recently had an idea for a better way, lets try to build up to it gradually.

A potential solution

Below is an atom that handles both cases separately (processes the data in sync if it is available, returns a promise otherwise).

const userAtom: Atom<User | Promise<User>> = ...;

// typeof uppercaseNameAtom is Atom<string | Promise<String>>, which means we can derive it
// again and get the same dual behavior.
const uppercaseNameAtom = atom((get) => {
  const process = (user: User) => user.name.toUppercase();

  const userOrPromise = get(userAtom);
  if (isPromise(userOrPromise)) {
    // data not available yet, process it asynchronously
    return userOrPromise.then(process);
  }

  // data is here, process it now
  return process(userOrPromise);
});

It can be a bit too verbose though, and tougher to do more complex processing with. Let's try to simplify this a little. Say we have a function soon with the following signature:

type PromiseOrValue<T> = T | Promise<T>;

function soon<T, R>(data: PromiseOrValue<T>, process: (data: T) => R): PromiseOrValue<R> {
  // run `process` synchronously if `data` is known, return promise otherwise.
}

const userAtom: Atom<User | Promise<User>> = ...;

// typeof uppercaseNameAtom is Atom<string | Promise<String>>
const uppercaseNameAtom = atom(
  (get) => soon(get(userAtom), (user) => user.name.toUppercase())
);

For multiple dependencies, I made a Promise.all equivalent aptly called soonAll, which is an identity function if all arguments are known, and returns a Promise to an array with the concrete input values otherwise.

Since most of my derived atoms were just a few sync/async dependencies that were transformed in some way, I wanted to create a simple helper to reduce the amount of boilerplate I had to write:

const userAtom: Atom<User | Promise<User>> = ...;

// typeof uppercaseNameAtom is Atom<string | Promise<String>>
const uppercaseNameAtom = derive([userAtom], (user) => user.name.toUppercase());

Or an example with multiple dependencies:

const usersAtom: Atom<User[] | Promise<User[]>> = ...;
const nameFilterAtom = atom('');

// typeof filteredUsersAtom is Atom<User[] | Promise<User[]>>
const filteredUsersAtom = derive(
  [usersAtom, nameFilterAtom],
  (users, nameFilter) => {
    if (!nameFilter) {
      return users;
    }
    
    return users.filter((user) => user.name.includes(nameFilter));
  }
);

Potential enhancements

So far I explored making the soon inspect the passed in Promise for a status field, and if it is fulfilled, it returns the value of the Promise. I wrote a couple tests for it, and it seems to be working fine. The main advantage of this approach, is that we can derive fully async atoms, and still be able to process the data in sync (for example, we change a sync dependency, but we also depend on 3 async atoms that are already fulfilled).

Conclusion

This has thus far been a real help in my codebase, decreasing redundant computations. If this is too specific to be included in the library, I would be happy to create a separate module, maybe jotai-derive?

[dai-shi]

Thanks for opening up a discussion.

This dual nature is nice (concrete value or a promise) because derived atoms could process the value in sync if it is available, and fallback to awaiting and returning a Promise if it is not.

Yes, this is pretty interesting. I knew this was possible and thought it's kind of advanced usage.

However, handling those cases looks to be more trouble that it is worth, so the easiest is to just always await.

Exactly.

causing tiny loading states and unnecessary remounts.

Exactly.

Below is an atom that handles both cases separately (processes the data in sync if it is available, returns a promise otherwise).

I'm so glad to see you've reached there. I think I did something similar somewhere.

Say we have a function soon with the following signature:

Niceeee.

If this is too specific to be included in the library, I would be happy to create a separate module, maybe jotai-derive?

I'm very excited to see your solution.
Yes, my suggestion is to create a separate module. (Mainly due to that it allows you rapid development without constraints. We might consider adding the outcome in this library, but my gut feeling is that it's not necessary. A separate module would be just fine. We should have it in docs though.)
May I invite you to https://github.com/jotaijs ?

[iwoplaza]

I will make it a separate module then 🫡. Thanks for the quick response, and yes, I would love an invite to https://github.com/jotaijs 💜

[dai-shi]

Cool. Sent an invitation.

[sebinsua]

I quite frequently do these things within my own applications but haven't yet attempted to generalise and abstract this out in the way you have. I have swr wrappers (for getting the latest value) and many usages of unwrap and conditional checks, etc (to avoid aspects of my application needing loading indicators).

I agree that there should be a nicer way of doing these things, but some of this API I find kind of unusual. Like, calling a function derive when really any atom can "derive" -- it doesn't really describe to me what is different about what this does compared to just "deriving" from an async atom normally.

And then internally we have soon and soonAll but these are also not very descriptive of what they do either. I was thinking that there is some similarity between soon and unwrap -- unwrap always unwraps a value (and doesn't apply a transform), while this applies a transformation and then only unwraps to a synchronous value if the underlying atom does not contain a promise.

I don't have a good name for either right now though. It's kind of like a preferSyncAtom that takes a list of atoms it's dependent on and then tries to derive synchronously before falling back to an asynchronous derive?! On the other hand, the API is radically different from that of an atom, because you can't use get internally without running into the original issue you resolved, etc.

Or maybe another way of looking at it, is that it could be called something like transformPreferUnwrap or maybe derivePreferUnwrap...

[iwoplaza]

Thank you for the comment! Glad to see I am not the only one struggling with this. I agree that derive is very general, but I so far have not found a better name for it. My thinking on that was that I want to derive the inherent value of other atoms, irrespective of when they are available (known or a promise).

I do like the suggestions, maybe something with unwrap, transform or prefer would be better. I do worry that using the unwrap naming may clash and cause confusion with the already existing unwrap utility though.

const usersAtom: Atom<User[] | Promise<User[]>> = ...;
const nameFilterAtom = atom('');

// typeof filteredUsersAtom is Atom<User[] | Promise<User[]>>
const filteredUsersAtom = atomWithUnwrap(
  [usersAtom, nameFilterAtom],
  (users, nameFilter) => {
    if (!nameFilter) {
      return users;
    }
    
    return users.filter((user) => user.name.includes(nameFilter));
  }
);

[PhilipGrefe]

Hi all, I'd like to share my use case since I'm also struggling with potentially async values.

I'm writing a mobile app using react-native and expo-sqlite as my database. Reads and writes are async. My atoms typically look like so:

export const fooAtom = atomWithStorage(
  'foo',
  Promise.resolve(undefined as string | undefined), // trying to get the correct type inference here 🤔 🤷‍♂️
  {
    getItem: async () =>  await db.query.fooTable.findFirst() // some query,
    setItem: async (key, newFoo) => {
      await db.update() // some updater
    },
    removeItem: async () => {
      await db.delete() // some deleter
    },
    subscribe: // some subscriber with DB-listener
  },
  { getOnInit: true },
)

For now, I simply unwrap all usages of atomWithStorage. If I don't do that, the components disappear for a split-second (not using Suspense).

Looking forward to future updates here 🥳