on error in createDataStreamResponse

[pedrocarnevale]

Hello all!!
I'm having a problem here. Sometimes openai is not available, and the API is down. So, to increase availability of my app, I use a togetherAi model if the openAi API is down.

The function getResponse = async (isError: boolean, dataStream: DataStreamWriter) is where I get the streamText. If isError is false, I use openAi, if it's true I use togetherAi. So I use this try ... catch to try to call togetherAi if the first response fails. But when an error happens it does not go to the catch, it goes to the onError.

So, when OpenAi is down, my endpoint returns an error, without trying to use togetherAi

How can I implement the logic that I want, so the availability of my endpoint increases?

return createDataStreamResponse({
        onError: (error: any) => {
          throw new Error(error);
        },
        execute: async (dataStream) => {
          let response = null;

          try {
            response = await getResponse(false, dataStream);
          } catch (e) {
            response = await getResponse(true, dataStream);
          }

          if (!response) {
            throw new Error('Error generating response');
          }

          response.mergeIntoDataStream(dataStream);
        },
      });

[lgrammel]

Use a fallback middleware instead https://github.com/remorses/ai-fallback