Any way to gracefully fail GraphQL queries?

[samajammin]

Hey folks - first off, thanks for creating such a great dev product & community! Loving Gatsby so far.

I'm curious if there's a way for GraphQL queries to fail gracefully vs. breaking the build.

My project uses the GitHub GraphQL API and occasionally we hit their rate limit. I'm wondering if there's a way to have our API request fail gracefully (e.g. by returning empty results) vs. causing the entire build to fail when the API returns a 403 response code?

Here's an example of a broken build in Netlify:

4:30:59 PM: success run static queries - 2.796s - 9/9 3.22/s
4:34:39 PM: error There was an error in your GraphQL query:
4:34:39 PM: API rate limit exceeded
4:34:39 PM: File path: /opt/build/repo/src/templates/docs.js
4:34:39 PM: Url path: /sl/developers/docs/smart-contracts/libraries/
4:34:39 PM: Plugin: none

Full logs can be found here.

Here's an example of the error when running yarn start locally:

error There was an error in your GraphQL query:

Response not successful: Received status code 403

  12 |       title
  13 |       description
  14 |       incomplete
  15 |       sidebar
  16 |       sidebarDepth
  17 |     }
  18 |     body
  19 |     tableOfContents
  20 |   }
  21 |   gitData: github {
> 22 |     repository(name: "ethereum-org-website", owner: "ethereum") {
     |     ^
  23 |       ref(qualifiedName: "master") {
  24 |         target {
  25 |           __typename
  26 |           ... on GitHub_Commit {
  27 |             history(path: $relativePath) {
  28 |               edges {
  29 |                 node {
  30 |                   message
  31 |                   commitUrl
  32 |                   author {

I'm still able to run the app locally when this error takes place but trying to yarn build completely fails.

Thanks in advance!

[KyleAMathews]

Is this with gatsby-source-graphql?

[samajammin]

Thanks for the quick reply! Yes, this is with gatsby-source-graphql.

Our full config is here:
https://github.com/ethereum/ethereum-org-website/blob/dev/gatsby-config.js#L212

Now that you mention, I see there's ability to batch queries:
https://www.gatsbyjs.com/plugins/gatsby-source-graphql/#performance-tuning

Is that the approach you'd recommend here to avoid rate limits? I'll look into that!

[KyleAMathews]

Our general recommendation is to use apollo-link which you can see some documentation in the README about https://www.gatsbyjs.com/plugins/gatsby-source-graphql/

Another helpful link https://medium.com/@joanvila/productionizing-apollo-links-4cdc11d278eb

We're discussing making our fetching more sophisticated by default to handle stuff like retries after errors and throttling requests when you hit rate limits. If you're interested in helping us work on that, lemme know!

[vladar]

We are going to add a new section in docs about this. Not published yet but you can check it here:

https://github.com/gatsbyjs/gatsby/blob/47a48dc0739c3fe8303694d164d87348da11c66a/packages/gatsby-source-graphql/README.md#composing-apollo-links-for-production-network-setup

[samajammin]

Hey @KyleAMathews @vladar - thanks again for your replies here. I'm digging back into this now & following this code example: https://www.gatsbyjs.com/plugins/gatsby-source-graphql/#composing-apollo-links-for-production-network-setup

Current problem is I hit this error:

error "gatsby-source-graphql" threw an error while running the sourceNodes lifecycle:

ApolloLink is not defined

  66 |
  67 |   if (createLink) {
> 68 |     link = await createLink(options);
     |                  ^
  69 |   } else {
  70 |     const options = {
  71 |       uri: url,

When I try to import ApolloLink in my gatsby-configs.js with

const { ApolloLink } = require(`apollo-link`)

...I hit this new error:

error "gatsby-source-graphql" threw an error while running the sourceNodes lifecycle:


fetch is not found globally and no fetcher passed, to fix pass a fetch for
your environment like https://www.npmjs.com/package/node-fetch.

For example:
import fetch from 'node-fetch';
import { createHttpLink } from 'apollo-link-http';

const link = createHttpLink({ uri: '/graphql', fetch: fetch });

  66 |
  67 |   if (createLink) {
> 68 |     link = await createLink(options);
     |                  ^
  69 |   } else {
  70 |     const options = {
  71 |       uri: url,


  Error: Invariant Violation:
  fetch is not found globally and no fetcher passed, to fix pass a fetch for
  your environment like https://www.npmjs.com/package/node-fetch.
  For example:
  import fetch from 'node-fetch';
  import { createHttpLink } from 'apollo-link-http';
  const link = createHttpLink({ uri: '/graphql', fetch: fetch });

Including const fetch = require("node-fetch") within gatsby-config.js doesn't solve the issue.

Any further guidance here is greatly appreciated 🙏

I tried upgrading to latest version of gatsby & gatsby-source-graphql to no avail. Here's my full gatsby-config.js & package.json for reference:

• https://gist.github.com/samajammin/a841a29a78ecc11a6f173fb8445a29c9
• https://gist.github.com/3c1163d39e989d20b3afdfcdecfee9ab

[samajammin]

I suppose one thing I'm unclear on: should I be using Apollo Link with Apollo client? As of now I'm not, as it wasn't mentioned in your docs & the Apollo Link docs do mention is can be used "as a standalone client".

[samajammin]

Ok I believe I got around that issue by passing fetch to createHttpLink (more on that here), so my gatsby-config.js looks like this:

const { ApolloLink } = require(`apollo-link`)
const { createHttpLink } = require(`apollo-link-http`)
const { onError } = require("apollo-link-error")
const fetch = require("node-fetch")

const errorLink = onError(({ graphQLErrors, networkError }) => {
  if (graphQLErrors) {
    graphQLErrors.map(({ message, locations, path }) => {
      console.log(`GraphQL Error:`)
      console.log({ message, locations, path })
    })
  }
  if (networkError) {
    console.log(`Network Error: ${networkError.message}`)
  }
})

...

module.exports = {

  ...

  plugins: [

  ...

    {
      resolve: `gatsby-source-graphql`,
      options: {
        typeName: `GitHub`,
        fieldName: `github`,
        url: `https://api.github.com/graphql`,
        createLink: (pluginOptions) => {
          ApolloLink.from([
            errorLink,
            createHttpLink({ url: pluginOptions.url, fetch }),
          ])
        },
        headers: {
          Authorization: `Bearer ${GITHUB_TOKEN_READ_ONLY}`,
        },
      },
    },
  ]

Now I'm facing a new error:

 ERROR #11321  PLUGIN

"gatsby-source-graphql" threw an error while running the sourceNodes lifecycle:

Cannot read property 'request' of undefined

  86 |
  87 |     if (!sdl) {
> 88 |       introspectionSchema = await introspectSchema(linkToExecutor(link));
     |                                   ^
  89 |       sdl = printSchema(introspectionSchema);
  90 |     } else {
  91 |       introspectionSchema = buildSchema(sdl);

File: node_modules/gatsby-source-graphql/gatsby-node.js:88:35



  TypeError: Cannot read property 'request' of undefined

  - ApolloLink.js:55 Object.execute
    [ethereum-org-website]/[@apollo]/client/link/core/ApolloLink.js:55:22

Any sense of what's going on here?

[samajammin]

Update: I believe I have this working! See this commit: ethereum/ethereum-org-website@12b9310

Now I'm trying to add query batching. I'm keen to know why you recommend HttpLinkDataLoader vs apollo-link-batch-http:
https://www.gatsbyjs.com/plugins/gatsby-source-graphql/#apollo-style-batching

I just tried using apollo-link-batch-http in the same way I set up apollo-link-http:

    {
      resolve: `gatsby-source-graphql`,
      options: {
        typeName: `GitHub`,
        fieldName: `github`,
        url: `https://api.github.com/graphql`,
        createLink: (pluginOptions) => {
          return ApolloLink.from([
            loggerLink,
            errorLink,
            new BatchHttpLink({
              uri: pluginOptions.url,
              fetch,
              headers: {
                Authorization: `Bearer ${GITHUB_TOKEN_READ_ONLY}`,
              },
            }),
          ])
        },
        headers: {
          Authorization: `Bearer ${GITHUB_TOKEN_READ_ONLY}`,
        },
      },

But that creates an error:

 ERROR #11321  PLUGIN

"gatsby-source-graphql" threw an error while running the sourceNodes lifecycle:

Response not successful: Received status code 400



  ServerError: Response not successful: Received status code 400

  - index.ts:114 Object.exports.throwServerError
    [ethereum-org-website]/[apollo-link-http-common]/src/index.ts:114:17

  - index.ts:145
    [ethereum-org-website]/[apollo-link-http-common]/src/index.ts:145:11

  - task_queues.js:97 processTicksAndRejections
    internal/process/task_queues.js:97:5


not finished source and transform nodes - 41.350s
not finished Generating image thumbnails - 18.112s

error Command failed with exit code 1.

Thanks!

[samajammin]

FYI we ended up going with the batching approach mentioned in the Gatsby docs to solve this:
https://www.gatsbyjs.com/plugins/gatsby-source-graphql/#performance-tuning

Here's our PR for reference:
ethereum/ethereum-org-website#2295

Thanks folks!