Split gatsbyjs.org into themes

[tesseralis]

Summary

Split parts of gatsbyjs.org into different themes in so that they can be iterated on faster.

Motivation

gatsbyjs.org takes a long time to build locally: up to ten minutes on a fresh build. This is due to (1) image optimization and (2) computationally expensive sources such as the list of starters and the entire packages directory which contains the source code for all of Gatsby and its official plugins.

This makes working on the website extremely difficult, if not impossible, especially if one needs to make changes in gatsby-node.js or gatsby-config.js, since changes to these files clear the cache and force a complete rebuild. Many potential website changes, such as creating routing for translations are blocked on the slow builds of the website.

By splitting off different logical sections of the Gatsby website into themes, we can run those themes individually without building other sections of the website that are more intensive. For example, we can run the "Tutorial" theme without compiling all the sources for "Plugins".

In addition, some of these themes, like for Documentation and Blogs, would be useful to provide to the Gatsby Community.

Detailed Design

Each section of gatsbyjs.org will be split into its own theme:

• Docs/Tutorial/Contributing
• Blog
• Plugins/Packages
• Creators
• Features
• Showcase
• Starters
• Guidelines

Each section should be able to:

• be run independently as a theme
• be excluded from the main website build, e.g. by commenting out its theme from gatsby-config.js

Each theme should start off as a local plugin and can freely use shared components from www/src. If deemed useful to the gatsby community as a whole, the theme can be moved to the packages directory.

Implementation

Steps:

• Move <Layout> component to layouts/index.js (www) Factor out <Layout> component in dot-org from most pages/templates #21229
• Split up www/gatsby-node.js refactor(www): Split up gatsby-node #21217
• Split up docs and blog page creation (www) Split up docs and blog page creation #21323
• Factor out components by section (www) Organize components in dot-org by section #21331

Problems

So far, I've run into the following issues when trying to refactor into themes:

Creators

template-creator-details queries allSitesYaml to determine which creators worked on what, but that node is associated with the "Showcase" theme.

Showcase

gatsby-transformer-screenshot looks for SitesYaml nodes by default. However, this and "Starters" would be their own themes, and I don't know if including the plugin in both themes would cause any problems. Ideally the plugin should just not have a default node.

In addition, because this section uses a lot of modals, it makes it hard to factor out Layout.

Drawbacks

Splitting sections up into themes won't help build times if the website in its entirety needs to be worked on. Individual sections can still take a long time (for example, Starters).

Alternatives

Other ways to potentially reduce build time:

• Add better checks in the frontend code so that source plugins can be commented out without crashing the build.
• Disable image processing on development, or disable images completely.

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜