Fixing the static site vibes: papercuts and head-scratching

[daniloc]

I volunteer as tribute.

I will dedicate immediate bandwidth to address this but I'm starting an issue before diving headlong into the problem:

• To accept your shortcuts
• To prevent duplicated effort
• To build context

It is hard to work on the static site

Local builds take awhile, Vercel preview builds take eons, and transient errors in both domains compound with the build times to bleed away time, energy and forward momentum.

Example case

1. Vercel build is failing. Why? I cannot say because, absent a Vercel seat to inspect the logs and errors, the failure is fully opaque.
2. Very well. I will build the site locally. It's just a few minutes of yarn build.
3. Except the local build is failing on paths entirely unrelated to the WIP, like this business with the /careers path:

Building static HTML failed for path "/careers"

See our docs page for more info on this error: https://gatsby.dev/debug-html


  125 |     const [processedHtml, setProcessedHtml] = useState('')
  126 |     const [websiteDescription, setWebsiteDescription] = useState('')
> 127 |     const teamsField = selectedJob.parent.customFields.find((field: { title: string }) => field.title ===
'Teams')
      |                                    ^
  128 |     const teams = teamsField ? JSON.parse(teamsField.value) : []
  129 |     const [selectedTeamName, setSelectedTeamName] = useState(teams[0])
  130 |     const selectedTeam = allTeams.find((team) => team.name.toLowerCase() === selectedTeamName.toLowerCase())


  WebpackError: TypeError: Cannot read properties of undefined (reading 'parent')

(My gut says we've got a failed Ashby API request in the mix failing to populate a job's properties, but I don't know. In any case, the mental overhead of reasoning through the content of /careers while working elsewhere has limited ROI.)

4. Eventually the build will work again, but I cannot predict the timeline, so forward motion on the WIP stops.

Not ideal, and over time, corrodes the leverage of all the salaries going into site content.

What can we do?

All of my suggestions are naive, I'm just starting the convo:

• Can we just give Vercel what they want short term to get more seats or more throughput?
• Could we pipe the contents of failure or logs into pull requests?
• Do we need an API caching proxy layer over Ashby to prevent our builds being rate-limited? We're surely an outlier in how often we're hitting their endpoints (or maybe I'm too attached to my pet theory)
• Are we hoping to give Vercel the boot, and need technical diligence or test deployments on a new platform?
• Something else? Some refactoring we've not had time to wade hip-deep into?

If there's stuff backlogged and you need me to be your developer experience donkey to pull it out of the mud, you can have my unlimited and immediate bandwidth to accomplish this! The compound returns fully justify the side quest imo.

Thank you for your time and generous indulgence!

[corywatilo]

I'll start by ultimately saying: don't worry about the builds! 😊 That's what we're here for – to monitor things, fix issues as they arise, and get PRs merged that are ready to go live. Hopefully builds aren't getting in the way of you writing content and putting up PRs!

If you're having issues running the site locally, let's definitely get to the bottom of it! Reach out in Slack and we'll get you taken care of.

But generally speaking we're always around and monitoring the build situation. If a build is extremely time-sensitive, definitely shoot a message in Slack and someone can restart a build if it fails. Otherwise, we'll usually get to it within an hour or two. (And you can definitely have a Vercel seat if it's helpful for you - we just don't hand them out to everyone by default due to cost.)

Vercel

Vercel builds fails mostly due to memory spikes during the build process. When this happens, this is all you see in Vercel:

We've spent a lot of time debugging and making optimizations – and it's still ongoing – and we've been in touch with Vercel, though they've been mostly unhelpful. Our issue would ultimately be fixed by a memory increase, though they won't give it to us unless we entertain upgrading to enterprise. Our monthly bill is north of $500/mo as it stands now – more than half of which is just seats. Our bill would multiply by a significant factor if we went the enterprise route, which I think we can all agree is absurd for a static website.

In working with Vercel support, I've found them to be well-intended though mostly unhelpful ("Try upgrading to Node 20") or suggesting things we've already tried – and extremely slow to respond at that (last reply took 4 days).

We've also explored other hosting options (like Amplify) where build consistently work (and with even less RAM than Vercel, mind you), but there is a security risk in that, out of the box, we can't require approvals for external contributors. (You could theoretically print out all of our environment variables, for example.) Eli is looking at workarounds for this.

Other build issues

This isn't to say 100% of the issues are on Vercel. Sometimes builds will fail for other reasons like:

• Missing a custom field value in Ashby (like when a new role is added and it's not set up correctly)
• Missing a photo for a new team member (I accidentally uploaded a photo to the wrong field recently)
• Small team renames
  • When this happens, any previous build will fail as files move around on disk, so any existing branch needs to have master merged into it to get the latest updates. This is a mostly recent issue, as we've been going through more and more renames, something that was a rare occurrence in the past. We'll find a better process for this at some point. It just hasn't been top of the priority list yet.

Some of these we fix as they come up, some we haven't as they're rare and not worth fixing compared to other priorities.

Building locally

When running the site locally, there are some steps that are omitted like building the careers section as well as the merch store – areas of the site that regular contributors don't need. This actually greatly speeds up starting the local development server.

To build the full site locally, you just need API keys. This would likely be why you're getting an Ashby failure or having issues building locally. (I've personally never seen that Ashby error before – but I also run with many more API keys.)

Generally speaking as we see issues with running or building the site, we put in a fix if it's warranted. But we also don't know of all issues everyone else has with running the site locally, so definitely give a shout if you're having specific problems. We'll at least be able to point you in the right direction – if not give you the fix immediately – and it will It definitely save you hours of trying to reverse engineer 3+ years of code. We've got lots of knowledge here - happy to share it, just ask!

[daniloc]

Thank you for the backup!

Hopefully builds aren't getting in the way of you writing content and putting up PRs!

Unfortunately, they have, enough I felt compelled to raise this issue and re-prioritize things if it would be helpful. I should emphasize that I'm speaking specifically of builds that yield deploy previews, as:

• These present an opportunity for QA
• They allow low-friction reviews of someone else's PR or newly-added commits
• They confirm that thornier changes worked as hoped, especially as relates to things like compiling .mdx content
• The deployed environment is the final word and its output can differ from the output of yarn start for all the reasons you mentioned, so in cases of ambiguity, it's nice to see exactly what will ship before a merge

A single flat .md file is straightforward enough under status quo.

But wading into the docs end of the house this week has been made thornier by the unpredictable, elliptical orbits of the deploy previews. In one particular case, I'd put down serious money I could have doubled my output for a day or two had they worked reliably.

I think our approach to docs and content folded in as first-class citizens of the .com site is infinitely better than what's typical, so I'm happy to sustain the long-term complexity lift it demands.

But man, there's some shin-bangers lurking there.

[corywatilo]

Let's just start with inviting you to Vercel so you can peep your own builds. Sent you an invite.

[corywatilo]

Forgot to mention - we also have a hey at posthog account for Vercel - creds in 1pass for anyone else who might want to check a build

[daniloc]

@corywatilo Thank you! I missed the invitation cutoff, trouble you to resend it?