Skip to content

feat: pre-compute developer apps selections in data layer#17180

Merged
wackerow merged 4 commits into
page-developer-appsfrom
unstable-cache
Feb 3, 2026
Merged

feat: pre-compute developer apps selections in data layer#17180
wackerow merged 4 commits into
page-developer-appsfrom
unstable-cache

Conversation

@wackerow
Copy link
Copy Markdown
Member

Extends page-developer-apps branch

Summary

  • Move randomization of highlighted apps and category previews from page-level unstable_cache to data-layer (trigger.dev task)
  • Add DeveloperToolsDataEnvelope type with appsById lookup + pre-computed selections
  • Remove unstable_cache wrappers from utils.ts
  • Update page components to use pre-computed selections with O(1) lookups

Problem

The previous implementation used unstable_cache at the page level, but unstable_cache creates cache keys from both the keyParts array and function
arguments. Passing the full apps array or searchParams as arguments created unique cache entries for each request, defeating the cache entirely.

Solution

Pre-compute all randomized selections in the trigger.dev task (daily refresh):

  • mainPageHighlights - 3 apps for landing page
  • categoryHighlights - 3 apps per category page (7 categories)
  • categoryPreviews - 5 apps per category card on landing page

Store as an envelope with appsById Record for O(1) modal lookups.

Audit Results

Area Status
Best practices ✅ O(1) lookups, pre-computation, strong types, deterministic seeded RNG
Star count filtering ✅ Preserved - sorts by max stargazers, takes top 9, then shuffles
unstable_cache removed ✅ Only remains in generic data layer, removed from apps code
searchParams issues ✅ Resolved - selections pre-computed, searchParams only used for O(1) lookups

Test plan

  • Verify highlights section shows 3 apps on main page
  • Verify highlights section shows 3 apps on each category page
  • Verify category cards show 5 preview apps each
  • Verify app modal opens correctly via ?appId= param
  • Verify tag filtering works via ?tag= param
  • Confirm no hydration mismatches

🤖 Generated with Claude Code

Move randomization of highlighted apps and category previews from page-level unstable_cache to data-layer (trigger.dev task).

Changes:
- Add DeveloperToolsDataEnvelope type with appsById lookup + selections
- Pre-compute selections (mainPageHighlights, categoryHighlights, categoryPreviews) in trigger.dev task
- Remove unstable_cache wrappers from utils.ts, revert unused `every` time util addition
- Update page components to use pre-computed selections with O(1) lookups
- Transform mock data to envelope format

This fixes cache invalidation issues caused by unstable_cache creating unique keys from function arguments (searchParams). Selections now persist daily and are
consistent for all users.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@netlify
Copy link
Copy Markdown

netlify Bot commented Jan 27, 2026

Deploy Preview for ethereumorg ready!

Name Link
🔨 Latest commit 18bc8ef
🔍 Latest deploy log https://app.netlify.com/projects/ethereumorg/deploys/69823f449e13980008215a03
😎 Deploy Preview https://deploy-preview-17180.ethereum.it
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
7 paths audited
Performance: 59 (no change from production)
Accessibility: 94 (no change from production)
Best Practices: 100 (🟢 up 1 from production)
SEO: 100 (no change from production)
PWA: 59 (no change from production)
View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions github-actions Bot added the tooling 🔧 Changes related to tooling of the project label Jan 27, 2026
Move all developer tools fetching logic into a dedicated folder to
maintain data layer isolation and prevent bidirectional imports
between data layer and app code.

Changes:
- Create src/data-layer/fetchers/developer-tools/ module with:
  - index.ts: main fetchDeveloperTools orchestrator
  - utils.ts: types, constants, and utility functions
  - fetchBuidlGuidl.ts: BuidlGuidl API fetcher
  - fetchGitHub.ts: GitHub GraphQL enrichment
  - fetchNpmJs.ts: npm downloads enrichment

- Move envelope types (DeveloperToolsDataEnvelope,
  DeveloperAppsComputedSelections) from app code to data layer

- Keep only UI-specific code in app (getCategoryTagStyle)

- Update all imports to use the new module structure

This ensures the data layer remains self-contained and doesn't depend
on app-level code for runtime functions.
Copy link
Copy Markdown
Member

@pettinarip pettinarip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wackerow hey, I think this works but I don't love the non-isolation code in the data-layer.

Check out the proposal made in #17213

Basically, grouped all related code in a subfolder inside the data-layer.

getMainPageHighlights,
getRandomPreviewsByCategory,
transformDeveloperAppsData,
} from "../../../app/[locale]/developers/apps/utils"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like too much this injection from things outside the data-layer. This module should be completely isolated from external code. The fetcher should do its thing and should have all the necessary logic to operate alone.

The fetcher is becoming too fat, logic heavy. Just for move this forward, I'd suggest putting all this logic next to the fetcher (it is logic closer to data fetching than UI).

Created a PR with a proposal for refactor.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wackerow wackerow merged commit 4d8400f into page-developer-apps Feb 3, 2026
6 checks passed
@wackerow wackerow deleted the unstable-cache branch February 3, 2026 20:34
This was referenced Feb 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tooling 🔧 Changes related to tooling of the project

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants