fix(host-service): dedupe PR refresh calls with repo-keyed cache

[Kitenite]

Multiple projects targeting the same GitHub repo each fired their own GraphQL query every 10s, and force=true paths bypassed the per-project debounce entirely. Replace with a single repo-keyed response cache so N projects on the same repo collapse to 1 call, branch-sync/tRPC bursts share the in-flight promise, and the polling cadence drops to 20s.

Description

Related Issues

Type of Change

• Bug fix
• New feature
• Documentation
• Refactor
• Other (please describe):

Testing

Screenshots (if applicable)

Additional Notes

---

Summary by cubic

Deduplicates GitHub PR refresh calls in host-service with a repo-keyed cache and caches PR detail fetches to cut duplicate API/CLI calls across projects and views.

• Bug Fixes
  • Added repo-keyed PR list cache (10s TTL) and gh pr view detail cache (30s TTL); both share in-flight promises and evict on failure.
  • Explicit refreshes (branch sync and refreshByWorkspaces) bypass the repo cache to avoid stale reads; bypassed fetches still populate the cache while poll ticks keep deduping.
  • Increased project refresh interval to 20s; removed per-project debounce/force and rely on in-flight dedupe via a unified refresh path.

^{Written for commit d1c299c. Summary will update on new commits. Review in cubic}

Summary by CodeRabbit

• Refactor
  • Reduced redundant GitHub requests to lower network traffic
  • Added repository-level deduplication with a 10s TTL to speed repeated PR listings
  • Cached pull request content fetches to avoid repeated external CLI calls
  • Unified refresh behavior across startup and workspace/branch triggers
  • Increased project refresh interval from 10s to 20s
  • Failed cache entries are evicted so subsequent attempts retry promptly

[coderabbitai]

📝 Walkthrough

Walkthrough

Repository-level promise dedupe cache for GitHub PR fetches (10s TTL, evict on failure) added; per-project refresh throttling (nextProjectRefreshAt) removed and PROJECT_REFRESH_INTERVAL_MS increased to 20s; refreshEligibleProjects no longer accepts force; refreshProject now accepts { bypassCache?: boolean } and bypasses repo cache when requested. Also adds a TTL-backed in-flight/fulfilled promise cache for gh pr view in the workspace-creation procedure.

Changes

Cohort / File(s)	Summary
Pull-request polling packages/host-service/src/runtime/pull-requests/pull-requests.ts	Increased PROJECT_REFRESH_INTERVAL_MS to 20s; removed per-project nextProjectRefreshAt throttling and force propagation; refreshEligibleProjects no longer takes force; refreshProject signature changed to accept { bypassCache?: boolean }; added repoPullRequestCache that deduplicates in-flight GraphQL/PR-node fetches by owner/name with a 10s TTL and evicts entries on fetch failure.
GitHub PR content caching packages/host-service/src/trpc/router/workspace-creation/procedures/get-github-pull-request-content.ts	Introduced a normalized key TTL-based Map cache that stores in-flight/completed promises for gh pr view calls; reuses cached promise within TTL; evicts entry on rejection; parses and normalizes results (lowercased state, defaulted body, nullable logins -> null).

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant HostService
  participant RepoCache
  participant GitHubAPI

  Caller->>HostService: request PR data (project-trigger or explicit)
  HostService->>RepoCache: lookup cache by owner/name
  alt cache hit (valid TTL)
    RepoCache-->>HostService: return cached promise/result
  else cache miss or expired
    HostService->>RepoCache: store in-flight promise
    HostService->>GitHubAPI: fetch PR nodes (GraphQL / gh)
    GitHubAPI-->>HostService: PR data or error
    alt success
      RepoCache-->>HostService: resolved result (store with fetchedAt)
      HostService-->>Caller: return PR data
    else failure
      HostService->>RepoCache: evict entry
      HostService-->>Caller: return error
    end
  end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 I hop through code with whiskers bright,
Bundling promises, keeping fetches light.
Ten seconds of shelter, errors set free,
Refreshes simpler—now dance with me! 🥕✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: adding a repo-keyed cache to deduplicate PR refresh calls. It's concise, clear, and directly reflects the primary objective of the changeset.
Description check	✅ Passed	The description includes a helpful summary and cubic-generated details about the changes, but the PR template sections (Related Issues, Type of Change, Testing, Screenshots, Additional Notes) are empty checkboxes and placeholder comments.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch gh-cli-throttle-debug

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Review rate limit: 2/8 reviews remaining, refill in 41 minutes and 2 seconds.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

packages/host-service/src/runtime/pull-requests/pull-requests.ts (1)

372-395: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Restore a cache-bypass path for explicit refreshes.

Now that refreshProject no longer accepts a force/bypass option, both refreshPullRequestsByWorkspaces() and the branch-sync follow-up are forced through the settled repo cache too. That means a PR opened/closed or checks update right after the last poll can still come back stale here for up to 10 seconds, which regresses the old “refresh now” behavior.

Suggested direction

- private async refreshProject(projectId: string): Promise<void> {
+ private async refreshProject(
+   projectId: string,
+   options: { bypassRepoCache?: boolean } = {},
+ ): Promise<void> {

- await Promise.all(
-   projectIds.map((projectId) => this.refreshProject(projectId)),
- );
+ await Promise.all(
+   projectIds.map((projectId) =>
+     this.refreshProject(projectId, { bypassRepoCache: true }),
+   ),
+ );

Thread that option down to getCachedRepoPullRequests() so only poll-driven refreshes reuse settled cache entries.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@packages/host-service/src/runtime/pull-requests/pull-requests.ts` around
lines 372 - 395, refreshProject currently forces all callers through the settled
repo cache because it no longer accepts a force/bypass flag; restore a bypass
path by adding a boolean parameter (e.g., forceRefresh) to refreshProject and
thread it into performProjectRefresh so callers like
refreshPullRequestsByWorkspaces and the branch-sync follow-up can pass true to
bypass settled cache logic; ensure performProjectRefresh forwards the flag down
to getCachedRepoPullRequests (and any intermediate helpers) so only poll-driven
refreshes reuse settled cache entries while explicit refreshes skip it; preserve
inFlightProjects behavior by keying by projectId only and storing the promise
returned when forceRefresh is false, but for forceRefresh === true always call
performProjectRefresh directly (or store a separate in-flight entry) so explicit
bypasses are not blocked by settled cached promises.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@packages/host-service/src/runtime/pull-requests/pull-requests.ts`:
- Around line 372-395: refreshProject currently forces all callers through the
settled repo cache because it no longer accepts a force/bypass flag; restore a
bypass path by adding a boolean parameter (e.g., forceRefresh) to refreshProject
and thread it into performProjectRefresh so callers like
refreshPullRequestsByWorkspaces and the branch-sync follow-up can pass true to
bypass settled cache logic; ensure performProjectRefresh forwards the flag down
to getCachedRepoPullRequests (and any intermediate helpers) so only poll-driven
refreshes reuse settled cache entries while explicit refreshes skip it; preserve
inFlightProjects behavior by keying by projectId only and storing the promise
returned when forceRefresh is false, but for forceRefresh === true always call
performProjectRefresh directly (or store a separate in-flight entry) so explicit
bypasses are not blocked by settled cached promises.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1bf7432f-4888-49ad-94eb-d6416ada85b8

📥 Commits

Reviewing files that changed from the base of the PR and between f5cf3f1 and fdcef1b.

📒 Files selected for processing (1)

• packages/host-service/src/runtime/pull-requests/pull-requests.ts

[github-actions]

🧹 Preview Cleanup Complete

The following preview resources have been cleaned up:

• ✅ Neon database branch

Thank you for your contribution! 🎉

[coderabbitai]

♻️ Duplicate comments (1)

packages/host-service/src/runtime/pull-requests/pull-requests.ts (1)

508-538: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Keep in-flight repo fetches reusable when bypassing settled cache.

bypassCache currently skips the lookup entirely, and fetchedAt is stamped before the GitHub call resolves. That lets two explicit refresh bursts for the same repo—or any slow fetch that runs longer than the 10s TTL—fan back out into duplicate GraphQL requests, which is the main failure mode this cache is meant to prevent.

Suggested fix

 private readonly repoPullRequestCache = new Map<
   string,
-  { promise: Promise<GraphQLPullRequestNode[]>; fetchedAt: number }
+  {
+    promise: Promise<GraphQLPullRequestNode[]>;
+    fetchedAt: number | null;
+  }
 >();

 private async getCachedRepoPullRequests(
   repo: NormalizedRepoIdentity,
   options: { bypassCache?: boolean } = {},
 ): Promise<GraphQLPullRequestNode[]> {
   const cacheKey = `${repo.owner.toLowerCase()}/${repo.name.toLowerCase()}`;
-  if (!options.bypassCache) {
-    const cached = this.repoPullRequestCache.get(cacheKey);
-    if (
-      cached &&
-      Date.now() - cached.fetchedAt < REPO_PULL_REQUEST_CACHE_TTL_MS
-    ) {
-      return cached.promise;
-    }
+  const cached = this.repoPullRequestCache.get(cacheKey);
+  if (
+    cached &&
+    (cached.fetchedAt === null ||
+      (!options.bypassCache &&
+        Date.now() - cached.fetchedAt < REPO_PULL_REQUEST_CACHE_TTL_MS))
+  ) {
+    return cached.promise;
   }

-  const fetchedAt = Date.now();
   const promise = (async () => {
     const octokit = await this.github();
-    return fetchRepositoryPullRequests(octokit, {
+    const nodes = await fetchRepositoryPullRequests(octokit, {
       owner: repo.owner,
       name: repo.name,
     });
+    const entry = this.repoPullRequestCache.get(cacheKey);
+    if (entry?.promise === promise) {
+      entry.fetchedAt = Date.now();
+    }
+    return nodes;
   })();
   // Evict on failure so the next caller retries instead of serving a
   // poisoned cache entry for the rest of the TTL.
   promise.catch(() => {
     if (this.repoPullRequestCache.get(cacheKey)?.promise === promise) {
       this.repoPullRequestCache.delete(cacheKey);
     }
   });
-  this.repoPullRequestCache.set(cacheKey, { promise, fetchedAt });
+  this.repoPullRequestCache.set(cacheKey, { promise, fetchedAt: null });
   return promise;
 }

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@packages/host-service/src/runtime/pull-requests/pull-requests.ts` around
lines 508 - 538, The cache currently skips lookup when bypassCache is true and
stamps fetchedAt before the GitHub call settles, causing duplicate GraphQL
requests; update getCachedRepoPullRequests to always check repoPullRequestCache
for an existing entry and, if an entry exists whose promise is still pending,
return that promise even when options.bypassCache is true, but only bypass a
settled (non-pending) cached result when bypassCache is true; also avoid setting
fetchedAt before the fetch resolves—set fetchedAt when the promise fulfills
(e.g., in promise.then) so TTL is based on the completion time, while keeping
the existing promise.catch eviction logic that deletes the cacheKey if the
stored promise fails.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@packages/host-service/src/runtime/pull-requests/pull-requests.ts`:
- Around line 508-538: The cache currently skips lookup when bypassCache is true
and stamps fetchedAt before the GitHub call settles, causing duplicate GraphQL
requests; update getCachedRepoPullRequests to always check repoPullRequestCache
for an existing entry and, if an entry exists whose promise is still pending,
return that promise even when options.bypassCache is true, but only bypass a
settled (non-pending) cached result when bypassCache is true; also avoid setting
fetchedAt before the fetch resolves—set fetchedAt when the promise fulfills
(e.g., in promise.then) so TTL is based on the completion time, while keeping
the existing promise.catch eviction logic that deletes the cacheKey if the
stored promise fails.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6cceb4ff-2495-4bfa-b017-1c414fde3617

📥 Commits

Reviewing files that changed from the base of the PR and between 4975b41 and d1c299c.

📒 Files selected for processing (1)

• packages/host-service/src/runtime/pull-requests/pull-requests.ts