diff --git a/docs/contributing/packages-and-manifests.md b/docs/contributing/packages-and-manifests.md index cd6f59f596..b052db4276 100644 --- a/docs/contributing/packages-and-manifests.md +++ b/docs/contributing/packages-and-manifests.md @@ -17,6 +17,8 @@ Use `package.json` as the canonical source of truth for saved package metadata. - `kody.id` — user-scoped Kody package id - `kody.description` — package description for search/detail - `kody.tags` — search tags +- `kody.dependencies` — direct static saved package dependencies imported via + `kody:@...` - `kody.app` — optional hosted package app config - `kody.jobs` — optional package-owned schedules - `kody.retrievers` — optional package-owned search/context retrievers @@ -68,6 +70,18 @@ The top-level saved identity is the package. - Cross-package imports use the full package name, for example `kody:@scope/my-package/export-name`. +- Static `kody:@...` imports are bundled snapshots. During checks and + publish-time artifact rebuilds, Kody records the imported saved package's + published commit in bundle dependency metadata. A later publish of that + imported package does not rewrite already-published dependent bundles. +- Direct static `kody:@...` imports are a breaking manifest contract: they must + be listed in `package.json#kody.dependencies` by package name, for example + `"dependencies": ["@scope/my-package"]` inside the `kody` object. Repo checks + fail when a static import is missing from the list or when the list contains a + package that is not statically imported. Type-only imports do not count, and + declaration files such as `.d.ts` are treated as type-only. Literal dynamic + `import("kody:@...")` expressions are bundled snapshots too, so they must be + declared. - Callable exports are resolved from package exports, not from a second Kody registry. - Packages may also export non-callable helper modules and values for reuse. @@ -218,6 +232,19 @@ checks and do not mutate D1, KV snapshots, published bundle artifacts, package projections, or vectors. Non-fast-forward external heads are refused unless the caller passes `allow_force: true`. +When publish succeeds, `package_publish_external_push` decorates the response +with `static_dependents`, a bounded summary of direct saved packages whose +published bundle artifact dependency metadata references the package that was +just published. `already_published` responses include the same summary when the +published commit is available. The stale count compares each dependent +artifact's captured dependency commit to the current published commit. + +This summary is visibility only. Do not add automatic fanout republishing to the +publish path. Agents should inspect and republish dependent packages only when +the static snapshot semantics matter for the change. Dynamic runtime invocation +through package execution, where available, resolves the current published +target at invocation time and should not force a dependent package republish. + The scheduled reconcile job in `packages/worker/src/jobs/reconcile-artifacts-pushes.ts` is a safety net for pushed-but-unpublished commits. Every five minutes it scans a small batch of diff --git a/docs/use/packages.md b/docs/use/packages.md index b1bcbcbf4b..2fa9a3af7a 100644 --- a/docs/use/packages.md +++ b/docs/use/packages.md @@ -30,6 +30,8 @@ Important fields: - `kody.id` — user-scoped Kody package id - `kody.description` — package description for search/detail - `kody.tags` — package tags +- `kody.dependencies` — direct saved package names imported through static + `kody:@...` imports - `kody.app` — optional hosted package app config - `kody.services` — optional package-owned service runtimes - `kody.jobs` — optional package-owned schedules @@ -72,6 +74,17 @@ exhaustive. - Cross-package imports use the full package name such as `kody:@scope/my-package/export-name`. +- Static `kody:@...` imports in saved package code are bundled into published + runtime artifacts as snapshots of the imported package's then-published + bundle. If the imported package is published again later, already-published + dependents keep using the older bundled snapshot until they are republished. +- Every direct static `kody:@...` import must be declared in + `package.json#kody.dependencies` using the imported package name, for example + `"dependencies": ["@scope/my-package"]` inside the `kody` object. Package + checks fail when static imports and declarations differ. Type-only imports do + not count, and declaration files such as `.d.ts` are treated as type-only; + literal dynamic `import("kody:@...")` expressions are bundled and must be + declared. - Exports are normal modules. They may expose a default export, named exports, or both. - Direct package invocation calls the resolved module's default export when that @@ -221,6 +234,20 @@ edit it with a normal git client without round-tripping each file change through projections. If the pushed HEAD is already current, the tool returns `already_published`. If checks fail, it returns `checks_failed` with the failed check entries and leaves the underlying storage state unchanged. + Successful `published` responses, and `already_published` responses when the + metadata is available, include a bounded `static_dependents` summary of + direct saved packages whose published bundle artifacts statically reference + this package. Stale entries mean the dependent bundle captured an older + dependency commit. Kody does not automatically republish those dependents; + inspect and republish only the ones whose static snapshot should pick up the + new publish. + +Dynamic package invocation is different from static bundled imports. When a +runtime feature invokes another package dynamically through the package +execution path, it resolves the current published package at invocation time +instead of embedding a source snapshot in the dependent bundle. Dynamic +invocation should not require republishing a dependent package just because the +called package was republished. Choose the narrowest token scope that fits the task. Use `read` for inspection or local diffing, and `write` only when the git client needs to push. Keep TTLs diff --git a/packages/worker/src/mcp/capabilities/packages/publish-external-push.node.test.ts b/packages/worker/src/mcp/capabilities/packages/publish-external-push.node.test.ts index b4b706222f..aad87fd060 100644 --- a/packages/worker/src/mcp/capabilities/packages/publish-external-push.node.test.ts +++ b/packages/worker/src/mcp/capabilities/packages/publish-external-push.node.test.ts @@ -7,6 +7,7 @@ const mockModule = vi.hoisted(() => ({ getEntitySourceById: vi.fn(), resolveArtifactSourceHead: vi.fn(), publishFromExternalRef: vi.fn(), + getStaticPackageDependentsSummary: vi.fn(), })) vi.mock('@sentry/cloudflare', () => ({ @@ -38,6 +39,11 @@ vi.mock('#worker/repo/repo-session-do.ts', () => ({ }), })) +vi.mock('#worker/package-runtime/static-package-dependents.ts', () => ({ + getStaticPackageDependentsSummary: (...args: Array) => + mockModule.getStaticPackageDependentsSummary(...args), +})) + const { publishExternalPushCapability } = await import('./publish-external-push.ts') @@ -66,6 +72,14 @@ beforeEach(() => { created_at: '2026-05-04T00:00:00.000Z', updated_at: '2026-05-04T00:00:00.000Z', }) + mockModule.getStaticPackageDependentsSummary.mockResolvedValue({ + total: 0, + stale: 0, + truncated: false, + items: [], + recommended_next_action: + 'No published bundle artifacts currently declare a static dependency on this package.', + }) }) function createContext() { @@ -104,6 +118,14 @@ test('publishes a new external Artifacts HEAD', async () => { ) expect(result.status).toBe('published') + expect(result).toEqual( + expect.objectContaining({ + static_dependents: expect.objectContaining({ + total: 0, + items: [], + }), + }), + ) expect(mockModule.publishFromExternalRef).toHaveBeenCalledWith( expect.objectContaining({ sourceId: 'source-1', @@ -129,10 +151,91 @@ test('returns already_published when Artifacts HEAD matches D1', async () => { expect(result).toEqual({ status: 'already_published', published_commit: 'commit-old', + static_dependents: { + total: 0, + stale: 0, + truncated: false, + items: [], + recommended_next_action: + 'No published bundle artifacts currently declare a static dependency on this package.', + }, }) expect(mockModule.publishFromExternalRef).not.toHaveBeenCalled() }) +test('published output lists stale static dependents for the new dependency commit', async () => { + mockModule.resolveArtifactSourceHead.mockResolvedValue({ + branch: 'main', + commit: 'commit-new', + }) + mockModule.publishFromExternalRef.mockResolvedValue({ + status: 'published', + previous_commit: 'commit-old', + published_commit: 'commit-new', + manifest: {}, + checks: [{ kind: 'manifest', ok: true, message: 'ok' }], + }) + mockModule.getStaticPackageDependentsSummary.mockResolvedValue({ + total: 1, + stale: 1, + truncated: false, + items: [ + { + package_id: 'package-b', + kody_id: 'package-b', + name: '@kentcdodds/package-b', + source_id: 'source-b', + published_commit: 'commit-b', + stale: true, + artifact_count: 1, + entrypoints: ['src/index.ts'], + entrypoints_truncated: false, + bundled_dependency_commit: 'commit-a-old', + current_dependency_commit: 'commit-new', + recommended_action: + 'Inspect this dependent package and republish it if its bundled static kody:@ snapshot should include the newly published dependency commit.', + }, + ], + recommended_next_action: + 'Inspect stale static dependents and republish only the packages whose bundled snapshot should include this package publish. Kody does not republish dependents automatically.', + }) + + const result = await publishExternalPushCapability.handler( + { package_id: 'package-1' }, + createContext(), + ) + + expect(result).toEqual( + expect.objectContaining({ + status: 'published', + static_dependents: expect.objectContaining({ + total: 1, + stale: 1, + items: [ + expect.objectContaining({ + package_id: 'package-b', + stale: true, + bundled_dependency_commit: 'commit-a-old', + current_dependency_commit: 'commit-new', + }), + ], + }), + }), + ) + expect(mockModule.getStaticPackageDependentsSummary).toHaveBeenCalledWith({ + db: {}, + userId: 'user-1', + sourceId: 'source-1', + currentDependencyCommit: 'commit-new', + }) + expect(publishExternalPushCapability.outputTypeDefinition).toContain( + 'static_dependents', + ) + expect(publishExternalPushCapability.outputTypeDefinition).toContain( + 'current_dependency_commit', + ) +}) + test('surfaces not-fast-forward refusal without force', async () => { mockModule.resolveArtifactSourceHead.mockResolvedValue({ branch: 'main', @@ -289,6 +392,14 @@ test('returns already_published when a retry observes that the reset attempt com expect(result).toEqual({ status: 'already_published', published_commit: 'commit-new', + static_dependents: { + total: 0, + stale: 0, + truncated: false, + items: [], + recommended_next_action: + 'No published bundle artifacts currently declare a static dependency on this package.', + }, }) expect(mockModule.publishFromExternalRef).toHaveBeenCalledTimes(1) } finally { diff --git a/packages/worker/src/mcp/capabilities/packages/publish-external-push.ts b/packages/worker/src/mcp/capabilities/packages/publish-external-push.ts index 66687f13b2..80b5955825 100644 --- a/packages/worker/src/mcp/capabilities/packages/publish-external-push.ts +++ b/packages/worker/src/mcp/capabilities/packages/publish-external-push.ts @@ -4,6 +4,10 @@ import { defineDomainCapability } from '#mcp/capabilities/define-domain-capabili import { capabilityDomainNames } from '#mcp/capabilities/domain-metadata.ts' import { getErrorMessage } from '#mcp/capabilities/error-message.ts' import { requireMcpUser } from '#mcp/capabilities/meta/require-user.ts' +import { + getStaticPackageDependentsSummary, + type StaticPackageDependentsSummary, +} from '#worker/package-runtime/static-package-dependents.ts' import { repoSessionRpc } from '#worker/repo/repo-session-do.ts' import { resolveArtifactSourceHead } from '#worker/repo/artifacts.ts' import { resolveOwnedPackageSource } from './resolve-package-source.ts' @@ -80,10 +84,100 @@ const checkSchema = z.object({ message: z.string(), }) +const staticDependentItemSchema = z + .object({ + package_id: z + .string() + .describe('Saved package id of the dependent package.'), + kody_id: z + .string() + .describe('User-scoped kody.id of the dependent package.'), + name: z + .string() + .describe('Scoped package.json name of the dependent package.'), + source_id: z + .string() + .describe('Repo-backed source id for the dependent package.'), + published_commit: z + .string() + .nullable() + .describe('Current published commit of the dependent package.'), + stale: z + .boolean() + .describe( + 'True when at least one dependent bundle artifact captured an older dependency commit than the just-published package commit.', + ), + artifact_count: z + .number() + .int() + .nonnegative() + .describe( + 'Number of published bundle artifacts that reference this dependency.', + ), + entrypoints: z + .array(z.string()) + .describe( + 'Bounded list of dependent bundle entrypoints referencing this dependency.', + ), + entrypoints_truncated: z + .boolean() + .describe( + 'True when the dependent has more matching entrypoints than returned.', + ), + bundled_dependency_commit: z + .string() + .nullable() + .describe( + 'Dependency commit captured in the dependent bundle, or null when matching artifacts have mixed or missing commits.', + ), + current_dependency_commit: z + .string() + .describe( + 'Current published commit for the package that was just published.', + ), + recommended_action: z + .string() + .describe('Agent guidance for this dependent package.'), + }) + .describe('A direct static dependent package from persisted bundle metadata.') + +const staticDependentsSchema = z + .object({ + total: z + .number() + .int() + .nonnegative() + .describe('Total direct static dependent packages found.'), + stale: z + .number() + .int() + .nonnegative() + .describe( + 'Count of direct static dependent packages with stale bundled snapshots.', + ), + truncated: z + .boolean() + .describe( + 'True when more dependent packages exist than are returned in items.', + ), + items: z + .array(staticDependentItemSchema) + .describe('Bounded direct static dependent package summaries.'), + recommended_next_action: z + .string() + .describe( + 'Agent guidance explaining whether to inspect or republish dependents. Kody does not republish dependents automatically.', + ), + }) + .describe( + 'Bounded summary of saved packages whose published bundles statically captured this package through kody:@ imports.', + ) + const outputSchema = z.discriminatedUnion('status', [ z.object({ status: z.literal('already_published'), published_commit: z.string().nullable(), + static_dependents: staticDependentsSchema, }), z.object({ status: z.literal('not_fast_forward'), @@ -103,15 +197,40 @@ const outputSchema = z.discriminatedUnion('status', [ published_commit: z.string(), manifest: z.unknown(), checks: z.array(checkSchema), + static_dependents: staticDependentsSchema, }), ]) +async function getPublishStaticDependents(input: { + db: D1Database + userId: string + sourceId: string + publishedCommit: string | null +}): Promise { + if (!input.publishedCommit) { + return { + total: 0, + stale: 0, + truncated: false, + items: [], + recommended_next_action: + 'No published commit is available, so static dependent bundle metadata cannot be compared.', + } + } + return await getStaticPackageDependentsSummary({ + db: input.db, + userId: input.userId, + sourceId: input.sourceId, + currentDependencyCommit: input.publishedCommit, + }) +} + export const publishExternalPushCapability = defineDomainCapability( capabilityDomainNames.packages, { name: 'package_publish_external_push', description: - 'Publish the current Artifacts git HEAD for a saved package after a package_get_git_remote clone/edit/push workflow and server-side checks pass.', + 'Publish the current Artifacts git HEAD for a saved package after a package_get_git_remote clone/edit/push workflow and server-side checks pass. Published and already_published responses include bounded static dependent metadata so agents can decide whether stale kody:@ bundled snapshots need inspection or dependent republish; Kody does not republish dependents automatically.', keywords: ['package', 'publish', 'git', 'artifacts', 'external', 'push'], readOnly: false, idempotent: true, @@ -148,6 +267,12 @@ export const publishExternalPushCapability = defineDomainCapability( return { status: 'already_published', published_commit: publishedCommit, + static_dependents: await getPublishStaticDependents({ + db: ctx.env.APP_DB, + userId: user.userId, + sourceId: source.id, + publishedCommit, + }), } as const } const sessionId = @@ -155,7 +280,7 @@ export const publishExternalPushCapability = defineDomainCapability( ? `external-publish-${source.id}` : `external-publish-${source.id}-retry-${attempt}` try { - return await repoSessionRpc( + const result = await repoSessionRpc( ctx.env, sessionId, ).publishFromExternalRef({ @@ -167,6 +292,29 @@ export const publishExternalPushCapability = defineDomainCapability( allowForce: args.allow_force, baseUrl: ctx.callerContext.baseUrl, }) + if (result.status === 'already_published') { + return { + ...result, + static_dependents: await getPublishStaticDependents({ + db: ctx.env.APP_DB, + userId: user.userId, + sourceId: source.id, + publishedCommit: result.published_commit, + }), + } as const + } + if (result.status !== 'published') { + return result + } + return { + ...result, + static_dependents: await getPublishStaticDependents({ + db: ctx.env.APP_DB, + userId: user.userId, + sourceId: source.id, + publishedCommit: result.published_commit, + }), + } as const } catch (error) { if (!isTransientDurableObjectResetError(error)) { throw error diff --git a/packages/worker/src/package-registry/types.ts b/packages/worker/src/package-registry/types.ts index 6c5f4cde6c..808de1815f 100644 --- a/packages/worker/src/package-registry/types.ts +++ b/packages/worker/src/package-registry/types.ts @@ -111,11 +111,41 @@ export const packageExportTargetSchema = z.union([ export type PackageExportTarget = z.infer +const scopedPackageNamePattern = /^@[a-z0-9][a-z0-9._-]*\/[a-z0-9][a-z0-9._-]*$/ + +export const kodyPackageDependencySchema = z + .string() + .min(1) + .regex(scopedPackageNamePattern, { + message: + 'Static Kody package dependencies must be scoped package names like "@scope/package".', + }) + +const kodyPackageDependenciesSchema = z + .array(kodyPackageDependencySchema) + .superRefine((dependencies, ctx) => { + const seen = new Set() + for (const [index, dependency] of dependencies.entries()) { + const normalized = dependency.trim() + if (seen.has(normalized)) { + ctx.addIssue({ + code: 'custom', + path: [index], + message: `Duplicate static Kody package dependency "${normalized}".`, + }) + } + seen.add(normalized) + } + }) + +export type KodyPackageDependency = z.infer + export const authoredPackageKodySchema = z.object({ id: z.string().regex(kodyPackageIdPattern), description: z.string().min(1), tags: z.array(z.string().min(1)).optional(), searchText: z.string().min(1).optional(), + dependencies: kodyPackageDependenciesSchema.optional(), secretMounts: z .record(z.string().min(1), packageSecretMountDefinitionSchema) .optional(), diff --git a/packages/worker/src/package-runtime/import-specifiers.node.test.ts b/packages/worker/src/package-runtime/import-specifiers.node.test.ts new file mode 100644 index 0000000000..45c7a18037 --- /dev/null +++ b/packages/worker/src/package-runtime/import-specifiers.node.test.ts @@ -0,0 +1,25 @@ +import { expect, test } from 'vitest' +import { collectLiteralImportSpecifiers } from './import-specifiers.ts' + +test('collectLiteralImportSpecifiers ignores type-only import and export sources', () => { + const specifiers = collectLiteralImportSpecifiers( + [ + 'import type { Config } from "kody:@kentcdodds/types/config"', + 'import helper from "kody:@kentcdodds/helper/run"', + 'export { type Result } from "kody:@kentcdodds/types/result"', + 'export { run, type RunInput } from "kody:@kentcdodds/runner"', + 'export type { Other } from "kody:@kentcdodds/types/other"', + 'export * from "kody:@kentcdodds/all"', + 'const dynamic = import("kody:@kentcdodds/dynamic")', + 'void dynamic', + 'void helper', + ].join('\n'), + ) + + expect(specifiers).toEqual([ + 'kody:@kentcdodds/helper/run', + 'kody:@kentcdodds/runner', + 'kody:@kentcdodds/all', + 'kody:@kentcdodds/dynamic', + ]) +}) diff --git a/packages/worker/src/package-runtime/import-specifiers.ts b/packages/worker/src/package-runtime/import-specifiers.ts index 045b16da09..404e5d1605 100644 --- a/packages/worker/src/package-runtime/import-specifiers.ts +++ b/packages/worker/src/package-runtime/import-specifiers.ts @@ -39,6 +39,30 @@ function readLiteralStringNode( return null } +function hasOnlyTypeSpecifiers(node: ModuleAstNode) { + const specifiers = (node as { specifiers?: unknown }).specifiers + if (!Array.isArray(specifiers) || specifiers.length === 0) { + return false + } + return specifiers.every( + (specifier) => + specifier && + typeof specifier === 'object' && + ((specifier as { importKind?: unknown }).importKind === 'type' || + (specifier as { exportKind?: unknown }).exportKind === 'type'), + ) +} + +function isTypeOnlyImportOrExport(node: ModuleAstNode) { + if ( + (node as { importKind?: unknown }).importKind === 'type' || + (node as { exportKind?: unknown }).exportKind === 'type' + ) { + return true + } + return hasOnlyTypeSpecifiers(node) +} + export function collectLiteralImportNodes( source: string, ): Array { @@ -59,6 +83,9 @@ export function collectLiteralImportNodes( typedNode.type === 'ExportAllDeclaration' || typedNode.type === 'ExportNamedDeclaration' ) { + if (isTypeOnlyImportOrExport(typedNode)) { + return + } const literalNode = readLiteralStringNode(typedNode.source) if (literalNode) { nodes.push(literalNode) diff --git a/packages/worker/src/package-runtime/module-graph.node.test.ts b/packages/worker/src/package-runtime/module-graph.node.test.ts index 7b6c46c410..e6427717d4 100644 --- a/packages/worker/src/package-runtime/module-graph.node.test.ts +++ b/packages/worker/src/package-runtime/module-graph.node.test.ts @@ -1051,6 +1051,130 @@ test('buildKodyModuleBundle keeps dependencies for scoped packages with the same ]) }) +test('buildKodyModuleBundle records only entrypoint-reachable kody package dependencies', async () => { + mockModule.createWorker.mockResolvedValue( + createBundleResult('reachable-deps'), + ) + mockModule.getSavedPackageByName.mockImplementation( + async ( + _db: unknown, + input: { + name: string + }, + ) => { + if (input.name === '@alice/reachable-package') { + return { + id: 'pkg-reachable', + userId: 'user-1', + name: '@alice/reachable-package', + kodyId: 'reachable-package', + description: 'Reachable package', + tags: [], + searchText: null, + sourceId: 'source-reachable', + hasApp: false, + createdAt: '2026-05-10T00:00:00.000Z', + updatedAt: '2026-05-10T00:00:00.000Z', + } + } + if (input.name === '@bob/unreachable-package') { + return { + id: 'pkg-unreachable', + userId: 'user-1', + name: '@bob/unreachable-package', + kodyId: 'unreachable-package', + description: 'Unreachable package', + tags: [], + searchText: null, + sourceId: 'source-unreachable', + hasApp: false, + createdAt: '2026-05-10T00:00:00.000Z', + updatedAt: '2026-05-10T00:00:00.000Z', + } + } + return null + }, + ) + mockModule.loadPackageSourceBySourceId.mockImplementation( + async (input: { sourceId: string }) => ({ + source: { + id: input.sourceId, + user_id: 'user-1', + entity_kind: 'package', + entity_id: `pkg-${input.sourceId}`, + repo_id: `repo-${input.sourceId}`, + published_commit: `commit-${input.sourceId}`, + indexed_commit: null, + manifest_path: 'package.json', + source_root: '/', + last_external_check_at: null, + created_at: '2026-05-10T00:00:00.000Z', + updated_at: '2026-05-10T00:00:00.000Z', + }, + manifest: { + name: + input.sourceId === 'source-reachable' + ? '@alice/reachable-package' + : '@bob/unreachable-package', + exports: { + '.': './src/index.ts', + }, + kody: { + id: + input.sourceId === 'source-reachable' + ? 'reachable-package' + : 'unreachable-package', + description: 'Dependency package', + }, + }, + files: { + 'package.json': '{}', + 'src/index.ts': 'export default async function run() { return "ok" }', + }, + }), + ) + + const { buildKodyModuleBundle } = await import('./module-graph.ts') + + const result = await buildKodyModuleBundle({ + env: { + APP_DB: {}, + REPO_SESSION: {}, + } as Env, + baseUrl: 'https://heykody.dev', + userId: 'user-1', + sourceFiles: { + 'package.json': JSON.stringify({ + name: '@kentcdodds/local-package', + exports: { + '.': './src/index.ts', + './unused': './src/unused.ts', + }, + kody: { + id: 'local-package', + description: 'Local package', + }, + }), + 'src/index.ts': + 'import "./reachable"; export default async function run() { return "ok" }', + 'src/reachable.ts': + 'import reachable from "kody:@alice/reachable-package"; export { reachable }', + 'src/unused.ts': + 'import unreachable from "kody:@bob/unreachable-package"; export { unreachable }', + }, + entryPoint: 'src/index.ts', + }) + + expect(result.dependencies).toEqual([ + { + sourceId: 'source-reachable', + publishedCommit: 'commit-source-reachable', + kodyId: 'reachable-package', + packageName: '@alice/reachable-package', + }, + ]) +}) + test('buildKodyModuleBundle keeps virtual package paths distinct for scoped packages with the same leaf', async () => { mockModule.createWorker.mockResolvedValue( createBundleResult('shared-leaf-prefix'), diff --git a/packages/worker/src/package-runtime/module-graph.ts b/packages/worker/src/package-runtime/module-graph.ts index f6e2fa5771..a672116f7b 100644 --- a/packages/worker/src/package-runtime/module-graph.ts +++ b/packages/worker/src/package-runtime/module-graph.ts @@ -29,6 +29,10 @@ import { } from './package-import-resolution.ts' import { loadPublishedBundleArtifactByIdentity } from './published-bundle-artifacts.ts' import { assertPublishedSourceCanRebuildWithoutInstallingDeps } from './published-source-dependencies.ts' +import { + collectStaticKodyPackageImportsFromFiles, + isTypeDeclarationFilePath, +} from './static-kody-imports.ts' import { collectLiteralImportNodes, collectLiteralImportSpecifiers, @@ -256,6 +260,153 @@ function isBundlerRootDependencyPath(path: string) { return path === 'node_modules' || path.startsWith('node_modules/') } +function normalizeWorkspaceModulePath(path: string) { + const parts: Array = [] + for (const segment of path.replace(/\\/g, '/').split('/')) { + if (!segment || segment === '.') continue + if (segment === '..') { + parts.pop() + continue + } + parts.push(segment) + } + return parts.join('/') +} + +function resolveLocalImportPath(input: { + files: Record + fromPath: string + specifier: string +}) { + if (!input.specifier.startsWith('./') && !input.specifier.startsWith('../')) { + return null + } + const basePath = normalizeWorkspaceModulePath( + joinPath(dirname(input.fromPath), input.specifier), + ) + const candidates = [ + basePath, + `${basePath}.ts`, + `${basePath}.tsx`, + `${basePath}.js`, + `${basePath}.jsx`, + `${basePath}.mts`, + `${basePath}.cts`, + joinPath(basePath, 'index.ts'), + joinPath(basePath, 'index.tsx'), + joinPath(basePath, 'index.js'), + joinPath(basePath, 'index.jsx'), + joinPath(basePath, 'index.mts'), + joinPath(basePath, 'index.cts'), + ] + return candidates.find((candidate) => input.files[candidate] != null) ?? null +} + +function collectReachableSourceFilePaths(input: { + files: Record + entryPoint: string +}) { + const reachable = new Set() + const stack = [normalizePackageWorkspacePath(input.entryPoint)] + while (stack.length > 0) { + const filePath = stack.pop() + if ( + !filePath || + reachable.has(filePath) || + isTypeDeclarationFilePath(filePath) + ) { + continue + } + const source = input.files[filePath] + if (source == null) continue + reachable.add(filePath) + for (const node of collectLiteralImportNodes(source)) { + const localPath = resolveLocalImportPath({ + files: input.files, + fromPath: filePath, + specifier: node.specifier, + }) + if (localPath && !reachable.has(localPath)) { + stack.push(localPath) + } + } + } + return reachable +} + +async function resolveDirectKodyDependenciesForEntryPoint(input: { + env: Env + baseUrl: string + userId: string + sourceFiles: Record + entryPoint: string + loadedPackages?: Map< + string, + LoadedPackageSource & { row: SavedPackageRecord; prefix: string } + > +}) { + const rootPackage = readRootPackage(input.sourceFiles) + const reachable = collectReachableSourceFilePaths({ + files: input.sourceFiles, + entryPoint: input.entryPoint, + }) + const reachableFiles = Object.fromEntries( + Object.entries(input.sourceFiles).filter(([filePath]) => + reachable.has(filePath), + ), + ) + const importedPackages = new Map() + for (const imported of collectStaticKodyPackageImportsFromFiles( + reachableFiles, + )) { + if (imported.packageName === rootPackage?.manifest.name) continue + importedPackages.set(imported.packageName, imported.specifier) + } + const dependencies: Array = [] + for (const specifier of [...importedPackages.values()].sort((left, right) => + left.localeCompare(right), + )) { + const parsed = parseKodyPackageSpecifier(specifier) + const cached = input.loadedPackages?.get(parsed.packageName) + const row = + cached?.row ?? + (await resolveSavedPackageImport({ + db: input.env.APP_DB, + userId: input.userId, + specifier: parsed, + })) + if (!row) { + throw new Error( + `Saved package "${parsed.packageName}" was not found for this user.`, + ) + } + const loaded = + cached ?? + (await loadPackageSourceBySourceId({ + env: input.env, + baseUrl: input.baseUrl, + userId: input.userId, + sourceId: row.sourceId, + })) + if (!loaded.source.published_commit) { + throw new Error( + `Saved package "${row.name}" source "${row.sourceId}" has no published commit.`, + ) + } + dependencies.push({ + sourceId: loaded.source.id, + publishedCommit: loaded.source.published_commit, + kodyId: row.kodyId, + packageName: row.name, + }) + } + return dependencies.sort( + (left, right) => + left.kodyId.localeCompare(right.kodyId) || + left.sourceId.localeCompare(right.sourceId), + ) +} + function* iterateModuleSourceTexts( modules: WorkerLoaderModules, ): Generator<[modulePath: string, source: string]> { @@ -440,6 +591,10 @@ async function ensurePackageLoaded( for (const [filePath, content] of Object.entries(loaded.files)) { const normalizedPath = normalizePackageWorkspacePath(filePath) const targetPath = joinPath(entry.prefix, normalizedPath) + if (isTypeDeclarationFilePath(normalizedPath)) { + state.files[targetPath] = content + continue + } state.files[targetPath] = await rewriteKodyImports({ state, source: content, @@ -544,7 +699,7 @@ export async function buildKodyModuleBundle(input: { sourceFiles: Record entryPoint: string }) { - const { files, dependencies } = await prepareKodyGraphFiles({ + const { files, packages } = await prepareKodyGraphFiles({ env: input.env, baseUrl: input.baseUrl, userId: input.userId, @@ -572,7 +727,10 @@ export async function buildKodyModuleBundle(input: { return { mainModule: bundle.mainModule, modules: bundle.modules as WorkerLoaderModules, - dependencies: [...dependencies.values()], + dependencies: await resolveDirectKodyDependenciesForEntryPoint({ + ...input, + loadedPackages: packages, + }), } } @@ -583,7 +741,7 @@ export async function buildKodyImportableModuleBundle(input: { sourceFiles: Record entryPoint: string }) { - const { files, dependencies } = await prepareKodyGraphFiles({ + const { files, packages } = await prepareKodyGraphFiles({ env: input.env, baseUrl: input.baseUrl, userId: input.userId, @@ -611,7 +769,10 @@ export async function buildKodyImportableModuleBundle(input: { return { mainModule: bundle.mainModule, modules: bundle.modules as WorkerLoaderModules, - dependencies: [...dependencies.values()], + dependencies: await resolveDirectKodyDependenciesForEntryPoint({ + ...input, + loadedPackages: packages, + }), } } @@ -646,6 +807,10 @@ async function prepareKodyGraphFiles(input: { continue } const normalizedPath = joinPath(rootSourcePrefix, normalizedSourcePath) + if (isTypeDeclarationFilePath(normalizedSourcePath)) { + files[normalizedPath] = content + continue + } if (normalizedSourcePath === packageManifestPath) { files[normalizedPath] = content continue @@ -658,6 +823,7 @@ async function prepareKodyGraphFiles(input: { } return { files, + packages: state.packages, dependencies: state.dependencies, } } @@ -671,7 +837,7 @@ export async function buildKodyAppBundle(input: { cacheKey?: string | null }) { const buildBundle = async () => { - const { files, dependencies } = await prepareKodyGraphFiles({ + const { files, packages } = await prepareKodyGraphFiles({ env: input.env, baseUrl: input.baseUrl, userId: input.userId, @@ -699,7 +865,10 @@ export async function buildKodyAppBundle(input: { return { mainModule: bundle.mainModule, modules: bundle.modules as WorkerLoaderModules, - dependencies: [...dependencies.values()], + dependencies: await resolveDirectKodyDependenciesForEntryPoint({ + ...input, + loadedPackages: packages, + }), } } diff --git a/packages/worker/src/package-runtime/published-bundle-artifacts.ts b/packages/worker/src/package-runtime/published-bundle-artifacts.ts index 6389d536d2..899cc21173 100644 --- a/packages/worker/src/package-runtime/published-bundle-artifacts.ts +++ b/packages/worker/src/package-runtime/published-bundle-artifacts.ts @@ -33,6 +33,7 @@ import { parseKodyPackageSpecifier, resolveSavedPackageImport, } from './package-import-resolution.ts' +import { collectStaticKodyPackageImportsFromFiles } from './static-kody-imports.ts' import { buildPackageSubscriptionArtifactName } from './subscription-artifacts.ts' type DependencyResolutionState = { @@ -105,22 +106,19 @@ async function collectDependenciesFromFiles(input: { userId: string files: Record }) { - const packageSpecifierPattern = /['"](kody:@[^'"]+)['"]/g const state: DependencyResolutionState = { env: input.env, userId: input.userId, visited: new Set(), dependencies: [], } - for (const content of Object.values(input.files)) { - for (const match of content.matchAll(packageSpecifierPattern)) { - const specifier = match[1]?.trim() - if (!specifier) continue - await resolveDependencyForPackage({ - state, - specifier, - }) - } + for (const imported of collectStaticKodyPackageImportsFromFiles( + input.files, + )) { + await resolveDependencyForPackage({ + state, + specifier: imported.specifier, + }) } return state.dependencies.sort( (left, right) => diff --git a/packages/worker/src/package-runtime/static-kody-imports.ts b/packages/worker/src/package-runtime/static-kody-imports.ts new file mode 100644 index 0000000000..ca37db8d01 --- /dev/null +++ b/packages/worker/src/package-runtime/static-kody-imports.ts @@ -0,0 +1,53 @@ +import { + parseKodyPackageSpecifier, + packageSpecifierPrefix, +} from './package-import-resolution.ts' +import { collectLiteralImportNodes } from './import-specifiers.ts' + +export type StaticKodyPackageImport = { + filePath: string + specifier: string + packageName: string + exportName: string +} + +export function isTypeDeclarationFilePath(filePath: string) { + return /\.d\.[cm]?ts$/.test(filePath) +} + +function collectStaticKodyPackageImportsFromSource(input: { + filePath: string + source: string +}): Array { + const imports: Array = [] + for (const node of collectLiteralImportNodes(input.source)) { + if (!node.specifier.startsWith(packageSpecifierPrefix)) continue + const parsedSpecifier = parseKodyPackageSpecifier(node.specifier) + imports.push({ + filePath: input.filePath, + specifier: node.specifier, + packageName: parsedSpecifier.packageName, + exportName: parsedSpecifier.exportName, + }) + } + return imports +} + +export function collectStaticKodyPackageImportsFromFiles( + files: Record, +): Array { + return Object.entries(files) + .filter(([filePath]) => !isTypeDeclarationFilePath(filePath)) + .flatMap(([filePath, source]) => + collectStaticKodyPackageImportsFromSource({ + filePath, + source, + }), + ) + .sort( + (left, right) => + left.packageName.localeCompare(right.packageName) || + left.exportName.localeCompare(right.exportName) || + left.filePath.localeCompare(right.filePath), + ) +} diff --git a/packages/worker/src/package-runtime/static-package-dependents.node.test.ts b/packages/worker/src/package-runtime/static-package-dependents.node.test.ts new file mode 100644 index 0000000000..e3a7b06dc5 --- /dev/null +++ b/packages/worker/src/package-runtime/static-package-dependents.node.test.ts @@ -0,0 +1,264 @@ +import { expect, test } from 'vitest' +import { buildStaticPackageDependentsSummary } from './static-package-dependents.ts' + +test('buildStaticPackageDependentsSummary marks a static dependent stale after dependency republish', () => { + const summary = buildStaticPackageDependentsSummary({ + total: 1, + stale: 1, + currentDependencyCommit: 'commit-a-new', + rows: [ + { + packageId: 'package-b', + packageKodyId: 'package-b', + packageName: '@kentcdodds/package-b', + sourceId: 'source-b', + publishedCommit: 'commit-b', + artifactKind: 'module', + artifactName: '.', + entryPoint: 'src/index.ts', + packageStale: true, + matchingArtifactCount: 1, + matchingEntrypointCount: 1, + packageBundledDependencyCommit: 'commit-a-old', + bundledDependencyCommit: 'commit-a-old', + }, + ], + }) + + expect(summary).toEqual({ + total: 1, + stale: 1, + truncated: false, + items: [ + { + package_id: 'package-b', + kody_id: 'package-b', + name: '@kentcdodds/package-b', + source_id: 'source-b', + published_commit: 'commit-b', + stale: true, + artifact_count: 1, + entrypoints: ['src/index.ts'], + entrypoints_truncated: false, + bundled_dependency_commit: 'commit-a-old', + current_dependency_commit: 'commit-a-new', + recommended_action: + 'Inspect this dependent package and republish it if its bundled static kody:@ snapshot should include the newly published dependency commit.', + }, + ], + recommended_next_action: + 'Inspect stale static dependents and republish only the packages whose bundled snapshot should include this package publish. Kody does not republish dependents automatically.', + }) +}) + +test('buildStaticPackageDependentsSummary returns an empty summary with no dependents', () => { + const summary = buildStaticPackageDependentsSummary({ + total: 0, + stale: 0, + currentDependencyCommit: 'commit-a', + rows: [], + }) + + expect(summary).toEqual({ + total: 0, + stale: 0, + truncated: false, + items: [], + recommended_next_action: + 'No published bundle artifacts currently declare a static dependency on this package.', + }) +}) + +test('buildStaticPackageDependentsSummary bounds returned dependents and entrypoints', () => { + const summary = buildStaticPackageDependentsSummary({ + total: 2, + stale: 2, + currentDependencyCommit: 'commit-a-new', + packageLimit: 1, + artifactsPerPackageLimit: 1, + rows: [ + { + packageId: 'package-b', + packageKodyId: 'package-b', + packageName: '@kentcdodds/package-b', + sourceId: 'source-b', + publishedCommit: 'commit-b', + artifactKind: 'module', + artifactName: '.', + entryPoint: 'src/index.ts', + packageStale: true, + matchingArtifactCount: 2, + matchingEntrypointCount: 2, + packageBundledDependencyCommit: 'commit-a-old', + bundledDependencyCommit: 'commit-a-old', + }, + { + packageId: 'package-b', + packageKodyId: 'package-b', + packageName: '@kentcdodds/package-b', + sourceId: 'source-b', + publishedCommit: 'commit-b', + artifactKind: 'job', + artifactName: 'nightly', + entryPoint: 'src/nightly.ts', + packageStale: true, + matchingArtifactCount: 2, + matchingEntrypointCount: 2, + packageBundledDependencyCommit: 'commit-a-old', + bundledDependencyCommit: 'commit-a-old', + }, + { + packageId: 'package-c', + packageKodyId: 'package-c', + packageName: '@kentcdodds/package-c', + sourceId: 'source-c', + publishedCommit: 'commit-c', + artifactKind: 'module', + artifactName: '.', + entryPoint: 'src/index.ts', + packageStale: true, + matchingArtifactCount: 1, + matchingEntrypointCount: 1, + packageBundledDependencyCommit: 'commit-a-old', + bundledDependencyCommit: 'commit-a-old', + }, + ], + }) + + expect(summary.truncated).toBe(true) + expect(summary.items).toHaveLength(1) + expect(summary.items[0]).toEqual( + expect.objectContaining({ + package_id: 'package-b', + artifact_count: 2, + entrypoints: ['src/index.ts'], + entrypoints_truncated: true, + }), + ) +}) + +test('buildStaticPackageDependentsSummary keeps stale true when stale artifact is not returned', () => { + const summary = buildStaticPackageDependentsSummary({ + total: 1, + stale: 1, + currentDependencyCommit: 'commit-a-new', + artifactsPerPackageLimit: 1, + rows: [ + { + packageId: 'package-b', + packageKodyId: 'package-b', + packageName: '@kentcdodds/package-b', + sourceId: 'source-b', + publishedCommit: 'commit-b', + artifactKind: 'module', + artifactName: '.', + entryPoint: 'src/index.ts', + packageStale: true, + matchingArtifactCount: 6, + matchingEntrypointCount: 6, + packageBundledDependencyCommit: null, + bundledDependencyCommit: 'commit-a-new', + }, + ], + }) + + expect(summary.items[0]).toEqual( + expect.objectContaining({ + stale: true, + artifact_count: 6, + entrypoints_truncated: true, + bundled_dependency_commit: null, + }), + ) +}) + +test('buildStaticPackageDependentsSummary reports null commit for mixed missing dependency commits', () => { + const summary = buildStaticPackageDependentsSummary({ + total: 1, + stale: 1, + currentDependencyCommit: 'commit-a-new', + rows: [ + { + packageId: 'package-b', + packageKodyId: 'package-b', + packageName: '@kentcdodds/package-b', + sourceId: 'source-b', + publishedCommit: 'commit-b', + artifactKind: 'module', + artifactName: '.', + entryPoint: 'src/index.ts', + packageStale: true, + matchingArtifactCount: 2, + matchingEntrypointCount: 2, + packageBundledDependencyCommit: null, + bundledDependencyCommit: 'commit-a-new', + }, + { + packageId: 'package-b', + packageKodyId: 'package-b', + packageName: '@kentcdodds/package-b', + sourceId: 'source-b', + publishedCommit: 'commit-b', + artifactKind: 'job', + artifactName: 'nightly', + entryPoint: 'src/nightly.ts', + packageStale: true, + matchingArtifactCount: 2, + matchingEntrypointCount: 2, + packageBundledDependencyCommit: null, + bundledDependencyCommit: null, + }, + ], + }) + + expect(summary.items[0]?.bundled_dependency_commit).toBeNull() +}) + +test('buildStaticPackageDependentsSummary truncates by distinct entrypoints, not artifact rows', () => { + const summary = buildStaticPackageDependentsSummary({ + total: 1, + stale: 0, + currentDependencyCommit: 'commit-a-new', + artifactsPerPackageLimit: 1, + rows: [ + { + packageId: 'package-b', + packageKodyId: 'package-b', + packageName: '@kentcdodds/package-b', + sourceId: 'source-b', + publishedCommit: 'commit-b', + artifactKind: 'module', + artifactName: '.', + entryPoint: 'src/index.ts', + packageStale: false, + matchingArtifactCount: 2, + matchingEntrypointCount: 1, + packageBundledDependencyCommit: 'commit-a-new', + bundledDependencyCommit: 'commit-a-new', + }, + { + packageId: 'package-b', + packageKodyId: 'package-b', + packageName: '@kentcdodds/package-b', + sourceId: 'source-b', + publishedCommit: 'commit-b', + artifactKind: 'importable-module', + artifactName: '.', + entryPoint: 'src/index.ts', + packageStale: false, + matchingArtifactCount: 2, + matchingEntrypointCount: 1, + packageBundledDependencyCommit: 'commit-a-new', + bundledDependencyCommit: 'commit-a-new', + }, + ], + }) + + expect(summary.items[0]).toEqual( + expect.objectContaining({ + artifact_count: 2, + entrypoints: ['src/index.ts'], + entrypoints_truncated: false, + }), + ) +}) diff --git a/packages/worker/src/package-runtime/static-package-dependents.ts b/packages/worker/src/package-runtime/static-package-dependents.ts new file mode 100644 index 0000000000..3d86ee6be9 --- /dev/null +++ b/packages/worker/src/package-runtime/static-package-dependents.ts @@ -0,0 +1,208 @@ +import { + countStaticDependentBundleArtifactPackages, + listStaticDependentBundleArtifactRows, + type StaticDependentBundleArtifactRow, +} from '#worker/repo/published-bundle-artifacts-repo.ts' + +export const defaultStaticDependentPackageLimit = 10 +const defaultStaticDependentArtifactsPerPackageLimit = 5 + +/** + * One saved package whose published bundle artifacts statically captured another + * saved package through a `kody:@scope/package` import. + */ +export type StaticDependentPackageSummaryItem = { + package_id: string + kody_id: string + name: string + source_id: string + published_commit: string | null + stale: boolean + artifact_count: number + entrypoints: Array + entrypoints_truncated: boolean + bundled_dependency_commit: string | null + current_dependency_commit: string + recommended_action: string +} + +/** + * Bounded publish-result summary of direct static dependents. Kody exposes this + * so agents can decide whether to inspect and republish dependent packages; it + * deliberately does not fan out or republish anything automatically. + */ +export type StaticPackageDependentsSummary = { + total: number + stale: number + truncated: boolean + items: Array + recommended_next_action: string +} + +type BuildStaticPackageDependentsSummaryInput = { + total: number + stale: number + rows: Array + currentDependencyCommit: string + packageLimit?: number + artifactsPerPackageLimit?: number +} + +type StaticDependentPackageAccumulator = { + package_id: string + kody_id: string + name: string + source_id: string + published_commit: string | null + packageStale: boolean + matchingArtifactCount: number + matchingEntrypointCount: number + artifactRows: Array + entrypoints: Array + packageBundledDependencyCommit: string | null +} + +function createRecommendedNextAction(input: { total: number; stale: number }) { + if (input.total === 0) { + return 'No published bundle artifacts currently declare a static dependency on this package.' + } + if (input.stale === 0) { + return 'Static dependents already reference the current published dependency commit; dependent republish is not indicated by bundle metadata.' + } + return 'Inspect stale static dependents and republish only the packages whose bundled snapshot should include this package publish. Kody does not republish dependents automatically.' +} + +function createRecommendedItemAction(input: { stale: boolean }) { + if (!input.stale) { + return 'No action indicated by bundle metadata; this dependent already references the current dependency commit.' + } + return 'Inspect this dependent package and republish it if its bundled static kody:@ snapshot should include the newly published dependency commit.' +} + +function createAccumulator( + row: StaticDependentBundleArtifactRow, +): StaticDependentPackageAccumulator { + return { + package_id: row.packageId, + kody_id: row.packageKodyId, + name: row.packageName, + source_id: row.sourceId, + published_commit: row.publishedCommit, + packageStale: row.packageStale, + matchingArtifactCount: row.matchingArtifactCount, + matchingEntrypointCount: row.matchingEntrypointCount, + artifactRows: [], + entrypoints: [], + packageBundledDependencyCommit: row.packageBundledDependencyCommit, + } +} + +function addRowToAccumulator(input: { + accumulator: StaticDependentPackageAccumulator + row: StaticDependentBundleArtifactRow +}) { + input.accumulator.artifactRows.push(input.row) + input.accumulator.matchingArtifactCount = Math.max( + input.accumulator.matchingArtifactCount, + input.row.matchingArtifactCount, + ) + input.accumulator.matchingEntrypointCount = Math.max( + input.accumulator.matchingEntrypointCount, + input.row.matchingEntrypointCount, + ) + if (!input.accumulator.entrypoints.includes(input.row.entryPoint)) { + input.accumulator.entrypoints.push(input.row.entryPoint) + } + input.accumulator.packageStale = + input.accumulator.packageStale || input.row.packageStale + input.accumulator.packageBundledDependencyCommit = + input.accumulator.packageBundledDependencyCommit === + input.row.packageBundledDependencyCommit + ? input.accumulator.packageBundledDependencyCommit + : null +} + +export function buildStaticPackageDependentsSummary( + input: BuildStaticPackageDependentsSummaryInput, +): StaticPackageDependentsSummary { + const packageLimit = input.packageLimit ?? defaultStaticDependentPackageLimit + const artifactsPerPackageLimit = + input.artifactsPerPackageLimit ?? + defaultStaticDependentArtifactsPerPackageLimit + const packages = new Map() + for (const row of input.rows) { + const accumulator = packages.get(row.packageId) ?? createAccumulator(row) + packages.set(row.packageId, accumulator) + addRowToAccumulator({ + accumulator, + row, + }) + } + const items = [...packages.values()].slice(0, packageLimit).map( + (item) => + ({ + package_id: item.package_id, + kody_id: item.kody_id, + name: item.name, + source_id: item.source_id, + published_commit: item.published_commit, + stale: item.packageStale, + artifact_count: item.matchingArtifactCount, + entrypoints: item.entrypoints.slice(0, artifactsPerPackageLimit), + entrypoints_truncated: + item.matchingEntrypointCount > + Math.min(item.entrypoints.length, artifactsPerPackageLimit), + bundled_dependency_commit: item.packageBundledDependencyCommit, + current_dependency_commit: input.currentDependencyCommit, + recommended_action: createRecommendedItemAction({ + stale: item.packageStale, + }), + }) satisfies StaticDependentPackageSummaryItem, + ) + return { + total: input.total, + stale: input.stale, + truncated: input.total > items.length, + items, + recommended_next_action: createRecommendedNextAction({ + total: input.total, + stale: input.stale, + }), + } +} + +export async function getStaticPackageDependentsSummary(input: { + db: D1Database + userId: string + sourceId: string + currentDependencyCommit: string + packageLimit?: number + artifactsPerPackageLimit?: number +}): Promise { + const packageLimit = input.packageLimit ?? defaultStaticDependentPackageLimit + const artifactsPerPackageLimit = + input.artifactsPerPackageLimit ?? + defaultStaticDependentArtifactsPerPackageLimit + const [counts, rows] = await Promise.all([ + countStaticDependentBundleArtifactPackages(input.db, { + userId: input.userId, + sourceId: input.sourceId, + currentDependencyCommit: input.currentDependencyCommit, + }), + listStaticDependentBundleArtifactRows(input.db, { + userId: input.userId, + sourceId: input.sourceId, + currentDependencyCommit: input.currentDependencyCommit, + packageLimit, + artifactsPerPackageLimit, + }), + ]) + return buildStaticPackageDependentsSummary({ + total: counts.totalPackages, + stale: counts.stalePackages, + rows, + currentDependencyCommit: input.currentDependencyCommit, + packageLimit, + artifactsPerPackageLimit, + }) +} diff --git a/packages/worker/src/repo/checks.node.test.ts b/packages/worker/src/repo/checks.node.test.ts index 9444abff92..780400fe41 100644 --- a/packages/worker/src/repo/checks.node.test.ts +++ b/packages/worker/src/repo/checks.node.test.ts @@ -88,13 +88,17 @@ function createPackageManifest(input: { packageName: string kodyId: string description: string - exports?: Record + exports?: Record< + string, + string | { import?: string; default?: string; types?: string } + > jobs?: Record }> subscriptions?: Record< string, { handler: string; description?: string; filters?: Record } > services?: Record + kodyDependencies?: Array retrievers?: Record< string, { @@ -116,6 +120,7 @@ function createPackageManifest(input: { kody: { id: input.kodyId, description: input.description, + dependencies: input.kodyDependencies, app: input.appEntry ? { entry: input.appEntry, @@ -409,7 +414,8 @@ test('runRepoChecks accepts execute runtime globals for package-owned jobs', asy expect.objectContaining({ kind: 'dependencies', ok: true, - message: 'package.json declares no npm dependencies.', + message: + 'package.json declares no npm dependencies. package.json#kody.dependencies declares no static Kody package dependencies.', }), expect.objectContaining({ kind: 'typecheck', @@ -1048,7 +1054,325 @@ test('runRepoChecks reports declared npm dependencies in package.json', async () expect.objectContaining({ kind: 'dependencies', ok: true, - message: 'package.json declares 1 npm dependency: "kleur".', + message: + 'package.json declares 1 npm dependency: "kleur". package.json#kody.dependencies declares no static Kody package dependencies.', + }), + ]), + ) +}) + +test('runRepoChecks requires static kody package imports to be declared', async () => { + const files = new Map([ + [ + 'package.json', + createPackageManifest({ + packageName: '@kody/uses-static-package', + kodyId: 'uses-static-package', + description: 'Uses a static Kody package import', + }), + ], + [ + 'src/index.ts', + 'import helper from "kody:@kentcdodds/helper/run"\nexport const ready = helper\n', + ], + ]) + const snapshot = createSnapshotFromFiles(files) + mockModule.createFileSystemSnapshot.mockResolvedValue(snapshot) + + const result = await runRepoChecks({ + workspace: { + async readFile(path: string) { + return files.get(path) ?? null + }, + async glob() { + return Array.from(files.keys()).map((path) => ({ path, type: 'file' })) + }, + }, + manifestPath: 'package.json', + sourceRoot: '/', + }) + + expect(result.ok).toBe(false) + expect(result.results).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + kind: 'dependencies', + ok: false, + message: expect.stringContaining( + 'package.json#kody.dependencies must match direct static kody:@ imports (missing "@kentcdodds/helper").', + ), + }), + ]), + ) +}) + +test('runRepoChecks accepts declared static kody package imports and ignores type-only imports', async () => { + const files = new Map([ + [ + 'package.json', + createPackageManifest({ + packageName: '@kody/declared-static-package', + kodyId: 'declared-static-package', + description: 'Declares static Kody package imports', + kodyDependencies: ['@kentcdodds/helper'], + }), + ], + [ + 'src/index.ts', + [ + 'import helper from "kody:@kentcdodds/helper/run"', + 'import type { HelperConfig } from "kody:@kentcdodds/types/config"', + 'export { type HelperResult } from "kody:@kentcdodds/types/result"', + 'export const ready: HelperConfig | unknown = helper', + ].join('\n'), + ], + ]) + const snapshot = createSnapshotFromFiles(files) + mockModule.createFileSystemSnapshot.mockResolvedValue(snapshot) + + const result = await runRepoChecks({ + workspace: { + async readFile(path: string) { + return files.get(path) ?? null + }, + async glob() { + return Array.from(files.keys()).map((path) => ({ path, type: 'file' })) + }, + }, + manifestPath: 'package.json', + sourceRoot: '/', + }) + + expect(result.ok).toBe(true) + expect(result.results).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + kind: 'dependencies', + ok: true, + message: expect.stringContaining( + 'package.json#kody.dependencies declares 1 static Kody package dependency: "@kentcdodds/helper".', + ), + }), + ]), + ) +}) + +test('runRepoChecks ignores kody imports in declaration files', async () => { + const files = new Map([ + [ + 'package.json', + createPackageManifest({ + packageName: '@kody/declaration-file-types', + kodyId: 'declaration-file-types', + description: 'Exports declaration-only types', + exports: { + '.': { + import: './src/index.ts', + types: './src/index.d.ts', + }, + }, + }), + ], + ['src/index.ts', 'export const ready = true\n'], + [ + 'src/index.d.ts', + 'import { HelperConfig } from "kody:@kentcdodds/types/config"\nexport type Options = HelperConfig\n', + ], + ]) + const snapshot = createSnapshotFromFiles(files) + mockModule.createFileSystemSnapshot.mockResolvedValue(snapshot) + + const result = await runRepoChecks({ + workspace: { + async readFile(path: string) { + return files.get(path) ?? null + }, + async glob() { + return Array.from(files.keys()).map((path) => ({ path, type: 'file' })) + }, + }, + manifestPath: 'package.json', + sourceRoot: '/', + }) + + expect(result.ok).toBe(true) + expect(result.results).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + kind: 'dependencies', + ok: true, + message: expect.stringContaining( + 'package.json#kody.dependencies declares no static Kody package dependencies.', + ), + }), + ]), + ) +}) + +test('runRepoChecks requires mixed value export-from kody package imports to be declared', async () => { + const files = new Map([ + [ + 'package.json', + createPackageManifest({ + packageName: '@kody/mixed-export-package', + kodyId: 'mixed-export-package', + description: 'Exports a value from another Kody package', + }), + ], + [ + 'src/index.ts', + 'export { run, type RunInput } from "kody:@kentcdodds/runner"\n', + ], + ]) + const snapshot = createSnapshotFromFiles(files) + mockModule.createFileSystemSnapshot.mockResolvedValue(snapshot) + + const result = await runRepoChecks({ + workspace: { + async readFile(path: string) { + return files.get(path) ?? null + }, + async glob() { + return Array.from(files.keys()).map((path) => ({ path, type: 'file' })) + }, + }, + manifestPath: 'package.json', + sourceRoot: '/', + }) + + expect(result.ok).toBe(false) + expect(result.results).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + kind: 'dependencies', + ok: false, + message: expect.stringContaining('missing "@kentcdodds/runner"'), + }), + ]), + ) +}) + +test('runRepoChecks requires literal dynamic kody package imports to be declared', async () => { + const files = new Map([ + [ + 'package.json', + createPackageManifest({ + packageName: '@kody/dynamic-import-package', + kodyId: 'dynamic-import-package', + description: 'Dynamically imports another Kody package', + }), + ], + [ + 'src/index.ts', + 'export async function load() { return await import("kody:@kentcdodds/dynamic/run") }\n', + ], + ]) + const snapshot = createSnapshotFromFiles(files) + mockModule.createFileSystemSnapshot.mockResolvedValue(snapshot) + + const result = await runRepoChecks({ + workspace: { + async readFile(path: string) { + return files.get(path) ?? null + }, + async glob() { + return Array.from(files.keys()).map((path) => ({ path, type: 'file' })) + }, + }, + manifestPath: 'package.json', + sourceRoot: '/', + }) + + expect(result.ok).toBe(false) + expect(result.results).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + kind: 'dependencies', + ok: false, + message: expect.stringContaining('missing "@kentcdodds/dynamic"'), + }), + ]), + ) +}) + +test('runRepoChecks rejects invalid static kody package dependency declarations', async () => { + const files = new Map([ + [ + 'package.json', + JSON.stringify({ + name: '@kody/invalid-dependency-declaration', + exports: { + '.': './src/index.ts', + }, + kody: { + id: 'invalid-dependency-declaration', + description: 'Declares an invalid Kody dependency', + dependencies: ['@kentcdodds/helper/run'], + }, + }), + ], + ['src/index.ts', 'export const ready = true\n'], + ]) + + await expect( + runRepoChecks({ + workspace: { + async readFile(path: string) { + return files.get(path) ?? null + }, + async glob() { + return Array.from(files.keys()).map((path) => ({ + path, + type: 'file', + })) + }, + }, + manifestPath: 'package.json', + sourceRoot: '/', + }), + ).rejects.toThrow( + 'Static Kody package dependencies must be scoped package names like "@scope/package".', + ) +}) + +test('runRepoChecks rejects unused static kody package dependency declarations', async () => { + const files = new Map([ + [ + 'package.json', + createPackageManifest({ + packageName: '@kody/unused-static-package', + kodyId: 'unused-static-package', + description: 'Declares an unused Kody package dependency', + kodyDependencies: ['@kentcdodds/unused'], + }), + ], + ['src/index.ts', 'export const ready = true\n'], + ]) + const snapshot = createSnapshotFromFiles(files) + mockModule.createFileSystemSnapshot.mockResolvedValue(snapshot) + + const result = await runRepoChecks({ + workspace: { + async readFile(path: string) { + return files.get(path) ?? null + }, + async glob() { + return Array.from(files.keys()).map((path) => ({ path, type: 'file' })) + }, + }, + manifestPath: 'package.json', + sourceRoot: '/', + }) + + expect(result.ok).toBe(false) + expect(result.results).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + kind: 'dependencies', + ok: false, + message: expect.stringContaining( + 'package.json#kody.dependencies must match direct static kody:@ imports (unused "@kentcdodds/unused").', + ), }), ]), ) @@ -1195,7 +1519,8 @@ test('runRepoChecks validates package runtime bundles with npm dependencies', as expect.objectContaining({ kind: 'dependencies', ok: true, - message: 'package.json declares 1 npm dependency: "marked".', + message: + 'package.json declares 1 npm dependency: "marked". package.json#kody.dependencies declares no static Kody package dependencies.', }), expect.objectContaining({ kind: 'bundle', diff --git a/packages/worker/src/repo/checks.ts b/packages/worker/src/repo/checks.ts index c255d0c569..541f1d2e94 100644 --- a/packages/worker/src/repo/checks.ts +++ b/packages/worker/src/repo/checks.ts @@ -13,6 +13,7 @@ import { buildKodyImportableModuleBundle, buildKodyModuleBundle, } from '#worker/package-runtime/module-graph.ts' +import { collectStaticKodyPackageImportsFromFiles } from '#worker/package-runtime/static-kody-imports.ts' import { hasTopLevelDefaultExport } from '#worker/module-source.ts' import { createRepoCodemodeModuleTypecheckHarness, @@ -363,7 +364,7 @@ function collectPackageCallableTypecheckTargets( return Array.from(targets.values()) } -function parseDeclaredDependencies(packageJsonContent: string | null) { +function parseDeclaredNpmDependencies(packageJsonContent: string | null) { if (!packageJsonContent) return [] try { const parsed = JSON.parse(packageJsonContent) as { @@ -377,6 +378,89 @@ function parseDeclaredDependencies(packageJsonContent: string | null) { } } +function pluralize(count: number, singular: string, plural: string) { + return count === 1 ? singular : plural +} + +function formatQuotedList(values: Array) { + return values.map((value) => `"${value}"`).join(', ') +} + +function formatNpmDependencyCheckMessage(input: { + packageJsonMissing: boolean + dependencies: Array +}) { + if (input.packageJsonMissing) { + return 'No package.json found in source root; dependency check skipped.' + } + if (input.dependencies.length === 0) { + return 'package.json declares no npm dependencies.' + } + return `package.json declares ${input.dependencies.length} npm ${pluralize( + input.dependencies.length, + 'dependency', + 'dependencies', + )}: ${formatQuotedList(input.dependencies)}.` +} + +function getDeclaredStaticKodyPackageDependencies( + manifest: AuthoredPackageJson, +) { + return Array.from( + new Set( + (manifest.kody.dependencies ?? []).map((dependency) => dependency.trim()), + ), + ).sort((left, right) => left.localeCompare(right)) +} + +function getImportedStaticKodyPackageDependencies(input: { + manifest: AuthoredPackageJson + sourceFiles: Record +}) { + return Array.from( + new Set( + collectStaticKodyPackageImportsFromFiles(input.sourceFiles) + .map((imported) => imported.packageName) + .filter((packageName) => packageName !== input.manifest.name), + ), + ).sort((left, right) => left.localeCompare(right)) +} + +function validateStaticKodyPackageDependencyDeclarations(input: { + manifest: AuthoredPackageJson + sourceFiles: Record +}) { + const declared = getDeclaredStaticKodyPackageDependencies(input.manifest) + const imported = getImportedStaticKodyPackageDependencies(input) + const missing = imported.filter( + (packageName) => !declared.includes(packageName), + ) + const unused = declared.filter( + (packageName) => !imported.includes(packageName), + ) + if (missing.length === 0 && unused.length === 0) { + return { + ok: true, + message: + declared.length === 0 + ? 'package.json#kody.dependencies declares no static Kody package dependencies.' + : `package.json#kody.dependencies declares ${declared.length} static Kody package ${pluralize( + declared.length, + 'dependency', + 'dependencies', + )}: ${formatQuotedList(declared)}.`, + } + } + const details = [ + missing.length > 0 ? `missing ${formatQuotedList(missing)}` : null, + unused.length > 0 ? `unused ${formatQuotedList(unused)}` : null, + ].filter((detail): detail is string => detail != null) + return { + ok: false, + message: `package.json#kody.dependencies must match direct static kody:@ imports (${details.join('; ')}).`, + } +} + async function validatePackageBundles(input: { env: Env baseUrl: string @@ -656,18 +740,22 @@ export async function runRepoChecks(input: { ) const packageJson = snapshot.read('package.json') - const declaredDependencies = parseDeclaredDependencies(packageJson) + const declaredNpmDependencies = parseDeclaredNpmDependencies(packageJson) + const staticKodyDependencyCheck = + validateStaticKodyPackageDependencyDeclarations({ + manifest, + sourceFiles, + }) results.push({ kind: 'dependencies', - ok: true, - message: - packageJson == null - ? 'No package.json found in source root; dependency check skipped.' - : declaredDependencies.length === 0 - ? 'package.json declares no npm dependencies.' - : `package.json declares ${declaredDependencies.length} npm dependenc${ - declaredDependencies.length === 1 ? 'y' : 'ies' - }: ${declaredDependencies.map((dependency) => `"${dependency}"`).join(', ')}.`, + ok: staticKodyDependencyCheck.ok, + message: [ + formatNpmDependencyCheckMessage({ + packageJsonMissing: packageJson == null, + dependencies: declaredNpmDependencies, + }), + staticKodyDependencyCheck.message, + ].join(' '), }) const bundleTargets = collectPackageBundleTargets(manifest) diff --git a/packages/worker/src/repo/published-bundle-artifacts-repo.node.test.ts b/packages/worker/src/repo/published-bundle-artifacts-repo.node.test.ts new file mode 100644 index 0000000000..782f3f15da --- /dev/null +++ b/packages/worker/src/repo/published-bundle-artifacts-repo.node.test.ts @@ -0,0 +1,113 @@ +import { expect, test, vi } from 'vitest' +import { + countStaticDependentBundleArtifactPackages, + listStaticDependentBundleArtifactRows, +} from './published-bundle-artifacts-repo.ts' + +function createStaticDependentsDb(input: { + first?: Record | null + results?: Array> + onBind?: (query: string, values: Array) => void +}) { + return { + prepare(query: string) { + return { + bind(...values: Array) { + input.onBind?.(query, values) + return { + async first() { + return input.first ?? null + }, + async all() { + return { results: input.results ?? [] } + }, + } + }, + } + }, + } as unknown as D1Database +} + +test('countStaticDependentBundleArtifactPackages counts direct JSON dependency rows by source id', async () => { + const onBind = vi.fn() + const db = createStaticDependentsDb({ + first: { + total_packages: 2, + stale_packages: 1, + }, + onBind, + }) + + const result = await countStaticDependentBundleArtifactPackages(db, { + userId: 'user-1', + sourceId: 'source-a', + currentDependencyCommit: 'commit-a-new', + }) + + expect(result).toEqual({ + totalPackages: 2, + stalePackages: 1, + }) + expect(onBind).toHaveBeenCalledWith( + expect.stringContaining('json_each(artifact.dependencies_json)'), + ['commit-a-new', 'user-1', 'source-a', 'source-a'], + ) + expect(onBind.mock.calls[0]?.[0]).toContain('artifact.source_id != ?') + expect(onBind.mock.calls[0]?.[0]).toContain( + 'artifact.published_commit = source.published_commit', + ) +}) + +test('listStaticDependentBundleArtifactRows maps bounded dependent package artifact rows', async () => { + const onBind = vi.fn() + const db = createStaticDependentsDb({ + results: [ + { + package_id: 'package-b', + package_kody_id: 'package-b', + package_name: '@kentcdodds/package-b', + source_id: 'source-b', + published_commit: 'commit-b', + artifact_kind: 'module', + artifact_name: '.', + entry_point: 'src/index.ts', + package_stale: 1, + matching_artifact_count: 1, + matching_entrypoint_count: 1, + package_bundled_dependency_commit: 'commit-a-old', + bundled_dependency_commit: 'commit-a-old', + }, + ], + onBind, + }) + + const rows = await listStaticDependentBundleArtifactRows(db, { + userId: 'user-1', + sourceId: 'source-a', + currentDependencyCommit: 'commit-a-new', + packageLimit: 10, + artifactsPerPackageLimit: 5, + }) + + expect(rows).toEqual([ + { + packageId: 'package-b', + packageKodyId: 'package-b', + packageName: '@kentcdodds/package-b', + sourceId: 'source-b', + publishedCommit: 'commit-b', + artifactKind: 'module', + artifactName: '.', + entryPoint: 'src/index.ts', + packageStale: true, + matchingArtifactCount: 1, + matchingEntrypointCount: 1, + packageBundledDependencyCommit: 'commit-a-old', + bundledDependencyCommit: 'commit-a-old', + }, + ]) + expect(onBind).toHaveBeenCalledWith( + expect.stringContaining('DENSE_RANK() OVER'), + ['commit-a-new', 'user-1', 'source-a', 'source-a', 10, 5], + ) +}) diff --git a/packages/worker/src/repo/published-bundle-artifacts-repo.ts b/packages/worker/src/repo/published-bundle-artifacts-repo.ts index 02da99e8f7..25a2d1ed29 100644 --- a/packages/worker/src/repo/published-bundle-artifacts-repo.ts +++ b/packages/worker/src/repo/published-bundle-artifacts-repo.ts @@ -23,6 +23,27 @@ export type PublishedBundleArtifactUpsertInput = { dependenciesJson: string } +export type StaticDependentBundleArtifactRow = { + packageId: string + packageKodyId: string + packageName: string + sourceId: string + publishedCommit: string | null + artifactKind: string + artifactName: string | null + entryPoint: string + packageStale: boolean + matchingArtifactCount: number + matchingEntrypointCount: number + packageBundledDependencyCommit: string | null + bundledDependencyCommit: string | null +} + +export type StaticDependentBundleArtifactCounts = { + totalPackages: number + stalePackages: number +} + function mapRow(row: Record): PublishedBundleArtifactRecord { return { id: String(row['id']), @@ -40,6 +61,34 @@ function mapRow(row: Record): PublishedBundleArtifactRecord { } } +function mapStaticDependentBundleArtifactRow( + row: Record, +): StaticDependentBundleArtifactRow { + return { + packageId: String(row['package_id']), + packageKodyId: String(row['package_kody_id']), + packageName: String(row['package_name']), + sourceId: String(row['source_id']), + publishedCommit: + row['published_commit'] == null ? null : String(row['published_commit']), + artifactKind: String(row['artifact_kind']), + artifactName: + row['artifact_name'] == null ? null : String(row['artifact_name']), + entryPoint: String(row['entry_point']), + packageStale: Number(row['package_stale'] ?? 0) === 1, + matchingArtifactCount: Number(row['matching_artifact_count'] ?? 0), + matchingEntrypointCount: Number(row['matching_entrypoint_count'] ?? 0), + packageBundledDependencyCommit: + row['package_bundled_dependency_commit'] == null + ? null + : String(row['package_bundled_dependency_commit']), + bundledDependencyCommit: + row['bundled_dependency_commit'] == null + ? null + : String(row['bundled_dependency_commit']), + } +} + export async function getPublishedBundleArtifactByIdentity( db: D1Database, input: { @@ -69,6 +118,177 @@ export async function getPublishedBundleArtifactByIdentity( return row ? mapRow(row) : null } +export async function countStaticDependentBundleArtifactPackages( + db: D1Database, + input: { + userId: string + sourceId: string + currentDependencyCommit: string + }, +): Promise { + const row = await db + .prepare( + `WITH matching AS ( + SELECT + p.id AS package_id, + CASE + WHEN json_extract(dependency.value, '$.publishedCommit') IS NULL + OR json_extract(dependency.value, '$.publishedCommit') != ? + THEN 1 + ELSE 0 + END AS stale + FROM published_bundle_artifacts AS artifact + JOIN json_each(artifact.dependencies_json) AS dependency + JOIN entity_sources AS source + ON source.id = artifact.source_id + AND source.user_id = artifact.user_id + AND source.entity_kind = 'package' + JOIN saved_packages AS p + ON p.id = source.entity_id + AND p.user_id = artifact.user_id + WHERE artifact.user_id = ? + AND artifact.source_id != ? + AND artifact.published_commit = source.published_commit + AND json_extract(dependency.value, '$.sourceId') = ? + ) + SELECT + COUNT(DISTINCT package_id) AS total_packages, + COUNT(DISTINCT CASE WHEN stale = 1 THEN package_id END) AS stale_packages + FROM matching`, + ) + .bind( + input.currentDependencyCommit, + input.userId, + input.sourceId, + input.sourceId, + ) + .first>() + return { + totalPackages: Number(row?.['total_packages'] ?? 0), + stalePackages: Number(row?.['stale_packages'] ?? 0), + } +} + +export async function listStaticDependentBundleArtifactRows( + db: D1Database, + input: { + userId: string + sourceId: string + currentDependencyCommit: string + packageLimit: number + artifactsPerPackageLimit: number + }, +) { + const result = await db + .prepare( + `WITH matching AS ( + SELECT + p.id AS package_id, + p.kody_id AS package_kody_id, + p.name AS package_name, + artifact.source_id, + source.published_commit, + artifact.artifact_kind, + artifact.artifact_name, + artifact.entry_point, + json_extract(dependency.value, '$.publishedCommit') AS bundled_dependency_commit, + CASE + WHEN json_extract(dependency.value, '$.publishedCommit') IS NULL + OR json_extract(dependency.value, '$.publishedCommit') != ? + THEN 1 + ELSE 0 + END AS stale + FROM published_bundle_artifacts AS artifact + JOIN json_each(artifact.dependencies_json) AS dependency + JOIN entity_sources AS source + ON source.id = artifact.source_id + AND source.user_id = artifact.user_id + AND source.entity_kind = 'package' + JOIN saved_packages AS p + ON p.id = source.entity_id + AND p.user_id = artifact.user_id + WHERE artifact.user_id = ? + AND artifact.source_id != ? + AND artifact.published_commit = source.published_commit + AND json_extract(dependency.value, '$.sourceId') = ? + ), + package_rollup AS ( + SELECT + package_id, + MAX(stale) AS package_stale, + COUNT(*) AS matching_artifact_count, + COUNT(DISTINCT entry_point) AS matching_entrypoint_count, + CASE + WHEN COUNT(DISTINCT COALESCE(bundled_dependency_commit, '__missing__')) = 1 + THEN MIN(bundled_dependency_commit) + ELSE NULL + END AS package_bundled_dependency_commit + FROM matching + GROUP BY package_id + ), + entrypoint_representatives AS ( + SELECT + matching.*, + package_rollup.package_stale, + package_rollup.matching_artifact_count, + package_rollup.matching_entrypoint_count, + package_rollup.package_bundled_dependency_commit, + ROW_NUMBER() OVER ( + PARTITION BY matching.package_id, matching.entry_point + ORDER BY stale DESC, artifact_kind ASC, COALESCE(artifact_name, '') ASC, entry_point ASC + ) AS entrypoint_artifact_rank + FROM matching + JOIN package_rollup + ON package_rollup.package_id = matching.package_id + ), + ranked_entrypoints AS ( + SELECT + entrypoint_representatives.*, + ROW_NUMBER() OVER ( + PARTITION BY package_id + ORDER BY stale DESC, entry_point ASC + ) AS entrypoint_rank + FROM entrypoint_representatives + WHERE entrypoint_artifact_rank = 1 + ), + ranked_packages AS ( + SELECT + ranked_entrypoints.*, + DENSE_RANK() OVER ( + ORDER BY package_stale DESC, package_name ASC, package_id ASC + ) AS package_rank + FROM ranked_entrypoints + ) + SELECT + package_id, + package_kody_id, + package_name, + source_id, + published_commit, + artifact_kind, + artifact_name, + entry_point, + package_stale, + matching_artifact_count, + matching_entrypoint_count, + package_bundled_dependency_commit, + bundled_dependency_commit + FROM ranked_packages + WHERE package_rank <= ? AND entrypoint_rank <= ? + ORDER BY package_rank ASC, entrypoint_rank ASC`, + ) + .bind( + input.currentDependencyCommit, + input.userId, + input.sourceId, + input.sourceId, + input.packageLimit, + input.artifactsPerPackageLimit, + ) + .all>() + return (result.results ?? []).map(mapStaticDependentBundleArtifactRow) +} + export async function listPublishedBundleArtifactsBySourceId( db: D1Database, userId: string, diff --git a/packages/worker/src/repo/published-bundle-artifacts-repo.workers.test.ts b/packages/worker/src/repo/published-bundle-artifacts-repo.workers.test.ts new file mode 100644 index 0000000000..e823666ee8 --- /dev/null +++ b/packages/worker/src/repo/published-bundle-artifacts-repo.workers.test.ts @@ -0,0 +1,267 @@ +import { env } from 'cloudflare:workers' +import { expect, test } from 'vitest' +import { getStaticPackageDependentsSummary } from '#worker/package-runtime/static-package-dependents.ts' + +async function runSql(sql: string, ...values: Array) { + await env.APP_DB.prepare(sql) + .bind(...values) + .run() +} + +async function ensurePublishedBundleArtifactDependencySchema() { + await runSql(`CREATE TABLE IF NOT EXISTS entity_sources ( + id TEXT PRIMARY KEY, + user_id TEXT NOT NULL, + entity_kind TEXT NOT NULL, + entity_id TEXT NOT NULL, + repo_id TEXT NOT NULL, + published_commit TEXT, + indexed_commit TEXT, + manifest_path TEXT NOT NULL DEFAULT 'package.json', + source_root TEXT NOT NULL DEFAULT '/', + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL + )`) + await runSql(`CREATE TABLE IF NOT EXISTS saved_packages ( + id TEXT PRIMARY KEY NOT NULL, + user_id TEXT NOT NULL, + name TEXT NOT NULL, + kody_id TEXT NOT NULL, + description TEXT NOT NULL, + tags_json TEXT NOT NULL DEFAULT '[]', + search_text TEXT, + source_id TEXT NOT NULL, + has_app INTEGER NOT NULL DEFAULT 0 CHECK (has_app IN (0, 1)), + created_at TEXT NOT NULL DEFAULT (CURRENT_TIMESTAMP), + updated_at TEXT NOT NULL DEFAULT (CURRENT_TIMESTAMP) + )`) + await runSql(`CREATE TABLE IF NOT EXISTS published_bundle_artifacts ( + id TEXT PRIMARY KEY, + user_id TEXT NOT NULL, + source_id TEXT NOT NULL, + published_commit TEXT NOT NULL, + artifact_kind TEXT NOT NULL, + artifact_name TEXT, + entry_point TEXT NOT NULL, + kv_key TEXT NOT NULL, + dependencies_json TEXT NOT NULL DEFAULT '[]', + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL + )`) +} + +async function insertPackage(input: { + userId: string + packageId: string + kodyId: string + name: string + sourceId: string + publishedCommit: string +}) { + const now = new Date().toISOString() + await runSql( + `INSERT INTO saved_packages ( + id, user_id, name, kody_id, description, tags_json, search_text, + source_id, has_app, created_at, updated_at + ) VALUES (?, ?, ?, ?, ?, '[]', NULL, ?, 0, ?, ?)`, + input.packageId, + input.userId, + input.name, + input.kodyId, + `${input.name} package`, + input.sourceId, + now, + now, + ) + await runSql( + `INSERT INTO entity_sources ( + id, user_id, entity_kind, entity_id, repo_id, published_commit, + indexed_commit, manifest_path, source_root, created_at, updated_at + ) VALUES (?, ?, 'package', ?, ?, ?, NULL, 'package.json', '/', ?, ?)`, + input.sourceId, + input.userId, + input.packageId, + `repo-${input.sourceId}`, + input.publishedCommit, + now, + now, + ) +} + +async function insertArtifact(input: { + userId: string + sourceId: string + publishedCommit: string + artifactName: string + entryPoint: string + dependencies: Array> +}) { + const now = new Date().toISOString() + await runSql( + `INSERT INTO published_bundle_artifacts ( + id, user_id, source_id, published_commit, artifact_kind, artifact_name, + entry_point, kv_key, dependencies_json, created_at, updated_at + ) VALUES (?, ?, ?, ?, 'module', ?, ?, ?, ?, ?, ?)`, + crypto.randomUUID(), + input.userId, + input.sourceId, + input.publishedCommit, + input.artifactName, + input.entryPoint, + `kv:${input.sourceId}:${input.artifactName}`, + JSON.stringify(input.dependencies), + now, + now, + ) +} + +test('static dependent summary runs JSON dependency queries against D1', async () => { + await ensurePublishedBundleArtifactDependencySchema() + const unique = crypto.randomUUID() + const userId = `user-${unique}` + const sourceA = `source-a-${unique}` + const sourceB = `source-b-${unique}` + const sourceC = `source-c-${unique}` + const sourceD = `source-d-${unique}` + await insertPackage({ + userId, + packageId: `package-a-${unique}`, + kodyId: `package-a-${unique}`, + name: `@kentcdodds/package-a-${unique}`, + sourceId: sourceA, + publishedCommit: 'commit-a-new', + }) + await insertPackage({ + userId, + packageId: `package-b-${unique}`, + kodyId: `package-b-${unique}`, + name: `@kentcdodds/package-b-${unique}`, + sourceId: sourceB, + publishedCommit: 'commit-b', + }) + await insertPackage({ + userId, + packageId: `package-c-${unique}`, + kodyId: `package-c-${unique}`, + name: `@kentcdodds/package-c-${unique}`, + sourceId: sourceC, + publishedCommit: 'commit-c', + }) + await insertPackage({ + userId, + packageId: `package-d-${unique}`, + kodyId: `package-d-${unique}`, + name: `@kentcdodds/package-d-${unique}`, + sourceId: sourceD, + publishedCommit: 'commit-d-current', + }) + for (let index = 0; index < 5; index += 1) { + await insertArtifact({ + userId, + sourceId: sourceB, + publishedCommit: 'commit-b', + artifactName: `a-current-${index}`, + entryPoint: `src/current-${index}.ts`, + dependencies: [ + { + sourceId: sourceA, + publishedCommit: 'commit-a-new', + kodyId: `package-a-${unique}`, + packageName: `@kentcdodds/package-a-${unique}`, + }, + ], + }) + } + await insertArtifact({ + userId, + sourceId: sourceB, + publishedCommit: 'commit-b', + artifactName: 'a-current-0-importable', + entryPoint: 'src/current-0.ts', + dependencies: [ + { + sourceId: sourceA, + publishedCommit: 'commit-a-new', + kodyId: `package-a-${unique}`, + packageName: `@kentcdodds/package-a-${unique}`, + }, + ], + }) + await insertArtifact({ + userId, + sourceId: sourceB, + publishedCommit: 'commit-b', + artifactName: 'z-stale', + entryPoint: 'src/stale.ts', + dependencies: [ + { + sourceId: sourceA, + publishedCommit: 'commit-a-old', + kodyId: `package-a-${unique}`, + packageName: `@kentcdodds/package-a-${unique}`, + }, + ], + }) + await insertArtifact({ + userId, + sourceId: sourceC, + publishedCommit: 'commit-c', + artifactName: 'missing-commit', + entryPoint: 'src/missing.ts', + dependencies: [ + { + sourceId: sourceA, + kodyId: `package-a-${unique}`, + packageName: `@kentcdodds/package-a-${unique}`, + }, + ], + }) + await insertArtifact({ + userId, + sourceId: sourceD, + publishedCommit: 'commit-d-obsolete', + artifactName: 'removed-import', + entryPoint: 'src/removed-import.ts', + dependencies: [ + { + sourceId: sourceA, + publishedCommit: 'commit-a-old', + kodyId: `package-a-${unique}`, + packageName: `@kentcdodds/package-a-${unique}`, + }, + ], + }) + + const summary = await getStaticPackageDependentsSummary({ + db: env.APP_DB, + userId, + sourceId: sourceA, + currentDependencyCommit: 'commit-a-new', + }) + + expect(summary).toEqual( + expect.objectContaining({ + total: 2, + stale: 2, + truncated: false, + }), + ) + expect(summary.items[0]).toEqual( + expect.objectContaining({ + name: `@kentcdodds/package-b-${unique}`, + stale: true, + artifact_count: 7, + entrypoints_truncated: true, + bundled_dependency_commit: null, + }), + ) + expect(summary.items[0]?.entrypoints).toHaveLength(5) + expect(summary.items[0]?.entrypoints[0]).toBe('src/stale.ts') + expect(summary.items[1]).toEqual( + expect.objectContaining({ + name: `@kentcdodds/package-c-${unique}`, + stale: true, + bundled_dependency_commit: null, + }), + ) +})