Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/guides/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,10 +90,17 @@ defineConfig({
build: {
outDir: "dist", // Output directory
trailingSlash: false, // Add trailing slashes to URLs
serverExternalPackages: ["knex", "@prisma/client"],
},
});
```

Use `serverExternalPackages` for npm packages that must run only on the server,
such as database, cache, or messaging clients. Veryfront leaves these imports
external so the runtime resolves the installed package instead of sending it
through the browser module CDN. Use package roots only. Do not include versions
or subpaths.

### Layout

```ts
Expand Down
10 changes: 5 additions & 5 deletions src/config/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@ field. The complete validated config is also passed to extensions and included
in render-cache identity, so compatibility-only fields cannot be removed as
incidental cleanup.

| Ownership | Fields |
| ---------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Core runtime/build | `projectSlug`, `react.version`, `directories.app/pages/components`, `router`, `layout`, `app`, `experimental.esmLayouts/rsc`, `build.outDir`, `build.ssg`, `cache`, supported `dev` fields, `resolve.importMap`, `security`, `middleware.custom`, `fs.veryfront`, `fs.github`, AI primitive discovery, `client`, `styles.stylesheet`, `integrations`, `extensions`, and core `openapi` fields |
| CLI or diagnostics | `experimental.precompileMDX`, `generate.preferredRouter`, `ai.enabled`, and provider API-key checks |
| Accepted for extension compatibility, without built-in semantics | `title`, `description`, `directories.ai`, `theme.colors`, `build.trailingSlash/esbuild`, `dev.host/open/hmrPort`, `theming`, `assetPipeline`, tracing/metrics project config, `search`, `fs.local.baseDir`, `fs.memory`, provider defaults, `ai.work`, `ai.mcp`, and `openapi.mcp` |
| Ownership | Fields |
| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Core runtime/build | `projectSlug`, `react.version`, `directories.app/pages/components`, `router`, `layout`, `app`, `experimental.esmLayouts/rsc`, `build.outDir`, `build.ssg`, `build.serverExternalPackages`, `cache`, supported `dev` fields, `resolve.importMap`, `security`, `middleware.custom`, `fs.veryfront`, `fs.github`, AI primitive discovery, `client`, `styles.stylesheet`, `integrations`, `extensions`, and core `openapi` fields |
| CLI or diagnostics | `experimental.precompileMDX`, `generate.preferredRouter`, `ai.enabled`, and provider API-key checks |
| Accepted for extension compatibility, without built-in semantics | `title`, `description`, `directories.ai`, `theme.colors`, `build.trailingSlash/esbuild`, `dev.host/open/hmrPort`, `theming`, `assetPipeline`, tracing/metrics project config, `search`, `fs.local.baseDir`, `fs.memory`, provider defaults, `ai.work`, `ai.mcp`, and `openapi.mcp` |

Keep public documentation aligned with this table. Implementing a
compatibility-only field requires an owned consumer and end-to-end tests;
Expand Down
27 changes: 27 additions & 0 deletions src/config/schemas/config.schema.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -478,6 +478,33 @@ describe("configSchema", () => {
);
});

it("accepts bare package names in build.serverExternalPackages", () => {
const config = validateVeryfrontConfig({
build: {
serverExternalPackages: ["knex", "@prisma/client"],
},
});

assertEquals(config.build?.serverExternalPackages, ["knex", "@prisma/client"]);
});

it("rejects versions, subpaths, duplicates, and empty server external packages", () => {
for (
const serverExternalPackages of [
["knex@3.1.0"],
["@prisma/client/runtime"],
["knex", "knex"],
[],
]
) {
assertThrows(
() => validateVeryfrontConfig({ build: { serverExternalPackages } }),
Error,
"Invalid veryfront.config at build.serverExternalPackages",
);
}
});

it("returns registered validation errors without retaining the full config", () => {
const input = {
dev: { port: "invalid" },
Expand Down
25 changes: 25 additions & 0 deletions src/config/schemas/config.schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,12 @@ import {
} from "#veryfront/utils/discovery-path-policy.ts";
import { MAX_PATH_LENGTH_CHARS } from "#veryfront/utils/constants/limits.ts";
import { isCanonicalProjectRelativePath } from "#veryfront/utils/project-relative-path.ts";
import {
hasUniqueServerExternalPackages,
isValidServerExternalPackageName,
MAX_SERVER_EXTERNAL_PACKAGE_COUNT,
MAX_SERVER_EXTERNAL_PACKAGE_NAME_LENGTH,
} from "#veryfront/config/server-external-packages.ts";

const integrationNames = new Set<string>(ALL_INTEGRATION_NAMES);
const MAX_CSRF_EXCLUDE_PATH_COUNT = 64;
Expand Down Expand Up @@ -356,6 +362,25 @@ export const getVeryfrontConfigSchema = defineSchema((v) =>
.object({
outDir: v.string().optional(),
trailingSlash: v.boolean().optional(),
/** Bare npm package roots that the runtime resolves instead of bundling. */
serverExternalPackages: v
.array(
v
.string()
.min(1)
.max(MAX_SERVER_EXTERNAL_PACKAGE_NAME_LENGTH)
.refine(
isValidServerExternalPackageName,
"Expected a bare npm package name without a version or subpath",
),
)
.min(1)
.max(MAX_SERVER_EXTERNAL_PACKAGE_COUNT)
.refine(
hasUniqueServerExternalPackages,
"Server external package names must be unique",
)
.optional(),
/**
* Generate static HTML for all routes during `veryfront build`.
* Defaults to true; disabling it produces no pages, so only turn it
Expand Down
64 changes: 64 additions & 0 deletions src/config/server-external-packages.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import {
primordialArrayPush,
primordialArraySort,
} from "#veryfront/platform/compat/primordials/array.ts";

export const MAX_SERVER_EXTERNAL_PACKAGE_COUNT = 128;
export const MAX_SERVER_EXTERNAL_PACKAGE_NAME_LENGTH = 214;

const ObjectFreeze = Object.freeze;
const RegExpExec = RegExp.prototype.exec;
const ReflectApply = Reflect.apply;
const SetAdd = Set.prototype.add;
const SetHas = Set.prototype.has;
const SERVER_EXTERNAL_PACKAGE_PATTERN = /^(?:@[a-z0-9][a-z0-9._-]*\/)?[a-z0-9][a-z0-9._-]*$/;

/** Return whether a value names one bare npm package root. */
export function isValidServerExternalPackageName(value: string): boolean {
return value.length <= MAX_SERVER_EXTERNAL_PACKAGE_NAME_LENGTH &&
ReflectApply(RegExpExec, SERVER_EXTERNAL_PACKAGE_PATTERN, [value]) !== null;
}

/** Return whether every configured package name occurs exactly once. */
export function hasUniqueServerExternalPackages(values: readonly string[]): boolean {
const seen = new Set<string>();
for (let index = 0; index < values.length; index++) {
const value = values[index]!;
if (ReflectApply(SetHas, seen, [value]) as boolean) return false;
ReflectApply(SetAdd, seen, [value]);
}
return true;
}

/** Capture an immutable, order-independent package list for one transform graph. */
export function canonicalizeServerExternalPackages(
values: readonly string[] | undefined,
): readonly string[] | undefined {
if (!values || values.length === 0) return undefined;

const canonical: string[] = [];
const seen = new Set<string>();
for (let index = 0; index < values.length; index++) {
const value = values[index]!;
if (ReflectApply(SetHas, seen, [value]) as boolean) continue;
ReflectApply(SetAdd, seen, [value]);
primordialArrayPush(canonical, value);
}
primordialArraySort(canonical, (left, right) => left < right ? -1 : left > right ? 1 : 0);
return ObjectFreeze(canonical);
}

/** Build a stable framed identity for cache keys whose output depends on this list. */
export function buildServerExternalPackagesIdentity(
values: readonly string[] | undefined,
): string | undefined {
const canonical = canonicalizeServerExternalPackages(values);
if (!canonical) return undefined;

let identity = "";
for (let index = 0; index < canonical.length; index++) {
const value = canonical[index]!;
identity += `${value.length}:${value};`;
}
return identity;
}
2 changes: 2 additions & 0 deletions src/modules/react-loader/component-loader.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ export async function loadModuleFromSource(
dev,
contentSourceId: options?.contentSourceId,
reactVersion: options?.reactVersion,
serverExternalPackages: options?.serverExternalPackages,
moduleServerOrigin,
dependencyPinningCacheKey: dependencySnapshot.cacheKey,
dependencyPinningDependencies: dependencySnapshot.dependencies,
Expand All @@ -71,6 +72,7 @@ export async function loadModuleFromSource(
vendorBundleHash: options?.vendorBundleHash,
ssr: false,
reactVersion: options?.reactVersion,
serverExternalPackages: options?.serverExternalPackages,
dependencyPinningCacheKey: dependencySnapshot.cacheKey,
dependencyPinningDependencies: dependencySnapshot.dependencies,
dependencyPinningSource,
Expand Down
8 changes: 8 additions & 0 deletions src/modules/react-loader/ssr-module-loader/loader.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,14 @@ describe("SSRModuleLoader", { sanitizeResources: false, sanitizeOps: false }, ()
}),
undefined,
);
const externalA = __ssrModuleLoaderInternals.getMdxEsmCacheVariant({
serverExternalPackages: ["knex", "@prisma/client"],
});
const externalB = __ssrModuleLoaderInternals.getMdxEsmCacheVariant({
serverExternalPackages: ["@prisma/client", "knex"],
});
assertEquals(externalB, externalA);
assert(externalA?.startsWith("on:server-externals-"));
});

it("invalidates stale cache entries with missing local dependencies and retransforms", async () => {
Expand Down
11 changes: 8 additions & 3 deletions src/modules/react-loader/ssr-module-loader/loader.ts
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ import {
createDependencyHashCache,
type DependencyHashCache,
} from "#veryfront/cache/dependency-graph.ts";
import { buildDependencyPinningCacheVariant } from "#veryfront/cache/keys/dependency-pinning.ts";
import { getMdxModuleCacheVariant } from "#veryfront/transforms/mdx/esm-module-loader/module-fetcher/cache-keys.ts";

const logger = rendererLogger.component("ssr-module-loader");
const CACHE_FILE_MISSING_PREFIX = "Cache file missing:";
Expand Down Expand Up @@ -136,11 +136,15 @@ function publishTransformCacheIfCurrent(input: {
}

function getMdxEsmCacheVariant(
options: Pick<SSRModuleLoaderOptions, "dependencyPinningCacheKey" | "moduleServerOrigin">,
options: Pick<
SSRModuleLoaderOptions,
"dependencyPinningCacheKey" | "moduleServerOrigin" | "serverExternalPackages"
>,
): string | undefined {
return buildDependencyPinningCacheVariant(
return getMdxModuleCacheVariant(
options.dependencyPinningCacheKey,
options.moduleServerOrigin,
options.serverExternalPackages,
);
}

Expand Down Expand Up @@ -811,6 +815,7 @@ export class SSRModuleLoader {
apiBaseUrl: this.options.apiBaseUrl,
moduleServerOrigin: this.options.moduleServerOrigin,
reactVersion: this.options.reactVersion,
serverExternalPackages: this.options.serverExternalPackages,
Comment thread
kojiwakayama marked this conversation as resolved.
dependencyHashCache,
dependencyPinningCacheKey: this.options.dependencyPinningCacheKey,
dependencyPinningDependencies: this.options.dependencyPinningDependencies,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,43 @@ describe("SSRCacheManager", { sanitizeResources: false, sanitizeOps: false }, ()
}
});

it("separates SSR module cache identity by server external packages", async () => {
const projectDir = await makeTempDir({ prefix: "vf-ssr-cache-manager-" });
const baseOptions = {
projectDir,
projectId: "project-a",
contentSourceId: "preview-main",
adapter: denoAdapter,
dev: true,
};

try {
const noExternals = new SSRCacheManager(baseOptions);
const externalReact = new SSRCacheManager({
...baseOptions,
serverExternalPackages: ["react"],
});
const externalReactDom = new SSRCacheManager({
...baseOptions,
serverExternalPackages: ["react", "react-dom"],
});
const reorderedExternals = new SSRCacheManager({
...baseOptions,
serverExternalPackages: ["react-dom", "react"],
});

assertNotEquals(noExternals.getConfigHash(), externalReact.getConfigHash());
assertNotEquals(externalReact.getConfigHash(), externalReactDom.getConfigHash());
assertEquals(externalReactDom.getConfigHash(), reorderedExternals.getConfigHash());
assertNotEquals(
externalReact.getCacheKey("/project/pages/index.tsx"),
externalReactDom.getCacheKey("/project/pages/index.tsx"),
);
} finally {
await remove(projectDir, { recursive: true });
}
});

it("recovers missing vfmod dependencies for redis cache entries", async () => {
const projectDir = await makeTempDir({ prefix: "vf-ssr-cache-manager-" });
const distributedCache = new FakeDistributedCache();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@
import { RUNTIME_VERSION } from "#veryfront/utils/version.ts";
import { INVALID_ARGUMENT } from "#veryfront/errors";
import { buildSSRModuleCacheKey } from "#veryfront/cache/keys.ts";
import { hashString } from "#veryfront/cache/hash.ts";
import { computeConfigHashSync } from "#veryfront/cache/config-hash.ts";
import { buildServerExternalPackagesIdentity } from "#veryfront/config/server-external-packages.ts";
import { createFileSystem } from "#veryfront/platform/compat/fs.ts";
import { rendererLogger } from "#veryfront/utils";
import { hashCodeHex } from "#veryfront/utils/hash-utils.ts";
Expand Down Expand Up @@ -49,13 +51,19 @@ export class SSRCacheManager {
/** Lazily compute config hash once per manager instance. */
getConfigHash(): string {
if (!this.cachedConfigHash) {
this.cachedConfigHash = computeConfigHashSync({
const baseConfigHash = computeConfigHashSync({
reactVersion: this.options.reactVersion,
dev: this.options.dev,
apiBaseUrl: this.options.apiBaseUrl,
moduleServerOrigin: this.options.moduleServerOrigin,
dependencyPinningCacheKey: this.options.dependencyPinningCacheKey,
});
const serverExternalPackagesIdentity = buildServerExternalPackagesIdentity(
this.options.serverExternalPackages,
);
this.cachedConfigHash = serverExternalPackagesIdentity
? `${baseConfigHash}:server-externals:${hashString(serverExternalPackagesIdentity)}`
: baseConfigHash;
}
return this.cachedConfigHash;
}
Expand Down
2 changes: 2 additions & 0 deletions src/modules/react-loader/ssr-module-loader/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ export interface SSRModuleLoaderOptions {
contentSourceId?: string;
/** React version for transforms (defaults to DEFAULT_REACT_VERSION) */
reactVersion?: string;
/** Bare npm package roots that the runtime resolves without bundling. */
serverExternalPackages?: readonly string[];
/** Stable VERYFRONT_DEPENDENCY_PINNING + package dependency-map state. */
dependencyPinningCacheKey?: string;
/** Immutable package map paired with dependencyPinningCacheKey. */
Expand Down
2 changes: 2 additions & 0 deletions src/modules/react-loader/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ export interface LoadComponentOptions {
contentSourceId?: string;
/** React version for transforms (from project config) */
reactVersion?: string;
/** Bare npm package roots that the runtime resolves without bundling. */
serverExternalPackages?: readonly string[];
/** Internal stable flag + package dependency-map key for cache isolation. */
dependencyPinningCacheKey?: string;
/** Immutable package map paired with dependencyPinningCacheKey. */
Expand Down
20 changes: 20 additions & 0 deletions src/modules/server/module-batch-handler.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,26 @@ describe(
buildModuleTransformCacheKey("project", "pages/index.js", false),
);
});

it("isolates transforms by the configured server external package set", () => {
const args = [
"project",
"pages/index.js",
false,
"off",
"release-a",
"content-a",
"https://app.example",
] as const;
const baseline = buildBatchTransformCacheKey(...args);
const knex = buildBatchTransformCacheKey(...args, ["knex"]);
const prismaAndKnex = buildBatchTransformCacheKey(...args, ["@prisma/client", "knex"]);
const reordered = buildBatchTransformCacheKey(...args, ["knex", "@prisma/client"]);

assertEquals(knex === baseline, false);
assertEquals(prismaAndKnex === knex, false);
assertEquals(reordered, prismaAndKnex);
});
});

describe("clearBatchCache / getBatchCacheStats", () => {
Expand Down
Loading