astro:cache core component

[FineWolf]

Summary

Create a cache interface within Astro to simplify the development of integrations requiring binary blobs / JSON value caching, and of user space components requiring caching of binary blobs / JSON value.

Background & Motivation

At the moment, in user space, there is no way of effectively caching content that is costly to generate. One common use-case for this is the creation of Astro Endpoints to generate OpenGraph share images.

Integrations are also encouraged to use config.cacheDir to cache results of expensive operations / transformation, but there is no guidance or standard way of doing so.

astro:cache would be a new core component available to both integrations and user space components. This integration would provide by default two adapters:

• NodeFsCacheAdapter: the default configured provider that allows interaction with config.cacheDir. Used at build time, and in user space for static targets.
• InMemoryCacheAdapter: Cache provider that stores values in-memory. Compatible with all targets/runtime.

Third-party integrations would be able to provide and expose their own cache adapters (for example: AWS S3 Cache Adapter, Redis Cache Adapter, etc.).

Possible Use Cases

• astro:assets – uniforming the caching mechanism for astro:assets to astro:cache.
• User space endpoints generating images (for sharing or other purposes) via Satori or canvaskit-wasm could now cache their results for faster build times.

Goals

• Providing a standard interface to read and set binary BLOBs and JSON values into the cache.
• Allow interacting with the cache in user space without requiring the development of a custom integration to do so.
• Normalizing and codifying the use of the cache folder in order to prevent collisions between integrations / user space.

Example

astro:cache would export the following for use in user space and in integrations:

declare module "astro:cache" {
  import type { ZodSchema } from "zod";

  /**
   * Returns a scoped BinaryCacheProvider
   * @param namespace Namespace with which to scope the returned cache provider
   */
  export function getCache(namespace: string): BinaryCacheProvider;

  /**
   * Returns a scoped CacheProvider
   * @param namespace Namespace with which to scope the returned cache provider
   * @param schema Schema which which to optionally validate the unserialized values with
   */
  export function getValueCache<T>(
    namespace: string,
    schema?: ZodSchema<T> | undefined,
  ): CacheProvider<T>;

  /**
   * Helper function to generate a cache key from a given serializable value (for example, a ImageMetadata object).
   * Usage of this is completely optional. An example implementation would be a SHA-256-SUM of the serialized JSON value.
   *
   * @param sourceObj Any JSON serializable object
   */
  export function generateCacheKey<T>(
    sourceObj: T & Serializable<T>,
    options?: GenerateCacheKeyOptions | undefined,
  ): Promise<string>;

  interface GenerateCacheKeyOptions {
    prefix?: string;
    suffix?: string;
  }

  /**
   * A cache provider is a facade of a cache adapter within a given namespace in order to provide
   * user space caching of data.
   */
  export interface CacheProvider<T extends Uint8Array | (T & Serializable<T>)> {
    /**
     * Constructs a new cache provider; This is handled by the factory methods above.
     * @param namespace Namespace of this cache provider. (PRIVATE)
     * @param cacheAdapter Cache adapter associated with this namespace. (PRIVATE)
     */
    constructor(namespace: string, cacheAdapter: CacheAdapter);

    /**
     * Retrieves a value from cache.
     * @param key Cache key
     * @returns {T} The cached data
     */
    get(key: string): Promise<T>;

    /**
     * Retrieve a value from cache. If not present, the value factory is called, and the result of the factory is cached automatically.

     * @param key Cache key
     * @param valueFactory Function that creates a value to be cached. Called when fetching the value from cache failed.
     * @returns {T} The cached data or the newly created data
     */
    getOrCreate(key: string, valueFactory: () => Promise<T> | T): Promise<T>;

    /**
     * Returns true if an item exists in cache, false if not.
     * @param key Cache key
     * @returns {boolean} True if an item exists in cache, false if not.
     */
    has(key: string): Promise<boolean>;

    /**
     * Sets an item in cache
     * @param key Cache key
     * @param value Value to cache
     */
    set(key: string, value: T): Promise<void>;
    delete(key: string): Promise<void>;
  }

  /**
   * A binary cache provider allows writing and reading binary data to cache (images, blobs, etc.).
   */
  export type BinaryCacheProvider = CacheProvider<Uint8Array>;

  /**
   * A CacheAdapter is the actual adapter in charge of interacting with the underlying storage.
   * This only is used by integrators wanting to provide their own cache adapters.
   */
  export interface CacheAdapter {
    /**
     * Verifies if a namespace value is valid for this cache adapter.
     *
     * @param {string} namespace Cache Adapter
     * @returns {boolean} True if the namespace value conforms to the limitations of this cache adapter.
     */
    isValidNamespace(namespace: string): boolean;
    /**
     * Verifies if a key value is valid for this cache provider.
     *
     * @param {string} key Cache Key
     * @returns {boolean} True if the key value conforms to the limitations of this cache adapter.
     */
    isValidKey(key: string): boolean;

    /**
     *
     * @param namespace
     * @param key
     */
    get(namespace: string, key: string): Promise<Uint8Array>;
    has(namespace: string, key: string): Promise<boolean>;
    set(namespace: string, key: string, value: Uint8Array): Promise<void>;
    delete(namespace: string, key: string): Promise<void>;
  }

  /**
   * Internal helper type to determine if a type can be represented as JSON
   */
  type Serializable<T> = T extends string | number | boolean | null
    ? T
    : // eslint-disable-next-line @typescript-eslint/ban-types
    T extends Function
    ? never
    : T extends object
    ? { [K in keyof T]: Serializable<T[K]> }
    : never;
}

Here is an example of a usage of a cache in user space. Imagine an endpoint that generates OpenGraph share images from content collection entries. Automatically caching the result of the generation during a build becomes trivial.

[slug]-share.png.ts

import type { APIContext, GetStaticPathsResult } from "astro";
import { getCollection, type CollectionEntry } from "astro:content";
import { generateCacheKey, getCache } from "astro:cache";

import { generateShareImage } from "../generators/share-image";

const cache = getCache("posts-share-images");

export async function getStaticPaths(): Promise<GetStaticPathsResult> {
  const posts = await getCollection("posts");

  return posts.map((post) => ({
    params: { slug: post.slug },
    props: { post },
  }));
}

export async function get({ props }: APIContext) {
  const post: CollectionEntry<"posts"> = props.post;

  // Create cache key
  const cacheKey = await generateCacheKey(post, { suffix: ".png" });

  // Get from cache, or call the factory for the image if not present in the cache.
  const imageData = await cache.getOrCreate(cacheKey, () =>
    generateShareImage(post),
  );

  // Return the response
  return new Response(imageData);
}

Astro would provide two adapters by default... A nodeFsCacheAdapter and a memoryCacheAdapter.

In astro.config.mjs, adapters could be configured per-namespace.

import { defineConfig, nodeFsCacheAdapter, memoryCacheAdapter } from 'astro/config';

export default defineConfig({
  caching: {
    defaultAdapter: nodeFsCacheAdapter,
    namespaces: {
      myCustomNamespace: memoryCacheAdapter,
    },
  },
});

getCache and getValueCache would return the proper cache provider facade with the right adapter based on the configuration.

[spaceemotion]

Not sure why this hasn't been discussed further - we came across the need for better caching this month as well. We're generating images and data asynchronously when a page loads. This isn't quite the issue when statically building the site, but it does make the build process longer, as well as local development.

Having a way to say "based on the input, tell me if we need to recalculate stuff - otherwise, use the output from last time" would be very helpful and save tons of time and resources (our website now takes multiple minutes to build, including reusing the image cache).

[markomitranic]

This is definitely a valuable thing to have, and a great proposal in terms of its interface.

I would suggest looking into integrating a well-known, standardized cache library instead. For example NestJS uses Cacheable which is a very mature implementation of (already mature) keyv cache adapters for different stores. It not only gives us all the adapters for different stores out of the box, but also has built-in stampede prevention, L1&L2 cache, cache nesting, revalidation, and many other niceities. NestJS relies on it A LOT, and managed to "wrap" it in different ways, to offer composite features like queues, remote procedure calls, blob cache etc. In fact it also is becoming a popular replacement for unstable_cache, in the Next.JS ecosystem, as their cache is crap, I use it with an in-memory LRU L1 cache and Upstash Redis as L2 cache, and use it both for data and component/page caching.

My suggestion isn't to limit our interfaces in any way, but instead to take what is already there as a solid base, for writing our own wrappers, much like Nest did.