Skip to content

Commit

Permalink
Implement RFC "A core story for images" (#6344)
Browse files Browse the repository at this point in the history
* feat(assets): Add Vite plugin

* feat(images): Set up Image component

* fix(types): Attempt to fix type generation

* Revert "fix(types): Attempt to fix type generation"

This reverts commit 063aa27.

* fix(image): Fix image types causing build to fail

* feat(image): Implement client side part

* feat(services): Allow arbitrary transforms parameters

* fix(image): Fix paths and types

* config(types): Update config types to provide completions for available services

* feat(image): Add serving in dev

* feat(image): Improve type error messages

* refactor(image): Move sharp's parseParams to baseService

* refactor(image): Skip work in dev for remote servies

* feat(image): Add support for remote images

* feat(image): Add squoosh service

* chore: update export map

* refactor(image): Abstract attributes handling by services

* config(vercel): Remove test image service

* feat(image): Support for relative images in Markdown (WIP)

* feat(images): Add support for relative images in Markdown

* feat(image): Update with RFC feedback

* fix(image): Fix alt error on getImage

* feat(image): Add support for assets validation through content collections

* feat(image): Remove validateTransform

* feat(image): Move to assets folder

* fix(image): Fix package exports

* feat(image): Add static imports references to virtual moduel

* fix(image): Fix images from content collections not working when embedded

* chore: lockfile

* fix(markdown): Fix type

* fix(images): Flag enhanced images behing an experimental flag

* config(example): Update images example conifg

* fix(image): Fix types

* fix(image): Fix asset type for strict, allow arbritary input and output formats

* chore: fix example check

* feat(image): Emit assets for ESM imported images

* Add initial core image tests (#6381)

* feat(images): Make frontmatter extraction more generic than images for future

* feat(image): Add support for building

* fix(image): Fix types

* fix(images): Fix compatibility with image integration

* feat(images): Cuter generation stats

* fix(images): Globals are unsafe, it turns out

* fix(images): Only generate images if flag is enabled

* fix(images): Only create `addStaticImage` in build

* feat(images): Add SSR endpoint

* fix(images): Only inject route in SSR

* Add tests for SSR

* Remove console.log

* Updated lockfile

* rename to satisfy the link gods

* skip build tests for now

* fix(images): Fix WASM files not being copied in dev

* feat(images): Add quality presets

* fix build tests running

* Remove console.log

* Add tests for getImage

* Test local services

* Test the content collections API

* Add tests for quality

* Skipping content collections test

* feat(image): Add support for `~/assets` alias

* test(image): Add tests for aliases in dev

* Fix windows + content collections

* test(image): Add tests for aliased images and images in Markdown

* Fix markdown images being built

* Should be posix join

* Use the optimized image

* fix test

* Fixes windows smoke

* fix(image): Nits

* feat(images): Add automatic update for `env.d.ts` when experimental images are enabled

* fix(images): Revert env.d.ts change if the user opted-out of the experimental image support

* chore: remove bad image example project

* feat(image): Rename `experimental.images` to `experimental.assets`

* fix(images): Remove unused code in MDX integration

* chore: Remove unrelated change

* fix(images): Remove export from astro/components

* Fix, esm import on Win

* test(images): Add test for format

* fix(images): Add `client-image.d.ts` to export map

* chore: changeset

* fix(images): Adjust with feedback, no more automatic refine, asset() -> image()

* fix(images): Fix types

* fix(images): Remove unnecessary spread

* fix(images): Better types for parseUrl and transform

* fix(images): Fix types

* fix(images): Adjust from feedback

* fix(images): Pass width and height through getHTMLAttributes even if they're not added by the uesr

* fix(images): Recusirsively extract frontmatter assets

* fix(images): Use a reduce instead

* feat(images): Add support for data: URIs

* chore: changeset

* docs(images): Misc docs fixes

* Update .changeset/gold-rocks-cry.md

Co-authored-by: Chris Swithinbank <[email protected]>

* Update .changeset/gold-rocks-cry.md

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update packages/astro/src/@types/astro.ts

Co-authored-by: Chris Swithinbank <[email protected]>

* Update packages/astro/src/assets/services/service.ts

Co-authored-by: Chris Swithinbank <[email protected]>

* Update packages/astro/src/assets/services/service.ts

Co-authored-by: Chris Swithinbank <[email protected]>

* Update packages/astro/src/assets/services/service.ts

Co-authored-by: Chris Swithinbank <[email protected]>

* Update packages/astro/src/assets/types.ts

Co-authored-by: Chris Swithinbank <[email protected]>

* Update packages/astro/src/assets/types.ts

Co-authored-by: Chris Swithinbank <[email protected]>

---------

Co-authored-by: Matthew Phillips <[email protected]>
Co-authored-by: Matthew Phillips <[email protected]>
Co-authored-by: Chris Swithinbank <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>
  • Loading branch information
5 people authored Mar 7, 2023
1 parent 377530a commit 694918a
Show file tree
Hide file tree
Showing 113 changed files with 14,978 additions and 149 deletions.
15 changes: 15 additions & 0 deletions .changeset/gold-rocks-cry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
'astro': minor
'@astrojs/mdx': minor
'@astrojs/markdown-remark': minor
---

Add a new experimental flag (`experimental.assets`) to enable our new core Assets story.

This unlocks a few features:
- A new built-in image component and JavaScript API to transform and optimize images.
- Relative images with automatic optimization in Markdown.
- Support for validating assets using content collections.
- and more!

See [Assets (Experimental)](https://docs.astro.build/en/guides/assets/) on our docs site for more information on how to use this feature!
23 changes: 23 additions & 0 deletions packages/astro/client-base.d.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,28 @@
/// <reference path="./import-meta.d.ts" />

declare module 'astro:assets' {
// Exporting things one by one is a bit cumbersome, not sure if there's a better way - erika, 2023-02-03
type AstroAssets = {
getImage: typeof import('./dist/assets/index.js').getImage;
Image: typeof import('./components/Image.astro').default;
};

type WithRequired<T, K extends keyof T> = T & { [P in K]-?: T[P] };
type Simplify<T> = { [KeyType in keyof T]: T[KeyType] };
type ImgAttributes = WithRequired<
Omit<import('./types').HTMLAttributes<'img'>, 'src' | 'width' | 'height'>,
'alt'
>;

export type LocalImageProps = Simplify<
import('./dist/assets/types.js').LocalImageProps<ImgAttributes>
>;
export type RemoteImageProps = Simplify<
import('./dist/assets/types.js').RemoteImageProps<ImgAttributes>
>;
export const { getImage, Image }: AstroAssets;
}

type MD = import('./dist/@types/astro').MarkdownInstance<Record<string, any>>;
interface ExportedMarkdownModuleEntities {
frontmatter: MD['frontmatter'];
Expand Down
48 changes: 48 additions & 0 deletions packages/astro/client-image.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/// <reference path="./client-base.d.ts" />

type InputFormat = 'avif' | 'gif' | 'heic' | 'heif' | 'jpeg' | 'jpg' | 'png' | 'tiff' | 'webp';

interface ImageMetadata {
src: string;
width: number;
height: number;
format: InputFormat;
}

// images
declare module '*.avif' {
const metadata: ImageMetadata;
export default metadata;
}
declare module '*.gif' {
const metadata: ImageMetadata;
export default metadata;
}
declare module '*.heic' {
const metadata: ImageMetadata;
export default metadata;
}
declare module '*.heif' {
const metadata: ImageMetadata;
export default metadata;
}
declare module '*.jpeg' {
const metadata: ImageMetadata;
export default metadata;
}
declare module '*.jpg' {
const metadata: ImageMetadata;
export default metadata;
}
declare module '*.png' {
const metadata: ImageMetadata;
export default metadata;
}
declare module '*.tiff' {
const metadata: ImageMetadata;
export default metadata;
}
declare module '*.webp' {
const metadata: ImageMetadata;
export default metadata;
}
28 changes: 28 additions & 0 deletions packages/astro/components/Image.astro
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
import { getImage, type LocalImageProps, type RemoteImageProps } from 'astro:assets';
import { AstroError, AstroErrorData } from '../dist/core/errors/index.js';
// The TypeScript diagnostic for JSX props uses the last member of the union to suggest props, so it would be better for
// LocalImageProps to be last. Unfortunately, when we do this the error messages that remote images get are complete nonsense
// Not 100% sure how to fix this, seems to be a TypeScript issue. Unfortunate.
type Props = LocalImageProps | RemoteImageProps;
const props = Astro.props;
if (props.alt === undefined || props.alt === null) {
throw new AstroError(AstroErrorData.ImageMissingAlt);
}
// As a convenience, allow width and height to be string with a number in them, to match HTML's native `img`.
if (typeof props.width === 'string') {
props.width = parseInt(props.width);
}
if (typeof props.height === 'string') {
props.height = parseInt(props.height);
}
const image = await getImage(props);
---

<img src={image.src} {...image.attributes} />
23 changes: 20 additions & 3 deletions packages/astro/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
"./types": "./types.d.ts",
"./client": "./client.d.ts",
"./client-base": "./client-base.d.ts",
"./client-image": "./client-image.d.ts",
"./import-meta": "./import-meta.d.ts",
"./astro-jsx": "./astro-jsx.d.ts",
"./tsconfigs/*.json": "./tsconfigs/*",
Expand All @@ -47,6 +48,10 @@
"./client/*": "./dist/runtime/client/*",
"./components": "./components/index.ts",
"./components/*": "./components/*",
"./assets": "./dist/assets/index.js",
"./assets/image-endpoint": "./dist/assets/image-endpoint.js",
"./assets/services/sharp": "./dist/assets/services/sharp.js",
"./assets/services/squoosh": "./dist/assets/services/squoosh.js",
"./content/internal": "./dist/content/internal.js",
"./debug": "./components/Debug.astro",
"./internal/*": "./dist/runtime/server/*",
Expand Down Expand Up @@ -77,6 +82,7 @@
"env.d.ts",
"client.d.ts",
"client-base.d.ts",
"client-image.d.ts",
"import-meta.d.ts",
"astro-jsx.d.ts",
"types.d.ts",
Expand All @@ -86,10 +92,10 @@
],
"scripts": {
"prebuild": "astro-scripts prebuild --to-string \"src/runtime/server/astro-island.ts\" \"src/runtime/client/{idle,load,media,only,visible}.ts\"",
"build": "pnpm run prebuild && astro-scripts build \"src/**/*.ts\" && tsc",
"build": "pnpm run prebuild && astro-scripts build \"src/**/*.ts\" && tsc && pnpm run postbuild",
"build:ci": "pnpm run prebuild && astro-scripts build \"src/**/*.ts\"",
"dev": "astro-scripts dev --prebuild \"src/runtime/server/astro-island.ts\" --prebuild \"src/runtime/client/{idle,load,media,only,visible}.ts\" \"src/**/*.ts\"",
"postbuild": "astro-scripts copy \"src/**/*.astro\"",
"dev": "astro-scripts dev --copy-wasm --prebuild \"src/runtime/server/astro-island.ts\" --prebuild \"src/runtime/client/{idle,load,media,only,visible}.ts\" \"src/**/*.ts\"",
"postbuild": "astro-scripts copy \"src/**/*.astro\" && astro-scripts copy \"src/**/*.wasm\"",
"test:unit": "mocha --exit --timeout 30000 ./test/units/**/*.test.js",
"test:unit:match": "mocha --exit --timeout 30000 ./test/units/**/*.test.js -g",
"test": "pnpm run test:unit && mocha --exit --timeout 20000 --ignore **/lit-element.test.js && mocha --timeout 20000 **/lit-element.test.js",
Expand Down Expand Up @@ -128,6 +134,7 @@
"github-slugger": "^2.0.0",
"gray-matter": "^4.0.3",
"html-escaper": "^3.0.3",
"image-size": "^1.0.2",
"kleur": "^4.1.4",
"magic-string": "^0.27.0",
"mime": "^3.0.0",
Expand Down Expand Up @@ -173,6 +180,7 @@
"@types/rimraf": "^3.0.2",
"@types/send": "^0.17.1",
"@types/server-destroy": "^1.0.1",
"@types/sharp": "^0.31.1",
"@types/unist": "^2.0.6",
"astro-scripts": "workspace:*",
"chai": "^4.3.6",
Expand All @@ -187,10 +195,19 @@
"remark-code-titles": "^0.1.2",
"rollup": "^3.9.0",
"sass": "^1.52.2",
"sharp": "^0.31.3",
"srcset-parse": "^1.1.0",
"undici": "^5.20.0",
"unified": "^10.1.2"
},
"peerDependencies": {
"sharp": "^0.31.3"
},
"peerDependenciesMeta": {
"sharp": {
"optional": true
}
},
"engines": {
"node": ">=16.12.0",
"npm": ">=6.14.0"
Expand Down
41 changes: 40 additions & 1 deletion packages/astro/src/@types/astro.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import type { PageBuildData } from '../core/build/types';
import type { AstroConfigSchema } from '../core/config';
import type { AstroTimer } from '../core/config/timer';
import type { AstroCookies } from '../core/cookies';
import type { LogOptions } from '../core/logger/core';
import type { AstroComponentFactory, AstroComponentInstance } from '../runtime/server';
import { SUPPORTED_MARKDOWN_FILE_EXTENSIONS } from './../core/constants.js';
export type {
Expand All @@ -28,6 +29,8 @@ export type {
RemarkPlugins,
ShikiConfig,
} from '@astrojs/markdown-remark';
export type { ExternalImageService, LocalImageService } from '../assets/services/service';
export type { ImageTransform } from '../assets/types';
export type { SSRManifest } from '../core/app/types';
export type { AstroCookies } from '../core/cookies';

Expand Down Expand Up @@ -85,6 +88,7 @@ export interface CLIFlags {
port?: number;
config?: string;
drafts?: boolean;
experimentalAssets?: boolean;
}

export interface BuildConfig {
Expand Down Expand Up @@ -696,6 +700,16 @@ export interface AstroUserConfig {

server?: ServerConfig | ((options: { command: 'dev' | 'preview' }) => ServerConfig);

/**
* @docs
* @kind heading
* @name Image options
*/
image?: {
// eslint-disable-next-line @typescript-eslint/ban-types
service: 'astro/assets/services/sharp' | 'astro/assets/services/squoosh' | (string & {});
};

/**
* @docs
* @kind heading
Expand Down Expand Up @@ -918,7 +932,27 @@ export interface AstroUserConfig {
* Astro offers experimental flags to give users early access to new features.
* These flags are not guaranteed to be stable.
*/
experimental?: object;
experimental?: {
/**
* @docs
* @name experimental.assets
* @type {boolean}
* @default `false`
* @version 2.1.0
* @description
* Enable experimental support for optimizing and resizing images. With this enabled, a new `astro:assets` module will be exposed.
*
* To enable this feature, set `experimental.assets` to `true` in your Astro config:
*
* ```js
* {
* experimental: {
* assets: true,
* },
* }
*/
assets?: boolean;
};

// Legacy options to be removed

Expand Down Expand Up @@ -1432,6 +1466,11 @@ export interface AstroIntegration {
};
}

export interface AstroPluginOptions {
settings: AstroSettings;
logging: LogOptions;
}

export type RouteType = 'page' | 'endpoint';

export interface RoutePart {
Expand Down
3 changes: 3 additions & 0 deletions packages/astro/src/assets/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# assets

This directory powers the Assets story in Astro. Notably, it contains all the code related to optimizing images and serving them in the different modes Astro can run in (SSG, SSR, dev, build etc).
14 changes: 14 additions & 0 deletions packages/astro/src/assets/consts.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
export const VIRTUAL_MODULE_ID = 'astro:assets';
export const VIRTUAL_SERVICE_ID = 'virtual:image-service';
export const VALID_INPUT_FORMATS = [
'heic',
'heif',
'avif',
'jpeg',
'jpg',
'png',
'tiff',
'webp',
'gif',
] as const;
export const VALID_OUTPUT_FORMATS = ['avif', 'png', 'webp', 'jpeg', 'jpg'] as const;
66 changes: 66 additions & 0 deletions packages/astro/src/assets/image-endpoint.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
import mime from 'mime';
import type { APIRoute } from '../@types/astro.js';
import { isRemotePath } from '../core/path.js';
import { getConfiguredImageService } from './internal.js';
import { isLocalService } from './services/service.js';
import { etag } from './utils/etag.js';

async function loadRemoteImage(src: URL) {
try {
const res = await fetch(src);

if (!res.ok) {
return undefined;
}

return Buffer.from(await res.arrayBuffer());
} catch (err: unknown) {
return undefined;
}
}

/**
* Endpoint used in SSR to serve optimized images
*/
export const get: APIRoute = async ({ request }) => {
try {
const imageService = await getConfiguredImageService();

if (!isLocalService(imageService)) {
throw new Error('Configured image service is not a local service');
}

const url = new URL(request.url);
const transform = await imageService.parseURL(url);

if (!transform || !transform.src) {
throw new Error('Incorrect transform returned by `parseURL`');
}

let inputBuffer: Buffer | undefined = undefined;

// TODO: handle config subpaths?
const sourceUrl = isRemotePath(transform.src)
? new URL(transform.src)
: new URL(transform.src, url.origin);
inputBuffer = await loadRemoteImage(sourceUrl);

if (!inputBuffer) {
return new Response('Not Found', { status: 404 });
}

const { data, format } = await imageService.transform(inputBuffer, transform);

return new Response(data, {
status: 200,
headers: {
'Content-Type': mime.getType(format) || '',
'Cache-Control': 'public, max-age=31536000',
ETag: etag(data.toString()),
Date: new Date().toUTCString(),
},
});
} catch (err: unknown) {
return new Response(`Server Error: ${err}`, { status: 500 });
}
};
4 changes: 4 additions & 0 deletions packages/astro/src/assets/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
export { getConfiguredImageService, getImage } from './internal.js';
export { baseService } from './services/service.js';
export { type LocalImageProps, type RemoteImageProps } from './types.js';
export { imageMetadata } from './utils/metadata.js';
Loading

0 comments on commit 694918a

Please sign in to comment.