Skip to content

Commit

Permalink
Image integration refactor and cleanup (#4482)
Browse files Browse the repository at this point in the history
* WIP: simplifying the use of `fs` vs. the vite plugin

* removing a few node deps (etag and node:path)

* adding ts defs for sharp

* using the same mime package as astro's core App

* fixing file URL support in windows

* using file URLs when loading local image metadata

* fixing a bug in the etag helper

* Windows compat

* splitting out dev & build tests

* why do these suites fail in parallel?

* one last windows compat case

* Adding tests for treating /public images the same as remote URLs

* a couple fixes for Astro's `base` config

* adding base path tests for SSR

* fixing a bad merge, lost the kleur dependency

* adding a test suite for images + MDX

* chore: add changeset

* simplifying the with-mdx tests

* bugfix: don't duplicate the period when using existing file extensions

* let Vite cache the image loader service

* adding some docs for using /public images

* fixing changeset

* Update packages/integrations/image/README.md

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update packages/integrations/image/README.md

Co-authored-by: Sarah Rainsberger <[email protected]>

* nit: minor README syntax tweaks

Co-authored-by: Sarah Rainsberger <[email protected]>
  • Loading branch information
Tony Sullivan and sarah11918 authored Aug 30, 2022
1 parent 7429664 commit 00c605c
Show file tree
Hide file tree
Showing 42 changed files with 2,211 additions and 763 deletions.
12 changes: 12 additions & 0 deletions .changeset/lucky-mirrors-type.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
'@astrojs/image': minor
---

`<Image />` and `<Picture />` now support using images in the `/public` directory :tada:

- Moving handling of local image files into the Vite plugin
- Optimized image files are now built to `/dist` with hashes provided by Vite, removing the need for a `/dist/_image` directory
- Removes three npm dependencies: `etag`, `slash`, and `tiny-glob`
- Replaces `mrmime` with the `mime` package already used by Astro's SSR server
- Simplifies the injected `_image` route to work for both `dev` and `build`
- Adds a new test suite for using images with `@astrojs/mdx` - including optimizing images straight from `/public`
26 changes: 24 additions & 2 deletions packages/integrations/image/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,11 @@ In addition to the component-specific properties, any valid HTML attribute for t

Source for the original image file.

For images in your project's repository, use the `src` relative to the `public` directory. For remote images, provide the full URL.
For images located in your project's `src`: use the file path relative to the `src` directory. (e.g. `src="../assets/source-pic.png"`)

For images located in your `public` directory: use the URL path relative to the `public` directory. (e.g. `src="/images/public-image.jpg"`)

For remote images, provide the full URL. (e.g. `src="https://astro.build/assets/blog/astro-1-release-update.avif"`)

#### format

Expand Down Expand Up @@ -182,7 +186,7 @@ A `number` can also be provided, useful when the aspect ratio is calculated at b

Source for the original image file.

For images in your project's repository, use the `src` relative to the `public` directory. For remote images, provide the full URL.
For images in your project's repository, use the path relative to the `src` or `public` directory. For remote images, provide the full URL.

#### alt

Expand Down Expand Up @@ -341,6 +345,24 @@ import heroImage from '../assets/hero.png';
<Image src={import('../assets/hero.png')} />
```

#### Images in `/public`

Files in the `/public` directory are always served or copied as-is, with no processing. We recommend that local images are always kept in `src/` so that Astro can transform, optimize and bundle them. But if you absolutely must keep an image in `public/`, use its relative URL path as the image's `src=` attribute. It will be treated as a remote image, which requires an `aspectRatio` attribute.

Alternatively, you can import an image from your `public/` directory in your frontmatter and use a variable in your `src=` attribute. You cannot, however, import this directly inside the component as its `src` value.

For example, use an image located at `public/social.png` in either static or SSR builds like so:

```astro title="src/pages/page.astro"
---
import { Image } from '@astrojs/image/components';
import socialImage from '/social.png';
---
// In static builds: the image will be built and optimized to `/dist`.
// In SSR builds: the image will be optimized by the server when requested by a browser.
<Image src={socialImage} width={1280} aspectRatio="16:9" />
```

### Remote images

Remote images can be transformed with the `<Image />` component. The `<Image />` component needs to know the final dimensions for the `<img />` element to avoid content layout shifts. For remote images, this means you must either provide `width` and `height`, or one of the dimensions plus the required `aspectRatio`.
Expand Down
19 changes: 7 additions & 12 deletions packages/integrations/image/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,8 @@
"homepage": "https://docs.astro.build/en/guides/integrations-guide/image/",
"exports": {
".": "./dist/index.js",
"./endpoint": "./dist/endpoint.js",
"./sharp": "./dist/loaders/sharp.js",
"./endpoints/dev": "./dist/endpoints/dev.js",
"./endpoints/prod": "./dist/endpoints/prod.js",
"./components": "./components/index.js",
"./package.json": "./package.json",
"./client": "./client.d.ts",
Expand All @@ -41,19 +40,15 @@
"test": "mocha --exit --timeout 20000 test"
},
"dependencies": {
"etag": "^1.8.1",
"image-size": "^1.0.1",
"mrmime": "^1.0.0",
"sharp": "^0.30.6",
"slash": "^4.0.0",
"tiny-glob": "^0.2.9"
"image-size": "^1.0.2",
"magic-string": "^0.25.9",
"mime": "^3.0.0",
"sharp": "^0.30.6"
},
"devDependencies": {
"@types/etag": "^1.8.1",
"@types/sharp": "^0.30.4",
"@types/sharp": "^0.30.5",
"astro": "workspace:*",
"astro-scripts": "workspace:*",
"kleur": "^4.1.4",
"tiny-glob": "^0.2.9"
"kleur": "^4.1.4"
}
}
42 changes: 15 additions & 27 deletions packages/integrations/image/src/build/ssg.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ import { bgGreen, black, cyan, dim, green } from 'kleur/colors';
import fs from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { OUTPUT_DIR } from '../constants.js';
import type { AstroConfig } from 'astro';
import type { SSRImageService, TransformOptions } from '../loaders/index.js';
import { isRemoteImage, loadLocalImage, loadRemoteImage } from '../utils/images.js';
import { loadLocalImage, loadRemoteImage } from '../utils/images.js';
import { debug, info, LoggerLevel, warn } from '../utils/logger.js';
import { ensureDir } from '../utils/paths.js';
import { isRemoteImage } from '../utils/paths.js';

function getTimeStat(timeStart: number, timeEnd: number) {
const buildTime = timeEnd - timeStart;
Expand All @@ -16,12 +16,12 @@ function getTimeStat(timeStart: number, timeEnd: number) {
export interface SSGBuildParams {
loader: SSRImageService;
staticImages: Map<string, Map<string, TransformOptions>>;
srcDir: URL;
config: AstroConfig;
outDir: URL;
logLevel: LoggerLevel;
}

export async function ssgBuild({ loader, staticImages, srcDir, outDir, logLevel }: SSGBuildParams) {
export async function ssgBuild({ loader, staticImages, config, outDir, logLevel }: SSGBuildParams) {
const timer = performance.now();

info({
Expand All @@ -35,15 +35,21 @@ export async function ssgBuild({ loader, staticImages, srcDir, outDir, logLevel
const inputFiles = new Set<string>();

// process transforms one original image file at a time
for (const [src, transformsMap] of staticImages) {
for (let [src, transformsMap] of staticImages) {
let inputFile: string | undefined = undefined;
let inputBuffer: Buffer | undefined = undefined;

// Vite will prefix a hashed image with the base path, we need to strip this
// off to find the actual file relative to /dist
if (config.base && src.startsWith(config.base)) {
src = src.substring(config.base.length - 1);
}

if (isRemoteImage(src)) {
// try to load the remote image
inputBuffer = await loadRemoteImage(src);
} else {
const inputFileURL = new URL(`.${src}`, srcDir);
const inputFileURL = new URL(`.${src}`, outDir);
inputFile = fileURLToPath(inputFileURL);
inputBuffer = await loadLocalImage(inputFile);

Expand All @@ -62,39 +68,21 @@ export async function ssgBuild({ loader, staticImages, srcDir, outDir, logLevel
debug({ level: logLevel, prefix: false, message: `${green('▶')} ${src}` });
let timeStart = performance.now();

if (inputFile) {
const to = inputFile.replace(fileURLToPath(srcDir), fileURLToPath(outDir));
await ensureDir(path.dirname(to));
await fs.copyFile(inputFile, to);

const timeEnd = performance.now();
const timeChange = getTimeStat(timeStart, timeEnd);
const timeIncrease = `(+${timeChange})`;
const pathRelative = inputFile.replace(fileURLToPath(srcDir), '');
debug({
level: logLevel,
prefix: false,
message: ` ${cyan('└─')} ${dim(`(original) ${pathRelative}`)} ${dim(timeIncrease)}`,
});
}

// process each transformed versiono of the
for (const [filename, transform] of transforms) {
timeStart = performance.now();
let outputFile: string;

if (isRemoteImage(src)) {
const outputFileURL = new URL(path.join('./', OUTPUT_DIR, path.basename(filename)), outDir);
const outputFileURL = new URL(path.join('./', path.basename(filename)), outDir);
outputFile = fileURLToPath(outputFileURL);
} else {
const outputFileURL = new URL(path.join('./', OUTPUT_DIR, filename), outDir);
const outputFileURL = new URL(path.join('./', filename), outDir);
outputFile = fileURLToPath(outputFileURL);
}

const { data } = await loader.transform(inputBuffer, transform);

ensureDir(path.dirname(outputFile));

await fs.writeFile(outputFile, data);

const timeEnd = performance.now();
Expand Down
29 changes: 0 additions & 29 deletions packages/integrations/image/src/build/ssr.ts

This file was deleted.

3 changes: 0 additions & 3 deletions packages/integrations/image/src/constants.ts

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,45 +1,53 @@
import type { APIRoute } from 'astro';
import etag from 'etag';
import { lookup } from 'mrmime';
import mime from 'mime';
// @ts-ignore
import loader from 'virtual:image-loader';
import { isRemoteImage, loadLocalImage, loadRemoteImage } from '../utils/images.js';
import { etag } from './utils/etag.js';
import { isRemoteImage } from './utils/paths.js';

async function loadRemoteImage(src: URL) {
try {
const res = await fetch(src);

if (!res.ok) {
return undefined;
}

return Buffer.from(await res.arrayBuffer());
} catch {
return undefined;
}
}

export const get: APIRoute = async ({ request }) => {
try {
const url = new URL(request.url);
const transform = loader.parseTransform(url.searchParams);

if (!transform) {
return new Response('Bad Request', { status: 400 });
}

let inputBuffer: Buffer | undefined = undefined;

if (isRemoteImage(transform.src)) {
inputBuffer = await loadRemoteImage(transform.src);
} else {
const clientRoot = new URL('../client/', import.meta.url);
const localPath = new URL('.' + transform.src, clientRoot);
inputBuffer = await loadLocalImage(localPath);
}
// TODO: handle config subpaths?
const sourceUrl = isRemoteImage(transform.src)
? new URL(transform.src)
: new URL(transform.src, url.origin);
inputBuffer = await loadRemoteImage(sourceUrl);

if (!inputBuffer) {
return new Response(`"${transform.src} not found`, { status: 404 });
return new Response('Not Found', { status: 404 });
}

const { data, format } = await loader.transform(inputBuffer, transform);

return new Response(data, {
status: 200,
headers: {
'Content-Type': lookup(format) || '',
'Content-Type': mime.getType(format) || '',
'Cache-Control': 'public, max-age=31536000',
ETag: etag(inputBuffer),
ETag: etag(data.toString()),
Date: new Date().toUTCString(),
},
});
} catch (err: unknown) {
return new Response(`Server Error: ${err}`, { status: 500 });
}
};
}
32 changes: 0 additions & 32 deletions packages/integrations/image/src/endpoints/dev.ts

This file was deleted.

Loading

0 comments on commit 00c605c

Please sign in to comment.