Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: remote srcset images not being resized and deduplication not working in certain cases #8823

Merged
merged 12 commits into from
Oct 16, 2023

Conversation

Princesseuh
Copy link
Member

@Princesseuh Princesseuh commented Oct 13, 2023

Changes

Continuation of #8819

This PR also reworks completely how we do the transforms for srcset, putting special care into avoiding regenerating images when unnecessary. It's possible that there's still some edge cases where it doesn't work, nonetheless as far as I could tell, it does work!

This also fixes #8839, since it's in the same area.

Testing

Added tests!

Docs

N/A

@changeset-bot
Copy link

changeset-bot bot commented Oct 13, 2023

🦋 Changeset detected

Latest commit: b0444b9

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions github-actions bot added the pkg: astro Related to the core `astro` package (scope) label Oct 13, 2023
Comment on lines +268 to +269
// Caution: The logic below is a bit tricky, as we need to make sure we don't generate the same image multiple times
// When making changes, make sure to test with different combinations of local/remote images widths, densities, and dimensions etc.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a "There be dragons" comment here because it's honestly super tricky to completely avoid making transforms that are functionally equivalent to another one, but don't have quite the exact same shape.

Common examples includes:

  • A transform where webp is specified manually vs one where it's not. Due to the default, both of those results in the same image.
  • A transform where width is specified, but it's equivalent to the width of the image, so it's not resized.

@@ -1,3 +1,4 @@
import { deterministicString } from 'deterministic-object-hash';
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found this package in the backwoods somewhere in a random stack overflow thread, but it's honestly quite nice and does everything I wanted.

@Princesseuh
Copy link
Member Author

!preview srcset-remote

@github-actions
Copy link
Contributor

 > [email protected] release /home/runner/work/astro/astro > pnpm run build && changeset publish "--tag" "next--srcset-remote" > [email protected] build /home/runner/work/astro/astro > turbo run build --filter=astro --filter=create-astro --filter="@astrojs/*" --filter="@benchmark/*" �[2m• Packages in scope: @astrojs/alpinejs, @astrojs/cloudflare, @astrojs/internal-helpers, @astrojs/lit, @astrojs/markdoc, @astrojs/markdown-remark, @astrojs/mdx, @astrojs/netlify, @astrojs/node, @astrojs/partytown, @astrojs/preact, @astrojs/prefetch, @astrojs/prism, @astrojs/react, @astrojs/rss, @astrojs/sitemap, @astrojs/solid-js, @astrojs/svelte, @astrojs/tailwind, @astrojs/telemetry, @astrojs/underscore-redirects, @astrojs/vercel, @astrojs/vue, @benchmark/timer, astro, create-astro�[0m �[2m• Running�[0m �[2m�[1mbuild�[0m�[0m �[2min 26 packages�[0m �[2m• Remote caching enabled�[0m ::group::create-astro:build cache hit, suppressing logs �[2mb92067e8d3d6f572�[0m ::endgroup:: ::group::@astrojs/prism:build cache hit, suppressing logs �[2m1fbe4acc344b9100�[0m ::endgroup:: ::group::@astrojs/telemetry:build cache hit, suppressing logs �[2m8f791e5bb7108651�[0m ::endgroup:: ::group::@astrojs/internal-helpers:build cache hit, suppressing logs �[2mc6ad1a8f27dca66f�[0m ::endgroup:: ::group::@astrojs/markdown-remark:build cache hit, suppressing logs �[2m5ea7eda930ba6d10�[0m ::endgroup:: ::group::astro:build cache miss, executing �[2m8b861c93f39de5f8�[0m > [email protected] build /home/runner/work/astro/astro/packages/astro > pnpm run prebuild && astro-scripts build "src/**/*.{ts,js}" && tsc && pnpm run postbuild > [email protected] prebuild /home/runner/work/astro/astro/packages/astro > astro-scripts prebuild --to-string "src/runtime/server/astro-island.ts" "src/runtime/client/{idle,load,media,only,visible}.ts" > [email protected] postbuild /home/runner/work/astro/astro/packages/astro > astro-scripts copy "src/**/*.astro" && astro-scripts copy "src/**/*.wasm" ::endgroup:: ::group::@astrojs/tailwind:build cache miss, executing �[2m0ec0df837e5ffb85�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/tailwind > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/underscore-redirects:build cache miss, executing �[2m8c3985125f1a1221�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/underscore-redirects > astro-scripts build "src/**/*.ts" && tsc -p tsconfig.json ::endgroup:: ::group::@astrojs/lit:build cache miss, executing �[2m6bc80285972f9a92�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/lit > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/react:build cache miss, executing �[2mde1478f2c5dab7e7�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/react > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/rss:build cache miss, executing �[2m07da25e1ec31ea55�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/astro-rss > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/partytown:build cache miss, executing �[2m06882c90fde91ad6�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/partytown > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/prefetch:build cache miss, executing �[2m3c27b323ff1d6d6e�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/prefetch > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/preact:build cache miss, executing �[2mfcc32fe2cbf856e6�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/preact > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/node:build cache miss, executing �[2m7d93dde09ca2f278�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/node > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/vue:build cache miss, executing �[2me77dfa12a81f8ede�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/vue > astro-scripts build "src/index.ts" && astro-scripts build "src/editor.cts" --force-cjs --no-clean-dist && tsc ::endgroup:: ::group::@benchmark/timer:build cache miss, executing �[2m57a2c68073b64c63�[0m > @benchmark/[email protected] build /home/runner/work/astro/astro/benchmark/packages/timer > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/alpinejs:build cache miss, executing �[2me5b144bf93d88e65�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/alpinejs > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/vercel:build cache miss, executing �[2med75224702f9b7ac�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/vercel > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/solid-js:build cache miss, executing �[2m7183c1b61c159953�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/solid > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/markdoc:build cache miss, executing �[2md3793fc03b0bb9a2�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/markdoc > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/mdx:build cache miss, executing �[2m486c2e9df3a15001�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/mdx > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/sitemap:build cache miss, executing �[2mcdb1eaea46aba9f8�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/sitemap > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: ::group::@astrojs/svelte:build cache miss, executing �[2m039fd552cd20e89f�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/svelte > astro-scripts build "src/index.ts" && astro-scripts build "src/editor.cts" --force-cjs --no-clean-dist && tsc ::endgroup:: ::group::@astrojs/cloudflare:build cache miss, executing �[2m98806a2b29c05852�[0m > @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/cloudflare > astro-scripts build "src/**/*.ts" && tsc ::endgroup:: Tasks: 25 successful, 25 total Cached: 5 cached, 25 total Time: 1m6.61s 🦋 �[33mwarn�[39m �[31m===============================IMPORTANT!===============================�[39m 🦋 �[33mwarn�[39m Packages will be released under the next--srcset-remote tag 🦋 �[33mwarn�[39m �[31m----------------------------------------------------------------------�[39m 🦋 �[36minfo�[39m npm info astro 🦋 �[36minfo�[39m npm info @astrojs/prism 🦋 �[36minfo�[39m npm info @astrojs/rss 🦋 �[36minfo�[39m npm info create-astro 🦋 �[36minfo�[39m npm info @astrojs/alpinejs 🦋 �[36minfo�[39m npm info @astrojs/cloudflare 🦋 �[36minfo�[39m npm info @astrojs/lit 🦋 �[36minfo�[39m npm info @astrojs/markdoc 🦋 �[36minfo�[39m npm info @astrojs/mdx 🦋 �[36minfo�[39m npm info @astrojs/node 🦋 �[36minfo�[39m npm info @astrojs/partytown 🦋 �[36minfo�[39m npm info @astrojs/preact 🦋 �[36minfo�[39m npm info @astrojs/prefetch 🦋 �[36minfo�[39m npm info @astrojs/react 🦋 �[36minfo�[39m npm info @astrojs/sitemap 🦋 �[36minfo�[39m npm info @astrojs/solid-js 🦋 �[36minfo�[39m npm info @astrojs/svelte 🦋 �[36minfo�[39m npm info @astrojs/tailwind 🦋 �[36minfo�[39m npm info @astrojs/vercel 🦋 �[36minfo�[39m npm info @astrojs/vue 🦋 �[36minfo�[39m npm info @astrojs/internal-helpers 🦋 �[36minfo�[39m npm info @astrojs/markdown-remark 🦋 �[36minfo�[39m npm info @astrojs/telemetry 🦋 �[36minfo�[39m npm info @astrojs/underscore-redirects 🦋 �[36minfo�[39m astro is being published because our local version (0.0.0-srcset-remote-20231013141048) has not been published on npm 🦋 �[33mwarn�[39m @astrojs/prism is not being published because version 3.0.0 is already published on npm 🦋 �[33mwarn�[39m @astrojs/rss is not being published because version 3.0.0 is already published on npm 🦋 �[33mwarn�[39m create-astro is not being published because version 4.2.1 is already published on npm 🦋 �[33mwarn�[39m @astrojs/alpinejs is not being published because version 0.3.1 is already published on npm 🦋 �[33mwarn�[39m @astrojs/cloudflare is not being published because version 7.5.3 is already published on npm 🦋 �[33mwarn�[39m @astrojs/lit is not being published because version 3.0.1 is already published on npm 🦋 �[33mwarn�[39m @astrojs/markdoc is not being published because version 0.6.0 is already published on npm 🦋 �[33mwarn�[39m @astrojs/mdx is not being published because version 1.1.2 is already published on npm 🦋 �[33mwarn�[39m @astrojs/node is not being published because version 6.0.3 is already published on npm 🦋 �[33mwarn�[39m @astrojs/partytown is not being published because version 2.0.1 is already published on npm 🦋 �[33mwarn�[39m @astrojs/preact is not being published because version 3.0.1 is already published on npm 🦋 �[33mwarn�[39m @astrojs/prefetch is not being published because version 0.4.1 is already published on npm 🦋 �[33mwarn�[39m @astrojs/react is not being published because version 3.0.3 is already published on npm 🦋 �[33mwarn�[39m @astrojs/sitemap is not being published because version 3.0.1 is already published on npm 🦋 �[33mwarn�[39m @astrojs/solid-js is not being published because version 3.0.2 is already published on npm 🦋 �[33mwarn�[39m @astrojs/svelte is not being published because version 4.0.3 is already published on npm 🦋 �[33mwarn�[39m @astrojs/tailwind is not being published because version 5.0.2 is already published on npm 🦋 �[33mwarn�[39m @astrojs/vercel is not being published because version 5.0.2 is already published on npm 🦋 �[33mwarn�[39m @astrojs/vue is not being published because version 3.0.1 is already published on npm 🦋 �[33mwarn�[39m @astrojs/internal-helpers is not being published because version 0.2.1 is already published on npm 🦋 �[33mwarn�[39m @astrojs/markdown-remark is not being published because version 3.3.0 is already published on npm 🦋 �[33mwarn�[39m @astrojs/telemetry is not being published because version 3.0.3 is already published on npm 🦋 �[33mwarn�[39m @astrojs/underscore-redirects is not being published because version 0.3.1 is already published on npm 🦋 �[36minfo�[39m Publishing �[36m"astro"�[39m at �[32m"0.0.0-srcset-remote-20231013141048"�[39m 🦋 �[32msuccess�[39m packages published successfully: 🦋 [email protected] 🦋 Creating git tag... 🦋 New tag: [email protected]

@Princesseuh Princesseuh marked this pull request as ready for review October 13, 2023 17:06
@Princesseuh Princesseuh changed the title fix: remote srcset images not being resized fix: remote srcset images not being resized and deduplication not working in certain cases Oct 13, 2023
@itsmatteomanf
Copy link
Contributor

Is there any chance to know when is this gonna get merged? It's a major bug fix on the newly announced feature...

I know this issue exists so I won't use srcset yet, but I'd really like to ahah

@Princesseuh
Copy link
Member Author

Is there any chance to know when is this gonna get merged? It's a major bug fix on the newly announced feature...

I know this issue exists so I won't use srcset yet, but I'd really like to ahah

We'll release it Monday morning CET, it needs to get reviewed and we don't typically do releases on the weekend

Copy link
Member

@ematipico ematipico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic is very complex, I can't really understand it.

packages/astro/src/assets/services/service.ts Outdated Show resolved Hide resolved
packages/astro/src/assets/services/service.ts Outdated Show resolved Hide resolved
packages/astro/src/assets/services/service.ts Outdated Show resolved Hide resolved
packages/astro/src/assets/services/service.ts Outdated Show resolved Hide resolved
@Princesseuh
Copy link
Member Author

The logic is very complex, I can't really understand it.

Sorry, I tried making it more readable in this PR than the initial code, but it's kinda hard to make neat. Images are complicated 😔

I added a bunch of comments to clarify some parts, in addition to clarifying the points you commented on, hope it helps.

@ematipico
Copy link
Member

It's okay, you did a great job! Thank you for addressing my comments. Feel free to merge it whenever you want. I don't have big concerns

@Princesseuh Princesseuh merged commit 8946f2a into main Oct 16, 2023
@Princesseuh Princesseuh deleted the fix-srcset-remote branch October 16, 2023 19:02
@astrobot-houston astrobot-houston mentioned this pull request Oct 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pkg: astro Related to the core `astro` package (scope)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Generated srcset attribute puts the largest image on the top of the list
3 participants