Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions docs/contributing/architecture/data-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,11 +223,14 @@ descriptor from `BUNDLE_ARTIFACTS_KV` and streams the referenced R2 object.
Source files remain in the listing's pinned Artifacts commit; R2 stores only
validated derived output or a generated fallback.

- Keys use `community-icon:v1/{listingId}/{pinnedCommit}/asset`.
- Descriptor keys include the same listing id and pinned commit, so re-publish
cannot serve an older icon.
- Unpublish, admin hard delete, re-publish, and account deletion remove known
descriptor and object keys.
- Keys use `community-icon:v1/{listingId}/{commit}/asset`, where the commit is
the listing's icon commit (the owner package's current published commit) or
its pinned snapshot commit.
- Descriptor keys include the same listing id and commit, so package publish and
listing re-publish cannot serve an older icon.
- Unpublish, admin hard delete, and re-publish prune all descriptor and object
keys under the listing prefix; package publish prunes superseded commits;
account deletion removes the pinned and icon commit keys derived from D1.
- Bucket names are `kody-community-assets` in production and
`{worker}-community-assets` for preview deployments.

Expand Down Expand Up @@ -585,9 +588,10 @@ or a deliberate retention note.

App-owned R2 keys are:

- `community-icon:v1/{listingId}/{pinnedCommit}/asset` — processed public
community icon bytes. The listing id is the public ownership boundary; account
deletion derives keys from the owner's listing rows.
- `community-icon:v1/{listingId}/{commit}/asset` — processed public community
icon bytes at the listing's pinned or icon commit. The listing id is the
public ownership boundary; account deletion derives keys from the owner's
listing rows and their joined package source commits.

- `email-raw:v1:{userId}/{messageId}` — raw email MIME for the message row that
stores this key in `email_messages.raw_mime_key`. The `userId` prefix is part
Expand Down
10 changes: 6 additions & 4 deletions docs/contributing/architecture/primitives.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ primitives:
Public pinned snapshots of published saved packages on the same
deployment; inert forks, ratings, user reports, admin moderation at
/admin/community-reports, /community routes, and lazily derived package
icons pinned to the published commit.
icons that track the owner package's current published commit.
code:
- packages/worker/src/community/
- packages/worker/src/mcp/capabilities/community/
Expand Down Expand Up @@ -513,9 +513,11 @@ primitives:
name: Community asset storage (R2)
summary:
COMMUNITY_ASSETS R2 bucket holding validated, lazily derived community
icon bytes under community-icon:v1/{listingId}/{pinnedCommit}/asset.
Cachified descriptors live in Bundle artifacts KV; public reads remain
gated by an active listing and account deletion removes owner assets.
icon bytes under community-icon:v1/{listingId}/{commit}/asset, keyed by
the listing's pinned or current icon commit. Cachified descriptors live in
Bundle artifacts KV; public reads remain gated by an active listing,
package publish prunes superseded commits, and account deletion removes
owner assets.
code:
- packages/worker/src/community/community-icon.ts
- packages/worker/src/app/handlers/community-icon.ts
Expand Down
42 changes: 27 additions & 15 deletions docs/contributing/community-packages.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,19 +67,30 @@ map; the path metadata lets the icon route retrieve bytes from Artifacts.

### Community icon cache

Community listing icons are derived lazily from the pinned Artifacts commit.
`@epic-web/cachified` stores a small descriptor in `BUNDLE_ARTIFACTS_KV` under
`derived-cache:v1:community-icon:v1:{listingId}:{pinnedCommit}`. The processed
bytes live in the private `COMMUNITY_ASSETS` R2 bucket under
`community-icon:v1/{listingId}/{pinnedCommit}/asset`.

On a descriptor cache miss, the icon service checks the pinned community
snapshot for a standardized root `community-icon.*` path, reads that file as
bytes from the Artifacts git repo at `pinnedCommit`, validates it, and writes
the derived asset to R2 before returning the descriptor. A dangling descriptor
is deleted and regenerated once. Packages without an icon receive a generated
fallback. SVG input is rasterized to PNG; PNG, WebP, and JPEG input remains in
its validated source format.
Community listing icons are derived lazily from the listing's **icon commit**:
the owner package's current `entity_sources.published_commit` (falling back to
the listing's pinned commit when the source row is gone). Listing reads join the
entity source, so a plain package publish moves the icon commit forward — and
with it the icon URL — without a community republish. `@epic-web/cachified`
stores a small descriptor in `BUNDLE_ARTIFACTS_KV` under
`derived-cache:v1:community-icon:v1:{listingId}:{commit}`. The processed bytes
live in the private `COMMUNITY_ASSETS` R2 bucket under
`community-icon:v1/{listingId}/{commit}/asset`.

On a descriptor cache miss, the icon service resolves the standardized root
`community-icon.*` path (from the pinned snapshot when serving the pinned
commit, otherwise by probing the Artifacts git repo at the icon commit), reads
the bytes, validates them, and writes the derived asset to R2 before returning
the descriptor. A dangling descriptor is deleted and regenerated once. Packages
without an icon receive a generated fallback. SVG input is rasterized to PNG;
PNG, WebP, and JPEG input remains in its validated source format.

The icon route serves only the current icon commit and the pinned commit; stale
commit URLs 404. Package publish (`finalizePublishedEntitySource`) calls
`refreshCommunityIconForPackagePublish`, which prunes superseded KV/R2 icon
entries by listing prefix and invalidates the public listing data cache.
Community republish, unpublish, admin hard delete, and account deletion remove
icon entries for the listing.

## Service layer

Expand Down Expand Up @@ -134,8 +145,9 @@ Client routes: `packages/worker/client/routes/community*`
- `/community` — searchable index of active listings
- `/community/:listingId` — metadata, ratings, README, fork prompt, report link
(report requires login)
- `/community/:listingId/icon/:pinnedCommit` — cached package icon or generated
fallback; stale commit URLs are rejected
- `/community/:listingId/icon/:iconCommit` — cached package icon or generated
fallback; serves the current icon commit or the pinned snapshot commit, and
rejects stale commit URLs

Shared OG rendering lives in `packages/worker/src/og/`: a light-mode palette
mirroring app design tokens, satori layout + resvg rasterization, and the
Expand Down
5 changes: 5 additions & 0 deletions docs/guides/package-authoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,8 @@ Public community packages may include one root `community-icon.svg`,
`community-icon.jpeg`. Prefer a square visual with a simple silhouette that
remains legible at 56 pixels. Keep it under 2 MiB and 16 megapixels. Kody
generates a package-name fallback when the repository has no icon.

Publishing the package refreshes the community listing icon automatically. The
candidate paths win in the order listed above, so when replacing an icon with a
different format (for example svg → png), delete the old file in the same commit
or the old format keeps being served.
9 changes: 7 additions & 2 deletions docs/use/community-packages.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,13 @@ icons to PNG before serving them; PNG, WebP, and JPEG files are validated and
served in their original format. Packages without an icon receive a generated
visual based on the package name.

Icon URLs are pinned to the same published commit as the listing. Re-publish the
saved package and then re-run `community_publish` to update its icon.
Icon URLs embed the package's **current published commit** (falling back to the
listing's pinned commit if the package source no longer exists), so publishing a
new package version with an updated `community-icon.*` refreshes the listing
icon without re-running `community_publish`. Remember the priority order above:
an old `community-icon.svg` left at the package root keeps winning over a newly
added `community-icon.png`, so delete superseded icon files when switching
formats.

Re-running `community_publish` updates the public listing to the package's
current published commit. Private edits after publishing do not change the
Expand Down
39 changes: 29 additions & 10 deletions packages/worker/src/app/account-deletion.node.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -162,15 +162,25 @@ function createTestDb(initial: RowMap): {
return { results: results as Array<T>, meta: { changes: 0 } }
}
if (
lower ===
'select id, pinned_commit from community_listings where owner_user_id = ?'
lower.startsWith(
'select community_listings.id, community_listings.pinned_commit',
)
) {
results = (rows.community_listings ?? [])
.filter((row) => row['owner_user_id'] === userId)
.map((row) => ({
id: row['id'],
pinned_commit: row['pinned_commit'],
}))
.map((row) => {
const source = (rows.entity_sources ?? []).find(
(sourceRow) =>
sourceRow['id'] === row['source_id'] &&
sourceRow['user_id'] === row['owner_user_id'],
)
return {
id: row['id'],
pinned_commit: row['pinned_commit'],
source_published_commit:
source?.['published_commit'] ?? null,
}
})
return { results: results as Array<T>, meta: { changes: 0 } }
}
const m = lower.match(/^select id from (\w+) where user_id = \?/)
Expand Down Expand Up @@ -591,7 +601,12 @@ test('deleteUserAccount cascades user-scoped rows for the requested user', async
{ user_id: userBbb, resource: 'email_sends_per_day', day: '2026-07-05' },
],
community_listings: [
{ id: 'listing-1', owner_user_id: userAaa, pinned_commit: 'commit-1' },
{
id: 'listing-1',
owner_user_id: userAaa,
pinned_commit: 'commit-1',
source_id: 'src-1',
},
{ id: 'listing-2', owner_user_id: userBbb, pinned_commit: 'commit-2' },
],
community_forks: [
Expand Down Expand Up @@ -844,6 +859,7 @@ test('deleteUserAccount cascades user-scoped rows for the requested user', async
expect(deletedKvKeys.sort()).toEqual([
'bundle-artifact:v1:src-1',
'community-snapshot:v1:listing-1',
'derived-cache:v1:community-icon:v1:listing-1:abc123',
'derived-cache:v1:community-icon:v1:listing-1:commit-1',
'package-retriever-index-entry:v1:user-aaa:context:pkg-1:notes',
'package-retriever-index-entry:v1:user-aaa:search:pkg-1:notes',
Expand All @@ -869,10 +885,13 @@ test('deleteUserAccount cascades user-scoped rows for the requested user', async
expect(result.updatedRowCounts.community_reports).toBe(1)
expect(result.deletedRowCounts.community_bans).toBe(1)
expect(result.updatedRowCounts.community_bans).toBe(1)
expect(result.deletedKvKeys).toBe(12)
expect(result.deletedCommunityAssets).toBe(1)
expect(result.deletedKvKeys).toBe(13)
expect(result.deletedCommunityAssets).toBe(2)
expect(result.deletedEmailBlobs).toBe(1)
expect(deletedCommunityAssetKeys).toEqual([
// Both the pinned snapshot revision and the current icon commit revision
// (the source's published commit) are removed.
expect(deletedCommunityAssetKeys.sort()).toEqual([
'community-icon:v1/listing-1/abc123/asset',
'community-icon:v1/listing-1/commit-1/asset',
])
expect(result.deletedVectors).toBe(5)
Expand Down
41 changes: 33 additions & 8 deletions packages/worker/src/app/account-deletion.ts
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ type UserPackageServiceSnapshot = {
type UserCommunityListingSnapshot = {
id: string
pinnedCommit: string
iconCommit: string
}

type UserDeletionInventory = {
Expand Down Expand Up @@ -322,13 +323,26 @@ async function listUserEmailBlobKeys(env: Env, userId: string) {

async function listUserCommunityListings(env: Env, userId: string) {
const rows = await env.APP_DB.prepare(
`SELECT id, pinned_commit FROM community_listings WHERE owner_user_id = ?`,
`SELECT community_listings.id, community_listings.pinned_commit,
entity_sources.published_commit AS source_published_commit
FROM community_listings
LEFT JOIN entity_sources
ON entity_sources.id = community_listings.source_id
AND entity_sources.user_id = community_listings.owner_user_id
AND entity_sources.entity_kind = 'package'
AND entity_sources.entity_id = community_listings.package_id
WHERE community_listings.owner_user_id = ?`,
)
.bind(userId)
.all<{ id: string; pinned_commit: string }>()
.all<{
id: string
pinned_commit: string
source_published_commit: string | null
}>()
return (rows.results ?? []).map((row) => ({
id: row.id,
pinnedCommit: row.pinned_commit,
iconCommit: row.source_published_commit ?? row.pinned_commit,
}))
}

Expand Down Expand Up @@ -364,7 +378,12 @@ async function listUserBundleKvKeys(input: {
derivedCacheKeyPrefix +
buildCommunityIconCacheKey({
listingId: listing.id,
pinnedCommit: listing.pinnedCommit,
commit: listing.pinnedCommit,
}),
derivedCacheKeyPrefix +
buildCommunityIconCacheKey({
listingId: listing.id,
commit: listing.iconCommit,
}),
]),
])
Expand Down Expand Up @@ -1104,11 +1123,17 @@ export async function deleteUserAccount(input: {

result.deletedCommunityAssets = await deleteR2Objects({
blobs: input.env.COMMUNITY_ASSETS,
keys: inventory.communityListings.map((listing) =>
buildCommunityIconR2Key({
listingId: listing.id,
pinnedCommit: listing.pinnedCommit,
}),
keys: uniqueStrings(
inventory.communityListings.flatMap((listing) => [
buildCommunityIconR2Key({
listingId: listing.id,
commit: listing.pinnedCommit,
}),
buildCommunityIconR2Key({
listingId: listing.id,
commit: listing.iconCommit,
}),
]),
),
label: 'Community asset',
warnings,
Expand Down
4 changes: 2 additions & 2 deletions packages/worker/src/app/community-public.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ export function truncateCommunityText(text: string, maxLength: number) {

export function buildCommunityIconUrl(input: {
listingId: string
pinnedCommit: string
iconCommit: string
}) {
return routes.communityDetailIcon.href(input)
}
Expand All @@ -34,7 +34,7 @@ export function toPublicCommunityListing(
description: listing.description,
iconUrl: buildCommunityIconUrl({
listingId: listing.id,
pinnedCommit: listing.pinnedCommit,
iconCommit: listing.iconCommit,
}),
tags: listing.tags,
readmeContent: listing.readmeContent,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ const sampleListing = {
readmeContent: '# README',
license: 'MIT',
pinnedCommit: 'abc1234567890',
iconCommit: 'abc1234567890',
status: 'active',
createdAt: '2026-01-01T00:00:00.000Z',
updatedAt: '2026-01-01T00:00:00.000Z',
Expand Down
35 changes: 30 additions & 5 deletions packages/worker/src/app/handlers/community-icon.node.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,26 +38,27 @@ const listing = {
readmeContent: null,
license: 'MIT',
pinnedCommit: 'abc123',
iconCommit: 'def456',
status: 'active',
createdAt: '2026-07-10T00:00:00.000Z',
updatedAt: '2026-07-10T00:00:00.000Z',
publishedAt: '2026-07-10T00:00:00.000Z',
} satisfies CommunityListingRecord

function callHandler(pinnedCommit = listing.pinnedCommit) {
function callHandler(iconCommit = listing.iconCommit) {
const handler = createCommunityIconHandler({ APP_DB: {} } as Env)
return handler.handler({
request: new Request(
`https://example.com/community/${listing.id}/icon/${pinnedCommit}`,
`https://example.com/community/${listing.id}/icon/${iconCommit}`,
),
params: { listingId: listing.id, pinnedCommit },
params: { listingId: listing.id, iconCommit },
url: new URL(
`https://example.com/community/${listing.id}/icon/${pinnedCommit}`,
`https://example.com/community/${listing.id}/icon/${iconCommit}`,
),
} as never)
}

test('community icon handler serves cached R2 bytes for the pinned listing', async () => {
test('community icon handler serves cached R2 bytes for the current icon commit', async () => {
const bytes = Uint8Array.from([0x89, 0x50, 0x4e, 0x47])
mocks.getCommunityListingById.mockResolvedValue(listing)
mocks.getCommunityIconObject.mockResolvedValue({
Expand All @@ -77,6 +78,30 @@ test('community icon handler serves cached R2 bytes for the pinned listing', asy
expect(response.headers.get('ETag')).toBe('"icon-etag"')
expect(response.headers.get('X-Content-Type-Options')).toBe('nosniff')
expect(new Uint8Array(await response.arrayBuffer())).toEqual(bytes)
expect(mocks.getCommunityIconObject).toHaveBeenCalledWith(
expect.objectContaining({ iconCommit: listing.iconCommit }),
)
})

test('community icon handler keeps serving the pinned snapshot commit', async () => {
const bytes = Uint8Array.from([0x89, 0x50, 0x4e, 0x47])
mocks.getCommunityListingById.mockResolvedValue(listing)
mocks.getCommunityIconObject.mockResolvedValue({
descriptor: {
byteLength: bytes.byteLength,
contentType: 'image/png',
},
object: {
body: new Blob([bytes]).stream(),
httpEtag: '"icon-etag"',
},
})

const response = await callHandler(listing.pinnedCommit)
expect(response.status).toBe(200)
expect(mocks.getCommunityIconObject).toHaveBeenCalledWith(
expect.objectContaining({ iconCommit: listing.pinnedCommit }),
)
})

test('community icon handler rejects stale commit URLs and degrades to a safe fallback', async () => {
Expand Down
10 changes: 9 additions & 1 deletion packages/worker/src/app/handlers/community-icon.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,22 @@ export function createCommunityIconHandler(env: Env) {
listingId: params.listingId,
includeDelisted: false,
})
if (!listing || listing.pinnedCommit !== params.pinnedCommit) {
// The pinned snapshot commit stays servable alongside the current
// icon commit so pages cached just before a package publish do not
// break; anything else is a stale URL and is rejected.
if (
!listing ||
(params.iconCommit !== listing.iconCommit &&
params.iconCommit !== listing.pinnedCommit)
) {
return new Response('Not found', { status: 404 })
}

try {
const { descriptor, object } = await getCommunityIconObject({
env,
listing,
iconCommit: params.iconCommit,
})
return new Response(object.body, {
headers: {
Expand Down
Loading
Loading