feat(aws)!: unified container platforms — ECS Task/Service, EKS Deployment/Job/Manifest, Dockerfile.inline image composition

[sam-goodwin]

One consistent container interface across ECS, EKS, and Cloudflare, modeled on the Worker authoring triad (external / inline effect / tagged effect). Design doc: processes/AWS/design/container-platform-redesign.md (local).

Every platform takes one image source, flat on props — image composes with main (there is no baseImage):

{ main }                                   // bundle this Effect program (default bun base)
{ main, image }                            // bundle FROM your base image
{ main, dockerfile: Dockerfile.inline`…` } // bundle in an inline-Dockerfile environment
{ context, dockerfile? }                   // build your Dockerfile (a string is always a PATH)
{ dockerfile: Dockerfile.inline`…` }       // inline content as the whole Dockerfile
{ image }                                  // registry ref deployed verbatim, mirrored into ECR / CF registry

Servers return { fetch, ...rpc }; one-shots return { run, ...rpc }. Tagged forms are always P<Self, Shape>().

AWS.ECS.Task — one-shot

// external (remote image) — no impl, no Effect runtime. Note: no `cluster` —
// a Task maps to AWS::ECS::TaskDefinition, which is cluster-independent; the
// cluster is declared where the launch happens (RunTask / Schedule / Service).
const migrate = yield* AWS.ECS.Task("DbMigrate", {
  image: "public.ecr.aws/docker/library/busybox:stable",
  command: ["sh", "-c", "echo done"],
  cpu: 256,
  memory: 512,
});

// external (build your Dockerfile)
const render = yield* AWS.ECS.Task("RenderJob", {
  context: "./render",
  dockerfile: "./render/Dockerfile.gpu",  // a PATH
  cpu: 1024,
  memory: 4096,
});

// inline effect
const drainer = yield* AWS.ECS.Task(
  "QueueDrainer",
  { main: import.meta.url, image: "oven/bun:1", cpu: 256, memory: 512 },
  Effect.gen(function* () {
    const receive = yield* AWS.SQS.ReceiveMessage(queue);
    return { run: Effect.gen(function* () { /* runs to completion, container exits */ }) };
  }),
);

// tagged effect
export class Reindexer extends AWS.ECS.Task<Reindexer, {
  status: () => Effect.Effect<ReindexStatus>;
}>()("Reindexer") {}

export default Reindexer.make(
  { main: import.meta.url, cpu: 512, memory: 1024 },
  Effect.gen(function* () {
    const scan = yield* AWS.DynamoDB.Scan(table);
    return { run: Effect.gen(function* () { /* ... */ }), status: () => Effect.succeed({ phase: "idle" }) };
  }).pipe(Effect.provide(AWS.DynamoDB.ScanHttp)),
);

// invoke / schedule a Task from other compute — the cluster is explicit at
// the launch site, never on the Task
const runTask = yield* AWS.ECS.RunTask(cluster, thumbnailer);
yield* runTask({ overrides: { environment: { KEY: objectKey } } });
yield* AWS.ECS.Schedule("Nightly", { cluster, task: thumbnailer, schedule: "cron(0 3 * * ? *)" });

AWS.ECS.Service — long-running server (new platform)

// external (remote image) behind an ALB
const nginx = yield* AWS.ECS.Service("Edge", {
  cluster,
  image: "public.ecr.aws/nginx/nginx:1.27",
  port: 80,
  desiredCount: 2,
  loadBalancer: true,
});
nginx.url; // string | undefined

// external (run an existing Task's definition)
const apiFromTask = yield* AWS.ECS.Service("Api", {
  cluster,
  task: apiTask,
  desiredCount: 2,
  loadBalancer: true,
});

// inline effect
const api = yield* AWS.ECS.Service(
  "Api",
  { cluster, main: import.meta.url, port: 3000, desiredCount: 2, cpu: 256, memory: 512 },
  Effect.gen(function* () {
    const putItem = yield* AWS.DynamoDB.PutItem(table);
    return { fetch: Effect.gen(function* () { /* ... */ }) };
  }).pipe(Effect.provide(AWS.DynamoDB.PutItemHttp)),
);

// tagged effect
export class Api extends AWS.ECS.Service<Api, {
  health: () => Effect.Effect<string>;
}>()("Api") {}

export default Api.make(
  { cluster, main: import.meta.url, port: 3000, desiredCount: 2 },
  Effect.gen(function* () {
    return { fetch: Effect.gen(function* () { /* ... */ }), health: () => Effect.succeed("ok") };
  }),
);

AWS.EKS.Deployment — replicated K8s server (was ServerHost)

// external (remote image)
const nginx = yield* AWS.EKS.Deployment("Nginx", {
  cluster,
  image: "nginx:1.27",
  replicas: 3,
  port: 80,
  serviceType: "LoadBalancer",
});
nginx.url; nginx.deploymentName; nginx.serviceAccountName;

// external (build your Dockerfile)
const legacy = yield* AWS.EKS.Deployment("LegacyApp", { cluster, context: "./legacy", replicas: 2, port: 8080 });

// inline effect
const api = yield* AWS.EKS.Deployment(
  "Api",
  { cluster, main: import.meta.url, port: 3000, replicas: 2, serviceType: "LoadBalancer" },
  Effect.gen(function* () {
    const putItem = yield* AWS.DynamoDB.PutItem(table);   // → pod-identity IAM policy
    return { fetch: Effect.gen(function* () { /* ... */ }) };
  }).pipe(Effect.provide(AWS.DynamoDB.PutItemHttp)),
);

// tagged effect
export class Api extends AWS.EKS.Deployment<Api, {
  health: () => Effect.Effect<string>;
}>()("Api") {}

export default Api.make(
  Effect.gen(function* () {
    const cluster = yield* AWS.EKS.Cluster.ref("Primary");
    return { cluster, main: import.meta.url, port: 3000 };
  }),
  Effect.gen(function* () {
    return { fetch: Effect.gen(function* () { /* ... */ }), health: () => Effect.succeed("ok") };
  }),
);

// escape hatch: literal deep-partial Pod template merged onto the synthesized one
const tuned = yield* AWS.EKS.Deployment("Api", {
  cluster, main: import.meta.url, port: 3000,
  podTemplate: {
    spec: { tolerations: [{ key: "gpu", operator: "Exists" }], nodeSelector: { pool: "arm" } },
  },
});

AWS.EKS.Job — run-to-completion (new)

// external
const migrate = yield* AWS.EKS.Job("DbMigrate", { cluster, image: "ghcr.io/acme/migrator:v3", backoffLimit: 2 });

// inline effect
const seed = yield* AWS.EKS.Job(
  "SeedData",
  { cluster, main: import.meta.url },
  Effect.gen(function* () {
    const putItem = yield* AWS.DynamoDB.PutItem(table);
    return { run: Effect.gen(function* () { /* runs to completion */ }) };
  }),
);

// tagged effect
export class Backfill extends AWS.EKS.Job<Backfill, {
  progress: () => Effect.Effect<number>;
}>()("Backfill") {}

export default Backfill.make(
  { cluster, main: import.meta.url, backoffLimit: 1 },
  Effect.gen(function* () {
    const scan = yield* AWS.DynamoDB.Scan(table);
    return { run: Effect.gen(function* () { /* ... */ }), progress: () => Effect.succeed(0) };
  }).pipe(Effect.provide(AWS.DynamoDB.ScanHttp)),
);

// scheduled — synthesizes a K8s CronJob
const nightly = yield* AWS.EKS.Job("NightlyBackfill", { cluster, main: import.meta.url, schedule: "0 3 * * *" });

AWS.EKS.Manifest — raw manifests (new)

The manifest is a literal object, exactly as you would write it in YAML (CRDs resolve via API discovery):

const sts = yield* AWS.EKS.Manifest("Cache", {
  cluster,
  manifest: {
    apiVersion: "apps/v1",
    kind: "StatefulSet",
    metadata: { name: "cache", namespace: "apps" },
    spec: { replicas: 3, /* ... */ },
  },
});

const crd = yield* AWS.EKS.Manifest("Custom", {
  cluster,
  manifest: { apiVersion: "acme.io/v1", kind: "Widget", metadata: { name: "w" }, spec: { /* CRDs ok */ } },
});

Cloudflare.Container

// external (remote image)
class Alpine extends Cloudflare.Container<Alpine>()("Alpine", {
  image: "ghcr.io/alpine/alpine:latest",
  instanceType: "lite",
}) {}

// external (build your Dockerfile)
class Sandbox extends Cloudflare.Container<Sandbox>()("Sandbox", {
  context: `${import.meta.dirname}/context`,
}) {}

// tagged effect (the only effectful form — the impl ships to a different bundle than the DO)
export class Runner extends Cloudflare.Container<Runner, {
  ping: () => Effect.Effect<string>;
}>()("Runner") {}

export default Runner.make(
  { main: import.meta.url, image: "oven/bun:latest", instanceType: "standard" },
  Effect.gen(function* () {
    const bucket = yield* Cloudflare.R2.ReadWriteBucket(store);
    return { ping: () => Effect.succeed("pong"), fetch: /* ... */ };
  }).pipe(Effect.provide(Cloudflare.R2.ReadWriteBucketHttp)),
);

Shared load balancers (ECS.Service)

Rules attach to listeners; ALBs own listeners. String listen ⇒ owned; Listener ref ⇒ shared (the service owns only its TargetGroups + ListenerRules); mixing is a typed error.

// one ALB + listener owned at the stack level
const alb   = yield* AWS.ELBv2.LoadBalancer("Shared", { subnets, securityGroups });
const https = yield* AWS.ELBv2.Listener("Https", { loadBalancer: alb, port: 443, certificates: [cert] });

// services share it — each owns only its rules
const api = yield* AWS.ECS.Service("Api", {
  cluster, main: import.meta.url, port: 3000,
  loadBalancer: {
    listener: https,
    rules: [
      { path: "/api/*" },
      { host: "api.acme.com" },
      { forward: "9090/http", container: "metrics", path: "/metrics" },
    ],
  },
}, impl);

// simplest sharing: bare ref, default rule
const web = yield* AWS.ECS.Service("Web", { cluster, image: "ghcr.io/acme/web", port: 8080, loadBalancer: https });

// owned, many rules incl. redirect
loadBalancer: { rules: [{ listen: "443/https" }, { listen: "80/http", redirect: "443/https" }] }

• Priorities auto-derive (pinned FNV-1a of the rule's logical id, 1–50000); live collisions are a typed ListenerRulePriorityInUse suggesting an explicit priority.
• LB wiring is now composed resources (SecurityGroup/LoadBalancer/Listener/TargetGroup/ListenerRule under the service's namespace) instead of inline API calls; destroying a service removes exactly its rules/TGs and never touches shared infra. Legacy inline-created ingress is reaped on first reconcile (live migration test).
• Verified live: two services sharing one listener with distinct path rules; partial destroy removes one service's rule+TG while the sibling keeps routing and the ALB survives; zero-orphan scan clean.

ECS.Service Phase 2 (SST parity) + live-example hardening

const api = yield* AWS.ECS.Service("Api", {
  cluster, main: import.meta.url, port: 3000,
  loadBalancer: { domain: "api.acme.com", rules: [{ listen: "443/https" }, { listen: "80/http", redirect: "443/https" }],
                  health: { "3000/http": { path: "/health", interval: "10 seconds" } } },
  scaling: { min: 1, max: 8, cpuUtilization: 70, requestCount: 500 },
  capacity: "spot",
  secrets: { DB_PASSWORD: dbSecret.arn },
  logging: { retention: "3 days" },
  serviceRegistry: { namespace, port: 3000 },
  healthCheck: { command: ["CMD-SHELL", "curl -f localhost:3000/health || exit 1"] },
  volumes: [{ efs: files, path: "/mnt/data" }],
}, impl);

NLB via tcp/udp/tls rule protocols; ACM gains region; scaling composes ApplicationAutoScaling resources via a new onCreate hook. All features have live fixture tests (DNS-validated domain path env-gated).

examples/aws-ecs is rewritten as an orders app exercising every authoring form and image source (shared ALB, tagged-form Api + external Web with path rules, { run } seed Task launched via the RunTask binding, cron Schedule, context: build) — deployed/curled/destroyed with a zero-orphan sweep. Deploying it live surfaced and fixed six real bugs the suites missed: serve self-recursion OOM on { fetch } impls, { fetch } containers exiting into a crash-loop, Fargate credential-chain resolution, an invalid Scheduler trust policy, multi-arch image mirroring (docker push --platform), and tagged-form (X.make Layer) boot on ECS + main:-source drift never invalidating images.

Breaking changes & how to migrate

Scan your code for these; each has a detailed diff below.

If you have	Migrate to
docker: { base: "..." } on ECS/EKS props	image: "..." next to main
docker: { dockerfile: "<content>" } (ECS/EKS)	dockerfile: Dockerfile.inline content — or the external { image, command } form when the bundled program was a placeholder
Cloudflare Container dockerfile: "FROM ..." (content string)	image: "..." for a bare base, or dockerfile: Dockerfile.inline for content
One-shot ECS Task impls returning { fetch }	return { run }
AWS.EKS.ServerHost	AWS.EKS.Deployment (tagged form takes two type args)
AWS.EKS.Workload / LoadBalancedWorkload / PodIdentityWorkload	AWS.EKS.Deployment (or AWS.EKS.Manifest for raw specs)
AWS.EKS.AutoCluster	AWS.EKS.Cluster("...", { compute: "auto" })
Anything from alchemy/Kubernetes (resource providers, types, builders)	removed entirely — AWS.EKS.Manifest with a literal object
cluster on AWS.ECS.Task props	remove it — bind launches with RunTask(cluster, task) / StartTask(cluster, task); Schedule takes its own cluster

Redeploy impact (state, not code):

• ECS.Service ingress: previously deployed loadBalancer: true services redeploy onto composed resources — new ALB/TG/listener/SG physical names and a new URL; the old inline-created ingress is reaped automatically on first reconcile. The managed SG now admits the container port from 0.0.0.0/0 (tasks sit in public subnets); composed TargetGroups default to fast health checks (10s interval / threshold 2) and 30s deregistration delay.
• EKS renames change resource Type strings (ServerHost → Deployment): existing state rows no longer match, so the next deploy replaces those resources (new physical names; NLB/ELB URLs change). Plan a maintenance window if the URL is consumed externally.
• Image-source changes are hash-stable where semantics are unchanged: docker: { base } → image and CF dockerfile-content → image produce the same generated Dockerfile, so no image rebuild or task-definition churn on redeploy.

ECS.Task: docker: {} removed; external forms replace the empty-program hack

-class OneShotTask extends AWS.ECS.Task<OneShotTask>()(
-  "OneShotTask",
-  {
-    main: import.meta.filename,
-    docker: { dockerfile: 'FROM busybox:stable\nCMD ["sh","-c","echo done"]' },
-  },
-  Effect.gen(function* () {}),   // bundled program never executed
-) {}
+const oneShot = yield* AWS.ECS.Task("OneShotTask", {
+  image: "public.ecr.aws/docker/library/busybox:stable",
+  command: ["sh", "-c", "echo done"],
+});

 AWS.ECS.Task("Api", {
   main: import.meta.url,
-  docker: { base: "oven/bun:1" },
+  image: "oven/bun:1",
 }, impl)

For environments that need extra build steps, compose inline content instead of a bare base:

AWS.ECS.Task("Transcoder", {
  main: import.meta.url,
  dockerfile: Dockerfile.inline`
    FROM oven/bun:1
    RUN apt-get update && apt-get install -y ffmpeg
  `,
}, impl)

Dockerfile.inline produces { content } — a plain state-serializable object (structural discrimination, no symbol brand). Interpolations ride Output.interpolate, so FROM ${base.imageUri} is a real dependency edge; unresolved content defers plan-time hashing to reconcile. On AWS, a path dockerfile also composes with main (the env Dockerfile builds as a local stage; the bundle layers FROM it). Exclusivity is enforced with typed defects: image+dockerfile, image+context, inline+context.

ECS.Task does not accept cluster; RunTask/StartTask take it explicitly

A Task maps to AWS::ECS::TaskDefinition, which is cluster-independent — the cluster belongs to the launch, not the definition:

 const seedTask = yield* AWS.ECS.Task("SeedTask", {
-  cluster,
   main: import.meta.url,
   image: "oven/bun:1",
 }, impl);

-const runSeedTask = yield* AWS.ECS.RunTask(seedTask).pipe(Effect.orDie);
+const runSeedTask = yield* AWS.ECS.RunTask(cluster, seedTask);

The two-argument form is the only form (no bind-time error channel, no Effect.orDie); AWS.ECS.Schedule/every already take their own cluster.

ECS.Task one-shot impls return { run } (and the container exits when it completes)

 Effect.gen(function* () {
-  return { fetch: Effect.gen(function* () { /* pseudo-handler */ }) };
+  return { run: Effect.gen(function* () { /* runs to completion */ }) };
 })

{ fetch } remains for server tasks referenced by Service({ task }).

EKS: ServerHost → Deployment (no alias); resource Type string changes

-class Api extends AWS.EKS.ServerHost<Api>()(
+class Api extends AWS.EKS.Deployment<Api, {}>()(
   "Api",
-  { cluster, main: import.meta.url, port: 3000, docker: { base: "oven/bun:1" } },
+  { cluster, main: import.meta.url, port: 3000, image: "oven/bun:1" },
   impl,
 ) {}

EKS: Workload family deleted

-const web = yield* AWS.EKS.LoadBalancedWorkload("Web", {
-  cluster, namespace,
-  containers: [{ name: "web", image: "nginx:1.27", ports: [{ containerPort: 80 }] }],
-});
+const web = yield* AWS.EKS.Deployment("Web", {
+  cluster, namespace,
+  image: "nginx:1.27",
+  port: 80,
+  serviceType: "LoadBalancer",
+});

EKS: AutoCluster → Cluster({ compute: "auto" }); roleArn optional under auto

-const cluster = yield* AWS.EKS.AutoCluster("Primary", { ... });
+const cluster = yield* AWS.EKS.Cluster("Primary", { compute: "auto" });

alchemy/Kubernetes removed entirely — EKS is the only Kubernetes surface

The module (resource providers, hand-bridged types, zero-runtime builders) and the website's /kubernetes section are gone. AWS.EKS.Manifest takes a literal object; podTemplate on Deployment/Job is a literal deep-partial object. The apply machinery (server-side apply, discovery, ordering) is internal to AWS.EKS.

-import * as Kubernetes from "alchemy/Kubernetes";
-
-const cm = yield* Kubernetes.ConfigMap("Cfg", { cluster, data: { ... } });
+const cm = yield* AWS.EKS.Manifest("Cfg", {
+  cluster,
+  manifest: { apiVersion: "v1", kind: "ConfigMap", metadata: { name: "cfg" }, data: { ... } },
+});

Cloudflare.Container: dockerfile-as-content string → image (or Dockerfile.inline); hash-stable, no rebuilds

 Runner.make(
   {
     main: import.meta.url,
-    dockerfile: "FROM oven/bun:latest",
+    image: "oven/bun:latest",
     instanceType: "standard",
   },
   impl,
 );

A dockerfile string is now always a path (external context variant). Inline content is explicit: dockerfile: Dockerfile.inline\FROM oven/bun:latest\nRUN apk add ffmpeg`— withmainit becomes the environment preamble the bundle layers onto; withoutmain` it is the whole Dockerfile. The variant-strict props union makes the old string-content misuse a type error.

Verification

Live suites green with ALL slow paths enabled: test/AWS/ECS 32/0 (AWS_TEST_SLOW=1; Fargate smoke, tagged bootstrap, shared-ALB, Phase 2) and test/AWS/EKS 19/0 (AWS_TEST_SLOW=1; full cluster-create → serialized config → image → Deployment → NLB → in-pod DynamoDB write → destroy, 19 min). test/Cloudflare/Container 12/12. Zero-orphan censuses clean after every round. The gated EKS path surfaced and fixed six more real bugs on its first true exercise: updateClusterConfig category batching, uncapped exponential cluster polls, hook-output invisibility in alchemy-test, NLB url missing the Service port, missing kubernetes.io/role/elb subnet tags, and a nonexistent default base-image ref.

Image-source finalization re-verified: ECS 33/0 fast sweep post-rework; a live Fargate one-shot whose inline environment bakes a marker via RUN and whose bundled { run } reads it back exited 0 (InlineDockerfileTask.test.ts); 12 unit tests cover the composition/exclusivity matrix (test/Docker/Dockerfile.test.ts). Also fixed in passing: { main, dockerfile: <path> } without context crashed on path.resolve(undefined) — context defaults to ".".

🤖 Generated with Claude Code

[alchemy-version-bot]

Install the packages built from this commit:

alchemy

bun add alchemy@https://pkg.ing/alchemy/a5a5336

@alchemy.run/better-auth

bun add @alchemy.run/better-auth@https://pkg.ing/@alchemy.run/better-auth/a5a5336

@alchemy.run/pr-package

bun add @alchemy.run/pr-package@https://pkg.ing/@alchemy.run/pr-package/a5a5336

[sam-goodwin]

Never do synchronous I/O. appendHookLike should probably return an Effect

[sam-goodwin]

Fixed in a180ede: the capture site now only does Queue.offerUnsafe (in-memory enqueue, no I/O — it's called from the synchronous array-push interception so it can't yield an Effect itself); a forked writer fiber drains the queue with Effect FileSystem appends, and fileLog.close ends + awaits it at RunEnd so tail lines flush. The mid-hook streaming regression test still passes.

[sam-goodwin]

Why do we need to pass cluster here? Don't we know the cluster from the task? Or is it because we need the LogicalId of the cluster?

[sam-goodwin]

Great catch — two answers: (1) ECS task definitions are cluster-independent (any taskdef can run on any cluster), which is why the AWS API — and our binding — takes a cluster. (2) But since this Task declares its cluster, making callers repeat it was redundant. Fixed in 79a7b1b: Task attributes now record clusterArn when declared, and RunTask(task) / StartTask(task) infer it (explicit RunTask(cluster, task) remains as override; a cluster-less Task fails the single-arg form at bind time with typed RunTaskRequiresCluster). The example now reads RunTask(seedTask). One DX note: the task-only form carries the typed error in its channel, so inside a Platform init (which constrains errors to ConfigError) it needs .pipe(Effect.orDie) — happy to switch to a defect-carrying clean channel if you prefer bare yield* RunTask(task).

[sam-goodwin]

Can we take another look at this and see if we can solve the problem without a synchronous call?

[sam-goodwin]

Same fix — no synchronous calls remain in FileLog (NodeFs import is gone entirely); all disk I/O moved to the async writer fiber.