Skip to content

docs(readme): add Deploy on AWS/GCP with Terraform section - #29873

Closed
yassin-berriai wants to merge 21 commits into
litellm_internal_stagingfrom
chore/readme-tf-deploy-snippets
Closed

docs(readme): add Deploy on AWS/GCP with Terraform section#29873
yassin-berriai wants to merge 21 commits into
litellm_internal_stagingfrom
chore/readme-tf-deploy-snippets

Conversation

@yassin-berriai

Copy link
Copy Markdown
Contributor

Summary

Adds a quickstart for the two published Terraform modules on the public registry, inside the existing Get Started section.

  • AWS -- copy-paste `main.tf` for `BerriAI/litellm/aws`. ECS Fargate + Aurora + ElastiCache + ALB.
  • GCP -- copy-paste `main.tf` for `BerriAI/litellm/google`, plus the one-time `gcloud artifacts repositories create ... --mode=remote-repository` command (Cloud Run can't pull from ghcr.io directly).
  • Both snippets default to plaintext LB (`allow_plaintext_alb/lb = true`) with commented-out lines for prod TLS, since the README example is the "kick the tires" path.
  • Closes the loop on the BerriAI/project-releaser workflow that publishes the modules to the registry on each stable/rc release.

Where it lands

Inside Get Started, between the gateway-vs-SDK table and "Run in Developer Mode" -- where someone scanning the README for "how do I deploy this" will look.

Test plan

  • Skim the new section on the rendered README to confirm code blocks look right.
  • Click both registry links and verify they resolve to the latest published version.
  • Spot-check one `terraform init && terraform apply` end-to-end from the AWS snippet against a sandbox account.

🤖 Generated with Claude Code

shin-berri and others added 10 commits May 13, 2026 22:37
[Infra] Promote internal staging to main
[Infra] Promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
Adds a quickstart for the two published Terraform modules on the public
registry (BerriAI/litellm/aws and BerriAI/litellm/google). Copy-paste
main.tf for each cloud, the one-time GCP Artifact Registry remote-repo
command, and pointers to the registry pages for the full input surface.

Sits inside the Get Started section, between the gateway/SDK table and
Run in Developer Mode -- where someone scanning the README for "how do I
deploy this" will land.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@CLAassistant

CLAassistant commented Jun 7, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 4 committers have signed the CLA.

✅ yuneng-berri
❌ yassin-berriai
❌ shin-berri
❌ claude
You have signed the CLA already but the status is still pending? Let us recheck it.

@codspeed-hq

codspeed-hq Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing chore/readme-tf-deploy-snippets (2c56b4d) with main (8f41fdb)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a new "Deploy on AWS or GCP with Terraform" section to the root README.md, providing copy-paste main.tf quickstarts for the published BerriAI/litellm/aws and BerriAI/litellm/google Terraform Registry modules.

  • AWS snippet wires ECS Fargate + Aurora + ElastiCache + ALB with allow_plaintext_alb = true and a commented-out ACM cert line for production TLS.
  • GCP snippet adds a prerequisite gcloud artifacts repositories create command to proxy GHCR through Artifact Registry (required because Cloud Run rejects ghcr.io directly), then deploys Cloud Run + Cloud SQL + Memorystore + HTTPS LB.

Confidence Score: 3/5

The change is README-only and cannot break runtime behaviour, but it adds documentation directly to this repository in a way the team has explicitly decided against.

A team rule explicitly requires new documentation to live in the litellm-docs repo rather than here; this PR deposits 124 lines of Terraform quickstart content into the root README instead. Beyond the policy issue, the GCP snippet silently requires users to replace my-gcp-project in three separate locations — the project_id input, the image_registry path, and the gcloud --project flag — with no hint that all three need to match, which could produce a confusing Artifact Registry resolution failure at apply time.

README.md — the only changed file; the GCP section needs the placeholder concern addressed and the content should be evaluated for placement in the litellm-docs repo.

Important Files Changed

Filename Overview
README.md Adds a 124-line "Deploy on AWS or GCP with Terraform" section with copy-paste HCL quickstarts; violates the policy requiring new documentation to live in the litellm-docs repo, and the GCP snippet has a repeated placeholder that users may only partially substitute.

Comments Outside Diff (1)

  1. README.md, line 406-529 (link)

    P1 Documentation added to README violates litellm-docs repo policy

    Per the team's custom instruction, documentation additions should live in the litellm-docs repository rather than being committed here. This entire "Deploy on AWS or GCP with Terraform" section (124 lines) was added to the root README.md, which will diverge from the canonical docs site and creates a maintenance burden when these snippets need to be updated with new module versions.

    Rule Used: Prevent documentation from being added - needs to ... (source)

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Reviews (1): Last reviewed commit: "docs(readme): add Deploy on AWS/GCP with..." | Re-trigger Greptile

Comment thread README.md
Comment on lines +486 to +497
source = "BerriAI/litellm/google"
version = "~> 1.89"

project_id = "my-gcp-project"
region = "us-central1"
tenant = "acme"
env = "prod"

image_registry = "us-central1-docker.pkg.dev/my-gcp-project/litellm/berriai"

# Production: provide DNS already pointing at the LB IP for Google-managed certs.
# Without one, set allow_plaintext_lb = true (dev/trial only).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 my-gcp-project placeholder must be updated in three separate locations

The GCP snippet requires users to substitute my-gcp-project in three distinct places: the project_id input, the image_registry path, and the --project flag in the preceding gcloud command. It's easy to update project_id and miss the others — the image_registry value in particular looks like a module default rather than a user-specific placeholder, so users who copy-paste may end up pointing Cloud Run at a non-existent Artifact Registry path and hit a confusing deploy failure. Adding an inline comment like # Replace my-gcp-project with your GCP project ID on the image_registry line would help.

@codecov

codecov Bot commented Jun 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

GCP gets the real 1-click: Open in Cloud Shell badge that clones the repo
and walks through `terraform apply` via the existing DeployStack
tutorial (already shipped at terraform/litellm/gcp/examples/default/
TUTORIAL.md). User just picks a project.

AWS gets a soft 1-click: a Launch in AWS CloudShell badge that opens an
in-browser, already-authenticated shell. User runs four commands
(clone + cd + cp tfvars + terraform apply) once inside. There's no
native AWS deeplink that pre-clones a repo + runs a tutorial -- CFN
"Launch Stack" + CodeBuild would be needed for that, and that's a
separate piece of work.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@yassin-berriai
yassin-berriai enabled auto-merge June 7, 2026 00:47
@yassin-berriai
yassin-berriai disabled auto-merge June 7, 2026 00:48
@yassin-berriai
yassin-berriai changed the base branch from main to litellm_internal_staging June 7, 2026 01:29

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; thanks!

@yassin-berriai
yassin-berriai enabled auto-merge (squash) June 7, 2026 01:30
claude and others added 3 commits June 7, 2026 01:33
…on font

GitHub rewrites an image's height attribute to "height: auto; max-height: Npx", which only caps and never stretches, so each image renders at its intrinsic height. The AWS/GCP shields badges are intrinsically 28px while the Render/Railway buttons are 40px, leaving the row uneven regardless of the height="48" we set. Replace the two shields badges with committed 40px PNGs so all four header buttons render at the same 40px.

Also swap the Cloud Shell button from open-btn.svg to open-btn.png. The SVG renders its label as live text with font-family "Roboto, Sans" and no generic fallback; since neither font exists in GitHub's render environment, the text fell back to a serif (Times New Roman). The PNG bakes in the correct typeface.
The Railway button wrapped its img across indented lines, so the anchor contained leading and trailing whitespace. GitHub underlines link content, rendering that whitespace as a small blue underline beside the button. Put the anchor on one line like the other three buttons so there is no inner whitespace to underline.
@mateo-berri
mateo-berri force-pushed the chore/readme-tf-deploy-snippets branch from a0dce09 to 7cf1b26 Compare June 7, 2026 02:10
@mateo-berri mateo-berri closed this Jun 7, 2026
auto-merge was automatically disabled June 7, 2026 02:45

Pull request was closed

@mateo-berri
mateo-berri deleted the chore/readme-tf-deploy-snippets branch June 7, 2026 02:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants