Sync feature/nest-zappa-migration with main

[rudransh-shrivastava]

Proposed change

Resolves #(put the issue number here)

Add the PR description here.

Checklist

• Required: I followed the contributing workflow
• Required: I verified that my code works as intended and resolves the issue as described
• Required: I ran make check-test locally: all warnings addressed, tests passed
• I used AI for code, documentation, tests, or communication related to this PR

[coderabbitai]

Important

Review skipped

Too many files!

This PR contains 224 files, which is 74 over the limit of 150.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

Walkthrough

Replaces Ansible-based staging deployment with Terraform-driven infrastructure, introduces comprehensive AWS infrastructure modules (ALB, ECS, RDS, ElastiCache, networking, security), updates CI/CD pipeline to use Terraform for staging deployments, integrates backend with AWS SSM Parameter Store and Zappa for Lambda deployment, and removes Docker Compose staging configuration.

Changes

Cohort / File(s)	Summary
Ansible Deployment Removal .github/ansible/inventory.yaml, .github/ansible/staging/nest.yaml, .github/ansible/staging/proxy.yaml, cron/staging	Removed entire staging Ansible deployment infrastructure including inventory hosts, playbooks for backend/database/worker services, proxy configuration, and related cron jobs.
CI/CD Workflow Updates .github/workflows/run-ci-cd.yaml	Introduced Terraform installation, TFLint setup, new infrastructure testing job, plan-staging-nest and enhanced deploy-staging-nest jobs with Terraform artifact handling, AWS ECR image tagging, ECS task triggers, and Terraform output capture.
Terraform Backend Infrastructure infrastructure/backend/*	Added complete Terraform state management with S3 bucket, DynamoDB lock table, KMS encryption, IAM policies, versioning, and lifecycle configurations for reliable remote state management.
Terraform Networking Module infrastructure/modules/networking/*, infrastructure/modules/networking/modules/{nacl,vpc-endpoint}/*	Created VPC, subnets (public/private), Internet Gateway, NAT Gateway, route tables, flow logs, and optional VPC endpoints (CloudWatch Logs, ECR, S3, Secrets Manager, SSM) with comprehensive security and networking rules.
Terraform Security Module infrastructure/modules/security/*	Defined security groups and inbound/outbound rules for ALB, ECS, frontend, Lambda, RDS, RDS Proxy, and Redis with conditional VPC endpoint integration and multi-service communication rules.
Terraform Database Module infrastructure/modules/database/*	Configured RDS PostgreSQL instance with optional RDS Proxy, backup/maintenance windows, encryption, Secrets Manager integration, and SSM parameter store for credentials.
Terraform Cache Module infrastructure/modules/cache/*	Set up ElastiCache Redis with encryption, authentication, failover, CloudWatch logging, and SSM parameter store integration for Django cache settings.
Terraform ECS Module infrastructure/modules/ecs/*, infrastructure/modules/ecs/modules/task/*	Created ECS cluster, ECR repository with lifecycle policies, IAM roles/policies for task execution, and modular task definitions for migrate, load_data, index_data, and data sync operations with CloudWatch logging.
Terraform Frontend Module infrastructure/modules/frontend/*	Configured ECS service for frontend with auto-scaling, target group attachment to ALB, CloudWatch logging, ECR repository, and secret parameter injection.
Terraform ALB Module infrastructure/modules/alb/*	Deployed Application Load Balancer with ACM certificate, HTTP-to-HTTPS redirect, HTTPS listener, path-based routing, target groups (frontend and Lambda), health checks, and S3 logging with lifecycle management.
Terraform KMS Module infrastructure/modules/kms/*	Created KMS encryption key with rotation, CloudWatch Logs and account root permissions, alias, and tagging for broader infrastructure encryption.
Terraform Parameters Module infrastructure/modules/parameters/*	Defined AWS SSM parameters for Django settings, NextAuth configuration, database/Redis/API credentials, secrets generation via random_string, with SecureString and String parameter types.
Terraform Storage Module infrastructure/modules/storage/*, infrastructure/modules/storage/modules/s3-bucket/*	Created S3 buckets for fixtures and Zappa deployments with encryption, lifecycle policies (incomplete multipart, version expiration), HTTPS-only policy, versioning, and read-only IAM policy.
Terraform Staging Configuration infrastructure/staging/*	Wired all infrastructure modules together for staging environment with local variables, provider configuration, backend state, outputs, and comprehensive variable definitions for customization.
Terraform Linting and Pre-commit .pre-commit-config.yaml, infrastructure/.tflint.hcl, cspell/*	Added terraform_fmt and terraform_tflint hooks with configuration, enabled Terraform dictionary in cspell, and updated custom dictionary with infrastructure-related terms.
Backend Django Management Commands backend/apps/common/management/commands/load_data.py, backend/tests/apps/common/management/commands/load_data_test.py	Created new load_data management command to load fixture data with transactional handling and indexing suppression, plus comprehensive unit tests.
Backend Zappa Integration backend/zappa_callback.py, backend/zappa_settings.template.json	Added Zappa callback functions for package cleaning, alias updates, and version cleanup; created Zappa settings template with VPC, Lambda, SSM, and post-execution hooks.
Backend AWS Integration backend/wsgi.py	Integrated AWS Systems Manager Parameter Store for environment variables, X-Ray tracing for Lambda, and boto3 client initialization at import time.
Backend Dependencies and Settings backend/pyproject.toml, backend/settings/base.py, backend/settings/staging.py	Added aws-xray-sdk and zappa dependencies, changed Redis protocol to rediss:// for TLS when enabled, removed AWS credentials from staging settings, and updated ALLOWED_ORIGINS to support environment-based configuration.
Backend Import Optimizations backend/apps/github/models/common.py, backend/apps/github/models/repository.py	Moved GithubException and UnknownObjectException imports from module level to function-local scope for lazy import optimization.
Build and Makefile Updates Makefile, backend/Makefile, docker/backend/Dockerfile, infrastructure/Makefile	Added infrastructure Makefile inclusion, introduced EXEC_MODE for direct backend command execution, added Makefile copying and backend symlink in Docker image, and created test-infrastructure target.
Configuration Cleanup .gitignore, docker-compose/staging/compose.yaml	Added Terraform-related ignores (.tfstate, .tfvars, .terraform), backend artifact patterns, zappa_settings.json, and removed entire staging docker-compose service definitions.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• OWASP/Nest#3694 — Adds CI infrastructure testing job and Terraform test mocking alongside this PR's infrastructure test integration
• OWASP/Nest#2699 — Modifies Terraform remote state backend (S3/DynamoDB) and .gitignore patterns matching this PR's backend infrastructure additions
• OWASP/Nest#2431 — Introduces Zappa deployment and Lambda SSM integration changes directly overlapping with this PR's Zappa callback and wsgi.py modifications

Suggested labels

infrastructure, terraform, ci, backend, deploy

Suggested reviewers

• arkid15r
• kasya

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Description check	❓ Inconclusive	The description is a template placeholder with unchecked checklist items and no specific details about the changeset, making it vague and uninformative about what was actually changed.	Replace the template placeholder with actual details: describe the purpose of the sync, reference the resolved issue, summarize key changes, and check the completed items in the contribution checklist.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Sync feature/nest-zappa-migration with main' clearly summarizes the main change: syncing a feature branch with the main branch.
Docstring Coverage	✅ Passed	Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

PR validation failed: No linked issue and no valid closing issue reference in PR description

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[codecov]

Codecov Report

❌ Patch coverage is 60.31746% with 100 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.38%. Comparing base (7eea02f) to head (6f3bda2).
⚠️ Report is 26 commits behind head on feature/nest-zappa-migration.

Files with missing lines	Patch %	Lines
...c/app/my/mentorship/programs/[programKey]/page.tsx	11.76%	2 Missing and 13 partials ⚠️
...p/projects/dashboard/metrics/[projectKey]/page.tsx	44.00%	0 Missing and 14 partials ⚠️
frontend/src/app/community/snapshots/[id]/page.tsx	25.00%	0 Missing and 6 partials ⚠️
frontend/src/app/contribute/page.tsx	0.00%	0 Missing and 6 partials ⚠️
frontend/src/app/page.tsx	61.53%	0 Missing and 5 partials ⚠️
frontend/src/components/MetricsCard.tsx	0.00%	0 Missing and 5 partials ⚠️
frontend/src/app/chapters/page.tsx	0.00%	0 Missing and 4 partials ⚠️
[...nizationKey]/repositories/[repositoryKey]/page.tsx](https://app.codecov.io/gh/OWASP/Nest/pull/3888?src=pr&el=tree&filepath=frontend%2Fsrc%2Fapp%2Forganizations%2F%5BorganizationKey%5D%2Frepositories%2F%5BrepositoryKey%5D%2Fpage.tsx&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=OWASP#diff-ZnJvbnRlbmQvc3JjL2FwcC9vcmdhbml6YXRpb25zL1tvcmdhbml6YXRpb25LZXldL3JlcG9zaXRvcmllcy9bcmVwb3NpdG9yeUtleV0vcGFnZS50c3g=)	33.33%	0 Missing and 4 partials ⚠️
frontend/src/components/ItemCardList.tsx	78.94%	0 Missing and 4 partials ⚠️
frontend/src/app/chapters/[chapterKey]/page.tsx	0.00%	0 Missing and 3 partials ⚠️
... and 20 more

Additional details and impacted files

@@                       Coverage Diff                        @@
##           feature/nest-zappa-migration    #3888      +/-   ##
================================================================
+ Coverage                         90.24%   95.38%   +5.14%     
================================================================
  Files                               464      464              
  Lines                             14427    14554     +127     
  Branches                           1937     2061     +124     
================================================================
+ Hits                              13020    13883     +863     
+ Misses                              988      328     -660     
+ Partials                            419      343      -76     

Flag	Coverage Δ
backend	95.67% <92.85%> (+4.68%)	⬆️
frontend	94.61% <58.40%> (+6.43%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
backend/apps/api/rest/v0/committee.py	100.00% <100.00%> (+11.42%)	⬆️
backend/apps/api/rest/v0/structured_search.py	100.00% <100.00%> (+25.00%)	⬆️
backend/apps/common/eleven_labs.py	100.00% <ø> (ø)
backend/apps/core/api/internal/algolia.py	84.00% <100.00%> (+4.83%)	⬆️
backend/apps/owasp/api/internal/nodes/project.py	90.32% <100.00%> (+16.12%)	⬆️
...s/slack/management/commands/slack_sync_messages.py	90.11% <100.00%> (+11.71%)	⬆️
frontend/src/app/my/mentorship/page.tsx	96.55% <100.00%> (+15.51%)	⬆️
[...Key]/modules/[moduleKey]/issues/[issueId]/page.tsx](https://app.codecov.io/gh/OWASP/Nest/pull/3888?src=pr&el=tree&filepath=frontend%2Fsrc%2Fapp%2Fmy%2Fmentorship%2Fprograms%2F%5BprogramKey%5D%2Fmodules%2F%5BmoduleKey%5D%2Fissues%2F%5BissueId%5D%2Fpage.tsx&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=OWASP#diff-ZnJvbnRlbmQvc3JjL2FwcC9teS9tZW50b3JzaGlwL3Byb2dyYW1zL1twcm9ncmFtS2V5XS9tb2R1bGVzL1ttb2R1bGVLZXldL2lzc3Vlcy9baXNzdWVJZF0vcGFnZS50c3g=)	93.22% <100.00%> (+5.93%)	⬆️
frontend/src/app/projects/[projectKey]/page.tsx	100.00% <ø> (ø)
...ontend/src/app/projects/dashboard/metrics/page.tsx	99.11% <100.00%> (+24.77%)	⬆️
... and 51 more

... and 69 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7eea02f...6f3bda2. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.