Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
131 commits
Select commit Hold shift + click to select a range
bc40efb
Add hypothesis to poetry and fuzz test the index endpoint.
ahmedxgouda Mar 19, 2025
6c18da7
Refactor import statements in algolia_test.py for consistency
ahmedxgouda Mar 19, 2025
4a17780
Add the client ip address to the request META.
ahmedxgouda Mar 19, 2025
75aa9fe
Add fuzz testing for Slack event handlers and refactor algolia fuzz t…
ahmedxgouda Mar 21, 2025
625cc0e
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Mar 21, 2025
b078e92
Refactor fuzz tests for Slack event handlers to improve readability a…
ahmedxgouda Mar 21, 2025
f1f6718
Fix the poetry lock file.
ahmedxgouda Mar 21, 2025
c0ed1f9
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Mar 27, 2025
998bd29
Remove fuzz testing from algolia_search unit tests
ahmedxgouda Mar 28, 2025
6b240b4
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Mar 28, 2025
f060510
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Apr 1, 2025
ab68f1a
Create a docker file for fuzz-testing, add the run commands to the Ma…
ahmedxgouda Apr 1, 2025
d1d48ea
Refactor to improve quality
ahmedxgouda Apr 1, 2025
54f7fbe
Update fuzz testing setup: modify Makefile and Dockerfile, add entryp…
ahmedxgouda Apr 3, 2025
12b5139
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Apr 3, 2025
21283f7
Update poetry.lock to reflect dependency changes and version updates
ahmedxgouda Apr 3, 2025
c47b9fd
Create a fuzz configuration, update docker file, makefile, and the te…
ahmedxgouda Apr 4, 2025
de81b25
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Apr 4, 2025
5f79854
Refactor fuzz configuration by reorganizing imports and cleaning up w…
ahmedxgouda Apr 4, 2025
8330865
Update Dockerfile and entrypoint script to use Alpine base image and …
ahmedxgouda Apr 4, 2025
1ed4452
Run the server on port 8000 after the tests.
ahmedxgouda Apr 4, 2025
dd36f25
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Apr 5, 2025
0a9068d
Create a docker compose file for fuzz testing.
ahmedxgouda Apr 6, 2025
c5363db
Add 'graphqler' to custom dictionary
ahmedxgouda Apr 7, 2025
ccc28c8
Load data from nest.json and add graphqler to cspell dict.
ahmedxgouda Apr 7, 2025
ef20adb
Remove model-bakery dependency from pyproject.toml and update poetry.…
ahmedxgouda Apr 7, 2025
845e1c1
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Apr 7, 2025
8755056
Update graphqler command in docker compose and the healthcheck
ahmedxgouda Apr 8, 2025
6637d8c
Update graphql command to use backend service URL in docker-compose
ahmedxgouda Apr 8, 2025
decec65
Refactor docker-compose to build graphqler service from Dockerfile an…
ahmedxgouda Apr 9, 2025
672f097
Enhance fuzz testing setup: update Dockerfile and entrypoint scripts,…
ahmedxgouda Apr 10, 2025
ed62759
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Apr 10, 2025
49fa5f8
Update fuzz-test-backend command to abort on container exit
ahmedxgouda Apr 10, 2025
e1533ab
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Apr 10, 2025
dcba769
Add fuzz testing workflow and update image build steps
ahmedxgouda Apr 11, 2025
5e85b0b
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Apr 11, 2025
70312c5
Add .env file creation step for fuzz tests in CI/CD workflow
ahmedxgouda Apr 11, 2025
751f105
Add Docker Hub login step for fuzz tests in CI/CD workflow
ahmedxgouda Apr 11, 2025
4cb975a
Refactor for the checks
ahmedxgouda Apr 11, 2025
47daeda
Refactor fuzz testing workflow: replace Docker Hub login with buildx …
ahmedxgouda Apr 11, 2025
5e08107
Fix fuzz tests workflow: rename docker-compose file
ahmedxgouda Apr 11, 2025
de6bf2e
Refactor fuzz-tests job.
ahmedxgouda Apr 11, 2025
d908fcf
Add environment variables for fuzz tests configuration
ahmedxgouda Apr 11, 2025
d3f2987
Update fuzz tests environment variables
ahmedxgouda Apr 11, 2025
d3fd074
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Apr 19, 2025
7d5c20d
Resolve conflicts
ahmedxgouda Apr 22, 2025
03ecb0c
Fix poetry lock file
ahmedxgouda Apr 22, 2025
cf07402
Sort the custom-dict.
ahmedxgouda Apr 22, 2025
e0459d4
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda May 2, 2025
54ce26a
Update content hash in poetry.lock
ahmedxgouda May 2, 2025
deaf1d0
Add docker cache mounts to the backend image
ahmedxgouda May 2, 2025
2236c73
Add Redis configuration
ahmedxgouda May 2, 2025
8ada3b9
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda May 3, 2025
013537b
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda May 6, 2025
8ec969f
refactor yaml
ahmedxgouda May 6, 2025
3839ef1
Add docker cache mounts to graphql file
ahmedxgouda May 6, 2025
f1e2c56
Remove unnecessary chmod command for cache directories in Dockerfile
ahmedxgouda May 6, 2025
2fabe70
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda May 7, 2025
56acd52
Merge branch 'main' into 'feature/fuzz-testing'
ahmedxgouda May 27, 2025
a2436f5
Fix poetry lock file
ahmedxgouda May 27, 2025
25b98f5
Add cache mounts to backend tests
ahmedxgouda May 27, 2025
eb8c087
Update cache mounts in graphql image
ahmedxgouda May 27, 2025
e27ce00
Update mkdir in graphql image
ahmedxgouda May 27, 2025
8d8e3fe
Remove duplicates
ahmedxgouda May 27, 2025
303f825
Update tests
ahmedxgouda May 27, 2025
b57d2b2
Merge branch 'main' into feature/fuzz-testing
ahmedxgouda Oct 11, 2025
8bb36c9
Rename docker compose
ahmedxgouda Oct 11, 2025
c79570c
Update poetry lock
ahmedxgouda Oct 11, 2025
09fd7c5
Apply sonar
ahmedxgouda Oct 11, 2025
51a74b1
Migrate to OWASP repo
ahmedxgouda Oct 11, 2025
fdfc29d
Update docker
ahmedxgouda Oct 11, 2025
663b3e5
Use graphqler maintainer docker image
ahmedxgouda Oct 12, 2025
c4eab67
Add disable permissions, update docker compose, and update entrypoint
ahmedxgouda Oct 12, 2025
133c89f
Establish an e2e backend instance locally and in CI/CD (#2429)
ahmedxgouda Nov 23, 2025
59007a9
Fix running e2e backend (#2710)
ahmedxgouda Dec 12, 2025
2e4c7f8
Merge branch 'feature/e2e-backend' into feature/fuzz-testing
ahmedxgouda Dec 13, 2025
ab32308
Update code for e2e
ahmedxgouda Dec 13, 2025
166f387
Add runs-on
ahmedxgouda Dec 13, 2025
bf07032
Skip sonar and fix ci/cd
ahmedxgouda Dec 14, 2025
e7b6329
Apply rabbit suggestion and override entrypoint in ci/cd
ahmedxgouda Dec 14, 2025
a7862ac
Use env with csrf
ahmedxgouda Dec 14, 2025
958c84e
Add timeout
ahmedxgouda Dec 14, 2025
4076d72
Remove hypothesis and old test files
ahmedxgouda Dec 14, 2025
88858ca
Apply rabbit's suggestions
ahmedxgouda Dec 14, 2025
c65e4cb
Update ci/cd and makefile
ahmedxgouda Dec 14, 2025
7d33ba8
Use digest pinning with graphqler image
ahmedxgouda Dec 14, 2025
a48b578
Update dockerfile and fix the typeerror issue
ahmedxgouda Dec 16, 2025
27d468f
Apply sonar suggestion
ahmedxgouda Dec 16, 2025
6d9f6a1
Apply sonar and rabbit suggestions
ahmedxgouda Dec 16, 2025
891baaf
Remove cache from ci/cd
ahmedxgouda Dec 16, 2025
4247342
Use curl instead of wget
ahmedxgouda Dec 16, 2025
c75ff58
Separate e2e from fuzz
ahmedxgouda Dec 17, 2025
b697d7a
Update fuzz ci/cd
ahmedxgouda Dec 17, 2025
a4d740b
Update CI/CD
ahmedxgouda Dec 17, 2025
cd56848
Run precommit
ahmedxgouda Dec 17, 2025
1640b3b
Update code
arkid15r Dec 20, 2025
04c8f1d
Update code
arkid15r Dec 20, 2025
b7d7f5d
Update docs, ci/cd, and apply suggestions
ahmedxgouda Dec 21, 2025
fcc4927
Use digest pinning and parameters in workflow
ahmedxgouda Dec 21, 2025
240720b
Apply sonar suggestions
ahmedxgouda Dec 21, 2025
df73958
Apply rabbit suggestions
ahmedxgouda Dec 21, 2025
e308775
Run migrations for fuzz testing in ci/cd
ahmedxgouda Dec 21, 2025
c643ada
Apply rabbit suggestions
ahmedxgouda Dec 21, 2025
a6d5605
Fix exceptions
arkid15r Dec 24, 2025
52a9b68
Establish an e2e backend instance locally and in CI/CD (#2429)
ahmedxgouda Nov 23, 2025
029db6e
Fix running e2e backend (#2710)
ahmedxgouda Dec 12, 2025
de86aab
Add backend/data/nest.sql.gz
arkid15r Dec 24, 2025
47d0feb
Merge branch 'feature/e2e-backend' into pr/ahmedxgouda/1139
arkid15r Dec 24, 2025
e62fed6
Update code
arkid15r Dec 24, 2025
86184a7
Automate data loading
arkid15r Dec 24, 2025
5f7ad2d
Update dump_data to avoid exceptions
ahmedxgouda Dec 24, 2025
2f78434
Update dump_data and automated data_loading
ahmedxgouda Dec 24, 2025
c400be4
Update CI/CD
ahmedxgouda Dec 24, 2025
7ea774e
Update tests
ahmedxgouda Dec 24, 2025
fed517c
Add timeout for fuzz tests
ahmedxgouda Dec 24, 2025
02b5e63
Update timeout for fuzz
ahmedxgouda Dec 24, 2025
ea7e23d
Update CI/CD
ahmedxgouda Dec 24, 2025
2c59b86
Update CI/CD
ahmedxgouda Dec 24, 2025
f67d9ac
Update CI/CD
ahmedxgouda Dec 24, 2025
4edfb7b
Apply rabbit's suggestions
ahmedxgouda Dec 24, 2025
2fcf6b9
Update backend/Makefile
arkid15r Dec 24, 2025
db1a7a1
Update make targets and docker compose
ahmedxgouda Dec 25, 2025
cefb150
Add volume for graphql fuzzing results and add upload artifacts in ci/cd
ahmedxgouda Dec 25, 2025
7fb30f3
Update ci/cd
ahmedxgouda Dec 25, 2025
8444288
Update ci/cd
ahmedxgouda Dec 25, 2025
f79b403
Update ci/cd
ahmedxgouda Dec 25, 2025
6c2d0f0
Update ci/cd
ahmedxgouda Dec 25, 2025
6f15af2
Update docker compose and makefile
ahmedxgouda Dec 26, 2025
9886459
Apply rabbit's suggestions
ahmedxgouda Dec 26, 2025
c648ada
Update dump to match the last nest.json.gz
ahmedxgouda Dec 29, 2025
00b1f84
Merge branch 'feature/e2e-backend' into pr/ahmedxgouda/1139
arkid15r Dec 29, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 127 additions & 2 deletions .github/workflows/run-ci-cd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -233,8 +233,40 @@ jobs:
- name: Set up Docker buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f

- name: Setup E2E environment
uses: ./.github/workflows/setup-e2e-environment
- name: Setup Backend environment
uses: ./.github/workflows/setup-backend-environment
with:
db_username: nest_user_e2e
db_name: nest_db_e2e

- name: Start Backend in the background
run: |
docker run -d --rm --name e2e-nest-backend \
--env-file backend/.env.e2e.example \
--network host \
-e DJANGO_DB_HOST=localhost \
-p 9000:9000 \
owasp/nest:test-backend-latest \
sh -c '
python manage.py migrate &&
gunicorn wsgi:application --bind 0.0.0.0:9000
'

- name: Waiting for the backend to be ready
run: |
timeout 5m bash -c '
until wget --spider http://localhost:9000/a; do
echo "Waiting for backend..."
sleep 5
done
'
echo "Backend is up!"

- name: Load Postgres data
env:
PGPASSWORD: nest_user_e2e_password
run: |
pg_restore -h localhost -U nest_user_e2e -d nest_db_e2e < backend/data/nest.dump

- name: Build frontend end-to-end testing image
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83
Expand Down Expand Up @@ -267,6 +299,99 @@ jobs:
echo "release_version=$(date '+%y.%-m.%-d')-${GITHUB_SHA::7}" >> $GITHUB_OUTPUT
fi

run-fuzz-tests:
name: Run fuzz tests
needs:
- scan-code
- scan-ci-dependencies
runs-on: ubuntu-latest
timeout-minutes: 15
services:
db:
image: pgvector/pgvector:pg16
env:
POSTGRES_DB: nest_db_fuzz
POSTGRES_PASSWORD: nest_user_fuzz_password
POSTGRES_USER: nest_user_fuzz
options: >-
--health-cmd="pg_isready -U nest_user_fuzz -d nest_db_fuzz -h localhost -p 5432"
--health-interval=5s
--health-timeout=5s
--health-retries=5
ports:
- 5432:5432
steps:
- name: Check out repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8

- name: Set up Docker buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f

- name: Setup Backend environment
uses: ./.github/workflows/setup-backend-environment
with:
db_username: nest_user_fuzz
db_name: nest_db_fuzz

- name: Run backend with fuzz environment variables
run: |
docker run -d --rm --name fuzz-nest-backend \
--env-file backend/.env.fuzz.example \
--network host \
-e DJANGO_DB_HOST=localhost \
-p 9500:9500 \
owasp/nest:test-backend-latest \
sh -c '
python manage.py migrate &&
gunicorn wsgi:application --bind 0.0.0.0:9500
'

- name: Waiting for the backend to be ready
run: |
timeout 5m bash -c '
until wget --spider http://localhost:9500/a; do
echo "Waiting for backend..."
sleep 5
done
'
echo "Backend is up!"

- name: Load Postgres data
env:
PGPASSWORD: nest_user_fuzz_password
run: |
pg_restore -h localhost -U nest_user_fuzz -d nest_db_fuzz < backend/data/nest.dump

- name: Build Fuzz-testing image
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83
with:
cache-from: |
type=gha
type=registry,ref=owasp/nest:test-fuzz-backend-cache
cache-to: |
type=gha,compression=zstd
context: backend/docker
file: backend/docker/Dockerfile.fuzz
load: true
platforms: linux/amd64
tags: owasp/nest:test-fuzz-backend-latest

- name: Run backend fuzz tests
run: |
mkdir -p ${{ github.workspace }}/fuzzing_results &&
chmod -R 777 ${{ github.workspace }}/fuzzing_results &&
docker run -e BASE_URL=http://localhost:9500 --network host \
-v ${{ github.workspace }}/fuzzing_results:/home/owasp/fuzzing_results \
owasp/nest:test-fuzz-backend-latest

- name: Upload fuzzing results
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f
if: always()
with:
name: fuzzing-results
path: fuzzing_results/
retention-days: 30

build-staging-images:
name: Build Staging Images
env:
Expand Down
44 changes: 44 additions & 0 deletions .github/workflows/setup-backend-environment/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Set up Backend environment

description: Sets up the Backend environment testing.

inputs:
db_username:
description: 'Database username'
required: true
db_name:
description: 'Database name'
required: true

runs:
using: composite
steps:
- name: Wait for database to be ready
env:
DB_USERNAME: ${{ inputs.db_username }}
DB_NAME: ${{ inputs.db_name }}
run: |
timeout 5m bash -c '
until docker exec ${{ job.services.db.id }} pg_isready -U $DB_USERNAME -d $DB_NAME; do
echo "Waiting for database..."
sleep 5
done
'
shell: bash

- name: Install PostgreSQL client
run: sudo apt-get install -y postgresql-client
shell: bash

- name: Build backend image
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83
with:
cache-from: |
type=gha
cache-to: |
type=gha,compression=zstd
context: backend
file: backend/docker/Dockerfile
load: true
platforms: linux/amd64
tags: owasp/nest:test-backend-latest
13 changes: 13 additions & 0 deletions .github/workflows/update-nest-test-images.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,16 @@ jobs:
platforms: linux/amd64
push: true
tags: owasp/nest:test-frontend-e2e-latest

- name: Build and push fuzz-test-backend image
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83
with:
cache-from: type=registry,ref=owasp/nest:test-fuzz-backend-cache
cache-to: |
type=gha,compression=zstd
type=registry,ref=owasp/nest:test-fuzz-backend-cache
context: backend/docker
file: Dockerfile.fuzz
platforms: linux/amd64
push: true
tags: owasp/nest:test-fuzz-backend-latest
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ __pycache__
.cache
.coverage
.cursor/rules/snyk_rules.mdc
backend/fuzzing_results/
.DS_Store
.env*
!.env.example
!.env.e2e.example
!.env.fuzz.example
.github/instructions/snyk_rules.instructions.md
.idea
.lighthouseci/
Expand Down Expand Up @@ -44,3 +46,6 @@ logs
node_modules/
TODO
venv/

# Snyk Security Extension - AI Rules (auto-generated)
.cursor/rules/snyk_rules.mdc
60 changes: 44 additions & 16 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,31 +417,59 @@ make test
This command runs tests and checks that coverage threshold requirements are satisfied for both backend and frontend.
**Please note your PR won't be merged if it fails the code tests checks.**

### Setting Up e2e Testing Environment
### Running e2e Tests

Follow these steps to setup your e2e testing environment:
Run the frontend e2e tests with the following command:

1. Make sure you have `gzip` installed on your machine.
```bash
make test-frontend-e2e
```

2. Run the e2e backend instance with the following command:
This command automatically:

```bash
make run-backend-e2e
```
- Starts the database and backend containers
- Runs migrations and loads test data
- Executes the e2e tests
- Cleans up containers when done

3. Load the data into the e2e db with the following command (in another terminal session):
For debugging, you can run the e2e backend separately:

```bash
make load-data-e2e
```
```bash
make run-backend-e2e
```

4. Now, you can stop the backend instance, and run the frontend e2e tests with the following command:
Then load data manually in another terminal:

```bash
make test-frontend-e2e
```
```bash
make load-data-e2e
```

### Running Fuzz Tests

**Please note that you only need to do these steps once.**
Run the fuzz tests with the following command:

```bash
make test-fuzz
```

This command automatically:

- Starts the database and backend containers
- Runs migrations and loads test data
- Executes the fuzz tests
- Cleans up containers when done

For debugging, you can run the fuzz backend separately:

```bash
make run-backend-fuzz
```

Then load data manually in another terminal:

```bash
make load-data-fuzz
```

### Test Coverage

Expand Down
23 changes: 23 additions & 0 deletions backend/.env.fuzz.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
DJANGO_ALGOLIA_APPLICATION_ID=None
DJANGO_ALGOLIA_EXCLUDED_LOCAL_INDEX_NAMES=None
DJANGO_ALGOLIA_WRITE_API_KEY=None
DJANGO_ALLOWED_HOSTS=*
DJANGO_AWS_ACCESS_KEY_ID=None
DJANGO_AWS_SECRET_ACCESS_KEY=None
DJANGO_SETTINGS_MODULE=settings.fuzz
DJANGO_CONFIGURATION=Fuzz
DJANGO_DB_HOST=db
DJANGO_DB_NAME=nest_db_fuzz
DJANGO_DB_USER=nest_user_fuzz
DJANGO_DB_PASSWORD=nest_user_fuzz_password
DJANGO_DB_PORT=5432
DJANGO_OPEN_AI_SECRET_KEY=None
DJANGO_PUBLIC_IP_ADDRESS="127.0.0.1"
DJANGO_REDIS_HOST=None
DJANGO_REDIS_PASSWORD=None
DJANGO_RELEASE_VERSION=None
DJANGO_SECRET_KEY=None
DJANGO_SENTRY_DSN=None
DJANGO_SLACK_BOT_TOKEN=None
DJANGO_SLACK_SIGNING_SECRET=None
GITHUB_TOKEN=None
Loading