Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new plus docker images to v2 pipeline #779

Merged
merged 142 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
142 commits
Select commit Hold shift + click to select a range
06a8659
adding new images into the integration tests
oliveromahony Jul 24, 2024
56402db
double @
oliveromahony Jul 24, 2024
9c9497d
separate the compose fileS
oliveromahony Jul 24, 2024
87fba64
invalid docker-compose
oliveromahony Jul 24, 2024
9ac20d3
changes
oliveromahony Jul 24, 2024
21a05fa
wip: fix tests
oliveromahony Jul 24, 2024
bfd8179
update from
oliveromahony Jul 24, 2024
9bb9630
check if file path is the issue
aphralG Jul 29, 2024
2d1c079
check if file path is the issue
aphralG Jul 29, 2024
de74ad6
try this
aphralG Jul 29, 2024
f23f233
try this
aphralG Jul 29, 2024
66b6735
try this part 2
aphralG Jul 30, 2024
ebc72a2
try this part 2
aphralG Jul 30, 2024
0dc04f3
try this part 3
aphralG Jul 30, 2024
435a05c
Update docker-compose-apk.yml
oliveromahony Jul 30, 2024
ee3a2c1
check directory
aphralG Jul 30, 2024
1fb6fd7
check directory
aphralG Jul 30, 2024
047a545
check directory
aphralG Jul 30, 2024
acfd521
check directory
aphralG Jul 30, 2024
14d08e5
check directory
aphralG Jul 30, 2024
61b2f85
check directory
aphralG Jul 30, 2024
dcecd5f
check directory
aphralG Jul 30, 2024
32923ac
check directory
aphralG Jul 30, 2024
472c69f
check directory
aphralG Jul 30, 2024
d06c221
try again
aphralG Jul 30, 2024
bea48f2
remove install test for now
aphralG Jul 30, 2024
4689fba
remove install test for now
aphralG Jul 30, 2024
4f55068
remove install test for now
aphralG Jul 30, 2024
083c39f
just try one test at a time
aphralG Jul 30, 2024
3657115
try this
aphralG Jul 31, 2024
6b3a18f
try this
aphralG Jul 31, 2024
4101143
try this
aphralG Jul 31, 2024
62e746e
try this
aphralG Jul 31, 2024
360045d
try this
aphralG Jul 31, 2024
7b48c18
try this
aphralG Jul 31, 2024
aaff887
try api test instead
aphralG Jul 31, 2024
f804d94
try api test instead
aphralG Jul 31, 2024
31e3c01
add agent config
aphralG Jul 31, 2024
3bcf402
add features test
aphralG Jul 31, 2024
7191d2b
add features test
aphralG Jul 31, 2024
fe372c6
add grpc tests back in
aphralG Jul 31, 2024
7909b87
add grpc tests back in
aphralG Jul 31, 2024
8288b98
add back in test
aphralG Jul 31, 2024
553cb62
add back in test
aphralG Jul 31, 2024
8ba5a7d
add back in test
aphralG Jul 31, 2024
f8cfc74
check logs
aphralG Jul 31, 2024
c481f78
check logs
aphralG Jul 31, 2024
8890292
check logs
aphralG Jul 31, 2024
fff5ebd
try removing link
aphralG Jul 31, 2024
9f2b708
try removing link
aphralG Jul 31, 2024
d5ef6a3
try removing link
aphralG Jul 31, 2024
c0e07cb
add back some tests
aphralG Jul 31, 2024
1064e88
make deps
aphralG Jul 31, 2024
5b6a717
add install test back
aphralG Jul 31, 2024
dd09639
add install test back
aphralG Jul 31, 2024
ed24390
add install test back
aphralG Jul 31, 2024
5897b19
add install test back
aphralG Jul 31, 2024
9996064
remove install test to check others
aphralG Aug 1, 2024
c9c6a56
try pass path
aphralG Aug 1, 2024
13ab5f8
try pass path
aphralG Aug 1, 2024
290eeca
try this
aphralG Aug 1, 2024
d350b79
try this
aphralG Aug 1, 2024
b29769b
try this
aphralG Aug 1, 2024
a9f2d0c
try this
aphralG Aug 1, 2024
a893e10
try this
aphralG Aug 1, 2024
7f61669
try this
aphralG Aug 1, 2024
7b225b0
try changing tag
aphralG Aug 1, 2024
86b0dab
try new format
aphralG Aug 1, 2024
0cc8b59
change wait
aphralG Aug 1, 2024
e11181d
start clean up
aphralG Aug 1, 2024
3896846
start clean up
aphralG Aug 1, 2024
871b7de
start clean up
aphralG Aug 1, 2024
f20232b
start clean up
aphralG Aug 1, 2024
1aea089
start clean up
aphralG Aug 1, 2024
60b1963
start clean up
aphralG Aug 1, 2024
51d9ad2
start clean up
aphralG Aug 1, 2024
81f7fc2
start clean up
aphralG Aug 1, 2024
d4f6dc0
try reload nginx
aphralG Aug 2, 2024
94543fd
try reload nginx
aphralG Aug 2, 2024
4b28f97
try reload nginx
aphralG Aug 2, 2024
b85120c
try reload nginx
aphralG Aug 2, 2024
4bb60ce
try reload nginx
aphralG Aug 2, 2024
5265e51
try reload nginx
aphralG Aug 2, 2024
1398883
try reload nginx
aphralG Aug 2, 2024
78b0d39
try this
aphralG Aug 2, 2024
17ff810
try loggin config
aphralG Aug 2, 2024
6221c16
try copy config
aphralG Aug 2, 2024
f192750
check config
aphralG Aug 6, 2024
f7f8987
check nginx is running
aphralG Aug 6, 2024
b646975
check nginx is running
aphralG Aug 6, 2024
dbd68f0
install
aphralG Aug 6, 2024
7545cb7
install
aphralG Aug 6, 2024
eaf5e96
nginx oss
aphralG Aug 6, 2024
994a87b
fix tag
aphralG Aug 6, 2024
99ae9fe
fix
aphralG Aug 6, 2024
56f6821
try plus image
aphralG Aug 6, 2024
f186449
fix naming
aphralG Aug 6, 2024
41a4dad
fix naming
aphralG Aug 6, 2024
9eb47a1
clean up
aphralG Aug 6, 2024
da8cef0
try alpine
aphralG Aug 6, 2024
a33cb13
try alpine
aphralG Aug 6, 2024
dda7b08
rename job
aphralG Aug 6, 2024
2ef06b3
fix naming
aphralG Aug 6, 2024
d6203ff
try new tag
aphralG Aug 7, 2024
432acbe
try new tag
aphralG Aug 7, 2024
3df74ab
tag fix
aphralG Aug 7, 2024
397ee98
fix wait log
aphralG Aug 7, 2024
1455b80
remove duplicate docker compose
aphralG Aug 7, 2024
6d52780
fix test
aphralG Aug 7, 2024
a60ac4a
pr feedback
aphralG Aug 8, 2024
8437617
try remove duplicate dockerfiles
aphralG Aug 8, 2024
53a29b6
try remove duplicate dockerfiles
aphralG Aug 8, 2024
93e086c
add plus image
aphralG Aug 9, 2024
276ba3b
fix env
aphralG Aug 9, 2024
c26b3fd
fix env
aphralG Aug 9, 2024
98b0a31
fix env
aphralG Aug 9, 2024
d1e3d4c
fix env
aphralG Aug 9, 2024
1a46a27
try features test
aphralG Aug 9, 2024
096745f
log config
aphralG Aug 12, 2024
81fb6f9
change log level
aphralG Aug 12, 2024
6e45e38
add plus config
aphralG Aug 12, 2024
d119bd1
set log level to trace
aphralG Aug 12, 2024
299bdcc
set log level to trace
aphralG Aug 12, 2024
2d05e9a
tail access log
aphralG Aug 12, 2024
39954c4
add sleep
aphralG Aug 12, 2024
da960b9
add sleep
aphralG Aug 12, 2024
f20c67a
only check for oss
aphralG Aug 12, 2024
9376c00
only check for oss
aphralG Aug 12, 2024
145cfb4
add other os
aphralG Aug 12, 2024
5c28cda
fix
aphralG Aug 12, 2024
38bd2ee
fix test
aphralG Aug 12, 2024
41ed681
fix os's
aphralG Aug 12, 2024
9fba2da
fix upload
aphralG Aug 12, 2024
2f12280
try fix example
aphralG Aug 13, 2024
2945d0a
try fix example
aphralG Aug 13, 2024
fa91063
try set container compose
aphralG Aug 13, 2024
d9d9241
remove example from pipeline
aphralG Aug 13, 2024
54fa422
pr
aphralG Aug 13, 2024
2828d42
Merge branch 'integration-tests-using-oss-repo' into integration-test…
aphralG Aug 13, 2024
f465567
pr
aphralG Aug 13, 2024
e7dff93
merge main
aphralG Aug 20, 2024
2b43183
Merge branch 'main' into integration-tests-using-plus-repo
aphralG Aug 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 80 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -180,8 +180,8 @@ jobs:
path: /tmp/integration-test-logs/
retention-days: 3

official-image-integration-tests:
name: Integration Tests - Official Images
official-oss-image-integration-tests:
name: Integration Tests - Official OSS Images
needs: build-unsigned-snapshot
runs-on: ubuntu-24.04
strategy:
Expand Down Expand Up @@ -234,9 +234,86 @@ jobs:
if: success() || failure()
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
name: integration-test-logs-${{ matrix.container.image }}-${{ matrix.container.version }}-new
name: official-oss-integration-test-logs-${{ matrix.container.image }}-${{ matrix.container.version }}
path: /tmp/integration-test-logs/
retention-days: 3

official-plus-image-integration-tests:
name: Integration Tests - Official Plus Images
needs: build-unsigned-snapshot
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
container:
- image: "alpine"
version: "3.18"
plus: "r30"
release: "alpine"
path: "/nginx-plus/agent"
- image: "alpine"
version: "3.20"
plus: "r32"
release: "alpine"
path: "/nginx-plus/agent"
- image: "alpine"
version: "3.19"
plus: "r31"
release: "alpine"
path: "/nginx-plus/agent"
- image: "debian"
version: "bookworm"
plus: "r32"
release: "debian"
path: "/nginx-plus/agent"
- image: "debian"
version: "bookworm"
plus: "r31"
release: "debian"
path: "/nginx-plus/agent"
- image: "debian"
version: "bookworm"
plus: "r30"
release: "debian"
path: "/nginx-plus/agent"
steps:
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
with:
fetch-depth: 0
- uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491 # v5.0.0
with:
go-version-file: 'go.mod'
- name: Download Packages
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
name: nginx-agent-unsigned-snapshots
path: build
- name: Login to Docker Registry
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
with:
registry: ${{ secrets.REGISTRY_URL }}
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_PASSWORD }}
- name: Run Integration Tests
run: |
go install github.com/goreleaser/nfpm/v2/cmd/nfpm@${{ env.NFPM_VERSION }}
CONTAINER_NGINX_IMAGE_REGISTRY="${{ secrets.REGISTRY_URL }}" TAG="${{ matrix.container.plus }}-${{ matrix.container.image }}-${{ matrix.container.version }}" \
OS_RELEASE="${{ matrix.container.release }}" IMAGE_PATH="${{ matrix.container.path }}" \
make official-image-integration-test
- name: Container Output Logs
if: failure()
run: |
docker ps -a
dockerid=$(docker ps -a --format "{{.ID}}")
docker logs "$dockerid"

- name: Archive integration test logs
if: success() || failure()
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
with:
name: official-plus-integration-test-logs-${{ matrix.container.image }}-${{ matrix.container.version }}-${{ matrix.container.plus }}
path: /tmp/integration-test-logs/
retention-days: 3

performance-test:
name: Performance Tests
Expand Down
12 changes: 6 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ OS_RELEASE ?= ubuntu
OS_VERSION ?= 24.04
BASE_IMAGE = "${CONTAINER_REGISTRY}/${OS_RELEASE}:${OS_VERSION}"
IMAGE_TAG = "agent_${OS_RELEASE}_${OS_VERSION}"
AGENT_IMAGE_PATH = "/nginx/agent"
IMAGE_PATH ?= "/nginx/agent"

VERSION_WITH_V := v${VERSION}
LDFLAGS = "-w -X main.version=${VERSION_WITH_V} -X main.commit=${COMMIT} -X main.date=${DATE}"
Expand Down Expand Up @@ -227,14 +227,14 @@ integration-test:

official-image-integration-test:
PACKAGE_NAME=${PACKAGE_NAME} CONTAINER_NGINX_IMAGE_REGISTRY=${CONTAINER_NGINX_IMAGE_REGISTRY} ARCH=${OSARCH} OS_VERSION=${OS_VERSION} \
OS_RELEASE=${OS_RELEASE} TAG=${TAG} CONTAINER_OS_TYPE=${CONTAINER_OS_TYPE} IMAGE_PATH=${AGENT_IMAGE_PATH} DOCKER_COMPOSE_FILE="docker-compose-official-image.yml" \
${GOTEST} -v ./test/integration/api
PACKAGE_NAME=${PACKAGE_NAME} CONTAINER_NGINX_IMAGE_REGISTRY=${CONTAINER_NGINX_IMAGE_REGISTRY} ARCH=${OSARCH} OS_VERSION=${OS_VERSION} \
OS_RELEASE=${OS_RELEASE} TAG=${TAG} CONTAINER_OS_TYPE=${CONTAINER_OS_TYPE} IMAGE_PATH=${AGENT_IMAGE_PATH} DOCKER_COMPOSE_FILE="docker-compose-official-image.yml" \
OS_RELEASE=${OS_RELEASE} TAG=${TAG} CONTAINER_OS_TYPE=${CONTAINER_OS_TYPE} IMAGE_PATH=${IMAGE_PATH} DOCKER_COMPOSE_FILE="docker-compose-official-image.yml" \
${GOTEST} -v ./test/integration/features
PACKAGE_NAME=${PACKAGE_NAME} CONTAINER_NGINX_IMAGE_REGISTRY=${CONTAINER_NGINX_IMAGE_REGISTRY} ARCH=${OSARCH} OS_VERSION=${OS_VERSION} \
OS_RELEASE=${OS_RELEASE} TAG=${TAG} CONTAINER_OS_TYPE=${CONTAINER_OS_TYPE} IMAGE_PATH=${AGENT_IMAGE_PATH} DOCKER_COMPOSE_FILE="docker-compose-official-image.yml" \
OS_RELEASE=${OS_RELEASE} TAG=${TAG} CONTAINER_OS_TYPE=${CONTAINER_OS_TYPE} IMAGE_PATH=${IMAGE_PATH} DOCKER_COMPOSE_FILE="docker-compose-official-image.yml" \
${GOTEST} -v ./test/integration/grpc
PACKAGE_NAME=${PACKAGE_NAME} CONTAINER_NGINX_IMAGE_REGISTRY=${CONTAINER_NGINX_IMAGE_REGISTRY} ARCH=${OSARCH} OS_VERSION=${OS_VERSION} \
OS_RELEASE=${OS_RELEASE} TAG=${TAG} CONTAINER_OS_TYPE=${CONTAINER_OS_TYPE} IMAGE_PATH=${IMAGE_PATH} DOCKER_COMPOSE_FILE="docker-compose-official-image.yml" \
${GOTEST} -v ./test/integration/api

test-bench: ## Run benchmark tests
cd test/performance && GOWORK=off CGO_ENABLED=0 ${GOTEST} -mod=vendor -count 5 -timeout 2m -bench=. -benchmem metrics_test.go
Expand Down
8 changes: 6 additions & 2 deletions test/integration/api/api_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"encoding/json"
"fmt"
"net/http"
"os"
"strconv"
"strings"
"testing"
Expand Down Expand Up @@ -82,8 +83,11 @@ func TestAPI_Metrics(t *testing.T) {
assert.NotContains(t, resp.String(), "test_fail_metric")
// Validate that the agent can call the stub status API
assert.Contains(t, resp.String(), "nginx_http_request_count")
// Validate that the agent can read the NGINX access logs
assert.Contains(t, resp.String(), "nginx_http_status_2xx")

if os.Getenv("IMAGE_PATH") == "/nginx/agent" {
// Validate that the agent can read the NGINX access logs
assert.Contains(t, resp.String(), "nginx_http_status_2xx")
}

metrics := tutils.ProcessResponse(resp)

Expand Down
4 changes: 2 additions & 2 deletions test/integration/api/docker-compose-official-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,5 @@ services:
networks:
- monitoring
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
- ./nginx-agent.conf:/etc/nginx-agent/nginx-agent.conf
- ${NGINX_CONF_FILE}
- ${AGENT_CONF_FILE}
4 changes: 2 additions & 2 deletions test/integration/api/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,5 @@ services:
networks:
- monitoring
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
- ./nginx-agent.conf:/etc/nginx-agent/nginx-agent.conf
- ${NGINX_CONF_FILE}
- ${AGENT_CONF_FILE}
File renamed without changes.
50 changes: 50 additions & 0 deletions test/integration/api/nginx-plus.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
worker_processes 1;

events {
worker_connections 1024;
}

http {
include mime.types;
default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" '
'"$bytes_sent" "$request_length" "$request_time" '
'"$gzip_ratio" $server_protocol ';

access_log /var/log/nginx/access.log main;

sendfile on;
keepalive_timeout 65;

server {
listen 8080;
server_name localhost;

location / {
root /usr/share/nginx/html;
index index.html index.htm;
}

##
# Enable Metrics
##
location /api/ {
api write=on;
allow 127.0.0.1;
deny all;
status_zone my_location_zone1;
}

#error_page 404 /404.html;

# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
}
4 changes: 2 additions & 2 deletions test/integration/features/docker-compose-official-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,5 @@ services:
networks:
- monitoring
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
- ${CONF_FILE}
- ${NGINX_CONF_FILE}
- ${AGENT_CONF_FILE}
4 changes: 2 additions & 2 deletions test/integration/features/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,5 @@ services:
networks:
- monitoring
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
- ${CONF_FILE}
- ${NGINX_CONF_FILE}
- ${AGENT_CONF_FILE}
49 changes: 49 additions & 0 deletions test/integration/features/nginx-plus.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
worker_processes 1;

events {
worker_connections 1024;
}

http {
include mime.types;
default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" '
'"$bytes_sent" "$request_length" "$request_time" '
'"$gzip_ratio" $server_protocol ';

access_log /var/log/nginx/access.log main;

sendfile on;
keepalive_timeout 65;

server {
listen 8080;
server_name localhost;

location / {
root /usr/share/nginx/html;
index index.html index.htm;
}

##
# Enable Metrics
##
location /api/ {
api write=on;
allow 127.0.0.1;
deny all;
}

#error_page 404 /404.html;

# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
}
2 changes: 1 addition & 1 deletion test/integration/grpc/docker-compose-official-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@ services:
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- ./nginx-agent.conf:/etc/nginx-agent/nginx-agent.conf
- ${AGENT_CONF_FILE}
2 changes: 1 addition & 1 deletion test/integration/grpc/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ services:
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- ./nginx-agent.conf:/etc/nginx-agent/nginx-agent.conf
- ${AGENT_CONF_FILE}
2 changes: 1 addition & 1 deletion test/integration/install/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ services:
ports:
- 80
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf # default nginx apk conf does not listen on localhost
- ${NGINX_CONF_FILE} # default nginx apk conf does not listen on localhost
49 changes: 49 additions & 0 deletions test/integration/install/nginx-plus.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
worker_processes 1;

events {
worker_connections 1024;
}

http {
include mime.types;
default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" '
'"$bytes_sent" "$request_length" "$request_time" '
'"$gzip_ratio" $server_protocol ';

access_log /var/log/nginx/access.log main;

sendfile on;
keepalive_timeout 65;

server {
listen 8080;
server_name localhost;

location / {
root /usr/share/nginx/html;
index index.html index.htm;
}

##
# Enable Metrics
##
location /api/ {
api write=on;
allow 127.0.0.1;
deny all;
}

#error_page 404 /404.html;

# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
}
Loading
Loading