K8SPG-833 add "custom-envs" e2e test #1315

nmarukovich · 2025-10-08T12:20:55Z

CHANGE DESCRIPTION

Problem:
Add E2E test to verify custom environment variables across all components (instance, pgBouncer, pgBackRest )

Since we modified several independent parts of the code, I added repeated checks for the instance, pgBouncer, and pgBackRest types.

What this test covers:

Add env
Update env
Add secret (envFrom)
Update vars inside secret
Update secret name in cr -> we need to update secret and start to use variable from new secret
Delete env
Delete secret.

connected PRs
#1231
#1306

Cause:
Short explanation of the root cause of the issue if applicable.

Solution:
Short explanation of the solution we are providing with this PR.

CHECKLIST

Jira

Is the Jira ticket created and referenced properly?
Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

Is an E2E test/test case added for the new feature/change?
Are unit tests added where appropriate?

Config/Logging/Testability

Are all needed new/changed options added to default YAML files?
Are all needed new/changed options added to the Helm Chart?
Did we add proper logging messages for operator actions?
Did we ensure compatibility with the previous version or cluster upgrade process?
Does the change support oldest and newest supported PG version?
Does the change support oldest and newest supported Kubernetes version?

eleo007

I believe we should also add check for backup and restore pods.

eleo007 · 2025-10-10T10:33:58Z

e2e-tests/tests/custom-envs/04-add-env-secret-to-instance.yaml

+          kubectl create secret generic instance-env-secret \
+            --from-literal=DB_USER_TEST_ENV=myuser \
+            --from-literal=DB_PASSWORD_TEST_ENV='MyS3cretP@ss' \
+            -n "${NAMESPACE}" || true


Maybe it is better to fail a test if secret is not created? Without secret - the next step will fail anyway?

eleo007 · 2025-10-10T10:34:48Z

e2e-tests/tests/custom-envs/04-add-env-secret-to-instance.yaml

+
+          sleep 10  
+          wait_cluster_consistency custom-envs
+          check_env_in_pod add instance DB_USER_TEST_ENV myuser


We add 2 env vars, why do we check only 1 of them?

Add 2 vars for env and check both

eleo007 · 2025-10-10T10:46:15Z

e2e-tests/tests/custom-envs/06-update-secret-for-instance.yaml

+  
+          sleep 10  
+          wait_cluster_consistency custom-envs
+          check_env_in_pod add instance DB_USER_NEW myuser2


the same, pls add hceck for the seond var

eleo007 · 2025-10-10T11:22:29Z

e2e-tests/tests/custom-envs/05-update-env-in-secret-for-instance.yaml

+      sleep 10  
+      wait_cluster_consistency custom-envs  
+        
+      check_env_in_pod add instance DB_NEW_ENV new-value


Add check that old variables stay

eleo007 · 2025-10-10T11:40:51Z

e2e-tests/tests/custom-envs/06-update-secret-for-instance.yaml

+          kubectl create secret generic instance-env-secret-updated \
+            --from-literal=DB_USER_NEW=myuser2 \
+            --from-literal=DB_PASSWORD_NEW=NewS3cretP@ss \
+            --dry-run=client -o yaml | kubectl -n "${NAMESPACE}" apply -f -


It would be nice to unify the way we create secrets

…ostgresql-operator into K8SPG-833_add_e2e_test

github-actions · 2025-10-13T09:50:05Z

e2e-tests/functions

+  local namespace=$1
+  local pod=$2
+  local container=$3
+
+  if [ -n "$container" ]; then
+    kubectl exec -n "$namespace" "$pod" -c "$container" -- printenv 2>/dev/null || true
+  else
+    kubectl exec -n "$namespace" "$pod" -- printenv 2>/dev/null || true
+  fi


[shfmt] _{reported by reviewdog 🐶}

Suggested change

local namespace=$1

local pod=$2

local container=$3

if [ -n "$container" ]; then

kubectl exec -n "$namespace" "$pod" -c "$container" -- printenv 2>/dev/null || true

else

kubectl exec -n "$namespace" "$pod" -- printenv 2>/dev/null || true

fi

local namespace=$1

local pod=$2

local container=$3

if [ -n "$container" ]; then

kubectl exec -n "$namespace" "$pod" -c "$container" -- printenv 2>/dev/null || true

else

kubectl exec -n "$namespace" "$pod" -- printenv 2>/dev/null || true

fi

github-actions · 2025-10-13T09:50:05Z

e2e-tests/functions

+  local check_type=$1
+  local pod=$2
+  local var_name=$3
+  local expected_value=$4
+  local env_content=$5
+
+  local actual_value
+  actual_value=$(echo "$env_content" | grep -E "^${var_name}=" | cut -d'=' -f2- || true)
+
+  if [[ "$check_type" == "add" ]]; then
+    if [ "$actual_value" != "$expected_value" ]; then
+      echo "ERROR: $var_name in $pod — expected '$expected_value', got '${actual_value:-<missing>}'"
+      return 1
+    else
+      echo "OK: $var_name=$actual_value in $pod"
+    fi
+  elif [[ "$check_type" == "delete" ]]; then
+    if [ -n "$actual_value" ]; then
+      echo "ERROR: $var_name exists in $pod (should not exist)"
+      return 1
+    else
+      echo "OK: $var_name deleted in $pod"
+    fi
+  else
+    echo "ERROR: unknown check type '$check_type'"
+    return 1
+  fi


[shfmt] _{reported by reviewdog 🐶}

Suggested change

local check_type=$1

local pod=$2

local var_name=$3

local expected_value=$4

local env_content=$5

local actual_value

actual_value=$(echo "$env_content" | grep -E "^${var_name}=" | cut -d'=' -f2- || true)

if [[ "$check_type" == "add" ]]; then

if [ "$actual_value" != "$expected_value" ]; then

echo "ERROR: $var_name in $pod — expected '$expected_value', got '${actual_value:-<missing>}'"

return 1

else

echo "OK: $var_name=$actual_value in $pod"

fi

elif [[ "$check_type" == "delete" ]]; then

if [ -n "$actual_value" ]; then

echo "ERROR: $var_name exists in $pod (should not exist)"

return 1

else

echo "OK: $var_name deleted in $pod"

fi

else

echo "ERROR: unknown check type '$check_type'"

return 1

fi

local check_type=$1

local pod=$2

local var_name=$3

local expected_value=$4

local env_content=$5

local actual_value

actual_value=$(echo "$env_content" | grep -E "^${var_name}=" | cut -d'=' -f2- || true)

if [[ $check_type == "add" ]]; then

if [ "$actual_value" != "$expected_value" ]; then

echo "ERROR: $var_name in $pod — expected '$expected_value', got '${actual_value:-<missing>}'"

return 1

else

echo "OK: $var_name=$actual_value in $pod"

fi

elif [[ $check_type == "delete" ]]; then

if [ -n "$actual_value" ]; then

echo "ERROR: $var_name exists in $pod (should not exist)"

return 1

else

echo "OK: $var_name deleted in $pod"

fi

else

echo "ERROR: unknown check type '$check_type'"

return 1

fi

github-actions · 2025-10-13T09:50:05Z

e2e-tests/functions

+  local check_type=$1      # add | delete
+  local component=$2       # instance | pgbouncer | repohost
+  local vars=("${@:3}")    # everything after the 2nd argument
+
+  case "$component" in
+    instance)
+      POD=$(kubectl get -n "${NAMESPACE}" pod -l postgres-operator.crunchydata.com/instance-set=instance1 -o 'jsonpath={.items[0].metadata.name}')
+      CONTAINER=""
+      ;;
+    pgbouncer)
+      POD=$(kubectl get -n "${NAMESPACE}" pod -l postgres-operator.crunchydata.com/role=pgbouncer -o 'jsonpath={.items[0].metadata.name}')
+      CONTAINER="pgbouncer"
+      ;;
+    repohost)
+      POD=$(kubectl get -n "${NAMESPACE}" pod -l postgres-operator.crunchydata.com/data=pgbackrest -o 'jsonpath={.items[0].metadata.name}')
+      CONTAINER="pgbackrest"
+      ;;
+    *)
+      echo "ERROR: unknown component '$component'"
+      return 1
+      ;;
+  esac
+
+  echo "Fetching environment variables for $component pod $POD..."
+  local env_content
+  env_content=$(get_envs_from_pod "${NAMESPACE}" "$POD" "$CONTAINER")
+
+  local errors=0
+  for var_entry in "${vars[@]}"; do
+    if [[ "$check_type" == "add" ]]; then
+      local var_name="${var_entry%%=*}"
+      local var_expected="${var_entry#*=}"
+      check_env_in_pod add "$POD" "$var_name" "$var_expected" "$env_content" || errors=$((errors+1))
+    else
+      check_env_in_pod delete "$POD" "$var_entry" "" "$env_content" || errors=$((errors+1))
+    fi
+  done
+
+  if (( errors > 0 )); then
+    echo "$errors environment check(s) failed for component '$component'"
+    return 1
+  else
+    echo "All environment checks passed for component '$component'"
+  fi
+}


[shfmt] _{reported by reviewdog 🐶}

Suggested change

local check_type=$1 # add | delete

local component=$2 # instance | pgbouncer | repohost

local vars=("${@:3}") # everything after the 2nd argument

case "$component" in

instance)

POD=$(kubectl get -n "${NAMESPACE}" pod -l postgres-operator.crunchydata.com/instance-set=instance1 -o 'jsonpath={.items[0].metadata.name}')

CONTAINER=""

;;

pgbouncer)

POD=$(kubectl get -n "${NAMESPACE}" pod -l postgres-operator.crunchydata.com/role=pgbouncer -o 'jsonpath={.items[0].metadata.name}')

CONTAINER="pgbouncer"

;;

repohost)

POD=$(kubectl get -n "${NAMESPACE}" pod -l postgres-operator.crunchydata.com/data=pgbackrest -o 'jsonpath={.items[0].metadata.name}')

CONTAINER="pgbackrest"

;;

*)

echo "ERROR: unknown component '$component'"

return 1

;;

esac

echo "Fetching environment variables for $component pod $POD..."

local env_content

env_content=$(get_envs_from_pod "${NAMESPACE}" "$POD" "$CONTAINER")

local errors=0

for var_entry in "${vars[@]}"; do

if [[ "$check_type" == "add" ]]; then

local var_name="${var_entry%%=*}"

local var_expected="${var_entry#*=}"

check_env_in_pod add "$POD" "$var_name" "$var_expected" "$env_content" || errors=$((errors+1))

else

check_env_in_pod delete "$POD" "$var_entry" "" "$env_content" || errors=$((errors+1))

fi

done

if (( errors > 0 )); then

echo "$errors environment check(s) failed for component '$component'"

return 1

else

echo "All environment checks passed for component '$component'"

fi

}

local check_type=$1 # add | delete

local component=$2 # instance | pgbouncer | repohost

local vars=("${@:3}") # everything after the 2nd argument

case "$component" in

instance)

POD=$(kubectl get -n "${NAMESPACE}" pod -l postgres-operator.crunchydata.com/instance-set=instance1 -o 'jsonpath={.items[0].metadata.name}')

CONTAINER=""

;;

pgbouncer)

POD=$(kubectl get -n "${NAMESPACE}" pod -l postgres-operator.crunchydata.com/role=pgbouncer -o 'jsonpath={.items[0].metadata.name}')

CONTAINER="pgbouncer"

;;

repohost)

POD=$(kubectl get -n "${NAMESPACE}" pod -l postgres-operator.crunchydata.com/data=pgbackrest -o 'jsonpath={.items[0].metadata.name}')

CONTAINER="pgbackrest"

;;

*)

echo "ERROR: unknown component '$component'"

return 1

;;

esac

echo "Fetching environment variables for $component pod $POD..."

local env_content

env_content=$(get_envs_from_pod "${NAMESPACE}" "$POD" "$CONTAINER")

local errors=0

for var_entry in "${vars[@]}"; do

if [[ $check_type == "add" ]]; then

local var_name="${var_entry%%=*}"

local var_expected="${var_entry#*=}"

check_env_in_pod add "$POD" "$var_name" "$var_expected" "$env_content" || errors=$((errors + 1))

else

check_env_in_pod delete "$POD" "$var_entry" "" "$env_content" || errors=$((errors + 1))

fi

done

if ((errors > 0)); then

echo "$errors environment check(s) failed for component '$component'"

return 1

else

echo "All environment checks passed for component '$component'"

fi

}

JNKPercona · 2025-10-13T11:38:42Z

Test Name	Result	Time
backup-enable-disable	failure	00:00:00
custom-envs	failure	00:00:00
custom-extensions	failure	00:00:00
custom-tls	failure	00:00:00
database-init-sql	skipped	00:00:00
demand-backup	skipped	00:00:00
finalizers	skipped	00:00:00
init-deploy	skipped	00:00:00
monitoring	skipped	00:00:00
monitoring-pmm3	skipped	00:00:00
one-pod	skipped	00:00:00
operator-self-healing	skipped	00:00:00
pgvector-extension	skipped	00:00:00
pitr	skipped	00:00:00
scaling	skipped	00:00:00
scheduled-backup	skipped	00:00:00
self-healing	skipped	00:00:00
sidecars	skipped	00:00:00
start-from-backup	skipped	00:00:00
tablespaces	skipped	00:00:00
telemetry-transfer	skipped	00:00:00
upgrade-consistency	skipped	00:00:00
upgrade-minor	skipped	00:00:00
users	skipped	00:00:00
We run 4 out of 24		N/A

commit: 09c40b5
image: perconalab/percona-postgresql-operator:PR-1315-09c40b55c

K8SPG-833 add e2e test

17d6f03

nmarukovich requested review from eleo007, jvpasinatto and valmiranogueira as code owners October 8, 2025 12:20

Merge branch 'main' into K8SPG-833_add_e2e_test

b6f6bd0

hors changed the title ~~K8SPG-833 add e2e test~~ K8SPG-833 add "custom-envs" e2e test Oct 8, 2025

nmarukovich added 2 commits October 9, 2025 15:26

Merge branch 'main' into K8SPG-833_add_e2e_test

413370d

Merge branch 'main' into K8SPG-833_add_e2e_test

a785e80

eleo007 requested changes Oct 10, 2025

View reviewed changes

eleo007 reviewed Oct 10, 2025

View reviewed changes

nmarukovich added 2 commits October 13, 2025 11:48

fix PR comments

6219010

Merge branch 'K8SPG-833_add_e2e_test' of github.com:percona/percona-p…

feab0ae

…ostgresql-operator into K8SPG-833_add_e2e_test

github-actions bot reviewed Oct 13, 2025

View reviewed changes

Merge branch 'main' into K8SPG-833_add_e2e_test

09c40b5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

K8SPG-833 add "custom-envs" e2e test #1315

K8SPG-833 add "custom-envs" e2e test #1315

nmarukovich commented Oct 8, 2025 •

edited

Loading

Uh oh!

eleo007 left a comment

Uh oh!

eleo007 Oct 10, 2025

Uh oh!

eleo007 Oct 10, 2025

Uh oh!

eleo007 Oct 10, 2025

Uh oh!

eleo007 Oct 10, 2025

Uh oh!

eleo007 Oct 10, 2025

Uh oh!

eleo007 Oct 10, 2025

Uh oh!

github-actions bot Oct 13, 2025

Uh oh!

github-actions bot Oct 13, 2025

Uh oh!

github-actions bot Oct 13, 2025

Uh oh!

JNKPercona commented Oct 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

K8SPG-833 add "custom-envs" e2e test #1315

Are you sure you want to change the base?

K8SPG-833 add "custom-envs" e2e test #1315

Conversation

nmarukovich commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CHANGE DESCRIPTION

CHECKLIST

Uh oh!

eleo007 left a comment

Choose a reason for hiding this comment

Uh oh!

eleo007 Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

eleo007 Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

eleo007 Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

eleo007 Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

eleo007 Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

eleo007 Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

JNKPercona commented Oct 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nmarukovich commented Oct 8, 2025 •

edited

Loading