Skip to content

Add e2e tests for static targets #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gsanchezgavier merged 4 commits into from
Aug 5, 2022
Merged

Add e2e tests for static targets #30

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1ea0103
add e2e test based on local environment
gsanchezgavier Aug 3, 2022
67a9ae4
clean environment before running tests
gsanchezgavier Aug 4, 2022
b4af3dd
docs
gsanchezgavier Aug 5, 2022
a91741f
fix lint issue
gsanchezgavier Aug 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions .github/workflows/push_pr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,8 @@ jobs:
# if: steps.list-changed.outputs.changed == 'true'
# run: ct install --namespace ct --config .github/ct.yaml --debug --upgrade

tilt:
name: Check testing environment
e2e-test:
name: E2e Tests
runs-on: ubuntu-latest
# If the environment is broken this job could timeout since the default timeout for tilt ci is 30m.
timeout-minutes: 10
Expand All @@ -91,9 +91,16 @@ jobs:
- name: Install Tilt
run: |
curl -fsSL https://raw.githubusercontent.com/tilt-dev/tilt/master/scripts/install.sh | bash
- name: Run Tilt ci
run: |
make tilt-ci
- name: Run e2e-test
uses: newrelic/newrelic-integration-e2e-action@v1
with:
retry_seconds: 60
retry_attempts: 5
agent_enabled: false
spec_path: test/e2e/test-specs.yml
account_id: ${{ secrets.COREINT_E2E_ACCOUNT_ID }}
api_key: ${{ secrets.COREINT_E2E_API_KEY }}
license_key: ${{ secrets.COREINT_E2E_LICENSE_KEY }}

test:
name: Run tests
Expand Down
4 changes: 4 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,12 @@ Make sure you have these tools or install them:
- [Install Tilt](https://docs.tilt.dev/install.html)
- [Install Helm](https://helm.sh/docs/intro/install/)

A license key and cluster name are required to run the environment. Configure them by the environment variables `NR_PROM_CLUSTER` and `NR_PROM_LICENSE_KEY`.

Start the local environment:
```shell
export NR_PROM_CLUSTER=<cluster name>
export NR_PROM_LICENSE_KEY=<Newrelic ingest key>
make start-local-cluster
make tilt-up
```
Expand Down
9 changes: 7 additions & 2 deletions Tiltfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ allow_k8s_contexts(cluster_context)

local_resource('Configurator binary', 'make compile-multiarch', deps=[
'./cmd',
'./internal',
'./configurator',
])

Expand All @@ -18,7 +17,13 @@ docker_build('prometheus-configurator', '.')
docker_build('openmetrics-fake-exporter', './test/openmetrics-fake-exporter/.')

# Deploying Kubernetes resources.
k8s_yaml(helm('./charts/%s' % project_name, name=project_name, values=['values-dev.yaml']))
k8s_yaml(
helm(
'./charts/%s' % project_name,
name=project_name,
values=['values-dev.yaml'],
set=['licenseKey=%s' % os.getenv('NR_PROM_LICENSE_KEY'), 'cluster=%s' % os.getenv('NR_PROM_CLUSTER')],
))
k8s_yaml(helm('./charts/internal/test-resources', name='test-resources'))

# Tracking the deployment.
Expand Down
10 changes: 9 additions & 1 deletion configurator/parser_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,10 @@ import (
_ "github.com/prometheus/prometheus/discovery/kubernetes"
)

// nolint: paralleltest,tparallel
func TestParser(t *testing.T) {
t.Parallel()
t.Setenv(configurator.LicenseKeyEnvKey, "")
t.Setenv(configurator.DataSourceNameEnvKey, "")
Comment on lines +25 to +26
Copy link
Contributor Author

@gsanchezgavier gsanchezgavier Aug 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hitting an error since i configured the env vars to run the Tilt environment.

Copy link
Member

@paologallinaharbur paologallinaharbur Aug 4, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure I am following, are they colliding?

Copy link
Contributor Author

@gsanchezgavier gsanchezgavier Aug 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test also covers the code where these env vars are expanded, if you have them in the session you are execute them , the test will fail (because of the expectations are checking other values for these).
I realize this "week" isolation because I exported this envars to run the e2e locally.


// it relies on testdata/<placeholder>.yaml and testdata/<placeholder>.expected.yaml
testCases := []string{
Expand Down Expand Up @@ -51,6 +53,9 @@ func TestParser(t *testing.T) {

//nolint: tparallel
func TestDataSourceName(t *testing.T) {
t.Setenv(configurator.LicenseKeyEnvKey, "")
t.Setenv(configurator.DataSourceNameEnvKey, "")

configWithDataSourceName := `
data_source_name: %s
newrelic_remote_write:
Expand Down Expand Up @@ -79,6 +84,9 @@ newrelic_remote_write:

//nolint: tparallel
func TestLicenseKey(t *testing.T) {
t.Setenv(configurator.LicenseKeyEnvKey, "")
t.Setenv(configurator.DataSourceNameEnvKey, "")

configWithLicense := `
newrelic_remote_write:
license_key: %s
Expand Down
24 changes: 24 additions & 0 deletions test/e2e/metrics.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
entities:
- entityType: FakeSelfMetrics
metrics:
- name: prometheus_agent_active_series
- name: prometheus_target_interval_length_seconds
- name: prometheus_target_scrape_pool_targets
- name: prometheus_remote_storage_samples_pending
- name: prometheus_remote_storage_samples_in_total
- name: prometheus_remote_storage_samples_retried_total
- name: prometheus_agent_corruptions_total
- name: prometheus_remote_storage_shards
- name: prometheus_sd_kubernetes_events_total
- name: prometheus_agent_checkpoint_creations_failed_total
- name: prometheus_agent_checkpoint_deletions_failed_total
- name: prometheus_remote_storage_dropped_samples_total
- name: prometheus_remote_storage_failed_samples_total
- name: prometheus_sd_kubernetes_http_request_total
- name: prometheus_agent_truncate_duration_seconds_sum
- name: prometheus_build_info
- name: process_resident_memory_bytes
- name: process_virtual_memory_bytes
- name: process_cpu_seconds_total
- name: scrape_pool_targets
- name: scrape_duration_seconds
21 changes: 21 additions & 0 deletions test/e2e/test-specs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
description: Newrelic Prometheus Configurator E2E Test

custom_test_key: cluster_name

scenarios:
- description: Scrape test environment metrics
before:
- cd ../../ && NR_PROM_LICENSE_KEY=${LICENSE_KEY} NR_PROM_CLUSTER=${SCENARIO_TAG} make tilt-ci
Copy link
Member

@paologallinaharbur paologallinaharbur Aug 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, this is not that nice, could it be a step of the pipeline? How nri-kubernetes works without this?

Copy link
Member

@paologallinaharbur paologallinaharbur Aug 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean the cd ../../ &&

Copy link
Member

@paologallinaharbur paologallinaharbur Aug 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also below tilt down is way more clear and simple

Copy link
Contributor Author

@gsanchezgavier gsanchezgavier Aug 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree is ugly, but is not complex i would say, is the simpler and clear solution i came up to actually re-use the make target and not duplicate bash scripts all over the repo.

Copy link
Contributor Author

@gsanchezgavier gsanchezgavier Aug 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like there is another way but i found it more obscure WDYT
https://superuser.com/questions/370575/how-to-run-make-file-from-any-directory

after:
- tilt down
tests:
metrics:
- source: "metrics.yml"
except_metrics:
# TODO check why this ones don't appear
- prometheus_remote_storage_dropped_samples_total
- prometheus_remote_storage_failed_samples_total
- scrape_pool_targets

# Need to activate kubernetes_sd for this metric to appear
- prometheus_sd_kubernetes_http_request_total
10 changes: 8 additions & 2 deletions values-dev.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,11 @@ images:
pullPolicy: Never
tag: "latest"

cluster: tilt-local
licenseKey: fake-license
# Configure your <cluster name> and <license key> by the env vars NR_PROM_CLUSTER and NR_PROM_LICENSE_KEY
# cluster: <cluster name>
# licenseKey: <license key>
# nrStaging: true

config:
common:
scrape_interval: 15s