Generic end-to-end tests

[pavolloffay]

At the beginning a small explanation for people who don't know/use xdock test_driver:

test_driver's main starts agent, collector and query services using exec.command(cmd, args) with hardcoded parameters (flags). It also initializes C* storage by dropping keyspace and then creating the schema ( the script is present in the container).

test_driver container is very nice to use in tests because it contains all necessary jaeger processes. To be specific I use it with testcontainers. It's super simple/easy to write integration tests. For languages where frameworks like testcontainers do not exist it can be simply used with docker-compose.

I propose to make it more generic and make it work with different storages and maybe be able to pass arbitrary flags to jaeger components.

Any concerns/suggestions are welcome.

[pavolloffay]

We could also make all-in-one to work with different storages (it will need storage initialization part)

[yurishkuro]

I thought we had a similar issue elsewhere, but can't find it. I agree that the test_driver could be more generic. I am not sure why it needs to start other components vs. starting them via docker-compose and having the driver just run & validate the test.

[yurishkuro]

@pavolloffay this was the issue I was referring to jaegertracing/jaeger-kubernetes#22

[pavolloffay]

I have renamed this issue to Generic end-to-end tests. Currently we have this use cases:

• test Kubernetes/OpenShift/Docker-compose deployment (with various storages)
• Spark job dependency links tests. This test creates traces and verifies that query rest API returns correct dep.links.

To do the above we need a tracer and parse response objects. For dependency links we need also report zipkin shared spans (jaeger go client should be good to go, however this does not support merged PRC spans and all zipkin formats (json, thrift, v2 json)).

[pavolloffay]

This could be implemented as an app with rest API which would create data based on parameters and validate. Maybe there could be a java process running brave tracer for zipkin data creation.

[yurishkuro]

Sorry, I don't fully understand the use case with dependency links, maybe we can move that to a separate issue?

I would like the test_driver NOT be responsible for starting the backend components. The typical setup of a crossdock test is that the components are started by the docker-compose, the test waits for all of them to start responding to a health check, and then it executes one or more tests (permutations) by sending requests against a driver app (test_driver). The behavior currently tested by the test_driver is independent of how the backend components are started and what their implementations are, i.e. we can bring them with C or ES backends.

I am not sure if it can also work for testing K8s / OpenShift templates because there the components will be started by the respective orchestration rather than docker-compose.

[pavolloffay]

Generally we need two things:

• a component to create tracing data - e.g. xdock-java
• a component that calls xdock tracer and verifies tracing data - test_driver

I think it could work for testing k8s deployments too. We could use arquillan-cube to handle deployment of the jaeger resource files and add test resource file with definitions for test-driver and tracer/s being used (xdock images). The test would port-forward test-driver's http port and we would just call specific tests to be executed. So our test class would play role of crossdock service.

cc @jpkrohling

[jpkrohling]

What's the state of this issue? We'll eventually need something like this, so, would be nice to know what is still missing and plan accordingly.

[jpkrohling]

cc @objectiser

[pavolloffay]

@jpkrohling there is no update on this issue. I needed it for https://github.com/jaegertracing/jaeger-kubernetes/. In the end I have used junit+arqullian cube

[jpkrohling]

I assume this is still needed, then. @objectiser is this also part of what we'd need in the near future?

[objectiser]

Not sure - we need to first identify how the full testing would be performed with @jkandasa and once we have a gap analysis this may be appropriate.

[mdelapenya]

This issue came to my attention as it mentioned testcontainers as not existing in Go. It was Sep 2017, which was totally true at that time. But nowadays we have a Go flavor of the library, which you may want to explore: https://github.com/testcontainers/testcontainers-go.

We'd be more than happy to collaborate with you if needed 😃