Project that illustrates how to use contract testing with pact w/ a ktor backend.
This is an example project illustrating how to use contract testing (pact) with a javascript consumer and a ktor provider. This represents the common setup where there is a frontend consuming a REST-like API (JS consumer) and a backend API (the provider).
The backend is a very minimal ktor API. Some working knowledge of the framework is recommended.
Contract testing is a technique for testing an integration point by checking each application in isolation to ensure the messages it sends or receives conform to a shared understanding documented in a "contract." - Pact docs
In general pact works by asserting that the calls to the test doubles, return the same result as a call to the real application would. In practical terms, this means that:
- The consumer (JS client) runs its tests and generates a contract document;
- The consumer publishes that contract somewhere, where the provider can fetch it. In our case we will be using the pact broker;
- The provider is connected to the broker, and the contract is used to assert that the responses match the expectations of the client;
- The provider publishes its results back to the broker, giving the operators visibility whether the updates to the API break the consumer's expectations.
This is an interesting tool because it allows developers to cover a broad range of scenarios that are traditionally covered by expensive integration tests requiring the presence of the consumer and producer at the same time.
- Docker
- Docker compose
- NPM
- JDK 11 or above
In the /broker
folder there is a compose file (for docker compose) that allows you to launch an instance of the pact
broker, with the following command:
docker-compose -f pact-broker.yml up
You can verify that the broker is up by opening the browser on http://localhost:9292 For further information please check the pact broker documentation
The /consumer
folder contains a sample javascript client, that will consume our API. The plain old jest tests in the
/consumer/__test__/contract
folder exercise the actual calls to the API defined in /consumer/src/consumer.js
.
You can find more information about the specific pact features used in the tests here
In order to run the consumer tests, you need to run the following commands:
npm install
npm run test:consumer
You'll note that there is a new json file under /consumer/__test__/contract/pacts
, this is our contract, which we will now
publish to the pact broker, with the following command (you'll need the broker running):
npm run publish:contract
In order to get started with the tests on the consumer, there's a bit of boilerplate code at play, the /consumer/__test__/helpers
takes care of that:
- pactSetup.js - Setup the port of the mock server and several settings regarding the generation of the contract. In particular the consumer and provider names.
- pactTestWrapper.js - starts and stops the mock server before and after each test respectively.
- publish.js - publishes the generated contract to the broker
On the package.json
its also good to look at the test:consumer
and publish:contract
scripts.
After publishing the contract,
The consumer is adapted from this Test Automation University course
For the provider, we've implemented two different ways of running the pact verification: using the gradle task and using the junit 5 extension.
In either case, the broker needs to be running in order for the provider to fetch the contract.
In order to run with gradle, you can use the following commands (in the provider folder):
gradlew pactVerify # verify the contract
gradlew pactPublish # publish the results back to the broker
Behind the scenes:
With Gradle, we are using the pact gradle plugin. It adds a series of
tasks, namely the pactVerify
and pactPublish
tasks above.
pact {
// Turn on reporting: https://github.com/pact-foundation/pact-jvm/tree/master/provider/gradle#verification-reports
reports {
defaultReports()
json
}
// Add the broker where the contracts are stored: https://github.com/pact-foundation/pact-jvm/tree/master/provider/gradle#verification-reports
broker{
pactBrokerUrl = 'http://localhost:9292'
}
serviceProviders {
// The name of the provider needs to match what the client specified initially. See /consumer/__test__/helpers/pactSetup.js
ClientsApi {
// The provider needs to start and stop. "startProvider" and "terminateProvider" are gradle tasks
startProviderTask = startProvider
terminateProviderTask = terminateProvider
requestFilter = { req ->
// For some reason even when the server is up and running I needed to introduce a bit of delay :(
Thread.sleep(50)
}
protocol = 'http'
host = 'localhost'
port = 8080
path = '/'
// Signal that the pacts will come from the pact broker.
fromPactBroker {}
}
}
}
In order to publish the results of the verification back to the broker gradle needs to run with the flag pact.verifier.publishResults
set to true. A quick and easy way to do that is to use System.setProperty("pact.verifier.publishResults", "true");
in your build.gradle
.
You can also invoke gradle with: -Ppact.verifier.publishResults=true
.
One of the important aspects to keep in mind, is that the provider needs to execute in order to run the verification task. In this case, we created the following tasks to manage the provider's lifecycle in the context of the contract verification:
task startProvider(type: SpawnProcessTask, dependsOn: 'shadowJar') {
command "java -jar ${shadowJar.archiveFile.get().asFile.path}"
ready 'Started Application' // This is important, as it is the string the the plugin looks for in order to proceed
}
task terminateProvider(type: KillProcessTask)
The code here should be relatively straightforward as the startProvider
task runs after the uber jar has been created,
and it just starts that jar. The terminateProvider
task kills the process running the jar once the verification ends.
This requires the usage of two other gradle plugins:
This setup works well, but as the service becomes more elaborate it may be needed to switch from running the jar directly to using something like docker compose. Something like this plugin may be an interesting choice.
An alternative way to run the pact verification in this context is to use the junit 5 extension. It has some advantages over the gradle task, namely more control over the initial required state for each interaction in the contract, and if you're already using something like the test containers its a breeze to set up even a provider with other dependencies (databases, caches, etc).
With the junit integration, the pact verification and publication runs as part of the test suite. So you can run the tests with the following command:
gradlew test
Behind the scenes:
In order to run the pact verification as part of the test suite, the following dependency is required (in build.gradle
):
implementation 'au.com.dius.pact.provider:junit5:4.1.9'
With that out of the way, your test class is almost a regular junit test, with a few nuances:
- The class needs to be annotated with the
@Provider("ClientsApi")
annotation. ReplaceClientsApi
with the name of the provider that was defined when the consumer generated the contract; - In this case we're using a broker, so we also need to specify that via annotation on the class:
@PactBroker(host = "localhost", port = "9292", scheme = "http")
; - Before any test runs we need to ensure that the provider is up and running. In this case we're using the standard junit 5
@BeforeAll
method annotation; - When all the tests are done, the provider should shut down. In this case we're using the standard junit 5
@AfterAll
method annotation to do that; - A test target needs to be defined, using the @TestTarget annotation;
- For each interaction in the contract, a new method with the
@State
annotation is required. This allows the setup of any required state for a particular interaction. - Finally, test template needs to be provided, e.g:
@TestTemplate
@ExtendWith(PactVerificationInvocationContextProvider::class)
fun pactVerificationTestTemplate(context: PactVerificationContext) {
context.verifyInteraction()
}
Note that in order to publish the verification result, the pact.verifier.publishResults
needs to be set.
The full working example can be found at /provider/src/test/kotlin/com/example/PactVerificationTests.kt
.
This approach is more flexible than using the gradle task, note however that its probably a good idea not to run the pact verifications as part of the regular test suite (as it will slow down your regular unit tests). A good idea would be to tag your contract tests, so that you can selectively run only on specific environments.
For instance, if the contract test is annotated with @Tag("ContractTest")
, in build.gradle
we could have:
tasks.withType(Test) {
useJUnitPlatform{
if (System.getenv("CI") != "true") {
excludeTags "ContractTest"
}
}
}
- Fork it (https://github.com/felix19350/pact-ktor-example/fork)
- Create your feature branch (
git checkout -b feature/fooBar
) - Commit your changes (
git commit -am 'Add some fooBar'
) - Push to the branch (
git push origin feature/fooBar
) - Create a new Pull Request