- FEATURES
- USAGE
- Generate Allure Results
- ALLURE DOCKER SERVICE
- PORT 4040 Deprecated
- Opening & Refreshing Report
- Extra options
- Allure API
- Send results through API
- Customize Executors Configuration
- API Response Less Verbose
- Switching version
- Switching port
- Updating seconds to check Allure Results
- Keep History and Trends
- Override User Container
- Export Native Full Report
- Customize Emailable Report
- Allure Customized Plugins
- Allure Options
- SUPPORT
- DOCKER GENERATION (Usage for developers)
Allure Framework provides you good looking reports for automation testing. For using this tool is required to install a server. You could have this server running on Jenkins or if you want to see reports locally, you need to run some commands on your machine. This work results tedious, at least for me :)
For that reason, this docker container allows you to see up to date reports simply mounting your allure-results
directory (for a Single Project) or your projects
directory (for Multiple Projects). Every time appears new results (generated for your tests), Allure Docker Service will detect those changes and it will generate a new report automatically (optional: send results / generate report through API), what you will see refreshing your browser.
- Useful for developers who wants to run tests locally and want to see what were the problems during regressions.
- Useful for the team to check the tests status for every project.
- Repository: frankescobar/allure-docker-service
Docker container versions are based on binary Allure 2 releases
Allure Docker Service supports architectures amd64, arm32v7 and arm64v8.
The following table shows the variation of provided images.
Tag | Base Image | Arch | OS |
---|---|---|---|
0.20.7-amd64 | amd64/adoptopenjdk:11-jre-openj9-bionic | amd64 | ubuntu |
0.20.7-arm32v7 | arm32v7/adoptopenjdk:11-jdk-hotspot-bionic | arm32v7 | ubuntu |
0.20.7-arm64v8 | arm64v8/adoptopenjdk:11-jre-hotspot-bionic | arm64v8 | ubuntu |
The following table shows the provided Manifest Lists.
Tag | allure-docker-service Base Image |
---|---|
latest, 2.13.3 | frankescobar/allure-docker-service:2.13.3-amd64 |
frankescobar/allure-docker-service:2.13.3-arm32v7 | |
frankescobar/allure-docker-service:2.13.3-arm64v8 |
First at all it's important to be clear. This container only generates reports based on results. You have to generate allure results according to the technology what are you using.
We have some examples projects:
- allure-docker-java-testng-example
- allure-docker-java-junit4-example
- allure-docker-java-cucumber-jvm-example
- allure-docker-nodejs-cucumber-example
- allure-docker-nodejs-typescript-cucumber-example
- allure-docker-nodejs-typescript-mocha-example
- allure-docker-python-behave-example
- allure-docker-python-pytest-example
- AllureDockerCSharpExample
In this case we are going to generate results using the java project allure-docker-java-testng-example of this repository.
Go to directory allure-docker-java-testng-example via command line:
cd allure-docker-java-testng-example
Execute:
mvn test -Dtest=FirstTest
If everything is OK, you should see something like this:
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.allure.docker.FirstTest
13:19:03.028 [main] INFO com.allure.docker.FirstTest - test1
13:19:03.044 [main] DEBUG io.qameta.allure.AllureLifecycle - Adding attachment to item with uuid 4b282bd9-6a0f-4fc3-a5cc-be6e8220d3c6
13:19:03.124 [main] INFO com.allure.docker.FirstTest - test2
13:19:03.133 [main] DEBUG io.qameta.allure.AllureLifecycle - Adding attachment to item with uuid e2097440-e9e8-46e9-8b9d-09467b5a49b1
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.702 s <<< FAILURE! - in com.allure.docker.FirstTest
[ERROR] test2(com.allure.docker.FirstTest) Time elapsed: 0.028 s <<< FAILURE!
java.lang.AssertionError: FAILURE ON PURPOSE
at com.allure.docker.FirstTest.test2(FirstTest.java:37)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] FirstTest.test2:37 FAILURE ON PURPOSE
[INFO]
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.600 s
[INFO] Finished at: 2019-09-16T13:19:03+01:00
[INFO] ------------------------------------------------------------------------
There are 2 tests, one of them failed. Now you can see the allure-results
diretory was created inside of allure-docker-java-testng-example project.
Just it has left 1 step more. You have to run allure-docker-service
mounting your allure-results
directory.
Start the container for a single project -> SINGLE PROJECT - LOCAL REPORTS
Docker Image: https://hub.docker.com/r/frankescobar/allure-docker-service/
Project Type | Port | Volume Path | Container Volume Path |
---|---|---|---|
Single Project | 5050 | ${PWD}/allure-results | /app/allure-results |
${PWD}/allure-reports | /app/default-reports | ||
Multiple Projects | 5050 | ${PWD}/projects | /app/projects |
To improve the navigability is recommended to install an Extension/AddOn in your browser:
- Google Chrome > JSONView > https://chrome.google.com/webstore/detail/jsonview/chklaanhfefbnpoihckbnefhakgolnmc?hl=en
- Mozilla Firefox > JSONView > https://addons.mozilla.org/en-US/firefox/addon/jsonview/
This option is recommended for local executions. You should attach the volume where your results are being generated locally for your automation project.
All the information related local executions will be stored in the default
project what is created when you start the container. You can see the complete info using the GET /projects/default
endpoint:
From this directory allure-docker-java-testng-example execute next command:
docker run -p 5050:5050 -e CHECK_RESULTS_EVERY_SECONDS=3 -e KEEP_HISTORY="TRUE" \
-v ${PWD}/allure-results:/app/allure-results \
-v ${PWD}/allure-reports:/app/default-reports \
frankescobar/allure-docker-service
From this directory allure-docker-java-testng-example execute next command:
docker run -p 5050:5050 -e CHECK_RESULTS_EVERY_SECONDS=3 -e KEEP_HISTORY="TRUE" \
-v "/$(pwd)/allure-results:/app/allure-results" \
-v "/$(pwd)/allure-reports:/app/default-reports" \
frankescobar/allure-docker-service
Using docker-compose is the best way to manage containers: allure-docker-java-testng-example/docker-compose.yml
version: '3'
services:
allure:
image: "frankescobar/allure-docker-service"
environment:
CHECK_RESULTS_EVERY_SECONDS: 1
KEEP_HISTORY: "TRUE"
ports:
- "5050:5050"
volumes:
- ${PWD}/allure-results:/app/allure-results
- ${PWD}/allure-reports:/app/default-reports
From this directory allure-docker-java-testng-example execute next command:
docker-compose up allure
If you want to run in background:
docker-compose up -d allure
You can see the logs:
docker-compose logs -f allure
NOTE:
- Read about PORT 4040 Deprecated in case you are using previous versions.
- The
${PWD}/allure-results
directory could be in anywhere on your machine. Your project must generate results in that directory. - The
/app/allure-results
directory is inside of the container. You MUST NOT change this directory, otherwise, the container won't detect the new changes. - The
/app/default-reports
directory is inside of the container. You MUST NOT change this directory, otherwise, the history reports won't be stored.
NOTE FOR WINDOWS USERS:
${PWD}
determines the current directory. This only works for GIT BASH. If you want to use PowerShell or CMD you need to put your full path toallure-results
directory or find the way to get the current directory path using those tools.
Available from Allure Docker Service version 2.13.3
With this option you could generate multiple reports for multiple projects, you can create, delete and get projects using Project Endpoints. You can use Swagger documentation to help you.
docker run -p 5050:5050 -e CHECK_RESULTS_EVERY_SECONDS=3 -e KEEP_HISTORY="TRUE" \
-v ${PWD}/projects:/app/projects \
frankescobar/allure-docker-service
docker run -p 5050:5050 -e CHECK_RESULTS_EVERY_SECONDS=3 -e KEEP_HISTORY="TRUE" \
-v "/$(pwd)/projects:/app/projects" \
frankescobar/allure-docker-service
Using docker-compose is the best way to manage containers: allure-docker-multi-project-example/docker-compose.yml
version: '3'
services:
allure:
image: "frankescobar/allure-docker-service"
environment:
CHECK_RESULTS_EVERY_SECONDS: 1
KEEP_HISTORY: "TRUE"
KEEP_HISTORY_LATEST: 25
ports:
- "5050:5050"
volumes:
- ${PWD}/projects:/app/projects
docker-compose up allure
If you want to run in background:
docker-compose up -d allure
You can see the logs:
docker-compose logs -f allure
NOTE:
- Read about PORT 4040 Deprecated in case you are using previous versions.
- The
/app/projects
directory is inside of the container. You MUST NOT change this directory, otherwise, the information about projects won't be stored.
NOTE FOR WINDOWS USERS:
${PWD}
determines the current directory. This only works for GIT BASH. If you want to use PowerShell or CMD you need to put your full path toallure-results
directory or find the way to get the current directory path using those tools.
- Creating the project
my-project-id
using the endpointPOST /projects
:
- You can see all the existent projects using the endpoint
GET /projects
:
-
The
default
project is always created automatically, it shouldn't be removed. -
And get specific information using the endpoint
GET /projects/{id}
If we want to generate reports for this specific project we need to use the same Action Endpoints that we used for a single project, but the difference now is we need to use the query parameter project_id
to specify our new project.
For example if we want to get the latest
report for a single project
, generally we execute this command:
- http://localhost:5050/latest-report/ >> http://localhost:5050/projects/default/reports/latest/index.html?redirect=false
This command will return the latest
report from the default
project as you see in the url above
If we want to get the latest
report from our new project we need to execute this one:
- http://localhost:5050/latest-report?project_id=my-project-id >> http://localhost:5050/projects/my-project-id/reports/latest/index.html?redirect=false
You can appreciate the difference in the path /projects/{PROJECT_ID}/....
You can use any Action Endpoints, but don't forget to pass the parameter project_id
with the right project id that you want to interact with it.
'GET' /latest-report?project_id=my-project-id
'POST' /send-results?project_id=my-project-id
'GET' /generate-report?project_id=my-project-id
'GET' /clean-results?project_id=my-project-id
'GET' /clean-history?project_id=my-project-id
'GET' /emailable-report/render?project_id=my-project-id
'GET' /emailable-report/export?project_id=my-project-id
'GET' /report/export?project_id=my-project-id
We are going to attach our volume NOT for our local allure-results
. For this case is necessary to store the information regarding all our projects. The project structure is this one:
projects
|-- default
| |-- results
| |-- reports
| | |-- latest
| | |-- ..
| | |-- 3
| | |-- 2
| | |-- 1
|-- my-project-id
| |-- results
| |-- reports
| | |-- latest
| | |-- ..
| | |-- 3
| | |-- 2
| | |-- 1
NOTE:
- You MUST NOT MODIFY MANUALLY the structure directory of any project, you could affect the right behaviour.
- If you don't attach your volume with the proper path
/app/projects
you will lost the information about the projects generated for you.
The first versions of this container used port 4040
for Allure Report and port 5050
for Allure API.
The latest version includes new features Multiple Projects
& Navigate detailed previous history/trends
. These improvements allow us to handle multiple projects and multiple history reports.
The only change required from your side is start using only port 5050
and instead to use http://localhost:4040/ for rendering Allure report you should use http://localhost:5050/latest-report
If you are mounting your volume -v ${PWD}/allure-results:/app/allure-results
your allure results are being used and stored in default
project internally in the container, you don't need to change your volume path directory or do anything else. If you want to keep the history reports start to attach another path -v ${PWD}/allure-reports:/app/default-reports
.
If you are already using port 4040
, NO WORRIES. The Allure Report exposed in port 4040
will still being rendered for avoiding compatibility problems.
The only issue you will face will be when you try to navigate the HISTORY from the TREND chart or any other widget aiming to any historic data. The error you will see is HTTP ERROR 404 NOT FOUND
Version | Port | Volume Path | Container Volume Path | Get Latest Report |
---|---|---|---|---|
Previous to 2.13.3 | 4040 | ${PWD}/allure-results | /app/allure-results | http://localhost:4040/ |
From 2.13.3 | 5050 | ${PWD}/allure-results | /app/allure-results | http://localhost:5050/latest-report |
${PWD}/allure-reports | /app/default-reports |
Check the new commands to start the container for a single project or for multiple projects: ALLURE DOCKER SERVICE
If everything was OK, you will see this:
allure_1 | Generating default report
allure_1 | Overriding configuration
allure_1 | Checking Allure Results every 1 second/s
allure_1 | Creating executor.json for PROJECT_ID: default
allure_1 | Generating report for PROJECT_ID: default
allure_1 | * Serving Flask app "app" (lazy loading)
allure_1 | * Environment: production
allure_1 | WARNING: This is a development server. Do not use it in a production deployment.
allure_1 | Use a production WSGI server instead.
allure_1 | * Debug mode: off
allure_1 | * Running on http://0.0.0.0:5050/ (Press CTRL+C to quit)
allure_1 | Detecting results changes for PROJECT_ID: default ...
allure_1 | Creating history on results directory for PROJECT_ID: default ...
allure_1 | Copying history from previous results...
allure_1 | Creating executor.json for PROJECT_ID: default
allure_1 | Generating report for PROJECT_ID: default
allure_1 | Report successfully generated to allure-report
allure_1 | 127.0.0.1 - - [22/May/2020 17:30:16] "GET /emailable-report/render?project_id=default HTTP/1.1" 400 -
allure_1 | Retrying call http://localhost:5050/emailable-report/render?project_id=default in 2 seconds
allure_1 | 127.0.0.1 - - [22/May/2020 17:30:19] "GET /emailable-report/render?project_id=default HTTP/1.1" 400 -
allure_1 | Retrying call http://localhost:5050/emailable-report/render?project_id=default in 2 seconds
allure_1 | 127.0.0.1 - - [22/May/2020 17:30:21] "GET /emailable-report/render?project_id=default HTTP/1.1" 400 -
allure_1 | Retrying call http://localhost:5050/emailable-report/render?project_id=default in 2 seconds
allure_1 | Report successfully generated to allure-report
allure_1 | 127.0.0.1 - - [22/May/2020 17:30:21] "GET /emailable-report/render?project_id=default HTTP/1.1" 200 -
allure_1 | Status: 200
To see your latest
report simply open your browser and access to:
The latest
report is generated automatically and sometimes could be not available temporary until the new latest
report has been generated. If you access to the latest
report url when is not available you will see the NOT FOUND
page. It will take a few seconds until the latest report be available again.
When you use the latest-report
will be redirected to the resource url:
When you start the container for a single report, the default
project will be created automatically, for that reason you are redirected to this endpoint to get information about default
project, you can see this in the path .../projects/default/...
The redirect=false
parameter is used to avoid be redirected to the GET /projects/{id}
endpoint (default behaviour)
Now we can run other tests without being worried about Allure server. You don't need to restart or execute any Allure command.
Just go again to this directory allure-docker-java-testng-example via command line:
cd allure-docker-java-testng-example
And execute another suite test:
mvn test -Dtest=SecondTest
When this second test finished, refresh your browser and you will see there is a new report including last results tests.
We can run the same test suite again and navigate the history:
You can repeat these steps, but now executing the third test
mvn test -Dtest=ThirdTest
Available endpoints:
'GET' /version
'GET' /latest-report
'POST' /send-results
'GET' /generate-report
'GET' /clean-results
'GET' /clean-history
'GET' /emailable-report/render
'GET' /emailable-report/export
'GET' /report/export
'POST' /projects
'GET' /projects
'DELETE' /projects/{id}
'GET' /projects/{id}
'GET' /projects/{id}/reports/{path}
Access to http://localhost:5050 to see Swagger documentation with examples
Available from Allure Docker Service version 2.12.1
After running your tests, you can execute any script to send the generated results from any node/agent/machine to the Allure Docker server container using the Allure API. Use the endpoint POST /send-results
.
- Python script: allure-docker-api-usage/send_results.py
python send_results.py
- Declarative Pipeline script for JENKINS: allure-docker-api-usage/send_results_jenkins_pipeline.groovy
These scripts are sending these example results allure-docker-api-usage/allure-results-example
If you want to clean the results use the endpoint GET /clean-results
(Allure API).
Available from Allure Docker Service version 2.13.3
When you use the GET /generate-report
, you will see this in your report:
If you want to change the execution name
, you need to pass a parameter named execution_name
with the value. Example:
GET /generate-report?execution_name=my-execution-name
If you want to change the execution from
(by default is empty), you need to pass a parameter named execution_from
with the value. Example:
GET /generate-report?execution_from=http://my-jenkins-url/job/my-job/7/
This option allow you to come back to your executor server.
If you want to change the execution icon
(default is empty), you need to pass a parameter named execution_type
with the value. Example:
GET /generate-report?execution_type=jenkins
If the type is not recognized it will take the default icon. You can use different types like:
- jenkins
- teamcity
- bamboo
The icons are based on the native Allure2 Framework:
Available from Allure Docker Service version 2.13.1
Enable API_RESPONSE_LESS_VERBOSE
environment variable if you are handling big quantities of files. This option is useful to avoid to transfer too much data when you request the API. Have in mind the json response structure will change.
environment:
API_RESPONSE_LESS_VERBOSE: 1
You can switch the version container using frankescobar/allure-docker-service:${VERSION_NUMBER}
.
Docker Compose example:
allure:
image: "frankescobar/allure-docker-service:2.13.1"
or using latest version:
allure:
image: "frankescobar/allure-docker-service:latest"
By default it will take last version: https://hub.docker.com/r/frankescobar/allure-docker-service/tags
Inside of the container Allure API
use port 5050
.
You can switch the port according to your convenience.
Docker Compose example:
ports:
- "9292:5050"
Updating seconds to check results
directory to generate a new report up to date.
Docker Compose example:
environment:
CHECK_RESULTS_EVERY_SECONDS: 5
Use NONE
value to disable automatic checking results.
If you use this option, the only way to generate a new report up to date it's using the Allure API.
environment:
CHECK_RESULTS_EVERY_SECONDS: NONE
Available from Allure Docker Service version 2.12.1
Enable KEEP_HISTORY
environment variable to work with history & trends
Docker Compose example:
environment:
KEEP_HISTORY: "TRUE"
If you want to clean the history use the Allure API.
Allure framework allow you to see the latest 20 executions in the history allure-framework/allure2#1059
Available from Allure Docker Service version 2.13.3
You can access to previous history clicking on the Allure image in the report. If the report is not available you will be redirected to the endpoint GET /projects/{id}
Also, Allure Docker Service
by default keeps the latest 20 executions in the history, but you can extend that limit:
environment:
KEEP_HISTORY_LATEST: 28
or you can reduce it
environment:
KEEP_HISTORY_LATEST: 10
Available from Allure Docker Service version 2.13.1
Override the user container in case your platform required it. The container must have permissions to create files inside directories like allure-results
(Single Project) or projects
(Multiple Project) or any other directory that you want to mount.
1000:1000
is the user:group for allure
user
Docker Compose example:
user: 1000:1000
environment:
...
or you can pass the current user using environment variable
user: ${MY_USER}
environment:
...
MY_USER=$(id -u):$(id -g) docker-compose up -d allure
or from Docker you can use parameter --user
docker run --user="$(id -u):$(id -g)" -p 5050:5050 -e CHECK_RESULTS_EVERY_SECONDS=3 -e KEEP_HISTORY="TRUE" \
-v ${PWD}/allure-results:/app/allure-results \
-v ${PWD}/allure-reports:/app/default-reports \
frankescobar/allure-docker-service
Note: It's not a good practice to use root
user to manipulate containers.
Reference:
Available from Allure Docker Service version 2.13.1
You can export the native full report using the endpoint GET /report/export
Allure API.
Available from Allure Docker Service version 2.12.1
You can render and export the emailable report using the endpoints GET /emailable-report/render
and GET /emailable-report/export
Allure API.
By default this report template is using Bootstrap css. If you want to override the css, just you need to pass the enviroment variable EMAILABLE_REPORT_CSS_CDN
. Docker Compose example:
environment:
EMAILABLE_REPORT_CSS_CDN: "https://stackpath.bootstrapcdn.com/bootswatch/4.3.1/sketchy/bootstrap.css"
You can use all these themes: https://bootswatch.com/ or any other boostrap css like https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.css
If you want override the title of the Emailable Report, just you need to pass the environment variable EMAILABLE_REPORT_TITLE
.
environment:
EMAILABLE_REPORT_TITLE: "My Title"
If you want the Emailable Report to redirect to your Allure server, just you need to pass the environment variable SERVER_URL
.
environment:
SERVER_URL: "http://allure-any-cloud.net/latest-report"
If you want to develop a new template, create a local directory (my-template
as example) with a file named default.html
. In that file you can create your own html template, you can use as guide this example: allure-docker-api/templates/default.html using Jinja syntax. Don't rename your local template, always the file must be named default.html
.
Mount that directory to the container like in the example and pass the environment variable FLASK_DEBUG
with value 1
.
This variable will allow you to use hot reloading
, you can update the content of default.html
locally and use the endpoint emailable-report/render
(Allure API) to see your changes applied in the browser.
environment:
FLASK_DEBUG: 1
volumes:
- ${PWD}/my-template:/app/allure-docker-api/templates
If you want to use your own Allure plugins you can mount your plugin directory
environment:
...
volumes:
- ${PWD}/my-plugin:/allure/plugins/my-plugin
References:
Some frameworks/adaptors don't support allure properties to set up links for Tracker Management Systems
or Issue/Bug Trackers
. In that case you need to set up ALLURE_OPTS
environment variable:
- For Allure1 (XML results)
environment:
CHECK_RESULTS_EVERY_SECONDS: 1
ALLURE_OPTS: "-Dallure.tests.management.pattern=https://example.org/tms/%s -Dallure.issues.tracker.pattern=https://example.org/issue/%s"
- For Allure2 (JSON results). Generally it's not necessary to do this because the properties are configured it in the adaptor/framework and stored in
allure-results
directory. The properties format is different:
allure.link.mylink.pattern=https://example.org/mylink/{}
allure.link.issue.pattern=https://example.org/issue/{}
allure.link.tms.pattern=https://example.org/tms/{}
Reference:
- https://docs.qameta.io/allure/#_test_management_and_bug_tracking_systems_integrations
- https://www.swtestacademy.com/allure-testng/
- https://docs.qameta.io/allure/#_configuration
- https://docs.qameta.io/allure/#_config_samples
- https://docs.qameta.io/allure/#_job_dsl_plugin
sudo apt-get update
sudo apt install -y docker.io
If you want to use docker without sudo, read following links:
- https://docs.docker.com/engine/installation/linux/linux-postinstall/#manage-docker-as-a-non-root-user
- https://stackoverflow.com/questions/21871479/docker-cant-connect-to-docker-daemon
docker build -t allure-release -f docker/archive/Dockerfile --build-arg RELEASE=2.13.3 .
docker run -d -p 5050:5050 allure-release
docker container ls
docker exec -it ${CONTAINER_ID} bash
docker exec -it ${CONTAINER_ID} tail -f log
docker container rm $(docker container ls -a -q) -f
docker image rm $(docker image ls -a -q)
docker ps -q -f status=exited | xargs docker rm
docker images -f dangling=true | xargs docker rmi
docker login
docker tag allure-release frankescobar/allure-docker-service:${PUBLIC_TAG}
docker push frankescobar/allure-docker-service
docker tag allure-release frankescobar/allure-docker-service:latest
docker push frankescobar/allure-docker-service
docker run -d -p 5050:5050 frankescobar/allure-docker-service
docker run -d -p 5050:5050 frankescobar/allure-docker-service:2.13.3