Smoke Testing

Automated Smoke Testing

Description

This document describes how to run smoke tests and add a smoke tests for new configurations. For Smoke tests we run Platform UI tests. These takes about 30 minutes to run.

See Eclipse Automated Unit and Integration Testing for more details on the testing framework.

Set up

All the necessary jobs are listed under Smoke tests tab in Releng jenkins instance. Tests are triggered using Start Smoke tests job

There are two types of jobs

1. Jobs that run on dedicated agents, like ppc64le agents, Mac agents and Windows agents. These use a physical/virtual machine
2. Jobs that run on a containerized environment. Like centos/ubuntu/sles on x86_64. These use docker container with customized docker image

The setup required is we should be able to launch Eclipse SDK in those systems either on the dedicated agent or docker container. And GUI should be connected either by vnc or someother means.

Setup on dedicated agents

The OS needs to be configures in such a way that eclipse can start. Also need to install VNC or its equivalent for exporting UI for the jenkins use

Here is an example job running on CentOS ppc64le platform. This one has java 11 download link as a parameter. This job downloads java 11 using the link and uses it to run Eclipse during test run.

Setup required on docker containers

Docker image needs to be created with all necessary tools to run Eclipse. Here is a CentOS 8.x docker file.

Here is an example job running on CentOS 8.x x86_64 platform. This one uses installed java 11 from the jenkins.

Add new smoke tests

Need to decide on the following before adding

1. Container or dedicated agent
2. Installed Java or dynamically downloaded java

Container based tests (Recommended)

• Create a new docker image using a Dockerfile similar to CentOS 8.x docker file.
• Duplicate one of the container based tests and use installed JRE and above created Docker image

Dedicated Agent based tests (Only if containers are not possible)

• Create a new Machine with GUI.
• Attach it to releng jipp by taking help of foundation
• Duplicate dedicated agent based smoke tests job CentOS ppc64le platform
• For Java you can either install required java on the system or use dynamically downloaded java as used in the above job.