Feature request: same behavior for firstTimeOnly and lastTimeOnly in @Before/AfterMethod even when invocationCount is 1 or data provider test driven parallel or not #1711
Comments
|
Could you share some sample for each use case you imagine? They will be functional tests when we will implement it. |
|
Hi, Here it is: public class MySecondTestClass {
@BeforeMethod(firstTimeOnly = true)
public void beforeMethod() {
System.out.println("do nothing in beforeMethod");
System.out.println(" Thread in beforeMethod: " + Thread.currentThread().getId());
}
@AfterMethod(lastTimeOnly = true)
public void afterMethod() {
System.out.println("do nothing in afterMethod");
System.out.println(" Thread in afterMethod: " + Thread.currentThread().getId());
}
@Test
public void testSecondMethod1() {
System.out.println(" Inside test method testSecond from second class");
System.out.println("Thread in testSecondMethod1: " + Thread.currentThread().getId());
assert true;
}
@Test(groups = "broken")
public void testSecondMethod2() {
System.out.println(" Thread in testSecondMethod2: " + Thread.currentThread().getId());
fail(" inside testSecondMethod2 fail");
}
/**
* test with parameter
* @param param
*/
@Test(dataProvider = "dp3")
public void testSecondMethod3(String param) {
System.out.println(" testSecondMethod3 param: " + param);
System.out.println(" Thread in testSecondMethod3: " + Thread.currentThread().getId());
assert true;
}
/**
* data provider
*/
@DataProvider(name = "dp3", parallel = true)
public Object[][] getDP() {
return new Object[][] {
{
"T"
},
{
"F"
}
};
}
}Output: Having firstTimeOnly = true in BeforeMethod and lastTimeOnly = true in AfterMethod would have expected effect that these configuration methods will be run once for testSecondmethod3. It should work also for non-parallel data provider and regardless of invocationCount. It is quite interesting that it works well (6.11) on non-parallel data provider: |
|
Hi, Imagine having a test class with a dozen of test methods, each iterating multiple times based on data providers and a BeforeMethod, all this leading to a number of 10k+ iterations. So far it works well and quite quickly when run sequentially considering each test makes some actions (stimulus) and then queries some data from data containers (response) with timeout (few seconds). In cases there are some issues, from few minutes this test class can take hours because of queries waiting for valid data until timeout expires. BeforeMethod is used to clear data containers before each iteration so the queries are not looking into a lot of unnecessary data from previous tests. Logging would be also affected in this case of failures as it will log the data queries did not match on (huge amount of data for each iteration, if keeping all data from containers) for debug purposes. Making parallel data providers will solve a lot of time in such cases but this implies removing config method BeforeMethod since it may clear data container (a single provider) another thread is querying on. Thanks a lot! |
TestNG Version
Expected behavior
It would be really useful if firstTimeOnly and lastTimeOnly would be enhanced to work with parallel dataproviders in the same way they work for invocationCount greater than 1. So the result would be to run BeforeMethod/AfterMethod exactly once for parallel enabled data provider, before the first run/after the last run of the test method using that data provider. It makes more sense in the context of parallel data providers than on sequential ones. But it may be enhanced to have effect regardless of parallel state of data provider. Also, it should behave well also when test class is instantiated with factory multiple times and group-by-instances set to true.
Actual behavior
Is the issue reproductible on runner?
Test case sample
The text was updated successfully, but these errors were encountered: