Groups in @BeforeMethod and @AfterMethod don't work as expected #549

PW999 · 2014-09-19T09:40:42Z

Either I'm misinterpreting or misusing TestNG-6.8.8, but it seems like the groups attribute in the @BeforeMethod and @AfterMethod aren't working as expected.

Consider following class:

public class MyClass {
    public void method1() {
        System.out.println("Method 1 invocation");
    }
    public void method2() {
        System.out.println("Method 2 invocation");
    }
}

and following test

public class MyClassTest {
    MyClass o = new MyClass();
    @BeforeMethod(groups = "method1") 
    public void before1() {
        System.out.println("Before1");
    }
    @BeforeMethod(groups = "method2") 
    public void before2() {
        System.out.println("Before2");
    }

    @AfterMethod(groups = "method1") 
    public void after1() {
        System.out.println("After1");
    }
    @AfterMethod(groups = "method2") 
    public void after2() {
        System.out.println("After2");
    }

    @Test(groups = "method1")
    public void testMethod1() throws Exception {
        o.method1();
    }
    @Test(groups = "method2")
    public void testMethod2() throws Exception {
        o.method2();
    }
}

I would expect the output to be

Before1
Method 1 invocation
After1
Before2
Method 2 invocation
After2

However, the output is

Before1
Before2
Method 1 invocation
After1
After2
Before1
Before2
Method 2 invocation
After1
After2

The text was updated successfully, but these errors were encountered:

jeangb · 2016-12-16T15:12:03Z

I have the same problem. AfterMethod don't care about the group specified in groups="".

krmahadevan · 2016-12-16T17:57:31Z

@jeangb - I just now tried this in TestNG 6.10 and I cannot recreate the problem.

Here's my sample class

package org.rationale.emotions.github.issue549;

public class MyClass {
    public void method1() {
        System.out.println("Method 1 invocation");
    }

    public void method2() {
        System.out.println("Method 2 invocation");
    }
}

Here's my test class

package org.rationale.emotions.github.issue549;

import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

public class MyClassTest {
    MyClass o = new MyClass();

    @BeforeMethod (groups = "method1")
    public void before1() {
        System.out.println("Before1");
    }

    @BeforeMethod (groups = "method2")
    public void before2() {
        System.out.println("Before2");
    }

    @AfterMethod (groups = "method1")
    public void after1() {
        System.out.println("After1");
    }

    @AfterMethod (groups = "method2")
    public void after2() {
        System.out.println("After2");
    }

    @Test (groups = "method1")
    public void testMethod1() throws Exception {
        o.method1();
    }

    @Test (groups = "method2")
    public void testMethod2() throws Exception {
        o.method2();
    }
}

Here's how my suite xml looks like

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="1265_Suite" parallel="false" verbose="2">
    <test name="92">
        <groups>
            <run>
                <include name="method1"/>
            </run>
        </groups>
        <classes>
            <class name="org.rationale.emotions.github.issue549.MyClassTest"/>
        </classes>
    </test>
</suite>

and here's my output

XmlSuite] [WARN] 'parallel' value 'false' is deprecated, default value will be used instead: 'none'....
... TestNG 6.10 by Cédric Beust ([email protected])
...

[TestNG] Running:
  /Users/krmahadevan/playground/scrappable-project/src/test/resources/549.xml
Before1
Method 1 invocation
After1
PASSED: testMethod1

===============================================
    92
    Tests run: 1, Failures: 0, Skips: 0
===============================================

apreg · 2017-01-19T13:46:35Z

@krmahadevan Could you try to include method2 too like this:

<groups>
            <run>
                <include name="method1"/>
                <include name="method2"/>
            </run>
        </groups>

Does it work this way too for you?

krmahadevan · 2017-02-10T04:15:51Z

@apreg - If I do what you suggested above, then yes I can re-create what you are seeing.

@juherr - Do you think that this is an issue ? I am a bit confused on this.

Here's what TestNG is currently doing :

When user chooses to run by 1 or more groups, TestNG ends up invoking all the @BeforeMethod methods that belong to the groups chosen, for all the tests, instead of selectively running only the relevant @BeforeMethod methods.

juherr · 2017-02-10T10:00:24Z

Currently, it is how groups is working.
In fact, groups is just a way to select methods but there is no relation between methods with the same groups.

If you choose method1 group, then every method with method1 will be selected and run. If you choose method1 and method2 groups, then every method with method1 or method2 will be selected and run.

So, it is not a bug.

But many persons don't understand it (some StackOverflow questions about it) because the behavior is not really logic. So, it could be a good idea to fix the issue.
But it is a big (maybe complicate) behavior move with potential regressions and we need to think about it before.

Ping @cbeust

cbeust · 2017-02-10T17:39:28Z

@juherr What is not logic about the current behavior?

juherr · 2017-02-10T17:53:30Z

@cbeust In fact, many users expect to have groups activated when they select a test.

For example, if they run @Test(groups="a"), they expect @BeforeMethod(groups="a") will be run without specify any group in the test selection.

Another example: if they run @Test(dependsOnGroup="a"), they expect @Test(groups="a") will be run first.

I'm not saying the current behavior is bad but it is not the most intuitive.

cbeust · 2017-02-10T18:41:12Z

Isn't this what alwaysRun = true is for?

juherr · 2017-02-10T18:58:27Z

No, it is not exactly the same goal IMO.

alwaysRun will be always invoked, when groups will be only invoked if a method has one of the groups.

And if you check the sample of this issue, it goes a step further: it expects the group is only activated for the context of the test.

madhukarashok · 2017-06-13T21:09:08Z

Has this issue been resolved . i just tried with testng 6.9.14 and it didn't work for aftermethod groups .

krmahadevan · 2017-06-14T06:06:40Z

@madhukarashok - Can you please help clarify what did you try ? This is not resolved because there's some ambiguity in the expectation. Please feel free to include relevant java and xml samples.

madhukarashok · 2017-06-15T15:07:37Z

i think i overlooked a condition which caused that confusion and im might be able to rectify that.
but i wanted to check why does multiple @aftermethod shows as skipped in the summary but in actual all the methods were executed succesfully.
![aftermethod]
(https://user-images.githubusercontent.com/3843299/27188001-c8627ed4-51ba-11e7-9c5b-e47f5c173cd9.png)

krmahadevan · 2017-06-16T03:59:27Z

@madhukarashok - Am guessing that screenshot is from eclipse. But you still haven't shared an example that can be used to reproduce the problem. Can you please do that ?

madhukarashok · 2017-06-19T14:37:29Z

hi here is the code snippet

package com.testCases;

import org.testng.annotations.AfterGroups;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeGroups;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;


public class sampletest extends basetest {
	private String EcommerceTestCase;
	public Boolean iflag=false;
	//private String ecommerceTestCases;
 	AppDataFromDB apdfdb = null;
 	cartTestartTest sCart =null;
 	cartOverlayTest bopisAddtoCart=null;
 	createOrderTest bopisCreateOrder = null;
 	sPage bopisPage = null;
 	sCart  cart=null;
	private String ecomID; 
	
	public BopisRegressionTests(){
		
	}
	
	@Factory(dataProvider = "DP4")
	public BopisRegressionTests(String testCaseID){
		this.EcommerceTestCase = testCaseID;

	}
	
	@BeforeMethod(groups={"atc"})
	public void setUp(){
		common.setInsertResults(insertResults);
		apdfdb = new AppDataFromDB(insertResults);		
		EcommerceTestCases ecommerceTestCases = apdfdb.getNewTestCaseFROMDB(EcommerceTestCase);
		ecomID = ecommerceTestCases.getTestCaseDescription();
		cart = new ShoppingCart(common);
		bopisPage = new BopisPage(common);
		bopisCreateOrder = new BopisCreateOrderTest(EcommerceTestCase, common);
		bopisAddtoCart = new BopisAddtoCartOverlayTest();
	}
	
	@BeforeGroups(groups={"cart","atc"})
	 public void Login(){
		LoginSetup();	
	}
	
	@Test(groups={"atc"})
	public void baddToCartOverLayVerification(){
	if (ecomID.contains("AddToOverLay")) {
		bopisAddtoCart.addToCartOverlay(EcommerceTestCase, common);
		bopisAddtoCart.overlayverification(EcommerceTestCase,apdfdb);	
	    }
				
	}
	
	@Test(groups={"cart"})
	public void cartverfiy (){
	   if (ecomID.contains("Cart"))  {
		   sCart = new BopisCartTest();	
		   sCart.cart(EcommerceTestCase, common);
	    }  
	 
	}
		
	@AfterMethod(dependsOnGroups={"cart"},dependsOnMethods="cartverfiy ")
	public void addToCart1(){
	if (ecomID.contains("Cart"))  {
		sCart.addToCart();
	}
	}
    
	@AfterMethod(dependsOnGroups={"cart"},dependsOnMethods="addToCart1")
    public void cstoreHeaderValidation1(){
		if (sCart != null ) {
	    sCart.storeHeaderValidation();
		}
    }
}

here is the xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Sherwin Williams Chrome Suite" verbose="0" parallel="none"
	thread-count="3" configfailurepolicy="continue" preserve-order="true">

	<parameter name="configFile" value="config/chrome.win.properties" />
	<parameter name="logLevel" value="info" />
	
	<test name="bRegressionRun">
		<groups>
		  <run>
           <include name="atc"/>
           <include name="cart"/>
          </run> 
        </groups>
		<classes>	
		   <class name="com.sw.testcases.BopisRegressionTests" />				
		</classes>
	</test>
</suite>

my base execution flow is i hav two groups "atc" & "cart" and i have different methods for respective groups which needs to get kicked off when @test is executed. but im having hard time running dependonmethods in conjuncture with @AfterMethod as it keeps spitting out this error "com.sw.testcases.BopisRegressionTests.addToCart1() is depending on method public void com.sw.testcases.BopisRegressionTests.cartverfiy (), which is not annotated with @test or not included."

i jus need one execution for each of this groups in the following order

atc -> baddToCartOverLayVerification method execution completed
2)cart - > cartverify -> (then aftermethod annotations for this group should get kicked off based on dependOnMethod annoations) .

let me know if it makes sense ?

madhukarashok · 2017-06-20T13:35:01Z

hi i tried dependOnGroups & dependOnMethods annotations like shown below and was unsuccesfull

import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeGroups;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;

import com.ecommerce.store.dataaccess.EcommerceTestCases;
import com.sw.datahelper.AppDataFromDB;
import com.sw.pages.BopisPage;
import com.sw.pages.ShoppingCart;

public class BopisRegressionTests extends SWBaseTest {
	private String EcommerceTestCase;
	public Boolean iflag=false;
	//private String ecommerceTestCases;
 	AppDataFromDB apdfdb = null;
 	BopisCartTest sCart =null;
 	BopisAddtoCartOverlayTest bopisAddtoCart=null;
 	BopisCreateOrderTest bopisCreateOrder = null;
 	BopisPage bopisPage = null;
 	ShoppingCart  cart=null;
	private String ecomID; 
	
	public BopisRegressionTests(){
		
	}
	
	@Factory(dataProvider = "DP4")
	public BopisRegressionTests(String testCaseID){
		this.EcommerceTestCase = testCaseID;

	}
	
	@BeforeMethod(alwaysRun = true)
	public void setUp(){
		common.setInsertResults(insertResults);
		apdfdb = new AppDataFromDB(insertResults);		
		EcommerceTestCases ecommerceTestCases = apdfdb.getNewTestCaseFROMDB(EcommerceTestCase);
		ecomID = ecommerceTestCases.getTestCaseDescription();
		cart = new ShoppingCart(common);
		bopisPage = new BopisPage(common);
		bopisCreateOrder = new BopisCreateOrderTest(EcommerceTestCase, common);
		bopisAddtoCart = new BopisAddtoCartOverlayTest();
	}
	
	@Test(groups={"cart"})
	 public void aLogin(){
		LoginSetup();	
	}
	
	@Test(groups={"atc"})
	public void baddToCartOverLayVerification(){
	if (ecomID.contains("AddToOverLay")) {
		bopisAddtoCart.addToCartOverlay(EcommerceTestCase, common);
		bopisAddtoCart.overlayverification(EcommerceTestCase,apdfdb);	
	    }
				
	}
	
	@Test(groups="cart",dependsOnMethods="aLogin")
	public void d1 (){
	   if (ecomID.contains("Cart"))  {
		   sCart = new BopisCartTest();	
		   sCart.cart(EcommerceTestCase, common);
	    }  
	 
	}
		
	@AfterMethod(dependsOnGroups={"cart"},dependsOnMethods="d1")
	public void addToCart1(){
	if (ecomID.contains("Cart"))  {
		sCart.addToCart();
	}
	}
    
	@AfterMethod(dependsOnGroups={"cart"},dependsOnMethods="addToCart1")
    public void cstoreHeaderValidation1(){
		if (sCart != null ) {
	    sCart.storeHeaderValidation();
		}
    }
}

The execution stops at d1 and it doesn't enter the aftermethod annotations which is dependent on groups & dependentonmethod = "d1".

any help would be appreciated .

tnn24 · 2017-09-21T18:13:14Z

How is this not a bug? If BeforeMethod with assigned groups also run on tests not assigned to those groups, what's the groups assignment for? Sorry, I just don't see the logic here.

juherr · 2017-09-21T18:38:46Z

As I said, groups are used for the test selection. As an alternative, you can split your tests in many classes.

Jaruso · 2018-02-11T15:36:46Z

The functionality you are looking for is BeforeGroup.

@BeforeGroups(groups = "method1") 
public void before1() {
    System.out.println("Before1");
}
@BeforeGroups(groups = "method2") 
public void before2() {
    System.out.println("Before2");
}

@AfterGroups(groups = "method1") 
public void after1() {
    System.out.println("After1");
}
@AfterGroups(groups = "method2") 
public void after2() {
    System.out.println("After2");
}

@Test(groups = "method1")
public void testMethod1() throws Exception {
	System.out.println("Method 1 invocation");
}
@Test(groups = "method2")
public void testMethod2() throws Exception {
	System.out.println("Method 2 invocation");
}

Output:

Before1
Method 1 invocation
After1
Before2
Method 2 invocation
After2

The BeforeMethod or AfterMethod will run for EVERY method in the test class IF its group is enabled.

Since you choose to run the class with groups 'method1' and 'method2', methods before1(), before2(), after1(), and after2() are ALL flagged as true. Therefore, both before1() AND before2() will run before EVERY method, and both after1() AND after2() will run after every method.

stachenov · 2018-03-30T08:43:39Z

No, @BeforeGroup doens't do exactly what is needed here. It is only invoked once per group, whereas it would be nice to have a way to invoke a configuration method before every test from a specific group. Ditto for @AfterMethod. Right now, pretty much the only way to achieve it is the Testcase Class per Test Fixture strategy as xUnit Patterns book puts it. Sometimes it's fine, but sometimes one ends up with too many classes this way, especially if inheritance is involved.

I propose to add a kind of group filter to @BeforeMethod. Perhaps, @BeforeMethod(onlyForGroups = ...). Then this method will only be invoked before a test method belonging to one of the listed groups. I have just tried some minimal implementation, works fine for me. Should I submit a PR to discuss it further?

And as far as I can see, issue #780 is very similar, so this should fix it too.

krmahadevan · 2018-03-30T10:07:13Z

@stachenov - Please go ahead and raise a PR with your changes. That would help facilitating the discussion on the actual changes more aptly.

wenijinew · 2019-07-22T14:39:23Z

I tried both 6.10 and 7.0.0-beta, the test result is not expected. Here is the wiki page with details.

If groups cannot group Before and After methods, why don't remove it to avoid confusions. I am quite confused why we need to add one more attribute onlyForGroups rather than fix groups attribute to make it workable in the expected way. (@juherr, if groups is used for test selection, why did we add it to Before/After Method annotation also?)

But anyway, onlyForGroups can work in 7.0.0-beta7.

stachenov · 2019-07-22T16:15:50Z

The wiki page is broken. Opens up a error page instead.

I also find it confusing, but I understand the historical reasons behind it. If groups are used for test selection, it makes perfect sense that they work in exactly the same way for before/after methods. You may have, say, a “slow” group that also includes rather slow startup, and therefore mark the appropriate before method as belonging to the same group, to avoid expensive startup if the slow tests aren't selected. And once groups are defined as test selection mechanism, it makes little sense to use them for method invocation control.

The problem is that maybe it is wrong to have onlyForGroups attribute too, because it also refers to groups, and therefore should not be used for anything else than test selection. But I couldn't think of anything but groups to, well, group method invocations (save for the test-class-per-fixture approach, which works poorly for testing class hierarchies)... Adding another kind of group would make things even more complicated and force people to duplicate stuff in every test method, like @Test(group="slow", invocationGroup="slow"), which doesn't make any sense either.

juherr · 2019-08-03T17:19:27Z

Maybe a better name for TestNG groups could be "tags". But as @stachenov said: historical reason.

CoryKniefel · 2019-12-13T20:48:32Z

I find the documentation lacking for the use of onlyForGroups on AfterMethods. Currently it says:

onlyForGroups | Only for @BeforeMethod and @AfterMethod. If specified, then this setup/teardown method will only be invoked if the corresponding test method belongs to one of the listed groups.

src: https://testng.org/doc/documentation-main.html

In reality, for onlyForGroup to work, it must be accompanied with a "alwaysRun = true".
I'm using version 7.0.0

stachenov · 2019-12-14T07:01:19Z

What happens if you don't specify alwaysRun? I never used it and never had any problems. Maybe it's something v7-specific, but I'd like to clarify what behavior you're expecting and what you're getting instead.

wenijinew · 2020-09-09T04:14:12Z

@stachenov
If you meant this wiki page, then it's still alive.

stachenov · 2020-09-09T04:30:34Z

?! I was replying to @CoryKniefel about alwaysRun. I didn't even know about that wiki page.

wenijinew · 2020-09-29T06:33:47Z

@stachenov
I replied to your message "The wiki page is broken. Opens up a error page instead.".

stachenov · 2020-09-29T06:36:12Z

Oh. But it was apparently broken more than a year ago. I don't think it matters now whether it was indeed so.

sandeshmms · 2021-02-14T05:12:25Z

If we exclude the groups or do not use groups tag in testng.xml file, then @BeforeMethod/@AfterMethod(onlyForGroups) methods run. But if we include the groups, then the @BeforeMethod/@AfterMethod(onlyForGroups) methods do not run. After some research, I found out that both of the above behaviour is TestNG design behaviour. So, as a workaround, when we include any groups, then we need to club onlyForGroups with alwaysRun=true i.e. @BeforeMethod/@AfterMethod(onlyForGroups, alwaysRun=true). But in this case, if there is any SkipException in the preceding/parent config methods, then it is forced to execute the @BeforeMethod/@AfterMethod methods, even when the test method is going to be skipped. So, to avoid this issue, we need to club groups flag with onlyForGroups flag as suggested here. There is a similar issue logged here, when the preceding/parent config methods has failed.

juherr added Feature: before/after Feature: groups labels Aug 14, 2015

juherr added the Need: user answer label Dec 16, 2016

juherr removed the Need: user answer label Feb 10, 2017

stachenov mentioned this issue Mar 30, 2018

Add onlyForGroups parameter to @BeforeMethod #1738

Merged

4 tasks

cbeust closed this as completed in #1738 Apr 1, 2018

stachenov mentioned this issue Apr 1, 2018

Add onlyForGroups to @AfterMethod #1741

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Groups in @BeforeMethod and @AfterMethod don't work as expected #549

Groups in @BeforeMethod and @AfterMethod don't work as expected #549

PW999 commented Sep 19, 2014 •

edited by juherr

Loading

jeangb commented Dec 16, 2016

krmahadevan commented Dec 16, 2016

apreg commented Jan 19, 2017 •

edited by juherr

Loading

krmahadevan commented Feb 10, 2017

juherr commented Feb 10, 2017

cbeust commented Feb 10, 2017

juherr commented Feb 10, 2017 •

edited

Loading

cbeust commented Feb 10, 2017

juherr commented Feb 10, 2017 •

edited

Loading

madhukarashok commented Jun 13, 2017

krmahadevan commented Jun 14, 2017

madhukarashok commented Jun 15, 2017

krmahadevan commented Jun 16, 2017

madhukarashok commented Jun 19, 2017 •

edited by juherr

Loading

madhukarashok commented Jun 20, 2017 •

edited by juherr

Loading

tnn24 commented Sep 21, 2017

juherr commented Sep 21, 2017

Jaruso commented Feb 11, 2018

stachenov commented Mar 30, 2018

krmahadevan commented Mar 30, 2018

wenijinew commented Jul 22, 2019

stachenov commented Jul 22, 2019

juherr commented Aug 3, 2019

CoryKniefel commented Dec 13, 2019 •

edited

Loading

stachenov commented Dec 14, 2019

wenijinew commented Sep 9, 2020

stachenov commented Sep 9, 2020

wenijinew commented Sep 29, 2020

stachenov commented Sep 29, 2020

sandeshmms commented Feb 14, 2021 •

edited

Loading

Groups in @BeforeMethod and @AfterMethod don't work as expected #549

Groups in @BeforeMethod and @AfterMethod don't work as expected #549

Comments

PW999 commented Sep 19, 2014 • edited by juherr Loading

jeangb commented Dec 16, 2016

krmahadevan commented Dec 16, 2016

apreg commented Jan 19, 2017 • edited by juherr Loading

krmahadevan commented Feb 10, 2017

juherr commented Feb 10, 2017

cbeust commented Feb 10, 2017

juherr commented Feb 10, 2017 • edited Loading

cbeust commented Feb 10, 2017

juherr commented Feb 10, 2017 • edited Loading

madhukarashok commented Jun 13, 2017

krmahadevan commented Jun 14, 2017

madhukarashok commented Jun 15, 2017

krmahadevan commented Jun 16, 2017

madhukarashok commented Jun 19, 2017 • edited by juherr Loading

madhukarashok commented Jun 20, 2017 • edited by juherr Loading

tnn24 commented Sep 21, 2017

juherr commented Sep 21, 2017

Jaruso commented Feb 11, 2018

stachenov commented Mar 30, 2018

krmahadevan commented Mar 30, 2018

wenijinew commented Jul 22, 2019

stachenov commented Jul 22, 2019

juherr commented Aug 3, 2019

CoryKniefel commented Dec 13, 2019 • edited Loading

stachenov commented Dec 14, 2019

wenijinew commented Sep 9, 2020

stachenov commented Sep 9, 2020

wenijinew commented Sep 29, 2020

stachenov commented Sep 29, 2020

sandeshmms commented Feb 14, 2021 • edited Loading

PW999 commented Sep 19, 2014 •

edited by juherr

Loading

apreg commented Jan 19, 2017 •

edited by juherr

Loading

juherr commented Feb 10, 2017 •

edited

Loading

juherr commented Feb 10, 2017 •

edited

Loading

madhukarashok commented Jun 19, 2017 •

edited by juherr

Loading

madhukarashok commented Jun 20, 2017 •

edited by juherr

Loading

CoryKniefel commented Dec 13, 2019 •

edited

Loading

sandeshmms commented Feb 14, 2021 •

edited

Loading