TestProfile's getEnabledAlternatives CDI BEAN leaks to unrelated TestProfile's test

[yuan-kuan]

Describe the bug

Running a normal @QuarkusTest with TestProfile without any special getEnabledAlternatives() definition will mistakenly use another TestProfile's mock CDI BEAN declared in getEnabledAlternatives().

My Profiles:

public class MockProfile implements QuarkusTestProfile {
  public static class ATag implements QuarkusTestProfile {
    @Override
    public String getConfigProfile() {
      return "test-a-tag";
    }

    @Override
    public Set<String> tags() {
      return Set.of("a");
    }
    @Override
    public Set<Class<?>> getEnabledAlternatives() {
      return Collections.singleton(MockService.class);
    }
    
  }
  
  public static class BTag implements QuarkusTestProfile {
    @Override
    public String getConfigProfile() {
      return "test-b-tag";
    }

    @Override
    public Set<String> tags() {
      return Set.of("b");
    }
    @Override
    public Set<Class<?>> getEnabledAlternatives() {
      return Collections.emptySet();
    }
  }
}

TestProfile without mock service

@QuarkusTest
@TestProfile(MockProfile.BTag.class)
public class NormalTest {
    @Inject
    GreetingService greetingService;
    @Test
    public void testOk() {
      System.out.println("B tag test run");
      
      Assertions.assertEquals("special greeting", greetingService.getSpecialGreet());
    }
}

TestProfile with mock service

@QuarkusMainTest
@TestProfile(MockProfile.ATag.class)
public class BugTest {
    @Test
    @Launch({"person"})
    public void bugTest(LaunchResult result) {
        System.out.println("B tag test run"); 
        Assertions.assertTrue(result.getOutput().contains("Hello person, mocked greeting!"));
    }
}

Expected behavior

CDI BEAN used in tests under a different TestProfile should be isolated from each other.

Actual behavior

NormalTest failed because it uses the Mock Service defined in another TestProfile.

How to Reproduce?

1. clone my small reproducible project: https://github.com/yuan-kuan/quarkus-test-tags-bug/tree/main
2. quarkus dev
3. press o and press r to run the unit tests with output on.
4. Observe that okTest's failure

Output of uname -a or ver

Darwin 24.0.0 Darwin Kernel Version 24.0.0: Wed Aug 7 03:09:57 PDT 2024; root:xnu-11215.1.9~22/RELEASE_ARM64_T8112 arm64

Output of java -version

openjdk version "21.0.5" 2024-10-15 LTS

Quarkus version or git rev

quarkus 3.15.1

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.9.8 (36645f6c9b5079805ea5009217e36f2cffd34256)

Additional information

No response

[quarkus-bot]

/cc @Ladicek (arc), @manovotn (arc), @mkouba (arc)

[Ladicek]

Mixing @QuarkusTest and @QuarkusMainTest isn't documented in https://quarkus.io/guides/getting-started-testing#mixing-quarkustest-with-other-type-of-tests, but it feels like it isn't allowed either. If I move the @QuarkusTest execution into a separate execution of the Maven Surefire plugin (as documented), both tests pass.

Not sure who knows whether this is allowed. CC @geoand maybe?

[geoand]

Hm, My guess is that mixing those two works as long as you don't have profiles. I think this is fixable