Enhance MongoDb otel integration #40714

vkn · 2024-05-18T11:13:39Z

This PR continues #40191

Fix parent-child spans for reactive request
Add docs
Add more integration and unit tests

cc @brunobat

github-actions · 2024-05-18T11:40:33Z

🙈 The PR is closed and the preview is expired.

brunobat

A few things:

You need to add a module to quarkus-integration-tests-parent:

  <module>opentelemetry-mongodb-client-instrumentation</module>

Bellow <module>opentelemetry-jdbc-instrumentation</module>

These tests should only run when test container tests are active, to save build time. Tests need to be disabled by default:

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <skip>true</skip>
    </configuration>
</plugin>
<plugin>
    <artifactId>maven-failsafe-plugin</artifactId>
    <configuration>
        <skip>true</skip>
    </configuration>
</plugin>

And a new profile must be added. There is a task on the CI to run only these test-containers tests:

 <profiles>
    <!-- Note: the container is started via Dev Services -->
    <profile>
        <id>test-mongodb</id>
        <activation>
            <property>
                <name>test-containers</name>
            </property>
        </activation>
        <build>
            <plugins>
                <plugin>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <skip>false</skip>
                    </configuration>
                </plugin>
                <plugin>
                    <artifactId>maven-failsafe-plugin</artifactId>
                    <configuration>
                        <skip>false</skip>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

...ions/mongodb-client/deployment/src/test/resources/application-tracing-mongoclient.properties

...ngodb-client-instrumentation/src/test/java/io/quarkus/it/opentelemetry/BookResourceTest.java

...db-client/runtime/src/main/java/io/quarkus/mongodb/runtime/MongoReactiveContextProvider.java

brunobat · 2024-05-21T10:10:21Z

Can you also please add the it project opentelemetry-mongodb-client-instrumentation to this line:

quarkus/.github/native-tests.json

Line 120 in 8d8dc6e

    
           "test-modules": "picocli-native, gradle, micrometer-mp-metrics, micrometer-prometheus, logging-json, jaxp, jaxb, opentelemetry, opentelemetry-jdbc-instrumentation, opentelemetry-redis-instrumentation, web-dependency-locator",

Otherwise the tests will not run on native.

vkn · 2024-05-21T10:26:14Z

Can you also please add the it project opentelemetry-mongodb-client-instrumentation to this line:

done

brunobat · 2024-05-21T14:35:07Z

@vkn, I see these exceptions on the build, do you have an Idea of the cause?
https://github.com/quarkusio/quarkus/actions/runs/9173102213/job/25221800658?pr=40714#step:16:2124

[mongod output] 2024-05-21 11:42:16,111 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (vert.x-eventloop-thread-3) HTTP Request to /reactive-books/invalid failed, error id: 4980e026-f2c7-4790-b8eb-eab7111de4e7-1: com.mongodb.MongoQueryException: Command failed with error 2 (BadValue): 'unknown top level operator: $invalidop. If you have a field name that starts with a '$' symbol, consider using $getField or $setField.' on server localhost:27017. The full response is {"ok": 0.0, "errmsg": "unknown top level operator: $invalidop. If you have a field name that starts with a '$' symbol, consider using $getField or $setField.", "code": 2, "codeName": "BadValue"}

vkn · 2024-05-21T15:34:00Z

@vkn, I see these exceptions on the build, do you have an Idea of the cause? https://github.com/quarkusio/quarkus/actions/runs/9173102213/job/25221800658?pr=40714#step:16:2124

[mongod output] 2024-05-21 11:42:16,111 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (vert.x-eventloop-thread-3) HTTP Request to /reactive-books/invalid failed, error id: 4980e026-f2c7-4790-b8eb-eab7111de4e7-1: com.mongodb.MongoQueryException: Command failed with error 2 (BadValue): 'unknown top level operator: $invalidop. If you have a field name that starts with a '$' symbol, consider using $getField or $setField.' on server localhost:27017. The full response is {"ok": 0.0, "errmsg": "unknown top level operator: $invalidop. If you have a field name that starts with a '$' symbol, consider using $getField or $setField.", "code": 2, "codeName": "BadValue"}

Yes. This is the test for failed query case. So the exception log is expected

brunobat

Thanks for the effort @vkn!

gsmet

Nice addition. I added a few suggestions for the doc.

docs/src/main/asciidoc/mongodb.adoc

gsmet · 2024-05-28T09:54:33Z

@vkn nice work! I asked a few micro changes for the doc. Also, it would be awesome if you could squash the commits in one when done (we use merge commits and we can't squash with the GitHub UI when merging).
Let me know if you need help, I can help if needed.

Continue PR quarkusio#40191 - Add docs - Add tests - Fix parent-child spans for reactive request

vkn · 2024-05-28T10:47:06Z

@gsmet I've applied your suggestions and squashed the commits

quarkus-bot · 2024-05-28T11:07:48Z

Status for workflow `Quarkus Documentation CI`

This is the status report for running Quarkus Documentation CI on commit c9480af.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

⚠️ There are other workflow runs running, you probably need to wait for their status before merging.

quarkus-bot · 2024-05-28T13:29:56Z

Status for workflow `Quarkus CI`

This is the status report for running Quarkus CI on commit c9480af.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.

Flaky tests - Develocity

⚙️ JVM Tests - JDK 21

📦 extensions/smallrye-reactive-messaging-kafka/deployment

✖ io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase.sseStream - History

Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase Expecting size of: [] to be greater than or equal to 2 but was 0 within 10 seconds. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: 
Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase 
Expecting size of:
  []
to be greater than or equal to 2 but was 0 within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)

📦 integration-tests/reactive-messaging-kafka

✖ io.quarkus.it.kafka.KafkaConnectorTest.testFruits - History

Assertion condition defined as a Lambda expression in io.quarkus.it.kafka.KafkaConnectorTest expected: <6> but was: <5> within 10 seconds. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: Assertion condition defined as a Lambda expression in io.quarkus.it.kafka.KafkaConnectorTest expected: <6> but was: <5> within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1006)
	at org.awaitility.core.ConditionFactory.untilAsserted(ConditionFactory.java:790)
	at io.quarkus.it.kafka.KafkaConnectorTest.testFruits(KafkaConnectorTest.java:63)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)

vkn · 2024-06-06T12:17:36Z

Hi @gsmet, is there anything else needed from me before merging?

gsmet · 2024-06-06T13:18:16Z

Nope, just me getting to the second page of PRs :).

Thanks for your efforts!

quarkus-bot bot added area/documentation area/mongodb labels May 18, 2024

vkn mentioned this pull request May 18, 2024

[WIP] Mongodb otel refactor #40472

Closed

This comment has been minimized.

Sign in to view

brunobat added the area/tracing label May 20, 2024

brunobat requested changes May 20, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

quarkus-bot bot added the triage/flaky-test label May 20, 2024

quarkus-bot bot added the area/infra-automation anything related to CI, bots, etc. that are used to automated our infrastructure label May 21, 2024

This comment has been minimized.

Sign in to view

brunobat approved these changes May 21, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

gsmet reviewed May 28, 2024

View reviewed changes

docs/src/main/asciidoc/mongodb.adoc Outdated Show resolved Hide resolved

docs/src/main/asciidoc/mongodb.adoc Show resolved Hide resolved

Enhance mongodb otel integration

c9480af

Continue PR quarkusio#40191 - Add docs - Add tests - Fix parent-child spans for reactive request

vkn force-pushed the mongodb-otel-enhancements branch from 24ea28a to c9480af Compare May 28, 2024 10:42

gsmet approved these changes Jun 6, 2024

View reviewed changes

gsmet merged commit 8416855 into quarkusio:main Jun 6, 2024
54 checks passed

quarkus-bot bot added this to the 3.12 - main milestone Jun 6, 2024

gsmet added the release/noteworthy-feature label Jun 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhance MongoDb otel integration #40714

Enhance MongoDb otel integration #40714

vkn commented May 18, 2024

This comment has been minimized.

github-actions bot commented May 18, 2024 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

brunobat left a comment

This comment has been minimized.

This comment has been minimized.

brunobat commented May 21, 2024 •

edited

Loading

vkn commented May 21, 2024

This comment has been minimized.

brunobat commented May 21, 2024 •

edited

Loading

vkn commented May 21, 2024

brunobat left a comment

This comment has been minimized.

This comment has been minimized.

gsmet left a comment

gsmet commented May 28, 2024

vkn commented May 28, 2024

quarkus-bot bot commented May 28, 2024

quarkus-bot bot commented May 28, 2024 •

edited by github-actions bot

Loading

vkn commented Jun 6, 2024

gsmet commented Jun 6, 2024

Enhance MongoDb otel integration #40714

Enhance MongoDb otel integration #40714

Conversation

vkn commented May 18, 2024

This comment has been minimized.

github-actions bot commented May 18, 2024 • edited Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

brunobat left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

brunobat commented May 21, 2024 • edited Loading

vkn commented May 21, 2024

This comment has been minimized.

brunobat commented May 21, 2024 • edited Loading

vkn commented May 21, 2024

brunobat left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

gsmet left a comment

Choose a reason for hiding this comment

gsmet commented May 28, 2024

vkn commented May 28, 2024

quarkus-bot bot commented May 28, 2024

Status for workflow Quarkus Documentation CI

quarkus-bot bot commented May 28, 2024 • edited by github-actions bot Loading

Status for workflow Quarkus CI

Flaky tests - Develocity

⚙️ JVM Tests - JDK 21

📦 extensions/smallrye-reactive-messaging-kafka/deployment

📦 integration-tests/reactive-messaging-kafka

vkn commented Jun 6, 2024

gsmet commented Jun 6, 2024

github-actions bot commented May 18, 2024 •

edited

Loading

brunobat commented May 21, 2024 •

edited

Loading

brunobat commented May 21, 2024 •

edited

Loading

Status for workflow `Quarkus Documentation CI`

quarkus-bot bot commented May 28, 2024 •

edited by github-actions bot

Loading

Status for workflow `Quarkus CI`