Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance MongoDb otel integration #40714

Merged
merged 1 commit into from
Jun 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/native-tests.json
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@
{
"category": "Misc4",
"timeout": 130,
"test-modules": "picocli-native, gradle, micrometer-mp-metrics, micrometer-prometheus, logging-json, jaxp, jaxb, opentelemetry, opentelemetry-jdbc-instrumentation, opentelemetry-redis-instrumentation, web-dependency-locator",
"test-modules": "picocli-native, gradle, micrometer-mp-metrics, micrometer-prometheus, logging-json, jaxp, jaxb, opentelemetry, opentelemetry-jdbc-instrumentation, opentelemetry-mongodb-client-instrumentation, opentelemetry-redis-instrumentation, web-dependency-locator",
"os-name": "ubuntu-latest"
},
{
Expand Down
12 changes: 12 additions & 0 deletions docs/src/main/asciidoc/mongodb.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -613,9 +613,21 @@
If you are using the `quarkus-micrometer` or `quarkus-smallrye-metrics` extension, `quarkus-mongodb-client` can provide metrics about the connection pools.
This behavior must first be enabled by setting the `quarkus.mongodb.metrics.enabled` property to `true` in your `application.properties`.

So when you access the `/q/metrics` endpoint of your application you will have information about the connection pool status.

Check warning on line 616 in docs/src/main/asciidoc/mongodb.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.TermsSuggestions] Depending on the context, consider using 'by using' or 'that uses' rather than 'using'. Raw Output: {"message": "[Quarkus.TermsSuggestions] Depending on the context, consider using 'by using' or 'that uses' rather than 'using'.", "location": {"path": "docs/src/main/asciidoc/mongodb.adoc", "range": {"start": {"line": 616, "column": 114}}}, "severity": "INFO"}
When using xref:smallrye-metrics.adoc[SmallRye Metrics], connection pool metrics will be available under the `vendor` scope.


== Tracing

To use tracing with MongoDB, you need to add the xref:opentelemetry.adoc[`quarkus-opentelemetry`] extension to your project.

Check warning on line 622 in docs/src/main/asciidoc/mongodb.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.Fluff] Depending on the context, consider using 'Rewrite the sentence, or use 'must', instead of' rather than 'need to'. Raw Output: {"message": "[Quarkus.Fluff] Depending on the context, consider using 'Rewrite the sentence, or use 'must', instead of' rather than 'need to'.", "location": {"path": "docs/src/main/asciidoc/mongodb.adoc", "range": {"start": {"line": 622, "column": 34}}}, "severity": "INFO"}

Even with all the tracing infrastructure in place the mongodb tracing is not enabled by default, and you need to enable it by setting this property:

Check warning on line 624 in docs/src/main/asciidoc/mongodb.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.Fluff] Depending on the context, consider using 'Rewrite the sentence, or use 'must', instead of' rather than 'need to'. Raw Output: {"message": "[Quarkus.Fluff] Depending on the context, consider using 'Rewrite the sentence, or use 'must', instead of' rather than 'need to'.", "location": {"path": "docs/src/main/asciidoc/mongodb.adoc", "range": {"start": {"line": 624, "column": 106}}}, "severity": "INFO"}
vkn marked this conversation as resolved.
Show resolved Hide resolved
[source, properties]
----
# enable tracing
quarkus.mongodb.tracing.enabled=true
----

== Testing helpers

xref:#dev-services[Dev Services for MongoDB] is your best option to start a MongoDB database for your unit tests.
Expand Down
1 change: 1 addition & 0 deletions docs/src/main/asciidoc/opentelemetry.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -684,6 +684,7 @@ Additional exporters will be available in the Quarkiverse https://docs.quarkiver
* https://quarkus.io/guides/resteasy-client[`quarkus-resteasy-client`]
* https://quarkus.io/guides/scheduler[`quarkus-scheduler`]
* https://quarkus.io/guides/smallrye-graphql[`quarkus-smallrye-graphql`]
* https://quarkus.io/extensions/io.quarkus/quarkus-mongodb-client[`quarkus-mongodb-client`]
* https://quarkus.io/extensions/io.quarkus/quarkus-messaging[`quarkus-messaging`]
** AMQP 1.0
** RabbitMQ
Expand Down
5 changes: 5 additions & 0 deletions extensions/mongodb-client/deployment/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,11 @@
<groupId>io.quarkus</groupId>
<artifactId>quarkus-mongodb-client</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-opentelemetry-deployment</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>mongodb</artifactId>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
package io.quarkus.mongodb.deployment;

import java.util.List;

import com.mongodb.reactivestreams.client.ReactiveContextProvider;

import io.quarkus.builder.item.SimpleBuildItem;

/**
* Register additional {@link ReactiveContextProvider}s for the MongoDB clients.
*/
public final class ContextProviderBuildItem extends SimpleBuildItem {

private final List<String> classNames;

public ContextProviderBuildItem(List<String> classNames) {
this.classNames = classNames == null ? List.of() : classNames;
}

public List<String> getContextProviderClassNames() {
return classNames;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import com.mongodb.client.model.changestream.UpdateDescription;
import com.mongodb.event.CommandListener;
import com.mongodb.event.ConnectionPoolListener;
import com.mongodb.reactivestreams.client.ReactiveContextProvider;
import com.mongodb.spi.dns.DnsClientProvider;

import io.quarkus.arc.deployment.AdditionalBeanBuildItem;
Expand Down Expand Up @@ -69,6 +70,7 @@
import io.quarkus.mongodb.runtime.MongoClientRecorder;
import io.quarkus.mongodb.runtime.MongoClientSupport;
import io.quarkus.mongodb.runtime.MongoClients;
import io.quarkus.mongodb.runtime.MongoReactiveContextProvider;
import io.quarkus.mongodb.runtime.MongoServiceBindingConverter;
import io.quarkus.mongodb.runtime.MongodbConfig;
import io.quarkus.mongodb.runtime.dns.MongoDnsClient;
Expand Down Expand Up @@ -113,9 +115,11 @@ AdditionalIndexedClassesBuildItem includeDnsTypesToIndex() {
}

@BuildStep
AdditionalIndexedClassesBuildItem includeDnsTypesToIndex(MongoClientBuildTimeConfig buildTimeConfig) {
AdditionalIndexedClassesBuildItem includeMongoCommandListener(MongoClientBuildTimeConfig buildTimeConfig) {
if (buildTimeConfig.tracingEnabled) {
return new AdditionalIndexedClassesBuildItem(MongoTracingCommandListener.class.getName());
return new AdditionalIndexedClassesBuildItem(
MongoTracingCommandListener.class.getName(),
MongoReactiveContextProvider.class.getName());
}
return new AdditionalIndexedClassesBuildItem();
}
Expand Down Expand Up @@ -161,15 +165,27 @@ CommandListenerBuildItem collectCommandListeners(CombinedIndexBuildItem indexBui
return new CommandListenerBuildItem(names);
}

@BuildStep
ContextProviderBuildItem collectContextProviders(CombinedIndexBuildItem indexBuildItem) {
Collection<ClassInfo> contextProviders = indexBuildItem.getIndex()
.getAllKnownImplementors(DotName.createSimple(ReactiveContextProvider.class.getName()));
List<String> names = contextProviders.stream()
.map(ci -> ci.name().toString())
.collect(Collectors.toList());
return new ContextProviderBuildItem(names);
}

@BuildStep
List<ReflectiveClassBuildItem> addExtensionPointsToNative(CodecProviderBuildItem codecProviders,
PropertyCodecProviderBuildItem propertyCodecProviders, BsonDiscriminatorBuildItem bsonDiscriminators,
CommandListenerBuildItem commandListeners) {
CommandListenerBuildItem commandListeners,
ContextProviderBuildItem contextProviders) {
List<String> reflectiveClassNames = new ArrayList<>();
reflectiveClassNames.addAll(codecProviders.getCodecProviderClassNames());
reflectiveClassNames.addAll(propertyCodecProviders.getPropertyCodecProviderClassNames());
reflectiveClassNames.addAll(bsonDiscriminators.getBsonDiscriminatorClassNames());
reflectiveClassNames.addAll(commandListeners.getCommandListenerClassNames());
reflectiveClassNames.addAll(contextProviders.getContextProviderClassNames());

List<ReflectiveClassBuildItem> reflectiveClass = reflectiveClassNames.stream()
.map(s -> ReflectiveClassBuildItem.builder(s).methods().build())
Expand Down Expand Up @@ -256,6 +272,7 @@ void build(
PropertyCodecProviderBuildItem propertyCodecProvider,
BsonDiscriminatorBuildItem bsonDiscriminator,
CommandListenerBuildItem commandListener,
ContextProviderBuildItem contextProvider,
List<MongoConnectionPoolListenerBuildItem> connectionPoolListenerProvider,
BuildProducer<AdditionalBeanBuildItem> additionalBeanBuildItemProducer,
BuildProducer<SyntheticBeanBuildItem> syntheticBeanBuildItemBuildProducer) {
Expand All @@ -277,6 +294,9 @@ void build(
for (String name : commandListener.getCommandListenerClassNames()) {
additionalBeansBuilder.addBeanClass(name);
}
for (String name : contextProvider.getContextProviderClassNames()) {
additionalBeansBuilder.addBeanClass(name);
}
additionalBeanBuildItemProducer.produce(additionalBeansBuilder.build());

// create MongoClientSupport as a synthetic bean as it's used in AbstractMongoClientProducer
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package io.quarkus.mongodb;

import java.util.ArrayList;
import java.util.List;

import org.reactivestreams.Subscriber;

import com.mongodb.RequestContext;
import com.mongodb.reactivestreams.client.ReactiveContextProvider;

import io.quarkus.mongodb.runtime.MongoRequestContext;

public class MockReactiveContextProvider implements ReactiveContextProvider {

public static final List<String> EVENTS = new ArrayList<>();

@Override
public RequestContext getContext(Subscriber<?> subscriber) {
EVENTS.add(MongoRequestContext.class.getName());
return new MongoRequestContext(null);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
package io.quarkus.mongodb;

import static org.assertj.core.api.Assertions.assertThat;
import static org.awaitility.Awaitility.await;

import java.time.Duration;

import jakarta.inject.Inject;

import org.jboss.shrinkwrap.api.ShrinkWrap;
import org.jboss.shrinkwrap.api.spec.JavaArchive;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import io.quarkus.mongodb.reactive.ReactiveMongoClient;
import io.quarkus.test.QuarkusUnitTest;

public class MongoTracingEnabled extends MongoTestBase {

@Inject
ReactiveMongoClient reactiveClient;

@RegisterExtension
static final QuarkusUnitTest config = new QuarkusUnitTest().setArchiveProducer(
() -> ShrinkWrap.create(JavaArchive.class)
.addClasses(MongoTestBase.class, MockReactiveContextProvider.class, MockCommandListener.class))
.withConfigurationResource("application-tracing-mongoclient.properties");

@AfterEach
void cleanup() {
if (reactiveClient != null) {
reactiveClient.close();
}
}

@Test
void invokeReactiveContextProvider() {
String dbNames = reactiveClient.listDatabaseNames().toUni().await().atMost(Duration.ofSeconds(30L));
assertThat(dbNames).as("expect db names available").isNotBlank();
await().atMost(Duration.ofSeconds(30L)).untilAsserted(
() -> assertThat(MockReactiveContextProvider.EVENTS)
.as("reactive context provider must be called")
.isNotEmpty());
assertThat(MockCommandListener.EVENTS).isNotEmpty();

}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
package io.quarkus.mongodb;

import static org.assertj.core.api.Assertions.assertThat;

import jakarta.inject.Inject;

import org.jboss.shrinkwrap.api.ShrinkWrap;
import org.jboss.shrinkwrap.api.spec.JavaArchive;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import com.mongodb.client.MongoClient;

import io.quarkus.test.QuarkusUnitTest;

public class MongoTracingNotEnabledTest extends MongoTestBase {

@Inject
MongoClient client;

@RegisterExtension
static final QuarkusUnitTest config = new QuarkusUnitTest()
.setArchiveProducer(
() -> ShrinkWrap.create(JavaArchive.class).addClasses(MongoTestBase.class,
MockReactiveContextProvider.class))
.withConfigurationResource("default-mongoclient.properties");

@AfterEach
void cleanup() {
if (client != null) {
client.close();
}
}

@Test
void contextProviderMustNotBeCalledIfNoOpenTelemetryIsAvailable() {
assertThat(client.listDatabaseNames().first()).isNotEmpty();
assertThat(MockReactiveContextProvider.EVENTS).isEmpty();
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
package io.quarkus.mongodb.deployment;

import static org.assertj.core.api.Assertions.assertThat;

import java.util.List;

import org.junit.jupiter.api.Test;

class ContextProviderBuildItemTest {

@Test
void getContextProviderClassNames() {
ContextProviderBuildItem item = new ContextProviderBuildItem(List.of("foo.bar"));
assertThat(item.getContextProviderClassNames())
.hasSize(1)
.first()
.isEqualTo("foo.bar");
}

@Test
void emptyOrNull() {
ContextProviderBuildItem withNull = new ContextProviderBuildItem(null);
assertThat(withNull.getContextProviderClassNames()).isEmpty();

ContextProviderBuildItem empty = new ContextProviderBuildItem(List.of());
assertThat(empty.getContextProviderClassNames()).isEmpty();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
quarkus.mongodb.connection-string=mongodb://127.0.0.1:27018
quarkus.mongodb.tracing.enabled=true

5 changes: 5 additions & 0 deletions extensions/mongodb-client/runtime/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,11 @@
<artifactId>quarkus-kubernetes-service-binding</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-opentelemetry</artifactId>
<optional>true</optional>
</dependency>

<!-- Add the health extension as optional as we will produce the health check only if it's included -->
<dependency>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@
import com.mongodb.connection.SslSettings;
import com.mongodb.event.CommandListener;
import com.mongodb.event.ConnectionPoolListener;
import com.mongodb.reactivestreams.client.ReactiveContextProvider;

import io.quarkus.arc.Arc;
import io.quarkus.arc.InstanceHandle;
Expand Down Expand Up @@ -86,18 +87,21 @@ public class MongoClients {

private final Map<String, MongoClient> mongoclients = new HashMap<>();
private final Map<String, ReactiveMongoClient> reactiveMongoClients = new HashMap<>();
private final Instance<ReactiveContextProvider> reactiveContextProviders;
private final Instance<MongoClientCustomizer> customizers;

public MongoClients(MongodbConfig mongodbConfig, MongoClientSupport mongoClientSupport,
Instance<CodecProvider> codecProviders,
Instance<PropertyCodecProvider> propertyCodecProviders,
Instance<CommandListener> commandListeners,
Instance<ReactiveContextProvider> reactiveContextProviders,
@Any Instance<MongoClientCustomizer> customizers) {
this.mongodbConfig = mongodbConfig;
this.mongoClientSupport = mongoClientSupport;
this.codecProviders = codecProviders;
this.propertyCodecProviders = propertyCodecProviders;
this.commandListeners = commandListeners;
this.reactiveContextProviders = reactiveContextProviders;
this.customizers = customizers;

try {
Expand All @@ -121,15 +125,17 @@ public MongoClients(MongodbConfig mongodbConfig, MongoClientSupport mongoClientS
}

public MongoClient createMongoClient(String clientName) throws MongoException {
MongoClientSettings mongoConfiguration = createMongoConfiguration(clientName, getMatchingMongoClientConfig(clientName));
MongoClientSettings mongoConfiguration = createMongoConfiguration(clientName, getMatchingMongoClientConfig(clientName),
false);
MongoClient client = com.mongodb.client.MongoClients.create(mongoConfiguration);
mongoclients.put(clientName, client);
return client;
}

public ReactiveMongoClient createReactiveMongoClient(String clientName)
throws MongoException {
MongoClientSettings mongoConfiguration = createMongoConfiguration(clientName, getMatchingMongoClientConfig(clientName));
MongoClientSettings mongoConfiguration = createMongoConfiguration(clientName, getMatchingMongoClientConfig(clientName),
true);
com.mongodb.reactivestreams.client.MongoClient client = com.mongodb.reactivestreams.client.MongoClients
.create(mongoConfiguration);
ReactiveMongoClientImpl reactive = new ReactiveMongoClientImpl(client);
Expand Down Expand Up @@ -254,14 +260,18 @@ public void apply(ServerSettings.Builder builder) {
}
}

private MongoClientSettings createMongoConfiguration(String name, MongoClientConfig config) {
private MongoClientSettings createMongoConfiguration(String name, MongoClientConfig config, boolean isReactive) {
if (config == null) {
throw new RuntimeException("mongo config is missing for creating mongo client.");
}
CodecRegistry defaultCodecRegistry = MongoClientSettings.getDefaultCodecRegistry();

MongoClientSettings.Builder settings = MongoClientSettings.builder();

if (isReactive) {
reactiveContextProviders.stream().findAny().ifPresent(settings::contextProvider);
}

ConnectionString connectionString;
Optional<String> maybeConnectionString = config.connectionString;
if (maybeConnectionString.isPresent()) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package io.quarkus.mongodb.runtime;

import org.reactivestreams.Subscriber;

import com.mongodb.RequestContext;
import com.mongodb.reactivestreams.client.ReactiveContextProvider;

import io.opentelemetry.context.Context;

public class MongoReactiveContextProvider implements ReactiveContextProvider {

@Override
public RequestContext getContext(Subscriber<?> subscriber) {
return new MongoRequestContext(Context.current());
brunobat marked this conversation as resolved.
Show resolved Hide resolved
}

}
Loading
Loading