Skip to content

Commit fe002fa

Browse files
zhumin8zhumin8
authored
feat: add client side logging with slf4j (#3403)
Changes to introduce client logging debug feature to Gax. Guide on usage will be added separately to README file. - Brings in `slf4j-api` as optional dependency to gax-java - `LoggingUtils`, `LogData` and `LoggerProvider` are public because access are needed from gax-grpc and gax-httpjson packages - `LoggingUtils` handles logic to enable/disable from env var, and contains shared utility methods for record data for logging and record logs (abstracting away any need for SLF4J classes). - `Slf4jUtils` any logic interacting with SLF4J classes. - `LoggerProvider` provides the SLF4J Logger. This is so that Logger interceptor classes do not declare Logger directly. - This feature is guarded by env var GOOGLE_SDK_JAVA_LOGGING, only turned on if true. - By default it is off, user app should act as before (usual tests behaves as usual) __Tests added:__ - Unit tests in Gax that need either GOOGLE_SDK_JAVA_LOGGING unset, or do not depend on env var - LoggingEnabledTest: test for logger correctly setups when GOOGLE_SDK_JAVA_LOGGING = true. This is added to existing `envVarTest` profile - Showcase tests: - ITLoggingDisabled: test no logging event should record when env var is turned off. - ITLogging: test logging event with slf4j2.x+logback - ITLogging1x: test logging event with slf4j1.x+logback These tests may not compile depending on the logging dependency brought in. Set up profiles to control test compile and run: - Slf4j2_logback, slf4j1_logback, disabledLogging: brings in logging dependencies and setup compile config exclusions - Do not include `it/logging` folder for compile by default, or `enable-golden-tests`. `native` is also excluded for now. __Notable changes since last reviewed:__ - Changes related to get slf4j-api deps as optional - Logics to switch logging behavior based on slf4j major version - Moved helper methods into Gax - Revamped test setup - use Protobuf utils to serialize message __Context:__ [go/java-client-logging-design](http://goto.google.com/java-client-logging-design) TODO: - add rules for renovate bot to bypass some versions for testing
1 parent ad26cf9 commit fe002fa

File tree

36 files changed

+3090
-9
lines changed

36 files changed

+3090
-9
lines changed

‎.github/workflows/ci.yaml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ jobs:
3131
env:
3232
GOOGLE_CLOUD_UNIVERSE_DOMAIN: random.com
3333
GOOGLE_CLOUD_ENABLE_DIRECT_PATH_XDS: true
34+
GOOGLE_SDK_JAVA_LOGGING: true
3435
- run: bazelisk version
3536
- name: Install Maven modules
3637
run: |
@@ -82,6 +83,7 @@ jobs:
8283
env:
8384
GOOGLE_CLOUD_UNIVERSE_DOMAIN: random.com
8485
GOOGLE_CLOUD_ENABLE_DIRECT_PATH_XDS: true
86+
GOOGLE_SDK_JAVA_LOGGING: true
8587
- run: bazelisk version
8688
- name: Install Maven modules
8789
run: |
@@ -133,6 +135,7 @@ jobs:
133135
env:
134136
GOOGLE_CLOUD_UNIVERSE_DOMAIN: random.com
135137
GOOGLE_CLOUD_ENABLE_DIRECT_PATH_XDS: true
138+
GOOGLE_SDK_JAVA_LOGGING: true
136139

137140
build-java8-gapic-generator-java:
138141
name: "build(8) for gapic-generator-java"
@@ -255,6 +258,34 @@ jobs:
255258
-P enable-integration-tests \
256259
--batch-mode \
257260
--no-transfer-progress
261+
# The `slf4j2_logback` profile brings logging dependency and compiles logging tests, require env var to be set
262+
- name: Showcase integration tests - Logging SLF4J 2.x
263+
working-directory: showcase
264+
run: |
265+
mvn clean verify -P '!showcase,enable-integration-tests,loggingTestBase,slf4j2_logback' \
266+
--batch-mode \
267+
--no-transfer-progress
268+
# Set the Env Var for this step only
269+
env:
270+
GOOGLE_SDK_JAVA_LOGGING: true
271+
# The `slf4j1_logback` profile brings logging dependency and compiles logging tests, require env var to be set
272+
- name: Showcase integration tests - Logging SLF4J 1.x
273+
working-directory: showcase
274+
run: |
275+
mvn clean verify -P '!showcase,enable-integration-tests,loggingTestBase,slf4j1_logback' \
276+
--batch-mode \
277+
--no-transfer-progress
278+
# Set the Env Var for this step only
279+
env:
280+
GOOGLE_SDK_JAVA_LOGGING: true
281+
# The `disabledLogging` profile tests logging disabled when logging dependency present,
282+
# do not set env var for this step
283+
- name: Showcase integration tests - Logging disabed
284+
working-directory: showcase
285+
run: |
286+
mvn clean verify -P '!showcase,enable-integration-tests,loggingTestBase,disabledLogging' \
287+
--batch-mode \
288+
--no-transfer-progress
258289
259290
showcase-clirr:
260291
if: ${{ github.base_ref != '' }} # Only execute on pull_request trigger event

‎gapic-generator-java-pom-parent/pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@
3838
<j2objc-annotations.version>3.0.0</j2objc-annotations.version>
3939
<threetenbp.version>1.7.0</threetenbp.version>
4040
<junit.version>5.11.4</junit.version>
41+
<slf4j.version>2.0.16</slf4j.version>
4142
</properties>
4243

4344
<developers>

‎gax-java/dependencies.properties‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,8 @@ maven.com_google_http_client_google_http_client_gson=com.google.http-client:goog
7676
maven.org_codehaus_mojo_animal_sniffer_annotations=org.codehaus.mojo:animal-sniffer-annotations:1.24
7777
maven.javax_annotation_javax_annotation_api=javax.annotation:javax.annotation-api:1.3.2
7878
maven.org_graalvm_sdk=org.graalvm.sdk:nativeimage:24.1.2
79+
maven.org_slf4j_slf4j_api=org.slf4j:slf4j-api:2.0.16
80+
maven.com_google_protobuf_protobuf_java_util=com.google.protobuf:protobuf-java-util:3.25.5
7981

8082
# Testing maven artifacts
8183
maven.junit_junit=junit:junit:4.13.2

‎gax-java/gax-grpc/pom.xml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,12 @@
9292
<artifactId>auto-value-annotations</artifactId>
9393
<scope>provided</scope>
9494
</dependency>
95+
<!-- Logging dependency -->
96+
<dependency>
97+
<groupId>org.slf4j</groupId>
98+
<artifactId>slf4j-api</artifactId>
99+
<optional>true</optional>
100+
</dependency>
95101

96102
<!-- test dependencies -->
97103
<dependency>

‎gax-java/gax-grpc/src/main/java/com/google/api/gax/grpc/GrpcLoggingInterceptor.java‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
/*
2+
* Copyright 2025 Google LLC
3+
*
4+
* Redistribution and use in source and binary forms, with or without
5+
* modification, are permitted provided that the following conditions are
6+
* met:
7+
*
8+
* * Redistributions of source code must retain the above copyright
9+
* notice, this list of conditions and the following disclaimer.
10+
* * Redistributions in binary form must reproduce the above
11+
* copyright notice, this list of conditions and the following disclaimer
12+
* in the documentation and/or other materials provided with the
13+
* distribution.
14+
* * Neither the name of Google LLC nor the names of its
15+
* contributors may be used to endorse or promote products derived from
16+
* this software without specific prior written permission.
17+
*
18+
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
19+
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
20+
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
21+
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
22+
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
23+
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
24+
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
25+
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
26+
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
27+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28+
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29+
*/
30+
31+
package com.google.api.gax.grpc;
32+
33+
import static com.google.api.gax.logging.LoggingUtils.executeWithTryCatch;
34+
import static com.google.api.gax.logging.LoggingUtils.logRequest;
35+
import static com.google.api.gax.logging.LoggingUtils.logResponse;
36+
import static com.google.api.gax.logging.LoggingUtils.recordResponseHeaders;
37+
import static com.google.api.gax.logging.LoggingUtils.recordResponsePayload;
38+
import static com.google.api.gax.logging.LoggingUtils.recordServiceRpcAndRequestHeaders;
39+
40+
import com.google.api.core.InternalApi;
41+
import com.google.api.gax.logging.LogData;
42+
import com.google.api.gax.logging.LoggerProvider;
43+
import io.grpc.CallOptions;
44+
import io.grpc.Channel;
45+
import io.grpc.ClientCall;
46+
import io.grpc.ClientInterceptor;
47+
import io.grpc.ForwardingClientCall;
48+
import io.grpc.ForwardingClientCallListener.SimpleForwardingClientCallListener;
49+
import io.grpc.Metadata;
50+
import io.grpc.MethodDescriptor;
51+
import io.grpc.Status;
52+
import java.util.HashMap;
53+
import java.util.Map;
54+
55+
@InternalApi
56+
public class GrpcLoggingInterceptor implements ClientInterceptor {
57+
58+
private static final LoggerProvider LOGGER_PROVIDER =
59+
LoggerProvider.forClazz(GrpcLoggingInterceptor.class);
60+
61+
ClientCall.Listener<?> currentListener; // expose for test setup
62+
63+
@Override
64+
public <ReqT, RespT> ClientCall<ReqT, RespT> interceptCall(
65+
MethodDescriptor<ReqT, RespT> method, CallOptions callOptions, Channel next) {
66+
67+
return new ForwardingClientCall.SimpleForwardingClientCall<ReqT, RespT>(
68+
next.newCall(method, callOptions)) {
69+
LogData.Builder logDataBuilder = LogData.builder();
70+
71+
@Override
72+
public void start(Listener<RespT> responseListener, Metadata headers) {
73+
recordServiceRpcAndRequestHeaders(
74+
method.getServiceName(),
75+
method.getFullMethodName(),
76+
null, // endpoint is for http request only
77+
metadataHeadersToMap(headers),
78+
logDataBuilder,
79+
LOGGER_PROVIDER);
80+
SimpleForwardingClientCallListener<RespT> responseLoggingListener =
81+
new SimpleForwardingClientCallListener<RespT>(responseListener) {
82+
@Override
83+
public void onHeaders(Metadata headers) {
84+
recordResponseHeaders(
85+
metadataHeadersToMap(headers), logDataBuilder, LOGGER_PROVIDER);
86+
super.onHeaders(headers);
87+
}
88+
89+
@Override
90+
public void onMessage(RespT message) {
91+
recordResponsePayload(message, logDataBuilder, LOGGER_PROVIDER);
92+
super.onMessage(message);
93+
}
94+
95+
@Override
96+
public void onClose(Status status, Metadata trailers) {
97+
logResponse(status.getCode().toString(), logDataBuilder, LOGGER_PROVIDER);
98+
super.onClose(status, trailers);
99+
}
100+
};
101+
currentListener = responseLoggingListener;
102+
super.start(responseLoggingListener, headers);
103+
}
104+
105+
@Override
106+
public void sendMessage(ReqT message) {
107+
logRequest(message, logDataBuilder, LOGGER_PROVIDER);
108+
super.sendMessage(message);
109+
}
110+
};
111+
}
112+
113+
// Helper methods for logging
114+
private static Map<String, String> metadataHeadersToMap(Metadata headers) {
115+
116+
Map<String, String> headersMap = new HashMap<>();
117+
executeWithTryCatch(
118+
() -> {
119+
for (String key : headers.keys()) {
120+
// grpc header values can be either ASCII strings or binary
121+
// https://grpc.io/docs/guides/metadata/#overview
122+
// this condition identified binary headers and skip for logging
123+
if (key.endsWith(Metadata.BINARY_HEADER_SUFFIX)) {
124+
continue;
125+
}
126+
Metadata.Key<String> metadataKey =
127+
Metadata.Key.of(key, Metadata.ASCII_STRING_MARSHALLER);
128+
String headerValue = headers.get(metadataKey);
129+
130+
headersMap.put(key, headerValue);
131+
}
132+
});
133+
134+
return headersMap;
135+
}
136+
}

‎gax-java/gax-grpc/src/main/java/com/google/api/gax/grpc/InstantiatingGrpcChannelProvider.java‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -681,6 +681,7 @@ private ManagedChannel createSingleChannel() throws IOException {
681681
builder =
682682
builder
683683
.intercept(new GrpcChannelUUIDInterceptor())
684+
.intercept(new GrpcLoggingInterceptor())
684685
.intercept(headerInterceptor)
685686
.intercept(metadataHandlerInterceptor)
686687
.userAgent(headerInterceptor.getUserAgentHeader())

‎gax-java/gax-grpc/src/test/java/com/google/api/gax/grpc/GrpcLoggingInterceptorTest.java‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
/*
2+
* Copyright 2025 Google LLC
3+
*
4+
* Redistribution and use in source and binary forms, with or without
5+
* modification, are permitted provided that the following conditions are
6+
* met:
7+
*
8+
* * Redistributions of source code must retain the above copyright
9+
* notice, this list of conditions and the following disclaimer.
10+
* * Redistributions in binary form must reproduce the above
11+
* copyright notice, this list of conditions and the following disclaimer
12+
* in the documentation and/or other materials provided with the
13+
* distribution.
14+
* * Neither the name of Google LLC nor the names of its
15+
* contributors may be used to endorse or promote products derived from
16+
* this software without specific prior written permission.
17+
*
18+
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
19+
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
20+
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
21+
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
22+
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
23+
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
24+
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
25+
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
26+
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
27+
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28+
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29+
*/
30+
31+
package com.google.api.gax.grpc;
32+
33+
import static org.mockito.ArgumentMatchers.any;
34+
import static org.mockito.Mockito.mock;
35+
import static org.mockito.Mockito.spy;
36+
import static org.mockito.Mockito.verify;
37+
import static org.mockito.Mockito.when;
38+
39+
import com.google.api.gax.grpc.testing.FakeMethodDescriptor;
40+
import io.grpc.CallOptions;
41+
import io.grpc.Channel;
42+
import io.grpc.ClientCall;
43+
import io.grpc.ClientInterceptors;
44+
import io.grpc.Metadata;
45+
import io.grpc.MethodDescriptor;
46+
import io.grpc.Status;
47+
import org.junit.jupiter.api.Test;
48+
import org.junit.jupiter.api.extension.ExtendWith;
49+
import org.mockito.Mock;
50+
import org.mockito.Mockito;
51+
import org.mockito.junit.jupiter.MockitoExtension;
52+
53+
@ExtendWith(MockitoExtension.class)
54+
class GrpcLoggingInterceptorTest {
55+
@Mock private Channel channel;
56+
57+
@Mock private ClientCall<String, Integer> call;
58+
59+
private static final MethodDescriptor<String, Integer> method = FakeMethodDescriptor.create();
60+
61+
@Test
62+
void testInterceptor_basic() {
63+
when(channel.newCall(Mockito.<MethodDescriptor<String, Integer>>any(), any(CallOptions.class)))
64+
.thenReturn(call);
65+
GrpcLoggingInterceptor interceptor = new GrpcLoggingInterceptor();
66+
Channel intercepted = ClientInterceptors.intercept(channel, interceptor);
67+
@SuppressWarnings("unchecked")
68+
ClientCall.Listener<Integer> listener = mock(ClientCall.Listener.class);
69+
ClientCall<String, Integer> interceptedCall = intercepted.newCall(method, CallOptions.DEFAULT);
70+
// Simulate starting the call
71+
interceptedCall.start(listener, new Metadata());
72+
// Verify that the underlying call's start() method is invoked
73+
verify(call).start(any(ClientCall.Listener.class), any(Metadata.class));
74+
75+
// Simulate sending a message
76+
String requestMessage = "test request";
77+
interceptedCall.sendMessage(requestMessage);
78+
// Verify that the underlying call's sendMessage() method is invoked
79+
verify(call).sendMessage(requestMessage);
80+
}
81+
82+
@Test
83+
void testInterceptor_responseListener() {
84+
when(channel.newCall(Mockito.<MethodDescriptor<String, Integer>>any(), any(CallOptions.class)))
85+
.thenReturn(call);
86+
GrpcLoggingInterceptor interceptor = spy(new GrpcLoggingInterceptor());
87+
Channel intercepted = ClientInterceptors.intercept(channel, interceptor);
88+
@SuppressWarnings("unchecked")
89+
ClientCall.Listener<Integer> listener = mock(ClientCall.Listener.class);
90+
ClientCall<String, Integer> interceptedCall = intercepted.newCall(method, CallOptions.DEFAULT);
91+
interceptedCall.start(listener, new Metadata());
92+
93+
// Simulate respond interceptor calls
94+
Metadata responseHeaders = new Metadata();
95+
responseHeaders.put(
96+
Metadata.Key.of("test-header", Metadata.ASCII_STRING_MARSHALLER), "header-value");
97+
interceptor.currentListener.onHeaders(responseHeaders);
98+
99+
interceptor.currentListener.onMessage(null);
100+
101+
Status status = Status.OK;
102+
interceptor.currentListener.onClose(status, new Metadata());
103+
}
104+
}

‎gax-java/gax-httpjson/pom.xml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,12 @@
7878
<artifactId>auto-value-annotations</artifactId>
7979
<scope>provided</scope>
8080
</dependency>
81+
<!-- Logging dependency -->
82+
<dependency>
83+
<groupId>org.slf4j</groupId>
84+
<artifactId>slf4j-api</artifactId>
85+
<optional>true</optional>
86+
</dependency>
8187

8288
<!-- test dependencies -->
8389
<dependency>

0 commit comments

Comments
 (0)