Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Zeta] Support slf4j mdc tracing job id output into logs #7491

Merged
merged 1 commit into from
Sep 2, 2024

Conversation

hailin0
Copy link
Member

@hailin0 hailin0 commented Aug 24, 2024

Purpose of this pull request

[Feature][Zeta] Support slf4j mdc tracing job id output into logs

reference
https://www.slf4j.org/manual.html#mdc
https://logging.apache.org/log4j/2.x/manual/thread-context.html#configuration
https://logging.apache.org/log4j/2.x/manual/layouts.html#PatternLayout

subtask of #2725

Does this PR introduce any user-facing change?

No

How was this patch tested?

Added

Check list

@github-actions github-actions bot added core SeaTunnel core module Zeta e2e api labels Aug 24, 2024
@hailin0
Copy link
Member Author

hailin0 commented Aug 24, 2024

img_v3_02e2_5a13324d-dd6b-4d3c-94d5-d1f0fcb443fg
img_v3_02e2_e03e4732-5174-42ac-a91f-72cab838f89g
img_v3_02e2_a24373b2-777f-433d-99b7-130eb11c2f0g
img_v3_02e2_46670f2e-388e-40ec-9217-aa26034df05g

@hailin0 hailin0 force-pushed the dev-mdc-zeta-logs branch 2 times, most recently from b2e749e to b8fcb17 Compare August 28, 2024 06:46
@github-actions github-actions bot added the CI&CD label Aug 28, 2024
@hailin0 hailin0 force-pushed the dev-mdc-zeta-logs branch 2 times, most recently from c9c175d to 9c546ec Compare August 30, 2024 15:12
@github-actions github-actions bot removed the CI&CD label Aug 30, 2024
@hailin0 hailin0 force-pushed the dev-mdc-zeta-logs branch 3 times, most recently from 0c06e63 to a2860eb Compare August 30, 2024 16:20
@github-actions github-actions bot added the CI&CD label Aug 30, 2024
@github-actions github-actions bot removed the CI&CD label Aug 30, 2024
@hailin0 hailin0 force-pushed the dev-mdc-zeta-logs branch 3 times, most recently from 5c93167 to ed5e863 Compare August 30, 2024 17:33
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

img_v3_02e9_d9119e5c-9e1f-4b6d-9f87-b9d2aa80403g

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

img_v3_02e9_7f329aa6-f531-444e-80dd-63397ffa35ag

@hailin0 hailin0 marked this pull request as ready for review September 1, 2024 14:59
@@ -83,6 +83,7 @@ done

# Log4j2 Config
JAVA_OPTS="${JAVA_OPTS} -Dlog4j2.contextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector"
JAVA_OPTS="${JAVA_OPTS} -Dlog4j2.isThreadContextMapInheritable=true"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@Carl-Zhou-CN Carl-Zhou-CN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@Carl-Zhou-CN Carl-Zhou-CN merged commit e224bda into apache:dev Sep 2, 2024
7 checks passed
@arshadmohammad
Copy link
Collaborator

arshadmohammad commented Sep 2, 2024

Good feature, Thanks @hailin0 for working on this.
related to this feature, is there anyway to get all the logs corresponding to job id, using API or command ?

@hailin0
Copy link
Member Author

hailin0 commented Sep 2, 2024

Good feature, Thanks @hailin0 for working on this. related to this feature, is there anyway to get all the logs corresponding to job id, using API or command ?

@arshadmohammad
The logs of distributed programs(job) are scattered on different machines. What are your thoughts?

@arshadmohammad
Copy link
Collaborator

arshadmohammad commented Sep 2, 2024

I was looking for functionality similar to mapreduce and yarn, all mapreduce jobs have separate logs.

@hailin0
Copy link
Member Author

hailin0 commented Sep 2, 2024

I will do some verification later

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants