Expose JMX metrics for TrinoFileSystemCache#12024
Merged
findepi merged 1 commit intotrinodb:masterfrom Jun 8, 2022
Merged
Conversation
Contributor
Author
|
@hashhar sorry to bug you but do you know who a good person to ping for a review of this might be? |
Member
|
@findinpath @homar can you PTAL (or find someone eligible). Piotr can do the final review before merge. |
homar
approved these changes
May 10, 2022
plugin/trino-hive/src/main/java/io/trino/plugin/hive/HiveModule.java
Outdated
Show resolved
Hide resolved
findepi
reviewed
May 11, 2022
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCacheStats.java
Outdated
Show resolved
Hide resolved
findinpath
reviewed
May 12, 2022
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCacheStats.java
Outdated
Show resolved
Hide resolved
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
Member
|
@phd3 @Felicity-3786 PTAL (re #12364) |
Contributor
|
Do we want to enable clearing the cache via JMX ? |
Member
i don't know the implications of doing that, so wouldn't approve such a change. |
Member
|
@posulliv there are some unapplied comments. Do you want to update the PR? |
posulliv
force-pushed
the
trino-fs-cache-jmx-metrics
branch
from
bbab13a to
ed69cd8
Compare
Contributor
Author
|
@findepi I have applied all feedback now. |
posulliv
force-pushed
the
trino-fs-cache-jmx-metrics
branch
from
ed69cd8 to
462a89d
Compare
findepi
reviewed
Jun 2, 2022
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
findepi
reviewed
Jun 2, 2022
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
findepi
reviewed
Jun 2, 2022
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
findepi
reviewed
Jun 2, 2022
plugin/trino-hive/src/test/java/io/trino/plugin/hive/fs/TestTrinoFileSystemCacheStats.java
Outdated
Show resolved
Hide resolved
posulliv
force-pushed
the
trino-fs-cache-jmx-metrics
branch
from
462a89d to
72bdb9a
Compare
Member
|
@posulliv please rebase, the CI didn't run |
findepi
reviewed
Jun 3, 2022
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
posulliv
force-pushed
the
trino-fs-cache-jmx-metrics
branch
from
72bdb9a to
809b073
Compare
findepi
reviewed
Jun 6, 2022
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
findepi
reviewed
Jun 6, 2022
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCache.java
Outdated
Show resolved
Hide resolved
findepi
reviewed
Jun 6, 2022
plugin/trino-hive/src/main/java/io/trino/plugin/hive/fs/TrinoFileSystemCacheStats.java
Outdated
Show resolved
Hide resolved
posulliv
force-pushed
the
trino-fs-cache-jmx-metrics
branch
from
809b073 to
aa0a593
Compare
findepi
force-pushed
the
trino-fs-cache-jmx-metrics
branch
from
aa0a593 to
a0092c1
Compare
findepi
added
the
no-release-notes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The Hive connector in Trino has it's own implementation of Hadoop's
FileSystemCache. The size of this cache defaults to 1000 but there is no way to see what the current size of the cache is.In deployments where HDFS impersonation is enabled with many distinct users connecting to Trino, this cache can get large.
This PR exposes some statistics on the cache via JMX to help Trino admins know if the cache is getting close to capacity.
New JMX statistics for
TrinoFileSystemCache.Update to the Hive connector.
This change exposes new JMX stats which will be available at
trino.plugin.hive.fs:name=<catalog_name>,type=trinofilesystemcache.Related issues, pull requests, and links
Documentation
(X ) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.
Release notes
( X) No release notes entries required.
( ) Release notes entries required with the following suggested text: