Skip to content

Add hdfs stats for local and remote rack bytes read #92

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
bbeaudreault merged 1 commit into from
Apr 19, 2024
Merged

Add hdfs stats for local and remote rack bytes read #92

bbeaudreault merged 1 commit into from
Apr 19, 2024

Conversation

@bbeaudreault
Copy link
Member

@bbeaudreault bbeaudreault commented Apr 18, 2024

I would like to investigate inter-AZ network usage, which is a meaningful cost for our hbase deployment. These two metrics should help. If they work out, I will upstream.

@bbeaudreault
Copy link
Member Author

bbeaudreault commented Apr 19, 2024

I've tested this out, and it works well:

"totalBytesRead": 13812001004,
      "localBytesRead": 1016148869,
      "localRackBytesRead": 5978978624,
      "remoteRackBytesRead": 10155185232,
      "shortCircuitBytesRead": 1016148869,

From an EC cluster

@bbeaudreault bbeaudreault merged commit 9a3d0b3 into hubspot-2.5 Apr 19, 2024
@bbeaudreault bbeaudreault added the upstream The changes in this PR should submitted upstream once ready label Apr 19, 2024
bbeaudreault added a commit that referenced this pull request May 14, 2024
@bbeaudreault
Add hdfs stats for local and remote rack bytes read (#92)
54650dd
ndimiduk pushed a commit that referenced this pull request Sep 23, 2024
@bbeaudreault @ndimiduk
Add hdfs stats for local and remote rack bytes read (#92)
30cadcb
ndimiduk pushed a commit that referenced this pull request Sep 23, 2024
@bbeaudreault @ndimiduk
Add hdfs stats for local and remote rack bytes read (#92)
de63252
ndimiduk pushed a commit that referenced this pull request Sep 23, 2024
@bbeaudreault @ndimiduk
[HubSpot] Add region server metrics for local and remote rack bytes r…
f329ea0 
…ead (#92)
ndimiduk pushed a commit that referenced this pull request Sep 23, 2024
@bbeaudreault @ndimiduk
[HubSpot] Add region server metrics for local and remote rack bytes r…
a7e0734 
…ead (#92)
ndimiduk pushed a commit that referenced this pull request Sep 25, 2024
@bbeaudreault @ndimiduk
[HubSpot] Add region server metrics for local and remote rack bytes r…
568829b 
…ead (#92)
ndimiduk pushed a commit that referenced this pull request Sep 27, 2024
@bbeaudreault @ndimiduk
[HubSpot] Add region server metrics for local and remote rack bytes r…
93572d0 
…ead (#92)
ndimiduk pushed a commit that referenced this pull request Oct 1, 2024
@bbeaudreault @ndimiduk
[HubSpot] Add region server metrics for local and remote rack bytes r…
730af92 
…ead (#92)
ndimiduk pushed a commit that referenced this pull request Oct 14, 2024
@bbeaudreault @ndimiduk
[HubSpot] Add region server metrics for local and remote rack bytes r…
c4adbb6 
…ead (#92)
charlesconnell pushed a commit that referenced this pull request Dec 20, 2024
@bbeaudreault @charlesconnell
Add hdfs stats for local and remote rack bytes read (#92)
cc49246
charlesconnell pushed a commit that referenced this pull request Dec 20, 2024
@bbeaudreault @charlesconnell
HubSpot Edit: Add hdfs stats for local and remote rack bytes read (#92)
526a86d
charlesconnell pushed a commit that referenced this pull request Dec 20, 2024
@bbeaudreault @charlesconnell
HubSpot Edit: Add hdfs stats for local and remote rack bytes read (#92)
caf2763
charlesconnell pushed a commit that referenced this pull request Dec 20, 2024
@bbeaudreault @charlesconnell
HubSpot Edit: Add hdfs stats for local and remote rack bytes read (#92)
a88f36b
charlesconnell pushed a commit that referenced this pull request Dec 21, 2024
@bbeaudreault @charlesconnell
HubSpot Edit: Add hdfs stats for local and remote rack bytes read (#92)
d2e2f39
ndimiduk pushed a commit that referenced this pull request Feb 19, 2025
@bbeaudreault @ndimiduk
HubSpot Edit: Add hdfs stats for local and remote rack bytes read (#92)
b5a90d7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

upstream The changes in this PR should submitted upstream once ready

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@bbeaudreault