Skip to content

HDDS-12288. Improve bootstrap logging to indicate progress #7861

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sadanand48 merged 10 commits into from
Feb 24, 2025
Merged

HDDS-12288. Improve bootstrap logging to indicate progress #7861

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
bfef929
HDDS-12288. Improved the bootsrap logging to indcate the progress
Feb 12, 2025
4a810a5
HDDS-12288. Improved the bootsrap logging to indcate the progress
Feb 12, 2025
1dd90ac
HDDS-12288. Improved the bootsrap logging to indcate the progress
Feb 12, 2025
cf05f89
HDDS-12288. Improved the bootsrap logging to indcate the progress
Feb 12, 2025
9aa8c33
HDDS-12288. Improved the bootsrap logging to indcate the progress
Feb 13, 2025
2d8fe42
HDDS-12288. Improved the bootsrap logging to indcate the progress
Feb 13, 2025
d3576d4
HDDS-12288. Improved the bootsrap logging to indcate the progress
Feb 13, 2025
dd03f4f
HDDS-12288. Improve bootstrap logging to indicate progress
Gargi-jais11 Feb 13, 2025
e5f7a59
HDDS-12288. Improve bootstrap logging to indicate progress
Gargi-jais11 Feb 13, 2025
3cb99a2
HDDS-12288. Improve bootstrap logging to indicate progress
Gargi-jais11 Feb 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 32 additions & 3 deletions ...ager/src/main/java/org/apache/hadoop/ozone/om/ratis_snapshot/OmRatisSnapshotProvider.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.FileOutputStream;
import java.net.HttpURLConnection;
Comment on lines 22 to 26
Copy link
Contributor

@adoroszlai adoroszlai Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't merge with outdated CI results. Last run, although triggered on Feb 21, was against code from Feb 13, because that's when the last commit was pushed to the PR:

  HEAD is now at 7062329 Merge 3cb99a2b4539b3f57a1da40118f8673aed7552f3 into 28b9a03204e2238ca9b01f403811f433685a4440

https://github.com/apache/ozone/actions/runs/13305097960/job/37592785787#step:2:90

commit 28b9a03204e2238ca9b01f403811f433685a4440
Author:     Takanobu Asanuma <[email protected]>
AuthorDate: Thu Feb 13 16:28:01 2025 +0900
Commit:     GitHub <[email protected]>
CommitDate: Thu Feb 13 08:28:01 2025 +0100

    HDDS-12312. NodeManager log aggregation to Ozone FileSystem fails. (#7856)

Copy link
Contributor

@sadanand48 sadanand48 Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure , will keep this in mind.
After how many days/commits b/w PR branch & master should one rebase and re-run? Ideally it should be after every new commit in master as any new commit can cause the CI to fail but do we actually follow this i.e re-running CI even if no conflicts reported in the PR

Copy link
Contributor

@adoroszlai adoroszlai Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it depends on the kind of changes done on master in the meantime. With new checkstyle rules it can get broken in one commit. Sorry I didn't notice that this PR would fail despite having no conflicts.

If in doubt, check out the PR locally, merge master into it, then run some quick sanity checks:

gh pr checkout 7861
git merge --no-commit origin/master
mvn -DskipTests -DskipRecon -DskipShade clean verify
./hadoop-ozone/dev-support/checks/checkstyle.sh
./hadoop-ozone/dev-support/checks/pmd.sh

import java.net.URL;
import java.util.List;
Expand Down Expand Up @@ -143,8 +144,8 @@ public void downloadSnapshot(String leaderNodeID, File targetFile)
OMNodeDetails leader = peerNodesMap.get(leaderNodeID);
URL omCheckpointUrl = leader.getOMDBCheckpointEndpointUrl(
httpPolicy.isHttpEnabled(), true);
LOG.info("Downloading latest checkpoint from Leader OM {}. Checkpoint " +
"URL: {}", leaderNodeID, omCheckpointUrl);
LOG.info("Downloading latest checkpoint from Leader OM {}. Checkpoint: {} URL: {}",
leaderNodeID, targetFile.getName(), omCheckpointUrl);
SecurityUtil.doAsCurrentUser(() -> {
HttpURLConnection connection = (HttpURLConnection)
connectionFactory.openConnection(omCheckpointUrl, spnegoEnabled);
Expand All @@ -166,7 +167,7 @@ public void downloadSnapshot(String leaderNodeID, File targetFile)
}

try (InputStream inputStream = connection.getInputStream()) {
FileUtils.copyInputStreamToFile(inputStream, targetFile);
downloadFileWithProgress(inputStream, targetFile);
} catch (IOException ex) {
boolean deleted = FileUtils.deleteQuietly(targetFile);
if (!deleted) {
Expand All @@ -181,6 +182,34 @@ public void downloadSnapshot(String leaderNodeID, File targetFile)
});
}

/**
* Writes data from the given InputStream to the target file while logging download progress every 30 seconds.
*/
public static void downloadFileWithProgress(InputStream inputStream, File targetFile)
throws IOException {
try (FileOutputStream outputStream = new FileOutputStream(targetFile)) {
byte[] buffer = new byte[8 * 1024];
long totalBytesRead = 0;
int bytesRead;
long lastLoggedTime = System.currentTimeMillis();

while ((bytesRead = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, bytesRead);
totalBytesRead += bytesRead;

// Log progress every 30 seconds
if (System.currentTimeMillis() - lastLoggedTime >= 30000) {
LOG.info("Downloading '{}': {} KB downloaded so far...",
targetFile.getName(), totalBytesRead / (1024));
lastLoggedTime = System.currentTimeMillis();
}
}

LOG.info("Download completed for '{}'. Total size: {} KB",
targetFile.getName(), totalBytesRead / (1024));
}
}

/**
* Writes form data to output stream as any HTTP client would for a
* multipart/form-data request.
Expand Down
Loading