Skip to content

HDFS-16338. Correct fsimage error configuration message #3684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ayushtkn merged 19 commits into from
Dec 6, 2021
Merged

HDFS-16338. Correct fsimage error configuration message #3684

Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
3c51579
Merge pull request #1 from apache/trunk
GuoPhilipse Nov 29, 2020
a449d04
Merge pull request #2 from apache/trunk
GuoPhilipse Dec 12, 2020
d4f968b
Merge pull request #3 from apache/trunk
GuoPhilipse Apr 7, 2021
d522abf
Merge pull request #4 from apache/trunk
GuoPhilipse Apr 21, 2021
de633b8
Merge pull request #5 from apache/trunk
GuoPhilipse Oct 16, 2021
66ce06b
Merge pull request #6 from apache/trunk
GuoPhilipse Nov 8, 2021
c0aa861
'correctfsimage'
tclxgf13871 Nov 19, 2021
4743a6a
'triggerrebuild'
tclxgf13871 Nov 19, 2021
bea061f
Merge pull request #7 from apache/trunk
GuoPhilipse Nov 19, 2021
ad9eab5
Merge remote-tracking branch 'origin/trunk' into correctfsimage
tclxgf13871 Nov 19, 2021
ef4d239
'triggerrebuild'
tclxgf13871 Nov 19, 2021
95909ec
addtestcase
GuoPhilipse Nov 21, 2021
a883029
'triggerrebuild'
tclxgf13871 Nov 22, 2021
aa0bb69
'simplecode'
tclxgf13871 Nov 25, 2021
5072db4
'adjustformat'
tclxgf13871 Nov 25, 2021
faf3055
'fixcheckstyle'
tclxgf13871 Nov 26, 2021
f748f13
fixcomment
GuoPhilipse Dec 4, 2021
f05f24a
triggerrebuild
GuoPhilipse Dec 4, 2021
f70eddb
fixstyle
GuoPhilipse Dec 5, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ...dfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -582,12 +582,12 @@ void doImportCheckpoint(FSNamesystem target) throws IOException {

if (checkpointDirs == null || checkpointDirs.isEmpty()) {
throw new IOException("Cannot import image from a checkpoint. "
+ "\"dfs.namenode.checkpoint.dir\" is not set." );
+ "\"dfs.namenode.checkpoint.dir\" is not set.");
}

if (checkpointEditsDirs == null || checkpointEditsDirs.isEmpty()) {
Copy link
Contributor

@virajjasani virajjasani Dec 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw checkpointDirs and checkpointEditsDirs will not be null as they are assigned values returned by FSImage.getCheckpointDirs and FSImage.getCheckpointEditsDirs respectively.

Copy link
Member

@ayushtkn ayushtkn Dec 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, my IntelliJ was giving that warning as well, but that was something not touched here....
I thought Todd added it just for future safety, Didn't research why there was a null check considering it harmless.

Copy link
Contributor

@virajjasani virajjasani Dec 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it seems fine, just an additional null check, not a big deal. Also good to keep it for future safety as you predicted correctly, just in case if the methods themselves start returning null (less likely but still doable).

throw new IOException("Cannot import image from a checkpoint. "
+ "\"dfs.namenode.checkpoint.dir\" is not set." );
+ "\"dfs.namenode.checkpoint.edits.dir\" is not set.");
}

FSImage realImage = target.getFSImage();
Expand Down
24 changes: 24 additions & 0 deletions ...project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@
import org.junit.Test;

import static org.junit.Assert.assertArrayEquals;
import static org.junit.Assert.fail;

public class TestFSImage {

Expand Down Expand Up @@ -275,6 +276,29 @@ public void testSaveAndLoadStripedINodeFile() throws IOException{
}
}

@Test
public void testImportCheckpoint() {
Configuration conf = new Configuration();
conf.set(DFSConfigKeys.DFS_NAMENODE_CHECKPOINT_EDITS_DIR_KEY, "");
MiniDFSCluster cluster = null;
try {
cluster = new MiniDFSCluster.Builder(conf).build();
cluster.waitActive();
FSNamesystem fsn = cluster.getNamesystem();
FSImage fsImage= new FSImage(conf);
fsImage.doImportCheckpoint(fsn);
fail("Expect to throw IOException.");
} catch (IOException e) {
GenericTestUtils.assertExceptionContains(
"Cannot import image from a checkpoint. "
+ "\"dfs.namenode.checkpoint.edits.dir\" is not set.", e);
Copy link
Member

@ayushtkn ayushtkn Dec 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use LambdaTestUtils instead of try-catch-assert

Copy link
Member Author

@GuoPhilipse GuoPhilipse Dec 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ayushtkn for your advice, just updated.

} finally {
if (cluster != null) {
cluster.shutdown();
}
}
}

/**
* Test if a INodeFileUnderConstruction with BlockInfoStriped can be
* saved and loaded by FSImageSerialization
Expand Down