Skip to content

HDDS-5023. Datanodes should always use MLV 0 when no VERSION file is present. #2080

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
avijayanhwx merged 4 commits into from
Mar 25, 2021
Merged

HDDS-5023. Datanodes should always use MLV 0 when no VERSION file is present. #2080

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0ec544a
Quick fix to make datanode's use correct mlv when pre finalized for f…
errose28 Mar 23, 2021
9c4aa17
Add documentation for new method
errose28 Mar 23, 2021
2671ab6
Use datanode.id file to inidcate whether DN with no VERSION file shou…
errose28 Mar 24, 2021
98210aa
Remove unnecessary SLV check
errose28 Mar 24, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 35 additions & 2 deletions ...service/src/main/java/org/apache/hadoop/ozone/container/common/DatanodeLayoutStorage.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,22 +27,23 @@
import org.apache.hadoop.hdds.conf.OzoneConfiguration;
import org.apache.hadoop.hdds.protocol.proto.HddsProtos.NodeType;
import org.apache.hadoop.hdds.server.ServerUtils;
import org.apache.hadoop.hdds.upgrade.HDDSLayoutFeature;
import org.apache.hadoop.hdds.utils.HddsServerUtil;
import org.apache.hadoop.ozone.common.Storage;

/**
* DataNodeStorageConfig is responsible for management of the
* StorageDirectories used by the DataNode.
*/
public class DatanodeLayoutStorage extends Storage {

/**
* Construct DataNodeStorageConfig.
* @throws IOException if any directories are inaccessible.
*/
public DatanodeLayoutStorage(OzoneConfiguration conf, String dataNodeId)
throws IOException {
super(NodeType.DATANODE, ServerUtils.getOzoneMetaDirPath(conf),
DATANODE_LAYOUT_VERSION_DIR, dataNodeId, maxLayoutVersion());
DATANODE_LAYOUT_VERSION_DIR, dataNodeId, getDefaultLayoutVersion(conf));
}

public DatanodeLayoutStorage(OzoneConfiguration conf, String dataNodeId,
Expand All @@ -61,4 +62,36 @@ public File getCurrentDir() {
protected Properties getNodeProperties() {
return new Properties();
}

/**
* Older versions of the code did not write a VERSION file to disk for the
* datanode. Therefore, When a datanode starts up and does not find a metadata
* layout version written to disk, it must figure out whether
* it is being installed on a new system, or as part of an upgrade to an
* existing system. To do this, it will use the presence of the datanode.id
* file to indicate that it is being started as part of an upgrade and
* should start with the initial layout version. Otherwise, it will use the
* latest layout version.
*
* This is only an issue when upgrading from software layout version
* {@link HDDSLayoutFeature#INITIAL_VERSION} to
* {@link HDDSLayoutFeature#DATANODE_SCHEMA_V2}. After this, the upgrade
* framework will write a VERSION file for the datanode and future software
* versions will always know the MLV to use. The result of this method will
* not be used in this case.
*
* @return The layout version that should be used for the datanode if no
* layout version is found on disk.
*/
private static int getDefaultLayoutVersion(OzoneConfiguration conf) {
int defaultLayoutVersion = maxLayoutVersion();

File dnIdFile = new File(HddsServerUtil.getDatanodeIdFilePath(conf));
if (dnIdFile.exists()) {
defaultLayoutVersion =
HDDSLayoutFeature.INITIAL_VERSION.layoutVersion();
}

return defaultLayoutVersion;
}
}