HDDS-7182. Add property to control RocksDB max open files #3843
Conversation
ayushtkn
changed the title
| // one hour | ||
| static final long ROCKSDB_DELETE_OBSOLETE_FILES_PERIOD_MICRO_SECONDS_DEFAULT = | ||
| 6 * 60 * 60 * 1000 * 1000; | ||
| 1 * 60 * 60 * 1000 * 1000; | ||
| static final int ROCKSDB_MAX_OPEN_FILES_DEFAULT = 1024; |
There was a problem hiding this comment.
Is this working? looks like a numeric overflow case to me, Integer multiplication is being implicitly cast to long.
Tried a small assertion
assertEquals(1L * 60 * 60 * 1000 * 1000, ROCKSDB_DELETE_OBSOLETE_FILES_PERIOD_MICRO_SECONDS_DEFAULT);
It indeed gives me a -694967296
Secondly why are we changing the default value in the scope of adding a new property?
There was a problem hiding this comment.
@ayushtkn thanks, will check this.
The hitchhiking is trying to avoid another small patch for this.
| @Config(key = "rocksdb.delete_obsolete_files_period", | ||
| defaultValue = "6h", timeUnit = MICROSECONDS, | ||
| @Config(key = "rocksdb.delete-obsolete-files-period", |
There was a problem hiding this comment.
Are we changing the config name? replacing _ with -, Even above ROCKSDB_DELETE_OBSOLETE_FILES_PERIOD_MICRO_SECONDS_KEY has _, I am not sure but changing config names must be an incompatible change right?
There was a problem hiding this comment.
The main purpose of Replacing _ with - is to be consistent with other ozone property naming style. The property is recently added, should not introduce any compatibility issue.
There was a problem hiding this comment.
@ChenSammi do we not need to change _ with - here in the value of this variable as well
ROCKSDB_DELETE_OBSOLETE_FILES_PERIOD_MICRO_SECONDS_KEY =
"hdds.datanode.rocksdb.delete_obsolete_files_period";
There was a problem hiding this comment.
The property is recently added, should not introduce any compatibility issue.
@ChenSammi, please cherry-pick this to the branch ozone-1.3 to avoid change of config property name between releases.
|
Thanks @guihecheng and @ayushtkn for the code review. |
* master: (718 commits) HDDS-7342. Move encryption-related code from MultipartCryptoKeyInputStream to OzoneCryptoInputStream (apache#3852) HDDS-7413. Fix logging while marking container state unhealthy (apache#3887) Revert "HDDS-7253. Fix exception when '/' in key name (apache#3774)" HDDS-7396. Force close non-RATIS containers in ReplicationManager (apache#3877) HDDS-7121. Support namespace summaries (du, dist & counts) for legacy FS buckets (apache#3746) HDDS-7258. Cleanup the allocated but uncommitted blocks (apache#3778) HDDS-7381. Cleanup of VolumeManagerImpl (apache#3873) HDDS-7253. Fix exception when '/' in key name (apache#3774) HDDS-7182. Add property to control RocksDB max open files (apache#3843) HDDS-7284. JVM crash for rocksdb for read/write after close (apache#3801) HDDS-7368. [Multi-Tenant] Add Volume Existence check in preExecute for OMTenantCreateRequest (apache#3869) HDDS-7403. README Security Improvement (apache#3879) HDDS-7199. Implement new mix workload Read/Write Freon command (apache#3872) HDDS-7248. Recon: Expand the container status page to show all unhealthy container states (apache#3837) HDDS-7141. Recon: Improve Disk Usage Page (apache#3789) HDDS-7369. Fix wrong order of command arguments in Nonrolling-Upgrade.md (apache#3866) HDDS-6210. EC: Add EC metrics (apache#3851) HDDS-7355. non-primordial scm fail to get signed cert from primordial SCM when converting an unsecure cluster to secure (apache#3859) HDDS-7356. Update SCM-HA.zh.md to match the English version (apache#3861) HDDS-6930. SCM,OM,RECON should not print ERROR and exit with code 1 on successful shutdown (apache#3848) ... Conflicts: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/LegacyReplicationManager.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/TestLegacyReplicationManager.java
4 participants
https://issues.apache.org/jira/browse/HDDS-7182