YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover#1399

Closed

RogPodge wants to merge 3472 commits intoapache:trunkfrom

RogPodge:DNSCacheFix

This pull request is big! We're only showing the most recent 250 commits

Commits on Mar 25, 2020

HDFS-15095. Fix TestDecommissioningStatus. Contributed by Ahmed Hussein.

kihwal
authored and
RogPodge
committed
HADOOP-16590. Update OS login modules for IBM JDK.

macroadster
authored and
RogPodge
committed
YARN-10019. Improved container-executor exec() calls.

macroadster
authored and
RogPodge
committed
HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang.

Chen Liang
authored and
RogPodge
committed
HDFS-15108. RBF: MembershipNamenodeResolver should invalidate cache incase of active namenode update. Contributed by Ayush Saxena.

ayushtkn
authored and
RogPodge
committed
HDFS-14578. AvailableSpaceBlockPlacementPolicy always prefers local node. Contributed by Ayush Saxena.

ayushtkn
authored and
RogPodge
committed
YARN-10067. Add dry-run feature to FS-CS converter tool. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
YARN-9866. u:user2:%primary_group is not working as expected. Contributed by Manikandan R

szilard-nemeth
authored and
RogPodge
committed
HADOOP-16797. Add Dockerfile for ARM builds. Contributed by Vinayakumar B. (#1801)

vinayakumarb
authored and
RogPodge
committed
YARN-9989. Typo in CapacityScheduler documentation: Runtime Configuration. Contributed by Kevin Su

szilard-nemeth
authored and
RogPodge
committed
YARN-9868. Validate %primary_group queue in CS queue manager. Contributed by Manikandan R

szilard-nemeth
authored and
RogPodge
committed
YARN-9912. Capacity scheduler: support u:user2:%secondary_group queue mapping. Contributed by Manikandan R

szilard-nemeth
authored and
RogPodge
committed
HDFS-15097. Purge log in KMS and HttpFS. Contributed by Doris Gu.

jojochuang
authored and
RogPodge
committed
YARN-10028. Integrate the new abstract log servlet to the JobHistory server. Contributed by Adam Antal

szilard-nemeth
authored and
RogPodge
committed
YARN-9788. Queue Management API does not support parallel updates. Contributed by Prabhu Joseph

szilard-nemeth
authored and
RogPodge
committed
HADOOP-16005. NativeAzureFileSystem does not support setXAttr.

c-w
authored and
RogPodge
committed
MAPREDUCE-7256. Fix javadoc error in SimpleExponentialSmoothing. (#1804)

iwasakims
authored and
RogPodge
committed
YARN-9970. Refactor TestUserGroupMappingPlacementRule#verifyQueueMapping. Contributed by Manikandan R

szilard-nemeth
authored and
RogPodge
committed
HADOOP-16803. Upgrade jackson-databind to 2.9.10.2. (#1805)

iwasakims
authored and
RogPodge
committed
YARN-10082. FS-CS converter: disable terminal placement rule checking. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
HADOOP-16790. Add Write Convenience Methods.

David Mollitor
authored and
RogPodge
committed
HDFS-13616. Batch listing of multiple directories (#1725)

sunchao
authored and
RogPodge
committed
YARN-9512. [JDK11] TestAuxServices#testCustomizedAuxServiceClassPath fails because of ClassCastException. (#1806)

aajisaka
authored and
RogPodge
committed
MAPREDUCE-7247. Modify HistoryServerRest.html content,change The job attempt id‘s datatype from string to int (#1684)

kevinzhao1661
authored and
RogPodge
committed
YARN-10070. Fix NPE if no queue mapping defined for proxy user when using application tag based placement.

Prabhu Joseph
authored and
RogPodge
committed
HADOOP-16621. [pb-upgrade] Remove Protobuf classes from signatures of Public APIs. Contributed by Vinayakumar B. (#1803)

vinayakumarb
authored and
RogPodge
committed
HDFS-15112. RBF: Do not return FileNotFoundException when a subcluster is unavailable.

Inigo Goiri
authored and
RogPodge
committed
YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph

szilard-nemeth
authored and
RogPodge
committed
YARN-10081. Exception message from ClientRMProxy#getRMAddress is misleading. Contributed by Ravuri Sushma Sree

szilard-nemeth
authored and
RogPodge
committed
YARN-9462. TestResourceTrackerService.testNodeRemovalGracefully fails sporadically. Contributed by Prabhu Joseph

szilard-nemeth
authored and
RogPodge
committed
YARN-9525. IFile format is not working against s3a remote folder. Contributed by Adam Antal

szilard-nemeth
authored and
RogPodge
committed
YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg

szilard-nemeth
authored and
RogPodge
committed
HADOOP-16785. followup to abfs close() fix.

steveloughran
authored and
RogPodge
committed
HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li.

aajisaka
authored and
RogPodge
committed
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.

aajisaka
authored and
RogPodge
committed
Remove WARN log when ipc connection interrupted in Client#handleSaslConnectionFailure()

sunlisheng
authored and
RogPodge
committed
HADOOP-16346. Stabilize S3A OpenSSL support.

Sahil Takiar
authored and
RogPodge
committed
HDFS-15126. TestDatanodeRegistration#testForcedRegistration fails intermittently. Contributed by Ahmed Hussein.

Inigo Goiri
authored and
RogPodge
committed
HDFS-15092. TestRedudantBlocks#testProcessOverReplicatedAndRedudantBlock sometimes fails. Contributed by Fei Hui.

Inigo Goiri
authored and
RogPodge
committed
YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.

Inigo Goiri
authored and
RogPodge
committed
HADOOP-16759. Filesystem openFile() builder to take a FileStatus param (#1761). Contributed by Steve Loughran

steveloughran
authored and
RogPodge
committed
Revert "YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R."

Inigo Goiri
authored and
RogPodge
committed
HADOOP-16670. Stripping Submarine code from Hadoop codebase. Contributed by Zhankun Tang.

tangzhankun
authored and
RogPodge
committed
HDFS-14968. Add ability to log stale datanodes. Contributed by Ahmed Hussein.

kihwal
authored and
RogPodge
committed
YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal

szilard-nemeth
authored and
RogPodge
committed
HDFS-15117. EC: Add getECTopologyResultForPolicies to DistributedFileSystem. Contributed by Ayush Saxena

ayushtkn
authored and
RogPodge
committed
HADOOP-16732. S3Guard to support encrypted DynamoDB table (#1752). Contributed by Mingliang Liu.

liuml07
authored and
RogPodge
committed
HADOOP-16826. ABFS: update abfs.md to include config keys for identity transformation

karthick-rn
authored and
RogPodge
committed
HADOOP-16792: Make S3 client request timeout configurable.

Mustafa Iman
authored and
RogPodge
committed
HDFS-15119. Allow expiration of cached locations in DFSInputStream.

kihwal
authored and
RogPodge
committed
HADOOP-16827. TestHarFileSystem.testInheritedMethodsImplemented broken. (#1817)

steveloughran
authored and
RogPodge
committed
HDFS-15128. Unit test failing to clean testing data and crashed future Maven test run due to failure in TestDataNodeVolumeFailureToleration. Contributed by Ctest

ayushtkn
authored and
RogPodge
committed
HADOOP-16811: Use JUnit TemporaryFolder Rule in TestFileUtils (#1811). Contributed by David Mollitor.

belugabehr
authored and
RogPodge
committed
HADOOP-16746. mkdirs and s3guard Authoritative mode.

steveloughran
authored and
RogPodge
committed
MAPREDUCE-7262. MRApp helpers block for long intervals (500ms)

amahussein
authored and
RogPodge
committed
HDFS-15143. LocatedStripedBlock returns wrong block type. Contributed by Ayush Saxena.

ayushtkn
authored and
RogPodge
committed
YARN-10085. FS-CS converter: remove mixed ordering policy check. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
YARN-10104. FS-CS converter: dry run should work without output defined. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
MAPREDUCE-7259. testSpeculateSuccessfulWithUpdateEvents fails Intermittently

amahussein
authored and
RogPodge
committed
HDFS-14993. checkDiskError doesn't work during datanode startup. Contributed by Yang Yun.

ayushtkn
authored and
RogPodge
committed
YARN-10022. RM Rest API to validate the CapacityScheduler Configuration change

Prabhu Joseph
authored and
RogPodge
committed
HDFS-15145. HttpFS: getAclStatus() returns permission as null. Contributed by hemanthboyina.

Inigo Goiri
authored and
RogPodge
committed
YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.

Inigo Goiri
authored and
RogPodge
committed
HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein.

Inigo Goiri
authored and
RogPodge
committed
[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Contributed by Chen Liang.

Chen Liang
authored and
RogPodge
committed
Revert "[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Contributed by Chen Liang."

Chen Liang
authored and
RogPodge
committed
[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Conntributed by Chen Liang

Chen Liang
authored and
RogPodge
committed
YARN-10015. Correct the sample command in SLS README file. Contributed by Aihua Xu.

flyrain
authored and
RogPodge
committed
YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne.

ericbadger
authored and
RogPodge
committed
YARN-10107. Fix GpuResourcePlugin#getNMResourceInfo to honor Auto Discovery Enabled

Prabhu Joseph
authored and
RogPodge
committed
MAPREDUCE-7260. Cross origin request support for Job history server web UI. Contributed by Adam Antal

szilard-nemeth
authored and
RogPodge
committed
MAPREDUCE-7079: JobHistory#ServiceStop implementation is incorrect. Contributed by Ahmed Hussein (ahussein)

Eric E Payne
authored and
RogPodge
committed
HDFS-15146. TestBalancerRPCDelay.testBalancerRPCDelay fails

kihwal
authored and
RogPodge
committed
YARN-9743. [JDK11] TestTimelineWebServices.testContextFactory fails. (#1824) Contributed by Akira Ajisaka and Kinga Marton.

aajisaka
authored and
RogPodge
committed
HADOOP-16801. S3Guard listFiles will not query S3 if all listings are authoritative (#1815). Contributed by Mustafa İman.

mustafaiman
authored and
RogPodge
committed
YARN-10099. FS-CS converter: handle allow-undeclared-pools and user-as-default-queue properly and fix misc issues. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
YARN-8982. [Router] Add locality policy. Contributed by Young Chen.

Giovanni Matteo Fumarola
authored and
RogPodge
committed
HDFS-7175. Client-side SocketTimeoutException during Fsck. Contributed by Stephen O'Donnell, Akira Ajisaka.

authored and
RogPodge
committed
HDFS-12491. Support wildcard in CLASSPATH for libhdfs. Contributed by Muhammad Samir Khan.

kihwal
authored and
RogPodge
committed
HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang.

Chen Liang
authored and
RogPodge
committed
YARN-10116. Expose diagnostics in RMAppManager summary

hungj
authored and
RogPodge
committed
YARN-10101. Support listing of aggregated logs for containers belonging to an application attempt. Contributed by Adam Antal

szilard-nemeth
authored and
RogPodge
committed
HADOOP-16832. S3Guard testing doc: Add required parameters for S3Guard testing in IDE. (#1822). Contributed by Mukund Thakur.

mukund-thakur
authored and
RogPodge
committed
HADOOP-16845: Disable ITestAbfsClient.testContinuationTokenHavingEqualSign due to ADLS Gen2 service bug.

snvijaya
authored and
RogPodge
committed
HADOOP-16825: ITestAzureBlobFileSystemCheckAccess failing.

bilaharith
authored and
RogPodge
committed
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency (#1635). Contributed by Vinayakumar B.

vinayakumarb
authored and
RogPodge
committed
HADOOP-16834. Replace com.sun.istack.Nullable with javax.annotation.Nullable in DNS.java. Contributed by Xieming Li.

aajisaka
authored and
RogPodge
committed
Bump checkstyle from 8.26 to 8.29 (#1828)

dependabot[bot]
authored and
RogPodge
committed
HDFS-15136. LOG flooding in secure mode when Cookies are not set in request header. Contributed by Renukaprasad C

ayushtkn
authored and
RogPodge
committed
HDFS-15115. Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug. Contributed by wangzhixiang

ayushtkn
authored and
RogPodge
committed
YARN-9624. Use switch case for ProtoUtils#convertFromProtoFormat containerState. Contributed by Bilwa S T

ayushtkn
authored and
RogPodge
committed
YARN-10109. Allow stop and convert from leaf to parent queue in a single Mutation API call. Contributed by Prabhu Joseph

sunilgovind
authored and
RogPodge
committed
HDFS-15158. The number of failed volumes mismatch with volumeFailures of Datanode metrics. Contributed by Yang Yun.

ayushtkn
authored and
RogPodge
committed
HADOOP-16739. Fix native build failure of hadoop-pipes on CentOS 8.

iwasakims
authored and
RogPodge
committed
HADOOP-16847. Test can fail if HashSet iterates in a different order.

testfixer
authored and
RogPodge
committed
HADOOP-16851. Removed unused import in Configuration

HorizonNet
authored and
RogPodge
committed
HDFS-15150. Introduce read write lock to Datanode. Contributed Stephen O'Donnell.

Stephen O'Donnell
authored and
RogPodge
committed
YARN-10127. Remove setting App Ordering Policy to ParentQueue in FSQueueConverter

Prabhu Joseph
authored and
RogPodge
committed
HDFS-14758. Make lease hard limit configurable and reduce the default.

kihwal
authored and
RogPodge
committed
HADOOP-16849. start-build-env.sh behaves incorrectly when username is numeric only. Contributed by Jihyun Cho.

aajisaka
authored and
RogPodge
committed
HADOOP-16856. cmake is missing in the CentOS 8 section of BUILDING.txt. (#1841)

iwasakims
authored and
RogPodge
committed
HDFS-15127. RBF: Do not allow writes when a subcluster is unavailable for HASH_ALL mount points. Contributed by Inigo Goiri

ayushtkn
authored and
RogPodge
committed
MAPREDUCE-7263. Remove obsolete validateTargetPath() from FrameworkUploader. Contributed by Marton Hudaky

szilard-nemeth
authored and
RogPodge
committed
HDFS-15161. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close(). Contributed by Lisheng Sun

ayushtkn
authored and
RogPodge
committed
HDFS-13989. RBF: Add FSCK to the Router (#1832)

authored and
RogPodge
committed
YARN-9521. Handle FileSystem close in ApiServiceClient

Prabhu Joseph
authored and
RogPodge
committed
YARN-10029. Add option to UIv2 to get container logs from the new JHS API. Contributed by Adam Antal

szilard-nemeth
authored and
RogPodge
committed
HDFS-15086. Block scheduled counter never get decremet if the block got deleted before replication. Contributed by hemanthboyina.

surendralilhore
authored and
RogPodge
committed
YARN-10137. UIv2 build is broken in trunk. Contributed by Adam Antal

szilard-nemeth
authored and
RogPodge
committed
HADOOP-16823. Large DeleteObject requests are their own Thundering Herd.

steveloughran
authored and
RogPodge
committed
HADOOP-16850. Support getting thread info from thread group for JvmMetrics to improve the performance. Contributed by Tao Yang.

aajisaka
authored and
RogPodge
committed
YARN-10136. [Router] : Application metrics are hardcode as N/A in UI. Contributed by Bilwa S T.

brahmareddybattula
authored and
RogPodge
committed
HDFS-15164. Fix TestDelegationTokensWithHA. Contributed by Ayush Saxena.

ayushtkn
authored and
RogPodge
committed
HDFS-15135. EC : ArrayIndexOutOfBoundsException in BlockRecoveryWorker#RecoveryTaskStriped. Contributed by Ravuri Sushma sree.

surendralilhore
authored and
RogPodge
committed
HDFS-15173. RBF: Delete repeated configuration 'dfs.federation.router.metrics.enable' (#1849)

xiaoxiaopan118
authored and
RogPodge
committed
HADOOP-13666. Supporting rack exclusion in countNumOfAvailableNodes in NetworkTopology. Contributed by Inigo Goiri.

ayushtkn
authored and
RogPodge
committed
HADOOP-15961. S3A committers: make sure there's regular progress() calls.

lqjack
authored and
RogPodge
committed
HADOOP-16759. FileSystem Javadocs to list what breaks on API changes

steveloughran
authored and
RogPodge
committed
HADOOP-16833. InstrumentedLock should log lock queue time. Contributed by Stephen O'Donnell.

arp7
authored and
RogPodge
committed
YARN-8374. Upgrade objenesis to 2.6 (#1798)

aajisaka
authored and
RogPodge
committed
HDFS-13739. Add option to disable rack local write preference. Contributed by Ayush Saxena.

ayushtkn
authored and
RogPodge
committed
YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB. Contributed by Prabhu Joseph.

sunilgovind
authored and
RogPodge
committed
HADOOP-16868. ipc.Server readAndProcess threw NullPointerException. Contributed by Tsz-wo Sze.

jojochuang
authored and
RogPodge
committed
YARN-10147. FPGA plugin can't find the localized aocx file. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
HDFS-15165. In Du missed calling getAttributesProvider. Contributed by Bharat Viswanadham.

Inigo Goiri
authored and
RogPodge
committed
YARN-10143. YARN-10101 broke Yarn logs CLI. Contributed by Adam Antal

szilard-nemeth
authored and
RogPodge
committed
YARN-10119. Option to reset AM failure count for YARN Service

Prabhu Joseph
authored and
RogPodge
committed
HDFS-15052. WebHDFS getTrashRoot leads to OOM due to FileSystem object creation. (#1758)

iwasakims
authored and
RogPodge
committed
HADOOP-16869. Upgrade findbugs-maven-plugin to 3.0.5 to fix mvn findbugs:findbugs failure (#1855)

aajisaka
authored and
RogPodge
committed
HADOOP-16706. ITestClientUrlScheme fails for accounts which don't support HTTP

steveloughran
authored and
RogPodge
committed
HADOOP-16711.

mukund-thakur
authored and
RogPodge
committed
HDFS-15185. StartupProgress reports edits segments until the entire startup completes. Contributed by Konstantin V Shvachko.

shvachko
authored and
RogPodge
committed
HDFS-15172. Remove unnecessary deadNodeDetectInterval in DeadNodeDetector#checkDeadNodes(). Contributed by Lisheng Sun.

Inigo Goiri
authored and
RogPodge
committed
HDFS-14731. [FGL] Remove redundant locking on NameNode. Contributed by Konstantin V Shvachko.

shvachko
authored and
RogPodge
committed
HDFS-15182. TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk. Contributed by Lisheng Sun.

ayushtkn
authored and
RogPodge
committed
HDFS-15041. Make MAX_LOCK_HOLD_MS and full queue size configurable. Contributed by zhuqi.

ayushtkn
authored and
RogPodge
committed
HDFS-15176. Enable GcTimePercentage Metric in NameNode's JvmMetrics. Contributed by Jinglun.

ayushtkn
authored and
RogPodge
committed
HDFS-15166. Remove redundant field fStream in ByteStringLog. Contributed by Xieming Li.

ayushtkn
authored and
RogPodge
committed
HDFS-15187. CORRUPT replica mismatch between namenodes after failover. Contributed by Ayush Saxena.

ayushtkn
authored and
RogPodge
committed
HADOOP-16859: ABFS: Add unbuffer support to ABFS connector.

sahilTakiar
authored and
RogPodge
committed
HADOOP-16853. ITestS3GuardOutOfBandOperations failing on versioned S3 buckets (#1840)

steveloughran
authored and
RogPodge
committed
YARN-10157. FS-CS converter: initPropertyActions() is not called without rules file. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
YARN-10135. FS-CS converter tool: issue warning on dynamic auto-create mapping rules. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
HDFS-15174. Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations. Contributed by Lisheng Sun.

jojochuang
authored and
RogPodge
committed
HADOOP-16841. The description of hadoop.http.authentication.signature.secret.file contains outdated information. Contributed by Xieming Li.

tasanuma
authored and
RogPodge
committed
YARN-10074. Update netty to 4.1.42Final in yarn-csi. Contributed by Wei-Chiu Chuang. (#1807)

jojochuang
authored and
RogPodge
committed
YARN-10130. FS-CS converter: Do not allow output dir to be the same as input dir. Contributed by Adam Antal

szilard-nemeth
authored and
RogPodge
committed
YARN-8767. TestStreamingStatus fails. Contributed by Andras Bokor

szilard-nemeth
authored and
RogPodge
committed
HDFS-14861. Reset LowRedundancyBlocks Iterator periodically. Contributed by Stephen O'Donnell.

Stephen O'Donnell
authored and
RogPodge
committed
YARN-10156. Fix typo 'complaint' which means quite different in Federation.md (#1856)

sungpeo
authored and
RogPodge
committed
YARN-9593. Support Comma in the value of Scheduler Configuration Mutation API.

tanu.ajmera
authored and
RogPodge
committed
HDFS-15147. LazyPersistTestCase wait logic is flawed. Contributed by Ahmed Hussein.

kihwal
authored and
RogPodge
committed
YARN-10141.Interceptor in FederationInterceptorREST doesnt update on RM switchover. Contributed by D M Murali Krishna Reddy.

brahmareddybattula
authored and
RogPodge
committed
HDFS-15120. Refresh BlockPlacementPolicy at runtime. Contributed by Jinglun.

ayushtkn
authored and
RogPodge
committed
HDFS-15111. stopStandbyServices() should log which service state it is transitioning from. Contributed by Xieming Li.

ayushtkn
authored and
RogPodge
committed
YARN-10152. Fix findbugs warnings in hadoop-yarn-applications-mawo-core module (#1852)

aajisaka
authored and
RogPodge
committed
HDFS-15167. Block Report Interval shouldn't be reset apart from first Block Report. Contributed by Ayush Saxena.

surendralilhore
authored and
RogPodge
committed
HDFS-14668 Support Fuse with Users from multiple Security Realms (#1739)

fapifta
authored and
RogPodge
committed
HDFS-15124. Crashing bugs in NameNode when using a valid configuration for . Contributed by Ctest.

ayushtkn
authored and
RogPodge
committed
HADOOP-16730: ABFS: Support for Shared Access Signatures (SAS). Contributed by Sneha Vijayarajan.

snvijaya
authored and
RogPodge
committed
HDFS-15186. Erasure Coding: Decommission may generate the parity block's content with all 0 in some case. Contributed by Yao Guangdong.

ayushtkn
authored and
RogPodge
committed
YARN-10148. Add Unit test for queue ACL for both FS and CS. Contributed by Kinga Marton

szilard-nemeth
authored and
RogPodge
committed
YARN-10155. TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk. Contributed by Manikandan R.

Inigo Goiri
authored and
RogPodge
committed
YARN-10161. TestRouterWebServicesREST is corrupting STDOUT. Contributed by Jim Brennan.

Inigo Goiri
authored and
RogPodge
committed
HDFS-15199. NPE in BlockSender. Contributed by Ayush Saxena.

surendralilhore
authored and
RogPodge
committed
HADOOP-16891. Upgrade jackson-databind to 2.9.10.3 (#1865)

smengcl
authored and
RogPodge
committed
HDFS-15190. HttpFS: Add Support for Storage Policy Satisfier. Contributed by hemanthboyina.

tasanuma
authored and
RogPodge
committed
HDFS-15149. TestDeadNodeDetection test cases time-out. Contributed by Lisheng Sun.

Inigo Goiri
authored and
RogPodge
committed
HDFS-15033. Support to save replica cached files to other place and make expired time configurable. Contributed by Yang Yun.

ayushtkn
authored and
RogPodge
committed
HADOOP-16885. Encryption zone file copy failure leaks a temp file

xiaoyuyao
authored and
RogPodge
committed
YARN-10148. addendum: Fix method call parameter order of setAdminAndSubmitACL in TestCapacitySchedulerQueueACLs. Contributed by Kinga Marton

szilard-nemeth
authored and
RogPodge
committed
HADOOP-16767 Handle non-IO exceptions in reopen()

spoganshev
authored and
RogPodge
committed
HADOOP-16794. S3A reverts KMS encryption to the bucket's default KMS key in rename/copy.

mukund-thakur
authored and
RogPodge
committed
HADOOP-16897. Sort fields in ReflectionUtils.java.

cpugputpu
authored and
RogPodge
committed
HADOOP-14936. S3Guard: remove experimental from documentation.

Gabor Bota
authored and
RogPodge
committed
YARN-10175. FS-CS converter: only convert placement rules if a cmd line switch is defined. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
HADOOP-16885. Fix hadoop-commons TestCopy failure

steveloughran
authored and
RogPodge
committed
HADOOP-16899. Update HdfsDesign.md to reduce ambiguity. (#1871)

invincible-akshay
authored and
RogPodge
committed
HDFS-15204. TestRetryCacheWithHA testRemoveCacheDescriptor fails intermittently. Contributed by Ahmed Hussein.

Inigo Goiri
authored and
RogPodge
committed
HDFS-14977. Quota Usage and Content summary are not same in Truncate with Snapshot. Contributed by hemanthboyina.

Inigo Goiri
authored and
RogPodge
committed
YARN-10173. Make pid file generation timeout configurable in case of reacquired

ericbadger
authored and
RogPodge
committed
YARN-10167. FS-CS Converter: Need to validate c-s.xml after converting. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
YARN.10003. YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore. Contributed by Benjamin Teke

szilard-nemeth
authored and
RogPodge
committed
HDFS-15212. TestEncryptionZones.testVersionAndSuiteNegotiation fails in trunk. Contributed by Ayush Saxena.

Inigo Goiri
authored and
RogPodge
committed
HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches. (#1876)

jojochuang
authored and
RogPodge
committed
YARN-6924. Metrics for Federation AMRMProxy. Contributed by Young Chen

bibinchundatt
authored and
RogPodge
committed
HADOOP-14206. TestSFTPFileSystem#testFileExists failure: Invalid encoding for signature. Contributed by Jim Brennan.

jzhuge
authored and
RogPodge
committed
HADOOP-16840. AliyunOSS: getFileStatus throws FileNotFoundException in versioning bucket. Contributed by wujinhu.

yangwwei
authored and
RogPodge
committed
HADOOP-16871. Upgrade Netty version to 4.1.45.Final to handle CVE-2019-20444,CVE-2019-16869

brahmareddybattula
authored and
RogPodge
committed
HADOOP-16909 Typo in distcp counters.

sebastian-nagel
authored and
RogPodge
committed
HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden

steveloughran
authored and
RogPodge
committed
HADOOP-16898. Batch listing of multiple directories via an (unstable) interface

steveloughran
authored and
RogPodge
committed
YARN-9419. Log a warning if GPU isolation is enabled but LinuxContainerExecutor is disabled. Contribued by Andras Gyori

szilard-nemeth
authored and
RogPodge
committed
YARN-942. TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically Contributed by Ahmed Hussein (ahussein)

Eric E Payne
authored and
RogPodge
committed
YARN-10168. FS-CS Converter: tool doesn't handle min/max resource conversion correctly. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
YARN-10002. Code cleanup and improvements in ConfigurationStoreBaseTest. Contributed by Benjamin Teke

szilard-nemeth
authored and
RogPodge
committed
YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori

szilard-nemeth
authored and
RogPodge
committed
Hadoop 16890. Change in expiry calculation for MSI token provider.

bilaharith
authored and
RogPodge
committed
YARN-10191. FS-CS converter: call System.exit function call for every code path in main method. Contributed by Peter Bacsko.

sunilgovind
authored and
RogPodge
committed
YARN-10193. FS-CS converter: fix incorrect capacity conversion. Contributed by Peter Bacsko

szilard-nemeth
authored and
RogPodge
committed
YARN-9997. Code cleanup in ZKConfigurationStore. Contributed by Andras Gyori

szilard-nemeth
authored and
RogPodge
committed
HDFS-14442. Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId. Contributed by Ravuri Sushma sree.

surendralilhore
authored and
RogPodge
committed
HADOOP-15430. hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard (#1646)

steveloughran
authored and
RogPodge
committed
HDFS-14612. SlowDiskReport won't update when SlowDisks is always empty in heartbeat. Contributed by Haibin Huang.

Inigo Goiri
authored and
RogPodge
committed
HDFS-15039. Cache meta file length of FinalizedReplica to reduce call File.length(). Contributed by Yang Yun.

jojochuang
authored and
RogPodge
committed
YARN-10110. Adding RouterPolicyProvider for RM Federation.

BilwaST
authored and
RogPodge
committed
HDFS-15216. Wrong Use Case of -showprogress in fsck. Contributed by Ravuri Sushma sree.

S O'Donnell
authored and
RogPodge
committed
HDFS-15155. writeIoRate of DataNodeVolumeMetrics is never used. Contributed by Haibin Huang.

ayushtkn
authored and
RogPodge
committed
YARN-10195. Dependency divergence building Timeline Service on HBase 2.2.0 and above. (#1895)

jojochuang
authored and
RogPodge
committed
HDFS-14743. Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to support Authorization of mkdir, rm, rmdir, copy, move etc... (#1829)

jojochuang
authored and
RogPodge
committed
YARN-9538. Document scheduler/app activities and REST APIs. Contributed by Tao Yang.

yangwwei
authored and
RogPodge
committed
YARN-9567. Add diagnostics for outstanding resource requests on app attempts page. Contributed by Tao Yang.

yangwwei
authored and
RogPodge
committed
HDFS-15113. Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He.

Hexiaoqiao
authored and
RogPodge
committed
HDFS-14820. The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big. Contributed by Lisheng Sun.

jojochuang
authored and
RogPodge
committed
HDFS-15159. Prevent adding same DN multiple times in PendingReconstructionBlocks. Contributed by hemanthboyina.

ayushtkn
authored and
RogPodge
committed
HDFS-15197. [SBN read] Change ObserverRetryOnActiveException log to debug. Contributed by Chen Liang.

ayushtkn
authored and
RogPodge
committed
HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena.

surendralilhore
authored and
RogPodge
committed
MAPREDUCE-7237. Supports config the shuffle's path cache related parameters (#1397)

jiwq
authored and
RogPodge
committed
HADOOP-16661. Support TLS 1.3 (#1880)

jojochuang
authored and
RogPodge
committed
YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed

ericbadger
authored and
RogPodge
committed
HADOOP-16319. S3A Etag tests fail with default encryption enabled on bucket.

steveloughran
authored and
RogPodge
committed
HADOOP-16858. S3Guard fsck: Add option to remove orphaned entries (#1851). Contributed by Gabor Bota.

Gabor Bota
authored and
RogPodge
committed
HDFS-15208. Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs.

jojochuang
authored and
RogPodge
committed
HADOOP-16054. Update Dockerfile to use Bionic.

aajisaka
authored and
RogPodge
committed
HADOOP-16920 ABFS: Make list page size configurable.

bilaharith
authored and
RogPodge
committed
HDFS-14919. Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun.

ayushtkn
authored and
RogPodge
committed
YARN-10034. Remove Allocation Tags from released container from Decommission node

Prabhu Joseph
authored and
RogPodge
committed
HDFS-15223. FSCK fails if one namenode is not available. Contributed by Ayush Saxena.

ayushtkn
authored and
RogPodge
committed
HDFS-15200. Delete Corrupt Replica Immediately Irrespective of Replicas On Stale Storage. Contributed by Ayush Saxena.

ayushtkn
authored and
RogPodge
committed
HADOOP-16927. Update hadoop-thirdparty dependency version to 1.0.0 (#1900). Contributed by Vinayakumar B.

vinayakumarb
authored and
RogPodge
committed
HADOOP-16818. ABFS: Combine append+flush calls for blockblob & appendblob

ishaniahuja
authored and
RogPodge
committed
HDFS-15214. WebHDFS: Add snapshot counts to Content Summary. Contributed by hemanthboyina.

tasanuma
authored and
RogPodge
committed
HDFS-15227. NPE if the last block changes from COMMITTED to COMPLETE during FSCK. Contributed by Ayush Saxena.

ayushtkn
authored and
RogPodge
committed
YARN-10198. Remove Validation of Queue Exist under Managed Parent Queue

Prabhu Joseph
authored and
RogPodge
committed
ABFS Streams to update FileSystem.Statistics counters on IO.

mehakmeet
authored and
RogPodge
committed
HDFS-15113. Addendum: Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He.

Hexiaoqiao
authored and
RogPodge
committed
HDFS-15232. Fix libhdfspp test failures with GCC 7. (#1906)

aajisaka
authored and
RogPodge
committed
HDFS-15088. RBF: Correct annotation typo of RouterPermissionChecker#checkPermission. Contributed by Xiaoqiao He.

Hexiaoqiao
authored and
RogPodge
committed
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit (#1870)

karthikhw
authored and
RogPodge
committed
Revert ""HADOOP-16910. ABFS Streams to update FileSystem.Statistics counters on IO."

steveloughran
authored and
RogPodge
committed
Revert "HADOOP-16054. Update Dockerfile to use Bionic." (#1911)

aajisaka
authored and
RogPodge
committed
HDFS-13377. The owner of folder can set quota for his sub folder. Contributed by Yang Yun.

ayushtkn
authored and
RogPodge
committed
HDFS-15219. DFS Client will stuck when ResponseProcessor.run throw Error (#1902). Contributed by zhengchenyu.

isahekmat
authored and
RogPodge
committed
HDFS-15215. The Timestamp for longest write/read lock held log is wrong

brfrn169
authored and
RogPodge
committed
Merge branch 'trunk' of https://github.com/RogPodge/hadoop into trunk
RogPodge
committed
Merge branch 'trunk' of https://github.com/RogPodge/hadoop into DNSCacheFix
RogPodge
committed
Merge branch 'trunk' of https://github.com/apache/hadoop into DNSCacheFix
RogPodge
committed
remaining merge changes
RogPodge
committed
minor changes to fix merge errors
RogPodge
committed
lingering edit error
RogPodge
committed