Branch 2.7.4#267

Closed

wangzhen11aaa wants to merge 8617 commits intotrunkfrom

This pull request is big! We're only showing the most recent 250 commits

Commits on Aug 29, 2016

MAPREDUCE-6768. TestRecovery.testSpeculative failed with NPE. Contributed by Haibo Chen
jlowe
committed
HDFS-10625. VolumeScanner to report why a block is found bad. Contributed by Rushabh S Shah and Yiqun Lin.
Yongjun Zhang
committed
YARN-5550. TestYarnCLI#testGetContainers should format according to CONTAINER_PATTERN. Contributed by Jonathan Hung.
zhe-thoughts
committed

Commits on Aug 31, 2016

Commits on Sep 1, 2016

Commits on Sep 8, 2016

HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula)
arp7
committed

Commits on Sep 9, 2016

HADOOP-13579. Fix source-level compatibility after HADOOP-11252. Contributed by Tsuyoshi Ozawa.
aajisaka
committed

Commits on Sep 12, 2016

HDFS-5802. NameNode does not check for inode type before traversing down a path. (Xiao Chen via Yongjun Zhang)

Yongjun Zhang
authored and
zhe-thoughts
committed
HDFS-10276. HDFS should not expose path info that user has no permission to see. (Yuanbo Liu via Yongjun Zhang)

Yongjun Zhang
authored and
zhe-thoughts
committed
HDFS-9019. Adding informative message to sticky bit permission denied exception. Contributed by Xiaoyu Yao.

xiaoyuyao
authored and
zhe-thoughts
committed
HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable.

Tsz-Wo Nicholas Sze
authored and
zhe-thoughts
committed
HDFS-10673. Optimize FSPermissionChecker's internal path usage. Contributed by Daryn Sharp.

kihwal
authored and
zhe-thoughts
committed

Commits on Sep 13, 2016

In Balancer, the target task should be removed when its size < 0. Contributed by Yiqun Lin

Tsz-Wo Nicholas Sze
authored and
zhe-thoughts
committed

Commits on Sep 14, 2016

HDFS-10744. Internally optimize path component resolution. Contributed by Daryn Sharp.

kihwal
authored and
zhe-thoughts
committed

Commits on Sep 15, 2016

HDFS-8200. Refactor FSDirStatAndListingOp. Contributed by Haohui Mai.

Haohui Mai
authored and
zhe-thoughts
committed
HDFS-9621. Consolidate FSDirStatAndListingOp#createFileStatus to let its INodesInPath parameter always include the target INode. Contributed by Jing Zhao.

kihwal
authored and
zhe-thoughts
committed

Commits on Sep 19, 2016

Commits on Sep 20, 2016

HADOOP-13601. Fix a log message typo in AbstractDelegationTokenSecretManager. Contributed by Mehran Hassani.
liuml07
committed

Commits on Sep 23, 2016

Commits on Sep 26, 2016

HADOOP-12597. In kms-site.xml configuration hadoop.security.keystore.JavaKeyStoreProvider.password should be updated with new name. ( Contributed by Surendra Singh Lilhore via Brahma Reddy Battula)
brahmareddybattula
committed

Commits on Sep 27, 2016

Commits on Sep 28, 2016

Commits on Sep 29, 2016

MAPREDUCE-6771. RMContainerAllocator sends container diagnostics event after corresponding completion event. Contributed by Haibo Chen
jlowe
committed

Commits on Sep 30, 2016

HDFS-10713. Throttle FsNameSystem lock warnings. Contributed by Hanisha Koneru.

arp7
authored and
zhe-thoughts
committed

Commits on Oct 4, 2016

Commits on Oct 5, 2016

Commits on Oct 6, 2016

Commits on Oct 7, 2016

HDFS-8824. Do not use small blocks for balancing the cluster.

Tsz-Wo Nicholas Sze
authored and
zhe-thoughts
committed

Commits on Oct 11, 2016

Commits on Oct 13, 2016

HDFS-11002. Fix broken attr/getfattr/setfattr links in ExtendedAttributes.md. Contributed by Mingliang Liu.
aajisaka
committed

Commits on Oct 18, 2016

Commits on Oct 20, 2016

Commits on Oct 21, 2016

HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula
brahmareddybattula
committed

Commits on Oct 25, 2016

HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee.
zhe-thoughts
committed

Commits on Oct 26, 2016

Commits on Oct 28, 2016

MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera)
jlowe
committed

Commits on Oct 30, 2016

YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino)

Carlo Curino
authored and
brahmareddybattula
committed

Commits on Nov 1, 2016

YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen
jlowe
committed

Commits on Nov 2, 2016

Commits on Nov 4, 2016

YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter
jlowe
committed

Commits on Nov 8, 2016

HADOOP-13804. MutableStat mean loses accuracy if add(long, long) is used. Contributed by Erik Krogen.
zhe-thoughts
committed

Commits on Nov 9, 2016

HADOOP-13782. Make MutableRates metrics thread-local write, aggregate-on-read. Contributed by Erik Krogen.
zhe-thoughts
committed

Commits on Nov 10, 2016

HDFS-8307. Spurious DNS Queries from hdfs shell. Contributed by Andres Perez
anuengineer
committed

Commits on Nov 13, 2016

HDFS-11087. NamenodeFsck should check if the output writer is still writable. Contributed by Erik Krogen.

xkrogen
authored and
shvachko
committed

Commits on Nov 14, 2016

HDFS-11056. Concurrent append and read operations lead to checksum error. Contributed by Wei-Chiu Chuang.
jojochuang
committed

Commits on Nov 18, 2016

HDFS-10941. Improve BlockManager#processMisReplicatesAsync log. Contributed by Chen Liang.

xiaoyuyao
authored and
zhe-thoughts
committed

Commits on Nov 21, 2016

HDFS-10966. Enhance Dispatcher logic on deciding when to give up a source DataNode. Contributed by Mark Wagner and Zhe Zhang.
kihwal
committed

Commits on Nov 22, 2016

HADOOP-13655. document object store use with fs shell and distcp. Contributed by Steve Loughran
liuml07
committed

Commits on Nov 25, 2016

HDFS-11174. Wrong HttpFS test command in doc. Contributed by John Zhuge.
aajisaka
committed

Commits on Nov 28, 2016

HADOOP-13742. Expose NumOpenConnectionsPerUser as a metric. Brahma Reddy Battula.

kihwal
authored and
zhe-thoughts
committed

Commits on Nov 29, 2016

HADOOP-13812. Upgrade Tomcat to 6.0.48. Contributed by John Zhuge.
aajisaka
committed

Commits on Dec 3, 2016

HDFS-11180. Intermittent deadlock in NameNode when failover happens.
aajisaka
committed

Commits on Dec 5, 2016

YARN-5694. ZKRMStateStore can prevent the transition to standby if the ZK node is unreachable. Contributed by Daniel Templeton
jian-he
committed

Commits on Dec 9, 2016

HADOOP-11859. PseudoAuthenticationHandler fails with httpcomponents v4.4. Contributed by Eugene Koifman.
kihwal
committed

Commits on Dec 10, 2016

HDFS-11229. HDFS-11056 failed to close meta file. Contributed by Wei-Chiu Chuang.
jojochuang
committed

Commits on Dec 16, 2016

HDFS-11160. VolumeScanner reports write-in-progress replicas as corrupt incorrectly. Contributed by Wei-Chiu Chuang and Yongjun Zhang.
jojochuang
committed

Commits on Dec 20, 2016

HDFS-11263. ClassCastException when we use Bzipcodec for Fsimage compression. Contributed by Brahma Reddy Battula.
brahmareddybattula
committed

Commits on Dec 28, 2016

YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan.
sunilgovind
committed

Commits on Dec 31, 2016

HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao.
Haohui Mai
committed

Commits on Jan 3, 2017

Revert "HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao."
brahmareddybattula
committed

Commits on Jan 5, 2017

Commits on Jan 8, 2017

Commits on Jan 18, 2017

HDFS-10733. NameNode terminated after full GC thinking QJM is unresponsive. Contributed by Vinitha Gankidi.

vgankidi
authored and
shvachko
committed

Commits on Jan 19, 2017

HADOOP-14001. Improve delegation token validity checking.
aajisaka
committed

Commits on Jan 21, 2017

HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu.
macroadster
committed

Commits on Jan 24, 2017

Commits on Jan 26, 2017

Commits on Feb 2, 2017

YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path. Contributed by Xuan Gong

JunpingDu
authored and
zhe-thoughts
committed

Commits on Feb 7, 2017

YARN-6152. Used queue percentage not accurate in UI for 2.7 and below when using DominantResourceCalculator. Contributed by Jonathan Hung
erichadoop
committed

Commits on Feb 9, 2017

HADOOP-13433 Race in UGI.reloginFromKeytab. Contributed by Duo Zhang.
xiao-chen
committed

Commits on Feb 15, 2017

HDFS-11333. Print a user friendly error message when plugins are not found. Contributed by Wei-Chiu Chuang.
jojochuang
committed

Commits on Feb 17, 2017

HDFS-11379. DFSInputStream may infinite loop requesting block locations. Contributed by Daryn Sharp.
xiao-chen
committed

Commits on Feb 28, 2017

Commits on Mar 2, 2017

HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry.

steveloughran
authored and
jlowe
committed

Commits on Mar 3, 2017

YARN-6274. Documentation refers to incorrect nodemanager health checker interval property. Contributed by WeiWei Yang
jlowe
committed

Commits on Mar 7, 2017

Treat encrypted files as private. Contributed by Daniel Templeton.
aajisaka
committed

Commits on Mar 10, 2017

YARN-6310. OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions. Contributed by Haibo Chen
jlowe
committed

Commits on Mar 13, 2017

HDFS-11499. Decommissioning stuck because of failing recovery. Contributed by Lukas Majercak and Manoj Govindassamy.
jojochuang
committed

Commits on Mar 18, 2017

HADOOP-14169. Implement listStatusIterator, listLocatedStatus for ViewFs. Contributed by Erik Krogen.

xkrogen
authored and
shvachko
committed

Commits on Mar 23, 2017

HADOOP-9631. ViewFs should use underlying FileSystem's server side defaults. Contributed by Lohit Vijayarenu and Erik Krogen.
zhe-thoughts
committed

Commits on Mar 24, 2017

HADOOP-14211. FilterFs and ChRootedFs are too aggressive about enforcing 'authorityNeeded'. Contributed by Erik Krogen.

umbrant
authored and
zhe-thoughts
committed

Commits on Mar 28, 2017

HDFS-11486. Client close() should not fail fast if the last block is being decommissioned. Contributed by Wei-Chiu Chuang and Yiqun Lin.
iwasakims
committed

Commits on Mar 30, 2017

MAPREDUCE-6873. MR Job Submission Fails if MR framework application path not on defaultFS. Contributed by Erik Krogen.
zhe-thoughts
committed

Commits on Apr 5, 2017

HDFS-11628. Clarify the behavior of HDFS Mover in documentation. Contributed by Xiaobing Zhou.

Xiaobing Zhou
authored and
arp7
committed

Commits on Apr 7, 2017

HADOOP-14276. Add a nanosecond API to Time/Timer/FakeTimer. Contributed by Erik Krogen.
zhe-thoughts
committed

Commits on Apr 10, 2017

HADOOP-14293. Initialize FakeTimer with a less trivial value.
umbrant
committed

Commits on Apr 11, 2017

Commits on Apr 13, 2017

HADOOP-14195. CredentialProviderFactory$getProviders is not thread-safe. Contributed by Vihang Karajgaonkar.
jzhuge
committed

Commits on Apr 17, 2017

HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen.
zhe-thoughts
committed

Commits on Apr 18, 2017

Commits on Apr 26, 2017

HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R.

brahmareddybattula
authored and
zhe-thoughts
committed

Commits on May 1, 2017

Commits on May 2, 2017

HADOOP-14371. License error in TestLoadBalancingKMSClientProvider.java. Contributed by hu xiaodong.
aajisaka
committed

Commits on May 3, 2017

Commits on May 9, 2017

Commits on May 10, 2017

Commits on May 11, 2017

Commits on May 12, 2017

Commits on May 16, 2017

Commits on May 18, 2017

Commits on May 19, 2017

Commits on May 24, 2017

Commits on May 25, 2017

Commits on May 26, 2017

Fix NPE in LazyPersistFileScrubber. Contributed by Inigo Goiri.

goiri
authored and
shvachko
committed

Commits on May 27, 2017

HDFS-8674. Improve performance of postponed block scans. Contributed by Daryn Sharp.

kihwal
authored and
shvachko
committed

Commits on May 30, 2017

Commits on May 31, 2017

HDFS-10619. Cache path in InodesInPath. Contributed by Daryn Sharp.

kihwal
authored and
zhe-thoughts
committed
HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp.

kihwal
authored and
zhe-thoughts
committed

Commits on Jun 2, 2017

Commits on Jun 3, 2017

Commits on Jun 4, 2017

YARN-5988. RM unable to start in secure setup. Contributed by Ajith S.
sunilgovind
committed

Commits on Jun 5, 2017

Commits on Jun 6, 2017

HDFS-8549. Abort the balancer if an upgrade is in progress. Contributed by Andrew Wang. Backport HDFS-11808 by Akira Ajisaka.

umbrant
authored and
shvachko
committed

Commits on Jun 7, 2017

Commits on Jun 8, 2017

YARN-4017. container-executor overuses PATH_MAX. Contributed by Sidharta Seethana

varun-vasu
authored and
aajisaka
committed

Commits on Jun 10, 2017

YARN-5121. fix some container-executor portability issues. Contributed by Allen Wittenauer. Backport YARN-6698 by Akira Ajisaka

cnauroth
authored and
shvachko
committed

Commits on Jun 12, 2017

HADOOP-14440. Add metrics for connections dropped. Contributed by Eric Badger.

kihwal
authored and
zhe-thoughts
committed

Commits on Jun 15, 2017

Commits on Jun 19, 2017

Commits on Jun 26, 2017

MAPREDUCE-6433. launchTime may be negative. Contributed by Zhihai Xu.
aajisaka
committed

Commits on Jul 3, 2017

Commits on Jul 10, 2017

HADOOP-14044. Synchronization issue in delegation token cancel functionality. Contributed by Hrishikesh Gadre.
xiao-chen
committed

Commits on Jul 11, 2017

Commits on Jul 13, 2017

MAPREDUCE-6697. Concurrent task limits should only be applied when necessary. Contributed by Nathan Roberts.

aajisaka
authored and
shvachko
committed

Commits on Jul 14, 2017

Commits on Jul 17, 2017

Commits on Jul 19, 2017

HDFS-11345. Document the configuration key for FSNamesystem lock fairness. Contributed by Erik Krogen.
aajisaka
committed

Commits on Jul 20, 2017

HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang.
shvachko
committed

Commits on Jul 21, 2017

Commits on Jul 25, 2017

Commits on Jul 27, 2017

HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula.

brahmareddybattula
authored and
shvachko
committed

Commits on Jul 28, 2017