HADOOP-18891 hadoop distcp needs support to filter by file/directory attribute#6058

Closed

AuthurWang2009 wants to merge 1580 commits intoapache:trunkfrom

AuthurWang2009:release-3.3.6

This pull request is big! We're only showing the most recent 250 commits

Commits on Aug 23, 2022

HDFS-16669: Enhance client protocol to propagate last seen state IDs for multiple nameservices.

simbadzina
authored and
omalley
committed

Commits on Aug 24, 2022

HADOOP-18333. Upgrade jetty version to 9.4.48.v20220622 (#4600)

authored
HADOOP-18417. Upgrade to M7 of surefire plugin (#4795)

snmvaughan
authored and
steveloughran
committed
MAPREDUCE-7403. manifest-committer dynamic partitioning support. (#4728)
steveloughran
committed
Revert "HADOOP-18417. Upgrade to M7 of surefire plugin (#4795)"
ayushtkn
committed
HADOOP-13144. Enhancing IPC client throughput via multiple connections per user (#4542)

ZanderXu
authored and
omalley
committed
HADOOP-18406: Adds alignment context to call path for creating RPC proxy with multiple connections per user.

simbadzina
authored and
omalley
committed

Commits on Aug 25, 2022

HDFS-16732. [SBN READ] Avoid get location from observer when the block report is delayed (#4756)

zhengchenyu
authored and
xkrogen
committed

Commits on Aug 28, 2022

HDFS-16684. Exclude the current JournalNode (#4786)
snmvaughan
authored

Commits on Aug 30, 2022

HADOOP-18375. Fix failure of shelltest for hadoop_add_ldlibpath. (#4652)
iwasakims
committed

Commits on Aug 31, 2022

Commits on Sep 1, 2022

Commits on Sep 2, 2022

HADOOP-18408. ABFS: ITestAbfsManifestCommitProtocol fails on nonHNS configuration (#4758)

sreeb-msft
authored and
steveloughran
committed

Commits on Sep 6, 2022

HADOOP-17873. ABFS: Fix transient failures in ITestAbfsStreamStatistics and ITestAbfsRestOperationException (#3699)

sumangala-patki
authored and
steveloughran
committed

Commits on Sep 9, 2022

HADOOP-18439. Fix VectoredIO for LocalFileSystem when checksum is enabled. (#4862)
mukund-thakur
committed

Commits on Sep 12, 2022

YARN-11241. Add uncleaning option for local app log file with log-aggregation enabled (#4703)

authored and
aajisaka
committed

Commits on Sep 13, 2022

HDFS-16686. GetJournalEditServlet fails to authorize valid Kerberos request (#4724) (#4794)
snmvaughan
authored

Commits on Sep 19, 2022

HADOOP-18400. Fix file split duplicating records from a succeeding split when reading BZip2 text files (#4732)

authored and
aajisaka
committed

Commits on Sep 21, 2022

HADOOP-16769. LocalDirAllocator to provide diagnostics when file creation fails (#4896)
hotcodemacha
authored

Commits on Sep 22, 2022

HADOOP-18341: upgrade commons-configuration2 to 2.8.0 and commons-text to 1.9 (#4916)
pjfanning
authored

Commits on Sep 23, 2022

Commits on Sep 25, 2022

HADOOP-18443. Upgrade snakeyaml to 1.32 (#4873)

hotcodemacha
and
Ashutosh Gupta
authored

Commits on Sep 27, 2022

HDFS-16766. XML External Entity (XXE) attacks can occur while processing XML received from an untrusted source (#4886)

authored and
aajisaka
committed

Commits on Sep 28, 2022

Commits on Oct 5, 2022

YARN-11303. Upgrade jquery ui to 1.13.2 to mitigate CVE-2022-31160 (#4895)
hotcodemacha
authored

Commits on Oct 6, 2022

MAPREDUCE-7370. Parallelize MultipleOutputs#close call (#4248). Contributed by Ashutosh Gupta.

hotcodemacha
authored and
cnauroth
committed

Commits on Oct 7, 2022

Commits on Oct 9, 2022

HDFS-16024. RBF: Rename data to the Trash should be based on src location (#4962)
xinglin
authored

Commits on Oct 11, 2022

Commits on Oct 13, 2022

HADOOP-18292. Fix s3 select tests when running against unsupported storage class (#4489)

monthonk
authored and
steveloughran
committed

Commits on Oct 14, 2022

HADOOP-18382. AWS SDK v2 upgrade prerequisites (#4698)

ahmarsuhail
authored and
steveloughran
committed
HADOOP-18481. AWS v2 SDK upgrade log to not about standard AWS Credential Providers. (#4973)

ahmarsuhail
authored and
steveloughran
committed

Commits on Oct 15, 2022

HADOOP-17563. Upgrade BouncyCastle to 1.68 (#3980) (#5015)
steveloughran
authored

Commits on Oct 17, 2022

HADOOP-18493: upgrade jackson-databind to 2.12.7.1 (#5011). Contributed by PJ Fanning.

pjfanning
authored and
ayushtkn
committed
HADOOP-18360. Update commons-csv from 1.0 to 1.9.0. (#4928). Contributed by fanshilun.

slfan1989
authored and
ayushtkn
committed

Commits on Oct 18, 2022

Commits on Oct 19, 2022

Commits on Oct 21, 2022

Commits on Oct 22, 2022

HDFS-16566 Erasure Coding: Recovery may causes excess replicas when busy DN exsits (#4252) (#5059)

tasanuma
and
RuinanGu
authored

Commits on Oct 24, 2022

HDFS-16016. BPServiceActor to provide new thread to handle IBR (#2998)

virajjasani
authored and
tasanuma
committed

Commits on Oct 26, 2022

MAPREDUCE-7411: use secure XML parsers in mapreduce modules (#4980)

pjfanning
authored and
steveloughran
committed

Commits on Oct 27, 2022

Commits on Oct 28, 2022

YARN-11360: Add number of decommissioning/shutdown nodes to YARN cluster metrics. (#5060)
cnauroth
committed

Commits on Oct 29, 2022

HADOOP-18472. Upgrade to snakeyaml 1.33 (#4958)

pjfanning
authored and
aajisaka
committed

Commits on Oct 31, 2022

HADOOP-18233. Possible race condition with TemporaryAWSCredentialsProvider (#5024)

sabertiger
authored and
steveloughran
committed

Commits on Nov 1, 2022

HADOOP-18512: upgrade woodstox-core to 5.4.0 for security fix (#5087). Contributed by PJ Fanning.

pjfanning
authored and
ayushtkn
committed
document fix for MAPREDUCE-7425 (#5090)

wangteng13
authored and
cnauroth
committed
YARN-11363. Remove unused TimelineVersionWatcher and TimelineVersion from hadoop-yarn-server-tests (#5091)

hotcodemacha
authored and
cnauroth
committed
YARN-11364. Docker Container to accept docker Image name with sha256 digest (#5092)

authored and
cnauroth
committed

Commits on Nov 4, 2022

HADOOP-18484. Upgrade hsqldb to v2.7.1 to mitigate CVE-2022-41853 (#5101)
hotcodemacha
authored

Commits on Nov 5, 2022

HADOOP-18515. Backport HADOOP-17612 to branch-3.3(Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0) (#5097)

authored

Commits on Nov 7, 2022

Hadoop-18519. Backport HDFS-15383 and HADOOP-17835 to branch-3.3 (#5112)

authored

Commits on Nov 8, 2022

Commits on Nov 10, 2022

HDFS-16811. Support DecommissionBackoffMonitor parameters reconfigurable (#5122)
haiyang1987
authored

Commits on Nov 15, 2022

HADOOP-18528. Disable abfs prefetching by default (#5134)

mehakmeet
authored and
steveloughran
committed
HDFS-16836: StandbyCheckpointer shouldn't trigger rollback fs image after RU is finalized (#5135)

li-leyang
authored and
omalley
committed

Commits on Nov 18, 2022

Commits on Nov 21, 2022

HDFS-16832. [SBN READ] Follow-on to HDFS-16732. Fix NPE when check the block location of empty directory (#5099)

zhengchenyu
authored and
xkrogen
committed

Commits on Nov 28, 2022

HADOOP-18498. ABFS: Remove unwanted ? prefix from SAS Tokens (#5136)

sreeb-msft
authored and
steveloughran
committed

Commits on Nov 29, 2022

Commits on Nov 30, 2022

HDFS-16839 It should consider EC reconstruction work when we determine if a node is busy (#5128)

authored and
jojochuang
committed
HADOOP-18457. ABFS: Support account level throttling (#5034)

anmolanmol1234
authored and
steveloughran
committed

Commits on Dec 1, 2022

HDFS-16633. Fixing when Reserved Space For Replicas is not released on some cases (#4452)

hotcodemacha
authored and
aajisaka
committed

Commits on Dec 5, 2022

Commits on Dec 6, 2022

HADOOP-18560. AvroFSInput opens a stream twice and discards the second one without closing (#5186)
steveloughran
committed

Commits on Dec 8, 2022

Commits on Dec 9, 2022

HADOOP-18546. ABFS. disable purging list of in progress reads in abfs stream close() (#5176)

saxenapranav
authored and
steveloughran
committed

Commits on Dec 12, 2022

HADOOP-18329. Support for IBM Semeru JVM > 11.0.15.0 Vendor Name Changes (#4537) (#5208)
JackBuggins
authored

Commits on Dec 14, 2022

Commits on Dec 15, 2022

Commits on Dec 16, 2022

Commits on Dec 18, 2022

Commits on Dec 19, 2022

Commits on Dec 22, 2022

YARN-10879. Incorrect WARN text in ACL check for application tag based placement (#5231)

susheel-gupta
authored and
brumi1024
committed

Commits on Dec 23, 2022

YARN-11395. RM UI, RMAttemptBlock can not render FINAL_SAVING. Contributed by Bence Kosztolnik

K0K0V0K
authored and
brumi1024
committed

Commits on Dec 28, 2022

Commits on Jan 1, 2023

HADOOP-18586. Update the year to 2023. (#5265). Contributed by Ayush Saxena.
ayushtkn
committed

Commits on Jan 6, 2023

HADOOP-18587: upgrade to jettison 1.5.3 due to cve (#5270)

pjfanning
authored and
cnauroth
committed

Commits on Jan 10, 2023

Commits on Jan 11, 2023

HDFS-16886: Fixes error in documentation for StateStoreRecordsOperations. (#5290)

simbadzina
authored and
omalley
committed

Commits on Jan 12, 2023

HADOOP-18591. Fix a typo in Trash (#5291)

huangxiaopingRD
authored and
cnauroth
committed
MAPREDUCE-7375 JobSubmissionFiles don't set right permission after mkdirs (#4237)

skysiders
authored and
cnauroth
committed
HDFS-16887 Log start and end of phase/step in startup progress (#5292)

virajjasani
authored and
cnauroth
committed

Commits on Jan 16, 2023

HADOOP-18575: followup: try to avoid repeatedly hitting exceptions when transformer factories do not support attributes (#5253)

pjfanning
authored and
steveloughran
committed

Commits on Jan 18, 2023

HDFS-16891 Avoid the overhead of copy-on-write exception list while loading inodes sub sections in parallel (#5300)

virajjasani
authored and
cnauroth
committed

Commits on Jan 24, 2023

HADOOP-18582. skip unnecessary cleanup logic in distcp (#5251)

authored and
cnauroth
committed

Commits on Jan 27, 2023

HADOOP-17717. Update wildfly openssl to 1.1.3.Final. (#5310)
steveloughran
committed

Commits on Jan 28, 2023

HADOOP-18604. Add compile platform in the hadoop version output. (#5327). Contributed by Ayush Saxena.
ayushtkn
committed

Commits on Jan 31, 2023

Commits on Feb 1, 2023

HADOOP-18584. [NFS GW] Fix regression after netty4 migration. (#5252)
jojochuang
committed
HADOOP-18592. Sasl connection failure should log remote address. (#5294)

virajjasani
authored and
liuml07
committed
HADOOP-18279. Cancel fileMonitoringTimer even if trustManager isn't defined (#4789)

snmvaughan
and
Steve Vaughan Jr
authored

Commits on Feb 3, 2023

HDFS-16902. Add Namenode status to BPServiceActor metrics and improve logging in offerservice (#5334)
tomscut
committed

Commits on Feb 6, 2023

HADOOP-18612. Avoid mixing canonical and non-canonical when performing comparisons (#5339)

snmvaughan
authored and
steveloughran
committed

Commits on Feb 7, 2023

HADOOP-18621. Resource leak in CryptoOutputStream.close() (#5347)

gardenia
authored and
steveloughran
committed

Commits on Feb 9, 2023

HDFS-16904. Close webhdfs during TestSymlinkHdfs teardown (#5342)

snmvaughan
authored and
steveloughran
committed
HADOOP-18625. Fix method name of RPC.Builder#setnumReaders (#5301)

haiyang1987
authored and
steveloughran
committed

Commits on Feb 10, 2023

HDFS-18324. Fix race condition in closing IPC connections. (#5371)
omalley
committed

Commits on Feb 11, 2023

Commits on Feb 12, 2023

HDFS-16907. Add LastHeartbeatResponseTime for BP service actor (#5349) (#5358)
virajjasani
authored

Commits on Feb 13, 2023

HDFS-16904. Close webhdfs during TestSymlinkHdfs teardown (#5372)

snmvaughan
authored and
steveloughran
committed

Commits on Feb 14, 2023

Commits on Feb 15, 2023

HADOOP-18628. IPC Server Connection should log host name before returning VersionMismatch error (#5385)
virajjasani
authored

Commits on Feb 16, 2023

HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers (#4215)

bbeaudreault
authored and
omalley
committed

Commits on Feb 17, 2023

Commits on Feb 21, 2023

HDFS-16925. Namenode audit log to only include IP address of client (#5413)
virajjasani
authored

Commits on Feb 23, 2023

Commits on Feb 24, 2023

Commits on Feb 27, 2023

Commits on Feb 28, 2023

HADOOP-18641. Cloud connector dependency and LICENSE fixup. (#5429)
steveloughran
committed

Commits on Mar 1, 2023

HDFS-16935. Fix TestFsDatasetImpl#testReportBadBlocks (#5432)

virajjasani
authored and
steveloughran
committed

Commits on Mar 2, 2023

HDFS-16923. [SBN read] getlisting RPC to observer will throw NPE if path does not exist (#5400)

ZanderXu
authored and
xkrogen
committed
HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref… (#5322) (#5444)

mccormickt12
and
Tom McCormick
authored

Commits on Mar 5, 2023

YARN-11383. Workflow priority mappings is case sensitive (#5171)
varunsaxena
committed

Commits on Mar 6, 2023

HDFS-16934. TestDFSAdmin.testAllDatanodesReconfig regression (#5434)

slfan1989
authored and
steveloughran
committed

Commits on Mar 8, 2023

HADOOP-17836. Improve logging on ABFS error reporting (#3281)
steveloughran
committed

Commits on Mar 10, 2023

YARN-11394. Fix hadoop-yarn-server-resourcemanager module Java Doc Errors. (#5288)
slfan1989
authored

Commits on Mar 11, 2023

Commits on Mar 13, 2023

HADOOP-18661. Fix bin/hadoop usage script terminology. (#5473)
steveloughran
committed

Commits on Mar 15, 2023

HDFS-16942. Addendum. Send error to datanode if FBR is rejected due to bad lease (#5478). Contributed by Stephen O'Donnell/

sodonnel
authored and
S O'Donnell
committed
HADOOP-17746. Compatibility table in directory_markers.md doesn't render right. (#3116)

iwasakims
authored and
steveloughran
committed

Commits on Mar 28, 2023

Commits on Mar 29, 2023

HADOOP-18620 Avoid using grizzly-http-* APIs (#5356) (#5374)
virajjasani
authored

Commits on Mar 30, 2023

HADOOP-18647. x-ms-client-request-id to identify the retry of an API. (#5437)
saxenapranav
authored

Commits on Apr 3, 2023

HADOOP-18680: Insufficient heap during full test runs in Docker container on branch-3.3.
cnauroth
committed

Commits on Apr 5, 2023

Commits on Apr 15, 2023

HADOOP-18590. Publish SBOM artifacts (#5555). Contributed by Dongjoon Hyun.

dongjoon-hyun
authored and
ayushtkn
committed

Commits on Apr 16, 2023

HADOOP-18693. Bump derby from 10.10.2.0 to 10.14.2.0 in /hadoop-project (#5427)

dependabot[bot]
authored and
smengcl
committed

Commits on Apr 17, 2023

HADOOP-18696. ITestS3ABucketExistence arn test failures. (#5557)
steveloughran
committed

Commits on Apr 18, 2023

HADOOP-18470. Hadoop 3.3.5 release wrap-up (#5558)
steveloughran
committed

Commits on Apr 20, 2023

HADOOP-18691. Add a CallerContext getter on the Schedulable interface (#5540)

xBis7
authored and
smengcl
committed

Commits on Apr 23, 2023

Commits on Apr 24, 2023

Commits on Apr 25, 2023

Commits on Apr 26, 2023

HADOOP-18712. Upgrade to jetty 9.4.51 due to cve. Contributed by PJ Fanning. (#5574) (#5585)
pjfanning
authored

Commits on Apr 28, 2023

Commits on Apr 29, 2023

Commits on May 2, 2023

Commits on May 4, 2023

HADOOP-18727. Fix WriteOperations.listMultipartUploads function description (#5613)

dongjoon-hyun
authored and
steveloughran
committed

Commits on May 5, 2023

YARN-11482. Fix bug of DRF comparision DominantResourceFairnessComparator2 in fair scheduler. (#5607). Contributed by Xiaoqiao He.
Hexiaoqiao
committed

Commits on May 8, 2023

HADOOP-18671. Add recoverLease(), setSafeMode(), isFileClosed() as interfaces to hadoop-common (#5553) (#5619)

jojochuang
and
taklwu
authored

Commits on May 9, 2023

HADOOP-18687. Remove json-smart dependency. (#5549 + #5524)
rohit-kb
authored

Commits on May 10, 2023

Commits on May 15, 2023

Commits on May 16, 2023

Commits on May 18, 2023

Commits on May 20, 2023

HDFS-17022. Fix the exception message to print the Identifier pattern (#5678). Contributed by Nishtha Shah.

NishthaShah
authored and
ayushtkn
committed

Commits on May 21, 2023

HDFS-17017. Fix the issue of arguments number limit in report command in DFSAdmin (#5667). Contributed by Haiyang Hu.

haiyang1987
authored and
ayushtkn
committed

Commits on May 23, 2023

HDFS-16882. RBF: Add cache hit rate metric in MountTableResolver#getDestinationForPath (#5276) (#5423). Contributed by farmmamba
hfutatzhanghb
authored

Commits on Jun 1, 2023

HADOOP-18755. openFile builder new optLong() methods break hbase-filesystem (#5704)
steveloughran
committed

Commits on Jun 5, 2023

HADOOP-18433. Fix main thread name for . (#4838) (#5692)

adoroszlai
and
zhengchenyu
authored

Commits on Jun 8, 2023

Commits on Jun 9, 2023

HADOOP-17386. Change default fs.s3a.buffer.dir to be under Yarn container path on yarn applications (#3908)

monthonk
authored and
steveloughran
committed

Commits on Jun 10, 2023

HADOOP-18646. Upgrade Netty to 4.1.89.Final to fix CVE-2022-41881 (#5435) (#5729)

jojochuang
and
nao-it
authored

Commits on Jun 12, 2023

Commits on Jun 13, 2023

HADOOP-18622. Upgrade ant to 1.10.13 (#5360). Contributed by Aleksandr Nikolaev.

nao-it
authored and
jojochuang
committed
HADOOP-18538. Upgrade kafka to 2.8.2 (#5164)

dmmkr
authored and
jojochuang
committed

Commits on Jun 14, 2023

HADOOP-18763. Upgrade aws-java-sdk to 1.12.367 (#5741)

virajjasani
authored and
jojochuang
committed

Commits on Jun 18, 2023

3.3.6 Release LICENSE file update.
jojochuang
committed

Commits on Sep 12, 2023

add support for distcp dirs circumstance
AuthurWang2009
committed

Commits on Sep 13, 2023

fix Catalog Webapp module compile errors due to low nodejs version, and update it to latest stable version
AuthurWang2009
committed