[SPARK-7481] [build] Add spark-hadoop-cloud module to pull in object store access.#17834

Closed

steveloughran wants to merge 28 commits intoapache:masterfrom

steveloughran:cloud/SPARK-7481-current

Commits on Apr 25, 2017

[SPARK-7481] stripped down packaging only module
steveloughran
committed
[SPARK-7481] basic instantiation tests verify that dependency hadoop-azure, hadoop-aws, hadoop-openstack and implicitly their transitive dependencies are resolved. They don't verify all dependency …
steveloughran
committed
[SPARK-7481] tests restricted to instantiation; logging modified appropriately
steveloughran
committed
[SPARK-7481] declare httpcomponents:httpclient explicitly, as downstream tests which pulled in spark-cloud but not spark-hive were ending up with inconsistent versions. Add a test for the missing c…
steveloughran
committed
[SPARK-7481] update docs by culling section on cloud integration tests; link to remaning docs from top level.
steveloughran
committed
[SPARK-7481] updated documentation as per review
steveloughran
committed
[SPARK-7481] SBT will build this now, optionally
steveloughran
committed
[SPARK-7481] cloud POM includes jackson-dataformat-cbor, so that the CP is set up consistently for the later versions of the AWS SDK
steveloughran
committed
[SPARK-7481] rebase with master; Pom had got out of sync
steveloughran
committed
[SPARK-7481] rename spark-cloud module to spark-hadoo-cloud, in POMs and docs
steveloughran
committed
[SPARK-7841] bump up cloud pom to 2.2.0-SNAPSHOT; other minor pom cleanup
steveloughran
committed
[SPARK-7481] builds against Hadoop shaded 3.x clients failing as direct references to AWS classes failing. Cut them and rely on transitive load through FS class instantation to force the load. All …
steveloughran
committed
[SPARK-7481] update 2.7 dependencies to include azure, aws and openstack JARs, transitive dependencies on aws and azure SDKs
steveloughran
committed
[SPARK-7481] add joda time as the dependency. Tested against hadoop branch-2, s3 ireland
steveloughran
committed
SPARK-7481 purge all tests from the cloud module
steveloughran
committed
SPARK-7481 add cloud module to sbt sequence
steveloughran
committed
SPARK-7481 break line of mvn XML declaration
steveloughran
committed
SPARK-7481 cloud pom is still JAR (not pom). works against Hadoop 2.6 as well as 2.7, keeping azure the 2.7.x dependency. All dependencies are scoped @ hadoop.scope
steveloughran
committed

Commits on Apr 27, 2017

Commits on Apr 28, 2017

Commits on May 2, 2017

Commits on May 3, 2017

SPARK-7481 module rename, POM movement, docs
steveloughran
committed

Commits on May 4, 2017

SPARK-7481: artifact is spark-hadoop-cloud while project is hadoop-cloud. Added an extra callout to the docs "without some form of consistency layer, Amazon S3 cannot
steveloughran
committed

Commits on May 5, 2017

SPARK-7481: applied proofreading, moved links to https; also cut a couple of superflous blank lines
steveloughran
committed