initiateMultipartUpload and completeMultipartUpload S3 methods

[onsails]

No description provided.

[dwhjames]

@prettynatty, I’m happy to accept this, but I would strongly recommend that you make use of TransferManager API rather than these lower level methods.

[onsails]

Not in my case - I need to use lower level methods to be able to upload file parts as single objects and then merge them using copyPartRequest: http://docs.aws.amazon.com/AmazonS3/latest/dev/CopyingObjctsUsingLLJavaMPUapi.html

[onsails]

Also added copyPart wrapper

[dwhjames]

So with that approach you can do things that can’t be done with:
http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/TransferManager.html#copy(com.amazonaws.services.s3.model.CopyObjectRequest)
?

[onsails]

This method returns immediately and does not allow validation (what if source does not exist?).

[dwhjames]

That exception would be captured by the Copy transfer object, right?

Sorry, I’m not intending to tell you how you should implement things. I’m just surprised and curious that what you are trying to do can’t be done with TransferManager. My impression was that that API enabled you do everything that could be done with respect to transfers, while taking care of multi-part uploads, and efficient use of threads and connections. So if there are gaps in its capabilities, I’m certainly interested.

[onsails]

Hmm, I see.

It's no problem to make such suggestions 😃 I'm pleased to listen them. I am just started learning S3 api.
I am implementing a service which generates pre-signed urls for uploading file parts (one url per part), wait after client uploads parts directly to S3 using these urls and then concatenate parts using copy method.
Seems like I can use TransferManager for everything except generating signed urls. That's good.
But I still need to call blocking waitForCopyResult method, right?

[dwhjames]

See
https://dwhjames.github.io/aws-wrap/doc/s3.html

aws-wrap/src/main/scala/s3/s3.scala

Lines 504 to 514 in 087cbdf

	/**
	* A helper object providing a Scala Future interface for S3 Transfers.
	*
	* Transfers to and from S3 using the TransferManager provider a listener
	* interface, and [[FutureTransfer.listenFor]] adapts this interface to
	* Scala futures.
	*
	* @see [[http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/TransferManager.html TransferManager]]
	* @see [[http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/Transfer.html Transfer]]
	*/
	object FutureTransfer {

with regards to blocking on transfers.

I have no experience with the pre-signed urls part, so maybe your use case requires direct use of the low-level API? I just try to avoid it in favor of using the higher-level transfer API, which is suppose to do a good job with respect to resource usage. And with the transfer listeners, it’s easy to bridge that into Scala Futures, which is what FutureTransfer.listenFor does.

[onsails]

Thank you, I'll play with TransferManager and FutureTransfer. But yes, I still need low-level api for generating pre-signed urls. Would you merge this pull request?

[dwhjames]

Would you merge this pull request?

Sure. Would you mind squashing your commits down to one in this case, as they are all closely related.

I’ll be able to publish a new release later today—I don’t have access to my software signing keys right at the moment.

[onsails]

Great! Squashed

[dwhjames]

Thanks for your PR. Best of luck with implementing your upload feature.

[dwhjames]

@prettynatty, this PR was released as part of v0.7.2