S3 v4 Signature does not match

[JacobEvelyn]

Fine Uploader version

5.16.2

Browsers where the bug is reproducible

All

Operating systems where the bug is reproducible

macOS

All relevant Fine Uploader-related code that you have written

fineUploader = new qq.s3.FineUploaderBasic({
  debug: true,
  maxConnections: 6,
  chunking: {
    enabled: true,
    mandatory: true,
    concurrent: { enabled: true }
  },
  objectProperties: {
    acl: "private",
    bucket: data.bucket,
    serverSideEncryption: true,
    key: function(fileId) { return ... }, // Including in case this is relevant.
  },
  signature: {
    endpoint: "http://0.0.0.0:8080/vendor/fineuploader/php-s3-server/endpoint.php",
    version: 4 // THIS IS THE ONLY LINE THAT CHANGES! If I change this to 2, everything works.
  },
  retry: {
    enableAuto: true
  },
  resume: {
    enabled: true
  },
  request: {
    endpoint: "https://s3.amazonaws.com/" + BUCKET,
    accessKey: ACCESS_KEY_ID
  },
  callbacks: { // Including in case this is relevant.
    onError: function(id, name, errorReason) { ... },
    onComplete: function(id, name, responseJSON) { ... },
    onTotalProgress: function(uploaded, total) { ... }
  }
});

Detailed explanation of the problem

I'm not sure if this is a bug with FineUploader or the FineUploader PHP server (or, as I suspect, user error on my part). I had FineUploader working fine uploading directly to S3 using v2 signatures, but we'd like to switch to v4 signatures.

The exact code above, including the use of the example endpoint, works using the v2 signature. But when I switch to v4 I see this error from Amazon:

Specific problem detected initiating multipart upload request for 0: 'The request signature we calculated does not match the signature you provided. Check your key and signing method.'

Am I missing something obvious?

[rnicholus]

What is the server returning when you are expecting a V4 signature? Does it look different than a V2 signature? Have you looked at the V4 test suite to verify the signature returned by the server?

[JacobEvelyn]

The server's returning JSON that looks like { signature: "..." } as I'd expect, and it's definitely using the v4 algorithm to generate it. I've verified that the algorithm the PHP server's producing exactly matches what the example Node.js server produces using v4. I've further confirmed that I can upload files with the v4 signature when chunking is not used, so it appears to only be an issue when chunking is enabled with v4 signatures; since the signature works for non-chunked uploads I suspect there's a bug somewhere in the FineUploader client JS that handles the chunking algorithm.

I've looked at the V4 test suite but I'm having trouble understanding how the different stages in the test suite interact with the requests FineUploader makes to S3 when chunking is enabled.

[bagr001]

I have conducted some research about this and found out folowing. The whole problem is when using S3 endpoint with bucket in URL - eg. https://s3.amazonaws.com/mybucket. Root problem is with function getCanonicalUri.

getCanonicalUri: function(endOfUri) {
	var path = endOfUri, queryParamIdx = endOfUri.indexOf("?");
	if (queryParamIdx > 0) {
		path = endOfUri.substr(0, queryParamIdx);
	}
	return "/" + path;
},

This function is returning only canonical path to the requested object but without the bucket name (/file.txt except /mybucket/file.txt). This is resulting in wrong signig request that differs from actual request to S3.

If you modify the code to this, it works.

getCanonicalRequest: function(signatureSpec) {
	return qq.format("{}\n{}\n{}\n{}\n{}\n{}", signatureSpec.method, v4.getCanonicalUri(signatureSpec.bucket, signatureSpec.endOfUrl), v4.getCanonicalQueryString(signatureSpec.endOfUrl), signatureSpec.headersStr || "\n", v4.getSignedHeaders(signatureSpec.headerNames), signatureSpec.hashedContent);
},
getCanonicalUri: function(bucket, endOfUri) {
	var path = endOfUri, queryParamIdx = endOfUri.indexOf("?");
	if (queryParamIdx > 0) {
		path = endOfUri.substr(0, queryParamIdx);
	}
	return "/" + bucket + "/" + path;
},

But it's just a workaround. It definitely deserves a proper solution.

[rnicholus]

Can someone open up a PR with the fix? I’ll be happy to take a look and get that released in a hotfix update

[JacobEvelyn]

I think creating a fix might be a little bit beyond my technical ability. @bagr001 would you be able to?

[bagr001]

I have to make a closer look at the bloblem. But as far as i know, it will be much more complex because it affects the esential part of the library, particulary how host, bucket and canonical Uri is being determinated.

[JacobEvelyn]

@bagr001 hmm, the code you pasted above doesn't seem to work for me. Did you have to make any changes to the server signing code as well?

[bagr001]

@JacobEvelyn can you provide more info about your configuration?

• what is your setting
• what is sent to the server as a request to sign

I am using nodeJs handler...