Presigned URL for MSK #2985
-
I have an MSK cluster using IAM auth. I have successfully produced messages with a python client using the MSKAuthTokenProvider. I want to produce messages using a Ruby client. I see that Ruby isn't mentioned as one of the languages covered by the announcement. I'm working on a PR to add Oauth support to the rdkafka-ruby gem. While I'm working on that, I wanted to see if I could create a presigned url in Ruby land and then override the presigned url that is created by MSKAuthTokenProvider to prove that I'm creating a presigned url in Ruby correctly. Below is the code I'm using to create a pre-signed URL in Ruby. When I attempt to use the URL that is generated, the MSK logs show
Is this even a valid test? I'm wondering if I'm getting into encoding issues when pasting the string from Ruby into the MSKAuthTokenProvider. Ruby presign_url
|
Beta Was this translation helpful? Give feedback.
Replies: 7 comments 14 replies
-
Turns out
|
Beta Was this translation helpful? Give feedback.
-
Hello! Reopening this discussion to make it searchable. |
Beta Was this translation helpful? Give feedback.
-
I'm sorry for the late reply to this. Looking over the code, it looks like you have something that should work. I have never used MSK - without any deeper digging, I couldn't give you any more useful debugging advice. It seems like you have something that succeeds based on your answer above. Following up on this, I think this potentially makes sense as a feature in the SDK itself. We have pre-signers for S3, Polly, and some other services. Would you be willing to iterate with that on me? |
Beta Was this translation helpful? Give feedback.
-
Yes! I have a working pre-signer in ruby and this PR that adds OAuth Bearer support to rdkafka-ruby. Together, I have been able to publish messages to a multi-vpc Kafka Cluster that uses IAM Auth. I was just getting ready to create an aws-msk-iam-sasl-signer-ruby repo that published the pre-signer gem and show a working example of how to use it with the rdkafka library. Let me know how you want to proceed. |
Beta Was this translation helpful? Give feedback.
-
Do you know what happened to DefaultProvider from v1 of the aws-sdk-ruby library? It was Ruby's equivalent of "best guess" to find credentials, the same behavior that is available in Python, Go, TypeScript and .Net. Looks like DefaultProvider didn't look in all the places, only static, environment and ec2 metadata. The Python folks go all out and look in a ton of places. |
Beta Was this translation helpful? Give feedback.
-
Shall I create the repo in my org, bruce-szalwinski-he, and we can figure out how to get it to aws org later? |
Beta Was this translation helpful? Give feedback.
-
@mullermp @bruce-szalwinski-he is there any work going on for moving https://github.com/bruce-szalwinski-he/aws-msk-iam-sasl-signer-ruby to aws org? It would really help with adoption, and IAM auth support would be added to many more ruby gems. |
Beta Was this translation helpful? Give feedback.
Turns out
User-Agent
is required parameter and has to come after Operation and Authentication parameters.