Add aws configuration in 'Shoryuken.configure_server'

[h3poteto]

I add aws configuration in Shoryuken.configure_server, like sidekiq:
https://github.com/mperham/sidekiq/blob/master/lib/sidekiq.rb#L75-L83

Now, I have to call EnvironmentLoader.load in initializer of my Rails application to setup aws configuration. For example:

EnvironmentLoader.load(config_file: "config/shoryuken.yml")

I want to divide EnvironmentLoader.load method, because I try to fix #249.

To fix #249, shoryuken have to read shoryuken.yml before daemonize, and then load Rails.
So, I want to divide EnvironmentLoader.load.

But, now EnvironmentLoader.load is used in initializer of user's Rails application, therefore I propose alternate method to configure aws in initializer. That is Shoryuken.configure_server.

In sidekiq, user can configure redis information in initializer.

Sidekiq.configure_server do |config|
  config.redis = { url: "redis://localhost:6379" }
end

Sidekiq.configure_client do |config|
 config.redis = { url: "redis://localhost:6379" }
end

I aim the same way as sidekiq.

Shoryuken.configure_server do |config|
  config.aws = { :sqs_endpoint => '...', :access_key_id: '...', :secret_access_key: '...', region: '...' }
end

Shoryuken.configure_client do |config|
  config.aws = { :sqs_endpoint => '...', :access_key_id: '...', :secret_access_key: '...', region: '...' }
end

[houndci-bot]

Unused block argument - v. If it's necessary, use _ or _v as an argument name to indicate that it won't be used.

[houndci-bot]

Use attr_writer to define trivial writer methods.

[houndci-bot]

Extra empty line detected at class body beginning.

[phstc]

Hi @h3poteto

Not sure if I could follow the change completely. Is it basically extracting the Aws.config setup? How will it fix #249?

[h3poteto]

@phstc
Thank you for your reply.

I create sample pull request in my repository: h3poteto#2
I will divide EnvironmentLoader.load to fix #249, and intercept current EnvironmentLoader.load.

But, now developers use EnvironmentLoader.load, so I prepare alternate load method which is this pull request.

[phstc]

@h3poteto is @aws being used?

[h3poteto]

Sorry, @aws is not used!

[agustin]

Hi there! what do you think of this @phstc? looks like something that you would like to merge eventually? just wondering because I was looking for a good way to decouple the general AWS config of my application from the one that Shoryuken should use.

I have 3 different users with different permissions in my app, one is for general purposes like store stuff in S3. Other is handling keys with the KMS, and was expecting to be able to have the last one to be in charge of the queues and jobs/workers, but the way is right now seems that if I passed the [:aws] key in the shoryuken.yml config file this will override the Aws.config right? or I'm maybe missed something here 😟 .

And also on the Shoryuken::Client Class there is no way to pass over a different configuration to the SQS or SNS clients. I feel that, as I said before, there is something that I'm missing here.

Let me know if this is not clear at all, and also if you rather have this conversation on an issue or in another place, I wrote it in here because this PR looks like something that will help on this same topic. Maybe I'm wrong, don't hesitate to let me know!

Thanks a lot for all the hard work on all this! 🍺

[phstc]

hi @agustin

The change in this PR is still using Aws.config which configures the Aws globally. If I understood correct, you would like to configure the SQS client in Shoryuken using a specific creds and region and for other parts of your app to use a different creds, right?

For example, using this:

sqs = Aws::SQS::Client.new(
  region: region_name,
  credentials: credentials,
  # ...
)

Instead of globally setting it Aws.config, is that right?

[agustin]

exactly @phstc! maybe if the SNS and the SQS method on the Shoryuken::Client accepts to pass the client directly as an optional argument and if not fallback to the actual way of instance it, maybe that could work. WDYT?

If you are ok with that maybe I can open a PR with that feature, let me know!
Thankss

[phstc]

@agustin actually, I think we can continue what @h3poteto started in this PR and add AwsConfig.sqsand AwsConfig.sns instead of configuring Aws.config directly. WDYT?

If you are ok with that maybe I can open a PR with that feature, let me know!

Please do!

[phstc]

@h3poteto great work on this, thanks! Added to master 🍻

@agustin ^

[h3poteto]

thank you!