support response stubbing for all instances of a service client

[kalpitad]

This is the follow up issue from our discussion in the comments of this blog post: http://ruby.awsblog.com/post/Tx15V81MLPR8D73/Client-Response-Stubs

My goal is to set stub_responses in my test code that will impact the DynamoDB calls in my app code, when the tests are run.

My Rspec test code sets Aws.config[:dynamodb] = {stub_responses: true} in a helper file. However, since my code calls Aws::DynamoDB::Client.new.query (i.e. I always create new clients), how can my test code call the stub_responses method on a client instance that it doesn't have access to? I would like a way to be able to stub responses for all instances of a service client (e.g. DynamoDB), so that I can affect the return values of the clients within my app code when the test is running.

Thanks!

[trevorrowe]

To support this, the SDK would need to support statically providing response stubs for a service client. For example:

Aws.config[:dynamodb] = {
  stub_responses: true,
  stubbed_responses: {
    list_tables: ...,
    get_item: ...
  }
}

This would not be very difficult, but users relying on a globally configured list of stubbed responses would need to be certain to clear this shared state between tests:

# or replace the stubs
Aws.config[:dynamodb].delete(:stubbed_responses)

Another option would be to monkey patch Aws::DynamoDB::Client.new to provide the proper stubs:

class Aws::DynamoDB::Client
  def self.new(options = {})
     client = super(options.merge(stub_responses: true))
     # don't hard-code these, but instead load them from a fixtures file perhaps
     client.stub_responses(:list_tables, table_names:['aws-sdk'])
     client
  end
end

A more expanded example might look like:

Given fixtures/dynamodb_stubs.json as:

{
  "list_tables": { "table_names": ["aws-sdk"] }
}

class Aws::DynamoDB::Client
  def self.new(options = {})
     client = super(options.merge(stub_responses: true))
     # don't hard-code these, but instead load them from a fixtures file perhaps
     stubs = MultiJson.load(File.read('fixtures/dynamodb_subs.json'))
     stubs.each do |operation_name, responses|
       client.stub_resposnes(operation_name, responses)
     end
     client
  end
end

Thoughts?

[jtrost]

I would like to see something similar to AWS.stub!. In my Rails application I create S3 buckets and Cloudfront distributions in a service object, which is called from an after_create method. Because of this setup, using the stub_responses: true option isn't practical.

Using the webmock gem, I am able to kind of accomplish this with:

before(:each) do
    stub_request(:any, /cloudfront.amazonaws.com/).to_return({distribution: {id: "ABCDEFGHIJ"}})
end

However, the returned object is coerced into an instance of Aws::PageableResponse, so I cannot access the stubbed distribution ID. I think there's a way for me to work around this, but it would be nice if stubbing requests was easier.

[kalpitad]

@trevorrowe -- my apologies for the long delay in replying. I'm not exactly clear on how the monkey patch would work.

Is the following correct?

1. Add the monkey patch to my test code
2. My test runs the monkey patch on the Aws::DynamoDB::Client class first (e.g. in a before block) and then invokes a part of my app code.
3. Using your example, if my app code then calls Aws::DynamoDB::Client.new.list_tables, the values from the stubbed responses in my monkey patch will be returned.

If this flow is correct, it would be probably work fine for me. Let me know and I'll put it on my todo list to try out. Thanks!

[trevorrowe]

@kalpitad Doing this in a before block would be a bit tricky, as the patch is persistent. This could allow for state leak between tests. The patch would need some modifications to do this one when setting up your tests (i.e. in a spec or test helper). Then it could use a shared / global list of stubs. Its not pretty, but a workable solution is possible.

@jtrost You can simulate AWS.stub! with the following:

Aws.config[:stub_responses] = true

To specify the stubs when using the resource interfaces, you have to access the client object:

# enable stubs globally
Aws.config[:stub_responses] = true

# stub responses for a single client
s3 = Aws::S3::Resource.new
s3.client.stub_responses(:list_buckets, { buckets: ['aws-sdk'] })

s3.buckets.map(&:name)
#=> ['aws-sdk']

[trevorrowe]

It sounds like it would be helpful if you could enable stubs and specify stubbed data globally as well as per instance. Would the following be helpful?

# global stubbing
Aws.config[:s3] = {
  stub_responses: true
  stubs: {
    list_buckets: { buckets: 'aws-sdk' }
  }
}

# per client or per service resource
s3 = Aws::S3::Resource.new(  
  stub_responses: true
  stubs: {
    list_buckets: { buckets: 'aws-sdk' }
  }
)

Thoughts?

[ktheory]

TIL about Aws.config[:stub_responses] = true. Yay! 🎉 That's exactly what I want in my test helpers.

(@trevorrowe: would be great if you mentioned that on your blog post about stubbing.)

[jtrost]

Thanks @trevorrowe. Aws.config[:stub_responses] = true works perfectly! Is this documented anywhere? I was looking around for this solution for a while before asking here.

[trevorrowe]

@ktheory I've updated the blog post with mention of Aws.config[:stub_responses] = true. Thanks for the suggestion.

@jtrost Is is documented in each of the client constructors, and then Aws.config is briefly documented as a set of default params applied to each client. Its very non-obvious at this point. Currently I am working on putting together a developer guide that should cover these sort of features.

[trevorrowe]

This has been resolved now as of v2.1.0. You can specify the stub responses to Aws.config.

Aws.config[:s3] = {
  stub_responses: {
    list_buckets: { buckets:[{name:'aws-sdk'}]}
  }
}

Aws::S3::Client.new.list_buckets.buckets.map(&:name)
#=> ["aws-sdk"]

I plan to blog about this and a few other of the new 2.1 features shortly.

[kalpitad]

Awesome!

[ktheory]

Yay, thanks @trevorrowe!

[kalpitad]

Hi @trevorrowe, I'm trying to stub a DynamoDB ProvisionedThroughputExceededException. After looking at the current docs and the old SDK blog, I still couldn't get this to work. What am I missing?

    Aws.config[:dynamodb] = {
      stub_responses: {
        query: { error: Aws::DynamoDB::Errors::ProvisionedThroughputExceededException}
      }
    }

I'm doing my stubbing in Rspec, so my other question is how to turn stubbing on (e.g. in a before block) and then off (e.g. in the after block), so that other tests aren't affected?

Thanks and Happy 4th!

[trevorrowe]

@kalpitad If you are stubbing a response error, you can give just the error code:

Aws.config[:dynamodb] = {
  stub_responses: {
    query: 'ProvisionedThroughputExceededException'
  }
}

To disable stubbing, simply remove the relevant option from Aws.config:

Aws.config[:dynamodb].delete(:stub_responses)

[kalpitad]

Thanks @trevorrowe!

Update: The stubbing is working. It turns out that I had a separate issue with my DynamoDB local, which was causing an error. I think I'm all good now! :)

[iDiogenes]

@trevorrowe the stub_responses works great for the client object, but I am having trouble resource objects when globally stubbing. I am trying to write a test that will stub out a S3 move_to request which is in S3::Object. Trying to add move_to into the global stub_responses throws an error during Aws::S3::Client.new, which is no surprise. Is what I am trying to do possible on a global level?

Aws.config[:s3] = {
            stub_responses: {
                list_objects: { contents: [{key: 'offers/1/img.jgp', storage_class: "STANDARD"}]},
                move_to: { delete_marker: true}
            }
        }

[trevorrowe]

@iDiogenes It would be helpful to see more context, including a stack trace on the raised error. Also, would you consider posting this as an issue instead against aws/aws-sdk-ruby?

[iDiogenes]

@trevorrowe Happy to post as in issue if you think it warrants it rather than just a mistake on my part. For context I have the following code in a ruby module

bucket = Aws::S3::Bucket.new(Rails.application.secrets.aws['bucket'])
object = bucket.object(s3_obj.key)
object.move_to(bucket.object(s3_obj.key.split('archive/').last), acl: 'public-read')

I would like to stub out the response of object.move_to for testing purposes. When I try and add it to Aws.config[:s3] as previously shown, I receive the following error when: Aws::S3::Bucket.new(Rails.application.secrets.aws['bucket']) is run.

ArgumentError: unknown operation :move_to

[trevorrowe]

@iDiogenes Oh, I see the issue. There is no client operation call #move_to that you can stub. The Aws::S3::Object#move_to method actually calls multiple operations, #copy_object followed by a #delete_object.

Do you need a specific response from the object.move_to call? I suspect not, so you can probably get away with simply enabling response stubbing, without specific responses.

Aws.config[:s3] = { stub_responses: true }

[iDiogenes]

@trevorrowe A specific response would be helpful. Currently the stub shows a delete_marker of false, which is the response for when the delete part of the #move_to fails. Therefore, with the stub I can only test what happens during a failed attempt and not the successful ones. Another example would be testing the response of

Aws::S3::Object.new(bucket_name: Rails.application.secrets.aws['bucket'], key: s3_obj.key).restore

In practice it returns a string of: "ongoing-request="true"", "ongoing-request="false"" or nil. When using

Aws.config[:s3] = { stub_responses: true }

The response is "Restore", which is not useful for testing code logic that depends on the result of #restore

[trevorrowe]

@iDiogenes The default stubbed responses are really just placeholders. If you prefer to specify the values for the response, you need to configure them via :stub_responses, or you can use your own testing framework to return stubbed responses. This can be useful if you only care the stub the final operation. For example:

# example using rspec
resp = obj.client.stub_data(:delete_object,  version_id:'id', delete_marker: true)
allow(obj).to recieve(:restore).and_return(resp)

You can use any instance of any service client and call #stub_data. This will return the structured data object for the named operation. If you pass a hash, it will merged with the placeholder values. It will also validate the structure to ensure you have no typos.

[iDiogenes]

"If you prefer to specify the values for the response, you need to configure them via :stub_responses" Correct, that is exactly what I am trying to do and where my issue is happening. The #stub_data looks perfect, but it does not seem to work when setting in the Aws.config

Aws.config[:s3] = {
            stub_responses: {
                list_objects: { contents: [{key: 'archive/offers/1/img.jgp', storage_class: "GLACIER"}]},
            },
            stub_data: {
                delete_object: { delete_marker: true}
            }
        }

Throws ArgumentError: invalid configuration option:stub_data'`

The issue I am having is the same as the individual who created this thread. "how can my test code call the stub_responses method on a client instance that it doesn't have access to? I would like a way to be able to stub responses for all instances of a service client (e.g. DynamoDB), so that I can affect the return values of the clients within my app code when the test is running." My client is being initiated within the method that needs to be tested so I can't send in the object with the stub response set, it needs to be set globally.

[rocifier]

Hi, how can I use the above technique to stub the responses of an S3::Bucket object in production code? Specifically I am trying to mock the response for #objects(prefix). Am I correct in seeing that you can only stub methods of S3::Client ??