Forcing logger to log requests and responses before retries exhausted

[viralpraxis]

Requests made with Aws::S3::Client are being logged as soon as it receives successful response or retries limit exceeded. However, I'd like to make the client to log every request (including failed retries) for debug purposes.

Could you please provide correct way to implement it?

I've tried to search in source code and GH discussions but failed

Thanks!

[mullermp]

I believe this is supported by configuring a log formatter on the client. The default log formatter should be sufficient. https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Log/Formatter.html#default-class_method

[viralpraxis]

Using Aws::Log::Formatter.default as the client log formatter doesn't log intermediate requests, only the final one. Also, AFAIK Aws::Log::Formatter doesn't provide an option to configure this behaviour, unfortunately

[mullermp]

Ah, yeah, you're correct. It's wrapped very early in the handler stack here:

aws-sdk-ruby/gems/aws-sdk-core/lib/aws-sdk-core/plugins/logging.rb

Lines 39 to 45 in bef4299

	def call(context)
	context[:logging_started_at] = Time.now
	@handler.call(context).tap do |response|
	context[:logging_completed_at] = Time.now
	log(context.config, response)
	end
	end

I don't see a good use case to support this in the Log formatter since it doesn't provide much value. Since you're doing this for debug purposes, I think http_wire_trace is what you're looking for, and I would not recommend it in production.

[viralpraxis]

Got it, thanks 👍🏻

[alextwoods]

Are you looking for wire logging of request/response? If so - you can enable that by configuring the client with: http_wire_trace: true.

[viralpraxis]

ATM my logfile contains records like this

I, [2023-06-29T15:43:18.073330 #831901]  INFO -- : [489c8ff1-b951-481f-94fc-55cada26882f] [Aws::S3::Client 0 613.850965 10 retries] delete_objects(...)

please note "10 retries" message. As far as I can see intermediate requests with corresponding errors (for 1st, 2nd, ... 9th retry) are not logged.

If http_wire_trace option logs all the intermediate requests then I believe that'd be a solution (seems like logs will be too verbose but I guess that's still OK.

[mullermp]

Wire trace is the HTTP debug option for Net::HTTP. It's not suitable for production because it logs credentials.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.