Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

read timeout request option (part 2) #1022

Merged
merged 22 commits into from
Oct 17, 2019
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
2597943
Add support for setting Ruby Net::HTTP `read_timeout` option separately
springerigor Jul 18, 2019
3575ad6
Merge branch 'read-timeout-request-option' of https://github.com/spri…
technoweenie Sep 19, 2019
956c53f
space things out a bit
technoweenie Sep 19, 2019
dbe4fae
teach Faraday::RequestOptions to get the correct timeout configuration
technoweenie Sep 19, 2019
abade0d
test current EMHttp timeout config
technoweenie Sep 19, 2019
24cf49c
teach EMHttp adapter to use #fetch_timeout
technoweenie Sep 19, 2019
6c117f4
test current Excon timeout config
technoweenie Sep 19, 2019
a5a398c
teach Excon to use #fetch_timeout
technoweenie Sep 19, 2019
8dae875
lint
technoweenie Sep 19, 2019
b35f6a2
RequestOptions#fetch_timeout => Adapter#request_timeout
technoweenie Sep 19, 2019
55c0b74
test current httpclient timeout behavior
technoweenie Oct 14, 2019
711c257
teach httpclient adapter how to use #request_timeout
technoweenie Oct 14, 2019
6394bc0
teach patron adapter how to use #request_timeout
technoweenie Oct 14, 2019
0aea513
teach rack adapter to use #request_timeout
technoweenie Oct 14, 2019
b85589c
Merge branch 'master' into springerigor-read-timeout-request-option
technoweenie Oct 14, 2019
18c204e
lint fixes
technoweenie Oct 14, 2019
75b9804
unused
technoweenie Oct 14, 2019
296f336
Update lib/faraday/adapter/net_http.rb
technoweenie Oct 15, 2019
a91581a
Update lib/faraday/adapter/net_http.rb
technoweenie Oct 15, 2019
9de70e0
Merge branch 'master' into springerigor-read-timeout-request-option
iMacTia Oct 17, 2019
48e6c3f
Merge branch 'master' into springerigor-read-timeout-request-option
iMacTia Oct 17, 2019
68c53c4
prefer Hash#fetch
technoweenie Oct 17, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions lib/faraday/adapter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -65,5 +65,31 @@ def save_response(env, status, body, headers = nil, reason_phrase = nil)
env.response.finish(env) unless env.parallel?
env.response
end

# Fetches either a read, write, or open timeout setting. Defaults to the
# :timeout value if a more specific one is not given.
#
# @param type [Symbol] Describes which timeout setting to get: :read,
# :write, or :open.
# @param options [Hash] Hash containing Symbol keys like :timeout,
# :read_timeout, :write_timeout, :open_timeout, or
# :timeout
#
# @return [Integer, nil] Timeout duration in seconds, or nil if no timeout
# has been set.
def request_timeout(type, options)
key = TIMEOUT_KEYS[type]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could use a key = TIMEOUT_KEYS.fetch(type) do ... end and raise the error in there?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(We can do that in a follow-up PR, no biggie.)

if key.nil?
msg = "Expected :read, :write, :open. Got #{type.inspect} :("
raise ArgumentError, msg
end
options[key] || options[:timeout]
end

TIMEOUT_KEYS = {
read: :read_timeout,
open: :open_timeout,
write: :write_timeout
}.freeze
end
end
7 changes: 3 additions & 4 deletions lib/faraday/adapter/em_http.rb
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,9 @@ def configure_ssl(options, env)

# Reads out timeout settings from env into options
def configure_timeout(options, env)
timeout, open_timeout = request_options(env)
.values_at(:timeout, :open_timeout)
options[:connect_timeout] = options[:inactivity_timeout] = timeout
options[:connect_timeout] = open_timeout if open_timeout
req = request_options(env)
options[:inactivity_timeout] = request_timeout(:read, req)
olleolleolle marked this conversation as resolved.
Show resolved Hide resolved
options[:connect_timeout] = request_timeout(:open, req)
end

# Reads out compression header settings from env into options
Expand Down
16 changes: 8 additions & 8 deletions lib/faraday/adapter/excon.rb
Original file line number Diff line number Diff line change
Expand Up @@ -95,17 +95,17 @@ def amend_opts_with_ssl!(opts, ssl)
end

def amend_opts_with_timeouts!(opts, req)
timeout = req[:timeout]
return unless timeout
if (sec = request_timeout(:read, req))
opts[:read_timeout] = sec
end

opts[:read_timeout] = timeout
opts[:connect_timeout] = timeout
opts[:write_timeout] = timeout
if (sec = request_timeout(:write, req))
opts[:write_timeout] = sec
end

open_timeout = req[:open_timeout]
return unless open_timeout
return unless (sec = request_timeout(:open, req))

opts[:connect_timeout] = open_timeout
opts[:connect_timeout] = sec
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rubocop forced me to turn this into a guard check ^^^

end

def amend_opts_with_proxy_settings!(opts, req)
Expand Down
20 changes: 9 additions & 11 deletions lib/faraday/adapter/httpclient.rb
Original file line number Diff line number Diff line change
Expand Up @@ -101,19 +101,17 @@ def configure_ssl(ssl)

# @param req [Hash]
def configure_timeouts(req)
configure_timeout(req) if req[:timeout]
configure_open_timeout(req) if req[:open_timeout]
end
if (sec = request_timeout(:open, req))
client.connect_timeout = sec
end

def configure_timeout(req)
client.connect_timeout = req[:timeout]
client.receive_timeout = req[:timeout]
client.send_timeout = req[:timeout]
end
if (sec = request_timeout(:write, req))
client.send_timeout = sec
end

return unless (sec = request_timeout(:read, req))

def configure_open_timeout(req)
client.connect_timeout = req[:open_timeout]
client.send_timeout = req[:open_timeout]
client.receive_timeout = sec
end

def configure_client
Expand Down
18 changes: 9 additions & 9 deletions lib/faraday/adapter/net_http.rb
Original file line number Diff line number Diff line change
Expand Up @@ -165,17 +165,17 @@ def configure_ssl(http, ssl)
end

def configure_request(http, req)
if req[:timeout]
http.read_timeout = req[:timeout]
http.open_timeout = req[:timeout]
if http.respond_to?(:write_timeout=)
http.write_timeout = req[:timeout]
end
if (sec = request_timeout(:read, req))
http.read_timeout = sec
end
if (sec = http.respond_to?(:write_timeout=) &&
technoweenie marked this conversation as resolved.
Show resolved Hide resolved
request_timeout(:write, req))
http.write_timeout = sec
end
http.open_timeout = req[:open_timeout] if req[:open_timeout]
if req[:write_timeout] && http.respond_to?(:write_timeout=)
http.write_timeout = req[:write_timeout]
if (sec = request_timeout(:open, req))
technoweenie marked this conversation as resolved.
Show resolved Hide resolved
http.open_timeout = sec
end

# Only set if Net::Http supports it, since Ruby 2.5.
http.max_retries = 0 if http.respond_to?(:max_retries=)

Expand Down
38 changes: 25 additions & 13 deletions lib/faraday/adapter/patron.rb
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,8 @@ def call(env)
end

if (req = env[:request])
if req[:timeout]
session.timeout = session.connect_timeout = req[:timeout]
end
session.connect_timeout = req[:open_timeout] if req[:open_timeout]

if (proxy = req[:proxy])
proxy_uri = proxy[:uri].dup
proxy_uri.user = proxy[:user] &&
Utils.escape(proxy[:user]).gsub('+', '%20')
proxy_uri.password = proxy[:password] &&
Utils.escape(proxy[:password]).gsub('+', '%20')
session.proxy = proxy_uri.to_s
end
configure_timeouts(session, req)
configure_proxy(session, req[:proxy])
end

response = begin
Expand Down Expand Up @@ -91,6 +80,29 @@ def configure_ssl(session, ssl)
end
end

def configure_timeouts(session, req)
return unless req

if (sec = request_timeout(:read, req))
session.timeout = sec
end

return unless (sec = request_timeout(:open, req))

session.connect_timeout = sec
end

def configure_proxy(session, proxy)
return unless proxy

proxy_uri = proxy[:uri].dup
proxy_uri.user = proxy[:user] &&
Utils.escape(proxy[:user]).gsub('+', '%20')
proxy_uri.password = proxy[:password] &&
Utils.escape(proxy[:password]).gsub('+', '%20')
session.proxy = proxy_uri.to_s
end

private

CURL_TIMEOUT_MESSAGES = [
Expand Down
3 changes: 2 additions & 1 deletion lib/faraday/adapter/rack.rb
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ def call(env)
rack_env[name] = value
end

timeout = env[:request][:timeout] || env[:request][:open_timeout]
timeout = request_timeout(:open, env[:request])
timeout ||= request_timeout(:read, env[:request])
response = if timeout
Timer.timeout(timeout, Faraday::TimeoutError) do
execute_request(env, rack_env)
Expand Down
5 changes: 3 additions & 2 deletions lib/faraday/options/request_options.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@
module Faraday
# RequestOptions contains the configurable properties for a Faraday request.
class RequestOptions < Options.new(:params_encoder, :proxy, :bind,
:timeout, :open_timeout, :write_timeout,
:boundary, :oauth, :context, :on_data)
:timeout, :open_timeout, :read_timeout,
:write_timeout, :boundary, :oauth,
:context, :on_data)

def []=(key, value)
if key && key.to_sym == :proxy
Expand Down
31 changes: 31 additions & 0 deletions spec/faraday/adapter/em_http_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,35 @@

expect(req.connopts.inactivity_timeout).to eq(20)
end

context 'Options' do
let(:request) { Faraday::RequestOptions.new }
let(:env) { { request: request } }
let(:options) { {} }
let(:adapter) { Faraday::Adapter::EMHttp.new }

it 'configures timeout' do
request.timeout = 5
adapter.configure_timeout(options, env)
expect(options[:inactivity_timeout]).to eq(5)
expect(options[:connect_timeout]).to eq(5)
end

it 'configures timeout and open_timeout' do
request.timeout = 5
request.open_timeout = 1
adapter.configure_timeout(options, env)
expect(options[:inactivity_timeout]).to eq(5)
expect(options[:connect_timeout]).to eq(1)
end

it 'configures all timeout settings' do
request.timeout = 5
request.read_timeout = 3
request.open_timeout = 1
adapter.configure_timeout(options, env)
expect(options[:inactivity_timeout]).to eq(3)
expect(options[:connect_timeout]).to eq(1)
end
end
end
32 changes: 32 additions & 0 deletions spec/faraday/adapter/excon_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,36 @@

expect(conn.data[:debug_request]).to be_truthy
end

context 'config' do
let(:adapter) { Faraday::Adapter::Excon.new }
let(:request) { Faraday::RequestOptions.new }
let(:uri) { URI.parse('https://example.com') }
let(:env) { { request: request, url: uri } }

it 'sets timeout' do
request.timeout = 5
options = adapter.send(:opts_from_env, env)
expect(options[:read_timeout]).to eq(5)
expect(options[:write_timeout]).to eq(5)
expect(options[:connect_timeout]).to eq(5)
end

it 'sets timeout and open_timeout' do
request.timeout = 5
request.open_timeout = 3
options = adapter.send(:opts_from_env, env)
expect(options[:read_timeout]).to eq(5)
expect(options[:write_timeout]).to eq(5)
expect(options[:connect_timeout]).to eq(3)
end

it 'sets open_timeout' do
request.open_timeout = 3
options = adapter.send(:opts_from_env, env)
expect(options[:read_timeout]).to eq(nil)
expect(options[:write_timeout]).to eq(nil)
expect(options[:connect_timeout]).to eq(3)
end
end
end
54 changes: 54 additions & 0 deletions spec/faraday/adapter/httpclient_spec.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
# frozen_string_literal: true

RSpec.describe Faraday::Adapter::HTTPClient do
# ruby gem defaults for testing purposes
HTTPCLIENT_OPEN = 60
HTTPCLIENT_READ = 60
HTTPCLIENT_WRITE = 120

features :request_body_on_query_methods, :reason_phrase_parse, :compression,
:trace_method, :connect_method, :local_socket_binding

Expand All @@ -18,4 +23,53 @@
expect(client.keep_alive_timeout).to eq(20)
expect(client.ssl_config.timeout).to eq(25)
end

context 'Options' do
let(:request) { Faraday::RequestOptions.new }
let(:env) { { request: request } }
let(:options) { {} }
let(:adapter) { Faraday::Adapter::HTTPClient.new }
let(:client) { adapter.client }

it 'configures timeout' do
assert_default_timeouts!

request.timeout = 5
adapter.configure_timeouts(request)

expect(client.connect_timeout).to eq(5)
expect(client.send_timeout).to eq(5)
expect(client.receive_timeout).to eq(5)
end

it 'configures open timeout' do
assert_default_timeouts!

request.open_timeout = 1
adapter.configure_timeouts(request)

expect(client.connect_timeout).to eq(1)
expect(client.send_timeout).to eq(HTTPCLIENT_WRITE)
expect(client.receive_timeout).to eq(HTTPCLIENT_READ)
end

it 'configures multiple timeouts' do
assert_default_timeouts!

request.open_timeout = 1
request.write_timeout = 10
request.read_timeout = 5
adapter.configure_timeouts(request)

expect(client.connect_timeout).to eq(1)
expect(client.send_timeout).to eq(10)
expect(client.receive_timeout).to eq(5)
end

def assert_default_timeouts!
expect(client.connect_timeout).to eq(HTTPCLIENT_OPEN)
expect(client.send_timeout).to eq(HTTPCLIENT_WRITE)
expect(client.receive_timeout).to eq(HTTPCLIENT_READ)
end
end
end
14 changes: 14 additions & 0 deletions spec/faraday/adapter/net_http_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,31 @@
let(:http) { adapter.send(:net_http_connection, url: url, request: {}) }

it { expect(http.port).to eq(80) }

it 'sets max_retries to 0' do
adapter.send(:configure_request, http, {})

expect(http.max_retries).to eq(0) if http.respond_to?(:max_retries=)
end

olleolleolle marked this conversation as resolved.
Show resolved Hide resolved
it 'supports write_timeout' do
adapter.send(:configure_request, http, write_timeout: 10)

expect(http.write_timeout).to eq(10) if http.respond_to?(:write_timeout=)
end

it 'supports open_timeout' do
adapter.send(:configure_request, http, open_timeout: 10)

expect(http.open_timeout).to eq(10)
end

it 'supports read_timeout' do
adapter.send(:configure_request, http, read_timeout: 10)

expect(http.read_timeout).to eq(10)
end

context 'with https url' do
let(:url) { URI('https://example.com') }

Expand Down
Loading