Request/Response vs Streaming Model

[ioquatix]

The current implementation of Async::HTTP uses a request/response model. I am interested in exploring an alternative streaming model.

Middleware Design

Body::Writable is a queue of String chunks.

Request Response Model

class Request
	attr :verb
	attr :target
	attr :body
end

class Response
	attr :status
	attr :headers
	attr :body
end

def call(request)
	return response
end

def call(request)
	return @app.call(request)
end

Stream Model

class Stream
	attr :verb
	attr :target
	
	def respond(status, headers) = ...
	
	attr_accessor :input
	attr_accessor :output
end

class Response
	def initialize(verb, target)
		@input = Body::Writable.new
		@output = Body::Writable.new
	end
	
	def request(verb, target)
		# Create a request stream suitable for writing into the buffered response:
		Stream.new(verb, target, @input, @output)
	end
	
	def write(...)
		@input.write(...)
	end
	
	def read
		@output.read
	end
end

def call(stream)
	# nothing. maybe error
end

def call(stream)
	return @app.call(stream)
end

Client Design

Request Response Model

request = Request.new("GET", url)
response = call(request)

response.headers
response.read

Stream Model

response = Response.new
call(response.request("GET", url))

response.headers
response.read

Differences

The request/response model has a symmetrical design which naturally uses the return value for the result of executing the request. The result encapsulates the behaviour of how to read the response status, headers and body. Because of that, streaming input and output becomes a function of the result object itself. As in:

def call(request)
	body = Body::Writable.new
	
	Fiber.schedule do
		while chunk = request.input.read
			body.write(chunk.reverse)
		end
	end
	
	return Response[200, [], body]
end

input = Body::Writable.new
response = call(... body ...)

input.write("Hello World")
input.close
response.read -> "dlroW olleH"

The streaming model does not have the same symmetry, and instead opts for a uni-directional flow of information.

def call(stream)
	Fiber.schedule do
		while chunk = stream.read
			stream.write(chunk.reverse)
		end
	end
end

input = Body::Writable.new
response = Response.new(...input...)
call(response.stream)

input.write("Hello World")
input.close
response.read -> "dlroW olleH"

The value of this uni-directional flow is that it is natural for the stream to be taken out of the scope imposed by the nested call(request) model. However, the user must explicitly close the stream, since it's no longer scoped to the client and/or server.

[ioquatix]

WebSocket (and similar) Connection Handling

Out of band streaming comes naturally to the stream interface since it's life time is not bound to the scope of the caller.

def call(stream)
	# Immediately upgrade the connection:
	@connections = WebSocket.upgrade(stream)
end

For the request/response model, we must return a closure which indicates that we want to begin streaming.

def call(request)
	# Generate a response which creates a connection upgrade and invokes the given block.
	return WebSocket.upgrade(request) do |connection|
		@connections << connection
	end
end

In the latter case, the server needs to have some way to signal that the connection should be streaming.

[ioquatix]

Informational Responses

Informational responses are tricky because they represent a model which is more elaborate than typical 1:1 request response models.

def call(stream)
	stream.respond(101, headers)
	# ...
	stream.respond(200, headers)
	
	stream.output.write(...)
	stream.output.close
end

Informational responses are much harder to implement ergonomically with the request-response model since we don't have an interface for more than one response.

def call(request)
	# This?
	# request.respond(101, headers)
	
	# This?
	response = Response[101, headers]
	
	response.then do
		Response[200, headers]
	end
	
	return response
end

[ioquatix]

Is the Request Response a subset of the Streaming model?

There is some value in the request response model. It's conceptually a bit simpler. But it also makes it tricky to do more advanced things with HTTP. Informational responses and efficient streaming become more tricky.

def call(stream)
	request = Request.new(stream)
	response = @app.call(request)
	response.write(stream)
end

def call(request)
	output = Body::Writable.new
	stream = Stream.new(request, output)
	
	# Hard to implement correctly considering provisional responses and streaming responses - requires server hooks (hijack):
	return Response.new(stream.status, stream.headers, output)
end

Because of this, I'm inclined to think that the streaming model is more general.