Skip to content

puzza007/katipo

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

katipo

An HTTP/HTTP2 client library for Erlang built around libcurl-multi and libevent.

Status

build status Hex pm Hex Docs

Usage

{ok, _} = application:ensure_all_started(katipo).
Pool = api_server,
{ok, _} = katipo_pool:start(Pool, 2, [{pipelining, multiplex}]).
Url = <<"https://example.com">>.
ReqHeaders = [{<<"User-Agent">>, <<"katipo">>}].
Opts = #{headers => ReqHeaders,
         body => <<"0d5cb3c25b0c5678d5297efa448e1938">>,
         connecttimeout_ms => 5000,
         proxy => <<"http://127.0.0.1:9000">>,
         ssl_verifyhost => false,
         ssl_verifypeer => false},
{ok, #{status := 200,
       headers := RespHeaders,
       cookiejar := CookieJar,
       body := RespBody}} = katipo:post(Pool, Url, Opts).

Or passing the entire request as a map

{ok, _} = application:ensure_all_started(katipo).
Pool = api_server,
{ok, _} = katipo_pool:start(Pool, 2, [{pipelining, multiplex}]).
ReqHeaders = [{<<"User-Agent">>, <<"katipo">>}].
Req = #{url => <<"https://example.com">>.
        method => post,
        headers => ReqHeaders,
        body => <<"0d5cb3c25b0c5678d5297efa448e1938">>,
        connecttimeout_ms => 5000,
        proxy => <<"http://127.0.0.1:9000">>,
        ssl_verifyhost => false,
        ssl_verifypeer => false},
{ok, #{status := 200,
       headers := RespHeaders,
       cookiejar := CookieJar,
       body := RespBody}} = katipo:req(Pool, Req).

Why

We wanted a compatible and high-performance HTTP client so took advantage of the 15+ years of development that has gone into libcurl. To allow large numbers of simultaneous connections libevent is used along with the libcurl-multi interface.

Documentation

API

-type method() :: get | post | put | head | options.
katipo_pool:start(Name :: atom(), size :: pos_integer(), PoolOptions :: proplist()).
katipo_pool:stop(Name :: atom()).

katipo:req(Pool :: atom(), Req :: map()).
katipo:Method(Pool :: atom(), URL :: binary()).
katipo:Method(Pool :: atom(), URL :: binary(), ReqOptions :: map()).

Application Config

Option Values Default Notes
mod_metrics folsom | exometer | noop noop see erlang-metrics

Request options

Option Type Default Notes
headers [{binary(), iodata()}] []
cookiejar opaque (returned in response) []
body iodata() <<>>
connecttimeout_ms pos_integer() 30000 docs
followlocation boolean() false docs
ssl_verifyhost boolean() true docs
ssl_verifypeer boolean() true docs
capath binary() undefined
cacert binary() undefined
timeout_ms pos_integer() 30000
maxredirs non_neg_integer() 9
proxy binary() undefined docs
return_metrics boolean() false
tcp_fastopen boolean() false docs curl >= 7.49.0
interface binary() undefined docs
unix_socket_path binary() undefined docs curl >= 7.40.0
lock_data_ssl_session boolean() false docs curl >= 7.23.0
doh_url binary() undefined docs curl >= 7.62.0
http_version curl_http_version_none
curl_http_version_1_0
curl_http_version_1_1
curl_http_version_2_0
curl_http_version_2tls
curl_http_version_2_prior_knowledge
curl_http_version_none docs curl >= 7.62.0
sslcert binary() undefined docs
sslkey binary() undefined docs
sslkey_blob binary() (DER format) undefined docs curl >= 7.71.0
keypasswd binary() undefined docs
http_auth basic
digest
ntlm
negotiate
undefined docs
userpwd binary() undefined docs

Responses

{ok, #{status := pos_integer(),
       headers := headers(),
       cookiejar := cookiejar(),
       body := body(),
       metrics => proplist()}}

{error, #{code := atom(), message := binary()}}

Pool Options

Option Type Default Note
pipelining nothing
http1
multiplex
nothing HTTP pipelining CURLMOPT_PIPELINING
max_pipeline_length non_neg_integer() 100
max_total_connections non_neg_integer() 0 (no limit) docs

Metrics

  • ok
  • error
  • status.XXX
  • total_time
  • curl_time
  • namelookup_time
  • connect_time
  • appconnect_time
  • pretransfer_time
  • redirect_time
  • starttransfer_time

System dependencies

  • libevent-dev
  • libcurl4-openssl-dev
  • make
  • curl
  • libssl-dev
  • gcc

Testing

The official Erlang Docker image has everything needed to build and test Katipo

TODO

  • A more structured way to ifdef features based on curl version