httpclient - HTTP accessing library.
Copyright (C) 2000-2009  NAKAMURA, Hiroshi  <[email protected]>.

'httpclient' gives something like the functionality of libwww-perl (LWP) in
Ruby.  'httpclient' formerly known as 'http-access2'.

See HTTPClient for documentation.


== Features

* methods like GET/HEAD/POST/* via HTTP/1.1.
* HTTPS(SSL), Cookies, proxy, authentication(Digest, NTLM, Basic), etc.
* asynchronous HTTP request, streaming HTTP request.

* by contrast with net/http in standard distribution;
  * Cookies support
  * MT-safe
  * streaming POST (POST with File/IO)
  * Digest auth
  * Negotiate/NTLM auth for WWW-Authenticate (requires net/htlm module)
  * NTLM auth for Proxy-Authenticate (requires win32/sspi module)
  * extensible with filter interface
  * you don't have to care HTTP/1.1 persistent connection
    (httpclient cares instead of you)

* Not supported now
  * Cache
  * Rather advanced HTTP/1.1 usage such as Range, deflate, etc.
    (of course you can set it in header by yourself)


== Author

Name:: NAKAMURA, Hiroshi
E-mail:: [email protected]
Project web site:: http://dev.ctor.org/httpclient/


== License

This program is copyrighted free software by NAKAMURA, Hiroshi.  You can
redistribute it and/or modify it under the same terms of Ruby's license;
either the dual license version in 2003, or any later version.

httpclient/session.rb is based on http-access.rb in http-access/0.0.4.
Some part of code in http-access.rb was recycled in http-access2.rb.
Those part is copyrighted by Maehashi-san who made and distributed
http-access/0.0.4. Many thanks to Maehashi-san.


== Install

  $ ruby install.rb

It will install lib/* to your site_ruby directory such as
/usr/local/lib/ruby/site_ruby/1.8/.


== Uninstall

Delete installed files from your site_ruby directory.


== Usage

See HTTPClient for documentation.
You can also check sample/howto.rb how to use APIs.


== Download

* Stable: http://dev.ctor.org/download/httpclient-2.1.5.2.tar.gz (tar + gzip)
* Stable: http://dev.ctor.org/download/httpclient-2.1.5.2.zip (ZIP) 

* Older versions: http://dev.ctor.org/download/archive/ 

* Gem repository for stable version
  * (at default remove source at rubyforge.org) 
* Gem repository for development version
  * http://dev.ctor.org/download/ 

* svn: http://dev.ctor.org/svn/http-access2/trunk/ 

=== Gem

You can install httpclient with rubygems.

  % gem install httpclient --source http://dev.ctor.org/download/


== Bug report or Feature request

Please file a ticket at the project web site.

1. find a similar ticket from http://dev.ctor.org/http-access2/search?ticket=on
2. login as an anonymous user from http://dev.ctor.org/http-access2/login
3. create a new ticket from http://dev.ctor.org/http-access2/newticket 

Thanks in advance. 


== Changes


= Changes in 2.1.5 =

  Jun 25, 2009 - version 2.1.5.2

    * Added another cacert distribution certificate which uses
      sha1WithRSAEncryption.  OpenSSL/0.9.7 cannot handle non-SHA1 digest
      algorithm for certificate.  The new certificate is
      RSA 2048 bit + SHA1 + notAfter:2010/12/31.  Corresponding CA bundle file
      is cacert_sha1.p7s.  It is loaded only when cacert.p7s cannot be loaded
      with the original distribution certificate.

  Jun 11, 2009 - version 2.1.5.1

    * README update.

  Jun 8, 2009 - version 2.1.5

    * IMPORTANT update for HTTPS(SSL) connection
      * Trusted CA bundle file included in httpclient <= 2.1.4 expires in
        Nov 2009. Please update to 2.1.5 by Oct 2009 if your application
        depends on trusted CA bundle file.
      * Updated trusted CA certificates file (cacert.p7s). CA certs are
        imported from 'Java(TM) SE Runtime Environment (build 1.6.0_13-b03)'. 
      * Updated a cacert distribution certificate.
        RSA 2048 bit + SHA512 + notAfter:2037/12/31. (#215)

    * Feature
      * WWW authentication with Negotiate based on win32/sspi as same as Proxy
        authentication. Applied a patch from Paul Casto. Thanks! (#212)

    * Bug fixes
      * Infinite loop caused by EOF error while reading response message body
        without Content-Length.  IO#readpartial does not clear the second
        argument (buffer) when an exception raised.  Fixed by a patch from an
        user.  Thanks! (#216)
      * NoMethodError caused by the cookie string that includes a double
        semicolons ";;".  Fixed by a patch from an user.  Thanks! (#211)
      * CNONCE attribute in Digest Authentication was not properly generated by
        itself (used same nonce sent from the connecting server). Fixed by a
        patch from bterlson
        [https://github.com/bterlson/httpclient/commit/6d0df734840985a7be88a2d54443bbf892d50b9a]
        Thanks! (#209)
      * Cookie header was not set in authentication negotiation. Fixed. This
        bug was found and pointed out by bterlson at
        [https://github.com/bterlson/httpclient/commits/master]. Thanks! (#210)
      * Do not send 'Content-Length: 0' when a request doesn't have message
        body. Some server application (!EasySoap++/0.6 for example) corrupts
        with the request with Content-Length: 0. This bug was found by clay
        [http://daemons.net/~clay/2009/05/03/ruby-why-do-you-torment-me/].
        Thanks! (#217)
      * Ensure to reset connection after invoking HTTPClient singleton methods
        for accessing such as HTTPClient.get_content. Thanks to @xgavin! (#214)

  Feb 13, 2009 - version 2.1.4

    * Bug fixes
      * When we hit some site through http-proxy we get a response without
        Content-Length header.  httpclient/2.1.3 drops response body for such
        case. fixed. (#199)
      * Avoid duplicated 'Date' header in request. Fixed. (#194)
      * Avoid to add port number to 'Host' header.  Some servers like GFE/1.3
        dislike it. Thanks to anonymous user for investigating the behavior.
        (#195)
      * httpclient/2.1.3 does not work when you fork a process after requiring
        httpclient module (Passenger). Thanks to Akira Yamada for tracing down
        this bug. (#197)
      * httpclient/2.1.3 cannot handle Cookie header with 'expires=' and
        'expires=""'.  Empty String for Time.parse returns Time.now unlike
        ParseDate.parsedate. Thanks to Mark for the patch. (#200) 

  Jan 8, 2009 - version 2.1.3.1

    * Security fix introduced at 2.1.3.
      * get_content/post_content of httpclient/2.1.3 may send secure cookies
        for a https site to non-secure (non-https) site when the https site
        redirects the request to a non-https site.  httpclient/2.1.3 caches
        request object and reuses it for redirection.  It should not be cached
        and recreated for each time as httpclient <= 2.1.2 and http-access2.
      * I realized this bug when I was reading open-uri story on
        [ruby-core:21205].  Ruby users should use open-uri rather than using
        net/http directly wherever possible.

  Dec 29, 2008 - version 2.1.3

    * Features
      * Proxy Authentication for SSL.
      * Performance improvements.
      * Full RDoc. Please tell me any English problem. Thanks in advance.
      * Do multipart file upload when a given body includes a File. You don't
        need to set 'Content-Type' and boundary String any more.
      * Added propfind and proppatch methods. 

    * Changes
      * Avoid unnecessary memory consuming for get_content/post_content with
        block.  get_content returns nil when you call it with a block.
      * post_content with IO did not work when redirect/auth cycle is required.
        (CAUTION: post_content now correctly follows redirection and posts the
        given content)
      * Exception handling cleanups.
        * Raises HTTPClient::ConfigurationError? for environment problem.
          (trying to do SSL without openssl installed for example)
        * Raises HTTPClient::BadResponse? for HTTP response problem.  You can
          get the response HTTPMessage returned via $!.res.
        * Raises SocketError? for connection problem (as same as before). 

    * Bug fixes
      * Avoid unnecessary negotiation cycle for Negotiate(NTLM) authentication.
        Thanks Rishav for great support for debugging Negotiate authentication.
      * get_content/post_content with block yielded unexpected message body
        during redirect/auth cycle.
      * Relative URI redirection should be allowed from 2.1.2 but it did not
        work... fixed.
      * Avoid unnecessary timeout waiting when no message body returned such as
        '204 No Content' for DAV.
      * Avoid blocking on socket closing when the socket is already closed by
        foreign host and the client runs under MT-condition. 

  Sep 22, 2007 - version 2.1.2

    * HTTP
      * implemented Negotiate authentication with a support from exterior
        modules. 'rubyntlm' module is required for Negotiate auth with IIS.
        'win32/sspi' module is required for Negotiate auth with ISA.
      * a workaround for Ubuntu + SonicWALL timeout problem. try to send HTTP
        request in one chunk.

    * SSL
      * create new self-signing dist-cert which has serial number 0x01 and
        embed it in httpclient.rb.
      * update cacert.p7s. certificates are imported from cacerts in JRE 6
        Update 2. 1 expired CA certificate
        'C=US, O=GTE Corporation, CN=GTE CyberTrust Root' is removed.

    * Bug fix
      * [BUG] SSL + debug_dev didn't work under version 2.1.1.
      * [BUG] Reason-Phrase of HTTP response status line can be empty according
      * to RFC2616.

  Aug 28, 2007 - version 2.1.1

    * bug fix
      * domain_match should be case insensitive. thanks to Brian for the patch.
      * before calling SSLSocket#post_connection_check, check if
        RUBY_VERSION > "1.8.4" for CN based wildcard certificate. when
        RUBY_VERSION <= "1.8.4",  it fallbacks to the post_connection_check
        method in HTTPClient so httpclient should run on 1.8.4 fine as before.

    * misc
      * added HTTPClient#test_loopback_http_response which accepts test
        loopback response which contains HTTP header. 

  Jul 14, 2007 - version 2.1.0

    * program/project renamed from 'http-access2' to 'httpclient'.
      there's compatibility layer included so existing programs for
      http-access2 which uses HTTPAccess2::Client should work with
      httpclient/2.1.0 correctly.

    * misc
      * install.rb did not install cacerts.p7s.  Thanks to knu.
      * now HTTPClient loads http_proxy/HTTP_PROXY and no_proxy/NO_PROXY
        environment variable at initialization time. bear in mind that it
        doesn't load http_proxy/HTTP_PROXY when a library is considered to be
        running under CGI environment (checked by ENVREQUEST_METHOD existence.
        cgi_http_proxy/CGI_HTTP_PROXY is loaded instead.

  Jul 4, 2007 - version 2.0.9

    * bug fix
      * fix the BasicAuth regression problem in 2.0.8.  A server may return
        "BASIC" as an authenticate scheme label instead of "Basic".  It must be
        treated as a case-insensitive token according to RFC2617 section 1.2.
        Thanks to mwedeme for contributing the patch. (#159)

  Jun 30, 2007 - version 2.0.8

    * HTTP
      * added request/response filter interface and implemented DigestAuth
        based on the filter interface.  DigestAuth calc engine is based on
        http://tools.assembla.com/breakout/wiki/DigestForSoap
        Thanks to sromano. (#155)
      * re-implemented BasicAuth based on the filter interface.  send BasicAuth
        header only if it's needed. (#31)
      * handle a response which has 2XX status code as a successfull response
        while retry check.  applied the patch from Micah Wedemeyer.
        Thanks! (#158)

    * Connection
      * show more friendly error message for unconnectable URL. (#156)

    * bug fixes
      * to avoid MIME format incompatibility, add empty epilogue chunk
        explicitly.  Thanks to the anonymous user who reported #154 (#154)
      * rescue EPIPE for keep-alive reconnecting.  Thanks to anonymous user
        who posted a patch at #124. (#124)

  May 13, 2007 - version 2.0.7

    * HTTP
      * added proxyauth support. (#6)
      * let developer allow to rescue a redirect with relative URI. (#28)
      * changed last-chunk condition statement to allow "0000\r\n" marker from
        WebLogic Server 7.0 SP5 instead of "0\r\n". (#30)
      * fixed multipart form submit. (#29, #116)
      * use http_date format as a date in a request header. (#35)
      * avoid duplicated Date header when running under mod_ruby. (#127)
      * reason phrase in Message#reason contains \r. (#122)
      * trim "\n"s in base64 encoded BasicAuth value for interoperability.
        (#149)
      * let retry_connect return a Message not a content. (#119)
      * rescue SocketError and dump a message when a wrong address given. (#152)

    * HTTP-Cookies
      * changed "domain" parameter matching condition statement to allow
        followings; (#24, #32, #118, #147)
        * [host, domain] = [rubyforge.com, .rubyforge.com]
        * [host, domain] = [reddit.com, reddit.com]

    * SSL
      * bundles CA certificates as trust anchors.
      * allow user to get peer_cert. (#117, #123)
      * added wildcard certificate support. (#151)
      * SSL + HTTP keep-alive + long wait causes uncaught exception.  fixed.
        (#120)

    * Connection
      * fixed a loop condition bug that caused intermittent empty response.
        (#150, #26, #125)

  September 16, 2005 - version 2.0.6

    * HTTP
      * allows redirects from a "POST" request.  imported a patch from sveit.
        Thanks! (#7)
      * add 'content-type: application/application/x-www-form-urlencoded' when
        a request contains message-body. (#11)
      * HTTP/0.9 support.  (#15)
      * allows submitting multipart forms.  imported a patch from sveit.
        Thanks! (#7)

    * HTTP-Cookies
      * avoid NameError when a cookie value is nil. (#10)
      * added netscape_rule property to CookieManager (false by default).  You
        can turn on the domain attribute test of Netscape rule with the
        property.  cf. http://wp.netscape.com/newsref/std/cookie_spec.html
      * added HTTPClient#cookie_manager property for accessing its properties.
        (#13)
      * added save_all_cookies method to save unused and discarded cookies as
        well.  The patch is from Christian Lademann.  Thanks!  (#21)
      * allow to set cookie_manager.  raise an error when set_cookie_store
        called and cookie_store has already been set. (#20)

    * SSL
      * allows SSL connection debugging when debug_dev != nil. (#14)
      * skip post_connection_check when
        verify_mode == OpenSSL::SSL::VERIFY_NONE.  Thanks to kdraper. (#12)
      * post_connection_check: support a certificate with a wildcard in the
        hostname. (#18)
      * avoid NameError when no peer_cert and VERIFY_FAIL_IF_NO_PEER_CERT
        given.  Thanks to Christian Lademann.

    * Connection
      * insert a connecting host and port to an exception message when
        connecting failed. (#5)
      * added socket_sync property to HTTPClient(HTTPAccess2::Client) that
        controls socket's sync property.  the default value is true.  CAUTION:
        if your ruby is older than 2005-09-06 and you want to use SSL
        connection, do not set socket_sync = false to avoid a blocking bug of
        openssl/buffering.rb.

  December 24, 2004 - version 2.0.5
    This is a minor bug fix release.
    - Connect/Send/Receive timeout cannot be configured. fixed.
    - IPSocket#addr caused SocketError? on Mac OS X 10.3.6 + ruby-1.8.1 GA.
      fixed.
    - There is a server which does not like 'foo.bar.com:80' style Host header.
      The server for http://rubyforge.org/export/rss_sfnews.php seems to
      dislike HTTP/1.1 Host header "Host: rubyforge.net:80".  It returns
      HTTP 302: Found and redirects to the page again, causes
      HTTPAccess2::Client to raise "retry count exceeded".  Keat found that the
      server likes "Host: rubyforge.net" (not with port number).

  February 11, 2004 - version 2.0.4
    - add Client#redirect_uri_callback interface.
    - refactorings and bug fixes found during negative test.
    - add SSL test.

  December 16, 2003 - version 2.0.3
    - no_proxy was broken in 2.0.2.
    - do not dump 'Host' header under protocol_version == 'HTTP/1.0'

  December ?, 2003 - version 2.0.2
    - do not trust HTTP_PROXY environment variable. set proxy server manually.
      http://ftp.ics.uci.edu/pub/websoft/libwww-perl/archive/2001h1/0072.html
      http://ftp.ics.uci.edu/pub/websoft/libwww-perl/archive/2001h1/0241.html
      http://curl.haxx.se/mail/archive-2001-12/0034.html
    - follow ossl2 change.

  October 4, 2003 - version 2.0.1
    Query was not escaped when query was given as an Array or a Hash.  Fixed.
    Do not use http_proxy defined by ENV['http_proxy'] or ENV['HTTP_PROXY'] if
      the destination host is 'localhost'.
    Hosts which matches ENV['no_proxy'] or ENV['NO_PROXY'] won't be proxyed.
      [,:] separated. ("ruby-lang.org:rubyist.net")
      No regexp. (give "ruby-lang.org", not "*.ruby-lang.org")
      If you want specify hot by IP address, give full address.
	("192.168.1.1, 192.168.1.2")

  September 10, 2003 - version 2.0
    CamelCase to non_camel_case.
    SSL support (requires Ruby/OpenSSL).
    Cookies support.  lib/http-access2/cookie.rb is redistributed file which is
      originally included in Webagent by TAKAHASHI `Maki' Masayoshi.  You can
      download the entire package from http://www.rubycolor.org/arc/.

  January 11, 2003 - version J
    ruby/1.8 support.