Performance headache

[mergoktas]

This excerpt from stackoverflow
**
"When you make requests with "Connection: keep-alive" the subsequent request to the server will use the same TCP connection. This is called HTTP persistent connection. This helps in reduction CPU load on server side and improves latency/response time.

If a request is made with "Connection: close" this indicates that once the request has been made the server needs to close the connection. And so for each request a new TCP connection will be established.

By default HTTP 1.1 client/server uses keep-alive where as HTTP 1.0 client/server dont support keep-alive by default."
**
From your code, I understand that you force to browser to close the connection immediately.

private static void WriteConnectResponse(StreamWriter clientStreamWriter, string httpVersion) { clientStreamWriter.WriteLine(httpVersion + " 200 Connection established"); clientStreamWriter.WriteLine("Timestamp: {0}", DateTime.Now); clientStreamWriter.WriteLine("connection:close");// <-- This line force the browser disconnect. clientStreamWriter.WriteLine(); clientStreamWriter.Flush(); }

And establishing a secure connection takes too long time.
With this current version, although it runs almost perfect but the performance is really headache.
Is it because of not supporting the persistent connection or not as I am thinking or anything else?

Please reconsider the design issue of this wonderful work.
Lastly, anyhow many thanks for your effort and sharing it to community.

[justcoding121]

I am aware of random performance degradation but couldn't point at anything specific. I am looking at this.

[mergoktas]

I am interested in a site named like "www.mysite.com". So I made very small changes to HandleClient(TcpClient client) function in RequestHandler.cs as follows

if (httpVerb.ToUpper() == "CONNECT" && httpRemoteUri.Host == "www.mysite.com" && httpRemoteUri.Port == 443)
{
  //this portion of the function seems to be working fairly fast enough
  //I think, since we make authentication by ourselves, connection is faster   
}
 else if (httpVerb.ToUpper() == "CONNECT")
 {
     //remote hosts that we are  not interested just will be tunneled (like in by pass proxies)

   //this portion of the function causes long latency for establishing a secure connection.
 }

I hope this may give you a clue about performance enhancement.

[justcoding121]

I think the issue is with HttpWebRequest throttling traffic, which we have no idea how to optimize. I got rid of it in my breaking-change branch.

It haven't implemented connection cache yet, so it may not be super fast yet. But you may want to check out.

[justcoding121]

Performance has been improved in release branch.
Just released beta.

Install-Package Titanium.Web.Proxy -Pre

[mergoktas]

I tried breaking-change branch.
This is my OnResponse routine

public void OnResponse(object sender, SessionEventArgs e)
{
   if (!e.ProxySession.Request.RequestHostname.Equals("medeczane.sgk.gov.tr")) return; 
   if (e.RequestMethod == "GET" || e.RequestMethod == "POST")
   {
        if (e.ProxySession.Response.ResponseStatusCode == ((int) HttpStatusCode.OK)).ToString())
        {
            if (e.ProxySession.Response.ResponseContentType.Trim().ToLower().Contains("text/html"))
            {
                 string body = e.GetResponseBodyAsString(); //This line crashes
             }
          }
     } 
}

e.GetResponseBodyAsString() crashes in
public static byte[] DecompressGzip(Stream input) function.
That is the only modification I made to your test proxy.

GetRequestBodyAsString() could have the same problem.
Besides, for performance degradation, by pass proxy mode also should be considered.
Please observe that if the website is in excluded host list, establishing the secure connection takes too much time, longer than the not excluded ones.

[justcoding121]

Notice that API have changed in release branch on beta.
Try this on my latest change on release branch.

 if (!e.ProxySession.Request.Hostname.Equals("medeczane.sgk.gov.tr")) return;
            if (e.RequestMethod == "GET" || e.RequestMethod == "POST")
            {
                if (e.ProxySession.Response.ResponseStatusCode == "200")
                {
                    if (e.ProxySession.Response.ContentType.Trim().ToLower().Contains("text/html"))
                    {
                        string body = e.GetResponseBodyAsString(); 
                    }
                }
            }