set_connection_timeout() unexpected results

[Yimyom3]

I verified that the connection timeout was accurate by getting the timestamp of the remote service, here is my code:

#include <iostream>
#include "httplib.h"
using namespace std;
using namespace httplib;

long long Get_Server_Time()
{
    auto url = ("http://api.pinduoduo.com");
    Client cli(url);
    auto res = cli.Get("/api/server/_stm");
    string content = res->body;
    smatch match;
    regex_search(content, match, regex("\\d+"));
    long long result = stoll(match[0]);
    return result;
}

int main()
{
    long long T1 = Get_Server_Time();
    Client X("http://www.baidu.com:9999");
    X.set_connection_timeout(10, 0); 
    X.Get("/");
    long long T2 = Get_Server_Time();
    cout << "T1: " << T1 << endl;
    cout << "T2: " << T2 << endl;
    cout << T2 - T1 << endl;
    return 0;
}

The expected timeout is 10 seconds, but the actual result does not match the expectation.
The result of running on my arm64 win11 and macOS is

T1: 1720509275196
T2: 1720509295308
20112

T1: 1720512589129
T2: 1720512609233
20104

The results of running on the other two amd64 win11 machines are

T1:1720511647341
T2:1720511687532
40191

T1:1720512744571
T2:1720512824922
80351

None of the results were as expected,I want to know what caused this.

[freeboub]

I confirm this, the connection timeout is working on android (built with jni).
as a workaround I first try to connect to the device: (here with a 500ms of timeout)

bool Scanning::canConnect(const std::string &ip, int port) {
    int client_fd;
    if ((client_fd = socket(AF_INET, SOCK_STREAM, 0)) < 0) {
        Log("Socket creation error");
        return false;
    }
    struct sockaddr_in serv_addr;
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_port = htons(port);
    serv_addr.sin_addr.s_addr = inet_addr( ip.c_str() );

    struct timeval timeout;
    timeout.tv_sec = 0;
    timeout.tv_usec = 500000;

    if (setsockopt (client_fd, SOL_SOCKET, SO_RCVTIMEO, &timeout,
                    sizeof timeout) < 0)
        Log("setsockopt failed (1)");

    if (setsockopt (client_fd, SOL_SOCKET, SO_SNDTIMEO, &timeout,
                    sizeof timeout) < 0)
        Log("setsockopt failed (2)");

    if (connect(client_fd, (struct sockaddr*)&serv_addr,
                sizeof(serv_addr))< 0) {
        Log("Test ip: cannot connect");
        return false;
    }
    close(client_fd);
    return true;
}

[grootgordon]

I have encountered a similar case like this which occasionally happens on Windows 10, Android, and iOS in product apps that may be accessed via Wi-Fi, cellular networks, or both. It's not easy to reproduce in the local development environment.

I'm use cpp-httplib/0.10.3, and ready for bump to latest version.

I suspect that creating a socket over multiple network interfaces might lead to connection timeouts being longer than we expect.

In the create_socket function, it traverses the results; does it block when calling create socket()?

if (getaddrinfo(node, service.c_str(), &hints, &result)) {
#if defined __linux__ && !defined __ANDROID__
    res_init();
#endif
    return INVALID_SOCKET;
}

for (auto rp = result; rp; rp = rp->ai_next) {
...
}

I apologize for bringing up this closed issue again. I noticed the fix mentioned in this comment (7d86835) should help address the long timeout problem. However, I'm a little curious as to why even with a non-blocking socket setting there is still a chance of blocking during creation.

Could you please explain the deeper reasoning behind this?

Lastly, I would like to express my heartfelt gratitude once more for your incredible cross-platform http library;
it has truly been a tremendous support for us.

[yhirose]

@grootgordon, the following is the code of client socket connection.

cpp-httplib/httplib.h

Lines 3488 to 3506 in 52a18c7

	set_nonblocking(sock2, true);
	auto ret =
	::connect(sock2, ai.ai_addr, static_cast<socklen_t>(ai.ai_addrlen));
	if (ret < 0) {
	if (is_connection_error()) {
	error = Error::Connection;
	return false;
	}
	error = wait_until_socket_is_ready(sock2, connection_timeout_sec,
	connection_timeout_usec);
	if (error != Error::Success) {
	if (error == Error::ConnectionTimeout) { quit = true; }
	return false;
	}
	}
	set_nonblocking(sock2, false);

The reason why it uses non-blocking mode is to simulate 'connection timeout'. You can see the details in this stackoverflow post.

After connect system call gets called on non-blocking mode, it waits until the socket becomes ready or timeout with select or poll in wait_until_socket_is_ready. So even if it uses non-blocking mode, this function is still considered as 'blocking'.

My change at 7d86835 tries to quit from the results loop as soon as a timeout happens. In other words, If a timeout happens, create_socket won't visit the next result. I know it's not a perfect solution, but a simple solution to fix this issue.

I am not sure if this comment answers your question, but hope it helps.

[yhirose]

Lastly, I would like to express my heartfelt gratitude once more for your incredible cross-platform http library;
it has truly been a tremendous support for us.

Thank you for the kind words. Such a appreciation motivates me to keep maintaining cpp-httplib!