Thoughts on measuring server response times

[jonb2]

Hello all

As I might have mentioned, I am grading huge amounts of domains and URLs for marketing research purposes.

I am putting together a series of metrics of my own beyond those provided by the APIs I am working with. One of which I would like to include is a server response time, like a Ping, but dealing with individual pages (and their content) as well as server response times.

Remembering what both Danijel and Kevin have said about abusing the HTTP ports too and knowing that RTC has been heavily tested and benchmarked. I wondered what the most efficient way of talking to thousand of different servers without running into the problems outlined in our previous conversation. Should I just use winHTTP and no threading ?

Also, I know how to measure page download time. But what about server response - what is the best interject point with the TRTC client to measure this in this scenario?

J

[D.Tkalcec (RTC)]

Hi Jon,

RTC Client components were never designed to work with thousands of different Servers at the same time, nor have I tried using them in that scenario, so the only advice I can give you is to make your own benchmarks and find out which combination of properties would work best for your speciifc scenario.

Best Regards,
Danijel Tkalcec

[jonb2]

Thanks D.

I think I will use Indy or IPworks for this then. I am just aware of this 'hanging' open port issue you and K mentioned.

J.

[D.Tkalcec (RTC)]

The "hanging" open port issue has nothing to do with the RTC SDK. It is a result of the TCP/IP stack, which is implemented on the lowest socket API level (WinSock, WinInet, WinHTTP, BSD sockets). You will see the same, regardless of the component set you use.

And the RTC SDK gives you seamless access to all available Windows socket APIs (async WinSock, blocking WinSock, WinInet and WinHTTP), as well as single- and multi-threaded operation modes. So ... if you ask me, your safest "bet" is still to use the RTC SDK, but in order to find out which combination of RTC properties work best for your specific scenario, you will need to run your own benchmarks.

Unless ... you wanted to use a protocol which isn't supported by the RTC SDK (for example: "PING") to eliminate the "hanging" ports issue, in which case you would have no other choice but to use some other components.

Best Regards,
Danijel Tkalcec

[jonb2]

Ok D.

Danke.

I will try the RTC client and report back when I have finally finished this nightmare of a project - probably in 2023 :-)

J.

[D.Tkalcec (RTC)]

Google for "Avoid TCP/IP Port Exhaustion" if you need more information on that subject. You will find that there is a lot of information about this already on the internet.

You may also want to take a look at TXT files in the "Help\Config" folder included in the RTC SDK.

I don't know how many domains you need to grade, nor what you are doing in order to grade a domain, but there is no need for you to invest much time in testing performance until your project is close to completion. And the only thing you need to test, is how much time it takes on average to grade a single domain, when using each of the supported methods. To get accurate results, I'd say you need to run the same test 100 times on a few domains, using these settings:

1. Async WinSock in Single-threaded mode:
MultiThreaded:=False;
Blocking:=False;
useProxy:=False;
UseWinHTTP:=False;

2. Blocking WinSock in Single-threaded mode:
MultiThreaded:=False;
Blocking:=True;
useProxy:=False;
UseWinHTTP:=False;

3. WinHTTP in Single-threaded mode:
MultiThreaded:=False;
Blocking:=False;
useProxy:=False;
UseWinHTTP:=True;

4. WinInet in Single-threaded mode:
MultiThreaded:=False;
Blocking:=False;
useProxy:=True;
UseWinHTTP:=False;

5. Async WinSock in Multi-threaded mode:
MultiThreaded:=True;
Blocking:=False;
useProxy:=False;
UseWinHTTP:=False;

6. Blocking WinSock in Multi-threaded mode:
MultiThreaded:=True;
Blocking:=True;
useProxy:=False;
UseWinHTTP:=False;

7. WinHTTP in Multi-threaded mode:
MultiThreaded:=True;
Blocking:=False;
useProxy:=False;
UseWinHTTP:=True;

8. WinInet in Multi-threaded mode:
MultiThreaded:=True;
Blocking:=False;
useProxy:=True;
UseWinHTTP:=False;

Each API has its downsides and upsides.

The WinInet API has a built-in connection pool, but limits the number of outgoing connections per domain to 3 per outgoing domain/IP. This shouldn't be a problem for you, though, since you will have a lot of domains to choose from, so you would be better off using a single physical connection per domain, until you have finished all behcnmarks for that domain and have the final "grade" for that domain. The biggest problem with the WinInet API is a requirement for a user Windows account, which means that using this API from a Service might pose a problem.

The WinHTTP API also has a built-in connection pool, but it does not limit the number of outgoing connections per domain, and does not require a Windows user to operate, which is better for applications running as Windows Services. But, in case you are running behind a proxy, you will need to manually configure the WInHTTP API in Windows. WinInet and WinHTTP APIs also both have built-in support for SSL, so there is no need for 3rd-party encryption components if you need to work with "HTTPS://" domains.

Then there are Async and Blocking WinSock. Because WinSock does NOT have built-in support for HTTPS, you will need 3rd-party encryption components when working with "HTTPS://" domains. This is not a bad thing in terms of performance, it just requires the use of 3rd-party components, and needs a bit more "setup" work (setting the "useSSL" property to TRUE won't be enough).

Async WinSock has the unique abbility to work with multiple active connections at the same time by using a single thread, without blocking it during the actual communication. But this requires the use of Windows messages, so it will usually be slower in performance than blocking WinSock.

Multi-Threaded mode uses a RTC thread pool, which requires some time for setting up each threads, and requires a bit more CPU than single-threaded mode for thread-switching. This is why opening a connection with MultiThreded=True will usually take more time than with MultiThreded=False, and the overall communication performance on very fast networks could be somewhat slower. But, if you have more CPU and RAM than bandwidth, and you want to run multiple connections parallel, it shouldn't have a big impact on the average performance.

I hope this info helps.

Best Regards,
Danijel Tkalcec

[jonb2]

This is incredibly helpful Danijel. It is greatly appreciated.

Thank you!!

J