RTC and Keep Alives

[Andrew.W]

Hi

I'm using RTC and the results have been great.

I'm trying to make my application work well over poor networks, say where the 'ping' time to the server is 250 milliseconds. This has caused my to look at the subject of keep alives.

My finding have been that with a good network application performance is great and keep alives make no noticeable difference. However, with a 'poor' network, say as you get with 3G mobile adapters, it makes a great difference to how the application performs.

A great article about Keep Alives is here:
http://virtualthreads.blogspot.com/2006/01/tuning-apache-part-1.html

I'm using RTC client and RTC server, BUT the RTC server is hidden behind Apache which is acting as a reverse proxy server (the main reason for this is that there are other web applications installed on the box, and all need to work through port 443).

What seems to be happening with the RTC client is that if you fire two requests off in quick succession to the server, the second one will be performed on another connection to the server - meaning all the overhead of round trips is incurred, and two keep alive threads (instead of one) are used on the server. These resources are finite, and as I only need 1, I'd like to use 1.

I'm observing this through Apache's own server-status pages, which show you how it's keep alive threads are being allocated to clients.

My RTC client is single-threaded. The Multi-threaded flag on the client is false. Auto-connect is true.

My question is simple, after all that pre-amble. How do I work best with Keep Alives in RTC?

[D.Tkalcec (RTC)]

If you mean that there are two requests sent at the same time, or that your 2nd request is sent from the same TRtcHttpClient component before the response from your 1st request sent through the same TRtcHttpClient component returns, then I have to say that this can NOT happen because each TRtcHttpClient component can only fire a single request at a time and will always WAIT for a response before firing another one, no matter what you do with it and no matter how you configure the components.

All requests you send through a single TRtcHttpClient component, even if you would be using MultiThreading and HyperThreading and posting multiple requests from different threads at the same time, the TRtcHttpClient component would serialize all your requests by using a request queue, which makes sure that all your requests will be sent out in the order they have been issued, but there will be no more than one request/response cycle active at any time.

But ... there are situations where your Server would receive requests from more than one physical connection from a single Client, even when you are using a single TRtcHttpClient component:

1) When your clients are going through one or more Proxies, each Proxy decides for itself how many connections it wants to use to forward each request received from any client. This means that, when your Clients are connecting to your Server through a Proxy (like Apache, for example), the Proxy can make it so that your Server will receive requests from a single client through more than one physical connection, or that requests from multiple clients are forwarded to the Server through a single physical connection. The decision about how many physical connections to use is left entirely to the Proxy and neither RTC Client nor RTC Server have anything to do with this. But this Proxy behavior should NOT be a problem for you nor the RTC SDK, as long as you do NOT store your client-side information inside your physical connection objects (TRtcDataServer descentands) and use Session IDs to identify each client.

2) WinHTTP and WinINET APIs (which RTC SDK supports if you set useProxy=TRUE and/or useWinHTTP=TRUE on the TRtcHttpClient component) use a connection pool to send all your requests to the Server. Even if you use a single TRtcHttpClient component, WinHTTP and WinINET APIs can still send your requests to your Server through more than one physical connection. Your requests will still be serialized, meaning that you will NEVER get into a situation where more than one request is being processed at a time when using a single TRtcHttpClient component and there will NEVER be more than one request at the Server coming from the same TRtcHttpClient component, even if you use the WinHTTP or WinINET APIs.  At the same time, because of the connection pool WinINET and WinHTTP APIs use for sending out requests, in case you should decide to use more than one TRtcHttpClient component from each RTC Client and set them all up to use the WinHTTP or WinINET API, all your requests will be sent out from only 2 or 3 physical connections, regardless of how many TRtcHttpClient components you are using. Please note that this behavior has nothing to do with the RTC SDK. It is how Microsoft has decided to implement their WinINET and WinHTTP APIs.

I am not sure if this is obvious, but you should also know that there is ALWAYS a round-trip involved if you send out a request, no matter when you do it. This means that sending out 3 requests, no matter how fast you do it, will ALWAYS result in 3 trips to the server and back. If you want to reduce the number of requests sent to the Server and thus lower the number of your round-trips, you need to optimize your code to pack more data into a single request. If you are using RTC remote functions and you are using the standard event-driven mechanisms with the "TRtcClientModule.Call" method, you can pack more than one remote function call into a single request by using the "StartCalls" and "Post" methods. Please read the help file for more information on TRtcClientModule.StartCall and Post.

Please let me know if you need more information on this topic.

Best Regards,
Danijel Tkalcec

[Andrew.W]

Thanks for the long and detailed reply.

>If you mean that there are two requests sent at the same time, or that your 2nd request is sent from the same TRtcHttpClient component before the response from your....

No that's not what I mean. RTC is absolutely doing one request, then the next, and then the next - all sequenced nicely.

What is strange is the effect on the server, and on apache keep alive threads.

Let me try to explain with a sequence diagram. All of these requests are assumed to be happening rapidly.

Client: Wants to make Request A
<Server and client negotiate, and agree to open a connection, this involves min 3 round trips>
Server: Processes request, allocates client to Thread 100.

Client: Wants to make Request B
<Server has already allocated a worked thread (keep-alive!) and so this goes ahead straight away.>
Server: Processes request B, but now more quickly.

Client: Wants to make Request C
<Server has already allocated a worked thread (keep-alive!) and so this goes ahead straight away.>
Server: Processes request C, but now more quickly.

..... {current worker threads held by the client = 1, which is correct}

Client: Wants to make Request X
<For some reason, and this point, client and server decide to open another connection. This involves another 3-way round trip, and results in another worker thread being allocated to the client.>
Server: Processes request X on Thread 101.

.... { current worked threads help open by the client, or more accurately allocated to the client, now equals 2 }

>>Even if you use a single TRtcHttpClient component, WinHTTP and WinINET APIs can still send your requests to your Server through more than one physical connection....

That's really interesting, and maybe that is what I'm seeing. I'm using Wininet with SSL set to TRUE. Do you know of any way to control this behaviour?

I'll keep investigating. From the work I've done so far, and the results of the testing I've done, keep alive really matters if the network is poor. As an example, in a system with a ping of 300ms opening a connection will take around 600ms, and that's before you send any data over.

Cheers,

[D.Tkalcec (RTC)]

As far as I know, unless you have manually set up Timeouts on your Client to force the connection to close after an idle period (see the "Timeout" property on TRtcHttpClient), it is up to the WinInet/WinHTTP API to handle connection opening and closing at their own discretion. Applications using these APIs can request the connection to use "Keep Alive", which is what the RTC SDK does by default, but the API itself decides how long a connection will remain open and how many connections will be used per client.

As an alternative, in case your clients are NOT behind proxies and you are willing to invest a few $$ in third-party components for SSL encryption, I would recommend you to take a look at StreamSec Tools 2. Using StreamSec Tools 2 for SSL would allow you to make your RTC Server to use SSL natively and work on port 443, eliminating the need to use Apache as a Proxy. This would also make your Server more stable, give you better performance and more control over connections "keep alive" status, since you would then be using the WinSock API where connection opening and closing is handled entirely by the RTC SDK and thus fully configurable through TRtcHttpClient properties and settings.

Best Regards,
Danijel Tkalcec

[D.Tkalcec (RTC)]

Btw ... there is a set of parameters you can use to specify timeouts for the low-level API. Depending on the API you are using, different parameters will be available. The property is "TimeoutsOfAPI" and is available on the TRtcHttpClient and TRtcHttpServer components. There are no properties to specify how long a connection would be forced to remain open during idle periods (when there is no data being sent), though.

Best Regards,
Danijel Tkalcec

[Andrew.W]

Thanks for the replies.

>>...eliminating the need to use Apache as a Proxy.

We need Apache, or at least some reverse proxy. We need to have IIS application on the same box and on the same port as our RTC server, all using SSL. SSL wildcard certificates like we need are expensive, and this means we need only one.

Of course, an RTC ISAPI DLL may have been an option, but my company is moving away from Microsoft stuff, not towards.

>The property is "TimeoutsOfAPI" and is available on the TRtcHttpClient

Can you help me find this? I can't locate it anywhere. I think my RTC version is 3.30.

[D.Tkalcec (RTC)]

Then you should update to a later version, because the "TimeoutsOfAPI" properties have been added in version "3.31".

But there is one more thing I did to explicitly mention, because it seemed so obvious. GSM, 3G and other mobile connections are not only slow, but also highly unreliable and your disconnects might have been a result of bad connectivity rather than anything else.

Best Regards,
Danijel Tkalcec

[Andrew.W]

Thanks Daniel.

It's not that. I'm testing the effect of keep alives on a known good network, making changes, and then running tests on other types of network to see what the effects are.

I'll upgrade and see what happens.