Implementation strategy question

[HelgeLange]

Hey Danijel,

since you have a lot more experience with RTC than me, I want to ask you a small design question for a server with your fabulous components:

I write a mobile app that connects to a server, that works a bit like a chat server in your examples. The example uses delayed calls, I already used them in another project (as you may remember) and they work just fine.
My question now is : when I have 50k or 100k users connecting to my server and sending an update every 10 up to 60 seconds, that server already has a lot of open sockets and will take some memory. But I also need callbacks (or delayed calls), but only in certain situations. What would be the best strategy there... Delayed calls give me the advantage that I don't need to know a callback ip/port and if they're in a wifi environment then there is no real routing involved.
So if every client opens a second connection isn't that too much for a server to handle ?

Thanks for your input,
Helge

[D.Tkalcec (RTC)]

Using the TRtcGateway component, you might be able to handle 20K users connected to a single Server, if the Server is powerful enough and had a high enough bandwidth. But with Delayed Calls, no way. That is what I can tell you for sure. A few hundred users or maybe even a thousand users working with Delayed Calls would probably work, but expecting a single Server to handle 50k to 100k users at the same time, where each user requires one open connection at all times for "polling" and a separate connection for making requests is way too much. Your problem is not just Server memory, but also the number of physical TCP/IP connections which a single Server can handle. This is not just limited by Server RAM, but also by other System resources like input and output socket buffers and the fact that each physical connection requires a unique Port number on the Server.

Best Regards,
Danijel Tkalcec

[HelgeLange]

Well, I only did a test with 2000 users so far to see, how much I can throw at the server per second

So if I'd like to handle 50k users for example I need several servers and several gateway servers to handle the incoming traffic?
I'll try to find some information about how your gateway server works ...

[D.Tkalcec (RTC)]

Well, a lot depends on your actual implementation. But to handle the load you are talking about (50k to 100k users), you will definitely need some way to balance the load across multiple Servers. As the number of users goes up, response times will be going down, so there is also a question of performance.

Best Regards,
Danijel Tkalcec

[HelgeLange]

So what I could do is the following :

I could use 3 lines of servers : 1 Login, x connection server (each with 2-5k), 1 data server

Client connects to a login server that assigns a permanent server for that session, the client then connects to the assigned server and stays there as long as it is connected, every server in that line has a limit of 2k - 5k (some testing and adjusting has to be done there). It sends all updates directly to that server. But since I need all data centralized, every conection server makes 2 connections to the data server and centralizes all data there...

How does that sound ?

[D.Tkalcec (RTC)]

Sounds like a plan. In theory

Now, you need to implement and test it, to see how it works

Best Regards,
Danijel Tkalcec

[HelgeLange]

In your opinion with an update of let's say 5k users and each user sending every 30 seconds is it better to open and close the connection all the time (or use a post), or better keep the connection open ?

[D.Tkalcec (RTC)]

Opening and closing connections frequently makes things even worse, because a Port remains unavailable for new connections for some time after a connection is closed, because of which frequent opening and closing of connections can get you into a situation where could end up with a Server refusing to accept new connections, even though there are no connections currently open. In short, having a few thousand clients (even if it was 10K Clients) which keep their connections open all the time is better than having the same number of Clients opening and closing their connections all the time. Not to mention the performance loss and the TCP/IP overhead of opening and closing a connection.

Best Regards,
Danijel Tkalcec

[HelgeLange]

Thanks for clearing that up.
I'm still trying to figure out why a TRtcGateway is better, the forum doesn't give me a lot of hits there...

[D.Tkalcec (RTC)]

When you are working with the TRtcGateway component, each Client will establish two connections with the Server, one connection used exclusively for sending data from the Client to the Server and the other connection used only for sending data from the Server to one or more Clients and displatching the same packet to multiple Clients in a group, while keeping different Clients connection speeds in mind. A custom protocol was designed to allow individual packets sent from any Client to be split into smaller packet for faster sending and to reduce memory usage on the Server, so the Server will require less memory, data will be dispatched faster, negative latency effects and TCP/IP overhead will be minimized and there is no need for manual coding. It is a general-purpose Gateway component, which only handles packet dispatching.

RTC Delayed Calls, on the other hand, were introduced as a simple way to allow the Server to respond to Remote Function Calls later if data is not available immediately when a call comes in. It can be used for a lot of things where data polling is needed. Client sends a request to the Server, the Server checks if there is any data available, puts the connection to "sleep" until data is ready, "wakes" the connection up and sends a response back when there is something to send. While this also works for message dispatching (as shown in the RTC Messenger Client/Server example), every piece of data being sent from the Server to the Client requires a trip back and fort. It is not a continuous stream in one direction, as with the TRtcGateway component. This means that latency (PING times) will be playing an important role for the performance. Also, because Delayed Calls are there for use with remote functions (which are very flexible but require additional memory and a rather complex parser and serializer), there will be higher CPU and Memory requirements than using the TRtcGateway component.

In short, TRtcGateway is highly optimized and designed from ground up for message dispatching. Even dispatching larger packets (up to 16MB) between Clients would only require a few KB of RAM on the Server. Delayed Calls give you more freedom, but require more bandwidth, have negative latency effects, require more Server resources, with "user code" being an additional "risk factor" to further reduce performance and increase Server resource requirements.

Best Regards,
Danijel Tkalcec

[HelgeLange]

Actually now that you explained it quite well, the solution is in the between.
I can create another server that only manages events to group certain cellphones together for a short amount of time (1 minute), do their stuff and then release the connections again.

Image the whole thing like a Taxi app (like MyTaxi). So you have a lot of drivers, they update their position all the time. And you have passengers that need a taxi. So calculating distance you select a subset of all taxis (those that are reasonable close) and you want to inform those that there is a passenger that needs a ride. So one of them decides to take the offer and the group is released, the communication now runs between passenger and taxi only, all other connections can be closed. As soon as the passenger enters the taxi, this connection can be closed too.

Basically something similar I'm writing, the analogy fits.

[D.Tkalcec (RTC)]

With increased user counts, things do start getting complicated pretty fast. I'm not going to ask you about the details of your implementation, but I'm glad if my explanation helped you find a good solution to your problem

Best Regards,
Danijel Tkalcec

[HelgeLange]

If things are getting ugly, that means I will have a lot of users and get some good money, I'll hire you to check my client/server code