Memory usage with many clients connected

[Aeter]

Hello,

our gateway application uses a rtcrecord to keep in memory connected clients. However, it seems there isn't a way to free up the memory when a client disconnects.
The memory remains allocated, since the items in the list are not removed but only set to null.
Is there another way to manage this? We would see the RAM memory usage that decrease when clients disconnect.


 procedure TRtcAbsRecord.SetNull(const index:RtcWideString; const Value: boolean);
  begin
  if Value then
    SetObject(index, nil, True);
  end;
 
 FValues.Objects[idx]:=nil; // do not call Delete(idx), or other fields Index positions would change

[D.Tkalcec (RTC)]

The behavior of a "TRtcRecord" class (which you have noticed and reported here) is by design. Unless your Server has only a few users and very low traffic, you should NOT use a TRtcRecord class to keep track of your connected clients, but instead implement your own custom class for that purpose.

[Aeter]

Hello,

we've made a simple test using the sample applications provided with the Component.

We used the Gateway Test and the App_Client.

We noticed memory is allocated when the App_Client makes many connections. However, this memory is never released, even if the App_Client is disconnected and closed.
If you make new connections, the memory increases again. There is no possibility, it seems, the memory usage returns to the initial state, when you open the Gateway Test app.
How is it possible?

We've prepared a simple video to show what happens. Can I send it by email ?

[D.Tkalcec (RTC)]

You can upload your video to YouTube (or any other Service that provides free video uploads) and post the link here.

[Aeter]

Here is the link: https://streamable.com/s3fxn

As you can see, the private bytes allocated by the server application is never released.
If you see also in the Windows task manager, the RAM used by the server never returns to the value when it was started, even if all clients are disconnected and sessions expired.
How can we avoid this? The best scenario would be the RAM is completely freed when all clients are disconnected and there are no more connections active.

[D.Tkalcec (RTC)]

1. Which Delphi version are you using?

2. Are you using the default Memory Manager included with that Delphi version, or an alternative Memory Manager like FastMM4?

3. Do you know how Memory Management in Delphi works and when/how Operating System Memory is being allocated and released by the Delphi Memory Manager?

4. Are you aware of Memory Fragmentation and how it affects Memory Usage at the Operating System level when Memory is being managed by a Memory Manager (like it is the case in Delphi)?

5. Do you know how a Stack works in Delphi and how each Thread allocates Memory for its Stack when the Thread is created?

6. Do you know how a Thread Pool works and how a Thread Pool with active Threads, even if they are idle, affects Memory usage?

RTC SDK does NOT directly allocate memory from the Operating System. Delphi Memory Manager does that. So ... if you want to know much Memory is being used by your Delphi Application, you should ask the Memory Manager you are using and NOT the Operating System.

Here's more info about Memory Management in Delphi from the EMBT wiki:
http://docwiki.embarcadero.com/RADStudio/Tokyo/en/Memory_Management

If you are using an older Delphi version, make sure to use FastMM4 or some other Memory Manager, because the Memory Manager included with older Delphi versions (before it was replaced with FastMM) is poorly implemented and doesn't work well for Multi-Threaded Applications. In short, the old default Memory Manager in Delphi was very slow, caused a lot of memory fragmentation and resulted in memory usage to go higher and higher, until it was completely used up, so you would end up with a non-working Application after prolonged usage or a stress-test.

[Aeter]

Everything is in "default" mode.
Tried with Delphi XE2 or Delphi Berlin, same behavior.

Thanks for the points about Memory topics, however, I must be sure what is the cause of the continuosly increasing memory usage despite the absence of any active connection.
If you say the problem is the thread pool, how to avoid this?

In a real scenario, if I created a web server with RtcSdk, and no client is connected to the webserver, I expect the RAM memory I see in the task manager is less than 100 MB for example.
Instead, simply using the sample application, after thousands of connections and thousands of disconnections, the RAM usage remains GBs.
How to avoid this?

I cannot believe it is an issue of the Delphi memory manager, otherwise this would be a serious limitation to create a real web server with Delphi and RTC without filling the RAM of any real server machine.
Is it possible, with the sample application, to obtain this:

1. the gateway test app starts and only shows 50 MB memory usage
2. many connections are made by the test aplication: the gateway test app now uses 700 MB RAM
3. all connections are closed and freed up, all sessions are expired
4. the gateway test app now uses again only 50 MB RAM

Isn't it reasonable?

[D.Tkalcec (RTC)]

If your Server has to serve thousands of Clients, then your expectations are unreasonable. If you are reaching a point where your 32-bit Server uses up all of its RAM, then you should consider going to 64-bit and using a PC with more available RAM.

[Aeter]

I'm sorry but this reply doesn't contain any answer to my question.
Let's assume we're talking only about 64 bit applications and OS.
And let's assume I can create with RTC a web server that can allocate tens of GBs of RAM. That's not the point anyway.

If the RTC component never frees up the RAM it uses, it isn't a scalable component. But I'm sure it is!
It is a component to create an http web server, so isn't it reasonable to expect it can manage hundreds of thousands of connections without wasting tens of GBs of RAM?

What about reproducing the issue on your side?
I just would know if you can simply obtain this:

1. the gateway test app starts and only shows 50 MB memory usage
2. many connections are made by the test aplication: the gateway test app now uses 700 MB RAM
3. all connections are closed and freed up, all sessions are expired
4. the gateway test app now uses again only 50 MB RAM

[D.Tkalcec (RTC)]

You could try tinkering with socket buffer sizes (see SOCK_* variables in the "rtcSystem.pas" unit) and Thread Pool parameters (see RTC_THREAD_* variables in the "rtcThrPool.pas" unit) to see if you can find values that seriously reduce your Servers total Memory usage, without resulting in much higher CPU usage and/or reduced performance and/or reduced Network throughput.

Performance comes at a cost, so you need to decide what you want.

Do you want a fast Server, or a Server that doesn't need much RAM?

You can't have both. It's that simple.

[Aeter]

The point is not "much RAM". I can also have a server with 128 GB of RAM.
But, after, let's say, 1 month of thousands of connections and disconnections, it couldn't be enough anyway, since every new connection increases the RAM usage, but every disconnection DOES NOT DECREASE the RAM usage.

The question is: why the memory is never freed up?
I'm sure you've the solution. You can simply reproduce it with the sample app.
This would be a very important tip for all users, and will make the component absolutely scalable.

[D.Tkalcec (RTC)]

Like all things in Delphi, when any part of RTC SDK requires Memory for anything, it uses the Delphi Memory Manager to allocate and release Memory, while the Memory Manager handles the rest. So ... what you see at the Operating System level is Memory that is currently RESERVED by the Delphi Memory Manager for use by your Application. This is NOT necessarily how much Memory is IN USE by components of your Application, but ONLY the amount of RAM currently RESERVED by the MEMORY MANAGER for current and possible future use by the Application.

Here's a simple test, completely unrelated to RTC:

1. Allocate 1.000 dynamic arrays of 64 KB in size.

2. Check Memory Usage in the Operating System.

3. Allocate 1 more dynamic array of 100 KB in size (separately).

3. Check Memory Usage in the Operating System.

4. Release the first 1.000 dynamic arrays you've allocated previously.

5. Check Memory Usage in the Operating System and let me know what you see.

[D.Tkalcec (RTC)]

By the way ... I've already answered your question "how to reduce RTC memory usage". See my reply above ("rtcSystem.pas" and "rtcThrPool.pas" units), but ... keep in mind that reducing these parameters can have serious negative effects for your Server - like reduced performance, increased CPU usage and reduced network throughput. Going too low can also break your Server completely.

The values I've placed there are a result of my own stress-tests, which were mostly focused on finding the optimal values for the best possible performance, since the most important thing for a Server (IMO) is how fast it can handle all requests and respond, but these do NOT have to be THE BEST possible choices for ALL possible scenarios. For example, a slower CPU with more RAM and higher bandwidth might require increasing buffers to improve performance, while a faster CPU with less RAM and lower bandwidth might reach its peak with lower buffer sizes.

So ... if you are bothered by Memory Usage of your RTC Server, feel free to try out different values.

[D.Tkalcec (RTC)]

Here's the source code for a simple Memory Manager test (see my reply above) ...

Code:

type TByteArr=array of byte;
var arr:array[0..1000] of TByteArr;

procedure TForm1.AllocateThousandClick(Sender: TObject);
  var
    i:integer;
  begin
  // Dynamic arrays can be of any size, but 1.000 x 64KB 
  // should be high enough to show in the Task Manager
  for i:=0 to 999 do SetLength(arr[i],64000);
  end;

procedure TForm1.AllocateOneClick(Sender: TObject);
  begin
  // 100 KB does the trick
  SetLength(arr[1000],100000);
  end;

procedure TForm1.ReleaseThousandClick(Sender: TObject);
  var
    i:integer;
  begin
  // Setting Length to zero "deallocates" the array
  for i:=0 to 999 do SetLength(arr[i],0);
  end;

procedure TForm1.ReleaseOneClick(Sender: TObject);
  begin
  // Setting Length to zero "deallocates" the array
  SetLength(arr[1000],0);
  end;

You can do the same with pointers using GetMem and FreeMem. It makes no difference.

The problem is NOT that the code above is buggy, or that Delphi doesn't release memory used by dynamic arrays when you set their Length to zero (btw ... dynamic Strings are also dynamic Arrays), the "problem" is that the Memory Manager allocates Memory from the Operating System in LARGER blocks and releases these blocks ONLY if possible AND necessary.

If you are NOT happy how the Memory Manager works in Delphi, you can write your own, or use one of the alternatives available out there. FastMM4 (now used with Delphi 2009 and above) was written by a developer who does NOT work for EMBT, and ... "Nexus DB" have their own Memory Manager, which is used with their Database Engine (it is heavily multi-threaded).

[D.Tkalcec (RTC)]

Here's the same with a simple pointer using GetMem/FreeMem ...

Code:

var arr:array[0..1000] of pointer;

procedure TForm1.AllocateThousandClick(Sender: TObject);
  var
    i:integer;
  begin
  for i:=0 to 999 do GetMem(arr[i],64000);
  end;

procedure TForm1.AllocateOneClick(Sender: TObject);
  begin
  GetMem(arr[1000],100000);
  end;

procedure TForm1.ReleaseThousandClick(Sender: TObject);
  var
    i:integer;
  begin
  for i:=0 to 999 do
    begin
    FreeMem(arr[i]);
    arr[i]:=nil;
    end;
  end;

procedure TForm1.ReleaseOneClick(Sender: TObject);
  begin
  FreeMem(arr[1000]);
  arr[1000]:=nil;
  end;

NOTE: In this test, make sure to ONLY click the "Allocate" button ONCE and then click the appropriate "Release" button, because you would otherwise be creating a REAL MEMORY LEAK, since pointers to previously allocated memory blocks would be overwritten by GetMem by pointers to newly allocated blocks.

This is NOT the case when using dynamic arrays in Delphi, because dynamic arrays have a state (length), which the "SetLength" function knows about and makes use of it when the array size is changing. Another thing that goes in favor of dynamic arrays vs pointers, is that dynamic arrays are auto-reference-counted, same as Ansi and Unicode Strings.

Btw ... because of the two reasons above that go in favor for dynamic arrays, and because dynamic arrays are fully cross-platform, RTC SDK uses dynamic arrays for all its buffers and Delphi objects for the rest.