RTC Thread Pool

[barbarland]

Hello,
My service works as expected with RTC without multi-threading  
Thanks

Now I'll try to use the power of RTC when MultiThreaded=TRUE and Blocking=FALSE.
I am trying to understand the WebStressTool project.
I am reading the helpfile, source code etc...

In RTCHelp.chm I can read

VERY IMPORANT!!! If you create connection components at runtime:
      * NEVER! use Create(), Free or Destroy on ANY connection component.
      * Allways use the 'NEW'  class function (implemented by all TRtcConnection
        components) to create the connection object and
      * ONLY use the 'RELEASE'  method to free the connection object.

Is it only true in case I create my own class (inherited from a TRtcConnection)?
If not I wonder why you never use new and release in any project (like WebStressTool).

My knowledge in English is quite limited.
And I should appreciate if you can drop a word about the need to use New/Release.

I have another question concerning RTC_THREAD_POOL_MAX.
Sometimes I want to change the value of RTC_THREAD_POOL_MAX (when my service starts for instance).
(I know that I DONT have to change it until connections are active)

And sometimes I need to know if the pool is busy (not between a OpenThreadPool..CloseThreadPool in rtcThrPool.pas).
(I also need to know if the pool is busy to delay the service shutdown.)

I wrote a function ThreadPoolCount in rtcThrPool.pas

function ThreadPoolCount:Cardinal;
  begin
  CSThread.Enter;
  try
    if Assigned(ThreadPool) then
      Result:=ThreadPool.Count
    else
      Result:=0;
  finally
    CSThread.Leave;
  end;
  end;

Example of use:

   NewThreadPoolMax:=Ini.ReadInteger('Options', 'ThreadPoolMax',32);
    if (NewThreadPoolMax<>RTC_THREAD_POOL_MAX) then
    begin
      if ThreadPoolCount = 0 then
        RTC_THREAD_POOL_MAX:=NewThreadPoolMax
      else
        LogError(Format( 'Warning Init RTC ThreadPoolMax (RTC busy, %d<>%d)',[RTC_THREAD_POOL_MAX, NewThreadPoolMax ]));
    end;

Is there a better way to do the job ?

Kind Regards
Barbaland

[D.Tkalcec (RTC)]

1) Thank you for your feedback about your "Calling ReportStatus" question.

2) Please do NOT continue asking more questions in an old thread after the original question from the thread was answered. Please create a separate thread for every question you are asking and use the appropriate Subject line for every question. Thank you.

3) The comment about using New and Release instead of Create and Free are actually relics from the past, when I was thinking about making the RTC SDK compile cross-compatible with Delphi.NET. You can safely ignore them now.

4) You should NOT directly use any classes nor methods in the rtcThrPool unit, unless you are extending the Thread Pool implementation and that is most likely something you won't be doing anyway, so ... please forget the classes in that unit even exist. The only thing you can use from there are global variables and they should only be set once before the first connection component has been activated.

5) It is safe to change the RTC_THREAD_POOL_MAX variable even when your Server is running. The only reason I have recommended against it is that changing that value once your thread pool size has already exceeded the value you are currently setting will usually NOT have the effect of reducing the pool size. I also do not see a reason why anyone would want to do this. A thread pool size should be set only ONCE at application start and NOT changed anytime later. A thread pool will automatically be created before the first connection needs a thread and will automatically be released  on application shutdown.

6) Checking how many threads exist in a thread pool is comparable to checking how many database connections your database connection pool can hold. The same way a database connection pool will normally hold more connections than are currently being actively used and will not be closing a connection every time when it is not being used, a thread pool will keep threads running in idle state and waiting for a new job when there is nothing to do. This means that checking how many threads are currently in the pool will not tell you anything about the number of threads being actively used.

A thread pool will be automatically initialized by the RTC SDK at the time the first connection creates a virtual thread object and tries to post a job to that virtual thread. From that point on, the thread pool will grow as needed (depending on your Server load) and will normally not reduce in size -> unless you have explicitly specified a small value for the maximum number of allowed idle threads. But I would NOT recommend changing other values for the thread pool, because starting and stopping a thread is a time consuming task and you do not want to have your Server burdened by this on top of the tasks it has to handle.

7) This is the first time since 2005 anyone has mentioned that they might need to know how many threads are active in a thread pool. I also can not think of any reason why this would be important except maybe to generate some kind of thread pool size statistics. Can you please explain what situations you are talking about where you would need to know how many threads are active in the thread pool?

If the only reason for knowing this is to delay service shutdown, then I should tell you that a thread pool will only be active if a connection is actively being used. If there are no open connections and the Server is not listening, there will be no active threads in the thread pool. So ... what you SHOULD actually be doing on your Service shutdown is stopping the Server listener and waiting for all the connections to close, as explained in THIS topic.

Best Regards,
Danijel Tkalcec

[barbarland]

1) 2) 3) 4)
Ok.
I agree with you.
Thank you.

5) 6)
I need to send requests to different kind of servers.
For example:
Servers A = Intel386Ex, limited to 4 simultaneous requests by IP (or account)
Servers B= ARM9, limited to  64 simultaneous requests by IP
Servers C = Intel i7 , no limitation
Each server handles a pool of temperature sensors.

My idea was:
I set RTC_THREAD_POOL_MAX:=4
I request temperature of each sensor of servers A01-A02-A03 (one request by sensor).
I collect information and update a locale database (postgresql).
My application is in <Idle mode> until all threads are running.
I compute temperature average.

I request temperature of each sensor of server A04.
I collect information and update a locale database.
<Idle mode>
I compute temperature average.

I set RTC_THREAD_POOL_MAX:=16
I request temperature of each sensor of servers B01-B02-B03.
I collect information and update a locale database (postgresql).
<Idle mode>
I compute temperature average.

I set RTC_THREAD_POOL_MAX:=256
I request temperature of each sensor of server C01.
I collect information and update a locale database (postgresql).
<Idle mode>
I compute temperature average.
and I retry from the beginning

7)
I will use RTC on dedicated servers (7d/24h).
a) I think it can be useful to log informations about thread usage to track application load (for hardware management planification).
b) If my application displays the thread usage in real time an operator can immediately determine if he can stop/update/reset a firewall.

Of course I can use global variable.
But I just thought that the pool manager could be more informative about its internal state.
Now I will follow the indicated topic.

Thank you very much for the answers

[D.Tkalcec (RTC)]

5 and 6) Bad idea.

Thread pool size the RTC SDK uses has nothing to do with Servers you are connecting to. The only reason you can change the size of a thread pool in the RTC SDK is to define how many parallel operations you want to allow on your side. What you should do is determine the maximum number of threads you want to allow over the whole life of your application and stick to it.

Also, you need to know that the RTC SDK can handle ANY number of connections by using a single thread. Limiting your thread pool to a smaller size will not mean that you will have a smaller number of active connections to handle. For example, a RTC Server with a thread pool size set to max 64 threads can easily handle 4.000 active connections in asynchronous (non-blocking) mode and will also be able to handle a lot more than 64 connections in blocking mode by shifting between connections when no data is being sent or received.

To make a long story short, most RTC applications will be just fine using the default setting with 64 max threads. And I think this also applies to your case.

7) a) RTC Thread Pool was designed to operate fully autonomously and was not meant to be monitored. If the classes would have been designed to be monitored, they would have events to notify you when a thread is being started, stopped, activated or deactivated. Doing it any other way, for example by continuously calling methods to check the current status (which is what I guess you wanted to do) could result in a significant drain on the CPU but would not really give you real-time data. Not to mention that most of the methods there were ONLY meant to be used by the unit itself and the RTC SDK components, since these know when it is safe to call this-or-that method.

To get real-time monitoring you were thinking of, the Thread Pool implementation would need to be extended with the appropriate events and there would need to be a new component to allow you to implement these events. But since I personally do not think this would give you any relevant data. For example, a thread could be actively waiting for data, in which case its CPU usage would be next to 0. The number of events that would get triggered would also be quite high, so your code implemented on these events would need to be very short and very optimized to not drain too much of the CPU.

To make a long story short, even though it might be nice to have this kind of monitoring options added to the RTC SDK at a later point in time, its usefulness would be very limited and I am sure there are better ways to monitor your CPU usage (for example, using real system performance monitors).

7) b) The number of currently active connections should be more than enough for someone who wants to fiddle with a firewall. I do not see what the number of currently active threads would have to do with firewalls. I also do not see a reason for anyone tho first have to check the state of your application before doing anything with a firewall. And the same goes for stopping, pausing or restarting your application.

Best Regards,
Danijel Tkalcec

[D.Tkalcec (RTC)]

By the way ... if you want your application to send multiple parallel requests to your Server(s), you will have to write your code using events and not using the blocking mode. Blocking means that there would be only one request at a time. even-driven mode allows you to create as many connection components as you want, post your requests (for example, using a timer) and write your code in events.

Best Regards,
Danijel Tkalcec

[D.Tkalcec (RTC)]

Here is a general rule which applies to all RealThinClient SDK components and units:
If a unit, function or a class is not documented in the HELP file, it is NOT meant to be used by component users.

One reason for this is that any one of the functions, classes or complete units which are used internally by the RTC SDK can be completely redesigned, rewritten and replaced by implementations which might not even have the functions or methods you were using.

Another reason is that these functions, classes and units are designed to be used in a specific way, following specific (unwritten) rules. Using them without following these rules would most likely get you in trouble (make your application unstable).

And now, after re-reading your first post and looking at the rtcThrPool unit, I am curious about HOW you meant to access the private "ThreadPool" variable which is declared in units "implementation" section? I have placed it there to avoid a situation where someone might get wrong ideas and start accessing it directly, but now that you are asking about how to use its "Count" method to check how many threads are currently active, I am wondering how you wanted to do it.

PS. There is a "Thread Callback" mechanism implemented in the RTC Thread Pool, which could be used to monitor thread construction and destruction and do any thread initialization and deinitialization. This is used for ActiveX support, for example, because each thread where ActiveX components are used needs to be "Coinitialized". If you were using ActiveX components, you would need to use the "rtcActiveX" unit, which registers such a thread callback to initialize each thread for ActiveX. If you want to monitor how threads are being created and destroyed, you could implement your own class by extending TRtcThreadCallback and register it using the AddThreadCallback at the beginning of your application, but ... as wrote in a prior post, I fail to see what good this kind of information would be to you.

Best Regards,
Danijel Tkalcec

[barbarland]

Ok.
I need to review my whole algorithm (I've just got more code to write than expected).
RTC is very versatile.
And I need time to acquire a better understanding about it (I am not a Delphi engineer).

For 7) b) this is a hardware firewall.
When there is no active threads I can safely setup the firewall. During setup my server is like physically disconnected from the internet (no active thread=no broken request).

Thank you for detailed replies.
Barbaland

[D.Tkalcec (RTC)]

7) b) No currently active connection = no broken requests.

The number of active threads has absolutely nothing to do with the number of active connections or the number of running requests in the RTC SDK. Even with zero currently active threads, the RTC SDK could currently be in the process of sending or receiving data through any number of connections by using asynchronous (non-blocking, message-driven) sockets or cycling through connections by using a single thread or by using blocking sockets running from the main thread.

But ... I think your whole idea about having to wait for your application to become idle before you can make any changes in the firewall is moot. Instead of hoping that every one of your requests will be sent without any problems and that every response will be received for the requests you send, you have to take in account that your connection to the Server could break at any time. There are just too many things that can happen. For example, there could be a temporary loss of your internet connection (you were talking about a firewall, so I guess your servers are accessed over the Internet), or one of the Server(s) you are working might have to be restarted, or your connection might be lost because of packet loss due to network congestion, or, or, or.

The RTC SDK implements automated mechanisms which make it easier to handle connection loss. TRtcHttpClient component, for example, has AutoConnect and ReconnectOn properties which implement automatic reconnects in case of a connection loss. TRtcClientModule has the AutoRepost property to handle automatic request re-posting, as well as the OnRepostCheck property which allows you to decide for yourself when you want to have a broken request re-posted.

There are many ways you can write your code to be tolerant to connection failures. Writing your application as if it was running in a "perfect World" where everything is always working as you want it to and everyone is "behaving" according to your own rules is the worst possible thing you can do. Especially when there are so many things that can and most likely will go wrong over the lifetime of your application.

Best Regards,
Danijel Tkalcec

[barbarland]

Ok.
I admit my assertion was quite expeditiously.

For the rest of your post I agree with you completely.
I could add to sum up: Anything that can go wrong will go wrong (Murphy's law).

Barbaland