DoWaitForCompletion is not bloked?

[Director]

Hello again.

I have faced a new problem: DoWaitForCompletion isn't perform a real blocking call.
I send a demo project to your e-mail. There are 2 simple apps: client and server.
Server exports 2 different functions: "Function1" return a unique string and "Function2" returns current date.
Client call "Function1" in manual mode and "Function2" in automatic mode. But this automatic calls generates with window messages, which are sent by a secondary thread.

How to use it:
- start Server.exe;
- start Client.exe;
- press "Connect", "Call" and "Disconnect" buttons in Client form;
- watch log messages.

If you set STOP_FLOOD directive to False (default value) you'll never receive a correct answer from manual "Function1" call, because "Function2" result always rewrite it.
Directive ALTERNATE_METHOD does not influence result. This mean that the situation repeats with any receiver of flood messages: with an application form or with custom user window.

Is there any way to make DoWaitForCompletion really blocked?

[D.Tkalcec (RTC)]

Here is a short explanation of the "DoWaitForCompletion" method, as it is written in the RTC SDK Help file and method comments (source code):

"Wait for all posted requests and function calls to complete, be aborted, be calceled, or for the connection to close."

It may not be obvious from the above sentence, but "all posted requests" does not only mean requests which have been posted before the DoWaitForCompletion method was called, but also any requests posted while waiting for a response (or responses) from other, previously posted requests.

In other words, if a new request gets posted while DoWaitForCompletion is waiting for a response, then DoWaitForCompletion will not return until the last posted requests was sent and its response received.

If that does not explain what you are doing wrong, let me go through your example and try to explain it:

1) You expect the Client to be blocking, but you have left the "Blocking" property of your TRtcHttpClient component at its default value = FALSE. And this means your Client is per definition NON-BLOCKING. Would you have set the "Blocking" property to TRUE, even with your rather strange example, the results would be what you thought they should be. In other words, each of your blocking remote function calls would give you the Result received from that specific remote function, even if another function was posted while waiting for the result from the Server.

2) You have created a background thread from which you are posting messages to the main thread, from where you are  using the exact same TRtcHttpClient and TRtcClientModule component to post more remote function calls into the request queue. Because you are using a message-driven asynchronous WinSock implementation (which you are doing when Blocking=FALSE, useProxy=FALSE and useWinHTTP=FALSE on your TRtcHttpClient component), DoWaitForCompletion will have to process the message posted from your background thread even if you would call Execute with "AllowMessageProcessing=FALSE".

Hence, the message you are posting from your background thread is adding more remote function calls to the queue and DoWaitForCompletion will need to wait until the last one has been completed before returning. And because a single TRtcClientModule only has a single internal TRtcResult object for use with the blocking Execute method (it is supposed to be used in BLOCKING mode, so a single result is all it needs when used correctly, as there will always be only one call at a time), if you use Execute to call additional remote functions while waiting for the result on the last Execute call, all your Execute methods will return the same result. The result received from the last Execute call.  And that is exactly what you see in your example. Your result from Execute made in Call1 is being overridden with the result received from Execute made in Call2.

3) To make things even worse, you are using the non-blocking Sockets implementation (TRtcHttpClient.Blocking=FALSE,useProxy=FALSE, useWinHTTP=FALSE and useSSL=FALSE) with the "Execute" method called with "AllowMessageProcessing=TRUE". Without disabling any buttons or forms, this will allow your client application user to click any buttons *again* while you are still waiting for the result of your last posted remote function call, thus giving your user the ability to to post multiple remote function calls to the Server while still waiting for the 1st result to arrive - even without using background threads.

If your remote function calls are mostly blocking, I would recommend you to set the "Blocking" property of your TRtcHttpClient component to TRUE. This will make your implementation a lot easier to maintain. If your remote function calls are mostly non-blocking and you want to use events to get and process the results instead of using the blocking Execute method, make sure your client will know what to do with received results.

I hope this explains your problem. If you have more questions, feel free to ask.

Best Regards,
Danijel Tkalcec

[D.Tkalcec (RTC)]

On a side-note ...

I do not know what exactly you wanted to achieve with your example, but if this was an attempt at implementing a periodical "ping" by using a background thread, I strongly recommend you to reconsider your implementation.

Instead of using a background thread and posting a message to a window handle created on the main form, you could use the TTimer component to trigger your "ping" calls. And from the Timer event implementation, instead of using Execute and blocking the execution, you should use a TRtcResult component and implement the OnResult event to make your call fully non-blocking and event-driven, so it will not interfere with any blocking calls you may be making at the same time from the main thread.

When you change your code to use a TTimer instead of a background thread, you will also be able to disable the Timer every time the OnTimer event is triggered, so you won't get into a situation where one event is being processed while another one is being triggered. You should re-enable the Timer from the OnResult or OnAbort events.

This will make your implementation a lot shorter than it is now, it will be a lot easier to understand and to maintain, and ... it will NOT mess with the rest of your implementation using the Execute method to have parts of your code run in emulated blocking mode even if you are using the non-blocking message-driven sockets.

Best Regards,
Danijel Tkalcec

[Director]

I thought about Blocking property. But you wrote in help:
But, a down-side of blocking connection providers is that they will NOT inform you of a connection loss unless the connection is currently being used for sending or receiving data.
And I really see this behavior in my app when changing Blocking to True: when I shutdown server all clients looks fine, OnConnectLost and OnDisconnect doesn't fired. And in some cases the application even hangs up. It is inadmissible for me... Is there any trick to learn about a disconnect instantly?

There is no secondary thread in my real application. Messages are sent by other application. And it isn't a "ping" calls. Of course this calls aren't periodical and I can't change it to something else.

[D.Tkalcec (RTC)]

Blocking connection providers (which are used when you set Blocking to TRUE) use a blocking API which does NOT have notification mechanisms like the non-blocking asynchronous connection provider has. But if your application is correctly designed, it should not matter if you are physically connected to the Server "between requests". And the only thing you would NOT know when using Blocking sockets is the status of your connection when you are not actively using it.

It should actually be a normal thing for all your Clients to allow the connection to get closed between every request and open a new connection before a new request is sent out. For your users, it should not matter if a connection is open at all times or if it is opened and closed as needed. And for this purpose, I strongly recommend using the "AutoConnect" property (set it to TRUE) on the TRtcHttpClient component instead of manually calling Connect. You might also want to use the Timeouts property to have a connection closed when it is idle for too long (for example: 10 or 20 minutes).

As to your design, I find it a bit odd that your Client is being controlled by another application. But if that is how you want to do it, then you should seriously consider using the RTC SDK in non-blocking mode. Use TRtcResult components and implement their OnResult and RequestAborted events to handle the results from remote function calls made from your external processes.

The Execute method is really only used when your application is making the calls and you want a very simplified way of making blocking remote calls. The Execute method should NEVER be used from more than one thread at a time, nor should it be called in a message handler for messages received from background threads or (even worse) other applications.

As long as you do NOT try to use the Execute method from background threads, or from message handlers which are called from background threads, or from external applications, you will be fine, regardless of which connection provider you are using: blocking or non-blocking. So, your only real problem is that you have used Execute in a way it was not meant to be used (from a message handler called from a background thread or an external application).

Best Regards,
Danijel Tkalcec

[D.Tkalcec (RTC)]

Just for the info:

What you have seen in your example is a limitation of the Execute method and the fact that it uses a single internal TRtcResult component to handle all blocking remote calls sent from the TRtcClientModule, as well as a single TRtcValue object to keep the last Result received. And the reason why the Execute method is implemented this way is to keep the use of the Execute method as easy as possible. In other words, give you access to the last Result objects through the "LastResult:TRtcValue" property and making sure the result object is freed automatically for you so you do not have to do it manually (when "AutoFreeResult=TRUE" / default).

This limitation is NOT imposed by the use of the DoWaitForCompletion method and there is absolutely no reason why you should not be able to implement your own blocking function or class for using RTC remote functions in blocking way. You can look at the Execute method implementation in the "rtcDataCli.pas" unit to see what I mean.

Also note that you can always use the DoWaitForCompletion method to wait for all requests to be completed, even if you are using the non-blocking event-driven approach to making remote function calls. If your reasons for using the "Execute" method were to have the message handler wait for a result before continuing, you can simply use the DoWaitForCompletion method to block the message handler from exiting before you get the result in the OnResult or RequestAborted event.

Best Regards,
Danijel Tkalcec