Error receiving data

[ISIGest]

I try to receive from server a big record with multiple dataset inside.
All work with a relative small data, but if I try to receive a big data the function called abort.

I trace it with RTC_DEBUG enabled and in log I got this error:
"TRtcClientModule.DataReceived EXCEPTION Exception! ERtcInfo: Length value missing."

Can you help me please?

[D.Tkalcec (RTC)]

How much data are you talking about and how stable is the connection you are using? If the connection breaks while receiving data, the content received will be incomplete and then you will get errors like the one you've posted here.

If the connection should be stable (for example, when Client and Server are both inside LAN), then I need to see how the components are set up on the Client and the Server side (all components properties)  and I need the source code of the Client and the Server side, as well as the exact data package sent from one side and the data package received on the other side. Without this info, I can only guess what might be wrong. The error message you have posted means that the content being parsed is incorrect (length value is missing).

Best Regards,
Danijel Tkalcec

[ISIGest]

I found the problem.
The server raise an exception "Out of memory".
This is because the TRtcRecord (result param when function called) will be stored in memory.
There is a method to store it in a file and reply to client piecemeal?

[D.Tkalcec (RTC)]

As far as I know, Delphi does not have any options for working with objects from a disk drive. Since TRtcRecord is a Delphi object, it has to be in memory.

And now, I need to ask again: How much data are we talking about here?

Remote Functions can be used with data packages up to several MB in size, but if you are running out of RAM, your structure must be quite large.

The TRtcRecord structure will be released from memory immediately after the object is serialized and its contents moved to the buffer used for sending the response out. But during the serialization, about the same amount of RAM will be needed once for the serialization object and then again to put it all in the response buffers. So ... when working with remote functions, you actually need 3 times the ammount of RAM required for storing the object structure in memory (the other 2 copies will be automatically freed once the data is in the sending buffer). Naturally, the content whole sending buffer will be cleaned up from memory as it is being sent out, but it will have to stay in memory during the transfer.

If you need to send large amounts of data, you will either need to break down the communication into several remote function calls (like downloading "pages"), or you need to use the TRtcDataProvider component on the Server and TRtcDataRequest on the Client, which give you full control over the sending and receiving process, so you can prepare your response in a file and send it out in pieces (as you've suggested above).

Best Regards,
Danijel Tkalcec

[ISIGest]

Thank you for reply, have you an example to mix remote function calls and rtcdataprovider to send a file?

[D.Tkalcec (RTC)]

Sending of files from the Server to the Client is already explained in the Quick Start lessons and shown in the QuickStart example Projects. You can link any number of TRtcServerModule and TRtcDataProvider components to the same TRtcHttpServer component make the functionality accessible from the same port on the Server. All you need to take care of, is to respond to a different "Request.FileName" from each component, or use the "CheckOrder" property to define the ordr in which the OnCheckRequest events will be executed on the Server in case one component should be processing a subset of another components URIs.

Best Regards,
Danijel Tkalcec

[ISIGest]

I found my problem on server an solve sending the RtcDataSet as file with data provider.
Now the problem is on client: with a dataset with over 30k record and 30 fields when I load the TRtcDataSet

Code:

TRtcDataSet.FromCode(Read_File(myFile));

It occupy more of 400MB of memory

And this is not the biggest dataset to load.

Do you have any idea to solve this?
Thanks

[D.Tkalcec (RTC)]

I would never even think about sending 30K records containing 400 MB of data in a single request from the Server to the Client, but ... if you have a good reason for doing this, then I guess you also need to find a way to work with that data from a hard drive on the Client side, which means that you can NOT use the TRtcDataSet class for serialization on the Server side, since it was never designed for that purpose. From the RTC point of view, it makes absolutely no difference what you use for data serialization and deserialization. If you absolutely need to handle that kind of data in a single DataSet on the Client side, you will probably want to take a look at some local Database implementations for Delphi.

PS. Keep in mind that - if your Client needs 400 MB of RAM to work with that data using the TRtcDataSet class, the Server will need the same or more to serialize it from a TRtcDataSet instance and store it to the disk. So ... your problem is not only on the Client side, but also on the Server side. A single Client asking for data which requires 400 MB of RAM on the Server sounds like a bigger issue to me than a Client which requires 400 MB of RAM to work with it. I guess your Server will have to handle more than one Client at a time, so the RAM requirement is going to multiply on the Server.

Best Regards,
Danijel Tkalcec