RTC File Client Demo: download of large file corrupted...

[ManfredPfeiffer]

Danijel,
I am using RTCHttpclient to download larger files (50MB). Unfortunatly the files are stored corrupted.
Smaller files pose no problem. I could replicate the problem with the RTC File Client Demo:
Server Port 443
Adress: ch.synadoc.ch
File Name: autsynadoc/synadoccd-6.7.2.exe
If you download through Firefox, the file arrives undamaged...
Do you have any insight?
Kind regards,
Manfred

[D.Tkalcec (RTC)]

Even though I do get a PING response from your Server, I can not get a HTTPS connection open to that Server from my location (Trogir, Croatia), which either means that your Server is not running at the moment, or that you have an IP address filter.

About HTTPS, in general ...

TRtcHttpClient components have 4 different options for using HTTPS on Windows:

1) You can use the WinInet API by setting the "UseSSL" property to TRUE. When using the File Client Demo, simply check the "useSSL" checkbox. Alternatively, since you are downloading a larger file, you can also check the "Multi-Threaded" checkbox to keep the Application responsive while downloading.

2) You can also use the WinHTTP API, for which (in addition to setting the "useSSL" property to TRUE), you need to set the "useWinHTTP" property to TRUE. When using the FIle Client Demo, this is done by checking the "WinHTTP" checkbox. Same as with the WinInet API, you can check the "Multi-Threaded" checkbox to keep the Application responsive while downloading.

3 and 4) You can also use the WinSock API, either in blocking mode (by setting the "Blocking" property to TRUE) or non-blocking mode (by leaving the "Blocking" property as FALSE). Because there is no SSL support in WinSock, a 3rd-party encryption component is required as a plug-in and should be assigned through the "CryptPlugin" property when using WinSock.

For the standard File Client demo from the RTC SDK to work using 3rd-party components for SSL encryption, you will need StreamSec components installed and the File Client demo compiled with the StreamSecII compiler directive. Alternatively, if you are using SecureBlackBox components from Eldos, you can contact Eldos for a similar demo prepared to work with their components. In either case, should you have any problems or questions related to SSL encryption with 3rd-party encryption components, please contact the 3rd-party encryption component vendor directly.

Best Regards,
Danijel Tkalcec

[ManfredPfeiffer]

Sorry Danijel,
I forgot to mention that though the server listens on port 443, it is plain HTTP - so no encryption.
I pinned down the problem to the write_file routine in rtcinfo.pas:
The opening and closing of the file seems to confuse the operating system (WIN 8.1), resulting in a corrupted download.
But please verify yourself...
Kind regards,
Manfred

[D.Tkalcec (RTC)]

If you think the problem is in the opening and closing of the file, you can change the OnDataReceived event to ...

A) wait for Response.Done before using the Read or ReadEx method to access the content body, in which case RTC will be using internal buffers to store the content until it arrives, then you can read the whole file with a single Read or ReadEx method call and write it down with a single Write_File or Write_FileEx procedure call.

B) use some other method to write the contents of your file to the disk.

If none of the above solves your problem, my assumption is that you have some anti-virus software messing with the file access API on Windows, trying to scan the contents of the file as it is being written down and prevent the file from running if it is potentionally harmful, or ... a firewall software messing up the content you are receiving.

Best Regards,
Danijel Tkalcec

[ManfredPfeiffer]

Thank you Danijel, for your suggestions.

I could solve the problem by buffering the file entirely and writing it in one chunk.
However it would be helpful for others running into the same problem to know, if your RTC File Cient Demo works correctly with large
files or if it needs to be modified.
As I wrote earlier the problem seems to be in your "write_file" routine, although I could not find anything wrong with it.
In any case would it be helpful, if your RTC File Client Demo could implement the case A) from your suggestion as an alternative.

Kind regards,
Manfred

PS: the test file url has changed to "autsynadoc/synadoccd-6.7.4.exe"

[D.Tkalcec (RTC)]

I've figured out what was causing this problem.

Windows doesn't release a lock on the file immediately after it is closed, because of which an attempt to open a file for writing, shortly after it has been open for writing and closed, can fail. The good news is that the Windows file access API used by "Write_File" and "Write_FileEx" functions does return with an error result and the "Write_File" as well as "Write_FileEx" functions will return FALSE if that happens, so the solution is to check the result of each "Write_File/Ex" function and try again (after a short delay) in case the result of the function is FALSE.

To avoid writing manual checks and loops everywhere the "Write_File" or "Write_FileEx" function is used, I've extended these functions in RTC SDK v7.26 with new "Retries" and "RetryDelayMS" parameters (both optional), which can be used to tell the Write function to automatically retry up to "Retries" number of times, using a delay of "RetryDelayMS" milliseconds between each new attempt in case file access operation fails. These parameters can be specified individually in each "Write_File" or "Write_FileEx" function call, or globally by setting the new RTC_WRITE_RETRIES (default=0) and RTC_WRITE_RETRY_DELAY (default=100) global variables, which are declared in the "rtcInfo.pas" unit.

Best Regards,
Danijel Tkalcec