Help... ISA Proxy problem

[zsleo]

I have a client app and server server both running RTC.

This app has been live for almost a year and processes approx 40k transactions a day. The transactions comprise RTC functions, ThinDAC/RTC DB processing and RTC file transfer.

All is working perfectly with client side auto-accessing 4 servers in round-robin fail-over. There are more than 200 client apps running over the public internet.

... you can see I am VERY happy with the RTC ...

Now for my problem.

When I switch on (ISA) proxy for the client app the file transfer (upload and down load) stops working BUT all RTC function interfaces and all ThinDAC/RTC DB processing works perfectly!

... even more strange is that somewhere along the comms path the proxy port is changed from port 8080 to port 80 with the proxy address NOT changing - this only happens when files are being transferred.

The scenarios I have tried are
-blocking & non-blocking
- multi-thread on & off

The client app proxy authentication is set up for "none" but I have tested with both "basic" and "integrated".

Help please...

ZSL
P.S. been at this for 2 days - now 1.30 am and I am going to take a break

[D.Tkalcec (RTC)]

Hi Zane,

1) Is file transfer not working at all, even for very small files (for example 10KB) or do problems start once you try sending or receiving large files (for example, a few MB or more)?

2) Is your problem with file download (get file from server), file upload (send file to server) or both?

3) Does the PROXY you are using have limitations in the size of request/response content it can handle, or is the proxy configured to let only a set amount of bytes through before it blocks?

4) What exactly happens on the Client side and what happens on the Server side when "file transfer does not work"? Does the Server get the request from the Client, or is the request blocked by the PROXY, or is the connection interrupted in the middle of transferring a file? What does the Client get?

You should log client-side and server-side events used for sending/receiving files and also log connection events on both sides to get more information about what exactly is happening.

Best Regards,
Danijel Tkalcec

[Andrew.W]

First of all, think long and hard before using ISA

Could this be flood prevention. Depending on how you've set your transfer parameters, it might be that there are so many requests flying back and forwards ISA says 'no'.

Other types of request are usually not so intense.

[zsleo]

Danijel

1) Is file transfer not working at all, even for very small files (for example 10KB) or do problems start once you try sending or receiving large files (for example, a few MB or more)?

typically the files are sub-10kB but can get as high as 1MB

2) Is your problem with file download (get file from server), file upload (send file to server) or both?

Both

3) Does the PROXY you are using have limitations in the size of request/response content it can handle, or is the proxy configured to let only a set amount of bytes through before it blocks?

No. Neither my internal test server nor the client's server

4) What exactly happens on the Client side and what happens on the Server side when "file transfer does not work"? Does the Server get the request from the Client, or is the request blocked by the PROXY, or is the connection interrupted in the middle of transferring a file? What does the Client get?

My test files are sub-1kB.
The file gets to the proxy server but appears not to pass through; the file does not get to the RTC server. I say "appears to" because I am changing our internal test RTC server to log more detailed info.

I am also setting up packet analysers on RTC client, proxy server and RTC server.

Zane

I will sent more details in a few hours.

[zsleo]

Andrew

While I agree with you. I have no control over what the customer uses.

The problem in happening at a government site...

Zane

[D.Tkalcec (RTC)]

If requests aren't reaching the Server, it is possible that the PROXY is configured to block the URL you are using for file upload/download.

Since remote functions are working, try changing your Client and Server to use the same "FileName" for upload/download which you are using as "ModuleFileName" for remote functions, passing the file name as a parameter inside the URI and setting the "CheckOrder" of your rtcDataProvider to a value smaller than the one you have on the rtcServerModule to make sure its OnCheckResuest event is triggered first.

You then have to change rtcDataProvider's OnCheckRequest event to ONLY Accept requests for which your filename property is set, letting all the other requests be handled by the rtcServerModule.

Best Regards,
Danijel Tkalcec

[zsleo]

It has been a week since the last update on this problem.

First: Danijel, thank you VERY much for your assistance in reviewing the captured transmission logs. Your advice and direction on alternate approaches is also appreciated.

The problem turned out to be heavily loaded proxy servers resulting in missed packets and out of sequence packets processed though the proxy server. It was easy to repeat the problem on a ISA proxy server. On a Squid proxy server it took a bit more doing but I did get to repeat it.

With all of my RTC tests, only the HTTP/S file transfer presented the failed file transmission problem.

I have re-written the app to replace the HTTP file transfer with HTTP byte stream. It is now in testing.

In my RTC projects I don't think I will ever use HTTP file transfer again! The coding for byte streaming (RAM contraints put to one side) is a lot more efficient & faster, vastly more simplified and, because of the RTC architecture, very, very elegant - even if I say so myself.

Danijel, I must tell you that as I was prototyping and then coding the solution there were periods when I was just thrilled at the simplicity and elegance that the RTC library delivered.

One last request. I now have a problem that more than 90% of the 1st, 2nd and 3rd connection attempts fail after the client exe is started. Once the first connection to the server is made everything works fine until the client is shut down and started again. Time between start up and attempt to connect has not effect - I have tried delays up to 4 minutes.

Thanks again

Zane

[D.Tkalcec (RTC)]

Depending on how loaded the Proxy is, your connection attempts will fail because the Proxy will be unable to accept new connections. Adding delays on the Client side before making the first connection attempt won't solve the problem, because the Client does not know how the Proxy is loaded before it makes a connection attempt. So ... the only thing you can do is re-try connecting until you succeed (as you are doing already).

If you want to reduce the load on the Proxy while connecting, you can use wait delays between connection attempts. Adding a delay on one client could improve the time another Client will need to connect, but it won't improve that clients chances of success, so you are free to decide whether you want to give other clients priority while connecting by using longer wait delays between each connect attempt, or if you want to force your way through by using no delays or very short wait delays between each connect attempt.

Best Regards,
Danijel Tkalcec

[zsleo]

thank you.

Zane