Copying files

[mastinfo]

Hello, i'm new to RTC components 

I'm trying to copy files between a server and a client.
I have created both server and client applications using code from the demos and the classroom examples.
I'm close to do the task. My problem is that the destination files are created with size 0.

The code i'm using is shown below. Any idea what could be the problem ?

thanks 
Roberto
Delphi XE2, Windows 7
RTC ver. 6.38

Server side:

var
  MAX_SEND_BLOCK_SIZE:int64=1460*44;
  MAX_ACCEPT_BODY_SIZE:int64=128000;

// TRCTDATAPROVIDER events --------------------------------------------

procedure Trtc_server_dm.rdp_filesCheckRequest(Sender: TRtcConnection);
var
  vsFileName : string;
begin
  with TRtcDataServer(Sender) do
  begin
    vsFileName := Request.FileName;
    Accept;
    Request.Info['fname'] := vsFileName;
    Response.ContentLength:=File_Size(vsFileName);
    WriteHeader;
  end;
end;


// event assigned to ONDATARECEIVED and ONDATASENT
procedure Trtc_server_dm.rdp_filesDataReceived(Sender: TRtcConnection);
var
  vsFileName  : string;
  viSent      : integer;
begin
  with TRtcDataServer(Sender) do
    if Request.Complete then
    begin
      if Response.ContentLength > Response.ContentOut then
      begin
        vsFileName := Request.Info['fname'];

        if File_Exists(vsFileName) then
        begin
          if File_Size(vsFileName) = Response.ContentLength then
          begin
            viSent := Response.ContentLength - Response.ContentOut;

            if viSent > 16000 then viSent := 16000;

            Write(Read_File(vsFileName, Response.ContentOut, viSent) );
          end
          else
            Disconnect;
        end
        else
          Write('File not found on server: ' + vsFileName);
      end;
    end;
end;



CLIENT SIDE:

// TRCTDATAREQUEST EVENTS ---------------------------------------------------

procedure Trtc_client_dm.datarequestBeginRequest(Sender: TRtcConnection);
begin
  with TRtcDataClient(Sender) do
  begin
    WriteHeader;
    // No Body.
  end;
end;

procedure Trtc_client_dm.datarequestDataReceived(Sender: TRtcConnection);
var
  s:RtcString;
begin
  with TRtcDataClient(Sender) do
  begin
    if not inMainThread then
      Sync(datarequestDataReceived)
    else begin
      if Response.Started then
      begin
        if Request.info.asText['fname']<>'' then
        begin
          if not DirectoryExists(ExtractFilePath(ExpandFileName(Request.info.asText['fname']))) then
            ForceDIrectories(ExtractFilePath(ExpandFileName(Request.info.asText['fname'])));
          Delete_File(request.Info.asText['fname']);
        end;
      end;

      s:=Read;

      if Request.Info.asText['fname']<>'' then
        Write_File(Request.Info.asText['fname'], s, Response.ContentIn-length(s))
      else if Request.Info.asString['data']='' then
        Request.Info.asString['data']:=s;
    end;
  end;
end;


// I'm using this procedure to copy a file from remote server to local
// datarequest is a TRCTDATAREQUEST object.
procedure Trtc_client_dm.copy_file(fi,fo:string);
// fi=remote file name
// fo=local file name
begin
  with datarequest do
  begin
    with Request do
    begin
      Method:=RtcString('GET');
      FileName:=fi;
      Query.Text:=RtcString('');
      Info.asText['fname']:=fo;
    end;

    Post;
  end;
end;

[D.Tkalcec (RTC)]

First, you can not simply assign a complete local file name with a full path to the "Request.FileName" property on the Client side and send it like that to the Server. HTTP communication does not work like that. If you really want to send a complete file name with its fully qualified path to the Server as part of the URL, the least you have to do is URL_Encode on the Client side and use some form of request identification as a prefix, so you can separate file download requests on the Server from other kinds of requests. For example ....

On the Client side, change the copy_file function to this:

Code:

procedure Trtc_client_dm.copy_file(fi,fo:string);
begin
  with datarequest do
  begin
    with Request do
    begin
      Method:='GET';
      FileName:='/download';
      Query.Text:=URL_Encode(fi);
      Info.asText['fname']:=fo;
    end;
    Post;
  end;
end;

This will send a request with the URI "/download" containing the remote file name in the query part of the URL. On the Server side, to decode the file name correctly, change OnCheckRequest event to something like this:

Code:

procedure Trtc_server_dm.rdp_filesCheckRequest(Sender: TRtcConnection);
var
  vsFileName : string;
begin
  with TRtcDataServer(Sender) do
  begin
    if Request.FileName='/download' then // only handle "download" requests ...
      begin
      vsFileName := URL_Decode(Request.Query.Text);
      Accept;
      Request.Info['fname'] := vsFileName;
      Response.ContentLength:=File_Size(vsFileName);
      WriteHeader;
      end;
  end;
end;

Please note that I've only changed the way how the name of the remote file is being transferred from the Client to the Server, but you should do a lot more than this if you want to keep the Server safe, because providing this kind of an API on a Server would allow anyone to download anything from the Server, without any kind of authentication.

Best Regards,
Danijel Tkalcec

[mastinfo]

Thank you Danijel   

I understand my mistakes.

Roberto

[mastinfo]

Hello

Since now i was doing the file copy having both client and server in the same pc. it worked nicely.
Now i'm trying to do the same thing having the server and the client in different PC of the lan.
Unfortunately i'm having odd problems and can't figure out what is the mistake.

I'll ty to explain.

In the client side the copy process start and SEEMS to work but:

1) it's a bit slow. Takes about 45 seconds to copy all the files (200 mb)
2) some files are .exe and, after the copy process is completed, the created files can't be executed. Windows tell me they are not valid exe. The file size is equal to the original one. Others binary files (i have some .wav files) can be opened.
3) The last (and very strange thing) is that the copy lasts forever if first i don't open in Windows Explorer the folder when the files are copied. Only when i browse the folder the process is completed in the time i have said at point 1

Roberto

[D.Tkalcec (RTC)]

To make sure the binary data is not corrupted, I would recommend using ReadEx and WriteEx methods for reading and sending data, which work with byte arrays instead of Strings (which could result in problems with different encoding. You can use Read_FileEx and Write_FileEx functions for file access.

I don't know why you would need to open a folder in Windows Exporer before you can access the files. One explanation might be that your process from which you are trying to access the files does not have sufficient access rights to that folder, or to files in that folder. But since the RTC SDK does require nor change any access rights to your file system, I'm not really sure how I can help you with that.

Best Regards,
Danijel Tkalcec

[mastinfo]

Ok i have the solution of the mistery 
The problem is the Antivirus software 
Seems that Microsoft Security Essentials interferes with the copy process.
Excluding my exe from the Antivirus live check solved the problem. The copy process now takes 15 seconds that is fast enough.
Fortunately this antivirus is not very popular.
Another client running Norton Antivirus Security shows a different (more tollerant) behaviour: if the folder where i'm copying files is opened in Explorer... well .. it doesn't like to mr Norton. The exe is damaged/removed.
But if the folder is not opened in Explorer nothing bad happens.
I'm wondering what happens with Others Antivirus. Anyway if adding the app. exe to the exclusion list solve the problem ... it's not a problem.

Roberto

[D.Tkalcec (RTC)]

Thank you for your feedback. This is very useful information for others, who might experience similar problems. I have also had issues with some Antivirus Software at customers with the transfer process, but forgot about it. Disabling the Antivirus Software or adding the EXE to exclusion list solved the problem there, too.

Best Regards,
Danijel Tkalcec

[mastinfo]

Seems i can avoid antivirus problems if i change the OnDataReceive event as follow.
With this solution i use a Tmemorystream to save the data and only at the end data is saved into a phisical file.
This could be a problem if files are very large but it's not my case (total 200mb and larger is 100mb)
The code seems to work but maybe i'm missing something. Do you see something i have to change ?

thanks

Roberto 

procedure Trtc_client_dm.datarequestDataReceived(Sender: TRtcConnection);
var
  s:rtcBytearray;
begin
  with TRtcDataClient(Sender) do
  begin
    if not inMainThread then
      Sync(datarequestDataReceived)
    else begin
      if Response.Started then
      begin
         if not DirectoryExists(ExtractFilePath(ExpandFileName(Request.info.asText['fname']))) then
           ForceDIrectories(ExtractFilePath(ExpandFileName(Request.info.asText['fname'])));

         Fms.clear;
      end;

      s:=Readex;

      Fms.Write(s[0],length(s));

      if response.Done then
        Fms.SaveToFile(Request.Info.asText['fname']);
    end;
  end;
end;

[D.Tkalcec (RTC)]

Since RTC components already have a built-in receiving buffer for requests and responses, if you want to keep the whole content in memory until you get all of it, then write it all to disk at once, you don't need a separate Memory Stream. You can do it like this:

procedure Trtc_client_dm.datarequestDataReceived(Sender: TRtcConnection);
  var
    s:RtcByteArray;
    fname:String;
  begin
  with TRtcDataClient(Sender) do
      if Response.Done then
        begin
        fname:=ExpandFileName(Request.info.asText['fname']);
        if not DirectoryExists(ExtractFilePath(fname)) then
          ForceDirectories(ExtractFilePath(fname));
        Write_FileEx(fname, ReadEx);
        end;
  end;

If, for what ever reason, you want to use a Memory Stream and fill it up as the content commes in instead of using the built-in RTC buffers, you can do it like this:
 
procedure Trtc_client_dm.datarequestDataReceived(Sender: TRtcConnection);
  var
    s:rtcBytearray;
    fname:String;
  begin
  with TRtcDataClient(Sender) do
    begin
    if Response.Started then
      Fms.Clear;
    s:=ReadEx;
    Fms.Write(s[0],length(s));
    if Response.Done then
      begin
      fname:=ExpandFileName(Request.info.asText['fname']);
      if not DirectoryExists(ExtractFilePath(fname)) then
           ForceDirectories(ExtractFilePath(fname));
      Fms.SaveToFile(fname);
      Fms.clear;
      end;
    end;
  end;

Best Regards,
Danijel Tkalcec

[mastinfo]

Your first example is ok.

thanks

Roberto