"Last header intercepted with new header, before data sent out"

[mastinfo]

Delphi XE10.1 Update 2, RTC V8 RC 8, Android app that connects to a Windows service.

I have a sporadic exception with RTC: "Last header intercepted with new header, before data sent out"
I guss this is an RTC exception.
When the error happens i have to close the app or the exception won't stop.
The code runs a remote function the execute a query on a database (Nexudb) and returns the dataset.
The dataset is first copyed into a TrtcClientDataset and from this to an SQLite database.
The code is executed into an anonymous thread. During the execution some info is visualized into a wait form (FWaitForm)
Difficult to debug because it doesn't happens often enough.
What could be the cause ?
If it helps i have pasted the code below.
thanks


procedure Tmain_frm.myproc;
var
  sql:string;
  nsped_lette:integer;
  data_sped:Tdate;
  num_sped:integer;
  msgerr: String;
  nsped:Integer;
  codice_autista:string;
  SpedDset:TrtcClientDataset;
  PacchiSpedDset:TrtcClientDataset;
begin
  SpedDset:=TrtcClientDataset.Create(nil);
  PacchiSpedDset:=TrtcClientDataset.Create(nil);
  try
    // leggo codice autista da configuarzione
    codice_autista:=GetCfgParam(IDCFG_CODICE_AUTISTA);

    FWaitForm.SetupProgress(0, 0, 0, 'Connessione ad server in corso...');
    ConnectCrmServer;
    FWaitForm.SetupProgress(0, 0, 0, 'Connesso ad server');

    // trovo spedizioni da caricare per l'autista
    sql:='select s.data,s.numero,s.anno_doc_mag,s.cod_clifor_doc_mag,s.data_doc_mag,s.tipo_doc_mag,s.bollet_doc_mag,s.num_doc_mag,' + sLineBreak +
  'cl.codice as codice_cliente,cl.rag1,cl.rag2,cl.ind1,cl.ind2,cl.cap,cl.citta,cl.prov,cl.telefono,' + sLineBreak +
  'dc.cod_dest,dc.rag1 as rag1_dest,dc.rag2 as rag2_dest,dc.ind1 as ind1_dest,dc.ind2 as ind2_dest,dc.cap as cap_dest,dc.citta as citta_dest,dc.prov as prov_dest' + sLineBreak +
  'from sped s' + sLineBreak +
  'inner join tesdoc td on (td.anno=s.anno_doc_mag and td.cod_clifor=s.cod_clifor_doc_mag and cast(td.data_doc as DATE)=s.data_doc_mag and td.tipo_doc=s.tipo_doc_mag and td.bollet=s.bollet_doc_mag and td.numdoc=s.num_doc_mag)' + sLineBreak +
  'inner join clifor cl on (cl.codice=s.cod_clifor_doc_mag)' + sLineBreak +
  'left outer join destcli dc on (dc.codcli=td.cod_clifor and dc.cod_dest=td.cod_dest)' + sLineBreak +
  'where (s.in_consegna=false and s.codice_autista='+codice_autista+')' + sLineBreak +
  'order by s.data,s.numero';

    FWaitForm.SetupProgress(0, 0, 0, 'Invio query spedizioni');
    execute_remote_query('SELECT',sql,SpedDset,20000);

    // Salvo risultato su database sqlite
    TThread.Synchronize(
      TThread.CurrentThread,
      procedure
      begin
        if not(assigned(Fnuove_spedizioni)) then
          Fnuove_spedizioni:=TStringList.create
        else Fnuove_spedizioni.Clear;
      end);

    FWaitForm.SetupProgress(0, 0, SpedDset.recordcount, 'Lettura spedizioni...');

    dbc.StartTransaction;
    try
      nsped_lette:=0;
      nsped:=0;
      SpedDset.First;
      while not(SpedDset.Eof) do
      begin
        inc(nsped);

        FWaitForm.UpdateProgress(nsped,'%');

        with dbengine_md do
        begin
          data_sped:=SpedDset.FieldByName('data').AsDateTime;
          num_sped:=SpedDset.FieldByName('numero').AsInteger;

          // salvo nuove spedizioni
          if not(sped_tb.FindKey([data_sped,num_sped])) then
          begin
            // salvo cliente
            if clienti_tb.FindKey([SpedDset.FieldByName('codice_cliente').AsString]) then
              clienti_tb.edit
            else clienti_tb.Append;
            clienti_tb.FieldByName('codice').AsString:=SpedDset.fieldbyname('codice_cliente').AsString;
            clienti_tb.FieldByName('rag1').AsString:=SpedDset.fieldbyname('rag1').AsString;
            clienti_tb.FieldByName('rag2').AsString:=SpedDset.fieldbyname('rag2').AsString;
            clienti_tb.FieldByName('ind1').AsString:=SpedDset.fieldbyname('ind1').AsString;
            clienti_tb.FieldByName('ind2').AsString:=SpedDset.fieldbyname('ind2').AsString;
            clienti_tb.FieldByName('cap').AsString:=SpedDset.fieldbyname('cap').AsString;
            clienti_tb.FieldByName('citta').AsString:=SpedDset.fieldbyname('citta').AsString;
            clienti_tb.FieldByName('prov').AsString:=SpedDset.fieldbyname('prov').AsString;
            clienti_tb.FieldByName('telefono').AsString:=SpedDset.fieldbyname('telefono').AsString;

            try
              clienti_tb.Post;
            except
              clienti_tb.Cancel;
              raise;
            end;

            // salvo destinazione
            if destcli_tb.FindKey([SpedDset.FieldByName('codice_cliente').AsString,
                                   SpedDset.FieldByName('cod_dest').AsString]) then
              destcli_tb.edit
            else destcli_tb.Append;
            destcli_tb.FieldByName('codice_cliente').AsString:=SpedDset.fieldbyname('codice_cliente').AsString;
            destcli_tb.FieldByName('codice_dest').AsString:=SpedDset.fieldbyname('cod_dest').AsString;
            destcli_tb.FieldByName('rag1').AsString:=SpedDset.fieldbyname('rag1_dest').AsString;
            destcli_tb.FieldByName('rag2').AsString:=SpedDset.fieldbyname('rag2_dest').AsString;
            destcli_tb.FieldByName('ind1').AsString:=SpedDset.fieldbyname('ind1_dest').AsString;
            destcli_tb.FieldByName('ind2').AsString:=SpedDset.fieldbyname('ind2_dest').AsString;
            destcli_tb.FieldByName('cap').AsString:=SpedDset.fieldbyname('cap_dest').AsString;
            destcli_tb.FieldByName('citta').AsString:=SpedDset.fieldbyname('citta_dest').AsString;
            destcli_tb.FieldByName('prov').AsString:=SpedDset.fieldbyname('prov_dest').AsString;

            try
              destcli_tb.Post;
            except
              destcli_tb.Cancel;
              raise;
            end;

            sped_tb.Append;
            sped_tb.FieldByName('data').AsDateTime:=SpedDset.FieldByName('data').AsDateTime;
            sped_tb.FieldByName('numero').AsInteger:=SpedDset.FieldByName('numero').AsInteger;
            sped_tb.FieldByName('anno_doc_mag').AsString:=SpedDset.FieldByName('anno_doc_mag').AsString;
            sped_tb.FieldByName('cod_clifor_doc_mag').AsString:=SpedDset.FieldByName('cod_clifor_doc_mag').AsString;
            sped_tb.FieldByName('data_doc_mag').AsDateTime:=SpedDset.FieldByName('data_doc_mag').AsDateTime;
            sped_tb.FieldByName('tipo_doc_mag').AsString:=SpedDset.FieldByName('tipo_doc_mag').AsString;
            sped_tb.FieldByName('bollet_doc_mag').AsString:=SpedDset.FieldByName('bollet_doc_mag').AsString;
            sped_tb.FieldByName('num_doc_mag').AsInteger:=SpedDset.FieldByName('num_doc_mag').AsInteger;
            sped_tb.FieldByName('codice_dest').AsString:=SpedDset.FieldByName('cod_dest').AsString;
            sped_tb.FieldByName('consegna_effettuata').AsBoolean:=False;

            try
              sped_tb.Post;
            except
              sped_tb.Cancel;
              raise;
            end;


            // leggo i pacchi della spedizione
            sql:='select p.data_sped,p.num_sped,p.anno_pacco,p.id_pacco,p.anno_pancale,p.id_pancale,ro.codart,ro.metallo,ro.titolo,p.qta' + sLineBreak +
    'from pacchi_sped p' + sLineBreak +
    'inner join rigordcli ro on (ro.anno=p.anno_ord and' + sLineBreak +
    '                            ro.bloc=p.bloc_ord and' + sLineBreak +
    '                            ro.numero=p.num_ord and' + sLineBreak +
    '                            ro.riga=p.riga_ord)' + sLineBreak +
    'where (p.data_sped=DATE'+quotedstr(formatdatetime('yyyy-mm-dd',data_sped))+' and p.num_sped='+inttostr(num_sped)+')'+ sLineBreak +
    'order by p.data_sped,p.num_sped';
            esegui_query_remota('SELECT',sql,PacchiSpedDset,20000);

            PacchiSpedDset.First;
            while not(PacchiSpedDset.Eof) do
            begin
              pacchi_sped_tb.Append;
              pacchi_sped_tb.FieldByName('data_sped').AsDateTime:=PacchiSpedDset.FieldByName('data_sped').asdatetime;
              pacchi_sped_tb.FieldByName('num_sped').AsInteger:=PacchiSpedDset.FieldByName('num_sped').asInteger;
              pacchi_sped_tb.FieldByName('anno_pacco').AsString:=PacchiSpedDset.FieldByName('anno_pacco').AsString;
              pacchi_sped_tb.FieldByName('id_pacco').AsInteger:=PacchiSpedDset.FieldByName('id_pacco').asInteger;
              pacchi_sped_tb.FieldByName('anno_pancale').AsString:=PacchiSpedDset.FieldByName('anno_pancale').asString;
              pacchi_sped_tb.FieldByName('id_pancale').AsInteger:=PacchiSpedDset.FieldByName('id_pancale').asInteger;
              pacchi_sped_tb.FieldByName('codart').AsString:=PacchiSpedDset.FieldByName('codart').asString;
              pacchi_sped_tb.FieldByName('metart').AsString:=PacchiSpedDset.FieldByName('metallo').asString;
              pacchi_sped_tb.FieldByName('titart').AsFloat:=PacchiSpedDset.FieldByName('titolo').AsFloat;
              pacchi_sped_tb.FieldByName('pezzi').AsInteger:=PacchiSpedDset.FieldByName('qta').AsInteger;
              pacchi_sped_tb.FieldByName('prespunta').AsBoolean:=false;
              pacchi_sped_tb.FieldByName('consegna').AsBoolean:=false;

              try
                pacchi_sped_tb.Post;
              except
                pacchi_sped_tb.Cancel;
                raise;
              end;

              // Aggiorno pancali
              if pacchi_sped_tb.FieldByName('id_pancale').AsInteger>0 then
              begin
                if pancali_tb.FindKey([pacchi_sped_tb.FieldByName('anno_pancale').AsString,
                                       pacchi_sped_tb.FieldByName('id_pancale').AsInteger]) then
                  pancali_tb.edit
                else pancali_tb.Append;

                pancali_tb.FieldByName('anno').AsString:=pacchi_sped_tb.FieldByName('anno_pancale').AsString;
                pancali_tb.FieldByName('id').AsInteger:=pacchi_sped_tb.FieldByName('id_pancale').AsInteger;
                pancali_tb.FieldByName('codice_cliente').AsString:=SpedDset.FieldByName('codice_cliente').AsString;
                pancali_tb.FieldByName('codice_dest').AsString:=SpedDset.FieldByName('cod_dest').AsString;
                pancali_tb.FieldByName('data_sped').AsDateTime:=SpedDset.FieldByName('data').AsDateTime;
                pancali_tb.FieldByName('num_sped').AsInteger:=SpedDset.FieldByName('numero').AsInteger;
                pancali_tb.FieldByName('prespunta').AsBoolean:=false;
                pancali_tb.FieldByName('consegna').AsBoolean:=false;
                try
                  pancali_tb.Post;
                except
                  pancali_tb.Cancel;
                  raise;
                end;
              end;

              PacchiSpedDset.Next;
            end;

            inc(nsped_lette);


            TThread.Synchronize(
                TThread.CurrentThread,
                procedure
                begin
                  Fnuove_spedizioni.Add('DDT '+inttostr(SpedDset.FieldByName('num_doc_mag').AsInteger)+' del '+
                                        datetostr(SpedDset.FieldByName('data_doc_mag').AsDateTime)+
                                        ' '+SpedDset.FieldByName('rag1').asstring);
                end);

          end;
        end;

        SpedDset.Next;
      end;

      FWaitForm.SetupProgress(0, 0, SpedDset.recordcount, 'Commit');
      dbc.Commit;
    except
      FWaitForm.SetupProgress(0, 0, SpedDset.recordcount, 'Rollback');
      dbc.Rollback;
      raise;
    end;

    FWaitForm.SetupProgress(0, 0, SpedDset.recordcount, 'Aggiornamento server ...');
    sql:='update sped set in_consegna=true where in_consegna=false and codice_autista='+codice_autista;
    if not(execute_remote_query('UPDATE',sql,nil,20000)) then
      msgerr:='SI E'' VERIFICATO UN ERRORE NELLA FINALIZZAZIONE DELLA PROCEDURA'+#13+'RIPETERE LA PROCEDURA DI CARICO SPEDIZIONI.'
    else msgerr:='';
    FWaitForm.SetupProgress(0, 0, SpedDset.recordcount, 'Aggiornamento server completato');
  finally
    SpedDset.Free;
    PacchiSpedDset.Free;
  end;

  if nsped_lette=0 then
    Frisultato_procedura:='Nessuna nuova spedizione trovata.'+#13+msgerr
  else
     Frisultato_procedura:=msgerr;
end;


function Tmain_frm.execute_remote_query(tipo,sql:string;dset:TRtcClientDataset;timeout:integer):boolean;
begin
  result:=false;

  GetAppClientModule.rtcClientModule.Prepare('TestConnection');
  GetAppClientModule.rtcClientModule.execute(true,5);

  if GetAppClientModule.rtcClientModule.LastResult.isType=rtc_Null then
    raise Exception.Create('Esecuzione query remota test connessione: PARAMETRO DI RITORNO NON CORRETTO')
  else if GetAppClientModule.rtcClientModule.LastResult.isType=rtc_Exception then
    raise Exception.Create('Errore su server in eseuzione query test connessione: '+GetAppClientModule.rtcClientModule.LastResult.asException)
  else begin
    if GetAppClientModule.rtcClientModule.LastResult.isType=rtc_Boolean then
    begin
      result:=GetAppClientModule.rtcClientModule.LastResult.asBoolean;

      if not(result) then raise Exception.Create('Esecuzione procedura test connessione non corretto.');
    end
    else raise Exception.Create('Tipo risultato query test connessione non corretto');
  end;

  GetAppClientModule.rtcClientModule.Prepare('sqlquery');
  GetAppClientModule.rtcClientModule.Param['tipo']:=tipo;
  GetAppClientModule.rtcClientModule.Param['sql']:=sql;
  GetAppClientModule.rtcClientModule.Param['timeout']:=timeout;
  GetAppClientModule.rtcClientModule.execute(true,15);

  if GetAppClientModule.rtcClientModule.LastResult.isType=rtc_Null then
    raise Exception.Create('Esecuzione query remota: PARAMETRO DI RITORNO NON CORRETTO')
  else if GetAppClientModule.rtcClientModule.LastResult.isType=rtc_Exception then
    raise Exception.Create('Errore su server: '+GetAppClientModule.rtcClientModule.LastResult.asException)
  else begin
    if GetAppClientModule.rtcClientModule.LastResult.isType=rtc_Record then
    begin
      result:=GetAppClientModule.rtcClientModule.LastResult.asRecord.asValue['ok'];

      if assigned(dset) then
      begin
        TThread.Synchronize(
          TThread.CurrentThread,
          procedure
          begin
            if dset.Active then dset.Active := False;
            RtcDataSetFieldsToDelphi(GetAppClientModule.rtcClientModule.LastResult.asRecord.asDataSet['dset'],dset );
            dset.CreateDataSet;
            RtcDataSetRowsToDelphi(GetAppClientModule.rtcClientModule.LastResult.asRecord.asDataSet['dset'],dset );
            dset.Active := True;
          end);
      end;
    end
    else raise Exception.Create('Tipo risultato query non corretto');
  end;
end;

[D.Tkalcec (RTC)]

The exception you are getting means that you are trying to send a new request before a complete response has been received for the last request sent through the connection component. This is most likely caused by your implementation, which (according to your statement) is using the "Execute" method on the "TRtcClientModule" component from a background thread.

Another possible problem source could be the use of the "TRtcClientModule" component from multiple threads at the same time, without setting its "HyperThreading" property to TRUE, which is required to ensure that each thread will get a separate instance of the "Data" object used to prepare a remote function call.

But ... even setting "HyperThreading:=TRUE" won't help if you try to use the Execute method from more than one thread, because the "Execute" method is emulating blocking behavior by implementing a message queue, uses a single internal TRtcResult instance and a single "LastResult" object.

If you want your code to run reliably and from multiple threads, avoid using the "Execute" or the "WaitForCompletion" methods. Instead, you should implement events on RTC components to handle responses and results received from the Server.

Also, instead of creating anonymous threads, you should use the TRtcHttpClient component with MultiThreaded=TRUE and handle everything you can from inside events triggered by RTC components, which will be executed from background threads. And if you need to access the GUI, you can always use the Sender.Sync method from any event triggered by a RTC component to call it synchronized with the Main Thread.

Best Regards,
Danijel Tkalcec

[mastinfo]

I guess i have also to set true the MultiThreaded property of the TrtcHTTPClient object. Correct ?

Do you have others general suggestions for me on how to setup RTC objects (client and server) for this kind of appplication ? A service application  that provides data to clients.

thanks

[D.Tkalcec (RTC)]

As a general rule when working with RTC components (Client and Server), it is best to avoid any kid of blocking calls and implement everything as events. If you are not a fan of event-driven design (it does split your code in "pieces"), you can use anynomous methods to write all events for RTC components, provided you are using Delphi 2010 or newer (which is probably true in your case - or you wouldn't be mentioning "Android").

When writing mobile Applications, you should avoid keeping the Main Thread busy for too long anyway, because your Application could get terminated by the OS. This is true for Android as well as iOS. You can use TRtcHttpClient and TRtcHttpServer components with MultiThreaded=TRUE and keep most of your code inside RTC events to achieve this. Use the Sync method only if you need to execute code from the Main Thread. For example, if you need to access any GUI elements.

And ... while implementing everything as events, you could end up wanting to call a remote function from a RTC event. If you do, make sure to pass the Sender:TRtcConnection component as a parameter to the method being called, or your 2nd call could start going out before the 1st one has finished.

Best Regards,
Danijel Tkalcec

[mastinfo]

Hello, thanks  a lot for your suggestions.

Seems i have solved. I have set to True the properties TrtcHTTPCLient.Multithreaded and TrtcClientModule.HyperThreading
I have also found a bad mistake in the server: in the onstart event of the service i created the data module with rtc components but i didn't free it in the onstop event.
After many test i haven't seen a single error (so far).

[D.Tkalcec (RTC)]

Setting HyperThreading to TRUE solves the problem of using the same TRtcClientModule from multiple threads to prepare and post a remote function call, which should fix the exception you were getting earlier, but ... because there is only one LastResult object per TRtcClientModule component, you still have to avoid using the Execute method and/or the LastResult object from more than one thread at a time, or you could easily end up processing the wrong Result or getting Access Violations while using the LastResult object from one Thread when another Thread has already called Execute and auto-freed the LastResult object created in the first Thread.

In short, if you want to use a single TRtcClientModule component from more than one thread at a time, you should NOT use the Execute method or the LastResult property of that component, or you will (eventually) end up with Access Violations and/or other kinds of problems.

Best Regards,
Danijel Tkalcec

[mastinfo]

Actually i don't have multiple threads running at the same time. There's only one.
I use a thread because i want to display some data while the procedure is running
I don't think there's another way in Android (can't use application.processmessages)


FWaitForm := TWaitForm.CreateAndRun(
    'PLEASE WAIT',

    procedure(AWaitForm: TWaitForm)
    begin
       do_the_work; // this is my procedure executed into an anomymous thread. It updates a progress bar and display info in the FWaitForm
    end,

    procedure(AWaitForm: TWaitForm)
    begin
      if not AWaitForm.HadErrors then
      begin
        // the procedure has been complited with no errors
      end;
    end,

    procedure (AWaitForm: TWaitForm; AException: Exception)
    begin
      // the procedure rised some exception
    end);
end;

[D.Tkalcec (RTC)]

When working with RTC components, if you implement your code by using RTC events, you can use the Sync method at anytime to execute parts of your code from the Main Thread. As an alternative, use the PostQuickJob procedure to post data for execution in any other thread or the Main Thread.

If you need an example on using the PostQuickJob procedure to update elements of the GUI from RTC events running in background threads, download RTC SDK v8.00 rc10 and take a look at Web Socket Client and Server examples in the QuickStart folder. In these examples, I am creating a virtual RTC Thread and using the PostQuickJob procedure to add new text in a Memo. Even though this is a VCL example (so it will only compile for Windows), all RTC-related code I'm using there works the same on all platforms.

Best Regards,
Danijel Tkalcec

[mastinfo]

Thanks, i'll take a look

Roberto