Streaming Audio from Computer as TMemoryStream Object thru RTC Gateway failure

[mikotron]

Hi, I'm trying to send audio recordings from my computer to other thru RTC Gateway in a TMemoryStream Object using RTC Gate Client Link 'SendBytes' method.

I'm using this to convert audio from TMemoryStream to RtcByteArray:

StreamToByteArray(Stream: TMemoryStream): RtcByteArray;
begin
  // Check stream
  if Assigned(Stream) then
  begin
     // Reset stream position
      Stream.Position:=0;
     // Allocate size
    SetLength( result, Stream.Size);
    Stream.Read (result[0], Stream.Size);

  end
  else
     // Clear result
     SetLength(result, 0);

end;

OnDataReceived I'm using this to convert from RtcByteArray to TMemoryStream :

ByteArrayToStream(bt: RtcByteArray):TMemoryStream;
var streaming: TMemoryStream;
begin

streaming:= TmemoryStream.Create;
streaming.size:= length(bt);
streaming.writebuffer(bt[0], length(bt));
streaming.Position:=0;
result:= streaming;
end;

So when I try to reproduce TMemoryStream variable on receiver computer:

option 1:

PlaySound(memstream.memory,0, SND_Memory or SND_ASYNC);

option 2:

sndPlaySound(memstream.memory, SND_NODEFAULT or SND_ASYNC or SND_LOOP);
 

is not playing it.

Could you please let me know what's proper way to send audio and receiving it thru RTC Gateway components ?

Regards!

[D.Tkalcec (RTC)]

The functions you are using to convert a RtcByteArray to a TMemoryStream and back look fine to me, but ...

1. SendBytes method is limited to 16 MB. If you need to send more, you will have to split it up.

2. RTC Gateway splits all commands into even smaller chunks when streaming. The OnDataReceived event on the Client will be called for every single chunk of data received and not only once for each SendBytes call. You need to combine these chunks on the Client, either by using the built-in buffer in the TRtcGateClient component, or by writing data into your Stream as it arrives.

Best Regards,
Danijel Tkalcec

[mikotron]

Thank you for your Feedback Danijel, a couple more questions:

1.- Could you please tell me how can I combine these chucks on the Client side using built-in buffer in the TRtcGateClient component?
2.- Is this proper way to write data into my Stream from RTCGateClientLink Component?

Info:
       GMC = TRtcGateClientLink Component in Form
       memstream = TMemoryStream Global Variable

procedure TGateClientForm.DataReceived(Sender: TRtcConnection);
begin
 if GMC.Data.CallID= cid_Test then
 begin
  if GMC.Data.Header then
      GMC.Data.ToBuffer:= True;
 
  if GMC.Data.ToBuffer then
       memstream:= ByteArrayToStream(GMC.Data.Content);
 
  //if GMC.Data.Footer then
       //
 end;
end;

Regards!

[D.Tkalcec (RTC)]

When you get a Header with your CallID, check if you also have the Footer.
If you have the Footer, you have the complete content.
If you do NOT have the Footer, enable built-in buffers.

Here is the code:

procedure TGateClientForm.DataReceived(Sender: TRtcConnection);
  begin
  // do we have a Header with our "CallID" ?
  if (GMC.Data.CallID=cid_Test) and (GMC.Data.Header) then
    begin
    // do we also have the Footer?
    if GMC.Data.Footer then
      // Yes, we can access the complete content now (sent in one call to "SendBytes")
      memstream:= ByteArrayToStream(GMC.Data.Content);
    else
      // No, we should enable built-in buffers ...
      GMC.Data.ToBuffer:= True; // enable built-in buffers
    end;
  end;

PS. Since you are creating a new TMemoryStream instance in your ByteArrayToStream function, make sure to destroy it after use. Also, please keep in mind that the TRtcHttpGateClient component is always multi-threaded (there is no single-threaded mode), which means that all the events on the component will be executed from background threads. If you need to access the GUI from events triggered by the component, you can use the Sync method (available on the Sender object - passed as a parameter to all components events).

Best Regards,
Danijel Tkalcec

[mikotron]

Hi Danijel, thank you again for your help. I've tried to implement this feature with your suggestions and I would like to show you how I'm doing it and I still can't reproduce Audio Stream in Receiver computer.

Please see .gif's attached as follow:

Sender:
https://app.box.com/s/2gey39z6hb1nf02ykcb5qkld62hnion1

Receiver:
https://app.box.com/s/o2j35i4gpexlrswucmbfirgb188ye328

Can you please let me know if you detect something I've missed ?

Once again thank you for your help!!

[D.Tkalcec (RTC)]

I can not see much in these animated GIFs. If you can not post the source code here, it would be a lot easier for me to see what you are doing if you could send me the source code by E-Mail:


Best Regards,
Danijel Tkalcec

[mikotron]

Hi Danijel, Good I just sent you Sender, Gateway and Receiver Source code with current proof of concept in order to work with audio streaming thru RTC components. Subject of email is same than this Post.

Regards!!!

[D.Tkalcec (RTC)]

Mistery solved

To send a WAV File using the SendBytes method on the TRtcHttpGateClient component (provided the file is less than 16MB in size, since that is the limit for a single SendBytes method call), you can use something like this:

if GateClient.Ready then
  GateClient.SendBytes(MyReceiverID, 0, cid_MyTestCall, Read_FileEx(MyFileToSend));

* GateClient is a TRtcHttpGateClient component, set up and connected to the Gateway
* MyReceiverID is the UserID of the receiver
* cid_MyTestCall is the CallID you are using for this command
* MyFileToSend is the name of the WAV file you want to sent

To make that file play on the receiver side when it arrives by using the PlaySound API on Windows, you can write the OnDataReceived event for the TRtcHttpGateClient or a connected a TRtcGateClientLink component, like this:

  var
    Cli:TRtcHttpGateClient;
  begin
  Cli:=TRtcHttpGateClient(Sender.Owner);
  if (Cli.Data.CallID=cid_MyTestCall) and (Cli.Data.Header) then
    if Cli.Data.Footer then
      PlaySound( Addr(Cli.Data.Content[0]), 0, SND_MEMORY)
    else
      Cli.Data.ToBuffer:=True;
  end;

Naturally, playing the audio file directly from RTC event in blocking mode (as above) will also pause data reception on the receiver side (because audio is being played inside the event used to receive data from the Gateway), which (in turn) will pause data transmission to this Client on the Gateway and also pause data transmission from the Client sending the data.

If you do NOT want to block the receiver thread while playing the sound and you do not need seemless streaming, but just want any sound to be played immediately when it arrives, while stopping any sounds which might have been playing before, you can use a global RtcByteArray variable to store the content received and use the PlaySound API with in asynchronous mode, like this:

  var
    Cli:TRtcHttpGateClient;
  begin
  Cli:=TRtcHttpGateClient(Sender.Owner);
  if (Cli.Data.CallID=cid_MyTestCall) and (Cli.Data.Header) then
    if Cli.Data.Footer then
      begin
      PlaySound(NIL,0,0); // stop any other sounds playing, before changing the global sound variable
      GlobalSound:=Cli.Data.Content; // make a global copy of the audio file received
      PlaySound( Addr(GlobalSound

• ), 0, SND_MEMORY or SND_ASYNC); // play the global sound, asynchronously

     end
    else
      Cli.Data.ToBuffer:=True;
  end;

Using the event code above, a new "GlobalSound" variable has to be declared (somewhere in your unit) as a "RtcByteArray". This global variable is necessary for asynchronous execution of the PlaySound API, because the "Cli.Data.Content" variable will be cleared as soon as more data arrives.

Or ... if you want your Client to receive data from the Gateway as soon as possible, but do NOT want the currently playing sound to be stopped when a new file arrives (as was in the example above), for example - if you want seamless audio streaming, you can either implementing your own data queue and implement your own TThread class, or ... you can use the TRtcQuickJob component, which has its own data queue and utilizes the RTC Thread pool.

Simply drop one TRtcQuickJob component on your Form or DataModule, give it a name (for example: "SoundJob"), set its "Serialized" property to TRUE (which makes the execution of jobs serialized - one job at a time) and write its OnExecute event to play a sound using using the ByteArray receved as a parameter ...

  begin
  PlaySound(Addr(Data.asByteArray[0]),0,SND_MEMORY);
  end;

... then, modify the OnDataReceived event of your TRtcHttpGateClient or connected TRtcGateClientLink component (see code above) to post the content receive to the SoundJob component, instead of playing it directly, like this:

  var
    Cli:TRtcHttpGateClient;
  begin
  Cli:=TRtcHttpGateClient(Sender.Owner);
  if (Cli.Data.CallID=cid_TestFileSend) and (Cli.Data.Header) then
    begin
    if Cli.Data.Footer then
      SoundJob.Post(TRtcByteArray.Create(Cli.Data.Content))
    else
      Cli.Data.ToBuffer:=True;
    end;
  end;


Best Regards,
Danijel Tkalcec