Unicode

[Ronald van Tour]

Hi Danijel,

Is it possible to pass all asstring parameters in unicode  (UTF8).

[D.Tkalcec (RTC)]

It would be easier and safer to use "asText" for sending and receiving Unicode Strings in Delphi 2009 and later, but there are no restrictions imposed by the RTC SDK to what may or may not be inside an AnsiString sent using "asString" parameters, so ... if you wanted to, you could also manually encode UnicodeString or WideString into UTF-8 AnsiString on one end, send the UTF-8 encoded AnsiString using "asString" parameters and decode them back to UnicodeString or WideString on the other end.

The obvious gain of using "asText" is to have the transformation done for you automatically, but that only works if your strings are already in UnicodeString or WideString variables. On the other hand, if your source is an actual file encoded in UTF-8, sending it over using "asString" parameters is a better solution than converting already encoded UTF-8 data into Unicode- or Wide-Strings in memory before sending it over.

I hope that answers your question. If not, please give me more details about what it is exactly you want to do.

Best Regards,
Danijel Tkalcec

[Ronald van Tour]

Unfortunately sending the data astext did not work out for me.
My text allready was in UTF I still had to use UTF8Encode and UTF8Decode.

My code is basically like this:

astext['test']:=UTF8Encode(memo1.text);

This works.

[D.Tkalcec (RTC)]

Which Delphi version are you using?

Best Regards,
Danijel Tkalcec

[Ronald van Tour]

For this project I use D2009 ( also tested on D2010 )

[Ronald van Tour]

I have a UTF8 encoded txt file.

Code is like this:

  memo1.lines.loadfromfile('C:\test.txt',TEncoding.UTF8);

[D.Tkalcec (RTC)]

Try this ...

var
  myVar:TRtcValue;
  myUTF8:AnsiString; // UTF-8 Encoded AnsiString
  myUnicode:String; // = UnicodeString in D2009+
  myCode:AnsiString;

begin
// Load File content into memory
myUTF8 := Read_File('C:\test.txt');
// Decode UTF-8 AnsiString to Unicode String
myUnicode := UTF8Decode(myUTF8);

// Store Unicode String inside "asText"
myVar := TRtcValue.Create;
myVar.asText := myUnicode;

// Serialize TRtcValue object
myCode := myVar.toCode;
// Destroy the original
myVar.Free;

// Recreate TRtcValue object
myVar := TRtcValue.FromCode(myCode);

// Directly assign Unicode String to Memo.Text
memo1.Text := myVar.asText;

// Destroy the variable
myVar.Free;
end;

Do you see the correct Unicode text inside the Memo?

[Ronald van Tour]

It clears my memo.

[D.Tkalcec (RTC)]

Can you send me the file you are using for testing, please?

Send the file to:

[D.Tkalcec (RTC)]

Thanks for sending me the file. I can see now why the memo clears. The file you are using is stored in native Unicode format and NOT in UTF-8 format. Loading the file into Memo using the following code will also clear it up:

memo1.lines.loadfromfile('C:\test.txt',TEncoding.UTF8);

You can see the actual file encoding if you open the file in Notepad (standard Windows Text File Editor) and select "File/Save As...", there is a drop-down list "Encoding" left of the "Save" button.

[Ronald van Tour]

Ah verry nice.

At the serverside a save files using savetofile('c:\test.txt',Tencoding.UTF8)
and then read it with a rtc function
I asume that the .txt is now UTF8 right?

But when I load that file into a astext parameter without using UFT8Encode I dont get the right characters at the client side.
I use the code like this :
x:=Tstringlist.create;
x.loadfromfile('c:\text.txt',Tencoding.UTF8);
astext['test']:=x.text;

I know the stringlist.text is a string value and not ansistring.
When is say for example:
memo1.text:=x.text (x is the stinglist)
I still see the russian characters.

Is that a problem?

[Ronald van Tour]

Do I have to use something like this instead of the stringlist?

var
  myUTF8:AnsiString; // UTF-8 Encoded AnsiString
begin
  myUTF8 := Read_File('C:\test.txt');
 
////////////////////
  asstring['test']:=myUTF8;

[D.Tkalcec (RTC)]

Here is a modified example (from above), which would work for files stored in Unicode format:

var
  myVar:TRtcValue;
  myStr:TStrings;
  myUnicode:String; // = UnicodeString in D2009+
  myCode:AnsiString;
begin
myStr := TStringList.Create;
try
  // Load Unicode File into TStrings
  myStr.LoadFromFile('C:\test.txt',TEncoding.Unicode);
  // Assign file contents to Unicode String
  myUnicode := myStr.Text;
finally
  myStr.Free;
end;

// Store Unicode String inside "asText"
myVar := TRtcValue.Create;
myVar.asText := myUnicode;

// Serialize TRtcValue object
myCode := myVar.toCode;
// Destroy the original
myVar.Free;

// Recreate TRtcValue object
myVar := TRtcValue.FromCode(myCode);

// Directly assign Unicode String to Memo.Text
memo1.Text := myVar.asText;

// Destroy the variable
myVar.Free;
end;

Let me know if that example gives you the correct Unicode text inside the Memo when you use the file you've sent me by E-Mail (which is Unicode encoded). If the file is UTF-8 encoded, simply change the encoding parameter in the LoadFromFile method.

Best Regards,
Danijel Tkalcec

[D.Tkalcec (RTC)]

Using UTF8Encode() manually on a Unicode String before it is assigned to the asText parameter is wrong.

What I'm wondering now is WHY you are saving a Unicode String into a file on the Server side if all you want to do is send it out? Is there a reason why you aren't simply assigning the Unicode String to the asText parameter when you get it, without going through a file?

[Ronald van Tour]

Your example works.

I have a file with unicode chars in it.
I save and load this file on the server.

I have a function to save the file on the server. And I have a function to read it back from the server.

When I do not use the UTF8Encode/Decode it does not work.
That must mean that my string is not unicode?

Lets look at this (part of your example):

myStr := TStringList.Create;
try
  // Load Unicode File into TStrings
  myStr.LoadFromFile('C:\test.txt',TEncoding.Unicode);
  // Assign file contents to Unicode String
  myUnicode := myStr.Text;

You assign the myStr.Test to myUnicode.
Do you mean it is not possible to assign myStr.text directly to myVar.asText ?