Encryption and Compression

[Ronald van Tour]

Hi Danijel,

We have a program where users save data as plain text files on the server.
( many plain text files).

Is there a way to store the file directly encrypted and compressed on the server side with the use of RTC components.

[D.Tkalcec (RTC)]

Yes, there is.

For encryption and decryption, you can use "Crypt" and "Decrypt" procedures from the "rtcCrypt.pas" unit. For compression and decompression, you can the "ZCompress_Str" and "ZDecompress_Str" functions from the "rtcZLib.pas" unit. And for reading and writing files as AnsiString, you can use the "Read_File" and "Write_File" functions in the "rtcInfo.pas" unit. Here are two example procedure implementations. "rtcWritePacked" for compressing, encrypting and writing the content "packed" to a file, and then you can use the "rtcReadPacked" function to get the original content (decrypted and uncompressed) back into memory:

uses
  rtcInfo, // Read_File & Write_File
  rtcCrypt, // Crypt & Decypt
  rtcZLib; // ZCompress_Str & ZDEcompress_Str

procedure rtcWritePacked(const FileName:String; const Key:AnsiString; const Content:AnsiString);
  var
    PackedContent:AnsiString;
  begin
  // For compression to be effective, we need to compress our Content before encrypting it  ...

  { "ZCompress_Str" function returns compressed content as a result (AnsiString) }
  PackedContent := ZCompress_Str(Content, zcDefault); // using default compression strength

 { "Crypt" procedure encrypts content "in place", so the variable "PackedContent" will contain the result }
  Crypt(PackedContent, Key);

  { Write the resulting Packed Content to the file "FileName" }
  Write_File(FileName, PackedContent);
  end;  

function rtcReadPacked(const FileName:String; const Key:AnsiString):AnsiString;
  var
    PackedContent:AnsiString;
  begin
  { Read compressed and encrypted content }  
  PackedContent := Read_File(FileName);

  // Content has been compressed and then encrypted before being written,
  // so we need to do it in reverse order when reading (first decrypt, then decompress) ...

 { "Decrypt" procedure decrypts content "in place",
    so the variable "PackedContent" will also contain the result }
  Decrypt(PackedContent, Key);

  { "ZDecompress_Str" function returns decompressed content as a result (AnsiString) }
  Result := ZDecompress_Str(PackedContent);
  end;

If your content is in a WideString or Unicode String ("String" type in Delphi 2009 and later, "WideString" in any Delphi versions from Delphi 6 - XE2), you can use "Utf8Encode" and "Utf8DEcode" functions to get that Wide/Unicode String to and from a UTF-8 encoded AnsiString. "Utf8Encode" function will convert a WideString into an UTF-8 encoded AnsiString. And to get the AnsiString back into a WideString, you can use the "Utf8Decode" function. For example, the functions below accept WideString Key and Content parameters instead of AnsiString Key and Content:

procedure rtcWritePackedWide(const FileName:String; const Key:WideString; const Content:WideString);
  begin
  // Content and Key need to be encoded from WideString to AnsiString
  rtcWritePacked(FileName, Utf8Encode(Key), Utf8Encode(Content));
  end;

function rtcReadPackedWide(const FileName:String; const Key:WideString):WideString;
  begin
  // Key needs to be encoded from WideString to AnsiString, but the resulting
  // Content has to be Decoded from AnsiString back to WideString
  Result:= Utf8Decode( rtcReadPacked(FileName, Utf8Encode(Key)) );
  end;

Best Regards,
Danijel Tkalcec

[Ronald van Tour]

Thanks.
Easy to use.

[Ronald van Tour]

Danijel,

Can you explain how to use the new ZDecompress_Ex.

We have this code:
  Result := Utf8Decode(ZDecompress_str(PackedContent));

So we tried to use :
  Result := Utf8Decode(ZDecompress_Ex(PackedContent));

But this is not the correct way.

[D.Tkalcec (RTC)]

It depends on what "PackedContent" and "Result" are. If PacketContent is a RtcByteArray and Result is a String or a WideString, then you should use this:

Result := Utf8DecodeEx(ZDecompress_Ex(PackedContent));

All methods and functions with the "Ex" ending work with a RtcByteArray (array of bytes) for binary content, instead of the the old methods and procedures which have been using AnsiString.

Best Regards,
Danijel Tkalcec

[D.Tkalcec (RTC)]

Here is the same code as above, but working with RtcByteArray instead of AnsiString, so it is compatible with the nextgen compiler for mobile devices. Because the NextGen compiler does not have AnsiString support, "Ex" functions and methods are used, which use RtcByteArray instread of AnsiString for all binary content.

For encryption and decryption, you can use "CryptEx" and "DecryptEx" procedures from the "rtcCrypt.pas" unit. For compression and decompression, you can the "ZCompress_Ex" and "ZDecompress_Ex" functions from the "rtcZLib.pas" unit. And for reading and writing files as RtcByteArray, you can use the "Read_FileEx" and "Write_FileEx" functions in the "rtcInfo.pas" unit. Here are two example procedure implementations. "rtcWritePackedEx" for compressing, encrypting and writing the content "packed" to a file, and then you can use the "rtcReadPackedEx" function to get the original content (decrypted and uncompressed) back into memory:

uses
  rtcInfo, // Read_FileEx, Write_FileEx, Utf8EncodeEx & Utf8DecodeEx
  rtcCrypt, // CryptEx & DecyptEx
  rtcZLib; // ZCompress_Ex & ZDecompress_Ex

procedure rtcWritePackedEx(const FileName:String; const Key:RtcByteArray; const Content:RtcByteArray);
  var
    PackedContent:RtcByteArray;
  begin
  // For compression to be effective, we need to compress our Content before encrypting it  ...

  { "ZCompress_Ex" function returns compressed content as a result (RtcByteArray) }
  PackedContent := ZCompress_Ex(Content, zcDefault); // using default compression strength

 { "CryptEx" procedure encrypts content "in place", so the variable "PackedContent" will contain the result }
  CryptEx(PackedContent, Key);

  { Write the resulting Packed Content to the file "FileName" }
  Write_FileEx(FileName, PackedContent);
  end;  

function rtcReadPackedEx(const FileName:String; const Key:RtcByteArray):RtcByteArray;
  var
    PackedContent:RtcByteArray;
  begin
  { Read compressed and encrypted content }  
  PackedContent := Read_FileEx(FileName);

  // Content has been compressed and then encrypted before being written,
  // so we need to do it in reverse order when reading (first decrypt, then decompress) ...

 { "DecryptEx" procedure decrypts content "in place",
    so the variable "PackedContent" will also contain the result }
  DecryptEx(PackedContent, Key);

  { "ZDecompress_Ex" function returns decompressed content as a result (RtcByteArray) }
  Result := ZDecompress_Ex(PackedContent);
  end;

If your content is in a WideString or a Unicode String ("String" type in Delphi 2009 and later), you can use "Utf8EncodeEx" and "Utf8DecodeEx" functions to get that Wide/Unicode String to and from a UTF-8 encoded RtcByteArray. "Utf8EncodeEx" function will convert a WideString into an UTF-8 encoded RtcByteArray. And to get the RtcByteArray back into a WideString, you can use the "Utf8DecodeEx" function. For example, the functions below accept  WideString Key and Content parameters and return a WideString result:

procedure rtcWritePackedWideEx(const FileName:String; const Key:WideString; const Content:WideString);
  begin
  // Content and Key need to be encoded from WideString to RtcByteArray
  rtcWritePackedEx(FileName, Utf8EncodeEx(Key), Utf8EncodeEx(Content));
  end;

function rtcReadPackedWideEx(const FileName:String; const Key:WideString):WideString;
  begin
  // Key needs to be encoded from WideString to RtcByteArray, but the resulting
  // Content has to be Decoded from RtcByteArray back to WideString
  Result:= Utf8DecodeEx( rtcReadPackedEx(FileName, Utf8EncodeEx(Key)) );
  end;

Best Regards,
Danijel Tkalcec

[Ronald van Tour]

OK Thanks