mORMot Open Source

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Display wrong characters even though data are intact after AES enc/dec

I am trying to encrypt the content from the TMemo (Chinese characters), and then decrypt and show them. The data (bytes) seems to be the same after AES encryption/decryption, as can be seen by going to the address "plain[1]" in the Memory debug window. However, the TMemo cannot display the original/correct characters. Could you help to comment what is the correct way out here ? Many thanks ! O_O

PS: The situation is tested with Delphi 7 ( comment out type ChineseString = type Ansistring(936) ) and Delphi Tokyo.
PS: Under Delphi Tokyo, the line calling SHA256Weak generates complaint "W1058 Implicit string cast with potential data loss from 'string' to 'RawByteString'." I thought methods with RawByteString parameter are ready for any string argument... O_O Could you help to comment the correct way to call SHA256Weak ?

uses
  SynCommons, SynLog, SynCrypto;

const
  TEST_KEY: string = 'TEST_KEY';

procedure TMainForm.btnEncryptClick(Sender: TObject);
var
  Key: TSHA256Digest;
  plain, cipher: RawByteString;
begin
  SHA256Weak(TEST_KEY, Key);
  plain := StringToUTF8(mmoPlain.Text);
//  mmoPlain.Text := UTF8ToString(plain);
//  mmoPlain.Text := UTF8ToString(AnyAnsiToUTF8(plain));
//  Exit;
  cipher := SynCrypto.AES(Key, 256, plain, true);
  mmoCipher.Text :=  UTF8ToString(BinToBase64(cipher));
end;

type
  ChineseString = type Ansistring(936); // https://stackoverflow.com/questions/7222615/how-can-i-convert-string-encoded-with-windows-codepage-1251-to-a-unicode-string

procedure TMainForm.btnDecryptClick(Sender: TObject);
var
  Key: TSHA256Digest;
  plain, cipher: RawByteString;
  ChineseStr: ChineseString;
begin
  SHA256Weak(TEST_KEY, Key);
  cipher := Base64ToBin(StringToUTF8(mmoCipher.Text));
  plain := SynCrypto.AES(Key, 256, cipher, false);

  mmoPlain.Text := UTF8ToString(AnyAnsiToUTF8(plain)); // Does not work
//  mmoPlain.Text := UTF8ToString(CurrentAnsiConvert.AnsiToUTF8(plain)); // Does not work
//  mmoPlain.Text := UTF8ToUnicodeString(CurrentAnsiConvert.AnsiToUTF8(plain)); // Does not work

//  ChineseStr := UTF8ToString(AnyAnsiToUTF8(plain)); // Does not work
//  mmoPlain.Text := ChineseStr;

// mmoPlain.Text := TEncoding.Default.GetString(TBytes(@plain[1])); // Does not work
end;

The test files can be viewed at http://gist.github.com/anonymous/fa917e … 447137bcbb

Last edited by ComingNine (2017-11-18 14:27:39)

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: Display wrong characters even though data are intact after AES enc/dec

For Delphi, TMemo now shows correct characters after calls of CurrentAnsiConvert routines are modified to be "symmetric". 

procedure TMainForm.btnEncryptWrongClick(Sender: TObject);
var
  Key: TSHA256Digest;
  plain, cipher: RawByteString;
begin
  SHA256Weak(CurrentAnsiConvert.UTF8ToAnsi(StringToUTF8(AES_KEY)), Key);
  plain := StringToUTF8(mmoPlain.Text);
  cipher := SynCrypto.AES(Key, 256, plain, true);
//  cipher := SynCrypto.AES(Key, 256, CurrentAnsiConvert.UTF8ToAnsi(plain), true);
  mmoCipher.Text :=  UTF8ToString(BinToBase64(cipher));
end;

procedure TMainForm.btnDecryptWrongClick(Sender: TObject);
var
  Key: TSHA256Digest;
  plain, cipher: RawByteString;
begin
  SHA256Weak(CurrentAnsiConvert.UTF8ToAnsi(StringToUTF8(AES_KEY)), Key);
  cipher := Base64ToBin(StringToUTF8(mmoCipher.Text));
  plain := SynCrypto.AES(Key, 256, cipher, false);
//  plain := CurrentAnsiConvert.AnsiToUTF8(SynCrypto.AES(Key, 256, cipher, false)); // Calling of CurrentAnsiConvert routines has to be "symmetric"
  mmoPlain.Text := UTF8ToString(plain);
end;

However, for FPC (on Windows), TMemo displays some characters correctly but question marks for others... O_O Could you help to comment about the reason of this behavior and possible workaround ?

The updated test files can be viewed at https://gist.github.com/anonymous/1995e … 1bad7f2974

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,669

Website

Re: Display wrong characters even though data are intact after AES enc/dec

Isn't LCL UTF-8 encoded, not AnsiString?

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: Display wrong characters even though data are intact after AES enc/dec

Many thanks for your help !
It works after the following change:

...
  {$IFDEF FPC}
  plain := mmoPlain.Text;
  {$ELSE ~FPC}
  plain := StringToUTF8(mmoPlain.Text);
  {$ENDIF ~FPC}
...
  {$IFDEF FPC}
  mmoPlain.Text := plain;
  {$ELSE ~FPC}
  mmoPlain.Text := UTF8ToString(plain);
  {$ENDIF ~FPC}
...