Utf8ToConsole function cause program crash

xwg · 2022-01-07 02:30:33

param with noascii character will cause Utf8ToConsole crashes program, using FPC_3.2.0+mOrmot2。

writeln(Utf8ToConsole('{"code":2000,"message":"获取校验码成功","data":{"uuid":"A0D3BB6F-24C6-4E51-80AE-20ECC5A906B9","img":"iVBORw0KGgoAAAANSUhEUgAAAG4AAAAmCAIAAACqMjmtAAADnUlEQVR4nO2aPXaEIBDHuQ5X4Spew5LSK1BaegXLlFumTblHMOgA8i2D+PL2hXnkbaLrOPvb/wwDhpCR9NFmyJ+1223rKJtZR9nMOspm1lE2s46ymXWUzcxC+TXQjRBn0OFLnuAsfhxnfBrJaI+JV8YsBp7u7Kq9nt4psY0ph5xFD5/mq1J8U+BFv4VzA32cvfDRLYICPi6U03kCoFQseHcZlLtPvMPAOPO+9sLhJviLxVCK4V3LcZ2ZxxFM82VzxQf1NXEJEQ8l0N0u11CLdgwkhvLAeToBSf7U5I7Oaz4E+tOnsDnJGR/sb0U6CQVaEao2SHGX2hXHNYnyUKUCCkfeTvSlFqa2bTrNTRVOuUkveUfgWBFa0kKSskzS64+fQwluzFxTE5XgqZpoz2OXuwTOlctAD1W2KYthzB7JMo7rFUqnRFYAhRRWCnLbAFdcmBxfhmbTS8QckvsfZRzXPEr+Q8ISGYo0mX0uu9jdl4EnK2ki3OP9B3khqH0//PQVM4tkQXn0YkuhfNNUF5nJPpednruTpAzKonlcKXFmVHDZD1Gr/PJpD4XGCjLKoHmUAFV3WSzJNYPSGem5Oyu9S5Rn355H6WT0zMjerTi9i2k272lTkWR7eRxYvBPPBJlGyV6qVtrCTLGLVdIGKAFa2PQZs+92d7WjFzSHFkGXCFlezeCBSC/cOZUUgRJeto1s26m4kumFpK2UgQnG1SFHyvICZfW6G64JZ5XE0mIDiFZYFXP0vtwjIzX3QpL10WFledlXWsLErHZSCzL3XQsQlxBt2dY2Oj5K7NW+CJGyLEC5RiqmsTwy1aKfzaQ6DxqUR3QFUBWuQIwwvUR5AUpM/+JcHEgQUDbpK8ODRpgJlQVmFo4wdCKremrOTvNaKEa51BkTfQ+c4lUL3BhJbIaXocwKM2uqcwSCRzVcjEi5XlYiKqNCGZEedOx1raWOzutAcBleiLKyYu5Rggx1FksXZmFuUh7j70jwiPRuFUqjScPUjHJZlm792nvs5buWalKGRD4uFquX9RN+u+Sg5vSPwLG6P0+pb89wj2zGS8kDidgSCLnntqez/0DC36nEAYWG3Iw7JfK0U4CpxxIZoI8/JgsBSYj2Iuf4/WR6Z0/vb+1ZlCFEVRm9mX2KZObHAX0KpQcinF7OfI9usFt+msf2kD2CMhRjG7fl/5z3F0/226P8IB21tZYoP666tbXHZ/D/Yx1lM+som5lC2UeT8QtS0NGYb+V8RgAAAABJRU5ErkJggg=="}}'));

ab · 2022-01-07 09:27:21

I am not able to reproduce it.

On which system do you have the issue?
What is the exact source code line which makes a crash?
Are you sure your source code file is really marked as UTF-8 encoded via a {$CODEPAGE UTF8} conditional?

xwg · 2022-01-07 16:27:05

Hi ab, thank you for your reply.
I found my program crash when TSynlog EchoToConsole active, debug crash at Utf8ToConsole function, here is reproduce code:
runing in windows10 Simplified Chinese, cp_936.

program project1;

{$codepage utf8}
{$I mormot.defines.inc}
uses
  {$I mormot.uses.inc}
  SysUtils,
  mormot.core.base,
  mormot.core.os;

var s: RawByteString;
begin
  try
    //AV occur
    s:=Utf8ToConsole('{"code":2000,"message":"成功","data":{"uuid":"A0D3BB6F-24C6-4E51-80AE-20ECC5A906B9","img":"iVBORw0KGgoAAAANSUhEUgAAAG4AAAAmCAIAAACqMjmtAAADnUlEQVR4nO2aPXaEIBDHuQ5X4Spew5LSK1BaegXLlFumTblHMOgA8i2D+PL2hXnkbaLrOPvb/wwDhpCR9NFmyJ+1223rKJtZR9nMOspm1lE2s46ymXWUzcxC+TXQjRBn0OFLnuAsfhxnfBrJaI+JV8YsBp7u7Kq9nt4psY0ph5xFD5/mq1J8U+BFv4VzA32cvfDRLYICPi6U03kCoFQseHcZlLtPvMPAOPO+9sLhJviLxVCK4V3LcZ2ZxxFM82VzxQf1NXEJEQ8l0N0u11CLdgwkhvLAeToBSf7U5I7Oaz4E+tOnsDnJGR/sb0U6CQVaEao2SHGX2hXHNYnyUKUCCkfeTvSlFqa2bTrNTRVOuUkveUfgWBFa0kKSskzS64+fQwluzFxTE5XgqZpoz2OXuwTOlctAD1W2KYthzB7JMo7rFUqnRFYAhRRWCnLbAFdcmBxfhmbTS8QckvsfZRzXPEr+Q8ISGYo0mX0uu9jdl4EnK2ki3OP9B3khqH0//PQVM4tkQXn0YkuhfNNUF5nJPpednruTpAzKonlcKXFmVHDZD1Gr/PJpD4XGCjLKoHmUAFV3WSzJNYPSGem5Oyu9S5Rn355H6WT0zMjerTi9i2k272lTkWR7eRxYvBPPBJlGyV6qVtrCTLGLVdIGKAFa2PQZs+92d7WjFzSHFkGXCFlezeCBSC/cOZUUgRJeto1s26m4kumFpK2UgQnG1SFHyvICZfW6G64JZ5XE0mIDiFZYFXP0vtwjIzX3QpL10WFledlXWsLErHZSCzL3XQsQlxBt2dY2Oj5K7NW+CJGyLEC5RiqmsTwy1aKfzaQ6DxqUR3QFUBWuQIwwvUR5AUpM/+JcHEgQUDbpK8ODRpgJlQVmFo4wdCKremrOTvNaKEa51BkTfQ+c4lUL3BhJbIaXocwKM2uqcwSCRzVcjEi5XlYiKqNCGZEedOx1raWOzutAcBleiLKyYu5Rggx1FksXZmFuUh7j70jwiPRuFUqjScPUjHJZlm792nvs5buWalKGRD4uFquX9RN+u+Sg5vSPwLG6P0+pb89wj2zGS8kDidgSCLnntqez/0DC36nEAYWG3Iw7JfK0U4CpxxIZoI8/JgsBSYj2Iuf4/WR6Z0/vb+1ZlCFEVRm9mX2KZObHAX0KpQcinF7OfI9usFt+msf2kD2CMhRjG7fl/5z3F0/226P8IB21tZYoP666tbXHZ/D/Yx1lM+som5lC2UeT8QtS0NGYb+V8RgAAAABJRU5ErkJggg=="}}');
    writeln(s);
  except on e:exception do
    writeln(e.message);
  end;
end.

ab · 2022-01-07 19:51:01

I can't reproduce it here.

On which line does exactly the AV occur?

xwg · 2022-01-09 13:21:44

maybe can fix this issure

      tmp.InitZero(length(utf16) * 3);  //<- tmp.Init(length(utf16) * 3); 
      CharToOemBuffW(pointer(utf16), tmp.buf, length(utf16));  //<- CharToOemBuffW(pointer(utf16), tmp.buf, tmp.len);

ab · 2022-01-09 13:25:36

The API documentation is very misleading.

[in] cchDstLength
Type: DWORD
The number of characters to translate in the string identified by the lpszSrc parameter.

So it is not the DstLength - Dest Length, but Source Length.

I guess what you wrote should fix it.
Please check https://github.com/synopse/mORMot2/commit/861c5dee

xwg · 2022-01-09 14:42:16

@ab need use tmp.InitZero to replace tmp.Init, Otherwise StrLen return is not correct.

ab · 2022-01-10 09:37:46

You are right.
CharToOemBuffW() is really a broken API.

Please check https://github.com/synopse/mORMot2/commit/2a451227
I guess that using + 1 to include the ending #0 is better than InitZero since we need only a single #0 at the right position.
This is what https://cpp.hotexamples.com/examples/-/ … c-177,,183, does.

mORMot Open Source

#1 2022-01-07 02:30:33

Utf8ToConsole function cause program crash

#2 2022-01-07 09:27:21

Re: Utf8ToConsole function cause program crash

#3 2022-01-07 16:27:05

Re: Utf8ToConsole function cause program crash

#4 2022-01-07 19:51:01

Re: Utf8ToConsole function cause program crash

#5 2022-01-09 13:21:44

Re: Utf8ToConsole function cause program crash

#6 2022-01-09 13:25:36

Re: Utf8ToConsole function cause program crash

#7 2022-01-09 14:42:16

Re: Utf8ToConsole function cause program crash

#8 2022-01-10 09:37:46

Re: Utf8ToConsole function cause program crash

Board footer