You are not logged in.
- Topics: Active | Unanswered
#1 2021-04-14 18:28:57
- marcinb
- Member
- Registered: 2014-04-08
- Posts: 11
Non ASCII characters in TSQLDBStatement.RowDocVariant
Hello
I have legacy firebird database with code page WIN1250. Still on mORMot 1.18.
What should I do to get whole string ('usługa' instead of 'us') when getting column values via RowDocVariant method. Should I set some additional Zeos connection properties?
I've checked that TSQLDBStatement.FetchAllAsJSON method returns "unmodified" string...
Thanks in advance
Marcin
Last edited by marcinb (2021-04-14 18:39:40)
Offline
#2 2021-04-14 21:05:09
Re: Non ASCII characters in TSQLDBStatement.RowDocVariant
I assume there shouldn't be any problem in reading from WIN1250 FB DB. TDocVariant from RowDocVariant should contain a full RawUTF8 string. Could it be that you got the loss afterward? maybe due to direct string assigning
Offline
#3 2021-04-15 08:02:02
- marcinb
- Member
- Registered: 2014-04-08
- Posts: 11
Re: Non ASCII characters in TSQLDBStatement.RowDocVariant
Hi
I did some digging (debugging) and found guilty routine: it's UTF8ToWideChar from ZEncode unit called by UTF8ToSynUnicode from SynCommons.
I'm on Delphi 10.4, but finally I'd like to compile this project in FPC on linux.
TIA
Marcin
Offline
#4 2021-04-15 09:42:11
Re: Non ASCII characters in TSQLDBStatement.RowDocVariant
I'm also using 10.4.2. I specifically made a test with the direct select statement execution and RowDocVariant through Zeos (WIN1250 DB).
In my case UTF8ToSynUnicode from SynCommons calls UTF8ToWideChar from SynCommons. And it seems working ok... - checked the resulting variant value from the debugger.
I used FB 2.5.9, although I doubt that it matters. Btw, I checked both, with USE_SYNCOMMONS and without it. Probably I misunderstand something, sorry.
Offline
#5 2021-04-15 13:45:47
- marcinb
- Member
- Registered: 2014-04-08
- Posts: 11
Re: Non ASCII characters in TSQLDBStatement.RowDocVariant
Can someone help me here, please
For unknow reason my project uses UTF8ToWideChar from ZEncoding instead of this one from SynCommons. It might be some settings I'm not aware of
TIA
Marcin
Last edited by marcinb (2021-04-15 13:59:29)
Offline
#7 2021-04-15 18:32:48
- marcinb
- Member
- Registered: 2014-04-08
- Posts: 11
Re: Non ASCII characters in TSQLDBStatement.RowDocVariant
Hello Arnaud
Latest mORMot (downloaded today) and Zeos 7.2.10 with latest patches (of 8th of April)
function UTF8ToWideChar from SynCommons returns 'us' string instead of 'usługa' (note the 3rd character in "usługa")
Any clues?
TIA
Marcin
Offline
#9 2021-04-16 14:44:33
Re: Non ASCII characters in TSQLDBStatement.RowDocVariant
Zeos 7.2.10 with latest patches (of 8th of April)
I'm using the trunk Zeos version. You could try it, probably not all fixes go to patches for a stable Zeos.
https://sourceforge.net/p/zeoslib/code- … ree/trunk/ or https://github.com/frones/ZeosLib
Offline
#10 2021-04-16 14:58:17
- marcinb
- Member
- Registered: 2014-04-08
- Posts: 11
Re: Non ASCII characters in TSQLDBStatement.RowDocVariant
I have checked same with UTF8 database and it is ok.
So following Arnaud's 'guessing' I'll try to set proper charset for db connection and see where I'll end up.
Offline
#11 2021-04-20 03:55:25
- EgonHugeist
- Member
- From: Germany
- Registered: 2013-02-15
- Posts: 190
Re: Non ASCII characters in TSQLDBStatement.RowDocVariant
@marcinb
what's the database characterset? If it's characterset "none"? If so you'll propably need an extra advice.
Michael
Offline
#12 2021-04-20 07:05:50
- marcinb
- Member
- Registered: 2014-04-08
- Posts: 11
Re: Non ASCII characters in TSQLDBStatement.RowDocVariant
Hello
Actually two things did the trick
- update of mORMot and Zeos
- explicilty specifying required characterset for the connection.
I have to perform more tests, but so far it looks good
Thanks for all suggestions
Marcin
Offline