mORMot Open Source

igors233

Member

Registered: 2012-09-10

Posts: 241

TSQLDBStatement.ColumnToVariant

This isn't a bug in the strict sense but it's inconsistency compared to results with ColumnToUTF8.
If Firebird DB stores text in NONE charset and Zeos connection is used, by default it will use UTF8 connection charset.
Problem happens if text stored contains chars above >127, when reading from DB, ColumnToVariant will use ColumnToSQLVar which uses ColumnUTF8,
For example, text is "Invoice value is 100£, payment due in 3 days"

Since text in DB is stored without UTF8 encoding, ColumnUTF8 will read it as is and assume it's already UTF8 encoded (which is fine).
Problem is that ColumntToVariant uses UTF8ToSynUnicode which uses UTF8ToWideChar for conversion to Unicode and that functions stops conversion after it encouters char > 127, so with above example, returned value will be "Invoice value is 100" and that in some cases is a big problem.
If Zeos settings are changed to use charset NONE all is fine, however I do believe current behaviour is dangerous (trims text) and inconsistent to result that ColumnUTF8 gives.
I would suggest to change that behaviour  to either raise conversion error or returns whole text (as ColumnToUF8) does.