mORMot Open Source

delphi_911

Member

Registered: 2018-06-11

Posts: 31

[Solved]TOrmTable SortFields got uncorrect results

Env: Windows64 / Lazarus3.2+fpc3.2fix + mORMot2 trunk

var
  json: RawUtf8;
  dataset: TOrmTableDataSet;

  //////////

  json := '[{"F":1},{"F":2},{"F":3},{"F":2},{"F":null},{"F":0},{"F":null}]';
 
  dataset := TOrmTableDataSet.CreateFromJson(Application, json, [oftInteger]);  // Create a DataSet
 
  dataset.Table.SortFields('F', True);   // Sort Field

expected: null, null, 0, 1, 2, 2, 3
actual: 0, null, null, 1, 2, 2, 3

Did I make a mistake?

Another question: I want to sort Japanese/Chinese fields, the {CustomCompare} para in SortFields how should I put in?

Thanks!

Last edited by delphi_911 (2024-10-10 01:30:06)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: [Solved]TOrmTable SortFields got uncorrect results

Null is just handled as 0 when sorting as oftInteger, because it handles Utf8CompareInt64(), which does not support null.

A custom compare function is the right way to handle null, or sort with UTF-8 content.
I guess you could try to use our Utf8ICompReference() from mormot.core.unicode.

delphi_911

Member

Registered: 2018-06-11

Posts: 31

Re: [Solved]TOrmTable SortFields got uncorrect results

I tested [Utf8ICompReference] from mormot.core.unicode. did not get correct order when there were Chinese/Japanese characters.
And found [AnsiIComp] in sysstr.inc got the correct order but a lot more slower.

Last edited by delphi_911 (2024-10-09 09:12:18)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: [Solved]TOrmTable SortFields got uncorrect results

In fact, Utf8ICompReference()
- is platform-neutral, i.e. gives the same results on all OS and all pascal compilers;
- will be case-insensitive, i.e. convert uppercase/lowercase characters;
- will use the "natural Unicode" order, i.e. the 32-bit UCS4 code point integer value, which may not be what you expect.

I will add something using the OS comparison function.

EDIT:
Please try https://github.com/synopse/mORMot2/commit/2e6a52bb

delphi_911

Member

Registered: 2018-06-11

Posts: 31

Re: [Solved]TOrmTable SortFields got uncorrect results

Utf8CompareOS works OK with Chinese/Japanese characters, and much faster then sysstr - AnsiStrComp.

Thank you so much for your support!