mORMot Open Source

tbo

Member

Registered: 2015-04-20

Posts: 367

Not expected behaviour with Dictionary

Delphi 12.2; mORMot Commit 2.3.11508; 32-Bit

The following example shows a behaviour that I did not expect:

var
  v: Integer;
  k1, k2, k3: Variant;
  dic: IKeyValue<Variant, Integer>;
begin
  k1 := Byte(1);
  k2 := Byte(1);
  k3 := Int64(1);
  dic := Collections.NewKeyValue<Variant, Integer>;
  dic.Add(k1, 2);
  ShowMessage(dic.TryGetValue(k2, v).ToString(TUseBoolStrs.True));  // Result: True
  ShowMessage(dic.TryGetValue(k3, v).ToString(TUseBoolStrs.True));  // Result: False <==

In contrast, Delphi's TDictionary<Variant, Integer> delivers the expected result.

With best regards
Thomas

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 15,134

Website

Re: Not expected behaviour with Dictionary

It comes indeed from an implementation detail:
1) It sounds like if mORMot's VariantHash() from mormot.core.data is actually hashing the variant raw binary in memory, so with len = 1 for varByte and len = 8 for varInt64. And obviously, the hashes are not the same, so it can't find it.
2) Delphi, on the other hand, in its GetHashCode_Variant() function, is always using a conversion to text, which is slow, but works in this case.

I did not expect Int64(1) and Byte(1) to be the same variant during hashing. In fact, for performance reasons, working on the binary did make better sense.
But I now understand it is not coherent with what SortDynArrayVariant() - aka FastVarDataComp() - is doing.

The hash function should match what FastVarDataComp() does, i.e. call VariantCompSimple() which converts simple types to Int64 or double for the comparison.
Please try with
https://github.com/synopse/mORMot2/commit/4b594eb26

I have also added some internal callbacks to efficiently hash complex types like TDocVariant or TBsonVariant.
Now huge values are not fully serialized as JSON, but are processed at binary level, stopping after 255 bytes of hashed content.

I have added some corresponding regression tests, which includes your findings (and more):
https://github.com/synopse/mORMot2/commit/06e682845

danielkuettner

Member

From: Germany

Registered: 2014-08-06

Posts: 398

Re: Not expected behaviour with Dictionary

Why didn't you include Integer into your Int64 normalization?
What is working for Byte should also work for Integer.

Last edited by danielkuettner (2025-08-05 16:08:37)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 15,134

Website

Re: Not expected behaviour with Dictionary

Look again, varInteger has its own optimized path, to avoid a call to VariantToInt64().