mORMot Open Source

algalg

Member

Registered: 2020-05-18

Posts: 8

with _Safe( and Delphi 11

While looking in to migrating my project into Delphi 11 I found that code that worked fine in Delphi  XE4 raises an exception in Delphi 11.

The code bellow shows the issue. The reading of the string member of the Doc Variant works perfectly in the Read procedure, but fails in ReadKO procedure when compiled with Delphi 11. The same code works fine when compiled in Delphi XE.

Do you have any idea why this would be happening?

program Project1;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils, SynCommons, System.Generics.Collections;

var FRepeatingData: TList<Variant>;
    AttributeName: RawUTF8;

   procedure Insert;
    var RepeatingDataDocument: TDocVariantData;
    begin
      RepeatingDataDocument.InitFast(1, dvObject);

      RepeatingDataDocument.AddValue('__INSERT', true);
      RepeatingDataDocument.AddValue('__UPDATE', false);
      RepeatingDataDocument.AddValue(AttributeName, 'Test');

      FRepeatingData.Add(variant(RepeatingDataDocument));

    end;

    procedure Read;
    var tmp: string;
        DocVariantData: TDocVariantData;
    begin
      DocVariantData :=  _Safe(FRepeatingData.Items[0])^ ;
      with DocVariantData do
      begin
        tmp := S[AttributeName];
        Writeln(tmp);
      end;
    end;

    procedure ReadKO;
    var tmp: string;
    begin
      with _Safe(FRepeatingData.Items[0])^ do   // EDocVariant property not found exception raised
      begin
        tmp := S[AttributeName];
        Writeln(tmp);
      end;
    end;



begin
  try
    AttributeName := 'Name';

    FRepeatingData := TList<Variant>.Create;
    try
      Insert;
      Read;
      ReadKO;
    finally
        FRepeatingData.Clear;
        FRepeatingData.free;
    end;

  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
end.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,649

Website

Re: with _Safe( and Delphi 11

I don't have Delphi 11, so I can't reproduce it.

What if a local DocVariantData: PDocVariantData is used?
(no copy of the content, just copy of the pointer)

Also try to disable _Safe() inlinining.
It may be an inlining bug. Delphi has tons of issues/regressions when inlining.

algalg

Member

Registered: 2020-05-18

Posts: 8

Re: with _Safe( and Delphi 11

You are definitely up to something.  The same issue occurs with PDocVariantData local variable.

 
    procedure ReadKO2;
    var tmp: string;
        DocVariantDataPointer: PDocVariantData;
    begin
      DocVariantDataPointer := _Safe(FRepeatingData.Items[0]);
      tmp := DocVariantDataPointer^.S[AttributeName];   // EDocVariant property not found exception raised
      Writeln(tmp);
    end;

But when I disable inlining completely in the compiler it does no longer occur.

As far as I can tell the definition of _Safe function does not set it as inline except for FPC. But it seems to be that Delphi 11 is inclining it anyways.

function _Safe(const DocVariant: variant): PDocVariantData; overload;
  {$ifdef FPC}inline;{$endif} // Delphi has problems inlining this :(

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,649

Website

Re: with _Safe( and Delphi 11

It sounds like if Delphi 11 is doing automatic inlining...

And its inlining is failing... as with previous Delphi versions...

I guess they also introduced a new keyword to disable automatic inlining for a given function.
I was not able to find out which keyword they are using.

Could you try to add
{$INLINE OFF}
{$INLINE ON}
above and after the _Safe() definition?
https://docwiki.embarcadero.com/RADStud … _Directive

algalg

Member

Registered: 2020-05-18

Posts: 8

Re: with _Safe( and Delphi 11

I was looking into this more and actually it is not the _Safe()  function getting inlined, but the {System.Generics.Collections}TList<System.Variant>.GetItem method.

on XE4

Project1.dpr.56: DocVariantDataPointer := _Safe(FRepeatingData.Items[0]);
00000000005B71BD 488B0D04160400   mov rcx,[rel $00041604]
00000000005B71C4 488D5528         lea rdx,[rbp+$28]
00000000005B71C8 4D33C0           xor r8,r8
00000000005B71CB E8006A0000       call {System.Generics.Collections}TList<System.Variant>.GetItem
00000000005B71D0 488D4D28         lea rcx,[rbp+$28]
00000000005B71D4 E857C7FCFF       call _Safe
00000000005B71D9 48894548         mov [rbp+$48],rax

on Delphi 11

Project1.dpr.57: DocVariantDataPointer := _Safe(FRepeatingData.Items[0]);
00000000005A2677 488D7D60         lea rdi,[rbp+$60]
00000000005A267B 33C0             xor eax,eax
00000000005A267D 48AB             stosq
00000000005A267F 48AB             stosq
00000000005A2681 48AB             stosq
00000000005A2683 90               nop
00000000005A2684 488B056D430400   mov rax,[rel $0004436d]
00000000005A268B 83781000         cmp dword ptr [rax+$10],$00
00000000005A268F 7705             jnbe ReadKO2 + $A6
00000000005A2691 E86AEFEBFF       call ErrorArgumentOutOfRange
00000000005A2696 488D4D60         lea rcx,[rbp+$60]
00000000005A269A 488B0557430400   mov rax,[rel $00044357]
00000000005A26A1 488B5008         mov rdx,[rax+$08]
00000000005A26A5 E8368FEAFF       call @VarCopy
00000000005A26AA 488D4D60         lea rcx,[rbp+$60]
00000000005A26AE E89DF3FCFF       call _Safe
00000000005A26B3 48894538         mov [rbp+$38],rax
00000000005A26B7 90               nop
00000000005A26B8 488D4D60         lea rcx,[rbp+$60]
00000000005A26BC E88F82EAFF       call @VarClr

the last instruction clears the result of call to _Safe ....

and for some reason adding the {$INLINE OFF} / {$INLINE ON} to around the call line does not help either, inlining needs to be disabled completely to make the code work properly.

I'd say this is Delphi compiler bug.

Last edited by algalg (2021-09-21 07:00:41)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,649

Website

Re: with _Safe( and Delphi 11

Yes, weird compiler bug - or at leastg breaking change.
There is no _Safe() inlining problem here.

In fact, I doubt they will call it a bug. And that they perhaps will never fix it.
What I remember is that they changed the local variable lifetime.
So Items[] return a local variant variable which is passed to _Safe(), which then returns a pointer to it, but VarClr() is called before you could use this pointer.

Any function returning a pointer to a temporary parameter would fail.
This is a collateral domage of changing the behavior of the compiler since decades, for no real benefit. It won't make applications faster, but it could break a lot of existing code since years.
Some micro benchmarks may find some benefit of this new temporary variables allocation strategy. But if you expect better register allocation in a real application, you would have use a local function for the main processing loop.

I note that FPC used to release sooner the local variables than Delphi. We had to use a local ISynLog or an explicit "with" when using TSynLog.Enter.
But at least, FPC, make it correct for such temporary variables, and the _Safe() code is working as expected.
So for me, it is a bug. Since you have a Delphi 11 license, please create a bug report. Just write a function returning a PVariant to the supplied variant parameter. No need of anything more.

The most obvious workaround is to use not TList<Variant> but a TVariantDynArray - it would also be faster.
I have added some new overloaded _DV() functions which return a TDocVariantData and not a PDocVariantData: it would be slower, but should work with your use case.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,649

Website

Re: with _Safe( and Delphi 11

Please check https://blog.synopse.info/?post/2021/09 … ng-Changes

We are waiting perhaps for a RSP at least for _Safe() like functions.

I am curious how our mORMot 2 collections would behave for your use case.
I wonder how

 Collections.NewList<variant> 

would behave. Perhaps exactly the same...
https://github.com/synopse/mORMot2/blob … ctions.pas

algalg

Member

Registered: 2020-05-18

Posts: 8

Re: with _Safe( and Delphi 11

I've logged this as https://quality.embarcadero.com/browse/RSP-35589

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,649

Website

Re: with _Safe( and Delphi 11

Stefan just answered in the RSP that instead of using Items[0] the code should use the internal List/PList.
It is a workaround, but not very efficient nor easy to use for sure.
If the previous code worked "by chance", then at least it could have continued to work.