Posts by algalg

algalg · Delphi

I've logged this as https://quality.embarcadero.com/browse/RSP-35589

algalg · Delphi

I was looking into this more and actually it is not the _Safe() function getting inlined, but the {System.Generics.Collections}TList<System.Variant>.GetItem method.

on XE4

Project1.dpr.56: DocVariantDataPointer := _Safe(FRepeatingData.Items[0]);
00000000005B71BD 488B0D04160400   mov rcx,[rel $00041604]
00000000005B71C4 488D5528         lea rdx,[rbp+$28]
00000000005B71C8 4D33C0           xor r8,r8
00000000005B71CB E8006A0000       call {System.Generics.Collections}TList<System.Variant>.GetItem
00000000005B71D0 488D4D28         lea rcx,[rbp+$28]
00000000005B71D4 E857C7FCFF       call _Safe
00000000005B71D9 48894548         mov [rbp+$48],rax

on Delphi 11

Project1.dpr.57: DocVariantDataPointer := _Safe(FRepeatingData.Items[0]);
00000000005A2677 488D7D60         lea rdi,[rbp+$60]
00000000005A267B 33C0             xor eax,eax
00000000005A267D 48AB             stosq
00000000005A267F 48AB             stosq
00000000005A2681 48AB             stosq
00000000005A2683 90               nop
00000000005A2684 488B056D430400   mov rax,[rel $0004436d]
00000000005A268B 83781000         cmp dword ptr [rax+$10],$00
00000000005A268F 7705             jnbe ReadKO2 + $A6
00000000005A2691 E86AEFEBFF       call ErrorArgumentOutOfRange
00000000005A2696 488D4D60         lea rcx,[rbp+$60]
00000000005A269A 488B0557430400   mov rax,[rel $00044357]
00000000005A26A1 488B5008         mov rdx,[rax+$08]
00000000005A26A5 E8368FEAFF       call @VarCopy
00000000005A26AA 488D4D60         lea rcx,[rbp+$60]
00000000005A26AE E89DF3FCFF       call _Safe
00000000005A26B3 48894538         mov [rbp+$38],rax
00000000005A26B7 90               nop
00000000005A26B8 488D4D60         lea rcx,[rbp+$60]
00000000005A26BC E88F82EAFF       call @VarClr

the last instruction clears the result of call to _Safe ....

and for some reason adding the {$INLINE OFF} / {$INLINE ON} to around the call line does not help either, inlining needs to be disabled completely to make the code work properly.

I'd say this is Delphi compiler bug.

algalg · Delphi

You are definitely up to something. The same issue occurs with PDocVariantData local variable.

 
    procedure ReadKO2;
    var tmp: string;
        DocVariantDataPointer: PDocVariantData;
    begin
      DocVariantDataPointer := _Safe(FRepeatingData.Items[0]);
      tmp := DocVariantDataPointer^.S[AttributeName];   // EDocVariant property not found exception raised
      Writeln(tmp);
    end;

But when I disable inlining completely in the compiler it does no longer occur.

As far as I can tell the definition of _Safe function does not set it as inline except for FPC. But it seems to be that Delphi 11 is inclining it anyways.

function _Safe(const DocVariant: variant): PDocVariantData; overload;
  {$ifdef FPC}inline;{$endif} // Delphi has problems inlining this :(

algalg · Delphi

While looking in to migrating my project into Delphi 11 I found that code that worked fine in Delphi XE4 raises an exception in Delphi 11.

The code bellow shows the issue. The reading of the string member of the Doc Variant works perfectly in the Read procedure, but fails in ReadKO procedure when compiled with Delphi 11. The same code works fine when compiled in Delphi XE.

Do you have any idea why this would be happening?

program Project1;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils, SynCommons, System.Generics.Collections;

var FRepeatingData: TList<Variant>;
    AttributeName: RawUTF8;

   procedure Insert;
    var RepeatingDataDocument: TDocVariantData;
    begin
      RepeatingDataDocument.InitFast(1, dvObject);

      RepeatingDataDocument.AddValue('__INSERT', true);
      RepeatingDataDocument.AddValue('__UPDATE', false);
      RepeatingDataDocument.AddValue(AttributeName, 'Test');

      FRepeatingData.Add(variant(RepeatingDataDocument));

    end;

    procedure Read;
    var tmp: string;
        DocVariantData: TDocVariantData;
    begin
      DocVariantData :=  _Safe(FRepeatingData.Items[0])^ ;
      with DocVariantData do
      begin
        tmp := S[AttributeName];
        Writeln(tmp);
      end;
    end;

    procedure ReadKO;
    var tmp: string;
    begin
      with _Safe(FRepeatingData.Items[0])^ do   // EDocVariant property not found exception raised
      begin
        tmp := S[AttributeName];
        Writeln(tmp);
      end;
    end;



begin
  try
    AttributeName := 'Name';

    FRepeatingData := TList<Variant>.Create;
    try
      Insert;
      Read;
      ReadKO;
    finally
        FRepeatingData.Clear;
        FRepeatingData.free;
    end;

  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
end.

algalg · mORMot 1

Hello,

I am getting back to this. I have decided to implement chunked reading and writing of blobs from and to a TStream.

The initial implementation is now checked in to https://github.com/algalg/mORMot commints:

https://github.com/algalg/mORMot/commit … 1e2b1a13e4
https://github.com/algalg/mORMot/commit … dce5dcd97f

The use is:

For reading

var SQLDBRows: ISQLDBRows;
    FileStrm : TFileStream;
begin
  SQLDBRows := FConnection.Execute('select a from T', []);
  SQLDBRows.Step();
  FileStrm := TFileStream.Create('c:\test.txt-syn', fmCreate);
  try
    SQLDBRows.ColumnBlobToStream(0, FileStrm);
  finally
   FileStrm.Free;
  end;
end;

For writing

var SQLDBRows: ISQLDBRows;
    FileStrm : TFileStream;
begin

  FConnection.ExecuteNoResult('delete from T',[]);  // just to make sure we have only one row in the table for the demo
  FConnection.ExecuteNoResult('INSERT INTO T (a)  VALUES (EMPTY_BLOB())',[]);

  SQLDBRows := FConnection.Execute('select a from T for update', []);
  SQLDBRows.Step();

  FileStrm := TFileStream.Create('c:\test.txt', fmOpenRead or fmShareDenyNone);
  try

    SQLDBRows.ColumnBlobWriteFromStream(0, FileStrm);
  finally
    FileStrm.Free;
  end;
  SQLDBRows := nil;

I would like to pull this into the main branch after some tidying and commenting the code. What is the process? Are in interested in this implementation?

I am planning to also implement this on the TOleDBStatement class.

algalg · mORMot 1

Thank you for the explanation. I agree that it would be best to not to store huge blobs in DB, but I don't really have a choice at the moment.

Anyways, I can work with BindBlob and use Oracle DBMS_LOB.APPEND function to move the file from client to DB server in chunks.

algalg · mORMot 1

Thanks, I see this in the code now.

But what is the best way to get 2GB file into Oracle blob field using SynDb? Load it into memory and bind and parameter using the BindBlob method? Is there anyway to load the blob in chunks and avoid using all the RAM?

algalg · mORMot 1

Hello I am new to mORMot Framework and SynDB and I am looking to use SynDB to replace existing ADO connection to Oracle and MSSQL in an existing application.

First I am working with Oracle and looking at the options in working with BLOBs.

To insert / upload blobs to database I only see the option to bind the blob to a parameter of SQL statement, but this is limited by the MAX_INLINED_PARAM_SIZE = 32*1024*1024; constant to 32MB.

Is there a way in SynDB to post large BLOB (2 GB) into Oracle in chunks?

mORMot Open Source

#1 Re: Delphi » with _Safe( and Delphi 11 » 2021-09-21 10:32:34

#2 Re: Delphi » with _Safe( and Delphi 11 » 2021-09-21 06:51:38

#3 Re: Delphi » with _Safe( and Delphi 11 » 2021-09-20 15:08:06

#4 Delphi » with _Safe( and Delphi 11 » 2021-09-20 11:34:21

#5 Re: mORMot 1 » SynDB - Oracle OCI - Big Blob insert update » 2020-05-21 06:28:14

#6 Re: mORMot 1 » SynDB - Oracle OCI - Big Blob insert update » 2020-05-19 07:15:20

#7 Re: mORMot 1 » SynDB - Oracle OCI - Big Blob insert update » 2020-05-18 12:29:42

#8 mORMot 1 » SynDB - Oracle OCI - Big Blob insert update » 2020-05-18 09:58:18

Board footer