mORMot Open Source

Bascy

Member

From: The Netherlands

Registered: 2012-06-22

Posts: 108

Integer overflow in Iso8601ToDateTime

While running our own unittests on an oracle database, we get an Integer Overflow error while deleting records using batch mode

Call stack:

SynCommons.Div100(???)
SynCommons.Iso8601ToDateTimePUTF8CharVar('00-01-01T00:00:00',17,0)
SQLite3Commons.TPropInfo.SetValue($25032E0,'0000-01-01T00:00:00')
SQLite3Commons.TSQLRecordFill.Fill($7C570038)
SQLite3Commons.TSQLRecord.FillRow(1,nil)
SQLite3Commons.TSQLRecord.FillOne
uTestMormotDBBaseCommon.TMormotDBBaseCommonCRUDTest.DeleteModelData(True,True)
uTestMormotDBBaseCommon.TMormotDBBaseCommonCRUDTest.TestTimedCRUDBatchedTransaction

Code:

procedure TMormotDBBaseCommonCRUDTest.DeleteModelData(aUseBatch, aUseTransactions: boolean);
VAR cnt:integer;
  Starttime, Endtime: cardinal;
  Rec: TSQLUnitTestSampleRecord;
  Ida: TIntegerDynArray;
begin
  OutputDebugString('---DELETE TEST---');
  cnt:=0;
  if aUseBatch then
    Client.BatchStart(TSQLUnitTestSampleRecord);
  if aUseTransactions then
    Client.TransactionBegin(TSQLUnitTestSampleRecord);

  StartTime:=GetTickCount;
  try
    Rec := TSQLUnitTestSampleRecord.CreateAndFillPrepare(Client,'');
    EndTime:=GetTickCount;
    OutputDebugString(PChar(Format('Query took %.0n ms',[1.0*(EndTime-StartTime)])));
    StartTime:=GetTickCount;
    try
      while Rec.FillOne do
      begin
        if aUseBatch then
        begin
          Client.BatchDelete(Rec.ID);
          if (cnt mod cBatchSize) = 0 then
          begin
            Client.BatchSend(Ida);
            Client.BatchStart(TSQLUnitTestSampleRecord);
          end;
        end
        else Client.Delete(TSQLUnitTestSampleRecord,Rec.ID);

        if aUseTransactions and (cnt mod cCommitSize = 0) then
        begin
          Client.Commit;
          Client.TransactionBegin(TSQLUnitTestSampleRecord);
        end;
        inc(cnt);
      end;
    finally
      Rec.Free;
    end;
    if aUseBatch then
      Client.BatchSend(Ida);
    if aUseTransactions then
      Client.Commit;
  except
    if aUseBatch then
      Client.BatchAbort;
    if aUseTransactions then
      Client.RollBack;
    raise;
  end;
  EndTime:=GetTickCount;
  OutputDebugString(PChar(Format('%.0n records deleted, avg speed %.0n rps Batch=%s Transaction=%s',[1.0*cnt,1000.0*cnt/max(1,(EndTime-StartTime)),BoolToStr(aUseBatch),BoolToStr(aUseTransactions)])));
end;

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,660

Website

Re: Integer overflow in Iso8601ToDateTime

How could you get an Integer Overflow error?

They are disactivated on purpose in the framework units.
Please do not change those Synopse.inc settings, which are expected to be as such, and will affect only mORMot units (not your code, which will rely on your project options).
For your program, you can enable range and overflow checking, but the framework expect the compiler settings to stay as defined.
You may have other issues in some low-level bit-oriented code of the framework.

In your special case, such an Integer overflow will return a value later checked as invalid.
So I suspect there is no problem in the supplied code.

Also check how your Nexus wrapper returns 0 TDateTime content: other external units convert a TDateTime of 0 as ''.
See e.g. TTextWriter.AddDateTime() method.
I suspect there may be something wrong in your wrapper, which should NEVER return a date/time column value as '0000-01-01T00:00:00' JSON content.

Bascy

Member

From: The Netherlands

Registered: 2012-06-22

Posts: 108

Re: Integer overflow in Iso8601ToDateTime

This test is done against a oracle database, the empty datetime returned as '0000-01-01T00:00:00' is from the Oracle driver ....

It turns out the field was empty for another reason (on creating the record we used BatchAdd(Rec, false, true) in stead of BatchAdd(Rec, True, True))
This created empty records in the oracle table, and when they were read back, the TDatetime field was empty too of course

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,660

Website

Re: Integer overflow in Iso8601ToDateTime

You were right about the issue: there was a problem in TOracleDate encoding/decoding in such case.

See http://synopse.info/fossil/info/649076d067 and http://synopse.info/fossil/info/608399a061