Unit testing revealed an AV in BatchUpdate for modifications

h.hasenack · 2012-08-21 10:58:31

I adjusted a routine to avoid the AV. Obviously the buffer wasnt initialized properly. Around line 628 in SQLite3DB.pas

procedure TSQLRestServerStaticExternal.InternalBatchStop;
....
            if fBatchMethod=mPut then
            begin
             j:=Decode.FieldCount;
             if Values[j]=nil then // HH adjusted, ensure array is not nil to avoid AV,
                SetLength(Values[j],Math.Min(fBatchCount-BatchBegin,max));
              Values[j,n] := fBatchIDs[i]; // ?=ID parameter
            end;
....

The old lines looked like this:

....
            if fBatchMethod=mPut then
              Values[Decode.FieldCount,n] := fBatchIDs[i]; // ?=ID parameter
....

ab · 2012-08-21 11:54:34

How could you have Fields<>nil and Values[j]=nil at the same time?

Values[] is always initialized within the "if Fields=nil" block.

Could you step into this method with the debugger and find out in which circumstance it does not go within the "if Fields=nil" block?

Perhaps it is better with a

SetLength(Values[j],Math.Min(fBatchCount-BatchBegin,max)+1)

or

SetLength(Values[j],Math.Min(fBatchCount-BatchBegin+1,max))

h.hasenack · 2012-08-21 12:07:36

This code, with uUaseBatch=True cases the mishap.

procedure TMormotDBBaseCommonTest.ModifyModelData(aUseBatch, aUseTransactions: boolean);
VAR cnt:integer;
  Starttime, Endtime: cardinal;
  Rec: TSQLSampleRecord;
  Ida: TIntegerDynArray;
  GUID:TGUID;
begin
  cnt:=0;
  StartTime:=GetTickCount;
  if aUseBatch then
    Client.BatchStart(TSQLSampleRecord);
  if aUseTransactions then
    Client.TransactionBegin(TSQLSampleRecord);
  try
    Rec := TSQLSampleRecord.CreateAndFillPrepare(Client,'');
    try
      while Rec.FillOne do
      begin
        CreateGUID(GUID);
        Rec.Question:=GUIDToString(GUID);
        if aUseBatch then
        begin
          Client.BatchUpdate(Rec);
          if (cnt mod cBatchSize) = 0 then
          begin
            Client.BatchSend(Ida); << Causes the crash
            Client.BatchStart(TSQLSampleRecord);
          end;
        end
        else Client.Update(Rec);

        if aUseTransactions and (cnt mod cCommitSize = 0) then
        begin
          Client.Commit;
          Client.TransactionBegin(TSQLSampleRecord);
        end;

        inc(cnt);
      end;
    finally
      Rec.Free;
    end;
    if aUseBatch then
      Client.BatchSend(Ida);
    if aUseTransactions then
      Client.Commit;
  except
    if aUseBatch then
      Client.BatchAbort;
    if aUseTransactions then
      Client.RollBack;
    raise;
  end;
  EndTime:=GetTickCount;
  OutputDebugString(PChar(Format('%.0n records modified, avg speed %.0n rps Batch=%s Transaction=%s',[1.0*cnt,1000.0*cnt/(EndTime-StartTime),BoolToStr(aUseBatch),BoolToStr(aUseTransactions)])));
end;

Here's the call stack

System._LStrAsg(???,???)
SQLite3DB.TSQLRestServerStaticExternal.InternalBatchStop
SQLite3Commons.TSQLRestServer.RunBatch($ABB32D0,TSQLSampleRecord,'}','','')
SQLite3Commons.TSQLRestServer.URI('root/SampleRecord/0','POST','{"SampleRecord'#0#0'["PUT'#0#0'{"RowID":7345{"Time":135055073769,"Name":"","Question":"{DB304BC4-E286-440C-84E7-5896261F6613}","Address_":"","PostalCode":"","City":""}]}','','',$603BF9)
SQLite3.TSQLRestClientDB.InternalURI('root/SampleRecord/0','POST',$5E3548 {''},$18F5F8 {''},$18F6B8 {'{"SampleRecord'#0#0'["PUT'#0#0'{"RowID":7345{"Time":135055073769,"Name":"","Question":"{DB304BC4-E286-440C-84E7-5896261F6613}","Address_":"","PostalCode":"","City":""}]}'})
SQLite3Commons.TSQLRestClientURI.URI('root/SampleRecord/0','POST',$18F6B4 {''},nil {''},$18F6B8 {'{"SampleRecord'#0#0'["PUT'#0#0'{"RowID":7345{"Time":135055073769,"Name":"","Question":"{DB304BC4-E286-440C-84E7-5896261F6613}","Address_":"","PostalCode":"","City":""}]}'})
SQLite3Commons.TSQLRestClientURI.BatchSend(())
uTestMormotDBBaseCommon.TMormotDBBaseCommonTest.ModifyModelData(True,False)
uTestMormotDBBaseCommon.TMormotDBBaseCommonTest.TestTimedCRUDBatched
TestFramework.TTestCase.Invoke((uTestMormotDBBaseCommon.TMormotDBBaseCommonTest.TestTimedCRUDBatched,$2278588))
TestFramework.TTestCase.RunTest($22F5A30)

Here's my diagnosis:
the for j= loop goes to high(Fields) and not to high(Values) , therefor the last element in Values[] is not initialized.

Hope it helps.

ab · 2012-08-21 15:32:53

Decode.FieldCount=length(Values) since SetLenth(Values,n) is called with n=Decode.FieldCount so I guess this is not here.

Perhaps the issue comes from a mis-parsing of the Question text, which looks like a JSON object.

ab · 2012-08-21 15:44:46

OK.

I think I found the issue.
You were almost right from the beginning.
See http://synopse.info/fossil/info/c7c4d32960

Sorry for the problem, and thanks for the report.

mORMot Open Source

#1 2012-08-21 10:58:31

Unit testing revealed an AV in BatchUpdate for modifications

#2 2012-08-21 11:54:34

Re: Unit testing revealed an AV in BatchUpdate for modifications

#3 2012-08-21 12:07:36

Re: Unit testing revealed an AV in BatchUpdate for modifications

#4 2012-08-21 15:32:53

Re: Unit testing revealed an AV in BatchUpdate for modifications

#5 2012-08-21 15:44:46

Re: Unit testing revealed an AV in BatchUpdate for modifications

Board footer