Pages: 1

#1 Re: mORMot 1 » SynDB.TSQLDBStatementWithParamsAndColumns.ColumnIndex() broken » 2019-08-28 14:55:48

Joe
ab wrote:

I have made a deeper refactoring to avoid issues as you discovered.
Please check https://synopse.info/fossil/info/51eced20b8

It is working perfectly now (I have removed that dirty patch of mine). Thanks.

#2 Re: mORMot 1 » SynDB.TSQLDBStatementWithParamsAndColumns.ColumnIndex() broken » 2019-08-27 11:17:59

Joe

Ok, I have analyzed it a bit more. The problem is that extended RTTI used in the TDynArray.ToKnownType() method it returns djCustom and thus the comparison fails to dig deeper (to compare first string elem of record). I have patched it this way:

  {$ifdef ISDELPHI2010} // seems inconsistent with FPC - only for Delphi 2010+
-  if (result=djNone) and (fElemType2<>nil) then // try from extended RTTI
+  if (result=djNone) and (fElemType2<>nil) and not (PTypeKind(fElemType2)^ in [tkClass, tkInterface, tkRecord, tkArray, tkDynArray]) then begin // try from extended RTTI
    result := RTTI[TJSONCustomParserRTTI.TypeInfoToSimpleRTTIType(fElemType2)];
+    if result = djCustom then  // unknown types
+      Result:= djNone;
  end;
  {$endif}

#3 Re: mORMot 1 » SynDB.TSQLDBStatementWithParamsAndColumns.ColumnIndex() broken » 2019-08-01 15:16:26

Joe
ab wrote:

With which compiler?

Found with Delphi 10.3.1

#4 mORMot 1 » SynDB.TSQLDBStatementWithParamsAndColumns.ColumnIndex() broken » 2019-08-01 14:25:42

Joe
Replies: 5

Hello,
today I have discovered that commit https://synopse.info/fossil/info/888baa599a51ab0a broke functionality of SynDB.TSQLDBStatementWithParamsAndColumns.ColumnIndex() method - it never finds a column, because it compares TSQLDBColumnProperty records by pointers.

Regards,
Joe

#5 Re: mORMot 1 » TDynArrayHashed's bug triggered by a re-used TSQLDBZeosStatement » 2018-10-12 15:35:40

Joe

Hello,
after some experimenting I have found out that, it works correctly if I modify the TDynArrayHashed.HashAdd() method in SynCommons.pas:

...
      dec(fCountP^); // ignore latest entry (which is not filled yet)
---    ReHash;
+++    ReHash(True);
    if fCountP<>nil then
...

Best regards,
Joe

#6 Re: mORMot 1 » TDynArrayHashed's bug triggered by a re-used TSQLDBZeosStatement » 2018-10-02 13:13:40

Joe

I have tried it now, but unfortunately it makes no difference.

#7 mORMot 1 » TDynArrayHashed's bug triggered by a re-used TSQLDBZeosStatement » 2018-10-01 17:15:24

Joe
Replies: 3

Hello,
today I have encountered a very obscure problem. The way a TDynArrayHashed is used in a TSQLDBZeosStatement a malfunction can be triggered in the fColumn.AddAndMakeUniqueName() method. The array is re-inited with just:

      fColumnCount := 0;
      fColumn.ReHash;

That works fine, but it leaves the fHashFindCount property unchanged, thus it raises further and after several tries with a query of eleven columns it crashes.

If I am correct the cause is a hit of the condition and reset of the fHashCounterTrigger property to zero in TDynArrayHashed.FindHashed() method (the code using the query is fetching values by the names of the columns). The core of the problem lies in the first call to the AddAndMakeUniqueName() method on a statement re-used after the fHashCounterTrigger was set to zero. The TDynArrayHashed.HashAdd does a ReHash (which effectively sets fHashs to nil for an empty array) and tries to write a hash to fHashs afterwards. Also, the Assert() in TDynArrayHashed.HashAdd() is hit, because ReHash clears hash and the following HashFind() finds the column (the data remained there, the Count was already incremented, search goes without a hash).

Either the fHashCounterTrigger should not be set to zero, or the ReHash should be able to prepare fHashs even for an empty array (perhaps calling it from HashAdd() with forAdd = True ?).

Best regards,
Joe

#8 Re: mORMot 1 » Prepared Statement Cache breaks parallel requests (from ticket) » 2014-10-14 14:13:36

Joe

It is working perfectly now. I am running it through the SQLite3 virtual tables and the performance is good for my purposes. Thanks again for the improved fix.

#9 Re: mORMot 1 » Prepared Statement Cache breaks parallel requests (from ticket) » 2014-10-13 15:47:05

Joe

Thank you very much for the fix.

#10 Re: mORMot 1 » Prepared Statement Cache breaks parallel requests (from ticket) » 2014-10-13 14:16:38

Joe

Yes, that is it.

The query in question is: 

select A.*,B.value,C.value from MAIN_TABLE as A 
left outer join DETAIL_TABLE as B on B.id=A.field1 
left outer join DETAIL_TABLE as C on C.id=A.field2 
where A.id=:id

Best Regards, Joe

#11 Re: mORMot 1 » Prepared Statement Cache breaks parallel requests (from ticket) » 2014-10-13 11:12:18

Joe

I have read the answer. The problem is NOT in threading. Each thread has its own connection.

I run a SQL query directly to SQLite. The problem lies in the data fetch process: three instances of TSQLVirtualTableCursorExternal are created and initialized by method Search(): one for a master table A and two for a detail table B (using the same SQL for external data fetch). The joined data is not fetched correctly, because SQLite fetches the external records one-by-one and the second detail B conflicts with the first detail B. Both fetches of external detail data is supposed to return one record (JOIN by key). The problem in detail is that Cursor.Search() for first detail B prepares the statement and fetches first (the only one) record, then the second detail B is prepared by Cursor.Search() doing the same and effectively dumping data of first detail B. The column values are retrieved after all this, thus giving invalid data for the first detail B.

#12 mORMot 1 » Prepared Statement Cache breaks parallel requests (from ticket) » 2014-10-13 09:30:11

Joe
Replies: 8

Hello,
I am posting a problem description as suggested in ticket http://synopse.info/fossil/info/7362951 … 07b1e9aa58 - I am trying to solve a problem with SynDB, more specifically with TSQLVirtualTableExternal. The problem is not concurrent access from different threads (as there would exist two different connections, that is correct). The External connections do use prepared statement cache (SynDBSQLite3 and SynDBZEOS connections). If I do a SQL with multiple JOINs to the same table using the same field constraints with different values, the problem is easily visible. The only solution with current sources is to disable the prepared statement cache completely.

Is there something I am missing?

Pages: 1

Board footer

Powered by FluxBB