mORMot Open Source

tfopc

Member

Registered: 2024-01-08

Posts: 36

Problem/Bug - TSqlDBOdbcStatement.BindColumns cause endless loop

Hi ab,

after i give a software modul a try with the current source after some time, the software didn't start and runs in a busy loop...

Delphi7, Orm, Postgres, we are using the odbc-driver

after some debugging is see that the reason seems to be how the data is transfered to json. After each "numeric" element there are some "#0"

Here is the shortend Delphi (Debug-)Var content:

'[{"id":1'#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0',"lastchange":"2025-05-13T14:54:33","createdon":"2022-09-12T08:35:10","name":"TF","location":1'#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0'}]'#$A

Endless loop than happens in TOrmTableJson.ParseAndConvert GetFieldCountExpanded

I checked the older versions last working version with using the orm was from 24.06.25, with source from 12.05.2025, next git fetch and compiletime was 09.07.2025. so the breaking change i think was done in this timerange.

I also give the mormot.db.sql.postgres a try and use the libpq directly. But the currently used dll can't be loaded...
'TSqlDBPostgresLib.Resolve('PQFreeMem'): not found in libpq.dll'

The libpq.dll is used also from firedac in the same software. So it works in general.

Any hints? Resolution?

libpq.dll updates will be a litle bit problematic on many clients :-/

... ohh i see that "FreeMem" isn't used anywhere so i uncomment it and also google ai says:

"PGFreeMem is not part of the official PostgreSQL C library libpq, but rather a function used in the C++ library libpqxx. libpq is the C API for PostgreSQL and provides functions for communicating with the database server, while libpqxx is a C++ abstraction layer for interacting with libpq. PGFreeMem is used in libpqxx to free memory that was allocated by the underlying C library libpq."

After that the software starts again. but now there are some other problems the need to be resolved....
... next week - to much debugging for friday ;-)

tfopc

Member

Registered: 2024-01-08

Posts: 36

Re: Problem/Bug - TSqlDBOdbcStatement.BindColumns cause endless loop

FYI: the problem with "PQFreeMem" seems to be a case beautify search and replace problem ;-)
I looked via dll-export-viewer in the libpq.dll - the function ist "PGfreemem"
so

https://github.com/synopse/mORMot2/blob … s.pas#L350

should be lowercase made again.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 15,328

Website

Re: Problem/Bug - TSqlDBOdbcStatement.BindColumns cause endless loop

You are right.

Please try with
https://github.com/synopse/mORMot2/commit/4bf8b6a31

Thanks for the feedback, and investigation.

tfopc

Member

Registered: 2024-01-08

Posts: 36

Re: Problem/Bug - TSqlDBOdbcStatement.BindColumns cause endless loop

Hi ab,

i saw your commits on odbc from yesterday. so i give the problem (see first post, that brings me to the dessison to switch the db-interface) a new try with odbc. But the problem with the endless-loop in
TOrmTableJson.ParseAndConvert - GetFieldCountExpanded is still there because of the invalid data returned from TSqlDBOdbcStatement - BindColumns / GetData.
But the buffer now looks different to my first try: '[{"id":1'
Debugged in mormot.orm.core line 6056.
But this data also ends in a endless loop in "mormot.core.json - GetFieldCountExpanded"

Isn't the missing check in GetFieldCountExpanded critical, if someone put invalid data in the function?

Didn't test with a unicode Delphi-Version. Only with D7.

Thanks.

Last edited by tfopc (2025-09-11 09:18:07)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 15,328

Website

Re: Problem/Bug - TSqlDBOdbcStatement.BindColumns cause endless loop

It sounds like if the JSON was not copied before parsing, and it is parsed twice, which does not work.
It may be a caching problem.

We need a minimal reproducible example.

ThomasKalten

Member

Registered: 2022-11-15

Posts: 8

Re: Problem/Bug - TSqlDBOdbcStatement.BindColumns cause endless loop

Hi all,

the problem is based in the odbc.getData call, if the result is of type SQL_C_CHAR or SQL_C_WCHAR.
The fColData[ColIndex] of TSqlDBOdbcStatement of type TRawByteString will hold the result and has a predefined length, which does not change after the call to getData.
Since the API is C it is terminated with a 0-Char.

Later (TSqlDBOdbcStatement.ColumnToJson) this data is added to JSON with W(riter).AddString, which does not recognize for the 0 termination and takes the full length of fColData[ColIndex].

It does not run into this endless loop, if I change the length of fColData[ColIndex] to the correct length, which is returned by getData.

I think,, it it is not a good idea to change the length, because the length of fColData[ColIndex] is only set in bindcolumns, which probably is only called before retrieving all the data and not before each record.

So I suggest to save the length of the results of odbc.getData also in an array fColDataLen, which can be then be used to limit the data added in ColumnToJSon.

Viele Grüße
Thomas Kaltenbrunner

ThomasKalten

Member

Registered: 2022-11-15

Posts: 8

Re: Problem/Bug - TSqlDBOdbcStatement.BindColumns cause endless loop

After this post, I saw, that the size is already saved to the "fcolumns", so this small change solves this problem:

line 904 in mormot.db.sql.odbc.pas
-          W.AddString(fColData[Col]);  // already as SQL_C_CHAR
+          W.AddString(copy(fColData[Col], 1, ColumnDataSize));  // already as SQL_C_CHAR

Viele Grüße
Thomas Kaltenbrunner

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 15,328

Website

Re: Problem/Bug - TSqlDBOdbcStatement.BindColumns cause endless loop

Nice catch!

Please try with
https://github.com/synopse/mORMot2/commit/0bbf52579

Anyway, for PostgreSQL, I would not use ODBC but our own direct libpq client unit.
It would be faster and has more features (e.g. asynchronous execution).

ThomasKalten

Member

Registered: 2022-11-15

Posts: 8

Re: Problem/Bug - TSqlDBOdbcStatement.BindColumns cause endless loop

The fix works.

Viele Grüße
Thomas Kaltenbrunner