Possible Memory Leak on version 1.18 ?

O.Schwab · 2014-04-01 13:22:51

I'm using mORMot 1.18 with Delphi XE2 and I *seem* to have huge memory leaks :

Each query I perform on the database seems to leak a memory block.

Tracing with FASTMM4 gives me the following log :

A memory block has been leaked. The size is: 64164

This block was allocated by thread 0x16A4, and the stack trace (return addresses) at the time was:
404B22 [System][@GetMem$qqri]
57EDEC [SynSQLite3Static.pas][SynSQLite3Static][malloc$qui][250]
58230F [SynSQLite3Static]
582A93 [SynSQLite3Static][sqlite3_memory_highwater$qi]
5D3280 [SynSQLite3Static][sqlite3_shutdown$qv]
5D4D6D [SynSQLite3Static][sqlite3_errmsg$qui]
5D4DE4 [SynSQLite3Static][sqlite3_open$qpcrui]
4FF6EF [SynSQLite3.pas][SynSQLite3][TSQLDatabase.DBOpen$qqrv][3406]
4FE32D [SynSQLite3.pas][SynSQLite3][TSQLDatabase.$bctr$qqrx20System.UnicodeStringx31System.%AnsiStringT$us$i65001$%i][2967]
51171D [SynDBSQLite3.pas][SynDBSQLite3][TSQLDBSQLite3Connection.Connect$qqrv][393]
50FD59 [SynDB.pas][SynDB][TSQLDBStatement.Prepare$qqrx31System.%AnsiStringT$us$i65001$%o][5497]

The block is currently used for an object of class: Unknown

The allocation number is: 4439

Current memory dump of 256 bytes starting at pointer address 7EE11570:
00 FA 00 00 00 00 00 00 00 00 00 00 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80
80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80
80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80
80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80
80 80 80 80 80 80 80 80 78 15 E1 7E 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80
80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80
80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80
80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80
.  ú  .  .  .  .  .  .  .  .  .  .  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €
€  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €
€  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €
€  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €
€  €  €  €  €  €  €  €  x  .  á  ~  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €
€  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €
€  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €
€  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €  €

I've got a lot of these leak blocks reported, with various size.
I think I've checked all my objects creation and liberation, so I don't think that the problem is in my code. I've tested the last (today 01/04/14) trunk source of mORMot

Any ideas what's wrong ?

Ty

ab · 2014-04-01 15:16:02

I just launched the framework's regression tests (TestSQL3.dpr), which has a pretty big code coverage base.
More than 35,000,000 individual checks performed, in fact.

It also tests most of the Sqlite3 engine features, including virtual tables and external engines.

FastMM4 did not report any memory leak nor corruption in FullDebuggMode....

Could you try to run TestSQL3.dpr ?
I suspect there may be something wrong in your code.

Try to isolate the problem.
We need some code to reproduce the issue here, if there is something to fix.

O.Schwab · 2014-04-01 15:50:33

All tests in TestSQL3 are OK, good news for you, bad for me

Couldn't it be linked to the static SQLite 3 feature ?

The procedures performing the leak are apparently dead simple :

Procedure TTraducteurSIRWEB.AjouteLibellePrg(pIDLibelle: Integer; ConnexionBDD: TSQLDBSQLite3Connection);
Var
  stmt  : TSQLDBSQLite3Statement;
  id    : Integer;
  IdPrgm: Integer;
Begin
  stmt := TSQLDBSQLite3Statement.Create(ConnexionBDD);
  stmt.Execute('SELECT ID FROM LibellesProgramme WHERE Dest = ' + IntToStr(pIDLibelle) + ' AND SOURCE =' + IntToStr(FProgramme.GetID) + ';', false);
  stmt.step();
  id := stmt.ColumnInt('ID');
  If id = 0 Then
    Begin
      stmt.Execute('INSERT INTO LibellesProgramme VALUES(null,' + IntToStr(pIDLibelle) + ',' + IntToStr(FProgramme.GetID) + ');', false);
    End;
  stmt.free;
End;

Procedure TTraducteurSIRWEB.AjouteCorres(pIDLibelle: Integer; ConnexionBDD: TSQLDBSQLite3Connection);
Var
  stmt: TSQLDBSQLite3Statement;
Begin
  stmt := TSQLDBSQLite3Statement.Create(ConnexionBDD);
  stmt.Execute('INSERT INTO Corres VALUES(null,' + IntToStr(pIDLibelle) + ');', true);
  stmt.step;
  stmt.Free;
End;

Last edited by O.Schwab (2014-04-01 16:05:00)

ab · 2014-04-01 18:19:24

OK.

So you are using SynDBSQlite3.pas layer over SynSQLite.pas.
This is a good idea, since it may help you switch to any other SynDB-compatible classes.

But you are not using the SynDB classes as they should!

You should not work at connection level, nor handle the statements by hand.
The main entry point is the TSQLDBConnectionProperties class.

From one instance of it:
- you retrieve statements via NewThreadSafeStatementPrepared() overloaded methods, which return a ISQLDBStatement;
- or even better, you call Execute() and ExecuteNoResult() methods.
See the documentation of those methods, and how the SynDBExplorer tool work, for instance.

O.Schwab · 2014-04-02 10:23:34

Hi Arnaud,

All is OK now using Execute() and ExecuteNoResult() .

Sry for the inconvenience ....

O.Schwab · 2014-05-15 12:51:53

Hi Arnaud,

C'est reparti pour un tour.

My trainee has left my project with a huge memory leak, he tried to get rid of it but didn't succeed.
My first thought was he must have done something wrong.
But he did not.

in fact , it seems that there are leaks on query text (?) when query is not cached.

I can reproduce the phenomenon using this code :

Var
  rowsync   : ISQLDBRows;
  CnxBDDProp: TSQLDBSQLite3ConnectionProperties;
Begin
    CnxBDDProp := TSQLDBSQLite3ConnectionProperties.Create(Configuration.Options['BDD'].AsString, '', '', '');

    rowsync := CnxBDDProp.Execute('SELECT ID FROM ' + table + ' WHERE ID = ' + IntToStr(i), []);  // this is leaking
    rowsync := CnxBDDProp.Execute('SELECT ID FROM ' + table + ' WHERE ID = ?' , [i ]); // this is not leaking

    CnxBDDProp.Free;

End;

Here are other examples of query in leaking and not leaking version :

    
    // LEAKING
    row := CnxBDDProp.Execute('SELECT name FROM sqlite_master WHERE type =''table'' AND name like ''Libelle\_%'' ESCAPE ''\''', []);
    row := CnxBDDProp.Execute('SELECT name FROM sqlite_master WHERE type ="table" ', []);

    // NOT LEAKING
    row := CnxBDDProp.Execute('SELECT ''Libelle_FR'' as name', []);
    row := CnxBDDProp.Execute('SELECT name FROM sqlite_master', []);

As you can see , it seems that if query contains a ? or does'nt contains WHERE , it's cached and not leaking. In the other case, it leaks.

(tested with today's NightlyBuild)

Any idea what' wrong ?

TY !

ab · 2014-05-15 15:48:59

You should NEVER mix ISQLDBRows and local TSQLDBSQLite3ConnectionProperties instances in the same method.
Due to hidden try..finally generated code by the compiler to release rowsync, it is a mess.
This is a restriction of the compiler.
Well identified since the beginning (see the forum and the documentation).

Please try with a TSQLDBSQLite3ConnectionProperties instance which is created and freed outside of the method.

See for instance http://synopse.info/forum/viewtopic.php?id=852

O.Schwab · 2014-05-16 06:49:49

Sry again for that lack of RTFM, Arnaud.

Finally, at the stage where I am, using mORMot in my project is like trying to kill a fly with a warhammer.

Your documentation is good but far too heavy for what I do with your framework.

mORMot Open Source

#1 2014-04-01 13:22:51

Possible Memory Leak on version 1.18 ?

#2 2014-04-01 15:16:02

Re: Possible Memory Leak on version 1.18 ?

#3 2014-04-01 15:50:33

Re: Possible Memory Leak on version 1.18 ?

#4 2014-04-01 18:19:24

Re: Possible Memory Leak on version 1.18 ?

#5 2014-04-02 10:23:34

Re: Possible Memory Leak on version 1.18 ?

#6 2014-05-15 12:51:53

Re: Possible Memory Leak on version 1.18 ?

#7 2014-05-15 15:48:59

Re: Possible Memory Leak on version 1.18 ?

#8 2014-05-16 06:49:49

Re: Possible Memory Leak on version 1.18 ?

Board footer