mORMot Open Source

SoftTech

Member

Registered: 2016-01-19

Posts: 10

Memory Leak with TDocVariantData

I am creating hierarchical data using TDocVariantData and it seems to be working fine, but when I exit the system I have numerous memory leaks that seem to be directly attributable.

The relevant code is below.  I couldn't find a better way, using TDocVariantData, to add data to create the hierarchy.  The call to Clear is *NOT* cleaning up the memory and I can't figure out how to go about writing my own that does the cleanup for me.

Any ideas??

procedure SomeProcedure;
var
  DocVariantData: TDocVariantData;
  LocalImageCacheItem: ILocalImageCacheItem;
  BillingCodes: String;
begin
  DocVariantData := SomethingOrOther.CreateDocVariantData;
  Try
    SomethingElse.AddToDocVariantData(DocVariantData);

    For LocalImageCacheItem In TLocalImageCache.List Do
      If LocalImageCacheItem.ImageName.StartsWith('Embed_') Then
        DocVariantData.AddValue('Image:' + LocalImageCacheItem.ImageName,
                                LocalImageCacheItem.BitmapAsURL);

    BillingCodes := SomethingElse.BillingCodes;
    If BillingCodes <> String.Empty Then
      DocVariantData.AddValue('Billing:Codes',
                              BillingCodes + String.Empty);

    DocVariantData.AddValue('DataObjectAttribute:Values',
                            Variant(AnotherObject.CreateDocVariantData));

    //TFile.WriteAllText('Debug.json', String(VariantSaveJson(DocVariantData)));

    // Other stuff happens...
  Finally
    DocVariantData.Clear;
  End;
end;

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: Memory Leak with TDocVariantData

First of all, DocVariantData.Clear is not needed here, nor the try..finally block.
Memory is handled by the TDocVariantData record definition.

I guess your problem comes from

  DocVariantData := SomethingOrOther.CreateDocVariantData;

Here you are copying a TDocVariantData.
I guess the instance is copied BY REFERENCE, so the values still exist in SomethingOrOther.CreateDocVariantData when SomeProcedure leaves.

So you have to create a new copy of the data.
See http://synopse.info/files/html/Synopse% … l#TITLE_40

SoftTech

Member

Registered: 2016-01-19

Posts: 10

Re: Memory Leak with TDocVariantData

I got rid of the Try...Finally...End.  Every one of these methods are being implemented as something like:

function TSomethingOrOther.CreateDocVariantDataObject: TDocVariantData;
begin
  Result.Init([dvoValueCopiedByReference]);
  Result.AddValue('Something', 'Value');
  Result.AddValue(Whatever, YetAnotherObject.CreateDocVariantData);
end;

function TYetAnotherObject.CreateDocVariantData: TDocVariantData;
begin
  Result.Init([dvoValueCopiedByReference]);
  Result.AddValue('SomethingElse', 'Another Value');
end;

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: Memory Leak with TDocVariantData

Please read the doc (follow the link I supplied in my last post) about dvoValueCopiedByReference

Some optimization may take place by defining

procedure TSomethingOrOther.CreateDocVariantDataObject(out result: TDocVariantData);

to ensure there is a single TDocVariantData instance, with no copy.

But as your code is written, it should not leak memory at all.
And I can assure you that TDocVariant, in normal use, does not leak memory.
It is proven by the regression tests, and our servers running 24/7.

Please provide a simple test sample project which actually leaks memory.

SoftTech

Member

Registered: 2016-01-19

Posts: 10

Re: Memory Leak with TDocVariantData

I did read the article you pointed me to...  please note the init call...

Very very strange!  I converted all of my functions to procedures precisely like what you show and the memory leak disappears completely.

I don't understand this at all and I tried to create a small project that used functions just like what I've shown you.  That project did not leak.

Is it time for me to declare victory and rely on the procedure instead of the function or do you have any other ideas?  I'm really thrown for a loss!

Another question - when I use dvoValueCopiedByReference am I passing around the entire structure on the heap or am I passing around just a pointer to memory?  I'm hoping that it's the latter...

Last edited by SoftTech (2016-02-23 16:00:12)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: Memory Leak with TDocVariantData

When you use dvoValueCopiedByReference, you are passing the TDocVariantData structure memory, and both VValues[] and VNames[] dynamic arrays are passed by reference, using the reference counting feature of Delphi dynamic arrays.

In short, all names and values are not copied, the arrays containing them have their reference count incremented/decremented to manage the memory.

Memory allocation is only the TDocVariantData record itself, i.e. a few bytes.

SoftTech

Member

Registered: 2016-01-19

Posts: 10

Re: Memory Leak with TDocVariantData

Perfect!  I'm going to declare victory and go work on something else instead.

Thanks for your support!  I can live with converting from functions to procedures.  I can't live with the enormous memory leak I was experiencing in a routine called pretty frequently.