mORMot Open Source

Anze

Member

Registered: 2023-08-08

Posts: 2

mormot.core.text.ObjectToJson & PatchCode

Hi,

we just started implementing ObjectToJson from the mORMot 2 library, which works insanely fast btw, and we encountered on an issue that makes the solution unusable for us, either we have a memory leak (with our quick and dirty solution - details below) or memory check fails.

We are writing our software in a business area where code part of the executable in the ram has to be periodically checked and checksum has to match all the time or the software has to lock itself till the next restart.

ObjectToJson causes the call to PatchCode, which changes the code part in the memory and so the memory check fails.

We have at least two options to resolve this:
a) let our memory check module know which Bytes where changed and ignore them (we would need a waiver from the certificate agency to do this, so not the best solution for us)
b) disable the RTTI optimization that is caused by ObjectToJson
c) ?

We are looking for somebody who would help us with b) (or c) ) (the donation is implicit in this case)

PS: our quick and dirty solution, which causes memory leaks:

in procedure TRttiCustom.SetValueClass(aClass: TClass; aInfo: PRttiInfo);

//  vmt := Pointer(PAnsiChar(aClass) + vmtAutoTable);
//  if vmt^ = nil then
//    PatchCodePtrUInt(pointer(vmt), PtrUInt(self), {leaveunprotected=}true);
//  if vmt^ <> self then
//    raise ERttiException.CreateUtf8(
//      '%.SetValueClass(%): vmtAutoTable set to %', [self, aClass, vmt^]);

and

in function TRttiCustomList.DoRegister(Info: PRttiInfo): TRttiCustom;

//  if FindType(Info) <> result then // paranoid check
//    raise ERttiException.CreateUtf8('%.DoRegister(%)?', [self, Info.RawName]);

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: mormot.core.text.ObjectToJson & PatchCode

The runtime patch trick is used not only for ObjectToJson but for other processes...

The easiest may be to notify the agent that the patches are done.
You can patch the classes ahead of time, once at startup, calling the Rtti.RegisterClasses([....]) method.

Otherwise, you would need to be able to by-pass the vmtAutoTable lookup, and replace it with another process.
There are only a few dozen places where it is done, and could be changed using a conditional.
We could just replace it with a plain Rtti.Find() and its hash-table-of-the-poor process, like other (non-class) types... It would be less fast, but working, and without any memory leak...

BTW, which compiler are you using? Delphi? FPC? On which OS do you run the executable?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: mormot.core.text.ObjectToJson & PatchCode

Please try https://github.com/synopse/mORMot2/commit/fb49b066

If you define the NOVMTPATCH conditional for the project (and mORMot package on Lazarus) it should not patch the executable at runtime.
It is slightly slower than the default vmtAutoTable lookup trick, but it is not dead slow either.

Edit: I have renamed the conditional as NOPATCHVMT for consistency, with some fixing and optimization.
It should be not much slower in practice.

Anze

Member

Registered: 2023-08-08

Posts: 2

Re: mormot.core.text.ObjectToJson & PatchCode

Hi Arnaud,

We've tried your fixes and they work perfectly.

Measured performance impact on serialize and deserialize (1.000.000 times repeat in a separate performance project, with NOPATCHVMT, i7-12700H, Win10):
- of plain Integer is 1% faster with NOPATCHVMT
- of large record (50 vars) is 1% faster with NOPATCHVMT

We are using the latest version of Delphi (11.4) on Windows 10 embedded.

We are very thankful for your quick and perfect solution, the donation is on it's way.

Last edited by Anze (2023-08-09 13:29:27)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: mormot.core.text.ObjectToJson & PatchCode

Thanks a lot for the feedback.
My guess is that this 1% speed difference is within the error margin, so it sounds fine to me.
From the regression tests, I don't see any performance difference, and the slowest search path is almost never used by Rtti.FindType(). The simple caching mechanism we put in place seems to work well in practice.

We are happy that you like the solution.