New Statics for mORMot

ab · 2021-03-27 14:23:58

I have published the new statics for mORMot.
At least!
(there was some delay after stabilization of https://www.sqlite.org/releaselog/3_35_3.html for a few days...)

It includes the latest 3.35.3 revision of SQLite3 for both mORMot revisions.

For mORMot 2, a lot of new features, mainly:
- refactoring of the static linking (no need of libgcc.a anymore on most platforms) including the new mormot.lib.static.pas unit;
- new libdeflate, used for zip/deflate/gzip in-memory compression on FPC Intel (but Win64 due to a FPC linking bug) - much faster than the plain zlib;
- new QuickJS embedded but complete ES2020 JavaScript engine, patched for mORMot embedding, with FPC and Delphi static linking.
See https://github.com/synopse/mORMot2/releases/tag/3.35.3

More info about how the statics were build is available in the https://github.com/synopse/mORMot2/tree … res/static folder.

edwinsn · 2021-03-27 14:41:37

Well done! So QuickJS is now available in mORMot2

L_VV · 2021-03-28 12:03:01

Good afternoon,

Today I updated mORMot2 to the latest trunk version and downloaded new static libraries.
After that, when I open in Lazarus a project using mORMot2, and do "Clean up and Build...", two link errors occurrs:

multiple definition of `libdeflate_malloc'
multiple definition of `libdeflate_free'

Verbose: Linking /home/data/projects/TestAppSrv/appsrv.console

Warning: linker: /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libdeflate.a(utils.o): in function `libdeflate_malloc':
Error: (.text+0x30): multiple definition of `libdeflate_malloc'; /home/data/projects/TestAppSrv/dependencies/fpc_components/3rd_party/mORMot2/packages/lazarus/lib/x86_64-linux/mormot.lib.z.o:/home/data/projects/TestAppSrv/dependencies/fpc_components/3rd_party/mORMot2/packages/lazarus//../../src/lib/mormot.lib.z.pas:928: first defined here

Warning: linker: /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libdeflate.a(utils.o): in function `libdeflate_free':
Error: (.text+0x40): multiple definition of `libdeflate_free'; /home/data/projects/TestAppSrv/dependencies/fpc_components/3rd_party/mORMot2/packages/lazarus/lib/x86_64-linux/mormot.lib.z.o:/home/data/projects/TestAppSrv/dependencies/fpc_components/3rd_party/mORMot2/packages/lazarus//../../src/lib/mormot.lib.z.pas:934: first defined here

appsrv.console.lpr(174,1) Error: Error while linking
appsrv.console.lpr(174,1) Verbose: There were 1 errors compiling module, stopping
Verbose: Compilation aborted
Verbose: /opt/fpcupdeluxe/lazarus_trunk/fpc/bin/x86_64-linux/ppcx64 returned an error exitcode

As a result, mORMot2 is not compiled: (

OS: SMP Debian 5.10.19-1 (2021-03-02) x86_64 GNU/Linux
Lazarus/fpc: 2.1.0 / 3.3.1 (trunk)

locate libdeflate.a
/usr/lib/x86_64-linux-gnu/libdeflate.a

ab · 2021-03-28 13:17:02

I am not able to reproduce it here with FPC 3.2.0 - not trunk.
We provide our own libdeflate.a version in the static/x86_64-linux folder.
Perhaps there is a conflict on your machine with the libdeflate.a installed on the system path using apt.

L_VV · 2021-03-28 17:39:19

ab wrote:

I am not able to reproduce it here with FPC 3.2.0 - not trunk.

I just checked on my stable version of Lazarus/fpc (2.0.12 / 3.2.0).
When I doing "Run / Clean up and Build..." occurs exactly the same error.

ab wrote:

We provide our own libdeflate.a version in the static/x86_64-linux folder.
Perhaps there is a conflict on your machine with the libdeflate.a installed on the system path using apt.

Yes, it is clear that this is due to the conflict with the system library.
It is not clear how to fix it.
I can not remove libdeflate system library, it is used by the OS itself.

To reproducing the error, it is enough to create a new console application in the Lazarus, and add to "uses", for example, the "mormot.orm.core" unit.
After that, we need to do "Run / Clean Up and Build ...", and if the designated libraries conflict is in the system, we will get the such error.
I tried to add the "mormot.core.base" unit to "uses", but in this case the error does not occur - likely libdeflate is not used in this case.

flydev · 2021-03-29 06:41:18

Compiled / run successfully on Delphi Sydney 10.4.2 Thanks for QuickJS

Last edited by flydev (2021-03-29 06:42:08)

ab · 2021-03-29 07:30:15

@L_VV
I don't see how a *.a library may be used for the system at runtime.
This is a dev library, used only when compiling/linking a project.
On debian, it is e.g. the https://packages.debian.org/sid/libdeflate-dev package which includes it IIRC.

Are you sure your project options point to our folder /static/x86_64-linux for its dependencies?

L_VV · 2021-03-29 09:35:13

Good afternoon!

ab wrote:

Are you sure your project options point to our folder /static/x86_64-linux for its dependencies?

My project contains the mormot2 package in the requirements.
In the mormot2' package options, the path to the libraries (_FL) is specified as follow:
../../static/$(TargetCPU)-$(TargetOS)

ab wrote:

@L_VV
I don't see how a *.a library may be used for the system at runtime.
This is a dev library, used only when compiling/linking a project.
On debian, it is e.g. the https://packages.debian.org/sid/libdeflate-dev package which includes it IIRC

Yes, you, of course, right - the libdeflate-dev package is not used at runtime.
I did not quite correctly wrote - I installed it when I builded a some application.
I can delete it, but if it needs to be needed again, it will have to be installed again.
In general, a small problem.

I checked it - after removing libdeflate-dev, indeed, mORMot2 compiled successfully.
After re-installing libdeflate-dev, mORMot2 again stopped compiled.
As a result, I deleted libdeflate-dev, as long as it is needed again.

However, it is still not clear why Lazarus/fpc is looking for all such libraries in the system, if the library already found in the directory, specified in the package options.
But since libdeflate-dev is not needed constantly, we can assume that everything is fine, and on most systems this error will not occurs.

ab · 2021-03-29 09:41:36

I may eventually rename our libraries to ensure there is no conflict with system .a files.

L_VV · 2021-03-29 10:34:59

Yes, may be it helps...

ab · 2021-03-29 15:15:46

I have renamed it to libdeflatepas.a for Intel.

The .7z release should be downloaded again.

L_VV · 2021-03-29 15:55:34

I updated mORMot2 to the current trunk, downloaded the latest static libraries, opened the mormot2 package in Lazarus IDE, and tried to compile it.
mormot.core.json unit does not compile with an error:
Wrong number of parameters specified for call to "<Procedure Variable>"

function TSynDictionary.ForEach(const OnMatch: TOnSynDictionary;
  KeyCompare, ValueCompare: TDynArraySortCompare; const aKey, aValue;
  Opaque: pointer): integer;
var
  k, v: PAnsiChar;
  i, n, ks, vs: PtrInt;
begin
  fSafe.Lock;
  try
    result := 0;
    if not Assigned(OnMatch) or
       (not Assigned(KeyCompare) and          <---
        not Assigned(ValueCompare)) then      <---
      exit;

mormot.core.json.pas(9345,32) Error: Wrong number of parameters specified for call to "<Procedure Variable>"
mormot.core.json.pas(9346,34) Error: Wrong number of parameters specified for call to "<Procedure Variable>"

mpv · 2021-03-29 17:09:47

@ab, for a long time I uses a SQLite in multi-thread mode (I implement a TSQLDBConnectionPropertiesThreadSafe for SQLite3 to read from FTS tables (a single thread writer and multi-thread reader) and it works for me even on 200Gb FTS database. Is this still possible after updating to 3.35.x on Windows when SQLITE_NO_THREAD is defined for win64?

itSDS · 2021-03-29 19:42:53

@ab i also have a question,

i compile my sqlite.obj sqlite.o with bcc32/64 from Rad Studio 10.4.2 and they work, is there a reason not to do this ?

ab · 2021-03-29 19:50:07

@L_VV
I am not able to reproduce the problem.
Please check your source folders.

@mpv
SQLITE_NO_THREAD was set because the beginthreadex/endthreadex redirections without libmsvcrt.a appeared to be buggy in mORMot 2...
But it may change the expected behavior of SQlite3, especially with FTS.
I will set it back to the previous behavior, and I guess we are fine too with mORMot 2.

@itSDS
For Delphi, we also use C++ Builder.
But here we speak about FPC, which uses gcc/mingw for Win32/Win64.

AOG · 2021-03-29 20:22:47

I guess L_VV compiles with FPC trunk.
https://synopse.info/forum/viewtopic.ph … 542#p34542

L_VV · 2021-03-29 20:52:30

AOG wrote:

I guess L_VV compiles with FPC trunk.
https://synopse.info/forum/viewtopic.ph … 542#p34542

Exactly, I compiled in the Lazarus / FPC trunk.

After replacement

    if not Assigned(OnMatch) or
       (not Assigned(KeyCompare) and
        not Assigned(ValueCompare)) then
      exit;

to

    if not Assigned(OnMatch) or
        (not (Assigned(KeyCompare) or
        Assigned(ValueCompare))) then
      exit;

everything was compiled successfully.

Thank you very much, AOG!
And where can I read, what change in fpc was done?

Arnaud, could it be possible to change this block in the mORMot2 trunk?

L_VV · 2021-03-29 21:09:00

And I just installed again libdeflate-dev.

After changing the file name in mORMot to libdeflatepas.a, everything is also compiled without problems.

Great!

Last edited by L_VV (2021-03-29 21:09:26)

AOG · 2021-03-30 06:42:15

And where can I read, what change in fpc was done?

Dunno. Its a FPC compiler error. Hard to isolate. Easy to work around.

ab · 2021-03-30 07:20:05

Please try https://github.com/synopse/mORMot2/comm … c536afe983

L_VV · 2021-03-30 08:25:55

All compiled successfully, thank you!

Leslie7 · 2021-04-02 22:20:00

ab,

there is one more sqlite release (3.35.4.) with a few fixes.

ab · 2021-04-03 07:22:16

Indeed.
Hope they will stabilize this version!
- anyway, yesterday I merged the latest https://github.com/bellard/quickjs version so it was time for new binaries, anyway.

Please check https://github.com/synopse/mORMot2/rele … ite.3.35.4
and https://github.com/synopse/mORMot/commi … 4ef6f62add

Leslie7 · 2021-04-03 08:59:32

Lightning fast.
Thank you!

mORMot Open Source

#1 2021-03-27 14:23:58

New Statics for mORMot

#2 2021-03-27 14:41:37

Re: New Statics for mORMot

#3 2021-03-28 12:03:01

Re: New Statics for mORMot

#4 2021-03-28 13:17:02

Re: New Statics for mORMot

#5 2021-03-28 17:39:19

Re: New Statics for mORMot

#6 2021-03-29 06:41:18

Re: New Statics for mORMot

#7 2021-03-29 07:30:15

Re: New Statics for mORMot

#8 2021-03-29 09:35:13

Re: New Statics for mORMot

#9 2021-03-29 09:41:36

Re: New Statics for mORMot

#10 2021-03-29 10:34:59

Re: New Statics for mORMot

#11 2021-03-29 15:15:46

Re: New Statics for mORMot

#12 2021-03-29 15:55:34

Re: New Statics for mORMot

#13 2021-03-29 17:09:47

Re: New Statics for mORMot

#14 2021-03-29 19:42:53

Re: New Statics for mORMot

#15 2021-03-29 19:50:07

Re: New Statics for mORMot

#16 2021-03-29 20:22:47

Re: New Statics for mORMot

#17 2021-03-29 20:52:30

Re: New Statics for mORMot

#18 2021-03-29 21:09:00

Re: New Statics for mORMot

#19 2021-03-30 06:42:15

Re: New Statics for mORMot

#20 2021-03-30 07:20:05

Re: New Statics for mORMot

#21 2021-03-30 08:25:55

Re: New Statics for mORMot

#22 2021-04-02 22:20:00

Re: New Statics for mORMot

#23 2021-04-03 07:22:16

Re: New Statics for mORMot

#24 2021-04-03 08:59:32

Re: New Statics for mORMot

Board footer