Delphi 64bit sqlite compilation

johnnysynop · 2019-09-19 09:03:11

@ab Are you going to support oficially? Why only FPC x64 build is supported?
I need static linking of 64 bit obj, does anyone have?

ab · 2019-09-19 10:40:00

This is a Delphi linker issue.
It is not able to link properly .o files as generated by gcc.

johnnysynop · 2019-09-19 13:00:59

ab wrote:

This is a Delphi linker issue.
It is not able to link properly .o files as generated by gcc.

What about BCC 64? For those who has license on BCC it will be still useful.
Is there any technical problem with embarcadero c++ compiler x64 for sqlite object?

ab · 2019-09-19 14:34:10

I don't have any Bcc64 licence, so I can't use it.

Greg0r · 2019-09-19 21:06:05

@ab what about vc++ express and coff/comm converter?

ab · 2019-09-20 09:38:40

IIRC the problem was not about object file format, but reference linking issues.

pvn0 · 2019-09-20 12:01:21

I used to have this problem then I moved to FPC, problem solved

Greg0r · 2019-09-20 12:29:17

Please understand guys that it's not matter loving one or another but matter of business choices I like both FPC and Delphi the same.
Lack of full 64 bit support with static linking and encryption in SQLite is big pain in ass and makes such wonderful product like mormot little "retarded"

So if there is technical issue with object build let's open fund for @ab Arnaud and he will buy C++ builder compiler and make 64 bit version.

Last edited by Greg0r (2019-09-21 12:27:44)

emk · 2019-09-20 20:07:45

@ab

mORMot is a open-source project, so you can use C++ Builder Community Edition.

ab · 2019-09-20 20:11:57

The 10.3 community edition is using LLVM for Win64, so it would increase the problems in respect to the older bcc64 - at least this is what occurred last time I tried.
I will try doing something with an older bcc64 command line compiler I have available somewhere.

AFAIR there was some linking problem, but I will try to fix it.

Stay tuned!

johnnysynop · 2019-09-20 20:48:10

@ab, what a wonderfull news!
Just my suggestion - I would use 10.3.2 don't worry about older bcc64, if there is community edition for free now (finally) old licensed users of BCC 64 can download new.
I think in this case trying to satifying all could be not good idea.
I'd go with option of using latest LLVM from 10.3.2 community, I don't think it could be the problem for those who really need it will buy new version.
As Mormot is free project it should care more about free compilers rather than older which were paid.
Free is from 10.x v.

C++ compiler which is known from very serious bugs in the past first of x64 versions do another reason to use latest.

Last edited by johnnysynop (2019-09-20 20:49:42)

ab · 2019-09-21 09:10:47

Some results:

Windows 10 64bit (10.0.18362) (cp1252)
    4 x Intel(R) Core(TM) i7-3517U CPU @ 1.90GHz (x64)
Using mORMot 1.18.5367 PRTL
    TSQLite3LibraryStatic 3.29.0 with internal MM
Generated with: Delphi 10.3 Rio 64 bit compiler

Time elapsed for all tests: 1m31
Performed 2019-09-21 11:09:03 by User on ACER

Total assertions failed for all test suits:  0 / 41,852,771
! All tests passed successfully.

And encryption seemed to work as expected!

 2.9. External database:
  - TQuery: 2,003 assertions passed  19.47ms
  - SynDBRemote: 42,707 assertions passed  395.04ms
  - DB properties persistence: 7 assertions passed  1.22ms
  - External records: 2 assertions passed  290us
  - Auto adapt SQL: 942 assertions passed  7.40ms
  - Crypted database: 253,277 assertions passed  164.36ms
  - External via REST: 170,355 assertions passed  916.07ms
  - External via virtual table: 170,355 assertions passed  1.93s
  - External via REST with change tracking: 180,455 assertions passed  2.71s
  Total failed: 0 / 820,103  - External database PASSED  6.15s

I have committed it, with a dedicated sqlite3.o static file for Delphi+Win64.
It is now included in https://github.com/synopse/mORMot or you need to download again

See http://blog.synopse.info/post/2019/09/2 … lphi-Win64

Vitaly · 2019-09-21 11:12:34

Nice! Thank you!

Greg0r · 2019-09-21 12:54:19

Fantastic! Thank you so much..
1) Successfully build with bcc64 7.40 from 10.3 Community with warnings and without -6 switch
https://i.imgur.com/8d0jkGL.png
2) While build Delphi mormot sample and 64 bit .o build above, following errs:
[dcc64 Error] SynSQLite3Static.pas(1279): E2065 Unsatisfied forward or external declaration: '_FInf'
[dcc64 Error] SynSQLite3Static.pas(1279): E2065 Unsatisfied forward or external declaration: '_Log'
[dcc64 Fatal Error] TestSQL3.dpr(874): F2063 Could not compile used unit 'SynSQLite3Static.pas'

Last edited by Greg0r (2019-09-21 12:55:39)

pvn0 · 2019-09-22 07:30:23

That's so awesome, thanks ab!

emk · 2019-09-22 20:20:33

Thank you!

profh · 2019-09-22 23:20:14

@ab, thanks, and you are my hero.

johnnysynop · 2019-09-23 09:50:02

profh wrote:

@ab, thanks, and you are my hero.

mORMot king is getting even better now, wonderful. Thank you Arnaud.
Yes @ab is the boss, it reminds me this /watch?v=kKR1rrjvq-I from 1:36 to 1:41

BTW: Will it work with Community edition of c++ compiler?

ab · 2019-09-23 10:30:22

I used a XE7 bcc64 compiler.

Currently, the LLVM-based compiler (the one from 10.3) is not supported, so the Community edition will fail linking, as reported by Greg0r https://synopse.info/forum/viewtopic.ph … 241#p30241
Perhaps it may give better performance - even if we need real timing, not just marketing stuff.

IIRC I can't install Delphi and C++ Builder Community editions at the same time on the same machine.
This is why I didn't try to install it yet... building a VM just for a compiler is not worth it...

Greg0r · 2019-09-23 14:24:03

Yes, indeed preety strange that there is no RAD studio community and problem with installing both at same time.
Do you think sharing files would work? I could pack installed folder od 10.3 free and share archive for "lend"/test.

ab · 2019-09-24 08:48:34

Yes, I guess you could try to find out the correct files in the "bin" and "include" sub-folders...
Then send me an email with the link.

Greg0r · 2019-09-24 09:46:36

@ab, check msg

ab · 2019-09-24 11:07:15

I am looking into it now.

Greg0r · 2019-09-24 12:57:10

Ok thank you

ab · 2019-09-24 13:56:23

I was able to compile using the latest LLVM compiler, and let it link:

Windows 10 64bit (10.0.18362) (cp1252)
    4 x Intel(R) Core(TM) i7-3517U CPU @ 1.90GHz (x64)
Using mORMot 1.18.5377
    TSQLite3LibraryStatic 3.29.0 with internal MM
Generated with: Delphi 10.3 Rio 64 bit compiler

Time elapsed for all tests: 1m34
Performed 2019-09-24 15:54:15 by User on ACER

Total assertions failed for all test suits:  0 / 41,854,545
! All tests passed successfully.

So it seems not faster than the XE7 compiler.
But I guess I will use it, since you made a simple cut-down version of the command-line compiler, very easy to work with!

I will post the new sqlite3.o now.

ab · 2019-09-24 14:18:57

One issue is that with Delphi XE4 Win64, I have the following error with I use the LLVM-compiler Sqlite3.o:

Embarcadero Delphi for Win64 compiler version 25.0
Copyright (c) 1983,2013 Embarcadero Technologies, Inc.
D:\Dev\Lib\SynSQLite3Static.pas(1278) Fatal: F2084 Internal Error: URW1154

No problem with XE6:

Windows 10 64bit (10.0.18362) (cp1252)
    4 x Intel(R) Core(TM) i7-3517U CPU @ 1.90GHz (x64)
Using mORMot 1.18.5377
    TSQLite3LibraryStatic 3.29.0 with internal MM
Generated with: Delphi XE6 64 bit compiler

Time elapsed for all tests: 1m44
Performed 2019-09-24 16:16:09 by User on ACER

Total assertions failed for all test suits:  0 / 41,850,842
! All tests passed successfully.

Or XE7:

Windows 10 64bit (10.0.18362) (cp1252)
    4 x Intel(R) Core(TM) i7-3517U CPU @ 1.90GHz (x64)
Using mORMot 1.18.5377
    TSQLite3LibraryStatic 3.29.0 with internal MM
Generated with: Delphi XE7 64 bit compiler

Time elapsed for all tests: 1m48
Performed 2019-09-24 16:18:29 by User on ACER

Total assertions failed for all test suits:  0 / 41,852,585
! All tests passed successfully.

(see above for the Delphi 10.3 CE)

Note that the "time elapsed for all tests" above is not very consistent, and should not be taken as a proof of the performance of each compiler, since I did some other things in parallel on my computer, so a few seconds difference seems as expected.

Please check https://synopse.info/fossil/info/753c2bbe1d

Greg0r · 2019-09-24 15:03:05

Quick test, No problem noticed with Delphi 10.3.2 will continue testing.

Using mORMot 1.18.5378
    TSQLite3LibraryStatic 3.29.0 with internal MM
Generated with: Delphi 10.3 Rio 64 bit compiler

Time elapsed for all tests: 1m01
Performed 2019-09-24 16:54:48 by Greg0r on DESKTOP-GR

As for noticed problems looks like .o from new compiler must be used with XE6+ without issues
Anyway great results and lot of Delphi's are covered assuming usage of both "normal" and newer llvm compiler.

Last edited by Greg0r (2019-09-24 15:11:38)

Eugene Ilyin · 2019-09-24 20:58:20

ab, Greg0r,

I have a slightly different issue compared to Greg0r's 30241#p30241 on the most recent 1.18.5378 version with Delphi 10.3 Update 2 on Windows 64-bit target platform and sqlite3.o file taken from https://synopse.info/files/sqlite3obj.7z as it mentioned on Blog's article and the Download section.

The missed declarations are slightly different (for example log instead of _log):

[dcc64 Error] SynSQLite3Static.pas(1277): E2065 Unsatisfied forward or external declaration: '_fltused'
[dcc64 Error] SynSQLite3Static.pas(1277): E2065 Unsatisfied forward or external declaration: 'log'
[dcc64 Fatal Error] F2063 Could not compile used unit 'SynSQLite3Static.pas'

The sqlite3obj.7z archive contains sqlite3.o file modified Sep 20, 2019 (4 days ago).

ab is Sep 20 is the recent compiled sqlite3.o available (because you modified SynSQLite3Static code yesterday, Sep 23)?

I suggest to keep both declaration variants to support both compilers for the current available sqlite3.o
All that is needed is to add the next lines:

SynSQLite3Static.pas: 728

Before

function _log(x: double): double; cdecl; export;
begin
  result := ln(x);
end;

After

function _log(x: double): double; cdecl; export;
begin
  result := ln(x);
end;

function log(x: double): double; cdecl; export;
begin
  result := ln(x);
end;

SynSQLite3Static.pas: 751

Before

var
  _finf: double = 1.0 / 0.0; // compiles to some double infinity constant

After

var
  _finf: double = 1.0 / 0.0; // compiles to some double infinity constant

  _fltused: uint64 = 0; // required for linking with old non-LLVM bcc64 compiler

With these two small additions the compilations of 1.18.5378 with downloaded sqlite3.o under Delphi 10.3 Update 2 Windows 64-bit platform goes without errors.

Last edited by Eugene Ilyin (2019-09-24 22:47:06)

Greg0r · 2019-09-24 23:41:53

@Eugene Ilyin,

Nice addition anyway hmm strange for me works without this mod with D10.3.2 x64 under Win10x64 on .o included in .5378 as well in my own build of sqlite.o with bcc64 llvm

ab · 2019-09-25 07:58:31

Oups.... I guess I forgot to upload the new sqlite3.o file in https://synopse.info/files/sqlite3obj.7z
Should be available now.
Sorry!

I also added compatibility to link old non-LLVM bcc64 compiler in https://synopse.info/fossil/info/e3441c9689 as @Eugene proposed.

Eugene Ilyin · 2019-09-25 10:03:57

ab,

Confirmed. Thanks!
With updated sqlite3.o all compiled as expected.

edwinsn · 2020-01-15 11:05:46

Hello ab,

I'm getting a "[dcc64 Fatal Error] F2084 Internal Error: AV0858E903-R343E15EC-0" error when trying to compile with XE4/Win64.
I'm wondering if there is a any workaround, I'm currently using dynamic linking (.dll)? I'm using the latest github repository committed on Jan 14, 2020.
Thanks.

Last edited by edwinsn (2020-01-15 11:43:33)

ab · 2020-01-15 13:35:46

This is sadly a XE4 bug for Win64.
I was not able to find out what was wrong during linking - I tested with XE7 and later with no problem.

The only workaround I know is indeed to use dynamic linking with an external .dll - or switch to FPC or a newer Delphi compiler.

edwinsn · 2020-01-15 15:00:31

@ab,

No problem, I'll use dynamic linking for now.

ab · 2020-01-15 16:52:09

We still keep in synch the Win64 .dll with the latest SQLite3 stable version, as the static version.
But you loose the ability to encrypt the DB file...

edwinsn · 2020-01-15 17:52:16

In this case I don't need to encrypt the sqlite db file, I guess I don't lose any other features aside from slightly slower performance, right?

BTW, for dynamic linking I should download the sqlite3.dll from sqlite.org but not from somewhere on synopse.info, right?

ab · 2020-01-16 14:26:37

If you use the one from synopse.info, you will have the same SQLite3 conditionals than with the static version, so it could help in some cases.

edwinsn · 2020-01-16 17:47:57

Thank you, ab.
I downloaded sqlite3-64.dll from http://synopse.info/files/SQLite3-64.7z, and found it's 600+KB smaller than the official one from sqlite.org, not sure why. I assume it has all the FTS4/5 and json1, right?

ab · 2020-01-17 09:47:57

IIRC we use VC++ RTL library for some part, so it may be smaller for this reason.
FTS and JSON1 are included in our DLL.
Just the same conditionals than the static .o/.obj - except the encryption.

mORMot Open Source

#1 2019-09-19 09:03:11

Delphi 64bit sqlite compilation

#2 2019-09-19 10:40:00

Re: Delphi 64bit sqlite compilation

#3 2019-09-19 13:00:59

Re: Delphi 64bit sqlite compilation

#4 2019-09-19 14:34:10

Re: Delphi 64bit sqlite compilation

#5 2019-09-19 21:06:05

Re: Delphi 64bit sqlite compilation

#6 2019-09-20 09:38:40

Re: Delphi 64bit sqlite compilation

#7 2019-09-20 12:01:21

Re: Delphi 64bit sqlite compilation

#8 2019-09-20 12:29:17

Re: Delphi 64bit sqlite compilation

#9 2019-09-20 20:07:45

Re: Delphi 64bit sqlite compilation

#10 2019-09-20 20:11:57

Re: Delphi 64bit sqlite compilation

#11 2019-09-20 20:48:10

Re: Delphi 64bit sqlite compilation

#12 2019-09-21 09:10:47

Re: Delphi 64bit sqlite compilation

#13 2019-09-21 11:12:34

Re: Delphi 64bit sqlite compilation

#14 2019-09-21 12:54:19

Re: Delphi 64bit sqlite compilation

#15 2019-09-22 07:30:23

Re: Delphi 64bit sqlite compilation

#16 2019-09-22 20:20:33

Re: Delphi 64bit sqlite compilation

#17 2019-09-22 23:20:14

Re: Delphi 64bit sqlite compilation

#18 2019-09-23 09:50:02

Re: Delphi 64bit sqlite compilation

#19 2019-09-23 10:30:22

Re: Delphi 64bit sqlite compilation

#20 2019-09-23 14:24:03

Re: Delphi 64bit sqlite compilation

#21 2019-09-24 08:48:34

Re: Delphi 64bit sqlite compilation

#22 2019-09-24 09:46:36

Re: Delphi 64bit sqlite compilation

#23 2019-09-24 11:07:15

Re: Delphi 64bit sqlite compilation

#24 2019-09-24 12:57:10

Re: Delphi 64bit sqlite compilation

#25 2019-09-24 13:56:23

Re: Delphi 64bit sqlite compilation

#26 2019-09-24 14:18:57

Re: Delphi 64bit sqlite compilation

#27 2019-09-24 15:03:05

Re: Delphi 64bit sqlite compilation

#28 2019-09-24 20:58:20

Re: Delphi 64bit sqlite compilation

#29 2019-09-24 23:41:53

Re: Delphi 64bit sqlite compilation

#30 2019-09-25 07:58:31

Re: Delphi 64bit sqlite compilation

#31 2019-09-25 10:03:57

Re: Delphi 64bit sqlite compilation

#32 2020-01-15 11:05:46

Re: Delphi 64bit sqlite compilation

#33 2020-01-15 13:35:46

Re: Delphi 64bit sqlite compilation

#34 2020-01-15 15:00:31

Re: Delphi 64bit sqlite compilation

#35 2020-01-15 16:52:09

Re: Delphi 64bit sqlite compilation

#36 2020-01-15 17:52:16

Re: Delphi 64bit sqlite compilation

#37 2020-01-16 14:26:37

Re: Delphi 64bit sqlite compilation

#38 2020-01-16 17:47:57

Re: Delphi 64bit sqlite compilation

#39 2020-01-17 09:47:57

Re: Delphi 64bit sqlite compilation

Board footer