mORMot Open Source

sjerinic

Member

Registered: 2013-02-11

Posts: 51

mORMot.pas

Is it possible to use mORMot.pas in FPC (Code Typon) on Linux?
Compiler gives me an error: mORMot.pas(1071,2) Fatal: Can not open include file "Synopse.inc". I need to use some functions from mORMot.pas like a JSONFileToObject. Is there any other alternative?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: mORMot.pas

As stated by the doc, the server is only for Win32 and Win64.

But the clients can be built with FPC on Linux, with our SynCrossPlatform units.

sjerinic

Member

Registered: 2013-02-11

Posts: 51

Re: mORMot.pas

OK, that means I have to replace the functions like JSONFileToObject/ObjectToJSONFile from mormot.pas with some other on the client side.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: mORMot.pas

You will find in SynCrossPlatformJSON.pas the TJSONVariant record (and associated custom variant type - but ensure you read the FPC limitations/bugs in the doc) and some general purpose marshalling functions.
Those are faster and use less resources than the official "fpjson" unit, AFAIK.

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

Is it possible to use mORMot.pas in FPC (Code Typon) on Linux?
Compiler gives me an error: mORMot.pas(1071,2) Fatal: Can not open include file "Synopse.inc". I need to use some functions from mORMot.pas like a JSONFileToObject. Is there any other alternative?

It seems that this issue is not yet solved ? That is to say, attempt to compile "01 - In Memory ORM" using CodeTyphon 5.0 on CentOS 7.0 still fails complaining  mORMot.pas (XXXX,X) Fatal: Can not open include file "Synopse.inc".

I read in other posts that the restful orm works great with all the advances in the FPC compatility. Could you help to comment ?

Last edited by ComingNine (2014-10-17 14:11:24)

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Hello,

You have to configure CodeTyphon to find the include file !

https://drive.google.com/file/d/0B96fg3 … sp=sharing

Add the path to mORMot (the path to synopse.inc) to the include files !!

And please give us a littlebit more time (few days) .... we are fighting with the last few bugs to enable full mORMot on Linux.
It's 99.99% ready now !!

Greetings, Alfred.

Last edited by AOG (2014-10-17 15:17:27)

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

And .... if you want to test and help ....

An executable for Linux i386: https://drive.google.com/file/d/0B96fg3 … sp=sharing

An executable for Linux ARM: https://drive.google.com/file/d/0B96fg3 … sp=sharing

Both designed for Arch Linux, but I think ok for other Linux systems !

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

Hello,

You have to configure CodeTyphon to find the include file !

https://drive.google.com/file/d/0B96fg3 … sp=sharing

Add the path to mORMot (the path to synopse.inc) to the include files !!

And please give us a littlebit more time (few days) .... we are fighting with the last few bugs to enable full mORMot on Linux.
It's 99.99% ready now !!

Greetings, Alfred.

First of all, thank you for your excellent work very much !

Thank you for the tip. It should be noted that after CodeTyphon is configured to find the include file, the compilation of the "01 - In Memory ORM" using CodeTyphon 5.0 under CentOS 7.0 still fails but complains about several things:
(1) The case of file name of "SynLZ".
(2) The case of file name of "synopseCommit.inc"
(3) The ambiguity of "W.Add(ID);" in "procedure TSQLPropInfoRTTIID.GetJSONValues" in "mORMot.pas".
(4) The asm in "procedure x64FakeStub; ".

Looking forward to have full mORMot on Linux !

Last edited by ComingNine (2014-10-18 02:12:25)

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

And .... if you want to test and help ....

An executable for Linux i386: https://drive.google.com/file/d/0B96fg3 … sp=sharing

An executable for Linux ARM: https://drive.google.com/file/d/0B96fg3 … sp=sharing

Both designed for Arch Linux, but I think ok for other Linux systems !

It seems the executable has met some dead lock. Please see the log:

[root@localhost Downloads]# ./testi386 

   Synopse mORMot Framework Automated tests
  ------------------------------------------


1. Synopse libraries

 1.1. Low level common: 
  - System copy record: 20 assertions passed  193us
  - TRawUTF8List: 70,005 assertions passed  48.66ms
  - TDynArray: 1,027,706 assertions passed  312.20ms
  - TDynArrayHashed: 1,200,629 assertions passed  168.29ms
  - TObjectListHashed: 999,758 assertions passed  549.17ms
  - TObjectDynArrayWrapper: 167,501 assertions passed  35.61ms
  - Fast string compare: 7 assertions passed  23us
  - IdemPropName: 30 assertions passed  28us
  - Url encoding: 132 assertions passed  1.82ms
  - GUID: 9,005 assertions passed  7.49ms
  - IsMatch: 599 assertions passed  238us
  - Soundex: 35 assertions passed  46us
  - Numerical conversions: 1,114,439 assertions passed  453.67ms
  - crc32c: 20,020 assertions passed  114.26ms
      pas 33.02ms 180.6 MB/s fast 13.78ms 432.9 MB/s
  - Curr 64: 20,053 assertions passed  3.30ms
  - CamelCase: 11 assertions passed  36us
  - Bits: 4,774 assertions passed  112us
  - Ini files: 7,004 assertions passed  75.85ms
  - UTF8: 82,106 assertions passed  1.12s
  - Iso 8601 date and time: 36,015 assertions passed  5.72ms
  - Url decoding: 1,100 assertions passed  394us
  - Mime types: 23 assertions passed  51us
  - TSynTable: 457 assertions passed  7.71ms
  - TSynCache: 404 assertions passed  215us
  - TSynFilter: 804 assertions passed  4.72ms
  - TSynValidate: 677 assertions passed  935us
  - TSynLogFile: 36 assertions passed  1.32ms
  Total failed: 0 / 4,763,350  - Low level common PASSED  2.91s

 1.2. Cryptographic routines: 
  - Adler32: 1 assertion passed  62us
  - MD5: 1 assertion passed  51us
  - SHA1: 5 assertions passed  58us
  - SHA256: 5 assertions passed  70us
  - AES256: 12,177 assertions passed  976.47ms
  - RC4: 1 assertion passed  34us
  - Base64: 11,994 assertions passed  334.42ms
  - CompressShaAes: 1,683 assertions passed  7.07ms
  Total failed: 0 / 25,867  - Cryptographic routines PASSED  1.31s

 1.3. Compression: 
  - In memory compression: 12 assertions passed  597.78ms
  - GZIP format: 19 assertions passed  1.11s
  - SynLZO: 3,006 assertions passed  173.47ms
  - SynLZ: 21,010 assertions passed  794.32ms
  Total failed: 0 / 24,047  - Compression PASSED  2.68s


2. mORMot

 2.1. File based: 

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Thanks for your feedback !
Our very nice Linux mORMot needs sqlite3 installed on your system ....
This deadlock happens on systems without sqlite3 !

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

Thanks for your feedback !
Our very nice Linux mORMot needs sqlite3 installed on your system ....
This deadlock happens on systems without sqlite3 !

Do you happen to know how to "install 32 bit sqlite on 64 bit CentOS" ?

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Unfortunately I cannot help you ....

I only found this on the web:

https://www.syncovery.com/documentation … t-linuxes/

Few minutes ago, I gave Ab the latest FPC / Linux patches ... so keep in touch with the forum / timeline !

warleyalex

Member

From: Sete Lagoas-MG, Brasil

Registered: 2013-01-20

Posts: 250

Re: mORMot.pas

Hi, I've downloaded the executable for Linuxi386 and installed Sqlite3 on Ubuntu that way:

wget http://br.archive.ubuntu.com/ubuntu/poo … 1_i386.deb
wget http://br.archive.ubuntu.com/ubuntu/poo … 1_i386.deb

sudo dpkg -i libsqlite3-0_3.8.6-1_i386.deb
sudo dpkg -i libsqlite3-dev_3.8.6-1_i386.deb
---------------------------------------------------

Synopse mORMot Framework Automated tests
------------------------------------------

1. Synopse libraries

1.1. Low level common:
  - System copy record: 20 assertions passed  305us
  - TRawUTF8List: 70,005 assertions passed  453.12ms
  - TDynArray: 1,027,706 assertions passed  3.13s
  - TDynArrayHashed: 1,200,629 assertions passed  2.09s
  - TObjectListHashed: 999,770 assertions passed  5.63s
  - TObjectDynArrayWrapper: 167,501 assertions passed  365.54ms
  - Fast string compare: 7 assertions passed  297us
  - IdemPropName: 30 assertions passed  299us
  - Url encoding: 132 assertions passed  17.38ms
  - GUID: 9,005 assertions passed  96.74ms
  - IsMatch: 599 assertions passed  653us
  - Soundex: 35 assertions passed  71us
  - Numerical conversions: 1,114,209 assertions passed  4.73s
  - crc32c: 20,020 assertions passed  852.11ms
      pas 261.20ms 22.8 MB/s fast 89.67ms 66.5 MB/s
  - Curr 64: 20,053 assertions passed  40.00ms
  - CamelCase: 11 assertions passed  282us
  - Bits: 4,774 assertions passed  2.19ms
  - Ini files: 7,004 assertions passed  667.67ms
  - UTF8: 82,106 assertions passed  9.45s
  - Iso 8601 date and time: 36,015 assertions passed  81.53ms
  - Url decoding: 1,100 assertions passed  1.06ms
  - Mime types: 23 assertions passed  127us
  - TSynTable: 457 assertions passed  42.59ms
  - TSynCache: 404 assertions passed  428us
  - TSynFilter: 804 assertions passed  30.69ms
  - TSynValidate: 677 assertions passed  4.61ms
  - TSynLogFile: 36 assertions passed  22.64ms
  Total failed: 0 / 4,763,132  - Low level common PASSED  27.74s

1.2. Cryptographic routines:
  - Adler32: 1 assertion passed  138us
  - MD5: 1 assertion passed  77us
  - SHA1: 5 assertions passed  94us
  - SHA256: 5 assertions passed  111us
  - AES256: 12,177 assertions passed  12.04s
  - RC4: 1 assertion passed  308us
  - Base64: 11,994 assertions passed  2.84s
  - CompressShaAes: 1,683 assertions passed  72.95ms
  Total failed: 0 / 25,867  - Cryptographic routines PASSED  14.96s

1.3. Compression:
  - In memory compression: 12 assertions passed  3.82s
  - GZIP format: 19 assertions passed  7.82s
  - SynLZO: 3,006 assertions passed  1.32s
  - SynLZ: 21,010 assertions passed  5.48s
  Total failed: 0 / 24,047  - Compression PASSED  18.46s

2. mORMot

2.1. File based:
  - Database direct access: 10,135 assertions passed  4.26s
  - Virtual table direct access: 12 assertions passed  7.99ms
  - TSQLTableJSON: 106,068 assertions passed  1.16s
  - TSQLRestClientDB: 310,071 assertions passed  11.85s
  Total failed: 0 / 426,286  - File based PASSED  17.29s

2.2. File based memory map:
  - Database direct access: 10,135 assertions passed  4.65s
  - Virtual table direct access: 12 assertions passed  186.99ms
  - TSQLTableJSON: 106,068 assertions passed  1.53s
  - TSQLRestClientDB: 310,071 assertions passed  14.05s
  Total failed: 0 / 426,286  - File based memory map PASSED  20.43s

2.3. File based WAL:
  - Database direct access: 10,135 assertions passed  5.17s
  - Virtual table direct access: 12 assertions passed  7.95ms
  - TSQLTableJSON: 106,068 assertions passed  1.36s
  - TSQLRestClientDB: 310,071 assertions passed  15.67s
  Total failed: 0 / 426,286  - File based WAL PASSED  22.21s

2.4. Memory based:
  - Database direct access: 10,135 assertions passed  5.64s
  - Virtual table direct access: 12 assertions passed  4.00ms
  - TSQLTableJSON: 106,068 assertions passed  1.47s
  - TSQLRestClientDB: 402,396 assertions passed  22.70s
  - RTree: 140,000 assertions passed  13.97s
  Total failed: 0 / 658,611  - Memory based PASSED  43.81s

2.5. Basic classes:
  - TSQLRecord: 76 assertions passed  14.09ms
  - TSQLRecordSigned: 200 assertions passed  117.68ms
  - TSQLModel: 3 assertions passed  3.93ms
  - TSQLRestServerFullMemory: 562,309 assertions passed  15.93s
  Total failed: 0 / 562,588  - Basic classes PASSED  16.07s

2.6. Client server access:
! - TSQLHttpServer: 1 / 3 FAILED  11.89ms
! - TSQLHttpClient: 1 / 1 FAILED  1.07ms
! - HTTP client keep alive: 1 / 1 FAILED  329us
! - HTTP client multi connect: 1 / 1 FAILED  2.34ms
! - HTTP client encrypted: 1 / 1 FAILED  259us
  - Direct in process access: 356 assertions passed  180.29ms

! Exception ECrtSocket raised with messsage:
!  Socket bind creation error on 0.0.0.0:888 (13)

Synopse framework used: 1.18.337
SQlite3 engine used: 3.8.6
Generated with: Free Pascal 2.7.1 compiler
Tests performed at 10/18/2014 09:22:18 PM

Total assertions failed for all test suits:  0 / 7,313,103
! Some tests FAILED: please correct the code.
Done - Press ENTER to Exit

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Thank you for testing.
And sorry about this.

On Linux, access to port 888 is limited to root processes.
(and I did perform the tests as root ... so I forgot about it)

I have made another executable, that uses port 8888, that is also available for non-root users.

https://drive.google.com/file/d/0B96fg3 … sp=sharing

Please report ... ;-)

Greetings and good luck.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: mORMot.pas

Please see  for the latest revision of the source code.
Feature request http://synopse.info/fossil/tktview/3a79adc10f9 seems almost finished.
Thanks Alfred (AOG) for your hard work!

We have still one big issue with FPC which prevents to use interface based services.
See http://bugs.freepascal.org/view.php?id=26774

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Thanks Ab !!

Please note that this post (http://synopse.info/forum/viewtopic.php … 073#p13073) still lists some valid problems !
I am busy with solving these problems.

But cross-compiling for Linux i386 and ARM will now be successfull !

warleyalex

Member

From: Sete Lagoas-MG, Brasil

Registered: 2013-01-20

Posts: 250

Re: mORMot.pas

OS: Xubuntu - Ubuntu 12.10
For non-root users this is awesome!

Synopse mORMot Framework Automated tests
----------------------------------------------------
1. Synopse libraries

1.1. Low level common:
  - System copy record: 20 assertions passed  316us
  - TRawUTF8List: 70,005 assertions passed  235.14ms
  - TDynArray: 1,027,706 assertions passed  1.49s
  - TDynArrayHashed: 1,200,629 assertions passed  900.29ms
  - TObjectListHashed: 999,769 assertions passed  2.78s
  - TObjectDynArrayWrapper: 167,501 assertions passed  206.99ms
  - Fast string compare: 7 assertions passed  306us
  - IdemPropName: 30 assertions passed  1.07ms
  - Url encoding: 132 assertions passed  5.16ms
  - GUID: 9,005 assertions passed  45.20ms
  - IsMatch: 599 assertions passed  1.31ms
  - Soundex: 35 assertions passed  308us
  - Numerical conversions: 1,114,356 assertions passed  2.30s
  - crc32c: 20,020 assertions passed  431.10ms
      pas 119.26ms 50.0 MB/s fast 51.32ms 116.2 MB/s
  - Curr 64: 20,053 assertions passed  22.59ms
  - CamelCase: 11 assertions passed  361us
  - Bits: 4,774 assertions passed  505us
  - Ini files: 7,004 assertions passed  311.45ms
  - UTF8: 82,106 assertions passed  4.58s
  - Iso 8601 date and time: 36,015 assertions passed  38.91ms
  - Url decoding: 1,100 assertions passed  1.26ms
  - Mime types: 23 assertions passed  363us
  - TSynTable: 457 assertions passed  4.57ms
  - TSynCache: 404 assertions passed  672us
  - TSynFilter: 804 assertions passed  10.59ms
  - TSynValidate: 677 assertions passed  2.86ms
  - TSynLogFile: 36 assertions passed  15.86ms
  Total failed: 0 / 4,763,278  - Low level common PASSED  13.41s

1.2. Low level types:
  - Url encoding: 200 assertions passed  4.10ms
  - Encode decode JSON: 250,616 assertions passed  1.07s
  - Variants: 5 assertions passed  57us
  Total failed: 0 / 250,821  - Low level types PASSED  1.08s

1.3. Cryptographic routines:
  - Adler32: 1 assertion passed  107us
  - MD5: 1 assertion passed  77us
  - SHA1: 5 assertions passed  91us
  - SHA256: 5 assertions passed  114us
  - AES256: 12,177 assertions passed  5.62s
  - RC4: 1 assertion passed  350us
  - Base64: 11,994 assertions passed  1.34s
  - CompressShaAes: 1,683 assertions passed  35.59ms
  Total failed: 0 / 25,867  - Cryptographic routines PASSED  7.01s

1.4. Compression:
  - In memory compression: 12 assertions passed  1.75s
  - GZIP format: 19 assertions passed  3.64s
  - SynLZO: 3,006 assertions passed  631.60ms
  - SynLZ: 21,010 assertions passed  2.56s
  Total failed: 0 / 24,047  - Compression PASSED  8.58s

2. mORMot

2.1. File based:
  - Database direct access: 10,136 assertions passed  1.78s
  - Virtual table direct access: 12 assertions passed  4.06ms
  - TSQLTableJSON: 106,068 assertions passed  517.22ms
  - TSQLRestClientDB: 310,071 assertions passed  6.14s
  Total failed: 0 / 426,287  - File based PASSED  8.44s

2.2. File based memory map:
  - Database direct access: 10,136 assertions passed  1.85s
  - Virtual table direct access: 12 assertions passed  4.63ms
  - TSQLTableJSON: 106,068 assertions passed  707.87ms
  - TSQLRestClientDB: 310,071 assertions passed  6.41s
  Total failed: 0 / 426,287  - File based memory map PASSED  8.98s

2.3. File based WAL:
  - Database direct access: 10,136 assertions passed  1.85s
  - Virtual table direct access: 12 assertions passed  4.30ms
  - TSQLTableJSON: 106,068 assertions passed  848.63ms
  - TSQLRestClientDB: 310,071 assertions passed  6.06s
  Total failed: 0 / 426,287  - File based WAL PASSED  8.77s

2.4. Memory based:
  - Database direct access: 10,136 assertions passed  1.95s
  - Virtual table direct access: 12 assertions passed  3.83ms
  - TSQLTableJSON: 106,068 assertions passed  491.00ms
  - TSQLRestClientDB: 402,396 assertions passed  9.46s
  - RTree: 140,000 assertions passed  5.85s
  Total failed: 0 / 658,612  - Memory based PASSED  17.78s

2.5. Basic classes:
  - TSQLRecord: 77 assertions passed  6.73ms
  - TSQLRecordSigned: 200 assertions passed  48.22ms
  - TSQLModel: 3 assertions passed  428us
  - TSQLRestServerFullMemory: 562,309 assertions passed  6.42s
  Total failed: 0 / 562,589  - Basic classes PASSED  6.48s

2.6. Client server access:
  - TSQLHttpServer: 2 assertions passed  4.62ms
     using Synsock - Synapse Platform Independent Socket Layer.514
  - TSQLHttpClient: 3 assertions passed  115.87ms
  - HTTP client keep alive: 387 assertions passed  5.28s
  - HTTP client multi connect: 387 assertions passed  704.62ms
  - HTTP client encrypted: 387 assertions passed  5.26s
  - Direct in process access: 356 assertions passed  50.36ms
  - HTTP several DB servers: 1,504 assertions passed  34.23s
  Total failed: 0 / 3,026  - Client server access PASSED  45.65s

2.7. External database:
  - TQuery: 2,003 assertions passed  38.36ms
  - External records: 2 assertions passed  3.97ms
  - Auto adapt SQL: 624 assertions passed  242.04ms
  - Crypted database: 176,192 assertions passed  928.88ms
!  - External via REST: 21,981 / 168,339 FAILED  11.62s
!  - External via virtual table: 21,981 / 168,339 FAILED  12.12s
!  - External via REST with change tracking: 22,992 / 178,439 FAILED  17.64s
  Total failed: 66,954 / 693,938  - External database FAILED  42.61s

2.8. Multi thread process:
  - Create thread pool: 1 assertion passed  49.17ms
  - TSQLRestServerDB: 4,822 assertions passed  2.80s
     1=4324/s  2=3140/s  5=988/s  10=419/s  30=582/s  50=803/s 
  - TSQLRestClientDB: 4,822 assertions passed  887.31ms
     1=2635/s  2=3981/s  5=3587/s  10=3191/s  30=2711/s  50=1752/s 
  - Locked: 4,822 assertions passed  940.58ms
     1=2642/s  2=1885/s  5=2767/s  10=3445/s  30=3004/s  50=2508/s 
  - Unlocked: 4,822 assertions passed  1.03s
     1=3303/s  2=2019/s  5=2035/s  10=2036/s  30=2830/s  50=2609/s 
  - Background thread: 4,822 assertions passed  1.48s
     1=3321/s  2=1533/s  5=1470/s  10=1500/s  30=1739/s  50=1297/s 
  - Main thread: 4,822 assertions passed  677.80ms
     1=3023/s  2=4284/s  5=4316/s  10=4153/s  30=3640/s  50=3071/s 
  Total failed: 0 / 28,933  - Multi thread process PASSED  7.88s

Synopse framework used: 1.18.354
SQlite3 engine used: 3.8.6
Generated with: Free Pascal 2.7.1 compiler
Tests performed at 10/19/2014 08:43:51 AM

Total assertions failed for all test suits:  66,954 / 8,289,972
! Some tests FAILED: please correct the code.
Done - Press ENTER to Exit

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

Unfortunately I cannot help you ....

I only found this on the web:

https://www.syncovery.com/documentation … t-linuxes/

Few minutes ago, I gave Ab the latest FPC / Linux patches ... so keep in touch with the forum / timeline !

Thank you for your efforts !

Could you help to comment:
(1) whether it is possible to statically link SQLite when mORMot samples are compiled with CodeTyphon on Linux ?
(2) if mORMot is 100% compatible with FPC on Linux, what is the relation between previous ways of writing clients and current ways of writing clients using units under the crossplatform directory ?
(3) Can TSynLog capture the stacktrace if mORMot samples are compiled with CodeTyphon on Linux ?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: mORMot.pas

(1) not yet tested on Linux, but it should be possible, since we did it easily with FPC + MingGW under Windows

(2) from the client point of view, nothing changes.

(3) stack trace is not yet available on Linux. Logging works, but should be enhanced.

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

(1) not yet tested on Linux, but it should be possible, since we did it easily with FPC + MingGW under Windows

(2) from the client point of view, nothing changes.

(3) stack trace is not yet available on Linux. Logging works, but should be enhanced.

Many thanks for your comments ! A self-contained executable should be nice.

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

Compilation of the "01 - In Memory ORM" succeeds. Congratulations !

However, when I try to add the message, the "name" and the "msg" field are empty.
The "Project01.db" file contains

[
{1,0,"",""},
{2,0,"",""}]

Could you help to comment the reason and the workaround ?

OS: CentOS 7.0 x64
IDE: CodeTyphon Typhon64
mORMot: http://synopse.info/fossil/info/cdc7ed7 … ff5e33226c
SQLite:  (1) Status can be seen from yum. (2) Although it is written as 3.7.17, the library file is "/lib64/libsqlite3.so.0.8.6" and should thus be the latest. (3) It does not help to download from the official site and overwrite the one under the lib64 directory (4) It does not help to install also the i686 version from the repository.

[root@localhost ~]# yum info sqlite
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirrors.linode.com
 * extras: mirrors.linode.com
 * updates: mirrors.linode.com
Installed Packages
Name        : sqlite
Arch        : x86_64
Version     : 3.7.17
Release     : 4.el7
Size        : 795 k
Repo        : installed
From repo   : anaconda
Summary     : Library that implements an embeddable SQL database engine
URL         : http://www.sqlite.org/
License     : Public Domain
Description : SQLite is a C library that implements an SQL database engine. A
            : large subset of SQL92 is supported. A complete database is stored
            : in a single disk file. The API is designed for convenience and
            : ease of use. Applications that link against SQLite can enjoy the
            : power and flexibility of an SQL database without the
            : administrative hassles of supporting a separate database server.
            : Version 2 and version 3 binaries are named to permit each to be
            : installed on a single host

Available Packages
Name        : sqlite
Arch        : i686
Version     : 3.7.17
Release     : 4.el7
Size        : 396 k
Repo        : base/7/x86_64
Summary     : Library that implements an embeddable SQL database engine
URL         : http://www.sqlite.org/
License     : Public Domain
Description : SQLite is a C library that implements an SQL database engine. A
            : large subset of SQL92 is supported. A complete database is stored
            : in a single disk file. The API is designed for convenience and
            : ease of use. Applications that link against SQLite can enjoy the
            : power and flexibility of an SQL database without the
            : administrative hassles of supporting a separate database server.
            : Version 2 and version 3 binaries are named to permit each to be
            : installed on a single host

[root@localhost ~]# 

Last edited by ComingNine (2014-10-25 04:31:21)

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

Thanks for your feedback !
Our very nice Linux mORMot needs sqlite3 installed on your system ....
This deadlock happens on systems without sqlite3 !

Thank you for testing.
And sorry about this.

On Linux, access to port 888 is limited to root processes.
(and I did perform the tests as root ... so I forgot about it)

I have made another executable, that uses port 8888, that is also available for non-root users.

https://drive.google.com/file/d/0B96fg3 … sp=sharing

Please report ... ;-)

Greetings and good luck.

Using the test executable in your post above, there is still a deadlock even if SQLite has been installed...
Could you help to comment ?

OS: CentOS 7.0 x64
IDE: CodeTyphon Typhon64
mORMot: http://synopse.info/fossil/info/cdc7ed7 … ff5e33226c
SQLite:  (1) Status can be seen from yum. (2) Although it is written as 3.7.17, the library file is "/lib/libsqlite3.so.0.8.6" and "/lib64/libsqlite3.so.0.8.6" and should thus be the latest.

[root@localhost Downloads]# yum info sqlite
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirrors.linode.com
 * extras: mirrors.linode.com
 * updates: mirrors.linode.com
Installed Packages
Name        : sqlite
Arch        : i686
Version     : 3.7.17
Release     : 4.el7
Size        : 798 k
Repo        : installed
From repo   : base
Summary     : Library that implements an embeddable SQL database engine
URL         : http://www.sqlite.org/
License     : Public Domain
Description : SQLite is a C library that implements an SQL database engine. A
            : large subset of SQL92 is supported. A complete database is stored
            : in a single disk file. The API is designed for convenience and
            : ease of use. Applications that link against SQLite can enjoy the
            : power and flexibility of an SQL database without the
            : administrative hassles of supporting a separate database server.
            : Version 2 and version 3 binaries are named to permit each to be
            : installed on a single host

Name        : sqlite
Arch        : x86_64
Version     : 3.7.17
Release     : 4.el7
Size        : 795 k
Repo        : installed
From repo   : anaconda
Summary     : Library that implements an embeddable SQL database engine
URL         : http://www.sqlite.org/
License     : Public Domain
Description : SQLite is a C library that implements an SQL database engine. A
            : large subset of SQL92 is supported. A complete database is stored
            : in a single disk file. The API is designed for convenience and
            : ease of use. Applications that link against SQLite can enjoy the
            : power and flexibility of an SQL database without the
            : administrative hassles of supporting a separate database server.
            : Version 2 and version 3 binaries are named to permit each to be
            : installed on a single host

[root@localhost Downloads]# 

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Strange.

Compiled mORMot 1.18.392 .

Installed Ach Linux 64 bit.
And installed lib32-glibc, lib32-sqlite, lib32-zlib, lib32-gcc-libs.

Result:
mORMot runs as expected inside my 64 bit Arch VirtualBox.

Sidenote:
I did sent some patches for mORMot to allow for static binding with Sqlite on Linux.
For self-contained executables. Keep in touch with the timeline / forum.

Sidenote2:
mORMot is not yet 64bit FPC ready.
You still have to compile as 32bit executable on Linux.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: mORMot.pas

Using the test executable in your post above, there is still a deadlock even if SQLite has been installed...
Could you help to comment ?

I guess this is because 3.7.17 is too old.
We use some feature of the latest revision.

You should use your own .so - from http://www.sqlite.org/howtocompile.html

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

Strange.

Compiled mORMot 1.18.392 .

Installed Ach Linux 64 bit.
And installed lib32-glibc, lib32-sqlite, lib32-zlib, lib32-gcc-libs.

Result:
mORMot runs as expected inside my 64 bit Arch VirtualBox.

Sidenote:
I did sent some patches for mORMot to allow for static binding with Sqlite on Linux.
For self-contained executables. Keep in touch with the timeline / forum.

Sidenote2:
mORMot is not yet 64bit FPC ready.
You still have to compile as 32bit executable on Linux.

Thank you for your efforts and comments ! Right now I could not get CodeTyphon's Typhon64 to cross-compile via FPC32 on CentOS 7. Various packages are missing and I could not find them in CentOS repository.

Looking forward to the static binding with SQLite, and 64bit compatibility on Linux !

Last edited by ComingNine (2014-10-25 12:29:15)

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

I guess this is because 3.7.17 is too old.
We use some feature of the latest revision.

You should use your own .so - from http://www.sqlite.org/howtocompile.html

Thank you for your comments ! Compiling 32bit SQLite from source on 64bit CentOS 7.0 helps.

[root@localhost Downloads]# export LD_LIBRARY_PATH=/usr/local/lib ; ./test

   Synopse mORMot Framework Automated tests
  ------------------------------------------


1. Synopse libraries

 1.1. Low level common: 
  - System copy record: 20 assertions passed  118us
  - TRawUTF8List: 70,005 assertions passed  31.38ms
  - TDynArray: 1,027,706 assertions passed  309.93ms
  - TDynArrayHashed: 1,200,629 assertions passed  183.51ms
  - TObjectListHashed: 999,788 assertions passed  572.01ms
  - TObjectDynArrayWrapper: 167,501 assertions passed  35.51ms
  - Fast string compare: 7 assertions passed  25us
  - IdemPropName: 30 assertions passed  46us
  - Url encoding: 132 assertions passed  1.68ms
  - GUID: 9,005 assertions passed  6.47ms
  - IsMatch: 599 assertions passed  159us
  - Soundex: 35 assertions passed  32us
  - Numerical conversions: 1,117,835 assertions passed  511.30ms
  - crc32c: 20,020 assertions passed  110.29ms
      pas 32.62ms 182.8 MB/s fast 13.51ms 441.3 MB/s
  - Curr 64: 20,053 assertions passed  3.26ms
  - CamelCase: 11 assertions passed  27us
  - Bits: 4,774 assertions passed  85us
  - Ini files: 7,004 assertions passed  57.71ms
  - UTF8: 82,106 assertions passed  1.11s
  - Iso 8601 date and time: 36,015 assertions passed  5.71ms
  - Url decoding: 1,100 assertions passed  487us
  - Mime types: 23 assertions passed  201us
  - TSynTable: 457 assertions passed  1.03ms
  - TSynCache: 404 assertions passed  120us
  - TSynFilter: 804 assertions passed  3.21ms
  - TSynValidate: 677 assertions passed  621us
  - TSynLogFile: 36 assertions passed  1.76ms
  Total failed: 0 / 4,766,776  - Low level common PASSED  2.95s

 1.2. Low level types: 
  - Url encoding: 200 assertions passed  1.29ms
  - Encode decode JSON: 250,616 assertions passed  173.40ms
  - Variants: 5 assertions passed  26us
  Total failed: 0 / 250,821  - Low level types PASSED  174.83ms

 1.3. Cryptographic routines: 
  - Adler32: 1 assertion passed  42us
  - MD5: 1 assertion passed  42us
  - SHA1: 5 assertions passed  52us
  - SHA256: 5 assertions passed  36us
  - AES256: 12,177 assertions passed  993.36ms
  - RC4: 1 assertion passed  33us
  - Base64: 11,994 assertions passed  299.17ms
  - CompressShaAes: 1,683 assertions passed  5.17ms
  Total failed: 0 / 25,867  - Cryptographic routines PASSED  1.29s

 1.4. Compression: 
  - In memory compression: 12 assertions passed  600.99ms
  - GZIP format: 19 assertions passed  1.17s
  - SynLZO: 3,006 assertions passed  170.56ms
  - SynLZ: 21,010 assertions passed  832.11ms
  Total failed: 0 / 24,047  - Compression PASSED  2.78s


2. mORMot

 2.1. File based: 
  - Database direct access: 10,136 assertions passed  262.91ms
  - Virtual table direct access: 12 assertions passed  553us
  - TSQLTableJSON: 106,068 assertions passed  117.83ms
  - TSQLRestClientDB: 310,071 assertions passed  1.38s
  Total failed: 0 / 426,287  - File based PASSED  1.76s

 2.2. File based memory map: 
  - Database direct access: 10,136 assertions passed  229.55ms
  - Virtual table direct access: 12 assertions passed  582us
  - TSQLTableJSON: 106,068 assertions passed  105.71ms
  - TSQLRestClientDB: 310,071 assertions passed  1.38s
  Total failed: 0 / 426,287  - File based memory map PASSED  1.71s

 2.3. File based WAL: 
  - Database direct access: 10,136 assertions passed  236.65ms
  - Virtual table direct access: 12 assertions passed  597us
  - TSQLTableJSON: 106,068 assertions passed  104.38ms
  - TSQLRestClientDB: 310,071 assertions passed  1.46s
  Total failed: 0 / 426,287  - File based WAL PASSED  1.80s

 2.4. Memory based: 
  - Database direct access: 10,136 assertions passed  262.49ms
  - Virtual table direct access: 12 assertions passed  837us
  - TSQLTableJSON: 106,068 assertions passed  110.89ms
  - TSQLRestClientDB: 402,396 assertions passed  2.55s
  - RTree: 140,000 assertions passed  1.28s
  Total failed: 0 / 658,612  - Memory based PASSED  4.21s

 2.5. Basic classes: 
  - TSQLRecord: 77 assertions passed  1.05ms
  - TSQLRecordSigned: 200 assertions passed  8.59ms
  - TSQLModel: 3 assertions passed  64us
  - TSQLRestServerFullMemory: 562,309 assertions passed  1.54s
  Total failed: 0 / 562,589  - Basic classes PASSED  1.55s

 2.6. Client server access: 
  - TSQLHttpServer: 2 assertions passed  143.38ms
     using Synsock - Synapse Platform Independent Socket Layer.514
  - TSQLHttpClient: 3 assertions passed  34.41ms
  - HTTP client keep alive: 387 assertions passed  5.24s
  - HTTP client multi connect: 387 assertions passed  165.86ms
  - HTTP client encrypted: 387 assertions passed  5.24s
  - Direct in process access: 356 assertions passed  15.15ms
  - HTTP several DB servers: 1,504 assertions passed  34.32s
  Total failed: 0 / 3,026  - Client server access PASSED  45.17s

 2.7. External database: 
  - TQuery: 2,003 assertions passed  9.52ms
  - External records: 2 assertions passed  629us
  - Auto adapt SQL: 624 assertions passed  48.53ms
  - Crypted database: 176,192 assertions passed  219.16ms
  - External via REST: 168,339 assertions passed  2.02s
  - External via virtual table: 168,339 assertions passed  2.69s
  - External via REST with change tracking: 178,439 assertions passed  3.85s
  Total failed: 0 / 693,938  - External database PASSED  8.85s

 2.8. Multi thread process: 
  - Create thread pool: 1 assertion passed  15.04ms
  - TSQLRestServerDB: 4,822 assertions passed  493.50ms
     1=12141/s  2=13583/s  5=8495/s  10=4044/s  30=3112/s  50=2726/s  
  - TSQLRestClientDB: 4,822 assertions passed  230.97ms
     1=9839/s  2=15237/s  5=13795/s  10=12790/s  30=9323/s  50=7953/s  
  - Locked: 4,822 assertions passed  248.20ms
     1=15276/s  2=12477/s  5=12133/s  10=10952/s  30=7308/s  50=7220/s  
  - Unlocked: 4,822 assertions passed  215.93ms
     1=13576/s  2=13519/s  5=16025/s  10=14628/s  30=8716/s  50=8203/s  
  - Background thread: 4,822 assertions passed  419.83ms
     1=7062/s  2=6319/s  5=6387/s  10=5858/s  30=5238/s  50=4877/s  
  - Main thread: 4,822 assertions passed  272.44ms
     1=6747/s  2=9213/s  5=11397/s  10=11656/s  30=9132/s  50=8683/s  
  Total failed: 0 / 28,933  - Multi thread process PASSED  1.89s


Synopse framework used: 1.18.354
SQlite3 engine used: 3.8.7
Generated with: Free Pascal 2.7.1 compiler
Tests performed at 10/25/2014 12:36:07 PM

Total assertions failed for all test suits:  0 / 8,293,470
! All tests passed successfully.
                                Done - Press ENTER to Exit
[root@localhost Downloads]#

PS: There is little trouble in compiling 32bit SQLite from source on 64bit CentOS 7.0.
Could you help to comment how to properly compile 32bit SQLite from source on 64bit CentOS 7.0 ?

" make clean; export CFLAGS=-m32 ; ./configure ; make ; make install "

and

" make clean; CFLAGS=-m32 ./configure ; make ; make install "

both give 64bit library.

I have to append -m32 manually:

yum install install glibc-devel
/bin/sh ./libtool --tag=CC   --mode=compile gcc -DPACKAGE_NAME=\"sqlite\" -DPACKAGE_TARNAME=\"sqlite\" -DPACKAGE_VERSION=\"3.8.7\" -DPACKAGE_STRING=\"sqlite\ 3.8.7\" -DPACKAGE_BUGREPORT=\"http://www.sqlite.org\" -DPACKAGE_URL=\"\" -DPACKAGE=\"sqlite\" -DVERSION=\"3.8.7\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_FDATASYNC=1 -DHAVE_USLEEP=1 -DHAVE_LOCALTIME_R=1 -DHAVE_GMTIME_R=1 -DHAVE_DECL_STRERROR_R=1 -DHAVE_STRERROR_R=1 -DHAVE_POSIX_FALLOCATE=1 -I.    -D_REENTRANT=1 -DSQLITE_THREADSAFE=1  -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -g -O2 -MT sqlite3.lo -MD -MP -MF .deps/sqlite3.Tpo -m32 -c -o sqlite3.lo sqlite3.c
mv -f .deps/sqlite3.Tpo .deps/sqlite3.Plo
/bin/sh ./libtool --tag=CC   --mode=link gcc -D_REENTRANT=1 -DSQLITE_THREADSAFE=1  -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -g -O2 -no-undefined -version-info 8:6:8 -m32 -o libsqlite3.la -rpath /usr/local/lib sqlite3.lo  -ldl -lpthread 
/bin/sh ./libtool   --mode=install /bin/install -c   libsqlite3.la '/usr/local/lib'

Last edited by ComingNine (2014-10-25 12:42:48)

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Good result !! Glad to hear !!
However, I do not have detailed knowledge about compiling on Centos 64 bit.
But, looking at your self-test-results, you are ready to go ahaead with mORMot.

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

Good result !! Glad to hear !!
However, I do not have detailed knowledge about compiling on Centos 64 bit.
But, looking at your self-test-results, you are ready to go ahaead with mORMot.

Thank you for your comments very much !

Could you help to comment further about the recommended way to build Linux exetuables ? For example:

(1) CodeTyphon on Win to cross-compile, or CodeTyphon on Linux ?

(1.1) If CodeTyphon on Win to cross-compile, how to generate the SQLite static library that is compatible with the cross-compile OS library that CodeTyphon ships ? I mean, CodeTyphon seems to have its own set of cross-compile OS library.

(1.2) If CodeTyphon on Linux, which Linux distro ? 32bit or 64bit OS?

(2) Some screen shots or kickstart video ?

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Sure !

1)

My personal situation determines my development setup.

I have a Win 8.1 64 bit laptop.
Therefor, I use CodeTyphon or Lazarus for Windows. Mostly 32 bit.

CodeTyphon makes crosscompiling very easy.
My setup: https://drive.google.com/file/d/0B96fg3 … sp=sharing

CodeTyphon Center ->  CrossBuild -> i386-linux.
This will build the crosscompiler and the RTL for Linux i386.

You will need some Linux i386 libraries.
Normally, I just copy all files from \lib on my Linux-system ... way to much libraries, but ok.
You can also download the needed library by using CodeTyphon Center: CodeTyphon Center ->  CrossBuild -> Download Libraries.

Next, you will need to inform CodeTyphon about the location of your Linux libraries: https://drive.google.com/file/d/0B96fg3 … sp=sharing

Add some extra's into the compiler options: https://drive.google.com/file/d/0B96fg3 … sp=sharing

Add things like this into your source files (dpr):

{$IFDEF FPC}
  {$MODE Delphi}
{$ENDIF}
{$I Synopse.inc} // define HASINLINE USETYPEINFO CPU32 CPU64 OWNNORMTOUPPER
uses
  // in your dpr :
  {$ifdef Linux}
  // if you use threads
  cthreads,
  // widestring manager for Linux if needed !!
  // could also be put in another unit ... but doc states: as early as possible
  // cwstring,
  {$endif}

You are ready to go.

There is also FPCUP : http://wiki.freepascal.org/fpcup . Makes installing and crosscompiling also very easy.

Naturally, if you have a Linux system available, it would be much easier to install CodeTyphon (or FPC [and Lazarus]) directly on this system !
No crosscompiling needed anymore.

1.1)
The Linux SQLite static library is made on the Linux system itself. Using gcc.
I have made some new pathes, including a script to build the sqlite static library on Linux. Keep in touch with the timeline.

(1.2)
As mORMot is not yet FPC 64 bit ready, I just use a 32 bit distro.
But 64 bit readiness is on its way ....

2)
Setting up CodeTyphon is very straight forward. Lots of info on the net.
Lazarus is an automatic install.
FPCUP needs some more effort, but has a good Wiki.

If you encounter problems, please report back !

We are still making improvements regarding FPC and Linux.
And hunt for bugs !!

Ps:
Perhaps it will be an option to include Linux sqlite3 object files with mORMot, as a separate download. Same as for Windows. You could ask Ab.

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

FPC lpr:

program LinuxSynTestFPCLinuxi386;

//{$IFDEF FPC}
  //{$MODE Delphi} // Already defined in Synopse.inc line 175, mORMot commit 228976b436782ae914dc098ab6cec5588aef0cec
//{$ENDIF}

{$I Synopse.inc} // define HASINLINE USETYPEINFO CPU32 CPU64 OWNNORMTOUPPER

{$APPTYPE CONSOLE}

uses
  {$ifdef Linux}
  cthreads,
  // widestring manager for Linux !!
  // could also be put in another unit ... but doc states: as early as possible
  cwstring,
  {$endif}
  mORMotSelfTests;

begin
  SQLite3ConsoleTests;
end.       

Test output:

   Synopse mORMot Framework Automated tests
  ------------------------------------------


1. Synopse libraries

 1.1. Low level common: 
  - System copy record: 20 assertions passed  20.00s
  - TRawUTF8List: 70,005 assertions passed  39869.00s
  - TDynArray: 1,027,706 assertions passed  280943.00s
  - TDynArrayHashed: 1,200,629 assertions passed  123358.00s
  - TObjectListHashed: 999,746 assertions passed  359072.00s
  - TObjectDynArrayWrapper: 167,501 assertions passed  34513.00s
  - Fast string compare: 7 assertions passed  7.00s
  - IdemPropName: 30 assertions passed  11.00s
  - Url encoding: 132 assertions passed  1301.00s
  - GUID: 9,005 assertions passed  6086.00s
  - IsMatch: 599 assertions passed  165.00s
  - Soundex: 35 assertions passed  10.00s
  - Numerical conversions: 1,113,690 assertions passed  431471.00s
  - crc32c: 20,020 assertions passed  79054.00s
      pas 22984.00s 272 B/s fast 10404.00s 601 B/s
  - Curr 64: 20,053 assertions passed  3187.00s
  - CamelCase: 11 assertions passed  7.00s
  - Bits: 4,774 assertions passed  84.00s
  - Ini files: 7,004 assertions passed  69322.00s
  - UTF8: 82,106 assertions passed  1083725.00s
  - Iso 8601 date and time: 36,015 assertions passed  4961.00s
  - Url decoding: 1,100 assertions passed  260.00s
  - Mime types: 23 assertions passed  25.00s
!  - TSynTable: 1 / 50 FAILED  2918.00s
  - TSynCache: 404 assertions passed  112.00s
  - TSynFilter: 804 assertions passed  2988.00s
  - TSynValidate: 677 assertions passed  1109.00s
  - TSynLogFile: 36 assertions passed  1419.00s
  Total failed: 1 / 4,762,182  - Low level common FAILED  2526159.00s

 1.2. Low level types: 
  - Url encoding: 200 assertions passed  1521.00s
  - Encode decode JSON: 250,616 assertions passed  167348.00s
  - Variants: 5 assertions passed  9.00s
  Total failed: 0 / 250,821  - Low level types PASSED  168909.00s

 1.3. Cryptographic routines: 
  - Adler32: 1 assertion passed  41.00s
  - MD5: 1 assertion passed  24.00s
  - SHA1: 5 assertions passed  28.00s
  - SHA256: 5 assertions passed  48.00s
  - AES256: 12,177 assertions passed  971956.00s
  - RC4: 1 assertion passed  15.00s
  - Base64: 11,994 assertions passed  279630.00s
  - CompressShaAes: 1,683 assertions passed  4454.00s
  Total failed: 0 / 25,867  - Cryptographic routines PASSED  1256308.00s

 1.4. Compression: 
  - In memory compression: 12 assertions passed  684980.00s
  - GZIP format: 19 assertions passed  1241166.00s
  - SynLZO: 3,006 assertions passed  135743.00s
  - SynLZ: 21,010 assertions passed  718603.00s
  Total failed: 0 / 24,047  - Compression PASSED  2780521.00s


2. mORMot

 2.1. File based: 
  - Database direct access: 10,138 assertions passed  434300.00s
  - Virtual table direct access: 12 assertions passed  1999.00s
  - TSQLTableJSON: 106,068 assertions passed  134448.00s
  - TSQLRestClientDB: 310,072 assertions passed  1019676.00s
  Total failed: 0 / 426,290  - File based PASSED  1590468.00s

 2.2. File based memory map: 

! Exception EAccessViolation raised with messsage:
!  Access violation


Synopse framework used: 1.18.433
SQlite3 engine used: 3.8.7
Generated with: Free Pascal 2.7.1 compiler
Tests performed at 10/29/2014 05:55:48 PM

Total assertions failed for all test suits:  1 / 5,489,207
! Some tests FAILED: please correct the code.

Done - Press ENTER to Exit

OS: CentOS 5.11 x86
CodeTyphon: 5.0 x86
mORMot: 228976b436782ae914dc098ab6cec5588aef0cec
modification to mORMot: "synopseCommit.inc" to "SynopseCommit.inc"

Last edited by ComingNine (2014-10-29 09:13:00)

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Strange regression.
Few things you could try:

0)
At my system, the same happens when running the tests as non-root.
Could you try running the tests as root ?

1)
Inside SynCommons.
{$ifdef PUREPASCAL} <----------- change to {$ifdef FPC_OR_PUREPASCAL}
function Div100(Y: PtrUInt): TWordRec;

Inside Synopse.inc (last lines):

2) Did you use dynamic or static linking ?
{$define NOSQLITE3STATIC}

3) Did you enable encryption ?
{$define NOSQLITE3ENCRYPT}

Last edited by AOG (2014-10-29 20:36:11)

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Ok.

Found a working (at my system) workaround !

Disable:

Demo.MemoryMappedMB = 256;

(at two places) in the SynSelfTests.
Please report back if this solves your problems (when running as non-root).

I will look into this more closely.

Last edited by AOG (2014-10-30 08:20:33)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: mORMot.pas

Yes, memory mapped option in Sqlite3 is perhaps not mandatory - we did not find any real speed benefit, at least under Windows.
IMHO disabling/not using it under Linux may be a good idea.

But WAL may have some benefit, depending on your storage HW and the running file system driver.

About div100() see http://synopse.info/fossil/info/105fc5eb65

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

@Ab and @ComingNine,

Some progress to report !

Found out that this regression could be caused by a wrong use of packed record inside SynSQLite3Static.

TSQLFile = packed record ---------> change to ---------> TSQLFile = record
(remove the packed statement)

At my system, everything works as expected now, including WAL and Demo.MemoryMappedMB !

Testing reports welcome !!

EgonHugeist

Member

From: Germany

Registered: 2013-02-15

Posts: 190

Re: mORMot.pas

@AOG

there is propably an aligment issue with ARM-processors.
FPC has a buildin define to control that behavior:

  MyRecord = {$ifndef FPC_REQUIRES_PROPER_ALIGNMENT}packed{$endif} record...

Note it might be possible you need dummy pointer alignments if pointers are used in the record

Don't know anything about delphi issues in this case..

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: mORMot.pas

@AOG
Yes, I wondered why you did still use "packed" keyword for FPC.
This structure was designed to match the low-level SQLite3 C struct layout, which is not packed.
In our original version of TSQLFile, we did use bulk1/bulk2 to do some fake record packing.
I understand your confusion.

So with FPC, {$PACKRECORDS C} above + TSQLFile = record  (without the packed) sounds fine to me.

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Agreed ! My mistake ... perhaps caused by my ARM-obsession (@EgonHugeist) ... ;-)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: mORMot.pas

I suspect http://synopse.info/fossil/info/b086548a8 would work as expected.

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

2) Did you use dynamic or static linking ?
{$define NOSQLITE3STATIC}

I tested static linking. When I test dynamic linking, everything works.

On Linux, access to port 888 is limited to root processes.
(and I did perform the tests as root ... so I forgot about it)

I have made another executable, that uses port 8888, that is also available for non-root users.

Could you help to comment how you changed the port ?

I suspect http://synopse.info/fossil/info/b086548a8 would work as expected.

I got compilation error as follows:

SynCommons.pas(38215,11) Error: Wrong number of parameters specified for call to "Fpmmap"
Error: Found declaration: Fpmmap(Pointer;LongWord;LongInt;LongInt;LongInt;Int64):^untyped;

ps: Can you change the file name case of "synopseCommit.inc" to be consistent with "SynopseCommit.inc" which is referred to in Synopose.inc ?

ps: The forum seems to become NOT responsive and return 500 when I tried to post the content shown in this git:
https://gist.github.com/anonymous/8204f29244f7e7be0e0e

Last edited by ComingNine (2014-10-30 16:20:42)

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Glad to hear that dynamic works as expected !

Would you mind trying:

0): Running at least version 1.18.438 (see : http://synopse.info/fossil/info/8fd8f6f … 828c0288c6)
1): No succes: Disabling ---> Demo.MemoryMappedMB = 256; inside SynSelfTests (2-3 x)
2): No succes: Running as root.

Running on 8888 : replacing all 888 with 8888 inside SynSelfTests.

Last edited by AOG (2014-10-30 17:02:03)

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

Glad to hear that dynamic works as expected !

Would you mind trying:

0): Running at least version 1.18.438 (see : http://synopse.info/fossil/info/8fd8f6f … 828c0288c6)
1): No succes: Disabling ---> Demo.MemoryMappedMB = 256; inside SynSelfTests (2-3 x)
2): No succes: Running as root.

Running on 8888 : replacing all 888 with 8888 inside SynSelfTests.

As I reported, the latest mORMot gives compilation error :

SynCommons.pas(38215,11) Error: Wrong number of parameters specified for call to "Fpmmap"
Error: Found declaration: Fpmmap(Pointer;LongWord;LongInt;LongInt;LongInt;Int64):^untyped;

OS: CentOS 5.11 x86
CodeTyphon: 5.0 x86
mORMot: 4e64baa9eac75911acf9fb8b86e1677be05e8cb9
modification to mORMot: "synopseCommit.inc" to "SynopseCommit.inc"

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

Small typo !!
Replace
fBuf := Fpmmap(nil,fBufSize,PROT_READ,MAP_SHARED,fFile,0,aCustomOffset);
with
fBuf := Fpmmap(nil,fBufSize,PROT_READ,MAP_SHARED,fFile,aCustomOffset);
(remove zero).

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

The forum seems to become NOT responsive and return 500 when I tried to post the content shown in this git:
https://gist.github.com/anonymous/8204f29244f7e7be0e0e

Can you help to solve this ? It happened several times when I tried to report the results of testing the kind suggestions given by "AOG" or "ab". Frustrating...

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

Now compilation under linux32 works great using static SQLite library !  Congratulations to your great work !

OS: CentOS 5.11 x86
CodeTyphon: 5.1 x86
mORMot: 4e64baa9eac75911acf9fb8b86e1677be05e8cb9
modification to mORMot:
(1) "synopseCommit.inc" to "SynopseCommit.inc"
(2) "fBuf := Fpmmap(nil,fBufSize,PROT_READ,MAP_SHARED,fFile,0,aCustomOffset);" to "fBuf := Fpmmap(nil,fBufSize,PROT_READ,MAP_SHARED,fFile,aCustomOffset);"
(3) replacing all 888 with 8888 inside SynSelfTests

I will test cross-compilation from win32 to linux32 using static SQLite library, and report back.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: mORMot.pas

@ComingNine
The forum is NOT meant to publish huge piece of code, or text.
Such posts would probably be deleted.
Using Gist or any PasteBin like is better.

For the needed modifications, see http://synopse.info/fossil/info/511b4775cf
Hope it will work for you.

Thanks for your feedback!

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

@ab:

Thank you for your efforts and comments !

Can you help to make TSynLog capable of capturing the stacktrace with FPC on Linux ?

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,659

Website

Re: mORMot.pas

I guess we may use http://wiki.freepascal.org/Logging_exceptions

There is a lot of material in this post.
We may be able to intercept exceptions using FPC RTL, and also capture the stacktrace...

Could you please create a feature request ticket, linking to this forum post?
Thanks!

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

@ab

Thank you for your efforts very much ! The ticket was created

ComingNine

Member

Registered: 2010-07-29

Posts: 294

Re: mORMot.pas

LinuxSynTestFPCLinuxi386.lpr:

program LinuxSynTestFPCLinuxi386;
{$I Synopse.inc} // define HASINLINE USETYPEINFO CPU32 CPU64 OWNNORMTOUPPER
{$APPTYPE CONSOLE}
uses {$ifdef Linux} cthreads, cwstring, {$endif} mORMotSelfTests;
begin SQLite3ConsoleTests; end.   

Link error during cross-compilation from win32 to linux32 with static SQLite library

 
LinuxSynTestFPCLinuxi386.lpr(22,1) Warning: "crtbegin.o" not found, this will probably cause a linking failure
LinuxSynTestFPCLinuxi386.lpr(22,1) Warning: "crtend.o" not found, this will probably cause a linking failure
C:\DEV\_Delphi_Lib_\LinuxSynTestFPCLinuxi386\LinuxSynTestFPCLinuxi386.lpr(22,1) Warning: (9034) "crtbegin.o" not found, this will probably cause a linking failure
C:\DEV\_Delphi_Lib_\LinuxSynTestFPCLinuxi386\LinuxSynTestFPCLinuxi386.lpr(22,1) Warning: (9034) "crtend.o" not found, this will probably cause a linking failure
c:\codetyphon\fpc\fpc32\bin\i386-win32\i386-linux-ld.exe: warning: link.res contains output sections; did you forget -T?
c:\codetyphon\fpc\fpc32\bin\i386-win32\i386-linux-ld.exe: cannot find /lib/libpthread.so.0
c:\codetyphon\fpc\fpc32\bin\i386-win32\i386-linux-ld.exe: cannot find /usr/lib/libpthread_nonshared.a
LinuxSynTestFPCLinuxi386.lpr(22,1) Error: Error while linking

OS: Windows 7 SP1 X64
CodeTyphon: 5.1
Typhon: x86
FPC cross-build element: i386-linux
mORMot: 014f4dd542e609ba2e27b81eccc6890f66ff1a68
modification to mORMot: nothing

File system:
A VMWare CentOS 5.11 x86 mounted using Sftp Net Drive

linuxlibrary & mORMot

linuxlibrary & test project

IDE settings:

Last edited by ComingNine (2014-10-31 17:15:31)

AOG

Member

Registered: 2014-02-24

Posts: 490

Re: mORMot.pas

The lib directory on a Linux system contains a lot of links.
AFAIR Sftp Net Drive shows these links as files, but they are not.

First step:
Copy the lib-directory to your Windows system.
Check that all links are converted into real files !

Second step
crtbegin.o and crtend.so and libgcc.a can be found inside /lib/gcc
Copy these also.

And be sure that you link with 32bit library files ... as our mORMot is still 32bits.

Codetyphon offers complete i386 Linux libraries for download ... at my system this works 100%.

Success !

Last edited by AOG (2014-11-01 08:47:59)