#1 2023-02-07 10:14:36

edwinsn
Member
Registered: 2010-07-02
Posts: 1,218

Real high-concurrency & replication is coming to SQLite!

Good news from the SQLite tean:

Real high-concurrency & replication is coming to SQLite!

@ab,

Since you already marked mORMot 1 to be in maintenance mode, will you keep updating the SQLite engine for mORMot 1 in the future?


Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.

Offline

#2 2023-02-07 11:02:08

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,609
Website

Re: Real high-concurrency & replication is coming to SQLite!

Yes, mORMot 1 will continue to be upgraded with the latest SQLite3 statics.
There is a single build script for both mORMot 1 and mORMot 2.

This hctree prototype is very interesting for sure.
The fact that it is made by SQLite3 authors make it a good choice for the long term support - if it is not only a prototype but a stable enough product.
But note that this hctree database is a new DB format. It can open SQlite3 databases, but it is a new file layout.

Offline

#3 2023-02-07 11:13:31

edwinsn
Member
Registered: 2010-07-02
Posts: 1,218

Re: Real high-concurrency & replication is coming to SQLite!

Great!

ab wrote:

But note that this hctree database is a new DB format. It can open SQlite3 databases, but it is a new file layout.

I don't think it's an issue as of now...


Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.

Offline

#4 2023-02-07 18:38:57

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,570
Website

Re: Real high-concurrency & replication is coming to SQLite!

For sure, it's huge improvement. Currently I can't use sqlite3 on real productions where long transactions are commonplace (because of sqlite3_busy error), but hctree should solve this problem, and (at least where DB size < 500Gb)  sqlite can be usable.

Offline

#5 2023-02-08 03:17:04

edwinsn
Member
Registered: 2010-07-02
Posts: 1,218

Re: Real high-concurrency & replication is coming to SQLite!

mpv wrote:

...Currently I can't use sqlite3 on real productions where long transactions are commonplace (because of sqlite3_busy error)...

Would you share with us some pseudo code about how do you deal with the sqlite3_busy error?
I guess you use a retrying mechanism?


Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.

Offline

#6 2023-02-08 15:44:44

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,570
Website

Re: Real high-concurrency & replication is coming to SQLite!

mORMot uses global lock for SQLITE3 updates, so problem not happens where.

I use my own TubSQLite3ThreadSafeConnection = class(TSQLDBConnectionThreadSafe) without global lock (it actually exists inside sqlite) and sets

TSQLDataBase.WalMode := true;
TSQLDataBase.BusyTimeout := 10000; // 10 seconds

In case of sqlite3_busy  - this error is passed to user and on UI message "try again later" is displayed.

Offline

#7 2023-02-09 02:10:08

edwinsn
Member
Registered: 2010-07-02
Posts: 1,218

Re: Real high-concurrency & replication is coming to SQLite!

Thanks for sharing, when not using the ORM but the database layer directly, I also increase the value of BusyTimeout


Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.

Offline

Board footer

Powered by FluxBB