mORMot Open Source

edwinsn

Member

Registered: 2010-07-02

Posts: 1,218

Real high-concurrency & replication is coming to SQLite!

Good news from the SQLite tean:

Real high-concurrency & replication is coming to SQLite!

@ab,

Since you already marked mORMot 1 to be in maintenance mode, will you keep updating the SQLite engine for mORMot 1 in the future?

---

Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Real high-concurrency & replication is coming to SQLite!

Yes, mORMot 1 will continue to be upgraded with the latest SQLite3 statics.
There is a single build script for both mORMot 1 and mORMot 2.

This hctree prototype is very interesting for sure.
The fact that it is made by SQLite3 authors make it a good choice for the long term support - if it is not only a prototype but a stable enough product.
But note that this hctree database is a new DB format. It can open SQlite3 databases, but it is a new file layout.

edwinsn

Member

Registered: 2010-07-02

Posts: 1,218

Re: Real high-concurrency & replication is coming to SQLite!

Great!

But note that this hctree database is a new DB format. It can open SQlite3 databases, but it is a new file layout.

I don't think it's an issue as of now...

---

Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.

mpv

Member

From: Ukraine

Registered: 2012-03-24

Posts: 1,571

Website

Re: Real high-concurrency & replication is coming to SQLite!

For sure, it's huge improvement. Currently I can't use sqlite3 on real productions where long transactions are commonplace (because of sqlite3_busy error), but hctree should solve this problem, and (at least where DB size < 500Gb)  sqlite can be usable.

edwinsn

Member

Registered: 2010-07-02

Posts: 1,218

Re: Real high-concurrency & replication is coming to SQLite!

...Currently I can't use sqlite3 on real productions where long transactions are commonplace (because of sqlite3_busy error)...

Would you share with us some pseudo code about how do you deal with the sqlite3_busy error?
I guess you use a retrying mechanism?

---

Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.

mpv

Member

From: Ukraine

Registered: 2012-03-24

Posts: 1,571

Website

Re: Real high-concurrency & replication is coming to SQLite!

mORMot uses global lock for SQLITE3 updates, so problem not happens where.

I use my own TubSQLite3ThreadSafeConnection = class(TSQLDBConnectionThreadSafe) without global lock (it actually exists inside sqlite) and sets

TSQLDataBase.WalMode := true;
TSQLDataBase.BusyTimeout := 10000; // 10 seconds

In case of sqlite3_busy  - this error is passed to user and on UI message "try again later" is displayed.

edwinsn

Member

Registered: 2010-07-02

Posts: 1,218

Re: Real high-concurrency & replication is coming to SQLite!

Thanks for sharing, when not using the ORM but the database layer directly, I also increase the value of BusyTimeout

---

Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.