DSqlite - an interesting project

edwinsn · 2019-08-30 16:34:40

Distributed SQLite but still embedded, by the company that made Ubuntu.

No sure how difficult it'll be to integrate it with mORMot

project site: https://github.com/canonical/dqlite

Last edited by edwinsn (2019-08-30 16:35:04)

pvn0 · 2019-08-30 18:22:52

mORMot already has all the features of Dqlite and more

edwinsn · 2019-08-31 04:32:51

@pvn0, actually it's quite different

pvn0 · 2019-08-31 06:51:10

Everything dqlite does you can already do in mORMot with little effort, which feature of dqlite are you missing from mORMot?

edwinsn · 2019-09-01 06:56:12

The "d" in dsqlite means distributed, from what I read, it allows your data to be automatically distributed into multiple servers, and if any of the servers is down, the data service will still keep working.

As a matter of fact, I don't think DSqlite can be easily integrated into mORMot.

Since you asked, in a perfect world, I want to program against mORMot's API, and my data to be transparently distributed in multiple nodes and take advantages of the multiple computing power.

ab · 2019-09-05 10:04:44

After more investigation, I like very much https://github.com/canonical/dqlite
- it allows very safe and efficient RAFT (master/slave) replication of a SQLite3 engine
- perhaps worth investigating e.g. for our solutions - it is not meant for huge data, but efficient for some metadata server
- only drawback I see is that it doesn't run on Windows, so we couln't use it on premise hosted on that OS
- I guess I will eventually support it in mORMot, with a fallback emulation with no replication on Windows

edwinsn · 2019-09-05 10:29:27

Hello ab,

I'm glad you find this project useful and intend to extend mORMot with it - that will make mORMot a **distributed** system!

Re not supporting Windows, maybe you wait a little time, also for its maturity.

It seems to use a patched sqlite engine.

ab · 2019-09-05 11:55:42

I guess we could use the patched sqlite engine for our static linking code.
From what I have seen in the patch, distribution is disabled by default, so it should work as a regular local SQlite3 engine.

The main benefit of dsqlite is its C-Raft integration: properly implementing Raft is challenging, and their asychronous implementation seems just perfect.
It is safer (and much easier) to rely on an existing solution like theirs than trying to reimplement Raft in pascal.

My only concern is about security: it seems that the wired protocol is not secured at all (no authentication, no encryption).
So it could be used only in trusted local networks, whereas I would like to use it other the Internet, e.g. from two providers.

mpv · 2019-09-05 17:57:00

Since it's unix only we can use a ssh tunnels between hosts. It solves all security problems.

pvn0 · 2019-09-10 11:27:26

I guess I should eat my words then, I still don't think it's very useful unless you're targeting highly volatile devices.

mORMot Open Source

#1 2019-08-30 16:34:40

DSqlite - an interesting project

#2 2019-08-30 18:22:52

Re: DSqlite - an interesting project

#3 2019-08-31 04:32:51

Re: DSqlite - an interesting project

#4 2019-08-31 06:51:10

Re: DSqlite - an interesting project

#5 2019-09-01 06:56:12

Re: DSqlite - an interesting project

#6 2019-09-05 10:04:44

Re: DSqlite - an interesting project

#7 2019-09-05 10:29:27

Re: DSqlite - an interesting project

#8 2019-09-05 11:55:42

Re: DSqlite - an interesting project

#9 2019-09-05 17:57:00

Re: DSqlite - an interesting project

#10 2019-09-10 11:27:26

Re: DSqlite - an interesting project

Board footer