#1 2010-06-22 09:48:24

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 11,474
Website

Synopse Database announcement

Synopse is releasing a new open source database engine, founded on some successfull projects like Ictus Win or ROC.

It's a whole new database engine, not SQL-based, but purely object/XML related.

The Synopse company is able to develop business rules and tuned solutions using the Synopse database engine. This software development can be made private by an appropriate contract, but will rely on the proven Synopse database engine.

The fully featured Synopse database engine itself is published under a dual licence model, similar to the one used by Sun™ for MySQL™:
- a commercial licence for OEMs, ISVs, VARs and other distributors of commercial applications;
- a true GPL licence version 3 or later for open source projects (FOSS).

Offline

#2 2010-08-18 13:13:15

yogiyang
Member
Registered: 2010-08-18
Posts: 21

Re: Synopse Database announcement

Does this support encryption of data at engine level?

Can you give some speed statistics of inserting data, updating data, retrieving and populating UI controls, etc. ?

Offline

#3 2010-08-18 14:25:24

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 11,474
Website

Re: Synopse Database announcement

Warning: this thread is about the Synopse database engine, which is NOT the SQLite3 framework.
I answered here: http://synopse.info/forum/viewtopic.php?pid=397

Offline

#4 2011-03-16 16:48:23

edwinsn
Member
Registered: 2010-07-02
Posts: 1,021

Re: Synopse Database announcement

purely object/XML related? Great! Is it for Delphi only?

Can't read what Ictus Win and ROC are since it's in French smile


Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.

Offline

#5 2011-03-17 08:47:12

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 11,474
Website

Re: Synopse Database announcement

You can use google translate to get an idea of what they are.
It's more or less a database system like ROC with the full text and cross-referencing capabilities of IctusWin.

I'd have to go back to this Synopse project sooner or later.

Offline

#6 2012-09-02 06:15:55

sh17
Member
Registered: 2011-08-09
Posts: 25

Re: Synopse Database announcement

Any new infos about this project?

Where i can get the source code?

Offline

#7 2012-09-02 10:06:47

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 11,474
Website

Re: Synopse Database announcement

This project has not been released, finally.
Its core is used in some military projects, so I would rather have the green light from the Army to release it.

I'd probably merge this engine within mORMot.
There is a corresponding item in the mORMot RoadMap:

Add customer-defined fields in mORMot ORM, via a new field type, stored as RawByteString (BLOB at the DB level), and using TSynTable / TSynTableData / TSynTableFieldProperties existing classes of SynCommons.pas; the UI units shall be able to handle edition of such fields just like regular TSQLRecord published properties, and a new unit shall provide customization of the record layout; this will allow end-users to customize the database layout according to their needs, whereas some default properties will be handled by the shared business logic (at least the ID field);

We would need also some other kind of field content, like MGRS.

Offline

#8 2012-09-03 07:25:31

sh17
Member
Registered: 2011-08-09
Posts: 25

Re: Synopse Database announcement

i search a database engine without the sql overhead. fast and save agains loose of data.
is it possible to use some parts from sqllite for indexmanagement? or adapt sqllite generally?
I hope I can express myself well enough.

Sven

Last edited by sh17 (2012-09-03 07:25:52)

Offline

#9 2012-09-03 08:04:23

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 11,474
Website

Re: Synopse Database announcement

I found out that if SQL statements are prepared, there is no overhead to use SQL.
In fact, prepared SQL statements will not require to process the SQL command, and the SQLite3 virtual machine will call directly the database back-end (i.e. B-Tree, Pager, indexes) as fast as possible.
arch2.gif
See http://www.sqlite.org/arch.html for details.
Our mORMot units cache and re-use SQL statements, so there is no overhead of using SQL in most cases, when you use parameters (?) in your queries.

In order to be ACID, your process will be slower.
SQLite3 has two modes:
- Synchrounous = FULL means that is 100% ACID, even in case of power failure - but it is very slow since it was for the data to be written on disk - around 10 inserts / second!
- Synchrounous = OFF is 100% ACID, unless you cut the power - then you have about 400 inserts per second.
If you nest INSERTs into a transaction, speed is very high.

If you want something faster, you'll loose ACID at every insert.
But even the TSQLRestServerStaticInMemory can be ACID at writing: all data remain in memory, and is flushed to disk at once, on request.
If a power loss or application crash occurs before this write phase, you may loose some data, but you won't loose the Atomicity of the table content.
TSQLRestServerStaticInMemory will give the best speed possible, both for searching and insertion, if your requests are per ID, or about one "stored false" defined field (i.e. declared as UNIQUE).

See http://blog.synopse.info/post/2012/07/26/ACID-and-speed for details and benchmarks, and http://blog.synopse.info/post/2012/08/0 … properties about "stored false" abilities.

Offline

#10 2014-05-04 17:37:44

ComingNine
Member
Registered: 2010-07-29
Posts: 288

Re: Synopse Database announcement

Page 156 in the SAD doc v1.18 says "A fast in-memory engine is included, which outperforms any SQL-based solution in terms of speed - but to the price of a non ACID behavior."

However, in the post above, first it is mentioned "If you want something faster, you'll loose ACID at every insert.", then it is mentioned "But even the TSQLRestServerStaticInMemory can be ACID at writing."

I am wondering whether or not TSQLRestServerStaticInMemory is ACID ? If not, in which area ? Could you help to elaborate ? :-)

Last edited by ComingNine (2014-05-04 17:40:01)

Offline

#11 2014-05-05 07:10:53

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 11,474
Website

Re: Synopse Database announcement

AFAIR everything is explained in the "In-Memory virtual tables" paragraph of the SAD 1.18 pdf.

I just added a new paragraph, to make it even more straightforward:

In-Memory and ACID wrote:

For data stored in memory, the TSQLRestServerStaticInMemory table is ACID.
It means that concurrent access will be consistent and work safely, as expected.

On disk, this kind of table is ACID only when its content is written to the file.
I mean, the whole file which will be written in an ACID way. The file will always be consistent.

The exact process of these in-memory tables is that each time you write some new data to a TSQLRestServerStaticInMemory table:
- It will be ACID in memory (i.e. work safely in concurrent mode);
- Individual writes (INSERT/UPDATE/DELETE) won't automatically be written to file;
- COMMIT will by default write the whole table to file (either as JSON or compressed binary);
- COMMIT won't write the data to file if the CommitShouldNotUpdateFile property is set to TRUE;
- ROLLBACK process won't do anything, so won't be ACID - but since your code may later use a real RDBMS, it is a good habit to always write the command, like in the sample code above, as except aClient.RollBack.

When you write the data to file, the whole file is rewritten: it seems not feasible to write the data to disk at every write - in this case, SQLite3 in exclusive mode will be faster, since it will write only the new data, not the whole table content.

This may sound like a limitation, but on our eyes, it could be seen more like a feature. For a particular table, we do not need nor want to have a whole RDBMS/SQL engine, just direct and fast access to a TObjectList. The feature is to integrate it with our REST engine, and still be able to store your data in a regular database later (SQLite3 or external), if it appears that TSQLRestServerStaticInMemory storage is too limited for your process.

Offline

#12 2014-05-05 07:51:04

ComingNine
Member
Registered: 2010-07-29
Posts: 288

Re: Synopse Database announcement

Thank you very much for your elaborations !!

Offline

Board footer

Powered by FluxBB