Couldn't see changes by other process, because of caching?

ab · 2015-04-01 10:54:52

Please refresh the page.
It should point to "5.10.3. Replication use cases".

edwinsn · 2015-04-01 17:29:37

Congratulations on the new replication feature!

Two questions if you don't mind:
1 - For the "Corporate Servers Master/Slave Replication With Private Local Data" mode, are the two arrows in the graph have wrong directions? Shouldn't it look like this (notice the two red arrows I added):

2 - I think one drawback of the master/slave replication mode is that the slave should be "read-only" to avoid duplicated version numbers. I'm not sure if I understand it correctly, but can we make this feature becoming two-way syncing by using different prefix for the server-wide TRecordVersion properties? For example, for rest server A in office A we use 1 as the version number prefix, while rest server B in office B has prefix of 2, and vice versa.

AOG · 2015-04-01 19:03:29

Got it !
Now for the really stupid questions ;-) !!

I use

aServer.RecordVersionSynchronize(TSQLRecordPeopleVersioned,aClient);

to get data from the master to my local FPC Android mORMot client.
As I understand correctly, this will update all records in the local PeopleVersioned-table.

(All) aClient(s) need to have full access to the master-server.
However, the master is equipped with a lot of methods to limit direct access.

Questions:
1) How do you see Synchronize ? Using a message loop / timer / thread ? Or during method-access of the master-server ?
2) How do you see Access ? Most tables on master are closed for direct access.
3) Would I need to make a special replication-user with full access ? This could be dangerous.

Sorry for stupidity, if applicable. Thanks for help in advance !

ab · 2015-04-01 21:05:01

edwinsn wrote:

2 - I think one drawback of the master/slave replication mode is that the slave should be "read-only" to avoid duplicated version numbers. I'm not sure if I understand it correctly, but can we make this feature becoming two-way syncing by using different prefix for the server-wide TRecordVersion properties? For example, for rest server A in office A we use 1 as the version number prefix, while rest server B in office B has prefix of 2, and vice versa.

Yes, this is prepared for a future implementation.
The TSQLRestServer.InternalRecordVersionComputeNext function just need to be overriden with a timestamp + node suffix to have a monotonic multi-node version number.
There is just a limit of 58 bits, due to how TSQLRecordTableDelete.ID works.

To synchronize, the easiest is to use a timer.
It will work from a mobile device.

To synchronize in real time, I have just introduced TSQLRestServer.RecordVersionSynchronizeCallback() method, able to register a callback interface which would be called each time a write operation is performed on a given TSQLRecord with a TRecordVersion field.
With WebSockets support, it is a pretty good way of having full synchronization of a database.
See http://synopse.info/fossil/info/e78b7e3e24

AOG · 2015-04-02 07:03:53

Thanks for these additions !

Could you shed some light on my very closed setup: how do I access my master-server, that only allows methods, and no direct CRUD on tables ?
Just use a privileged replication-user ?

Minor problem with latest mORMot. This

(PPointer(instance)^=TInterfacedObjectFakeServer)

does not compile with FPC.

ab · 2015-04-02 08:07:38

FPC compilation should be fixed by http://synopse.info/fossil/info/4462a6f626 (together with other changes).

The slave need to have read access to the master tables involved (i.e. the TSQLRecord table containing the TRecordVersion field, and the TSQLRecordTableDeleted table).
So specific user rights may be necessary.

AOG · 2015-04-02 08:37:20

Thanks ! Your fix compiles !! Now I can go testing on FPC and Android.

As a sidenote, and in addition to the need for specific user rights:

Would it be an idea to add the replication-user to the mORMot framework.
Because this replication will only work with full read access of the Master.
So it can be added by default to the framework.
Replication will not work without full read acccess to all tables.

edwinsn · 2015-04-02 09:01:36

ab wrote:

Yes, this is prepared for a future implementation.
The TSQLRestServer.InternalRecordVersionComputeNext function just need to be overriden with a timestamp + node suffix to have a monotonic multi-node version number.
There is just a limit of 58 bits, due to how TSQLRecordTableDelete.ID works.

Sounds like a good idea! Does it mean once such a two-way syncing feature is implemented, I can have several databases synced with the very same set of data? I can help to test when it's added, what's your plan? Thanks.

ab · 2015-04-02 13:15:49

edwinsn wrote:

Does it mean once such a two-way syncing feature is implemented, I can have several databases synced with the very same set of data? I can help to test when it's added, what's your plan? Thanks.

Yes, this is the point.
But we won't use it directly this feature here, so testing will be welcome!

I suspect we may be able to add this TRecordVersion per-node generation in a very short term.

edwinsn · 2015-04-02 14:02:43

ab wrote:

edwinsn wrote:
Does it mean once such a two-way syncing feature is implemented, I can have several databases synced with the very same set of data? I can help to test when it's added, what's your plan? Thanks.
Yes, this is the point.
But we won't use it directly this feature here, so testing will be welcome!
I suspect we may be able to add this TRecordVersion per-node generation in a very short term.

Great! I'll monitor this thread to get updates. Thanks.

edwinsn · 2015-04-03 02:33:04

Hi Arnaud, I assume even with the above mentioned two-way syncing implemented, we still have to take care of the IDs of the records not to be duplicated across offices, right?

Jameson · 2015-05-07 13:50:19

Good afternoon! Am new to mORMot, I developed an application to perform automatic backup in real time, however, would like to test the performance of mORMot with this relationship, someone could provide a simple example of database replication with this new available routine? Best Regards

mORMot Open Source

#51 2015-04-01 10:54:52

Re: Couldn't see changes by other process, because of caching?

#52 2015-04-01 17:29:37

Re: Couldn't see changes by other process, because of caching?

#53 2015-04-01 19:03:29

Re: Couldn't see changes by other process, because of caching?

#54 2015-04-01 21:05:01

Re: Couldn't see changes by other process, because of caching?

#55 2015-04-02 07:03:53

Re: Couldn't see changes by other process, because of caching?

#56 2015-04-02 08:07:38

Re: Couldn't see changes by other process, because of caching?

#57 2015-04-02 08:37:20

Re: Couldn't see changes by other process, because of caching?

#58 2015-04-02 09:01:36

Re: Couldn't see changes by other process, because of caching?

#59 2015-04-02 13:15:49

Re: Couldn't see changes by other process, because of caching?

#60 2015-04-02 14:02:43

Re: Couldn't see changes by other process, because of caching?

#61 2015-04-03 02:33:04

Re: Couldn't see changes by other process, because of caching?

#62 2015-05-07 13:50:19

Re: Couldn't see changes by other process, because of caching?

Board footer