You are not logged in.
- Topics: Active | Unanswered
#401 Re: mORMot 1 » the lack of scale solution » 2018-10-31 06:07:04
I came across the Google Cloud Tasks introduction blog post today, check it for ideas: https://cloud.google.com/blog/products/ … n-runtimes
with mORMot we can definitely implement a similar queue-based architecture!
#402 Re: mORMot 1 » the lack of scale solution » 2018-10-27 03:54:49
1. table scale to control single table row count (the valid Couppon info is just the recent few days, we do not need to lookup the whole table);
2. Rest Services load balancing.
3. rest services really need version control .
4. we only use delphi to build or whole project, only 3 coder , and we love delphi/pascal , we really dont wanna switch to Java or Go solution. maybe we used mORMot the wrong way . but any advice will be appreciated.
1. Did you create index for CreatedAt and ModifiedAt? I believe the database system can take care of the indexing and allow you query the records by the datetime efficiently.
2. I believe ab is not against the load balancing function, the current company he works for has several hundreds of server running mORMot-based service. Clue 1: http://blog.synopse.info/post/2016/07/3 … d-Big-Data
3. I don't know how other languages do the API versioning, but with mORMot you can version the API with interfaces or function names, eg. IMyInterfaceV1, IMyInterfaceV2...
4. You don't have to, maybe other languages offer more ready solutions for load balancing, but I believe Delphi/Pascal with mORMot is flexible enough to allow you to program solutions for the issues. Please take a read of this post (https://synopse.info/forum/viewtopic.php?id=4703), ab's company will have over one thousand servers running, it's based on a so-called **queue-based** architecture, so does the the OP's architecture.
And let's see other's suggestions.
#403 Re: mORMot 1 » I think there is a logic error in TSynLog » 2018-10-23 10:37:44
ok, thanks.
#404 Re: mORMot 1 » named parameters to interface functions » 2018-10-22 17:16:52
@erick, how can you write a huge app with ElevateWebBuilder without code-completion?
#405 Re: mORMot 1 » SynLog - To allow ignoring exceptions with code » 2018-10-22 05:02:48
@ab,
Is it possible to include the ID of the thread who's causing an exception with TSynLogExceptionContext? The thread ID can be helpful in the OnBeforeException event handler.
#406 mORMot 1 » I think there is a logic error in TSynLog » 2018-10-21 09:32:03
- edwinsn
- Replies: 2
Hello ab,
Even if CustomFileName is set, TSynLog.ComputeFileName still append the current datetime to the filename.
I think this is logically not right. Instead, it should use CustomFileName without appending the timestamp, what do you think?
#407 Re: mORMot 1 » Working with an offline client and central server » 2018-10-21 06:05:11
For mORMot's built-in support for your case, check TSQLModel.SetIDGenerator and SynCommons.TSynUniqueIdentifierGenerator.
or check https://github.com/suiyunonghen/DelphiSnowflake.
or check my little function if your data volume is low: https://synopse.info/forum/viewtopic.php?id=2522
#408 Re: mORMot 1 » CR is a wrong end-of-line character for Unix (at last for logs) » 2018-10-20 04:50:58
Everything work much faster. I'm surprised how much better Lazarus debugger works under Linux
Would love to hear some first-hand details 
#409 Re: mORMot 1 » Interface-based services: Receiving a list of objects » 2018-10-17 14:55:09
Great! It seems that polling has its own advantages - easy to implement and simple connection life-time management.
#410 Re: mORMot 1 » Interface-based services: Receiving a list of objects » 2018-10-17 14:05:42
Great to hear that! mORMot is so powerful
#411 Re: mORMot 1 » Interface-based services: Receiving a list of objects » 2018-10-17 10:58:38
@Beertjie,
Thanks for the details, I haven't used RabbitMQ before, however, I have the following custom-made, queue-based architecture in mind, utilizing TSynQueue and web-sockets provided by mORMot, how does it compare to your solution?
On the server side:
- expose a mORMot-based API function for the clients (also based on mORMot) to submit requests, and all the submitted requests won't be process immediately but instead, they will be put into a queue (based on TSynQueue), so the clients won't be blocked.
- there will be one (or multiple) worker thread(s) to process the requests in the queue one by one, when done, the corresponding client will be notified using mORMot's web-sockets-based interface.
I'm sure engines like RabbitMQ must have their advantages, for sure, but I'm still want to know how my idea compares, it's upside is that it's written in pure Delphi/Pascal
#412 Re: mORMot 1 » SynLog - To allow ignoring exceptions with code » 2018-10-17 08:11:04
/// callback signature used by TSynLogFamilly.OnBeforeException // - should return true to log the exception, or false to ignore it TSynLogOnBeforeException = function(aExceptionContext: TSynLogExceptionContext): boolean of object;ab, please correct description: should return false to log the exception, or true to ignore it.
What's why I have originally named the event property as "HandleExceptionBeforeLogging", the keyword is "handle", because if it handle it, it should return True, and then TSynLog should ignore it
However I'm certainly OK with ab's new naming, but you are right, the comment you mentioned is wrong.
#413 Re: mORMot 1 » SynLog - To allow ignoring exceptions with code » 2018-10-17 08:04:40
Thank you ab for merging it.
#414 Re: mORMot 1 » SynLog - To allow ignoring exceptions with code » 2018-10-16 16:21:45
@ab,
Did you forget the pull request? I have made the changes as you requested
#415 Re: mORMot 1 » Interface-based services: Receiving a list of objects » 2018-10-16 15:44:15
@ab, thanks, I discovered TSynQueue, but I wish I noticing it earlier, otherwise I won't have to use an alternative
And the ability to persist the queued items is really really an added value !
#416 Re: mORMot 1 » Interface-based services: Receiving a list of objects » 2018-10-16 12:21:56
...but my app has since transformed from an HTTP-based service to a queue-based one...
@Beertjie, would love to know some details about the queue-based architecture, if you can tell!
I has just recently discovered the 'queue' concept is so powerful and useful in a multi-thread environment, so I'm very curious about it
#417 Re: mORMot 1 » How to batch update with conditions? » 2018-10-16 12:19:01
BTW, I use mORMot's TModTime field type for the so-called 'modification date', and it's a perfect fit
#418 Re: mORMot 1 » How to batch update with conditions? » 2018-10-16 03:33:56
The feature @edwinsn request is known as "Optimistic locking", and as @ab note it usually should be implemented in the business logic layer.
I have it in my apps (every row store a modification date). Before run update on server side I compare modification date passed from client side with date stored in DB (yes, one more select, but it is quick) and in case it's not equal return error to client "Record is modified by another user".
I'm doing the similar thing - the logic is the same, except that I do it on the client-side before submitting the batch-update, so it's not wrapped by a DB transaction.
#419 Re: mORMot 1 » How to batch update with conditions? » 2018-10-15 12:35:00
OK, maybe I have a misunderstanding about SQLite transactions - can the following situation (step 3) happen on the server side?
1 - start transaction.
2 - read a record from db.
3 - the record read in step 2 is modified by another client.
4 - update the record.
5 - commit the transaction.
#420 Re: mORMot 1 » SynLog - To allow ignoring exceptions with code » 2018-10-15 12:28:30
@ab,
I added such ability, if no problem please merge pull request I made just now: https://github.com/synopse/mORMot/pull/142
#421 Re: mORMot 1 » How to batch update with conditions? » 2018-10-15 09:02:10
So first retrieve the IDs then apply the Batch.
What if the record has been changed in the database by **other clients**, in between 'first retrieve the ID' and 'apply the Batch'?
#422 mORMot 1 » How to batch update with conditions? » 2018-10-15 07:00:43
- edwinsn
- Replies: 10
The concept's illustrated with this example:
https://gist.github.com/edwinyzh/656250 … 7994811d7f
In short, to implement the 'UPDATE...WHERE...' mechanism in mORMot's batch-update.
Do you like this suggestion, ab?
#423 Re: mORMot 1 » Where is sqlite3fts.obj ? » 2018-10-12 03:58:03
I always pull from the mORMot github repository.
#424 Re: mORMot 1 » Is SessionClose the opposite of SetUser? » 2018-10-10 16:40:59
@pvn0, thank you! The info you provided is helpful for me to further understand the issue, however, at my side the solution is still not obvious, so I'll postpone the check. But i'll post updates when I find some details.
#425 Re: mORMot 1 » Is SessionClose the opposite of SetUser? » 2018-10-09 06:11:50
Original log entry (the words between ... and ... were translated from my local language):
EXC ECrtSocket ("SockRecvLn [10054 WSAECONNRESET \"... connection closed forcibly by remote host...\"]") at 0096414A SynCrtSock.TCrtSocket.SockRecvLn (5153) stack trace API
Using the latest mORMot as of today.
#426 Re: mORMot 1 » Is SessionClose the opposite of SetUser? » 2018-10-09 03:42:54
...I have only seen this error if Client calls SessionClose (manually or through destructor) while a session releated operation is still active on the server...
How do I know 'a session releated operation is still active on the server'? Your hint is helpful, Thanks!
#427 Re: mORMot 1 » Is SessionClose the opposite of SetUser? » 2018-10-08 18:00:10
No, I've not been able to fix the issue yet, but since the issue is not severe so it is of low priority here, but I'll be appreciated if anybody give me a pointer
#428 Re: mORMot 1 » TestSql fails at utf8 » 2018-10-08 07:06:28
This is the issue I couldn't find a solution yet, when running the mORMot tests, although I'm using mORMot without any issues, the amount of failures still frightens me.
#429 Re: mORMot 1 » Latest mORMot.pas does not compile under latest FPC. » 2018-09-30 06:03:08
I confirm the compilation issue's fixed.
#430 Re: mORMot 1 » SynLog - To allow ignoring exceptions with code » 2018-09-25 15:32:52
Hi ab,
I strongly strongly strongly prefer a callback event to check for the exception kind, it'll be much flexible! Thank you!
#431 mORMot 1 » SynLog - To allow ignoring exceptions with code » 2018-09-25 14:31:23
- edwinsn
- Replies: 11
Hello Arnaud,
To ignore certain exceptions I know there is alread the ExceptionIgnore property, but it's not very handy in case there are tons of exceptions to ignore.
For example in one of my programs which uses both mORMot and Indy, I definitely want SQLite3Log to intercept exceptions raised by the mORMot framework only but NOT exceptions caused by Indy.
But there are too much exceptions that's possible raised by Indy, so instead of calling ExceptionIgnore.Add() near one hundred times, I'd prefer to code some thing like
TMyChildSynLog = class(TSynLog)
...
function IsExceptionIgnorable(aException: Exception): Boolean;
...
implementation
...
function TMyChildSynLog.IsExceptionIgnorable(aException: Exception): Boolean;
begin
Result := aException.InheritesFrom(EIdException);
end;
...Hope it makes sense.
#432 Re: mORMot 1 » Latest mORMot.pas does not compile under latest FPC. » 2018-09-20 15:51:33
Update: With Lazarus/FPC trunk installed by fpcupdeluxe just now (Lazarus 2.1.0 r59094, FPC 3.3.1), the latest mORMot can be compiled successfully, even cross-compile for linux64!
#433 Re: mORMot 1 » The new SQLite 3.25.0 now supports window functions! » 2018-09-18 06:25:28
And mORMot now includes SQLite 3.25.0: https://github.com/synopse/mORMot/commi … 53bf38721d
#434 Re: mORMot 1 » Is SessionClose the opposite of SetUser? » 2018-09-10 14:00:17
@pvn0,
Thanks for the help. The program does free the client in the main form's Destroy destructor method, but the server-side
still produce the mentioned ECrtSocket error log.
And I noticed a strange phenomenon - when I run the program in debug mode and step into the TSQLRestClientURI.Destroy source code, no more ECrtSocket exception log, as if the program needs to close **slowly**, strange...
#435 mORMot 1 » Is SessionClose the opposite of SetUser? » 2018-09-09 12:38:46
- edwinsn
- Replies: 9
So I have been implementing the "Login" operation via TSQLRestClientURI.SetUser, but recently I noticed the server-side logs contains log messages something like "ECrtSocket (... connection closed forcibly by remote host...) " when a client program exits, so I thought maybe the program needs to do a "Logout" via SessionClose, **Is it correct?** If not, what method implements the "logout" operation Thanks.
#436 Re: mORMot 1 » Android compilation fails with the latest trunk FPC&Lazarus & mORMot) » 2018-09-06 10:22:37
Same error message and line number here when compiling "04 - HTTP Client-Server" for Windows with FPC/Lazarus trunk installed today.
#437 Re: SyNode » SpiderMonkey Lite Build » 2018-09-01 03:16:29
6.8MB seems to be already a great deduction of the file size!
#438 Re: mORMot 1 » How to see changes since... for ORM Objects? » 2018-08-31 08:53:19
Or you can use the OnUpdateEvent on the server side, this even is triggered when there is a update/insert/delete event happens on a specific table. Can you store a list of <tableName, LastUpdateTime> pairs in the memory, and expose an API to the clients.
#439 Re: mORMot 1 » How to see changes since... for ORM Objects? » 2018-08-31 07:05:46
Not at the ORM level but at the sqlite db engine level, few days ago I read an article on tracking sqlite table changes into JSON string, using triggers, not sure if it helps in your case: https://blog.budgetwithbuckets.com/2018 … gelog.html
#440 Re: mORMot 1 » What is best practice for multi-threaded in-memory database? » 2018-08-26 02:28:13
@xiwuping,
darkThreading has a lock-free and thread-safe queue implemented using array, not sure if it suits your needs.
#441 Re: mORMot 1 » [Fixed] Expose the version number for mORMot and SQLite » 2018-08-25 15:54:24
Great! Thanks!
#442 mORMot 1 » [Fixed] Expose the version number for mORMot and SQLite » 2018-08-25 10:34:31
- edwinsn
- Replies: 2
Hello ab,
How about expose publicly the version number of both mORMot itself and SQLite engine? That can be very helpful to show in, for example, the log file or the program's About window.
PS, currently I can see the EXPECTED_SQLITE3_VERSION constant in SynSQLite3Static.pas but it's only visible in the unit scope.
Thanks.
#443 Re: mORMot 1 » What is best practice for multi-threaded in-memory database? » 2018-08-25 06:15:48
3 thoughts:
- Try ScaleMM and benchmark to see if there are any improvements.
- Try TSQLRestServerDB with in-memory sqlite db (use ":memory:" as the db name).
- Use TSQLDataBase directly with in-memory sqlite db.
Out of curiosity, what kind of program is it?
#444 Re: mORMot 1 » The new SQLite 3.25.0 now supports window functions! » 2018-08-16 08:44:24
Ah you are right!
#445 mORMot 1 » The new SQLite 3.25.0 now supports window functions! » 2018-08-16 02:46:55
- edwinsn
- Replies: 5
I didn't know SQL's window function before, but from what I've seen so far it seems to be very helpful, and will learn it here:
https://www.windowfunctions.com/questions/over/
IIRC, in the old versions mORMot didn't handle the 'WITH...' statement (CTE) correctly until it's reported to ab and he fixed it quickly, so I'm not sure if the special syntax of the 'window function' statements needs special attentions to be paid to it when ab's integrating the new sqlite 3.25
#446 Re: mORMot 1 » mORMot and the new full featured Delphi Community Edition » 2018-07-19 15:38:40
Good!
#447 Re: mORMot 1 » Status of Wince support of the Cross-platform clients? » 2018-06-24 13:28:45
Aha! You are right. And what about future versions? Do you plan to keep the patch for future versions?
PS: The customization ability of fpcupdeluxe is amazing, I didn't know it can patch the installation
#448 Re: mORMot 1 » Status of Wince support of the Cross-platform clients? » 2018-06-23 14:22:07
@AOG,
By "Only 1.6.1c will patch Lazarus to be suitable for wince.", do you mean all newer versions since 1.6.1c will patch Lazarus for wince, or only this single version will do that? Thanks.
#449 Re: mORMot 1 » Status of Wince support of the Cross-platform clients? » 2018-06-23 14:18:45
update:
Yes, updated fpc/lazarus with 1.6.1c, successfully compiled a blank lcl project, selected arm/wince, without changing the 'widgetset', and the compiled program successfully run under wince 6.0.
I also have successfully compiled the mORMot sample 27 (lclclient) for wince, however, when running under wince 6.0, it starts, the form lasts for about 1 second, and then exit. This is the first time compile this sample for any platform and I haven't dig any further yet.
#450 Re: mORMot 1 » Status of Wince support of the Cross-platform clients? » 2018-06-23 13:30:13
@AOG,
I only have the arm-wince cross-compiler installed with 1.6.1.c, all the others were installed by 1.6.1b.
I will try again to update all the other modules with 1.6.1c now.