You are not logged in.
mpv does use it several heavily-loaded production sites, with more than 1,000,000 requests per day.
See http://synopse.info/forum/viewtopic.php?pid=9989#p9989SpiderMonkey over a mORMot kernel rocks!
Sounds great!
What do you mean?
Is it free to use, without royalties?
-> Yes, as soon as you follow one of the mORMot Open Source licences.
Actually, I mean the stability status of the current version of the spidermonkey wrapper, is it alpha, beta or production-ready? Thanks.
It is not included yet.
Following the node.js APIs is perhaps not worth it, unless it is easy to do.
Yes! I'ts worth it!
If you just have a look at the total package number (67,091) at nodejsmodules.org
In other words, we Delphi developers can suddenly have a bunch of new open source libraries (I assume node.js libs that are compatible with browsify will also work with this SM wrapper).
dll is here: http://synopse.info/files/synsm.7z
Great! Thanks!
Looks like it just works! Any thoughts as to can we use it in production?
Was trying the JSHttpApiServer demo, mozjs.dll is missing, not sure where to download the correct version for your wrapper...
We have just published huge code and documentation refactoring of SpiderMonkey related units:
- direct variant-based execution of JavaScript functions from Delphi
- direct variant-based execution of Delphi native code methods from JavaScript
- added JSON process, now ready to fulfill requirements needed for method-based / interface-based Delphi code automatic publication into JavaScript.
See http://synopse.info/fossil/info/74d11c1369
WOW! This is incredibly awesome!!!
With ES6 is being standardized, the JS language will be much more close to what Delphi/C# programmers would expect (think TypeScript).
I think this project can at some degree solve, 1) the shortage Delphi developers in the market, 2) the shortage of Delphi libraries nowadays. How? By delegating some of the implementations to JS developers!
I also think if this project done right and well, can add one of the major benefits of dynamic languages - interactive development (no compilation is needed)! Imagine when your programs runs in DEBUG mode, you can call up an integrated JavaScript code editor with code-complete, edit and click save, and our program is enhanced!
If you still don't understand the benefit of interactive programming, maybe I can put it in another way - this can completely replace debugging tools such as CodeSite, and does much more. But interactive programming is so much more than that, go do a google yourself ![]()
PS, re auto-publishing of Delphi objects, I think you should provide a way (maybe by using Attributes) to specify which objects we do/don't want to expose to JS.
ps2, I have a live html/css code editor called LIVEditor that might need this SpiderMonkey wrapper, and I wish it can be used standalone - without other parts of the Synopse framework - which are gems but I don't need ![]()
Thank you guys!
Hi, it seems that the latest SVN commit of code.google.com/p/delphi-javascript is Dec 6, 2013, does anybody knows if it uses the latest SpiderMonkey or not? Thanks.
Good news Arnaud!
Thanks Arnaud,
While little code overhead is good, what I actually want is a smaller codebase so that in the future it'll be easier to port to Free Pascal if I would ever need that.
How hard would that be?
Hi Arnaud,
This may sounds like a strange problem, but all I want to use is inter-process interface calls, like what the "Demo 14 - Interface based services" demonstrated, and I don't need database support.
So I'm thinking of extracting that part from mORMot framework, would you give me some guides? Thanks!
Hi Arnaud,
Well, I understand, but in my specific case, the server will read data only and the client will write data only, so directly accessing to the DB is ok, and I want things to be simplified. Does your answer means that there is no way to re-compile the engine to be thread-safe?
Hi Arnaud,
Thank you very much. That helps.
Hi Arnaud,
You said that the synopsis sqlite engine is not compiled to be thread-safe, can I recompile it to be thread-safe?
You know, there are chances we need to access the database using other ways, for example, to access the DB from programs written in other language. In my specific case, I'll write some part of the app in Python, the only reason is that, that part will be cross-platform ready.
Thanks and have a nice day!
the background: I'm designing a DB to store millions of objects of different structure, so to make things simplified, the DB will only have two types of objects, or in other words, two tables:
TMyDocument = class(TSQLRecord)
PrimaryKey: Integer; //primary key of the object
MetaDataAsJSON: RawUTF8; //object properties in JSON format.
end;
//store blob datas of TMyDocument, can have many instances that are associated with TMyDocument.
TMyBlob = class(TSQLRecord)
PKey: Integer;//primary key
FKey: Integer;//foreign key, maps to TMyDocument.PrimaryKey
MetaDataAsJSON: blob;//blob data, I still don't know what data type to use here, but you should get my point.
end;
Hi Arnaud,
Performance-wise, what's the best way to pass TSqlRecord derived object with large blob data, say 10 MB's? The client will get large amount of blob data from the internet, and pass the data to the server which in turn save it to the DB (via your framework, of course), Q#1: is the named pipe protocol is the best choice?
I did some research, the developer of the original sqlite project suggested to design the table as such - store the blob data as the last field in the table, or even store the blob data in a separate table. Others said that increasing the page size of the DB will help in performance.
Q#2: Any other hints you can offer?
Thanks!
unbelievable! Very few open source projects are so well documented!
Great job, Arnaud!
Thank you for your effort!
Thanks Arnaud, in the model layer I use a object model that uses properties such as Parent, PrevSibling, NextSibling, InsertChild, DeleteChild, etc, and these members are necessary when implement the program logic, just like a generic tree data structure.
But here we are taking about persisting the objects. When saving to the database I use "ParentId" and "Idx" table fields instead, to simply things.
I'll let you know how it goes when the project it's started ![]()
Hi Arnaud,
What approach do you recommend to persist tree-like object relationships? Currently with direct access to sqlite I use only two fields to store the relationships: "ParentId" and "Idx", where "ParentId" references a rowid of the parent nodes, while "Idx" indicates the index of a node among its siblings.
Thanks. I know I've got too many questions in this forum, I'm just new in the ORM thing ![]()
That's a little more code to write...
Thanks for the helpful info, Arnaud!
I'll be pleased to post some helpful code here if I even go for this approach, at this moment I'm just doing some technical research for a planned program.
If the Python code will talk to the server via http/json, I'm still wondering how the server side code will look like, which method should I use/extend?
Thanks!
Hi Arnaud,
I might need to use Python to developer some clients that talk to a restful server powered by Synopse sqlite, since the client need to access some data that's easier to access by Python.
In short, the Python clients to use a thread to read data from other data sources and then write data to the sqlite database. To simplify things I plan to directly write the data to the sqlite database, instead of posting JSON data to the restful server (what do you think?), but the clients still need to send some some logs to the restful server to indicate the operation process.
I'm wondering if you can give me some hints as to what method to use for the communication between the server (developed in Delphi) and the clients (developed in Python). All the server and programs will be running on the same machine.
Thanks!
Wow! The batch update mode is great!
This is just GREAT!
Hi Arnaud,
With my planned program, the server will mostly receive new objects (IOW, insert commands) from multiple clients. In my previous experience with sqlite, wrapping a bunch of insert commands with 'BEGIN TRANS...' and 'COMMIT TRANS...' Can speed things up a lot. How we do this in Synopse sqlite?
And other tips you can give me to speed up such bulk insert operations?
Thanks!
Oops, I was in a hurry, sorry.
Well done, Arnaud, you are fast.
But I suspect there is still a problem of design in your program. A more generic type of "document" would be better than multiple tables, IMHO.
Maybe the use of 'documents' misled you, actually, it'll be a program that need to store a wide variety of different types of objects.
This is an interesting question.
@thunya, I haven't tried it myself, but have you tried? Just a guess...
function CreateSampleModel: TSQLModel;
begin
result := TSQLModel.Create([TSQLInherited, TSQLInherited]);
end;
Hi Arnaud,
While it'll be perfect if you can remove the limit (imagine large projects that will have over 64 types of objects that need to persist), I think I've come up with a workaround:
For saving:
1 - On the client side, encode the TSQLRecord-derived objects to JSON strings;
2 - Call the server side method to pass the JSON strings;
3 - On the server side, decode the JSON strings back to objects;
4 - Call the direct sqlite restful server to persist the object
And do the other way around when the clients need to retrieve objects from the server.
It's still quite neat IMHO, thanks to the excellent framework.
Thanks for your effort, Arnaud!
HI Arnaud,
Sorry for the late response. Yes, I agree that in most apps 64 tables would be enough, but in some cases, my case include, 64 tables is not enough. As you might remember in another post I'm planning to use Synopse sqlite to store various "documents", and the app will continue adding support for more type of documents, it will reach the 64-table limit at the end but 256 will be enough.
Or, if I don't use the permission-related features, can we remove this limit? Maybe via a conditional define?
Thanks!
There is no limit if you use SQLite directly.
This limit is just from the ORM side.
Thanks Arnaud, are you planning to remove this limit? Thanks!
Hi Arnaud,
This is a serious limit, SQLITE itself doesn't have such limits, right? Will remove this limit in the future?
And is there any other limits Synopse SQLITE introduced?
Thanks!
Hi Arnaud,
I've a further question about utilizing your open source project to consuming the google data api if you don't mind, since you must good at the HTTP protocol.
When establishing connections, authentication and sending requests - which unit in the Synopse repository to use? Do you any hints for this? Thanks!
Kudos to Arnaud for this excellent multi-tier ORM framework for Sqlite!
I've been searching for a document-based database for a desktop app. As opposed to RDBMS, among other features, a document-based database gives you a great benefit - schema-free. That means you won't need to add/delete/update fields if the structure of the data you persist change;
I checked many of the NoSQL's that has been the buzz in recent years, first, Key/Value-based ones were eliminated, then feature-wise, MongoDB is closest to what I need and it's powerful, but unfortunately, maybe because it's mostly designed for high-volume web servers, on Win32 system it's DB size limit is 2GB; Moreover, it's installation size is too large for my embedded use.
With no result by checking the NOSQL's for my requirement, I then looked at my favorite embedded DBMS - Sqlite, I've used it in other projects with very good result, except that it's a pain when the database schema changes; So Arnaud's Synopse SQLITE framework came to my mind, after looking at the source, demos and forum posts a bit, I found several good things about it:
1 - The ORM framework will solve the pain of database schema changes, at least in my specific scenario, since the major change will be adding fields only;
2 - Its neat multi-tier feature will allow me to implement the addin modules of the app as standalone EXE's; And It will allow the app to extend to C/S architecture very easily - just change from named pipes to tcp/ip;
3 - While there is not a complete manual for the framework, its source is very very well documented, and it includes several very explanatory demo projects, and Arnaud's blog posts and his replies to the forum posts are very detailed and helpful.
4 - Arnaud's very active in the forum, man, this is only an open source project!
5 - There are many gems hidden inside this open source framework, for example:
a - A JSON parser which can serialize/deserialize a TPersistent-derived object very easily;
b - Some encryption/hassing classes including AES, MD5, SHA1;
c - A base64 encoding/decoding classe;
d - A unit that implements HTTP/1.1 client and server protocol;
e - ZLib compression, plus a zip writer?
f - A PDF renderer;
g - A GDI plus interface;
h - A Windows service management class for functions including, but not limited to stopping, starting a Windows service;
Wow!
Thank you Arnaud! You are a hero and genius (ifyou check other open source projects found in this site, you'll have to agree with me on this!)!
Great! So the CreateMissingTables method will handle the field adding. I don't need to rename or delete a field.
Thanks for the info, Arnaud!
Hi Ab,
Will there any problems (encoding, for example) if the server is written in D7 and some of the clients are written in Unicode-enabled Delphi versions, such as D2010, in order to access some new functions?
Thanks!
I'm wondering if you think that it's a good idea that offering multiple versions of the sqlite3 objects, for example, one with this ICUenabled option defined...
Hi ab,
Thanks for the info. I think I've come up with an alternative (maybe better) approach - The multiple clients send the logs to the single server via a RESTful JSON method on the server side, and in that method the log will in turn be stored into another database by via another TSQLRest object.
Thanks for the link, ab.
Is Synopse SQLITE compiled with the SQLITE_ENABLE_ICU pre-processor symbol defined? Thanks.
I got it!
And you might overlooked my new question - "is it possible to have multiple TSQLRestClientURINamedPipe instances in a single process?"
Thanks!
Thanks for the sample code! But I don't understand, what's the advantage of using TSQLTableJSON?
One more question, for some reasons, I can't use tbigtable but want to store the logs to a SQLITE DB instead. I'll use TSQLRestClientURINamedPipe server component, and I'm wondering if I can have multiple TSQLRestClientURINamedPipe instances - one for the main data while another for logs only, so that I can save my logs to a separate database. But I read the following line in the source code:
// - allows only one ExportServer*() by running process
Does it mean it's not possible? And I found this similar thread about TSQLite3HttpServer
Hi AB,
Sorry for so much questions in a single day, since I'm building a new app and I some requirements need to be well thought ![]()
Two questions about using FTS4 with this excellent sqlite framework:
1 - Do you have any tips/info as to support international characters when using FTS4?
2 - Do you have any hints/info as to removing stop words? I see in this article that this can significant reduce the database size.
Thank you again!
Well, forget about this question, I read the following release notes in the source file:
Version 1.7
- alter table with newly added fields to a TSQLRecord (see GetSQLAddField()
methods)
Hi ab,
What if published properties are removed or added to a TSqlRecord decedent? Will this framework automatically handle the table schema change? Thanks!
Great advise! I'll check TSynBigTable!
Wow, found another hidden gem in this lib following your clue - Class for controlling the windows services.
Thanks for the further info, ab.
At the application level, logging is made easy via the TSQLRecordLog record type
Hi ab,
Can you give me some hint or a few lines of sample code to get me started using TSQLRecordLog?
Can I store the logs to an external database, rather than the main database?
Thank you!