You are not logged in.
Last two week I work on refactoring my “scripting support” codebase to share it with mORMot. The work is in progress but first result I deploy in few days.
The main ideas is:
Why mORMot need scripting support
At the moment mORMot allows us to create well-designed, scalable, and fast application. But often we need to customize application logic in different way for different customers. For example if we develop application for accounting, entities is the same for all customers (store, goods, departments, invoice and so on) but transaction logic differ for every customers. In this case if we write separate logic for every customer, we go to “branch-hell”, if we start to add options for every case - the number of options increase and code become unsupported. Even if we move this logic to “customer specific” plugins (dll) we need to compile it and unable to change “this one little thing” directly on customer workplace. So moving such type of logic to scripting engine is a good idea. In this case we have one version of program with base functionality and different transaction building scripts for different customers. Also scripting is a must for report generation. Creation wrapper over SynPdf for scripting engine give us possibility to create reports without change main application codebase.
My opinion is: only speed/memory critical functionality must be “hardcoded” everything else is work for scripting engine. But this is IMHO.
Why JavaScript
There are many reasons to chose JavaScript as scripting engine.
- This is Web assembler. Every programmer in one way or another knows JS.
- JS is a very powerful language. See for example excellent article (http://javascript.crockford.com/inheritance.html). (IMHO Crockford is the best author for javaScript learn – "JavaScript – good parts" is a “must read” book);
- There are HUGE number of things written on JS: syntax sugar - CoffeScript,TypeScript, Dart; template engine – jade, mustache…; SOAP and LDAP client and many-many other. And node.js libraries of course.
- In case of Rich Internet Application we can directly share part of logic between client and server (validation / template rendering e.t.c.) without any middleware.
- We can use other scripting language (DWS for example) translated to JS and run it under the JavaScript engine.
Why SpiderMonkey
I look at many engine before chose SpiderMonkey.
- pure Delphi Besen http://code.google.com/p/besen/ ( bad performance speed/memory compared to V8 & SM)
- V8 (C++ hard to integrate with Delphi)
- Microsoft scripting host ( COM / poor API / bad performance / only for Windows)
I choose SpiderMonkey because:
- Stability. It is the oldest, most heavily tuned engine available.
- Open source with HUGE community.
- C API – easy to integrate with Delphi
- ECMA-262 Edition 5 (and Harmony soon)
- It is very fast. Really VERY fast.
The road map
- First of all I start from converting SpiderMonkey .h files to Delphi. All known conversion (http://code.google.com/p/delphi-javascript http://delphi.mozdev.org/javascript_bridge/) based on old SM engine (1.5 – no JIT and ECMAScript3). I convert SpiderMonkey 1.8.5 witch give us full ECMAScript5 support and JIT.
- The second step is create Delphi classes to easy deal with engine – I do it in a week.
- Third – implement CommonJS Modules specification and node.js “fs” module (it’s easy) – this give us possibility to use huge number of node.js modules.
- Create wrapper for mORMot classes (TSQLRecord and so on) to use scripting for implementing Interface based services (IBS). (additionally do something like “contract” for IBS may be using JSON-Schema for easy use IBS from AJAX client.
- Implement debugger
The goal
I admire the success of Node.js. But I think it has one big architectural problem: due to single thread node make all API async, and async code become unreadable and “undebuggerable”. mORMot is multithread – it is easy to make JS work the same way mORMot DB connection pool work and write JS code without callback what call callback what call callback and so on. Also we have great ORM engine – so let’s use it in script’s. And we native to Windows.
So the goal is: Delphi based, FAST multithread server with ORM and node.js modules compatible.
Suggestions are welcome!
Sorry for my English
Offline
This is great.
Source code you sent to me is just awesome.
We will work together to integrate it into 1.18 release (so in short time).
In addition to your proposal, I'd like to add a direct link to our interface-based services.
That is, direct connection to mORMot services on server side (to implement the business logic of services) and client side (to call the remote business logic).
The upcoming event-based mORMot feature will also benefit of it.
It could be very handy to add an event listener to an existing business core, then use JavaScript to listen to those events, and define some custom process, without touching the main business logic.
JavaScript can easily fit in a Domain Driven Design, at least at several levels:
- Client layer,to validate input and create custom UI or reports;
- Application layer, as the application logic should not be part of the client, but stay on the server;
- Application layer, to adapt the common business model to one customer or a given application (e.g. custom behavior, reporting);
- Business layer, to define the business logic using the domain objects (e.g. to adapt to a given change of policy).
I mean, interfaces will bring something unique to the node.js ecosystem.
JavaScript lacks of interface support and strong typing, this is why I look at DWS script and TypeScript.
TypeScript is an Open Source scalable JavaScript development with types, classes and modules. It is published by Microsoft, but is open source.
It comes with full source code in TypeScript, so can be embedded very easily to our SpiderMonkey modules.
TypeScript is great because it is a layer over JavaScript, adding strong typing and interfaces.
DWS script compiler to JavaScript is not free any more. So it is no option for us. Sorry Eric.
I suspect TypeScript could be a great optional syntax for our mORMot business layer.
http://www.typescriptlang.org/ and http://en.wikipedia.org/wiki/TypeScript
And... Anders Hejlsberg, lead architect of Delphi and C#, has worked on development of TypeScript!
What we need also is an editor and debugger integrated into the framework, I suspect.
I've written a blog article about this proposal.
http://blog.synopse.info/post/2013/01/2 … -to-mORMot
Offline
AB, I already have code for direct connection to mORMot services in server side from scripting via HTTP (wrapper around TWinHTTP + simple JS class to implement protocol details) (something like http://nodejs.org/api/http.html ) - I rewrite it in new way and shere when we finish refactoring of main part.
When implementing event is good to use EventEmitter pattern (http://nodejs.org/api/events.html ) - the main different from Delphi OnSomthing is:
1) posibility to add sereval different listiners to one event - very important thing. For example I use it on auth event (fired by auth procedure of my mORMot-based server to fill custom user information. One subsystem do App.on('auth', {obtain additional info from LDAP), other App.on('auth', {obtain additional info from database table) and so on.
2) posibility to create new event types "on the fly" (something like sendMessage in windows with custom code), i.e. Class.fireEvent('myNewEvent', param1, param2) and in other part of code Class.on('myNewEvent', function(){do something})
Anders Hejlsberg... I'll look on TypeScript once again!!
The reason I do not implement debugger yet is new Debuger API I waiting from Mozilla - no reason to implement it in old way. As editor I use (with grate success) old good SynEdit on delphi side and codemirror on web side (even with code insight)
Offline
AB, I already have code for direct connection to mORMot services in server side from scripting via HTTP (wrapper around TWinHTTP + simple JS class to implement protocol details)
I suspect this is far from optimized to use a true HTTP request instead of direct in-process RESTful request, on the server side.
Offline
I just checked out TypeScript implementation.
What is pretty awesome is that not either the compiler itself is written in TypeScript, so can run directly in SpiderMonkey in our Delphi applications, but also that they implemented a TypeScript editor as HTML!
If you take a look at the official web site, and click on "Open in Playground" you will find an editor / compiler in HTML in your browser.
See http://www.typescriptlang.org/Playground/
If there is some error in the TypeScript editor, it is underlined in red.
If you move your mouse on some instance or type in the TypeScript code, you will see its definition!
If you type something, you will see some auto-completion list, just like Intellisense!
It could very easily be able to integrate a very efficient business logic editor in mORMot projects.
Customer safely connect on the mORMot server in a private URI, then write and adapt its business logic in TypeScript.
"Wow effect", I suspect.
Offline
Yes, editor is cool. We can even use it in desktop applications using Delphi Chromium embedded (http://code.google.com/p/delphichromiumembedded/)
Offline
I am the developer of http://code.google.com/p/sejscript/ perhaps it might interest you.
It consists of an extendable object framework using a stripped down ecmascript implementation as "glue".
Pros: pure Delphi, easy to extend, very small footprint, multi-threaded, ...
Cons: 32-bits only, interpreted (simple VM), free,
A "like node.js but not-async web server sample" is included. Check the *.js files.
If interested just let me know.
Offline
Very interesting.
No revision on the svn tree?
Of course, there is only a simple VM, so performance won't be very high, but what I like very much is the low overhead.
Source is still readable.
Nice work!
How do you execute the regression tests?
What is your EcmaScript implementation coverage? Can you use some standard JavaScript libraries?
Offline
> No revision on the svn tree?
No. It's only me coding on different machines (some of them without inet) so, I just carry everything on a pen, when I get something stable I just upload it. Not very pro, I admit..
> performance won't be very high, but what I like very much is the low overhead.
I try to keep it extremely simple on purpose: api-mallocs instead of memory manager, no objects (the only one left (in the parser) is legacy), minimum DOM, as less Delphi-RTL/RTT as possible,...
If more performance is needed, the generate VM-code is easy to convert into something more efficient (ie. inline calls which would give us a native engine with a very bad code generator, but way faster than current implementation) or with more work (and much better performance) LLVM code.
> How do you execute the regression tests? What is your EcmaScript implementation coverage?
Haven't bothered with regression tests. My goal is not to produce an Ecma implementation, but a "glue" tool for different projects (in different languages).
I test the host applications (which are controlled by the "glue"). If results are the expected ones, stability (leaks, crashes, ...) is OK, and performance (time/resources) is reasonable...then I take it for good.
Nevertheless I like the language, and core issues: closures, functions as objects, scope-chain,... are implemented. Others (ie. regexp) not.
> Can you use some standard JavaScript libraries?
Which ones ?? most of them (jquery, ExtsJS,...) rely on a DOM that it's not present...
It's something do it on my spare time, so I don't try to cover a wide area of functionality, just the one I need.
Offline
Unfortunately I could not complete the implementation of the SpiderMonkey support before my vacation. In the next two weeks I will not have access to a computer, but I will continue to work after 10 February. I hope during the holidays I will have fresh ideas
Offline
Enjoy your vacation!
You own it!
See you after next month.
We will wait for your implementation, or include SpiderMonkey in 1.19, if we release the 1.18 officially in the next weeks (which is pretty much possible).
Offline
That's it...
Code has been committed...
Introducing JavaScript execution using the SpiderMonkey library in mORMot framework!
See http://synopse.info/fossil/info/744111d689
Thanks a lot Pavel for all this GREAT contribution!
Offline
Does anyone have example how to perform JavaScript execution using the SpiderMonkey library in mORMot?
Offline
I write example ASAP (very busy in the main work now). AB - it would be nice to include SynSMSelfTest.pas TestSynSM.dpr files to trunc - it give some basic examples of use SM. And compiled dll (mozjs.dll nspr4.dll) - compiling mozjs is not a simple task under windows .
Offline
Yes - I would like to include the self tests to the official mORMot test suite.
And also supply the .dlls as .zip resource!
I made a trial to load the .dll as dynamic library, but it is more complicated than expected, especially when defining callbacks.
Offline
Hi. I'm trying to compile SM units. And wonder where can I get precompiled Windows binaries of SpiderMonkey compatible with mOrmot SM units?
Offline
I just uploaded it at http://synopse.info/files/synsm.7z
Offline
Next time I plane to extend SynSM and add sample of how to add Delphi classes to SpiderMonkey. Unfortunately, my current implementation is highly dependent on the new RTTI + TValue and requires modifications to be compatible with Delphi 7 before merge it into mORMot. But in any case, I plan to get away from the new RTTI - it is VERY slow and is not compatible with FreePascal....
Last edited by mpv (2013-10-08 17:11:22)
Offline
I would like to add:
- an orm object able to access most useful REST/CRUD methods of the mORMot ORM (then returning objects from the JSON);
- a service object able to implement server-side services as with other mORMot SOA interface-based services;
- a client side (why not?) able to consume orm object and service object.
A KISS implementation (relying on JSON on most wanted methods) could be easy to implement.
What do you think?
Offline
Yes - this will be good. For client- side is more important IMHO to create wrappers for PDF creation - in this case user can move report generation logic to JavaScript
Last edited by mpv (2013-10-08 18:46:12)
Offline
Dear Sirs,
I have one question:
"- Third – implement CommonJS Modules specification and node.js “fs” module (it’s easy) – this give us possibility to use huge number of node.js modules."
Did you already successfully implement node.js “fs” module together with SpiderMonkey JavaScript engine -- or -- it's still in your plans for future?
I didn't found any samples of using node.js “fs” module in your code.
With best regards,
Offline
It is not included yet.
Following the node.js APIs is perhaps not worth it, unless it is easy to do.
I'm still considering a more "abstract" approach, based on interfaces, and Delphi-side implementation of low-level access.
See http://synopse.info/fossil/tktview?name=651c13009b
and http://synopse.info/fossil/tktview?name=bd94c11ab1
Offline
We found very unexpected issue while working with SpiderMonkey dll. To work correctly SpiderMonkey require setting of exInvalidOp FPU control word. In terms of Delphi this must be something like this(in dpr):
var
SavedExceptionMask: TArithmeticExceptionMask;
begin
SavedExceptionMask := SetExceptionMask([exInvalidOp,exDenormalized,exUnderflow,exPrecision]);
try
.......
Application.Run;
finally
SetExceptionMask(SavedExceptionMask);
end;
Usually Delphi allpication run with FPU parameters [exDenormalized,exUnderflow,exPrecision], we add exInvalidOp.
If we not set exInvalidOp flag, JavaScript code, what operate with undefined or NaN values rise invalid floating point operation (not in line where it actually raised, but on next call to FPU, so it impossible to handle such exception types)
'a' + (undefined + 1); 'a' + (undefined + 1);
Unfortunately, I'm not sure how it will affect the whole application since we change the way, Delphi usually work with floating point exceptions.
Last edited by mpv (2014-03-27 09:55:44)
Offline
Hi, it seems that the latest SVN commit of code.google.com/p/delphi-javascript is Dec 6, 2013, does anybody knows if it uses the latest SpiderMonkey or not? Thanks.
Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.
Offline
Hi, it seems that the latest SVN commit of code.google.com/p/delphi-javascript is Dec 6, 2013, does anybody knows if it uses the latest SpiderMonkey or not? Thanks.
Nope, AFAIK it still uses the 1.8.5 revision of the library.
The next version is not published yet:
The next release will be SpiderMonkey 31, around July 2014.
We have just published huge code and documentation refactoring of SpiderMonkey related units:
- direct variant-based execution of JavaScript functions from Delphi
- direct variant-based execution of Delphi native code methods from JavaScript
- added JSON process, now ready to fulfill requirements needed for method-based / interface-based Delphi code automatic publication into JavaScript.
See http://synopse.info/fossil/info/74d11c1369
I have investigated further about the FPU exception problems...
Changing globally the exception mask for the whole application is IMHO not a good idea.
That's why I propose the following changes: http://synopse.info/fossil/info/a47ba91f57 and http://synopse.info/fossil/info/f3fb92e3e1
Feedback is welcome!
Offline
Ups. We also preparing big changes to SynSM - we near to complete SM24 support. So huge merge is granted
To migrate to SM24 we make some changes in mozilla code to suport Delphi - starting from SM24 mozilla drop support for C api (only c++). But there is a lot of impovement in 24 version, include new jit(up to 20% faster on our real task) and asm.js suport. And arrow function as a bonus
In all case future migration to SM31 must be easier.
As far as i understand delphi-javascript use very old spidermonkey. Not shure, but seems pre https://developer.mozilla.org/en-US/Firefox/Releases/17
Offline
We have just published huge code and documentation refactoring of SpiderMonkey related units:
- direct variant-based execution of JavaScript functions from Delphi
- direct variant-based execution of Delphi native code methods from JavaScript
- added JSON process, now ready to fulfill requirements needed for method-based / interface-based Delphi code automatic publication into JavaScript.
See http://synopse.info/fossil/info/74d11c1369
WOW! This is incredibly awesome!!!
With ES6 is being standardized, the JS language will be much more close to what Delphi/C# programmers would expect (think TypeScript).
I think this project can at some degree solve, 1) the shortage Delphi developers in the market, 2) the shortage of Delphi libraries nowadays. How? By delegating some of the implementations to JS developers!
I also think if this project done right and well, can add one of the major benefits of dynamic languages - interactive development (no compilation is needed)! Imagine when your programs runs in DEBUG mode, you can call up an integrated JavaScript code editor with code-complete, edit and click save, and our program is enhanced!
If you still don't understand the benefit of interactive programming, maybe I can put it in another way - this can completely replace debugging tools such as CodeSite, and does much more. But interactive programming is so much more than that, go do a google yourself
PS, re auto-publishing of Delphi objects, I think you should provide a way (maybe by using Attributes) to specify which objects we do/don't want to expose to JS.
ps2, I have a live html/css code editor called LIVEditor that might need this SpiderMonkey wrapper, and I wish it can be used standalone - without other parts of the Synopse framework - which are gems but I don't need
Thank you guys!
Last edited by edwinsn (2014-04-04 06:37:54)
Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.
Offline
Was trying the JSHttpApiServer demo, mozjs.dll is missing, not sure where to download the correct version for your wrapper...
Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.
Offline
dll is here: http://synopse.info/files/synsm.7z
Offline
dll is here: http://synopse.info/files/synsm.7z
Great! Thanks!
Looks like it just works! Any thoughts as to can we use it in production?
Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.
Offline
It is not included yet.
Following the node.js APIs is perhaps not worth it, unless it is easy to do.
Yes! I'ts worth it!
If you just have a look at the total package number (67,091) at nodejsmodules.org
In other words, we Delphi developers can suddenly have a bunch of new open source libraries (I assume node.js libs that are compatible with browsify will also work with this SM wrapper).
Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.
Offline
I've just updated the source code to specify the libraries download URI.
Looks like it just works!
Of course!
Any thoughts as to can we use it in production?
What do you mean?
Is it free to use, without royalties?
-> Yes, as soon as you follow one of the mORMot Open Source licences.
What is the purpose of this library?
- Calling some external .js libraries from your Delphi code, server side or client side (e.g. here we use Shutdown.js to visualize the .md file format) - there are plenty of JavaScript libraries around;
- Write some end-user code in JavaScript (e.g. some business rules to be customized by the client, some report codes) - we will soon add some interface-driven methods to publish mORMot features to JavaScript (ORM, reporting), and allow to write SOA services in JavaScript.
Stay tuned!
Offline
What do you mean?
Is it free to use, without royalties?
-> Yes, as soon as you follow one of the mORMot Open Source licences.
Actually, I mean the stability status of the current version of the spidermonkey wrapper, is it alpha, beta or production-ready? Thanks.
Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.
Offline
mpv does use it several heavily-loaded production sites, with more than 1,000,000 requests per day.
See http://synopse.info/forum/viewtopic.php?pid=9989#p9989
SpiderMonkey over a mORMot kernel rocks!
Offline
mpv does use it several heavily-loaded production sites, with more than 1,000,000 requests per day.
See http://synopse.info/forum/viewtopic.php?pid=9989#p9989SpiderMonkey over a mORMot kernel rocks!
Sounds great!
Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.
Offline
I've added late-binding and a custom variant type for SpiderMonkey.
This TSMVariant custom variant type, can be used to store a native JavaScript object in Delphi, and expose its properties and functions via latebinding.
You can now write Delphi code to execute the showDownRunner() JavaScript method:
content := engine.Global.showDownRunner(FileName);
Which is the same as the slightly faster:
content := engine.GlobalObject.Run('showDownRunner',[SynUnicode(FileName)]);
Of course, you can access to any JavaScript's object properties via late-binding:
engine.Global.myobject.myproperty := 1.0594631; // here myobject.myproperty are accessed via late-binding!
This works also for parameters to any native Delphi function: when a JavaScript object is transmitted, such variant types are sent to TSMEngineMethodEventVariant callbacks...
So you access all JavaScript object internals in your Delphi native code, just as natural as from JavaScript...
See http://synopse.info/fossil/info/0871fa6785 and http://synopse.info/fossil/info/0baebae5d1.
Offline
ab, mpv, this is just AWESOME!
Since JS is case-sensitive, I assume
showDownRunner
in
engine.Global.showDownRunner(FileName);
must be case-sensitive, right?
I've added late-binding and a custom variant type for SpiderMonkey.
This TSMVariant custom variant type, can be used to store a native JavaScript object in Delphi, and expose its properties and functions via latebinding.
You can now write Delphi code to execute the showDownRunner() JavaScript method:
content := engine.Global.showDownRunner(FileName);
Which is the same as the slightly faster:
content := engine.GlobalObject.Run('showDownRunner',[SynUnicode(FileName)]);
Of course, you can access to any JavaScript's object properties via late-binding:
engine.Global.myobject.myproperty := 1.0594631; // here myobject.myproperty are accessed via late-binding!
This works also for parameters to any native Delphi function: when a JavaScript object is transmitted, such variant types are sent to TSMEngineMethodEventVariant callbacks...
So you access all JavaScript object internals in your Delphi native code, just as natural as from JavaScript...See http://synopse.info/fossil/info/0871fa6785 and http://synopse.info/fossil/info/0baebae5d1.
Delphi XE4 Pro on Windows 7 64bit.
Lazarus trunk built with fpcupdelux on Windows with cross-compile for Linux 64bit.
Offline
I've just updated the documentation - now SpiderMonkey integration is documented as needed.
See latest http://synopse.info/files/pdf/Synopse%2 … 201.18.pdf
See chapter "20. Scripting Engine".
Now the SAD pdf reaches more than 1400 pages..
Offline
AB, the next day (april 8), we plan to merge current version SynSM & SynSMAPI with our new SM24 implementation, so please - do not modify sources.
Full list of changes can to JavaScript be found here:
Changes in SM 18
Changes in SM 19
Changes in SM 20
Changes in SM 21
Changes in SM 22
Changes in SM 23
Changes in SM 24
After this we start to:
redesign our debugger implementation to be contributed to mORMot ( Yes, we have debugger )
redesign out TObject descendant import mechanism to be contributed to mORMot
Last allow write in delphi (for "old" RTTI):
TMyClass = class
published
function write(cx: PJSContext; argc: uintN; vp: pjsval): JSBool; // for OLD RTTI - only this signature of function
property myProp: integer; // but all kind of property
end;
TSMEngine.registerClass(TMyClass);
And when in SM
var inst = new TMyClass();
inst.write('some arg');
inst.myProp;
Also we have the same but for new RTTI - it allow to publish to SM almost all methods of TObject
Offline
Great!
So we won't change anything to those two units until you finish the merge.
For rtti access, we could do that directly via current JSON serialization, and via variant arguments for callbacks.
I've published a blog article, containing the latest state of the documentation:
http://blog.synopse.info/post/2014/04/0 … iderMonkey
Offline
Blog article is (as always) perfect!
I thought about using JSON serialization to marshal native to<->from SM. This is a simplest way, and I use it sometimes. But in complex scenario, when native call SM, and then from SM we call native and so on several times I discover big overhead. Also such scenario is a topic of investigation for FPU solution you propouse.
We found good solution for expose native object to SM with minimal performance lost. I provide it while merge. The main idea is to put pointer to native (TObject) to JSObject private and redefine JSClass members for sush JSObjects for access not real JSObject property, but properies of native instead. This is a way moziłla write his DOM..
Offline