Adding JavaScript support for mORMot framework

BeRo1985 · 2016-06-25 12:31:11

edwinsn wrote:

Well done BeRo1985! I wish you'll support FPC as well

Hm, all my stuff are beginning after the program/uses clause with "{$ifdef fpc}{$mode delphi} . . . {$endif}", so FPC support is already there, even in BESEN 1.x since it did exist . I'm developing my stuff almost always for FPC (for the then each most currently released stable version of FPC) and Delphi 7-XE7, except for example my SupraEngine Project now, which is FPC/Lazarus-only, since the SupraEngine Editor itself depends on the FPC compiler for the game-logic-code stuff.

You can convince yourself here at https://github.com/BeRo1985?tab=repositories there are BESEN PasMP PUCU FLRE PasVulkan Kraft PasENet CrossPascal BeRoTinyPascal and so on buildable with FreePascal and Delphi >= 7.

And I'm also co-author of CrossFPC http://www.crossfpc.com/ which integrates the FPC compiler into the Delphi IDE, and that is often helpful, because I can with this writting my code in Lazarus but then debugging my code with the Delphi compiler in the Delphi IDE and building my release binaries with FPC also directly into the Delphi IDE (or into the Lazarus IDE, as I like it on the respective day). I'm doing this so often in this way, because GDB-based Debugging under Lazarus under Windows (but not under Linux, *BSD or MacOSX, where it is at my computers then super smooth without any debugger-action-lags) is super damn slow, at least where a single instruction line code step action needs circa one to two seconds on my computers with mostly-yet-current fast hardware, incl. a Ivybridge-era Xeon Workstation with 32GB ECC RAM, and also because I can ensure it even with this Most-Always-Right-From-The-Prioject-Start-With-Two-Compiler-Development approach that my code is compatible with FPC and Delphi right from a project start.

The same applies more or less also to the being-Operating-System-and-CPU-target-cross-platform-topic for me, where I'm focusing most always Windows, Linux, Android and MacOS(X), except for OpenGL-based projects, because the OpenGL support is under Apple products a hell, for example MacOS(X) has only support for maximal OpenGL 4.1 (but current is 4.5) and the OpenGL drivers under MacOS(X) have very many bugs and are often much slower than the exactly same Apple hardware with OpenGL with Linux or Windows, so I'm dislike also the Metal stuff at Apple, because it is only ignoring on the part of Apple, that their OpenGL support is just super bad, and not OpenGL itself. And unfortunately Apple defends also against Vulkan support in MacOS(X), so my project MacOS(X)-supporting era will be maybe potentially over for ever now, because I don't want work with ignorance system-ecosystems (and this is sadly the case at Apple).

Last edited by BeRo1985 (2016-06-25 12:45:15)

BeRo1985 · 2016-06-26 01:08:33

Other small status update:

I've noticed at my BESEN 2.0 parsing benchmarks with the ten megabytes big Octane javascript benchmark sources, that the WideString-based Delphi 7 BESEN 2.0 build is much slower (by many huge performance worlds apart, no matter with which memory manager (FastMM4, etc.) and with which third-party performance-optimized system unit for Delphi 7) than the UnicodeString-based Delphi XE7 and FreePascal builds.

But this is no surprise, because the WideString type is managed mostly under Windows-targets by the Windows-API as "COM BSTR" strings and the WideString type is moreover also not reference-counted, while the UnicodeString type is managed by the RTL itself and which is also reference-counted.

Why am I saying this: Don't expect with BESEN 2.0 under Delphi 7 the same good performance from newer Delphi versions und FreePascal then, so that no one complained at the end, that the overall BESEN 2.0 performance with Delphi 7 is not so good like as with newer Delphi versions or with FreePascal.

And BESEN 1.x was and is mostly only WideString-based, so that this is a another performance issue cause at the old (but now yet current stable) BESEN 1.x version.

Last edited by BeRo1985 (2016-06-26 01:13:59)

ab · 2016-06-26 06:54:16

To speed up WideString a lot (if OleDB/COM is not to be used directly) in older versions of Delphi, just try https://github.com/synopse/mORMot/blob/ … String.pas
See http://blog.synopse.info/post/2014/09/1 … phi-6-2007

You would get similar performance than Delphi 2009+, if you use this unit in conjunction with FastMM4.

The only exception is that WideString does not support from the COW pattern, so getmem would be called much more often than UnicodeString.
This is why in-place modification, or simple str1 := str1 + str2 pattern is to be avoided.
There are therefore some part of BESEN which may benefit from speedup.
For instance, the JSON support may be much better optimized, in TBESENObjectJSON.NativeStringify.
"str1 := str1 + str2" is slow. A writer class would avoid most memory allocation. See e.g. TTextWriter in SynCommons.pas.
Check also "if Value.Obj is TBESEN... then" in cascade. They may be written using an enumerate and a case, or at least "if Value.ClassType=TBESEN..." which is faster (or directly PPointer(Value)^=TBESEN... for compilers which do not support inlining).

Did you try some real-world performance tuning, using a profiler?
It may identify the scope of what is needed to be optimized, and what could be left as this.
Of course, refactoring of the TBSONValue as a "double/NAN" value may definitively be worth it.

Integration of BESEN with mORMot may be pretty easy, since the object registration, and its methods via RegisterNativeFunction(), sounds crystal clear...
What I like with BESEN is that it has a very small code footprint, and it works with Delphi 7 and FPC, and supports most platforms.
It may be much easier to integrate than SpiderMonkey. Of course, SM outperforms BESEN, but for executing small business rules, BESEN performance may be enough.
For best performance, JSON support has to be optimized in BESEN, since JSON is the "native" value marshalling in mORMot: ORM, SOA, and even stubs/mocks uses JSON for value storage.
Of course, PascalScript has similar features, but it is a pascal-only language, and ECMAScript sounds a better candidate to me for business rules: JavaScript is much widely adopted, and you could share code with the client side (which is a great plus!). If we want strong types, we may even use DWS or TypeScript to generate the ECMAScript...

ab · 2016-06-26 07:30:59

Remark:
I guess that under newer versions of FPC, you have the "UnicodeString" type defined, even if the UNICODE conditional may not be set.
That is, under FPC, depending on the compilation options, you may have string=AnsiString, but still have an UnicodeString fast type at hand, with COW and heap allocation, which would be much faster than WideString/BSTR under Windows.

edwinsn · 2016-06-26 10:14:45

Sounds great, @BeRo1985!

PS, I'm OK if it doesn't support pre-unicode versions of Delphi.

BeRo1985 · 2016-06-26 12:41:30

BESEN 2.x uses PUCU ( https://github.com/BeRo1985/pucu ) for cross-compiler and cross-OS-target unicode support, where PUCU defines:

{$ifdef fpc}
...
 {$if declared(RawByteString)}
  {$define HAS_TYPE_RAWBYTESTRING}
 {$ifend} 
 {$if declared(UTF8String)}
  {$define HAS_TYPE_UTF8STRING}
 {$ifend}
 {$if declared(UnicodeString)}
  {$define HAS_TYPE_UNICODESTRING}
 {$ifend}
...
{$else}
 ...
 {$ifdef conditionalexpressions}
  .... 
  {$if declared(RawByteString)}
   {$define HAS_TYPE_RAWBYTESTRING}
  {$ifend} 
  {$if declared(UTF8String)}
   {$define HAS_TYPE_UTF8STRING}
  {$ifend}
  {$if declared(UnicodeString)}
   {$define HAS_TYPE_UNICODESTRING}
  {$ifend}
 {$endif} 
 ...
{$endif} 
....

     PPUCURawByteChar=PAnsiChar;
     TPUCURawByteChar=AnsiChar;

     PPUCURawByteCharSet=^TPUCURawByteCharSet;
     TPUCURawByteCharSet=set of TPUCURawByteChar;

     PPUCURawByteString=^TPUCURawByteString;
     TPUCURawByteString={$ifdef HAS_TYPE_RAWBYTESTRING}RawByteString{$else}AnsiString{$endif};

     PPUCUUTF8Char=PAnsiChar;
     TPUCUUTF8Char=AnsiChar;

     PPUCUUTF8String=^TPUCUUTF8String;
     TPUCUUTF8String={$ifdef HAS_TYPE_UTF8STRING}UTF8String{$else}AnsiString{$endif};

     PPUCUUTF16Char={$ifdef HAS_TYPE_UNICODESTRING}{$ifdef fpc}PUnicodeChar{$else}PWideChar{$endif}{$else}PWideChar{$endif};
     TPUCUUTF16Char={$ifdef HAS_TYPE_UNICODESTRING}{$ifdef fpc}UnicodeChar{$else}WideChar{$endif}{$else}WideChar{$endif};

     PPUCUUTF16String=^TPUCUUTF16String;
     TPUCUUTF16String={$ifdef HAS_TYPE_UNICODESTRING}UnicodeString{$else}WideString{$endif};

so that I've a cross-compiler (including the old non-so-true-unicode-aware Delphi 7) and cross-OS-target consistent Unicode support for my projects including BESEN 2.0, except at/for the NextGen Delphi Compiler targets, which I'm ignoring it simply now once, until somebody needs really really ... really NextGen Delphi Compiler support at BESEN 2.0 (but only against reward then, because I would not do it otherwise voluntarily ), because they are too codebreaking for me degrading the string type stuff, ARC, and so on.

At least my PUCU side-project has for me all important i18n stuff for my projects, including fast DFA-machine-based UTF8 sequence decoding, cross-compiler and cross-OS-target consistent code-page support and so on, for example for emulating Windows's MultibyteToWideChar and WideCharToMultibyte on non-Windows targets and on older non-unicode-aware compilers.

And the same applies to the string<->double-number-value conversion stuff, where BESEN 2.0 uses PasDblStrUtils ( https://github.com/BeRo1985/pasdblstrutils ) for cross-compiler and cross-OS-target consistent and exact conversion between double-precision floating point number values and strings, PasDblStrUtils uses the newer more fast and more exact Grisu algorithm (paper: http://www.cs.tufts.edu/~nr/cs257/archi … printf.pdf ) than the string<->double conversion algorithms which Delphi and older FreePascal versions itself are using in their RTLs, which are mostly based on the older Dragon4 algorithm or something in that direction, although newer FreePascal versions seems using also Grisu in its RTL now. And at least the V8 and SpiderMonkey ECMAScript engines uses also the Grisu algorithm now. And it's since for quite a while now a tradition that double-number-string-conversion algorithms have names that are dragon related. The latest one is called "Errol" ( https://github.com/marcandrysco/Errol but quote:

Our original evaluation of Errol against the prior work of Grisu3 was erroneous. The evaluation indicates a 2x speed improvement over Grisu3. However, corrected performance measurements show a 2x speed loss to Grisu3.

, so Grisu, which I'm using it, is but still the fastest known double-string-conversion-algorithm )

ab · 2016-06-26 19:44:59

Yes, the algorithm is great, but using a lot of string concatenation to generate the string may not be so great.

NextGen support is something we did not do also for the main mORMot units - only a client has been developed for this target.
Anyway, when we see https://plus.google.com/105481197125997 … nLvMteQ4dU we can both be happy that FPC support was the right direction...

erayszn · 2016-06-27 11:37:46

mpv wrote:

It's released 2 month ago. This is usual situation for Mozilla to keep a release notes in a draft stage
We are working on migration - almost all is finished. Since API is incompatible with SM24, we decide to create a separate units SynSM45/SynSMAPI45.

When will SynSM45 units be relased? Will it support x64?

erayszn · 2016-06-27 12:40:23

I see debugger wrappers are exists in SynSMAPI.pas file but not implemented in TSMEngine class.
Do you have a plan adding debugger support in TSMEngine class or do have any example for using debugger methonds in SynSMAPI?

mpv · 2016-06-27 13:53:42

I hope we finish SynSM45 to the end of July. The changes between 24 and 45 version of SpiderMonkey API is huge (completely different API).

In the SM45 debugger is a pure JavaScript object, so we do not need any special Delphi classes. See Debugger-API

First release will be without x64 support. We have a long-term plans to support x64 in our main product and therefore in SpiderMonkey also. But not in this year.

mpv · 2016-07-23 14:48:39

We are near to finish a SynSM45. All important parts are implemented:
- a wrapper itself
- a creation of JavaScript object based on a Delphi RTTI (both new and old)
- a remote debugger!

Now we are working on code clean-up, documentation and sample
A small demo is on the YouTube (this is the first video in my life, sorry for quality)

edwinsn · 2016-07-23 15:15:03

@mpv, the new SynSM45 is impressive! Thanks for your effort.

Int · 2016-07-25 18:04:11

@mpv Looks exiting. I'm looking forward to see the release! Is it available already?

mpv · 2016-08-25 16:29:05

A beta version of SyNode - the server-side JavaScript execution using the SpiderMonkey 45 library with nodeJS modules support is committed to both fossil and github repos. I create a separate topic for this

ab · 2017-11-10 09:43:15

I've created a dedicated forum entry for SyNode.
See https://synopse.info/forum/viewforum.php?id=22
Don't hesitate to create multiple threads in this new forum category!

mORMot Open Source

#151 2016-06-25 12:31:11

Re: Adding JavaScript support for mORMot framework

#152 2016-06-26 01:08:33

Re: Adding JavaScript support for mORMot framework

#153 2016-06-26 06:54:16

Re: Adding JavaScript support for mORMot framework

#154 2016-06-26 07:30:59

Re: Adding JavaScript support for mORMot framework

#155 2016-06-26 10:14:45

Re: Adding JavaScript support for mORMot framework

#156 2016-06-26 12:41:30

Re: Adding JavaScript support for mORMot framework

#157 2016-06-26 19:44:59

Re: Adding JavaScript support for mORMot framework

#158 2016-06-27 11:37:46

Re: Adding JavaScript support for mORMot framework

#159 2016-06-27 12:40:23

Re: Adding JavaScript support for mORMot framework

#160 2016-06-27 13:53:42

Re: Adding JavaScript support for mORMot framework

#161 2016-07-23 14:48:39

Re: Adding JavaScript support for mORMot framework

#162 2016-07-23 15:15:03

Re: Adding JavaScript support for mORMot framework

#163 2016-07-25 18:04:11

Re: Adding JavaScript support for mORMot framework

#164 2016-08-25 16:29:05

Re: Adding JavaScript support for mORMot framework

#165 2017-11-10 09:43:15

Re: Adding JavaScript support for mORMot framework

Board footer