Beta release of SyNode - ES6 JavaScript + NPM modules

George · 2017-09-14 09:53:16

Thanks, i figured out

George · 2017-09-18 21:29:47

Cant find reference for method readDir(path, true) from fs.js.
Is it incomplete or defined somewhere in tricky way?
I get error readDir is not defined

function findAllCoreModules(path) {
    let result = new Array;
    let filesAndFolders = fs.readdirSync(path);
    for (let item of filesAndFolders) {
        result.push(item);
    }
    return result;
}

mpv · 2017-09-20 10:46:42

Should be fixed by this github commit 72627db

@ab - Arnaud, I'm commit directly to github, may be better to create a pull request? (sorry, but it's much simpler to me to use a git instead of fossil)

UPD. let result = new Array; is anti-pattern. Better to write let result = [];

UPD2 Array rooting added into fs_readDir to prevent access to a phantom memory in case GC executed during call to readDir function

Last edited by mpv (2017-09-20 12:27:13)

ab · 2017-09-20 15:36:01

Direct commit is ok if you warn me. :-)

ab · 2017-09-21 16:47:43

About the commit, I'm afraid my git did delete your commit!

George · 2017-09-22 11:44:01

@mpv, when commit will be restored, check this post.

Last edited by George (2017-09-22 12:00:41)

mpv · 2017-09-23 10:34:43

@ab - I'm commit to both fossil & git (I'm lost all my data during last not-Petya virus attack, so commit manually, without SourceCodeRep tool)

George · 2017-10-02 19:53:13

I have code sample that cannot be executed in SyNode, but run without any problems inside node.js or pure ES2015
Seems iterator object for "arguments" is missing.
Where that internal member initialize?)

"use strict";
function test(...args) {
    console.log(`param count is: ${arguments.length}`);
}
function testExecutor(p1, p2) {
    test(...arguments); // <<<<<<<< arguments is not iterable
}
testExecutor('param1', 'param2');

Last edited by George (2017-10-03 11:52:47)

mpv · 2017-10-03 15:43:20

We use a SpiderMonkey 52 in SyNode - may be in this version it's not implemented. Next stable SM version (59) will be released in 6 month. Mozilla mark as stable each 7th version and create version once a 3 month.

George · 2017-10-11 12:06:14

~~How to get access to TSMEngine.ThreadData from cx: PJSContext?~~
TSMEngine(cx.PrivateData).ThreadData

Last edited by George (2017-10-11 12:10:05)

hsvandrew · 2017-10-18 02:51:17

Can I suggest a few improvements to SyNodeNewProto

Firstly I think TVal2JSVal should be function TVal2JSVal(cx: PJSContext; const Value: TValue; aParentProto: TSMCustomProtoObject; propType: TRttiType ): jsval;
This allows us to correctly pickup a TDateTime property/method arg etc and process correctly as currently dates only work one way (write/inwards), not read on a property)
Secondly, I suggest any 'int64' support be removed as Javascript can't cope with 64bit integers and it will cause more problems than it solves allowing it. Its better to force users to create string based int64 properties/method args etc

mpv · 2017-10-18 12:00:20

In practice I'm not recommend to use a RTTI-based binding. Preferred way is to expose a minimal set of functions to a JS Engine via native bindings and implement all logic inside a JS, like we did in fs.js + SyNodeBinding_fs.pas.

As a workaround to deal with TDateTime there is jsval.getAsDate / jsval.setAsDate. I'm not understand how we can recognize a TDateTme, since where is no DateTime in TTypeKind?

We use Int64 (actually Int52) to interact with a database primary key values. String / GUID is overhead in our case, Int32 is small for us (we have ~10Tb DB). The only place where we limit a Int64 size is a database primary key generator (sequences, etc)

Last edited by mpv (2017-10-18 12:03:54)

George · 2017-10-19 16:16:27

Hello, MPV!

mpv wrote:

We use a SpiderMonkey 52 in SyNode

Do you have compiled files for version 52.4.1?

Last edited by George (2017-10-19 16:16:44)

mpv · 2017-10-19 16:24:24

No. The only version I use is the same as in downloads link. You can compile by yourself using this instruction (2-5 hours depending on your PC)

hsvandrew · 2017-10-21 03:11:40

mpv wrote:

In practice I'm not recommend to use a RTTI-based binding. Preferred way is to expose a minimal set of functions to a JS Engine via native bindings and implement all logic inside a JS, like we did in fs.js + SyNodeBinding_fs.pas.
As a workaround to deal with TDateTime there is jsval.getAsDate / jsval.setAsDate. I'm not understand how we can recognize a TDateTme, since where is no DateTime in TTypeKind?
We use Int64 (actually Int52) to interact with a database primary key values. String / GUID is overhead in our case, Int32 is small for us (we have ~10Tb DB). The only place where we limit a Int64 size is a database primary key generator (sequences, etc)

I think you misunderstand my point about int64. We like and use int64 for database primary keys as well. The problem is that javascript doesn't support int64 properly as a basic type and therefore making it look like there is support for it will confuse developers who aren't aware of this fact and place bugs in their code that will be very hard to detect until their database grows and all of a sudden really crazy things start to happen, which could even include security holes exposing themselves.

This is why its safer to not support int64

See https://developer.mozilla.org/en-US/doc … ence/Int64

-----

To detect TDateTime types

function TVal2JSVal(cx: PJSContext; const Value: TValue; aParentProto: TSMCustomProtoObject; propType: TRttiType ): jsval;
...
begin
  Result.setNull;
  if Value.IsEmpty then
    exit;

  if assigned( propType ) then
  begin
    if propType.ToString = 'TDateTime' then
    begin
      result.asDate[cx] := value.AsVariant;
      exit;
    end;
  end;

George · 2017-10-27 22:36:49

How to access to prototype field "fld1" from binded delphi method?

var TBaseTest = (function () {
    function TBaseTest() {
    }
    return TBaseTest;
}());
TBaseTest.prototype.fld1 = 'fff';

bindedFunc(TBaseTest);

I use GetPrototype method, but in result object can't access to fld1 property...

mpv · 2017-10-28 11:23:21

NewRTTI based prototypes now can handle TDateTime as proposed by @hsvandrew - see [37849aee2e]. Thanks for idea!

2 @hsvandrew - about int64 - I understand you point of view. But we can't remove int64 support for legacy reason.

Last edited by mpv (2017-10-28 11:25:41)

mpv · 2017-10-28 11:36:40

George wrote:

How to access to prototype field "fld1" from binded delphi method?
...

GetPrototype method work in the same way as Object.getPrototypeOf, you should pass instance to the binding function but you pass "class".
Try this (I remove closure for simplicity):

function TBaseTest() {  }
TBaseTest.prototype.fld1 = 'fff'
let inst = new TBaseTest()
console.log(Object.getPrototypeOf(inst)) // {fld1: "fff", constructor: ƒ}
bindedFunc(inst);

in Delphi GetPrototype will reture object with property fld1 = 'fff '

mpv · 2017-10-30 09:40:54

@AB - my colleague ssoftpro work on Linux support for SyNode and create several pull requests to mORMot GitHub repository. Current state - all work except debugger. If you don't mind I will review / test & merge his pull requests to the trunk (and push fossil also)?

Last edited by mpv (2017-10-30 09:41:32)

ab · 2017-10-30 20:18:32

Yes: but please try to keep the compatibility with Delphi 6 and up, and also with Kylix...

edwinsn · 2017-10-31 13:18:03

JS engine for Linux? Well done MPV!

mpv · 2017-11-03 16:07:21

SyNode for Linux (x64) merged into trunk. We add a compiled SpiderMonkey to mORMot git repository (using LSF extension) - see fpc-linux64 folder
Build instruction is also available

Also added a Hello World sample

Thanks to @ssoftpro for contribution!

ab · 2017-11-03 21:31:24

Thanks a lot!
Looking forward to the merged code.

The fact that it uses FPC for Linux / x64 compilation is just great.
Much better alternative than the Delphi compiler, for sure.

Now, there is no reason why SyNode may not be seen as an alternative to Node.
A ready-to-use package may be a good idea!

ab · 2017-11-06 19:56:11

Where do we get production ready libmozjs-52.so ?

The file included in https://media.githubusercontent.com/med … ozjs-52.so seems huge, with no symbol stripped.
I would like to include a smaller version in the sqlite3fpc.7z archive.

mpv · 2017-11-07 14:48:45

This is a production ready version. Version with debug information is about 220 Mb.. Resulting file is so huge because it contains a full ICU library resources inside. We will try to reduce size by removing some exotic languages (like nodejs did)

ab · 2017-11-07 15:11:49

It is just that GitHub is complaining about the LFS feature of the repository... and start asking for money, even for an Open Source project!

You’ve used 100% of your data plan for Git LFS on your personal account synopse. Please purchase additional data packs to cover your bandwidth and storage usage:
https://github.com/account/billing/data/upgrade
Current usage as of 07 Nov 2017 06:32PM UTC:
Bandwidth: 1.02 GB / 1 GB (102%)
Storage: 0.03 GB / 1 GB (3%)

I guess we should get rid of the .so in the github repository itself, and provide a separate .7z link, just like with Windows.

mpv · 2017-11-07 19:16:52

Ups. Sorry, I didn't know about this. I'm move so file to my resources and remove it from lfs

mpv · 2017-11-07 19:33:42

@AB. As noted in official documentation about LFS - the only way to completely remove file from remote storage is recreate the repository. This is very strange.... But I found the issue (see last 2 post). Can you as repository owner write to support and ask to remove the LFS file with ID = 301ed4e5be01b883ac17113797214906e1d9ccbb8f04508ba6d366c8d58d80cf (result of git lfs ls-files --long command)?

Once again, sorry for this.

UPD. libmozjs-52.so ( Linux x64 ) can be downloaded from: http://unitybase.info/media/files/libmozjs-52.zip. Link also added to SyNode.pas

Last edited by mpv (2017-11-07 19:48:02)

hnb · 2017-11-08 08:58:01

IMO mORMot for github should use "releases" (like fpcupdeluxe and newpascal does - https://github.com/newpascal/fpcupdeluxe/tags, https://github.com/newpascal/newpascal/releases ).

also is possible to use different categories for releases (by tags) for example sqlite_object_files, js_libraries.

Releases - very good way to keep all binary files around git repository:

https://help.github.com/articles/creating-releases/

ab · 2017-11-08 11:48:46

@mpv
I've removed the file: https://github.com/synopse/mORMot/commi … 54a7f9f96b
And asked GitHub support to delete the associate LFS resource.
Now seems OK.

@hnb
Releases do make sense, for sure.
We are stuck to 1.17 code base from years ago...

hnb · 2017-11-08 12:14:03

@ab github release don't need to be related to stable release. I was thinking about github releases as of kind of storage for our all binaries. For example fpcupdelux has following "binary" releases not related (at all) to fpcupdelux source code:

Libraries for cross-compiling v1.1 : https://github.com/newpascal/fpcupdelux … slibs_v1.1
Darwin i386 binary toolchains for cross-compiling: https://github.com/newpascal/fpcupdelux … sbins_v1.0
GIT for Windows: https://github.com/newpascal/fpcupdelux … Git-2.13.2
GDB 7.11.1 for Windows: https://github.com/newpascal/fpcupdelux … gdb-7.11.1

etc.

we could do the same thing for mORMot for related libraries/object files

ab · 2017-11-10 09:42:23

I've created a dedicated forum entry for SyNode.
See https://synopse.info/forum/viewforum.php?id=22

Don't hesitate to create multiple threads in this new forum category!

Linux64 compatibility is really exciting!
A true Node.js alternative!

mpv · 2017-11-10 10:58:22

@AB - may be you can move a related threads to a new entry? (as far as I remember this is possible in administrative area of phpBB, not shure about fluxBB)

George · 2017-11-13 20:52:55

Hello!

When delphi function returns js object (via vp: JSArgRec) with some fields, what is correct steps to be sure that JS garbage collector will not destroy object too early?
1. create local rooted object on delphi.
2. fill.
3. assign to vp.rval and call FreeRootedObject?

Or RootedObject should be used only for objects that was passed to delphi as parameters (to add reference and prevent object destruction while working with it)?

Last edited by George (2017-11-13 20:53:42)

mpv · 2017-11-14 10:47:43

For a long-live values the simplest way is to assign a returned value to global variable (or closure variable) in the JS. RootedObject should be used mostly inside Delphi function to protect a value from GC due to function execution time.

George · 2017-11-14 20:01:26

About long-live objects i understand.
I'm going to create a new empty object from delphi side, add some fields (or instantiate object using known js class), fill data and then return it as result.
From js side, code may looks like listed below:

function exec() {
    // local context variable that store result until end of method execution
    let result = process.binding('delphi').getSomeObject();
    // do something
}

I found sample code in SyNodeBinding_fs.
In fs_fileStat method, steps are exact same as i described in previous post)

hsvandrew · 2017-12-22 23:53:58

mpv wrote:

SyNode for Linux (x64) merged into trunk. We add a compiled SpiderMonkey to mORMot git repository (using LSF extension) - see fpc-linux64 folder
Build instruction is also available
Also added a Hello World sample
Thanks to @ssoftpro for contribution!

Hi mpv

Do you plan on providing support for both FPC and the Delphi Linux compiler or just FPC?

I am trying to work out whether to buy the latest Delphi and some comments here suggest FPC is better than the Delphi Linux compiler but I'm having trouble finding any sources that tell me why.

Compiling Synode in Linux is very important to us and there is no point spending $4600AUD on Delphi if SyNode will only ever be for FPC

ab · 2017-12-23 09:21:08

We would support Delphi Linux for the cross-platform client units.

But we don't have any plan for the main mORMot units to support Delphi Linux.
We supported FPC Linux since years, and use this as main target now.

I don't like how the Delphi Linux "nextgen" compiler uses ARC, which is a PITA to maintain a shared code base with other compilers.
There is no benefit in respect to FPC: FPC compiles faster, generates better code (Delphi LLVM backend is not tuned at all), has a Delphi syntax, is free and Open Source, and supports much more targets, we already support: Linux x86, Arm32 and Arm64, BSD, OsX/Darwin, and soon PPC (most targets thanks to invaluable Alf's help)...

mpv · 2017-12-23 11:09:24

About SyNode - I try to switch all my projects to FPC, even for Windows target. Since we have many of projects in production what use XE2(Windows) + SyNode, we will continue to support Delphi/Windows during 1-2 year (while migrating on Linux/FPC). Delphi Linux compiler not tested at all (in case of SyNode) because we don't have it on hand (we have XE2/7 Delphi), and there is no plans to support it. In any case it's strange IMHO to provide a open source solution what require a commercial (expensive) compiler..

George · 2018-02-21 14:14:29

Hello!
readDir returns folders without trailing delimiter (\).
How correctly evaluate folders?

filesAndFolders = readDir('path', true); // get files and folders
And check every item using directoryExists?

Last edited by George (2018-02-21 14:40:28)

mpv · 2018-02-21 19:13:06

If you use SyNode(SM52) I strongly recommend to use fs + path modules(it's compatible with nodejs, so see docs on node web site) instead of direct call to readDir. See how we use it for wolking on folder tree.
In brunch fb_SyNode_fpc311 we make this modules cross platform so your JS code will work on win/linux without any modification (will be merged to trunk soon)

Last edited by mpv (2018-02-21 19:13:55)

George · 2018-02-22 20:04:38

I see, thanks!

mapes · 2018-04-22 09:39:21

Hi,

under linux fs.readdirSync is not reading Directories.

in fs_readDir

{$IFDEF MSWINDOWS}
if FindFirst(Dir + '*.*', searchAttr, F) = 0 then begin
{$ELSE}
if FindFirst(Dir + '*', searchAttr, F) = 0 then begin
{$ENDIF}

or FindFirst (Dir + '*',searchAttr,F) for both OS solves the problem.

Last edited by mapes (2018-04-22 09:41:27)

mpv · 2018-04-22 12:07:26

fixed by removing .*
Thanks!

Also @ssoftpro near to finish a cross-platform file descriptors for fs - will be merged soon

guxinglei · 2018-04-23 07:32:31

How to import jquery?

ab · 2018-04-23 07:53:21

JQuery on server side?
Weird idea - there is no HTML DOM with SyNode (as with node.js).
One can be mocked by tools such as jsdom. This can be useful for testing purposes.

guxinglei · 2018-04-23 08:40:18

ab wrote:

JQuery on server side?
Weird idea - there is no HTML DOM with SyNode (as with node.js).
One can be mocked by tools such as jsdom. This can be useful for testing purposes.

npm install jquery
npm install jsdom

Grab some website data, but some JS encryption functions are frequently modified and dynamic JS encryption is used.

example：
function getKey(a,page)
{
c2 = $.fun1($.md5(a));
e2 = eval(page);
return e2.toUppercase();
}
page = '$.func2($.md5(c2))';
pubKey = getKey('123456',page)

Last edited by guxinglei (2018-04-23 10:17:48)

mpv · 2018-04-23 09:30:37

For cryptographic operations we use either CryptoJS in case speed is not required, or expose functions from SynCrypto to JS (SynCrypto is very fast). See for example how we define sha256 - can be added to global by

aEngine.GlobalObject.ptr.DefineFunction(cx, 'nsha256', nsm_sha256, 1, JSPROP_ENUMERATE or JSPROP_READONLY);

inside DoOnCreateNewEngine handler

guxinglei · 2018-04-24 06:10:09

ChakraCore GitHub repository is now open

https://github.com/hsreina/ChakraCore-Delphi
https://github.com/hsreina/Chakra-Samples-Delphi

https://github.com/Microsoft/ChakraCore

edwinsn · 2018-04-24 07:12:32

@guxinglei,

This Delphi wrapper for ChakraCore is much more updated and complete: https://github.com/tondrej/chakracore-delphi

mORMot Open Source

#101 2017-09-14 09:53:16

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#102 2017-09-18 21:29:47

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#103 2017-09-20 10:46:42

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#104 2017-09-20 15:36:01

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#105 2017-09-21 16:47:43

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#106 2017-09-22 11:44:01

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#107 2017-09-23 10:34:43

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#108 2017-10-02 19:53:13

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#109 2017-10-03 15:43:20

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#110 2017-10-11 12:06:14

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#111 2017-10-18 02:51:17

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#112 2017-10-18 12:00:20

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#113 2017-10-19 16:16:27

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#114 2017-10-19 16:24:24

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#115 2017-10-21 03:11:40

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#116 2017-10-27 22:36:49

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#117 2017-10-28 11:23:21

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#118 2017-10-28 11:36:40

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#119 2017-10-30 09:40:54

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#120 2017-10-30 20:18:32

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#121 2017-10-31 13:18:03

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#122 2017-11-03 16:07:21

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#123 2017-11-03 21:31:24

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#124 2017-11-06 19:56:11

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#125 2017-11-07 14:48:45

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#126 2017-11-07 15:11:49

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#127 2017-11-07 19:16:52

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#128 2017-11-07 19:33:42

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#129 2017-11-08 08:58:01

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#130 2017-11-08 11:48:46

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#131 2017-11-08 12:14:03

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#132 2017-11-10 09:42:23

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#133 2017-11-10 10:58:22

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#134 2017-11-13 20:52:55

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#135 2017-11-14 10:47:43

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#136 2017-11-14 20:01:26

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#137 2017-12-22 23:53:58

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#138 2017-12-23 09:21:08

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#139 2017-12-23 11:09:24

Re: Beta release of SyNode - ES6 JavaScript + NPM modules

#140 2018-02-21 14:14:29