You are not logged in.
- Topics: Active | Unanswered
Pages: 1
#1 Re: mORMot 1 » Fast MM5 » 2020-06-22 23:19:15
On Win64 it just calls VirtualAlloc for the large blocks so if you don't reallocate them it relies on the OS.
Very interesting project your DAW!
Thanks.
Another fact:
My DAW allocates (and releases these shortly afterwards again) continuously new tiny and small memory blocks for the MIDI events and automation events in the PasMP-based playback threadsafe lockless queues under real-time audio conditions. This can be even several thousands per far below one millisecond, depending on the type of song. And so far SynFPCx64MM performs rather pretty well regarding this aspect, because otherwise, memory allocations within real-time DSP audio routines are actually a total no-go and taboo in the audio software industry, because one has no predictable duration of the operation for an memory allocation, which is rather a major disadvantage in real-time applications. But SynFPCx64MM handles it pretty good anyway, without PCM audio buffer underruns in the end, at least with tiny and small memory blocks.
#2 Re: mORMot 1 » Fast MM5 » 2020-06-22 11:41:18
And by the way, I once wrote a GPU memory manager, because with the very explicit Vulkan API one has to manage the GPU vRAM almost completely by oneself.
See:
https://github.com/BeRo1985/pasvulkan/b … k.pas#L656 and
https://github.com/BeRo1985/pasvulkan/b … .pas#L8238 and
https://youtu.be/EKcwo_mrBDM?t=395
for more details.
#3 Re: mORMot 1 » Fast MM5 » 2020-06-22 11:33:41
Thanks a lot BeRo for the feedback!
It is very difficult to have a correct memory manager, this is why we started from FastMM4 which had a very proven code base.Did you use Windows for your tests?
I guess the advantage of synFpcx64mm for large block is that it just wraps map/remap on Posix or VirtualAlloc on Windows.I am happy to have been helpful to you.
Do you have any information to publish about your upcoming Digital Audio Workstation project?
Yes, it's Win64, because of the VST plugin support.
On Youtube I've some videos of now some weeks/months older versions of it:
Mini-DAW-Project - Safri Duo - Played a live - BeRo Remake - Draft 2
Mini-DAW - PasMP Profiler integrated for to show how strong parallized the whole audio engine is
[Mini-DAW] Scorpions - Wind Of Change (with synthesized vocals) (together with my own pascal-native singing-able diphone-based speech synthesizer with the PSOLA-style "Multi-Band Resynthesis OverLap Add" algorithm)
Mini-DAW - Slavko Avsenik - Auf der Autobahn - Techno Remix with extra much BASS
It's fully pascal-native (with some inline assembler parts) and fully multithreaded and work-stealing-fork-join-style parallelized with the help of my PasMP project.
It has a VST 2.x and VST 3.x plugin host. The funny thing is that the VST 3.x plugin spec ABI/API is actually hardcoded against the MSVC C++ classes VMT data structure respectively ABI, wrapped into Fake-COM-interface-constructs (Steinberg is calling their C++ ABI Raw-MSVC-Class-VMT Fake-COM "VST-MA"). It took some time before my VST 3.x host code worked correctly with dirty hacks for the ABI-wrapping with pure-pascal-code, where most all VST3 plugins are working now, except these of izotope and few others, where I'm guessing that any other C++ ABI problems are occuring at these.
It has integrated also my own Sobanth Soundbank engine.
The most of the GUI is rendered over the GPU with OpenGL 4.3 or Direct2D/DirectWrite as compile-time-option, and with GDI as fallback option. I've hijacked the canvas stuff in the Lazarus LCL for this purpose, where most all GDI operations are redirected and emulated with OpenGL 4.3 or Direct2D/DirectWrite (with DirectX 12 feature level) then. The OpenGL render part uses shader-based 2D Signed-Distance-Fields for its rendering concept, especially for the antialiased text rendering.
I could write more, but I don't have the time right now. It is or it will be a a full DAW.
#4 Re: mORMot 1 » Fast MM5 » 2020-06-21 21:28:17
A status update from my side:
The latest SynFPCx64MM version now works with my Digital Audio Workstation project without access violations at Application.Initialize() .
As a result it is now possible to load multiple large multiple gigabytes big Soundfont 2.x soundbanks in my Digital Audio Workstation without corrupting the internal heap data structures when using SynFPCx64MM, but which happened before with FPC's own default memory manager after a longer runtime, where the internal heap data structures at the Lazarus LCL message queue stuff were often corrupted.
So somewhere in FPC own memory-manager seems to be a bug concerning these heap data structure corruptions at multiple large memory allocations after some runtime, but I haven't been able to find out which code location of FPC own memory manager is guilty for this so far yet.
Hence my warning: Avoid using FPC's own memory manager if one has to perform multiple several large memory allocations (>=512MB).
At least I can freeze my work on my own memory manager, where SynFPCx64MM now works for me, and my Digital Audio Workstation project is x86-64-only anyway. Thanks a lot.
#5 Re: mORMot 1 » Fast MM5 » 2020-05-14 04:36:29
@bero
We validated here with FPC 3.2 fixes, not 3.3.1/trunk.Also ensure you have the latest revision.
I fixed some problems specific to Win64.
The most current mormot.core.fpcx64mm.pas crashes still at me with Free Pascal 3.3.1 SVN revision 45361 and Lazarus 2.0.9 fixes branch SVN revision 63147 (with my small patch http://rootserver.rosseaux.net/stuff/la … .3.1.patch for to compile the stable Lazarus 2.0.9 fixes branch with fpc 3.3.1 SVN trunk), both updated and fresh compiled on 14.05.2020. And so far I can see, it crashes always somewhere at image-resource-loading (icon, bitmap, etc.) stuff at Application.Initialize for the app icon, or later at Application.CreateForm at TImage's TImageList's and so on.
I think, I should resume my work on my own lock-free memory manager, which is based on the core concepts of https://github.com/mjansson/rpmalloc.
#6 Re: mORMot 1 » Fast MM5 » 2020-05-09 18:49:47
Hm, it crashes om me at Applicatiion.Initialize unter Windows. Here is the stack trace:
Here the stack trace:
#0 _FREEMEM(0x0) at mormot.core.fpcx64mm.pas:1463
#1 ?? at :0
#2 ?? at :0
#3 SYSTEM_$$_FREEMEM$POINTER$$QWORD at :0
#4 FREEDATA({DESCRIPTION = {FORMAT = RICFGRAY, WIDTH = 310, HEIGHT = 310, DEPTH = 1, BITORDER = RIBOBITSINORDER, BYTEORDER = RIBOLSBFIRST, LINEORDER = RILOTOPTOBOTTOM, LINEEND = RILETIGHT, BITSPERPIXEL = 1, REDPREC = 1, REDSHIFT = 0, GREENPREC = 0, GREENSHIFT = 0, BLUEPREC = 0, BLUESHIFT = 0, ALPHAPREC = 0, ALPHASHIFT = 0, MASKBITSPERPIXEL = 0, MASKSHIFT = 0, MASKLINEEND = RILETIGHT, MASKBITORDER = RIBOBITSINORDER, PALETTECOLORCOUNT = 0, PALETTEBITSPERINDEX = 0, PALETTESHIFT = 0, PALETTELINEEND = RILETIGHT, PALETTEBITORDER = RIBOBITSINORDER, PALETTEBYTEORDER = RIBOLSBFIRST}, DATA = 0x0, DATASIZE = 0, MASK = 0x0, MASKSIZE = 0, PALETTE = 0x0, PALETTESIZE = 0}) at graphtype.pp:1310
#5 MASKHANDLENEEDED(0x61a00d0) at include\icon.inc:753
#6 GETMASKHANDLE(0x61a00d0) at include\icon.inc:576
#7 HANDLENEEDED(0x61a00d0) at include\icon.inc:1390
#8 RELEASEHANDLE(0x61a00d0) at include\icon.inc:1363
#9 BIGICONHANDLE(0x66927f0) at include\application.inc:1158
#10 ICONCHANGED(0x66927f0, 0x61a00d0) at include\application.inc:1115
#11 CHANGED(0x61a00d0, 0x61a00d0) at include\graphic.inc:56
#12 CHANGED(0x61a00d0, 0x61a00d0) at include\rasterimage.inc:395
#13 ENDUPDATE(0x61a00d0, true) at include\rasterimage.inc:271
#14 LOADFROMSTREAM(0x61a00d0, 0x4690910, 35898) at include\rasterimage.inc:457
#15 LOADFROMSTREAM(0x61a00d0, 0x4690910) at include\rasterimage.inc:417
#16 READDATA(0x61a00d0, 0x4690910) at include\icon.inc:801
#17 LOADFROMRESOURCEHANDLE(0x61a00d0, 4294967296, 4303999108) at include\icon.inc:1440
#18 INITIALIZE(0x66927f0) at include\application.inc:459
#19 main at BeRoSequencer.lpr:189
FPC Version is SVN trunk from 03.03.2020 and Lazarus version is the 2.0.7 patches SVN branch also from 03.03.2020 with some small fpc-syntax-change-bugfix changes of myself, so that it compiles with FPC 3.3.1 without any compilation errors.
#7 Re: SyNode » Adding JavaScript support for mORMot framework » 2016-06-26 12:41:30
BESEN 2.x uses PUCU ( https://github.com/BeRo1985/pucu ) for cross-compiler and cross-OS-target unicode support, where PUCU defines:
{$ifdef fpc}
...
{$if declared(RawByteString)}
{$define HAS_TYPE_RAWBYTESTRING}
{$ifend}
{$if declared(UTF8String)}
{$define HAS_TYPE_UTF8STRING}
{$ifend}
{$if declared(UnicodeString)}
{$define HAS_TYPE_UNICODESTRING}
{$ifend}
...
{$else}
...
{$ifdef conditionalexpressions}
....
{$if declared(RawByteString)}
{$define HAS_TYPE_RAWBYTESTRING}
{$ifend}
{$if declared(UTF8String)}
{$define HAS_TYPE_UTF8STRING}
{$ifend}
{$if declared(UnicodeString)}
{$define HAS_TYPE_UNICODESTRING}
{$ifend}
{$endif}
...
{$endif}
....
PPUCURawByteChar=PAnsiChar;
TPUCURawByteChar=AnsiChar;
PPUCURawByteCharSet=^TPUCURawByteCharSet;
TPUCURawByteCharSet=set of TPUCURawByteChar;
PPUCURawByteString=^TPUCURawByteString;
TPUCURawByteString={$ifdef HAS_TYPE_RAWBYTESTRING}RawByteString{$else}AnsiString{$endif};
PPUCUUTF8Char=PAnsiChar;
TPUCUUTF8Char=AnsiChar;
PPUCUUTF8String=^TPUCUUTF8String;
TPUCUUTF8String={$ifdef HAS_TYPE_UTF8STRING}UTF8String{$else}AnsiString{$endif};
PPUCUUTF16Char={$ifdef HAS_TYPE_UNICODESTRING}{$ifdef fpc}PUnicodeChar{$else}PWideChar{$endif}{$else}PWideChar{$endif};
TPUCUUTF16Char={$ifdef HAS_TYPE_UNICODESTRING}{$ifdef fpc}UnicodeChar{$else}WideChar{$endif}{$else}WideChar{$endif};
PPUCUUTF16String=^TPUCUUTF16String;
TPUCUUTF16String={$ifdef HAS_TYPE_UNICODESTRING}UnicodeString{$else}WideString{$endif};so that I've a cross-compiler (including the old non-so-true-unicode-aware Delphi 7) and cross-OS-target consistent Unicode support for my projects including BESEN 2.0, except at/for the NextGen Delphi Compiler targets, which I'm ignoring it simply now once, until somebody needs really really ... really NextGen Delphi Compiler support at BESEN 2.0 (but only against reward then, because I would not do it otherwise voluntarily 
), because they are too codebreaking for me degrading the string type stuff, ARC, and so on.
At least my PUCU side-project has for me all important i18n stuff for my projects, including fast DFA-machine-based UTF8 sequence decoding, cross-compiler and cross-OS-target consistent code-page support and so on, for example for emulating Windows's MultibyteToWideChar and WideCharToMultibyte on non-Windows targets and on older non-unicode-aware compilers.
And the same applies to the string<->double-number-value conversion stuff, where BESEN 2.0 uses PasDblStrUtils ( https://github.com/BeRo1985/pasdblstrutils ) for cross-compiler and cross-OS-target consistent and exact conversion between double-precision floating point number values and strings, PasDblStrUtils uses the newer more fast and more exact Grisu algorithm (paper: http://www.cs.tufts.edu/~nr/cs257/archi … printf.pdf ) than the string<->double conversion algorithms which Delphi and older FreePascal versions itself are using in their RTLs, which are mostly based on the older Dragon4 algorithm or something in that direction, although newer FreePascal versions seems using also Grisu in its RTL now. And at least the V8 and SpiderMonkey ECMAScript engines uses also the Grisu algorithm now. And it's since for quite a while now a tradition that double-number-string-conversion algorithms have names that are dragon related. The latest one is called "Errol" ( https://github.com/marcandrysco/Errol but quote:
Our original evaluation of Errol against the prior work of Grisu3 was erroneous. The evaluation indicates a 2x speed improvement over Grisu3. However, corrected performance measurements show a 2x speed loss to Grisu3.
, so Grisu, which I'm using it, is but still the fastest known double-string-conversion-algorithm ) 
#8 Re: SyNode » Adding JavaScript support for mORMot framework » 2016-06-26 01:08:33
Other small status update:
I've noticed at my BESEN 2.0 parsing benchmarks with the ten megabytes big Octane javascript benchmark sources, that the WideString-based Delphi 7 BESEN 2.0 build is much slower (by many huge performance worlds apart, no matter with which memory manager (FastMM4, etc.) and with which third-party performance-optimized system unit for Delphi 7) than the UnicodeString-based Delphi XE7 and FreePascal builds.
But this is no surprise, because the WideString type is managed mostly under Windows-targets by the Windows-API as "COM BSTR" strings and the WideString type is moreover also not reference-counted, while the UnicodeString type is managed by the RTL itself and which is also reference-counted.
Why am I saying this: Don't expect with BESEN 2.0 under Delphi 7 the same good performance from newer Delphi versions und FreePascal then, so that no one complained at the end, that the overall BESEN 2.0 performance with Delphi 7 is not so good like as with newer Delphi versions or with FreePascal.
And BESEN 1.x was and is mostly only WideString-based, so that this is a another performance issue cause at the old (but now yet current stable) BESEN 1.x version.
#9 Re: SyNode » Adding JavaScript support for mORMot framework » 2016-06-25 12:31:11
Well done BeRo1985! I wish you'll support FPC as well
Hm, all my stuff are beginning after the program/uses clause with "{$ifdef fpc}{$mode delphi} . . . {$endif}", so FPC support is already there, even in BESEN 1.x since it did exist . I'm developing my stuff almost always for FPC (for the then each most currently released stable version of FPC) and Delphi 7-XE7, except for example my SupraEngine Project now, which is FPC/Lazarus-only, since the SupraEngine Editor itself depends on the FPC compiler for the game-logic-code stuff.
You can convince yourself here at https://github.com/BeRo1985?tab=repositories there are BESEN PasMP PUCU FLRE PasVulkan Kraft PasENet CrossPascal BeRoTinyPascal and so on buildable with FreePascal and Delphi >= 7.
And I'm also co-author of CrossFPC http://www.crossfpc.com/ which integrates the FPC compiler into the Delphi IDE, and that is often helpful, because I can with this writting my code in Lazarus but then debugging my code with the Delphi compiler in the Delphi IDE and building my release binaries with FPC also directly into the Delphi IDE (or into the Lazarus IDE, as I like it on the respective day). I'm doing this so often in this way, because GDB-based Debugging under Lazarus under Windows (but not under Linux, *BSD or MacOSX, where it is at my computers then super smooth without any debugger-action-lags) is super damn slow, at least where a single instruction line code step action needs circa one to two seconds on my computers with mostly-yet-current fast hardware, incl. a Ivybridge-era Xeon Workstation with 32GB ECC RAM, and also because I can ensure it even with this Most-Always-Right-From-The-Prioject-Start-With-Two-Compiler-Development approach that my code is compatible with FPC and Delphi right from a project start.
The same applies more or less also to the being-Operating-System-and-CPU-target-cross-platform-topic for me, where I'm focusing most always Windows, Linux, Android and MacOS(X), except for OpenGL-based projects, because the OpenGL support is under Apple products a hell, for example MacOS(X) has only support for maximal OpenGL 4.1 (but current is 4.5) and the OpenGL drivers under MacOS(X) have very many bugs and are often much slower than the exactly same Apple hardware with OpenGL with Linux or Windows, so I'm dislike also the Metal stuff at Apple, because it is only ignoring on the part of Apple, that their OpenGL support is just super bad, and not OpenGL itself. And unfortunately Apple defends also against Vulkan support in MacOS(X), so my project MacOS(X)-supporting era will be maybe potentially over for ever now, because I don't want work with ignorance system-ecosystems (and this is sadly the case at Apple).
#10 Re: SyNode » Adding JavaScript support for mORMot framework » 2016-06-25 03:09:34
Status upgrade:
The now mostly-written-from-scratch rewrite of BESEN (called BESEN 2.0) have already 13874 code lines now, where the following code parts are already done and working:
The new garbage collector (ported over from my BESEN-sister project POCA)
The new ECMAScript 5.1 compliant lexer and parser, which's also faster than the old one in BESEN 1.x now (parses the whole set of the megabyte-large typescript js files also really fast) including a new and more compact (more memory-usage-friendly) abstract-syntax-tree data representation
The new high level abstract-syntax-tree optimizer
The new abstract-syntax-tree-back-to-ECMAScript-code-dumper (needed for the non-official Function.prototype.toString() stuff, which a lot of another ECMAScript does support also it, even although Function.prototype.toString() is not in the official ECMAScript standard)
UTF-16 anywhere as much as possible (because the ECMAScript standard specification is very very very UTF-16 centric for the internal string encoding stuff), even although I'm preferring UTF-8 personally more (like at POCA), but sadly specification is specification.
I've tested the current state with JQuery 1.12.4, JQuery 2.2.4, JQuery 3.0.0 and TypeScript 1.8.10, without any parsing or high-level-AST-optimizer errors.
The next steps will be:
The new byte code generator and the new again register-based virtual machine
The new native code just-in-time compiler (which'll be then with a more generally (very simplified) LLVM-like concept)
The new standard ECMAScript host object implementations
and then incrementally update the stuff from ECMAScript 5.1 to ECMAScript 2015 and ECMAScript 2016.
and then put BESEN 2.0 onto GitHub.
#11 Re: SyNode » Adding JavaScript support for mORMot framework » 2016-06-18 17:48:35
My current "lobby goal" is to convince BaRo for using mORMot for presented Unity-style engine
Erm ... a Entity-Component-System-implementation in a modern game engine is "just" like a key-value database, but really only just like and not really exactly like, because real-time-game-oriented Entity-Component-Systems do need heavily-cpu-cache-optimized memory-contiguous Array-Of-Structs or Struct-of-Arrays (depends on the usage) implementations including (if possible) fast O(1) entity<->component vice-versa-lookups (without hash tables, hash maps, self-balanced search trees, etc.) for a smooth game experience without unnecessary frame-drops etc., how I've implemented it also in my SupraEngine, see:
http://t-machine.org/index.php/2014/03/ … us-memory/
http://gamedev.stackexchange.com/questi … ame-engine
http://gamedev.stackexchange.com/questi … ty-systems
http://www.randygaul.net/2014/06/10/san … y-systems/
http://www.randygaul.net/2013/05/05/dat … arting-up/
http://entity-systems.wikidot.com/
and of course also https://gist.github.com/paniq/c32ac8447a7cc5c33a45 of my Farbrausch demogroup colleague paniq
#12 Re: SyNode » Adding JavaScript support for mORMot framework » 2016-06-18 15:59:03
And for those interested in it: here are some informations, resources and benchmark results of my BESEN-sister research-project called POCA, because BESEN was/is too slow for my Object-Pascal-based Unity-style GameEngine called SupraEngine, which I'm developing also now:
http://vserver.rosseaux.net/stuff/pocastuff/doc.md.html (very incomplete documentation, it documents circa. only 25% of the syntax features of POCA)
http://vserver.rosseaux.net/stuff/pocastuff/ (pocarun.exe binary and some *.poca examples)
https://gist.github.com/BeRo1985/0a87de … e72adb10fa (Prime Search Benchmark against Lua 5.3)
POCA has just nulls, numbers, strings, arrays, objects, ghosts and functions as data types, but no extra boolean or undefined data types, what had simplified the POCA VM design a lot in contrast to the BESEN VM design.
POCA has already all that, what BESEN 2.0 should become then. NaN-Tagging/NaN-Boxing, a better code generator, a better Incremental Generational Mark&Sweep Garbage Collector, and a better virtual-machine design with a better performance, better inline-caching-design-concept and better native code Just-In-Time Compiler, and less memory-usage-bloating-issues than BESEN.
And POCA's syntax is ECMAScript-style but a lot with syntax idea additions from other script languages, for example operator overloading, <=> spaceship operator, C#-style safe navigation operators, TypeScript-style / Squirrel-style classes, Java-style super keyword, and so on.
The POCA Code is more low level Non-OOP Pascal-based (therefore also the overall better execution performance than BESEN), but with some Object-Pascal constructs, for example to have a TBESENNativeObject-style construct in POCA called TPOCANativeObject, so that here is no
OOP-class-object-binding-convenience-disadvantage over to BESEN.
Source code on request (which's smaller than the source code from BESEN, even with the fact than POCA has more features than BESEN), since it's a research project, and not a real-usage project (at least yet). POCA is indeed already nice and even stable usable, but POCA is a own non-standardized and non-tested language, so I would not recommend it to use it, simply because it's a non-standardized language, even if it's ECMAScript-like, unless you convinced me to release it POCA as a stable open source product, which I'll maintaining further then.
And at least as already said I'll try now to get BESEN to the same performance level like my research project POCA, if necessary also with a mostly-from-scratch rewrite of BESEN, then with less design flaws and with POCA-style code structure.
I just wanted to have POCA, SupraEngine and Supraleiter mentioned here, so that you know my motives, why I want to rework BESEN now.
And here some videos to my SupraEngine stuff:
https://www.youtube.com/watch?v=L_2olISxXWw
https://www.youtube.com/watch?v=YB5mNY0DHQc
https://www.youtube.com/watch?v=gRvL2KjMdUU
And here some in-game Supraleiter videos with some older versions of my SupraEngine before the Vulkan-based rewrite now:
https://www.youtube.com/watch?v=_SVL0_Z3WL8
https://www.youtube.com/watch?v=SWanRYNNTUA
https://www.youtube.com/watch?v=dR8OJoHWuGs
#13 Re: SyNode » Adding JavaScript support for mORMot framework » 2016-06-18 07:53:15
V8 is using SMI, Shifted-Minimum-Integer as Tagging-Concept (see http://thlorenz.com/talks/demystifying-v8/talk.pdf ), and SpiderMonkey (where it is called NuNBoxing and PuNBoxing, see https://github.com/mozilla/gecko-dev/bl … a/jsval.py and https://github.com/mozilla/gecko-dev/bl … ic/Value.h ) is internally using also NaN-Boxing/NaN-Tagging like LuaJIT (and BESEN in future and my BESEN-sister project POCA already since 2011), even all another state-of-the-art script-VM-designs (not only at ECMAScript) are using often the NaN-Boxed/NaN-Tagging base concept, at least at the internal execution layer.
If TBESENValue is just a 64-bit double, it has following advantages:
The automatic record initialization/finalization stuff will not more needed, but at the explict cases, if the TBESENValue contains a Object or a String over the NaN-boxing mechanism.
The most of BESEN values are just always 64-bit big => more CPU cache friendly
The native code Just-In-Compiler stuff can be also more simple then
Numeric arithmetics just works on these plain raw Non-Signaling-NaN double-precision float values, with fewer code overhead than now.
Something like BESENValueType(AValue) instead AValue.ValueType (like now yet) has also the advantage, that further future underlying TBESENValue data structure changes will be easier than now. It's indeed not so optimal for older ObjectPascal compilers, which don't supporting function inlining yet, but for more current ObjectPascal compilers with function inlining, it is a advantage for to decoupling the TBESENValue data structure internal field access from the BESEN API for the "end-user-programmer" for further future underlying data structure changes, without affecting the then new BESEN 2.0 based application code so much.
And I think, BESEN must be also still compatible with Delphi 7 and Kylix 3, because BESEN is already used by some companies with these old compiler-ecosystems for their products.
And support for TypeScript should be no problem then, when I've finished the first BESEN 2.0 stable release then.
#14 Re: SyNode » Adding JavaScript support for mORMot framework » 2016-06-17 19:16:21
@BeRo, that would be great!
Would the project backed by a complete test suite for the js langauge? I think that's crucial for quality control
BESEN passed already the offical ECMAScript 5th edition test suite since 2010, see https://youtu.be/w6sRxB4iDpk?t=1805 from the BESEN tech talk from the Revision 2011 demoparty from the year 2011.
And for be clarified: BESEN is primary a ECMAScript engine and less a JavaScript engine. Because ECMAScript isn't JavaScript, but JavaScript is in turn ECMAScript with possible some implementation-dependent extensions. So BESEN focuses on the ECMAScript conformity (where BESEN did passing the offical TC39 es5-test suite already since end 2010, see https://youtu.be/w6sRxB4iDpk?t=1805 ), less on the JavaScript conformity to a another ECMAScript implementation in its JavaScript ECMAScript dialect mode, because JavaScript conformity doesn't exist, because only the ECMAScript language core of JavaScript is specified, but JavaScript with its umpteen Implementation variants not. And for example ActionScript from Adobe Flash is also ECMAScript with Adobe-own extensions, but ActionScript is nevertheless mostly incompatible to the defacto lowest common denominator web JavaScript dialect variant, and the same does apply to JScript and so on. For more, see https://en.wikipedia.org/wiki/ECMAScript .
TLDR: JavaScript != JavaScript and ECMAScript != JavaScript, but JavaScript == ECMAScript + possible implementor-own extensions
So just download the offical ECMAScript 5th edition test suite (for example from https://github.com/tc39/test262/tree/es5-tests ), which BESEN passes already since it's public released (at least in the version state of BESEN, which I've used at the techtalk at the Revision 2011 demoparty) .
#15 Re: SyNode » Adding JavaScript support for mORMot framework » 2016-06-17 13:31:42
My NaN-Tagging / NaN-boxing throughts so far:
Layout of a IEEE 754 double-precision binary floating-point format:
1 bit sign
11 bit exponent
52 bit mantissaNaN => all exponent bits set and mantissa != 0, where the last bit of the mantissa sets, if it is a quiet or signalling NaN
Inf => all exponent bits set and mantissa == 0For NaN tagging/boxing, BESEN will using 51 bits of the 52 bits of the mantissa and the 1 bit sign for to have 52 bits again for the payload
Linux on x86_64, Win64, Solaris and Irix have following policies with the address spaces:
https://msdn.microsoft.com/en-us/librar … s.85).aspx
https://docs.oracle.com/cd/E19127-01/ul … 892-12.pdf
http://menehune.opt.wfu.edu/Kokua/More_ … /ch01.html
https://web.archive.org/web/20090405235 … /ch01.htmlIn short as summary:
- Microsoft guarantees 44 bits of process address space, 8 terabytes (7 terabytes on Itanium-based systems)
- SGI guarantees 40 bits of process address space
- Sun/Oracle guarantees 43 bits of process address space (in Solaris the stack points into the negative address space at $ffff..., but I don't care as BESEN values will never point there)
- Linux doesn't document this rigorously, but testing shows that it allows 47 bits of address space (and current x86_64 implementations are limited to 48 bits of virtual space anyway)So I'll choose 48 bits as the conservative compromise for our address space room for the BESEN values, where normal numeric values are just normal doubles, and otherwise all other things NaN-boxed/NaN-tagged pointers inside a Signalling NaN double value.
And Signalling NaN that can't itself be produced by normal numerics code, and NaN-boxed pointers in BESEN values will be guaranteed that all memory that can be pointed to by a memory position lives in the bottom 48 bits of memory.
So I've then 4 bit (52bits - 48bits = 4bits) for the value type of 16 possible value types (string, object, etc.).
Or here a possible alternative approach:
I'll use the whole 51-bit (without the 1-bit sign bit as additional storage) just for the pointers to the NAN-boxed objects (for possible future systems for bigger process address rooms than 48 bit), where the object header behind the NAN-boxed object pointers will have then a 32-bit value type tag itself.
#16 Re: SyNode » Adding JavaScript support for mORMot framework » 2016-06-17 11:00:11
Hi guys,
I've started a new BESEN branch https://github.com/BeRo1985/besen/tree/experimental where I'll try to solve the performance problems of BESEN (where BESEN will be then API-incompatible to the current old master branch version), and the I'll try update BESEN to ECMAScript 2015 & 2016.
My planned steps are:
Getting rid of TBESENValue big-record, the performance lack no. 1, and replacing it with LuaJIT-style NaN-tagging.
"Deoptimizing" the another whole code to naive non-optimized code back (but keeping the then new NaN-tagging Value container concept), so that the next step and the over-next step will be easier to manage then.
Adding support for the newer ECMAScript 2015 & 2016 specifications.
Optimizing the code again, but now in a better way, including a better function-call-frame-activation concept, what is the performance lack no. 2 at the current BESEN version, where I was too verbatim to the ECMAScript 5th Edition specification, so that it passes all official test cases overexemplary, but at a cost: very slow execution performance at function-calls.
Because I've started BESEN at that time, because I wanted, that the ObjectPascal ecosystem has a own fast ECMAScript implementation, and that stuff with SynSM doing now hurt in my eyes, so I must act now and lift BESEN to a new level.
Benjamin 'BeRo' Rosseaux
Pages: 1