mORMot Open Source

macfly,

If your nginx/apache/openlitespeed reverse proxy provides some HTTP header corrections you can use them and disable the related option on mORMotBP side (to skip of doing the same work twice). But from compression perspective (especially for apache http server which is based on thread-per-request model) I suggest to use mORMotBP because you get the maximum possible compression and save CPU/Response time on the reverse proxy side.

Measuring both ways under the expected parallel loads is a good appoach to make a decision.

macfly

Usually I use mORMotBP in the opposite way

In one of my projects I create React.js/Emotion.js front-end, build a bundle with Webpack and then embed all assets into the single executable file (Linux with FPC or Windows with Delphi). On production I can choose: delegate assets transfer to low-level http.sys core driver (with .StaticRoot property) and save threds time to something else or to transfer assets right from the memory.

You can do the same trick with Mustache Views - debug and pack them into another Views.res file then extract them with one method call into some folder on production. This is good for delivery, version control, fast rollbacks and any production assets/resources embedding or extraction.

One big advantage on this library is that you precompress all your static assets with maximum possible compression each time you compile your project (you embed not only asset as it is (identity version), but maximum Zopfli compression version (best possible gzip) + maximum Brotli compression version + hash calculation done for each compressed version preliminary which can be used in different cache strategies later like ETag). So you save network costs, get faster html/css/js/json/xml, etc. transfer, save CPU usage, save HDD/SSD usage and deliver maximum possible compressed version of your assets to clients right from the server RAM. Such compression is not possible when you try to compress file on production, especially when you have large app/webworker js code.

Also this library show good results when work behind nginx or other http reverse proxy - because you needn't to spend resources on compression on the fly, you have a lot of HTTP headers fixes and get best HTTP Headers delivery practices and protection available in HTTP Bolerplate framework. Started from version 2.2 - all possible Content Security Policy directives Level 2 and Level 3 are supported (see CSP.pas unit) with Hashes, Nonces, etc. to protect your clients even more. The library code is concentrated on speed, direct call stack usage where possible and inlinings.

If you want to read assets from some production folder and feed them to mORMotBP server just look how assetslz tool works, then do the same thing on production and fill your mORMotBP server with any assets you want.

Something like this:

TMyHTTPServer = class(TBoilerplateHTTPServer)
public
  procedure ReadAssetsFromPath(const Path: string);
...
end;

procedure TMyHTTPServer.ReadAssetsFromPath(const Path: string);
var
  Index: Integer;
  Files: TFindFilesDynArray;
begin
  Files := FindFiles(Path, '*', '', True, True, True);
  for Index := Low(Files) to High(Files) do
    FAssets.AddAsset(Path, Files[Index].Name)
end;

I see, you add both sqlite3.obj and sqlite3.o files under /SQLite3 directory of GitHub project.
This pull request is pointless.

Thanks for clarification!

ab, pvn0, ah, I see.

So funny, that I missed so basic Delphi & FPC fundamental specialty.
I was sure that anything in stack is a garbage initially.

But you right:

function Test: RawUTF8;
var
  S: RawUTF8;
begin
  Result := S;
end;

Gives us:

Test.dpr.86: begin
0000000000B00E10 55               push rbp
0000000000B00E11 4883EC30         sub rsp,$30
0000000000B00E15 488BEC           mov rbp,rsp
0000000000B00E18 48C7452800000000 mov qword ptr [rbp+$28],$0000000000000000
0000000000B00E20 48894D40         mov [rbp+$40],rcx
0000000000B00E24 90               nop
Test.dpr.87: Result := S;
0000000000B00E25 488B4D40         mov rcx,[rbp+$40]
0000000000B00E29 488B5528         mov rdx,[rbp+$28]
0000000000B00E2D E84E2291FF       call @LStrAsg
0000000000B00E32 90               nop
Test.dpr.88: end;
0000000000B00E33 488D4D28         lea rcx,[rbp+$28]
0000000000B00E37 E8541B91FF       call @LStrClr
0000000000B00E3C 488B4540         mov rax,[rbp+$40]
0000000000B00E40 488D6530         lea rsp,[rbp+$30]
0000000000B00E44 5D               pop rbp
0000000000B00E45 C3               ret

Now I see this Initialization (mov qword ptr [rbp+$28],$0000000000000000) and Finalization (call @LStrClr) auto code.

Thanks for clarification!
Will remove all nil/empty local stack managed-types initialization in my code in future.

ab,

Confirmed. Thanks!
With updated sqlite3.o all compiled as expected.

Confirmed, thanks for the fix!

@Edtech or in case if you don't want to use reverse proxy, your can set .StrictSSL property of mORMotBP HTTP server to
strictSSLOn
strictSSLIncludeSubDomains

Hi all,

The mORMotBP 2.0 has been released!

High-level changes:

• Now it contains 50+ properties and options which are fully aligned with HTML 5 Boilerplate v7.2.0.

• Utilize your network smaller and deliver your resources and assets up to 15%-30% faster with new Zopfli and Brotli compressions.

• Save your CPU cycles: now all assets are pre-compressed before embedding.

• Delegate compressed assets transferring to low-level API and free your threads for more interesting work.

• New cache busting strategies are available.

Release 2.0

• Align all boilerplate assets to recent HTML 5 Boilerplate 7.2.0

• All Delphi compilers support started from Delphi 6 (special BuildEvents IDE extenstion provided for old Delphi 6/7/2005/2006)

• Free Pascal support (for Lazarus IDE pre-build.sh scipt provided to compress and embed static assets over "Run / Build File" IDE option)

• Kylix 3 support (over CrossKilyx)

• Zopfli compression support for static assets (save up to 5-15% of traffic and delivery time compared to max GZip Level)

• Brotli compression support for static assets as per RFC 7932 (save another 15%-25% of traffic and delivery time compared to Zopfli)

• All assets compressions (GZip/Zopfli, and Brotli) now precomputed and embedded, so you save your CPU cycles by skipping any static assets compression on production

• Add additional cache bursting strategy. See bpoEnableCacheBustingBeforeExt

• Following RFC 7946 the GeoJSON applications now use application/geo+json MIME type

• MIME Type for RDF XML documents now application/rdf+xml following as per RFC 3870

• Add support of .mjs files with EcmaScript modules  (or JavaScript modules) MIME types

• Add web assembly (.wasm) MIME type support

• Woff fonts (.woff) now have updated font/woff MIME type

• Woff version 2 fonts (.woff2) now have updated font/woff2 MIME type

• True Type collection .ttc fonts now have separate font/collection MIME type

• TTF fonts (.ttf) now have separate font/ttf MIME type

• OTF fonts (.otf) now have separate font/otf MIME type

• Add support for .ics (text/calendar), and .markdown, .md (text/markdown) MIME types

• Upgrade the required 'charset=UTF-8' MIME type list

• Upgrade Content Sequrity Policy (CSP)

• New bpoEnableReferrerPolicy options

• The GZippedMimeTypes has been removed  (just pack your assets with updated assetslz tool)

• Deprecation of Iframes cookies support in Internet Explorer

• TAssets.SaveAssets remove regexp for assets matching (this excludes dependency over SynTable.pas)

Please see https://github.com/eugeneilyin/mORMotBP for details.

@ab,

Finally I got the recent build 1.18.5254 self-tests start executing under XUbuntu 18.04, Bionic Beaver LTS 32-bit

I do the next change in SynCommons.pas:

Comment/remove the original SystemInfo.dwPageSize initialization in SynCommons.pas:26135:

procedure RetrieveSystemInfo;
...
  modname := nil;
  // SystemInfo.dwPageSize := getpagesize; // use libc for this value
...

And put if before InitRedirectCode in SynCommons.pas:62569:

initialization
...
  MoveFast := @System.Move;
  {$ifndef MSWINDOWS}
  SystemInfo.dwPageSize := getpagesize; // use libc for this value
  {$endif}
  {$ifdef FPC}
  FillCharFast := @System.FillChar; // FPC cross-platform RTL is optimized enough
  {$ifdef Linux}
...

Unfortunately ECC cryptography tests failed.
The whole 1.6 Protocols section frozen and requires ^C to be pressed to let the other tests pass.

#44   !!! ECC cryptography - Certificates and signatures "" failed !!!
#55   !!! ECC cryptography - Certificates and signatures "" failed !!!
#90   !!! ECC cryptography - Certificates and signatures "" failed !!!
!  - Certificates and signatures: 3 / 91 FAILED  3.77s

#602 EControlC  !!! Protocols - RTSP over HTTP "EControlC" failed !!!
!  - RTSP over HTTP: 1 / 602 FAILED  5m27
  Total failed: 1 / 602  - Protocols FAILED  5m27

Here is the summary (please pay attention to the last line):

Ubuntu 18.04.2 LTS - Linux 4.18.0-22-generic (cp1252)
    2 x Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (x86)
Using mORMot 1.18.5254 PRTL
    TSQLite3LibraryDynamic 3.22.0 with internal MM
Generated with: Kylix 3 32 bit compiler

Time elapsed for all tests: 7m05
Performed 2019-06-24 21:26:31 by * on *

Total assertions failed for all test suits:  6 / 39,072,532
! Some tests FAILED: please correct the code.
? H@Vunknown: MM Operation after uninstalRuntime error 204 at 0804AD84

@ab, can you please make the fix, validate Kylix under 64-bit Linux system (CrossKylix is unable to build executable 32-bit code for the recent 64-bit platforms) and check why the couple of ECC tests and whole 1.6 Protocols section failed?

Yeap, the initialization section in SynCommons.pas:

initialization
  ...
  InitRedirectCode;
  InitSynCommonsConversionTables;
  RetrieveSystemInfo;

InitRedirectCode uses RedirectCode, which is uses PatchCode, which is based on SystemInfo.dwPageSize, which is equal zero and crush an execution code because dwPageSize initialized later in RetrieveSystemInfo call in initialization section.

Seems that is a reason why processor core goes 100% and execution frozen in some infinite loop inside the random broken execution.

@ab,

After testing of 26 mORMot versions between 4578 and 5254 (the recent one), I found that issue was introduced in 1.18.5037 (2019-02-13 17:11:25)

5037: ensure PatchCode() will work even if OS memory page size is not 4KB

Seems like you change something with SystemInfo.dwPageSize / AlignedAddr on non-Windows OS'es which is broke program execution after compilation under CrossKylix.

Can you add some {$IFDEF}'s and bring Kylix support back?

This is a good news for me. I'm not a big fun to support Delphi 5 and compile code under XP SP 2 virtual boxes

I'm updating mORMotBP to the resent HTML Boilerplate release now.
Some people here noticed that lack of support of old Delphi versions for this library stops them to use it.
So I decided to find a way to pass all library tests scenarios under Delphi 5 & 7. As the result this topic occurred.
I've add Delphi 7 support, so I keep it as is and stop trying to run THTTPServer under Delphi 5.

The dcc50.dll AV crush always occurred when the line 62570 displayed on compiling window (the last file line is 62569).
So some unit post-processing crushed this ancient compilator.

Isn't is easy to remove this 20 years old "Delphi 5" partial support notice from SAD 1.18 Documentation (only 2 places in a whole document)?

@ab, many thanks!

@ab, can you extend this method signature, please?
There are not so many usages of it:

SynCrtSock:6227       
  ctxt.Prepare(URL,Method,HeaderGetText(fRemoteIP),Content,ContentType,'');

SynBidirSock:1686     
  Ctxt.Prepare(URL,Method,InHeaders,InContent,InContentType,fRemoteIP);

SynBidirSock:3131     
  Ctxt.Prepare(url,method,header,data,dataType,'');

mORMotHttpServer:1157 
  ctxt.Prepare(FormatUTF8('%/%/%',[aSender.Model.Root, aInterfaceDotMethodName,aFakeCallID]),'POST','','['+aParams+']','','');

Thanks, fix confirmed

keinn, sorry for the long delay with the response.

The "X-Powered-By" HTTP header line is added explicitly by the mORMot core code - this is helps to understand the popularity of mORMot in the public Internet services and give some gratitude to his authors.

If you insist on this line removal you can search for NOXPOWEREDNAME definition in SynCrtSock.pas
Or you can add NOXPOWEREDNAME into Project Options -> Delphi Compiler -> Conditional Defines under "All Configurations" Target

Great, thanks!

Hi @ab,

Following standard login/logout web app techniques, I need to extend TMVCRendererAbstract.ExecuteCommand as such:

mORMotMVC.pas (1413)

procedure TMVCRendererAbstract.ExecuteCommand(aMethodIndex: integer);

----- old code -------------------------

        if (action.ReturnedStatus=HTML_TEMPORARYREDIRECT) or
           (action.ReturnedStatus=HTML_MOVEDPERMANENTLY) then

----- new code -------------------------

        if (action.ReturnedStatus=HTML_TEMPORARYREDIRECT) or
           (action.ReturnedStatus=HTML_FOUND) or
           (action.ReturnedStatus=HTML_SEEOTHER) or
           (action.ReturnedStatus=HTML_MOVEDPERMANENTLY) then

This is required to handle 302 or 303 POST requests provided by forms redirect to GET methods after login/logout process. POST to GET replacement is done by browsers automatically. Current 307 response code do not replace POST to GET on user agent side (as it required by 303 code).

For example in your 30 - MVC Server sample: if I press Ctrl+F5 after successful login the browser will show popup message with POST repeat. But if we will answer 302 (or more correct 303) on TBlogApplication.Login interface event then browser will replace POST to GET under the hood and all page refreshes after login will not impact the user.

Btw, why did you name HTTP status codes with HTML prefix? Isn't it an abstraction leak (low level HTTP response codes named with high level non-existing HTML_* prefixes)?

@itSDS,
Try clear and the rebuild the project.

Usually tests failed on SynCrtSock.pas (3937)

  // THttpServerGeneric thread preparation: launch any OnHttpThreadStart event
  NotifyThreadStart(self);
  // main server process loop
  if Sock.Sock>0 then

Seldom after call NotifyThreadStart the Sock become nil and AV has raised (all exceptions in this code are catch and ignored in the end of this method)
So if you run tests 5-8 times you will get this issue in this place (tested on XE8, Seattle, Berlin)

Just try to clear, then rebuild and let me know, thanks.

@ab,
Do you have any updates following integration into trunk?

Release 1.5

• fix EnableCacheByETag test scenarios