Wow! Full FPC support now??!!

AOG · 2014-10-28 07:47:10

You are right !
Sorry about this ... sometimes I forget that I use my own compiler settings to get things running in my setup.

But, if you encounter this asm problem, you can solve it by adding the "assembler" statement to the declaration:
e.g. your line 315 SynLZO error:

function _lzo1x_1_do_compress(in_p: PAnsiChar; in_len: PtrInt;
out_p: PAnsiChar; out out_len: integer): integer;
{$ifdef USEASM}
---------------> {$ifdef FPC} assembler;{$endif} <---------------

In FPC, pure assembler functions and procedures have to be defined by this "assembler" statement.
But if you set FPC into Delphi-mode, this is not necessary.

Hope this is clear.

edwinsn · 2014-10-28 10:28:58

AOG wrote:

You are right !
Sorry about this ... sometimes I forget that I use my own compiler settings to get things running in my setup.
But, if you encounter this asm problem, you can solve it by adding the "assembler" statement to the declaration:
e.g. your line 315 SynLZO error:
function _lzo1x_1_do_compress(in_p: PAnsiChar; in_len: PtrInt;
out_p: PAnsiChar; out out_len: integer): integer;
{$ifdef USEASM}
---------------> {$ifdef FPC} assembler;{$endif} <---------------
In FPC, pure assembler functions and procedures have to be defined by this "assembler" statement.
But if you set FPC into Delphi-mode, this is not necessary.
Hope this is clear.

Thanks! I got further.

- Suggest to allow defining the location for libkernel32.a and all otehr library files.

- where I can download the sqlite3.o and sqlite3fts3.o file? current I got:
TestMormotWin.lpr(23,1) Error: Undefined symbol: _sqlite3_close and a lot of similar errors.

thanks again.

AOG · 2014-10-28 11:17:01

I would suggest first using mORMot on FPC without static linking.
I then needs the normal sqlite3.dll library on Windows.

Inside Synopse.inc do this somewhere near the end (depends on your version):
{$ifdef CPU64}
{$define NOSQLITE3STATIC}
{$endif}
{$define NOSQLITE3STATIC} <------------- add this !!

mORMot now compiles without static libraries. Just add an appropriate sqlite3.dll in your exe-directory.

But, if you want static, there is a script for building the special version of sqlite3 (c-fpcmingw.bat).
On Windows, you will need MinGW to compile. And some MinGW libraries (libkernel32.a and libgcc.a).

Let me know if you are ready with the normal dynamic tests (with sqlite3.dll), and want to test static.
I will make the necessary files available for you !

edwinsn · 2014-10-28 17:50:34

Thanks AGO, successfully built on Windows without static lib!

However, first run it complains missing libgcc_s_sjlj-1 so I got it from a subfolder inside the CodeTyphon directory. and of course I've got the latest sqlite3.dll from sqlite.org.

second run it emit this error:
Application could not start normally (0xc000007b), click ok to terminate the program. (translated from my native language).

AOG · 2014-10-28 17:57:14

Wow ... now you have to explain to me how you are building and using !!

1) what is the system you are compiling on ?
2) CodeTypon version ?
3) what is your target system ?

And any more details you care to share !

edwinsn · 2014-10-29 08:36:42

Alfred,

>1) what is the system you are compiling on ?
>2) CodeTypon version ?
I'm not using CodeTyphon, but the latest Lazarus:

Lazarus version: 1.2.6
Lazarus svn revision: 46529
Lazarus build date: 2014/10/11
Lazarus was compiled for i386-win32
Lazarus was compiled with fpc 2.6.4

3) what is your target system ?
Target OS is win32 (I'm using win 7 64bit, but I don't think it's a problem).
Target CPU family is default
Target processor is default.

Changed values in Synopse.inc:

  {.$MODE DELPHI} // e.g. for asm syntax - disabled for FPC 2.6 compatibility
  {$define NOSQLITE3STATIC}

Let me know if you need more info.

ab · 2014-11-07 07:56:33

Variant late binding has already been fixed by the FPC team!
See http://bugs.freepascal.org/view.php?id=26773

We have included some additional fixes and compatibility enhancements for FPC
See http://synopse.info/fossil/info/31511cd597
Also including sample 01, 02 and 04.
Thanks Alf for the feedback!

EgonHugeist · 2014-11-10 16:24:34

@AB

propose you reply a "thank you" and close the bugreport?..

ab · 2014-11-10 17:44:04

AFAIK it is already closed!

EgonHugeist · 2014-11-11 00:43:13

Well the report is marked as Resolved and fixed.
AFAIU the FPC bugtracker: Closing should be done by reporter. Else they keep the report a long time until a core member decides to close it.

So they are waiting for a confirmation or you can reopen same ticket if you disagree, Arnaud.

Just a hint. Maybe AOG can compile the trunk and test the fix? Or you kindly should close the report. This way all are happy.

AOG · 2014-11-11 05:44:18

Already done, succeeded and reported !!

http://lists.freepascal.org/pipermail/f … 34808.html
http://lists.freepascal.org/pipermail/f … 34829.html

My mORMot now runs with Mustache wrappers enabled.

To test for yourself : fpcup makes life for users of trunk FPC / Lazarus very easy.
http://wiki.freepascal.org/fpcup

fpcup.exe --fpcURL="trunk" --lazURL="trunk"

Greetings, Alf.

Last edited by AOG (2014-11-11 05:49:26)

ab · 2014-11-11 09:57:51

@Egon I closed the report. The process was not obvious: I did not find the button last time I checked.

@AOG Thanks.
For the other issue (interface RTTI), could you report that reference has been added to the expected RTTI layout, in the report?
I think there is enough info to start the implementation.

Leslie7 · 2016-03-14 21:53:06

ab wrote:

What do you think of using http://gwan.com/ as our HTTP server under Linux?
Sounds like a great and proven system, to be used instead of http.sys (which sounds the best under Windows) and any FastCGI processes (e.g. Apache/NGinx/Lighttpd) under Linux.
Take a look at http://www.wikivs.com/wiki/G-WAN_vs_Nginx

AB,

How about this: https://github.com/zaphoyd/websocketpp

ab · 2016-03-15 07:35:25

@Leslie this is a Websockets only C++ library.
I do not understand why it may help for a HTTP server.

Leslie7 · 2016-03-15 09:51:22

It has the ability to respond to HTTP requests other than WebSocket Upgrade requests.

I may be wrong but it seems to me that it has all the low level stuff based on epoll, and mORMot has all the high level implementation to handle the http requests in its own httpserver implementation. They seem like a good match. It could be more than two birds with one stone to provide efficient websocket and http server implementation for a lot of platforms with a single library. Having the core compiled with eg a GNU compiler could boost the performance as well. You already have the sqlite obj files compiled from C in the framework. WebsocketPP could become part of mORMot in the same manner.

Last edited by Leslie7 (2016-03-15 10:15:55)

ab · 2016-03-15 10:58:40

Yes, it is interesting in this context.

But it is a C++ library, so a C flat API is needed to consume it from Delphi or FPC...
Is anyone willing to do this?

mpv · 2016-03-15 12:02:28

My opinion is to use libuv to replace a IOCP. All websocket part in mORMot is already implemented, we luck only a async IO.
Pros for this lib is:
- battle tested by millions of user
- fallback to the optimal mechanism depending on platform: epoll, kqueue, IOCP, event ports
- C API

Leslie7 · 2016-03-16 12:54:30

ab wrote:

Yes, it is interesting in this context.
But it is a C++ library, so a C flat API is needed to consume it from Delphi or FPC...
Is anyone willing to do this?

I am not a C/C++ guy. But maybe the developer is willing to create a C API.

Leslie7 · 2016-03-16 13:16:42

mpv wrote:

My opinion is to use libuv to replace a IOCP. All websocket part in mORMot is already implemented, we luck only a async IO.
Pros for this lib is:
- battle tested by millions of user
- fallback to the optimal mechanism depending on platform: epoll, kqueue, IOCP, event ports
- C API

They both seem to be good. Maybe we should create a positive/negative list for both.

" fallback to the optimal mechanism depending on platform: epoll, kqueue, IOCP, event ports"

this goes for Websocket Plus Plus too. It can be linked with different libraries. Boost being one of them uses the best choice for every platform. I only outlined epoll because the platform most developer has to deal with beside Windows is Linux, so this is where mORMot could use the improvement the most.

I think the question here is more than just async IO, websockets support is a huge topic as well:

WSPP passes all the Autobhan tests, it implements most websocket features, much more than mORMot does. For example one I am interested in is scatter&gather. I would argue that what it has to offer on the websockets front carries quite a weight.

Last edited by Leslie7 (2016-03-16 13:19:59)

ab · 2016-03-16 14:30:26

@Leslie7
If you take a look at our implementation, you will find out that it does implement a scatter & gather pattern, in its own optimized way.
Together with frame encryption and compression, performance is very huge.
Without the need to use a TLS layer, and its certificates plumbing.

And the mORMot implementation has some unique features, like REST emulation, or its integration with interface-based services, via interface parameters used as callbacks.
The very same code could be used in-process, without any WebSockets call at all, and no delay, nor plumbing.
I do not know any other framework or library leveraging websockets at such high level.

Leslie7 · 2016-03-16 15:01:22

ab,

It is good to know.

I am sure the current websockets implementation fits very well the specific needs of the framework. The question is if it can be used with browsers (which implement the official standards) as a client? What about crossplatform clients?

Last edited by Leslie7 (2016-03-17 22:42:49)

ab · 2016-03-16 15:43:12

You could use JSON as content for WebSockets clients, if our proprietary binary protocol is not possible.

marius maximus · 2016-03-17 10:12:08

I do not want to create a new topic.

On my platform Debian 64bit , I always after download mORMot source I need add {$PIC OFF} in unit SynCrypto.
http://www.freepascal.org/docs-html/prog/progsu112.html

Problem is with function sha256_sse4

        movdqu  xmm12,[PSHUFFLE_BYTE_FLIP_MASK] // SynCrypto.pas(3258,47) Error: Generating PIC, but reference is not PIC-safe
        movdqu  xmm10,[_SHUF_00BA] // SynCrypto.pas(3259,34) Error: Generating PIC, but reference is not PIC-safe
        movdqu  xmm11,[_SHUF_DC00] //SynCrypto.pas(3260,34) Error: Generating PIC, but reference is not PIC-safe
@loop0: mov     rbp,[K256Aligned] // SynCrypto.pas(3261,33) Error: Generating PIC, but reference is not PIC-safe

What is the best solution?
Compile whole project with "$PIC OFF"
Compile whole SynCrypto.pas with "$PIC OFF"
or compile only this 4 lines compile with "$PIC OFF"

ab · 2016-03-17 10:17:59

The framework is not yet compatible with Linux 64.
Try to by-pass all the asm, and use the "purepascal" version of SynCrypto instead.
Any input is welcome.

AOG · 2016-03-17 10:34:39

I am using the -Cg- switch for getting a 64 bit Linux mORMot.

If Lazarus: in Custom Options add -Cg-

marius maximus · 2016-03-18 22:46:18

In this place is problem on windows 64bit too in lazarus 64bit , I tessted trunk 3.1.1 and stabile Lazarus with fpc 3.0.0

        movdqu  xmm12,[PSHUFFLE_BYTE_FLIP_MASK] // here exception !!!
        movdqu  xmm10,[_SHUF_00BA] 
        movdqu  xmm11,[_SHUF_DC00] 
@loop0: mov     rbp,[K256Aligned]

After run "Synopse mORMot Framework Automated tests" I have exception "Project TestSQL3.exe raised exception class 'External: SIGSEGV' in file ..\synCrypto.pas at line 3256
After exception application exit

Last message on screen:

 1.5. Cryptographic routines:
  - Adler32: 1 assertion passed  770us
  - MD5: 86 assertions passed  372us
  - SHA1: 10 assertions passed  6.59ms

marius maximus · 2016-03-21 10:42:01

I tested TestSQL3.exe on 4 computers with i7 but only one has problem
I check TestSQL3.exe build on XE5 64bit version and have problem too but Delphi show exception message :

! Cryptographic routines - AES256
! Exception EOSError raised with messsage:
! System Error. Code: -2146893813.
Nieprawidłowy klucz do użycia w podanym stanie // Error in Polish language

Translated version
0x8009000B Key not valid for use in specified state

From this scope

procedure EnsureCryptoAPIAESProviderAvailable;
begin
  if CryptoAPIAESProvider=nil then
    raise ESynCrypto.Create('PROV_RSA_AES provider not installed') else
  if CryptoAPIAESProvider=HCRYPTPROV_NOTTESTED then begin
    CryptoAPIAESProvider := nil;
    if CryptoAPI.Available then begin
      if not CryptoAPI.AcquireContextA(CryptoAPIAESProvider,nil,nil,PROV_RSA_AES,0) then // AcquireContextA return FALSE !!!
        if (HRESULT(GetLastError)<>NTE_BAD_KEYSET) or not CryptoAPI.AcquireContextA(
           CryptoAPIAESProvider,nil,nil,PROV_RSA_AES,CRYPT_NEWKEYSET) then
          RaiseLastOSError; // exception here !!!
    end;
  end;
end;

And it is solution
https://www.youtube.com/watch?v=ZOpzfLt7Qpw

Last edited by marius maximus (2016-03-21 10:47:38)

ab · 2016-03-21 11:37:58

@mariusmaximus
The solution shown is to delete the %appdata%\microsoft\crypto folder.
But it would delete all your installed certificates and private keys, which may be used by other softwares...

Sounds one more reason not to use the Crypto API, but our direct AES implementation classes, which are faster and don't suffer from this issue!

mORMot Open Source

#51 2014-10-28 07:47:10

Re: Wow! Full FPC support now??!!

#52 2014-10-28 10:28:58

Re: Wow! Full FPC support now??!!

#53 2014-10-28 11:17:01

Re: Wow! Full FPC support now??!!

#54 2014-10-28 17:50:34

Re: Wow! Full FPC support now??!!

#55 2014-10-28 17:57:14

Re: Wow! Full FPC support now??!!

#56 2014-10-29 08:36:42

Re: Wow! Full FPC support now??!!

#57 2014-11-07 07:56:33

Re: Wow! Full FPC support now??!!

#58 2014-11-10 16:24:34

Re: Wow! Full FPC support now??!!

#59 2014-11-10 17:44:04

Re: Wow! Full FPC support now??!!

#60 2014-11-11 00:43:13

Re: Wow! Full FPC support now??!!

#61 2014-11-11 05:44:18

Re: Wow! Full FPC support now??!!

#62 2014-11-11 09:57:51

Re: Wow! Full FPC support now??!!

#63 2016-03-14 21:53:06

Re: Wow! Full FPC support now??!!

#64 2016-03-15 07:35:25

Re: Wow! Full FPC support now??!!

#65 2016-03-15 09:51:22

Re: Wow! Full FPC support now??!!

#66 2016-03-15 10:58:40

Re: Wow! Full FPC support now??!!

#67 2016-03-15 12:02:28

Re: Wow! Full FPC support now??!!

#68 2016-03-16 12:54:30

Re: Wow! Full FPC support now??!!

#69 2016-03-16 13:16:42

Re: Wow! Full FPC support now??!!

#70 2016-03-16 14:30:26

Re: Wow! Full FPC support now??!!

#71 2016-03-16 15:01:22

Re: Wow! Full FPC support now??!!

#72 2016-03-16 15:43:12

Re: Wow! Full FPC support now??!!

#73 2016-03-17 10:12:08

Re: Wow! Full FPC support now??!!

#74 2016-03-17 10:17:59

Re: Wow! Full FPC support now??!!

#75 2016-03-17 10:34:39

Re: Wow! Full FPC support now??!!

#76 2016-03-18 22:46:18

Re: Wow! Full FPC support now??!!

#77 2016-03-21 10:42:01

Re: Wow! Full FPC support now??!!

#78 2016-03-21 11:37:58

Re: Wow! Full FPC support now??!!

Board footer