#1 Re: mORMot 1 » Wow! Full FPC support now??!! » 2016-03-21 10:42:01

I tested  TestSQL3.exe on 4 computers with i7 but only one has problem
I check TestSQL3.exe build on XE5 64bit version and have problem too but Delphi show exception message :

! Cryptographic routines - AES256
! Exception EOSError raised with messsage:
!  System Error.  Code: -2146893813.
Nieprawidłowy klucz do użycia w podanym stanie  // Error in Polish language

Translated version
0x8009000B Key not valid for use in specified state

From this scope

procedure EnsureCryptoAPIAESProviderAvailable;
begin
  if CryptoAPIAESProvider=nil then
    raise ESynCrypto.Create('PROV_RSA_AES provider not installed') else
  if CryptoAPIAESProvider=HCRYPTPROV_NOTTESTED then begin
    CryptoAPIAESProvider := nil;
    if CryptoAPI.Available then begin
      if not CryptoAPI.AcquireContextA(CryptoAPIAESProvider,nil,nil,PROV_RSA_AES,0) then // AcquireContextA return FALSE !!!
        if (HRESULT(GetLastError)<>NTE_BAD_KEYSET) or not CryptoAPI.AcquireContextA(
           CryptoAPIAESProvider,nil,nil,PROV_RSA_AES,CRYPT_NEWKEYSET) then
          RaiseLastOSError; // exception here !!!
    end;
  end;
end;

And it is solution
https://www.youtube.com/watch?v=ZOpzfLt7Qpw

#2 Re: mORMot 1 » SynCommons - Lazarus v1.6 - FPC v3.0, compile fails » 2016-03-20 20:44:52

This is not a simple task.

Probably you should add windows unit to uses for WINCE
and {$define MSWINDOWS}   for  SynCommons and  SynCrypto (wince has similar api to win32) 


An then

while true do 
begin
  press_CTRL_F9;
  
  if found_error then 
  begin
    in_error_line_insert_DEFINE;  
      //   for example:  
      //   {$ifndef WINCE}  
      //      PROBLEMATIC CODE !!!
      //   {$else}
      //      writeln('to do WINCE!');
      //   {$endif}
  else
    break; 
  end;
end;

This help compile , 
It does not guarantee that the program will work, but first step.

Few years ago I was thinking about wince , but was problematic and I switch to Linux.
At this moment i don't have platofrom for test ,but my lazarus compile EXE for  wince-arm with  SynCrypto  but I can not see if it works

#3 Re: mORMot 1 » SynCommons - Lazarus v1.6 - FPC v3.0, compile fails » 2016-03-19 19:09:42

Look into uses SynCommons.pas

WinCE is not windows and not unix wink It is problem !

{$ifdef MSWINDOWS} 
...  
{$else MSWINDOWS}  

    {$ifdef FPC}
      BaseUnix,
    {$endif}  
{$endif MSWINDOWS}  

In my opinion WinCE at this moment is unsupported, but you have to fight !

Remove BaseUnix and see what happens.

#4 Re: mORMot 1 » Wow! Full FPC support now??!! » 2016-03-18 22:46:18

In this place is problem on windows 64bit too in lazarus 64bit , I tessted trunk 3.1.1 and stabile Lazarus with fpc 3.0.0

        movdqu  xmm12,[PSHUFFLE_BYTE_FLIP_MASK] // here exception !!!
        movdqu  xmm10,[_SHUF_00BA] 
        movdqu  xmm11,[_SHUF_DC00] 
@loop0: mov     rbp,[K256Aligned] 

After run "Synopse mORMot Framework Automated tests"  I have exception "Project TestSQL3.exe raised exception class 'External: SIGSEGV' in file ..\synCrypto.pas at line 3256
After exception application exit

Last message on screen:

 1.5. Cryptographic routines:
  - Adler32: 1 assertion passed  770us
  - MD5: 86 assertions passed  372us
  - SHA1: 10 assertions passed  6.59ms

#5 Re: mORMot 1 » Wow! Full FPC support now??!! » 2016-03-17 10:12:08

I do not want to create a new topic.

On my platform Debian 64bit , I always after download mORMot source I need add {$PIC OFF} in unit SynCrypto.
http://www.freepascal.org/docs-html/prog/progsu112.html

Problem is with function sha256_sse4

        movdqu  xmm12,[PSHUFFLE_BYTE_FLIP_MASK] // SynCrypto.pas(3258,47) Error: Generating PIC, but reference is not PIC-safe
        movdqu  xmm10,[_SHUF_00BA] // SynCrypto.pas(3259,34) Error: Generating PIC, but reference is not PIC-safe
        movdqu  xmm11,[_SHUF_DC00] //SynCrypto.pas(3260,34) Error: Generating PIC, but reference is not PIC-safe
@loop0: mov     rbp,[K256Aligned] // SynCrypto.pas(3261,33) Error: Generating PIC, but reference is not PIC-safe

What is the best solution?
Compile whole project with "$PIC OFF" 
Compile whole SynCrypto.pas with "$PIC OFF" 
or compile only this 4 lines compile with "$PIC OFF"

#6 Re: mORMot 1 » TSQLHttpServer and linux » 2015-08-25 19:11:23

My bookmarks (for future) about linux64.
In api.pdf (page 23) we found exaple for function with many  parameters (double, struct and long double)   
AMD64 ABI
Calling conventions for different C++ compilers and operating systems
Wikipedia
Stack frame layout on x86-64

and windows
Parameter Passing

#7 Re: mORMot 1 » TFakeCallStack -Duplicate identifier "RCX" » 2015-08-24 20:43:46

Next small step wink
Last to weeks I was on holiday . Now I begin to test x64 linux wink

#8 Re: mORMot 1 » TSQLHttpServer and linux » 2015-08-24 20:28:41

Next step to full suport x64 linux

After this fix "14 - Interface based services" server start works OK , client return correct result

procedure CallMethod(var Args: TCallMethodArgs);
{$ifdef CPUARM}
begin
  raise EInterfaceFactoryException.Create('FPC+ARM not supported yet');
end;
{$else}
{$ifdef CPU64}
{$ifdef FPC}
asm
    //.params 64    // size for 64 parameters
    //.pushnv r12   // generate prolog+epilog to save and restore non-volatile r12
    mov r12,Args
    // copy stack content (if any)
    mov RDI,[r12].TCallMethodArgs.StackAddr
    lea RSI,[rsp+$20]
    mov RDX, [r12].TCallMethodArgs.StackSize
    call Move
    // call method
    mov RDI,[r12+TCallMethodArgs.Regs+REGRDI*8-8]
    mov RSI,[r12+TCallMethodArgs.Regs+REGRSI*8-8]
    mov RDX, [r12+TCallMethodArgs.Regs+REGRDX *8-8]
    mov RCX, [r12+TCallMethodArgs.Regs+REGRCX *8-8]
    mov R8,[r12+TCallMethodArgs.Regs+REGR8*8-8]
    mov R9,[r12+TCallMethodArgs.Regs+REGR9*8-8]
    movsd xmm0,[r12+TCallMethodArgs.Regs+REGXMM0*8-8]
    movsd xmm1,[r12+TCallMethodArgs.Regs+REGXMM1*8-8]
    call [r12].TCallMethodArgs.method
    // retrieve result
    mov [r12].TCallMethodArgs.res64,rax
    mov cl,[r12].TCallMethodArgs.resKind
    cmp cl,smvDouble
    je @d
    cmp cl,smvDateTime
    je @d
    cmp cl,smvCurrency
    jne @e
@d: movsd [r12].TCallMethodArgs.res64,xmm0
@e:
end;
{$else}

#9 mORMot 1 » TFakeCallStack -Duplicate identifier "RCX" » 2015-08-24 13:40:05

marius maximus
Replies: 3
/// map the stack memory layout at TInterfacedObjectFake.FakeCall()
  TFakeCallStack = packed record
    {$ifdef CPU64}
    {$ifdef LINUX}
    XMM0, XMM1, XMM2, XMM3, XMM4, XMM5, XMM6, XMM7: double;
    {$else}
    XMM1, XMM2, XMM3: double;
    {$endif}
    MethodIndex: PtrUInt;
    Frame, Ret: pointer;
    {$ifdef LINUX}
    RDI, RSI, RDX, RCX, R8, R9: pointer;
    {$endif}
    RCX, RDX, R8, R9: pointer; // Here error !!! 
    {$else}
    EDX, ECX, MethodIndex, EBP, Ret: Cardinal;
    {$endif}
    Stack: array[word] of byte;
  end;

Fo example correct version

  /// map the stack memory layout at TInterfacedObjectFake.FakeCall()
  TFakeCallStack = packed record
    {$ifdef CPU64}
    {$ifdef LINUX}
    XMM0, XMM1, XMM2, XMM3, XMM4, XMM5, XMM6, XMM7: double;
    {$else}
    XMM1, XMM2, XMM3: double;
    {$endif}
    MethodIndex: PtrUInt;
    Frame, Ret: pointer;
    {$ifdef LINUX}
    RDI, RSI, RDX, RCX, R8, R9: pointer;
    {$else}
    RCX, RDX, R8, R9: pointer;
    {$endif}
    {$else}
    EDX, ECX, MethodIndex, EBP, Ret: Cardinal;
    {$endif}
    Stack: array[word] of byte;
  end; 

#10 Re: mORMot 1 » TSQLHttpServer and linux » 2015-08-06 15:13:34

Well done!
   
Now I open project from "14 - Interface based services" directory wink

First problem "procedure CallMethod(var Args: TCallMethodArgs);" it is only assembler sad
Exists "pure pascal" version ?

I need to think wink

#11 Re: mORMot 1 » TSQLHttpServer and linux » 2015-08-06 08:58:17

Next step:

Problem in "function THttpSocket.HeaderGetText: SockString;"

Is problem "fast length" on different platform wink




This error is similar to
http://synopse.info/forum/viewtopic.php?pid=16504


And similar is fix

function THttpSocket.HeaderGetText: SockString;
var i,L,n: integer;
    V: PtrInt;
    P: PAnsiChar;
begin
  // much faster than for i := 0 to Count-1 do result := result+Headers[i]+#13#10;
  result := '';
  n := length(Headers);
  if n=0 then
    exit;
  L := n*2; // #13#10 size
  dec(n);
  for i := 0 to n do
    if pointer(Headers[i])<>nil then
     {$ifdef FPC}
      inc(L,PInteger(PAnsiChar(pointer(Headers[i]))-sizeof(pointer))^); // fast add length(List[i])
     {$else} 
      inc(L,PInteger(PAnsiChar(pointer(Headers[i]))-4)^); // fast add length(List[i])
     {$endif}
    
  SetLength(result,L);
  P := pointer(result);
  for i := 0 to n do begin
    V := PtrInt(PAnsiChar(Headers[i]));
    if V<>0 then begin
      {$ifdef FPC}
      L := PInteger(V-sizeof(pointer))^;  // L := length(List[i])
      {$else}
      L := PInteger(V-4)^;  // L := length(List[i])
      {$eif}
      move(pointer(V)^,P^,L);
      inc(P,L);
    end;
    PWord(P)^ := 13+10 shl 8;
    inc(P,2);
  end;
end;   

Or use this metod
{$ifdef FPC}PStrRec(Pointer(PtrUInt(U)-STRRECSIZE))^.length ....

This remove error "'Error adding the data'" and all works OK

#12 Re: mORMot 1 » TSQLHttpServer and linux » 2015-08-05 20:48:49

sometimes   "'Error adding the data'" is from this function

function Send(s: TSocket; Buf: pointer; len,flags,timeout: Integer): Integer;
var maxTicks: Int64;
begin
  maxTicks := GetTickCount64+timeout;
  repeat
    {$ifdef KYLIX3}
    result := LibC.Send(s,Buf^,len,flags);
    {$else}
    result := fpSend(s,pointer(Buf),len,flags);

    writeln('ggggggg',result);

    {$endif}
    if result>=0 then
      exit; // success
    if timeout<=0 then
      break;
    if (errno<>WSATRY_AGAIN) and (errno<>WSAEINTR) then
      break;
    sleep(1);
  until GetTickCount64>maxTicks;
  writeln('errno Send()=',errno);
  result := -1; // error
end;    

fpSend return -1
on console i see "'errno Send()=32"


I need to think smile

#13 Re: mORMot 1 » TSQLHttpServer and linux » 2015-08-05 13:18:10

small mistake in your code

your read vInteger but value is in "vInt64^" field !!!

I have mistake too wink

Correct version:

    vtInt64:    begin
      Str(VInt64^,tmp);
      SockSend(@tmp[1],length(tmp));
    end;  

After this fix I see "04 - HTTP Client-Server" works almost correct.
linux server add data and return correct results to client(s).
Is only small bug in linux client after "Add the message" client show message "'Error adding the data'" but in database is new record smile
Windows client don't have this bug !

#14 Re: mORMot 1 » TSQLHttpServer and linux » 2015-08-05 10:52:59

Next step wink
Please add in TCrtSocket.SockSend

this caseoption

    vtInt64:begin
      Str(vtInt64,tmp);
      SockSend(@tmp[1],length(tmp));
    end;

#15 Re: mORMot 1 » TSQLHttpServer and linux » 2015-08-05 08:07:25

Thx very much !

Next step!
Now i test linux 64bit serwer by client from windows.

Problem is with function "WinHttpSendRequest" after call (function return FALSE) we have exception "Exception class EWinHTTP with message 'winhttp.dll error 12152 (invalid server response)'"

I look into wireshark and comapre windows<->windows and windows<-> linux connection
windows serwer <->windows client ALL WORKS OK !

GET /root/TimeStamp HTTP/1.1
Cache-Control: no-cache
Connection: Keep-Alive
Pragma: no-cache
Content-Type: application/json; charset=UTF-8
Accept: */*
Accept-Encoding: synlz
User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows; Synopse mORMot 1.18.1735 TWinHTTP)
Host: 192.168.10.8:8080

HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Server: mORMot/1.18.1735 (Windows) Microsoft-HTTPAPI/1.0
X-Powered-By: Synopse mORMot 1.18.1735 http://synopse.info
Server-InternalState: 17
Date: Wed, 05 Aug 2015 07:41:41 GMT
Content-Length: 12

135254276713

linux 64bit serwer and windows client

GET /root/TimeStamp HTTP/1.1
Cache-Control: no-cache
Connection: Keep-Alive
Pragma: no-cache
Content-Type: application/json; charset=UTF-8
Accept: */*
Accept-Encoding: synlz
User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows; Synopse mORMot 1.18.1735 TWinHTTP)
Host: 192.168.10.7:8080

HTTP/1.1 200 OK
Server-InternalState: 1
X-Powered-By: Synopse mORMot 1.18.1735 http://synopse.info
Server: mORMot/1.18.1735 (Linux)
Content-Length: 
Content-Type: text/plain; charset=UTF-8
Accept-Encoding: synlz,gzip
Connection: Keep-Alive

135254276817

the server responds correctly,
I need to think wink

#16 Re: mORMot 1 » TSQLHttpServer and linux » 2015-08-04 21:52:19

AB wrote:

x86-64 is not supported as a target for FPC, under Linux or Windows.
We started to support FPC with x86-32 under Linux and Windows.

Only Delphi as compiler is supported yet for Windows 64.

But  few comments to make it work ! One step to work !

The first problem( anyway very interesting )
In Delphi 32bit sizeof(TTextRec.Handle) = 4B
In Delphi 64bit sizeof(TTextRec.Handle) = 8B

In FPC/Lazarus
TTextRec.Handle type =  Longint (4B)

In function TCrtSocket.CreateSockIn and TCrtSocket.CreateSockOut you save into TTextRec.Handle  pointer


 
with TTextRec(SockIn^) do begin
    Handle := PtrInt(self);

My version use "userdata" to save pointer

 
with TTextRec(SockIn^) do begin
    Handle := PtrInt(self);
    PPtrInt(@UserData[1])^ :=  PtrInt(self);

And then replace all  (3x)
TCrtSocket(F.Handle)
with
TCrtSocket(PPtrInt(@f.UserData[1])^);

After that server reply HTML

#17 mORMot 1 » FPC compilation problem(s) » 2015-08-04 13:21:37

marius maximus
Replies: 3

My platform 64bit linux (debian 8)
fpc current svn trunk

I have problem with lines:


File SynCrypto.pas
Line 3221
        movdqu  xmm12,[PSHUFFLE_BYTE_FLIP_MASK]
        movdqu  xmm10,[_SHUF_00BA]
        movdqu  xmm11,[_SHUF_DC00]
@loop0: mov     rbp,[K256Aligned]


Compile Project, Mode: linux, Target: fpc/x86_64-linux/TestSQL3: Exit code 256, Errors: 4
SynCrypto.pas(3221,47) Error: Generating PIC, but reference is not PIC-safe
SynCrypto.pas(3222,34) Error: Generating PIC, but reference is not PIC-safe
SynCrypto.pas(3223,34) Error: Generating PIC, but reference is not PIC-safe
SynCrypto.pas(3224,33) Error: Generating PIC, but reference is not PIC-safe

On windows (64bit) this same fpc version no errors !

#18 mORMot 1 » TSQLHttpServer and linux » 2015-08-04 13:13:08

marius maximus
Replies: 16

Now I have 64bit Debian 8 wink

I compile server and client
"04 - HTTP Client-Server\Project04Server"
"04 - HTTP Client-Server\Project04Client"
Start server:
sudo ./Project04Server

Then I check 8080 port

"netstat -na |grep 8080
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN"

In my opinion LISTEN is OK wink

First test on linux:
1) telnet localhost 8080
2) wait 3s.
3) Message "Connection closed by foreign host."

In windows (server and telnet run on windows) this same test:
1) telnet localhost 8080
2) type "get index.html"
3) Result from serwer

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/str
ict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>

On windows server works correctly on linux serwer close Connection

#19 Re: mORMot 1 » Understanding sample 02 and 03 » 2015-07-06 07:43:12

Please look into DPR file
Sample 02:
https://github.com/synopse/mORMot/blob/ … ject02.dpr
line: 50
Form1.Database := TSQLRestServerDB.Create(Form1.Model,

About "where is the code/logic  that connects the client to the server?"
Server side:
  Server := TSQLRestServerDB.Create(Model,ChangeFileExt(ExeVersion.ProgramFileName,'.db3'));
  Server.CreateMissingTables;
  Server.ExportServerNamedPipe('03');
Client side:
  Form1.Database := TSQLRestClientURINamedPipe.Create(Form1.Model,'03');

#21 Re: mORMot 1 » Wrong length RawUTF8 » 2015-06-15 20:28:52

I was able compile "TestSql3" ("Synopse mORMot Framework Automated tests") in 64bit fpc.

test

a lot of red and a long way yet wink

#22 Re: mORMot 1 » Wrong length RawUTF8 » 2015-06-14 22:55:45

a step in the right direction,
I will be tested

#23 Re: mORMot 1 » Wrong length RawUTF8 » 2015-06-14 09:22:18

Thx very much ,

I always use svn but try git ,
I improvise wink

I find two bugs (very small)
https://github.com/mariuszekpl/mORMot/c … a20282853c

#24 Re: mORMot 1 » Wrong length RawUTF8 » 2015-06-13 21:38:08

Step by step and we(I) fix all wink

for example better version

{$IFDEF FPC}
SetLength(result,length(s)*2);
{$ELSE}
SetLength(result,PInteger(PtrInt(S)-sizeof(integer))^*2);
{$ENDIF FPC}

I look in asembler and fpc create better asm like Delphi
version with length(s) is near  "(PtrInt(S)-sizeof(integer)"

Or other version is create for example "CONST _str_length_pos"
and use "(PtrInt(S)-_str_length_pos)"
and add few "ifdef" in Synopse.inc


After this fix , next sample project start works smile
A find this bug when try run  "07 - SynTest"


I need to take a workshop next project wink

#25 mORMot 1 » Wrong length RawUTF8 » 2015-06-13 20:20:53

marius maximus
Replies: 9

In code I found few pleces where insted
length(RawUTF8)
is used metod like in "UnCamelCase"
(Sometimes i used this metod too wink)

function UnCamelCase(const S: RawUTF8): RawUTF8; overload;
begin
  result := '';
  if S='' then
    exit;
  SetLength(result,PInteger(PtrInt(S)-sizeof(integer))^*2); // max length
  ...
  ...

This metod works OK in Delphi (64 and 32 bit) but not always in FPC and Lazarus ,
for example
in 64 bit windows version length is on -8 position
in 32 bit windows version length is on -4 position (identical like Delphi)

This problem is in fundamental functions:

Utf8DecodeToRawUnicode
Utf8DecodeToRawUnicodeUI
UrlDecode
UnCamelCase

Is chanse to fix it ?

For example
{$IFDEF FPC}
SetLength(result,PInteger(PtrInt(S)-sizeof(pointer))^*2);
{$ELSE}
SetLength(result,PInteger(PtrInt(S)-sizeof(integer))^*2);
{$ENDIF FPC}

Always exist this difference ? Where read about fpc string ?
I try find "StrRec" definiction in FPC but nothing about this

I read this
http://wiki.freepascal.org/Character_an … AnsiString
On picture is 4B for length and othing about 64 bit  sad



I would like to run 64 bit ,
I will send more comments to this smile
It is ok ?

#26 Re: mORMot 1 » Demo application + some question / remarks. » 2015-06-12 22:27:28

My patch for lazarus version "synopse-sqlite-demo"
this project works identical in Delphi XE5 , and lazarus 32bit [this same code] smile (it is good news)
http://pastebin.com/9mj70KqE

bad news:
"synopse-sqlite-demo" use mORMotUI , but this unit have few problems in lazarus , I must create other patch wink

Other problem to comments in "mORMoti18n.pas"
In mORMoti18n.pas in comment block is "(* . msg)" , and this "completely smashes the code"

#27 Re: mORMot 1 » Demo application + some question / remarks. » 2015-06-12 10:48:45

Parametr fIds  is wrong type , please change
fIds: TIntegerDynArray;
to
fIds: TIDDynArray;

#28 Re: mORMot 1 » Using official MongoDB Connection String » 2015-06-11 23:11:20

In my opinion it is good idea but .....

... sometimes something you need do it yourself
it is open source wink

#29 Re: mORMot 1 » 200 tests for TSynTable failed in TestSQL3.dpr » 2015-06-11 22:44:16

I have similar

"TSynTable: 100/458 FAILED 8.78ms"

For example
20150612 00312207 fail  TTestLowLevelCommon(00303548) Low level common: TSynTable "" stack trace API 0046C050 004720FC 00470645 00523DD1 00471419 00563959
20150612 00312207 fail  #57  stack trace API 0046C432 00470630 00523DD1 00471419 00563959

My Windows  (64bit 8.1) code page is 1250

I use lazarus i386  build over fpcup

------------------------------------------------------------------------
PATH=c:\windows\system32;c:\windows;C:\Program Files\TortoiseSVN\bin
set

fpcup ^
    --noconfirm --verbose ^
    --fpcURL=trunk --lazURL=trunk ^
    --fpcopt="-g -gl -O1" --lazopt="-g -gl -O1" ^
    --installdir=d:\fpc32  --lazlinkname=LAZ_32  ^
    --lazrevision=49313  ^
    --fpcrevision=31024  ^
    --logfilename=fpc32.log
------------------------------------------------------------------------

First fail is in SynSelfTest.pas

// test TSynTableVariantType
      rec := T.Data;  // << HERE

In function TSynTable.Data in line "VValue := DefaultRecordData" is exception   
------------------------------------------------------------------------

In Delphi XE5 all is correct !!!

Board footer

Powered by FluxBB