Problem mORMotMVC with Delphi 10.4

tbo · 2020-07-11 11:54:59

Hallo!

mORMot 1.18.6085, Windows 10

I have a strange problem with mORMotMVC and Delphi 10.4. The problem can also be seen in example "30 - MVC Server". It crashes after takeoff.

Immediately after starting my server I get the following error message: Fatal exception EAccessViolation raised with message Zugriffsverletzung bei Adresse 0053B770 in Modul 'VGServer.exe'. Lesen von Adresse 001A4000.

In my program I can reproduce the following behaviour.

This function declaration works:
procedure ArticleBuy(var pmvIsUserLoggedIn: Boolean; out pmoSelectedArticle: TSQLArtikel);

The error comes when I change the function declaration as follows:
procedure ArticleBuy(pmArtikelID: TID; var pmvIsUserLoggedIn: Boolean; out pmoSelectedArticle: TSQLArtikel);

The behaviour can only be observed under Delphi 10.4. Under Delphi 10.3.3 it still works without problems.

I find following entry in log file:
000000000000DEF3 0 trace u_VGServerMain.TVGSRestServer(02f4ba50) BeginCurrentThread(THttpApiServer) root=Shop ThreadID=3f94 ThreadCount=15
000000000000DF4E ! EXCOS EAccessViolation (c0000005) [] at 53b770 stack trace API 57d63e 40b56c 77d25224 77d12b26 6f133e 6f0508 86cd54 8867db 886ced 895117 77bd0419 77d066dd 77d066ad
000000000000DFCD ! + u_VGServerMain.TVGSHttpServer(027bbb10).Destroy

With best regards
Thomas

ab · 2020-07-11 12:36:20

Please enable the detailed .map debugging information and check the exact stack trace with source file names and line numbers.
Or at least run it with the debugger and check the stack trace.

Sadly, I don't have Delphi 10.4 so I can't reproduce it.
As you wrote, I don't see any issue with Delphi 10.3.3 CE.

tbo · 2020-07-11 19:54:55

Thanks for your help.

When I do a DEBUG build with Delphi 10.4, there is NO error! When I do a RELEASE build, I get the following error message:
000000000000C757 ! EXCOS EAccessViolation (c0000005) [] at 53b770 SynCommons.MoveX87 stack trace API 6f133a mORMot.TInterfaceFactory.Create 6f0504 mORMot.TInterfaceFactory.Get 86cd44 mORMotMVC.TMVCApplication.Start 8867c3 u_VGServerMain.TVGServerMain.Create 886d29 u_VGServerDaemon.TVGServerDaemon.Start
000000000004B051 ! EXCOS EAccessViolation (c0000005) [] at 53b770 SynCommons.MoveX87 stack trace API

Fatal exception EAccessViolation raised with message Zugriffsverletzung bei Adresse 0053B770 in Modul 'VGServer.exe'. Lesen von Adresse 001A4000

The error occurs only when I add the parameter "pmArtikelID: TID" to the function in the example. Otherwise it also works with Delphi 10.4 in RELEASE build.

With Delphi 10.3.3 it works with the DEBUG and the RELEASE build.

If I can do more, please say so.

With best regards
Thomas

Last edited by tbo (2020-07-11 19:56:00)

ab · 2020-07-12 11:00:56

Isn't any line number in the stack trace?
Could you identify which line of TInterfaceFactory.Create is involved at address $6f0504 ?

tbo · 2020-07-12 13:58:08

ab wrote:

Isn't any line number in the stack trace?

Unfortunately I get no line numbers displayed. I rarely use the debugger and I don't need more than F7, F8, F9. Let me try it this way.

This is the line (54217) in my version of the "mORMot.pas" unit where the problems start.

case ValueType of
  smvInteger, smvCardinal, smvInt64:
    if TJSONCustomParserRTTI.TypeNameToSimpleBinary(

And since the error only occurs with the addition of the parameter "pmArticleID: TID", it might even be a useful result.

Then I continue with F7 and F8:

> TSynAnsiConvert.AnsiBufferToRawUTF8()
> TSynTempBuffer.Init()
> TSynAnsiFixedWidth.AnsiBufferToUTF8()
  Source: 'TID'#8'¸e', SourceChars: 3, NoTrailingZero: False
> TSynTempBuffer.Done()
> SynCommons.FastSetString(), line number 21211
  --> MoveFast()

After that I cannot continue working in the IDE. It stands still. I have to abort the process.

Hope this helps. If I can help in any other way, please ask.

With best regards
Thomas

ab · 2020-07-13 08:03:26

I don't know what happens...
Perhaps some Delphi 10.4 optimization issue....
Please try https://synopse.info/fossil/info/832b1be8bb

tbo · 2020-07-13 11:06:08

Hallo!

Thanks for the work. Unfortunately, the problem is not solved. The error occurs both in my program and in the example "30 - MVC Server" (only in RELEASE build).

It's no problem for me. I can continue working with older Delphi versions. If other users report bugs related to Delphi 10.4, you should keep the problem in mind. From my past experience I should have known better, never trust Delphi x.0 versions. But I couldn't resist the new LSP feature.

Thanks again for the time you invest in mORMot.

With best regards
Thomas

ab · 2020-07-13 14:32:42

Sorry it didn't circumvent the problem.

Any feedback is welcome!

array81 · 2020-07-14 13:52:00

I tried to compile demo 30 both in debug mode and release mode with Delphi 10.4 and it works without error.
Also my software (based on MVC) works.
I'm not sure of framework version used but one of last build. (is there a way to know the build?)

ab · 2020-07-14 14:00:15

Check SynopseCommit.inc file.

tbo · 2020-07-14 14:37:20

Hallo!

array81 wrote:

I tried to compile demo 30 both in debug mode and release mode with Delphi 10.4 and it works without error.

Interesting! Did you mean compile, or compile and run? My Delphi 10.4 Professional Patch 1 only installed the Spring4D and DSharp libraries. No other tools or libraries are installed. Maybe you can better isolate the error and provide Arnaud with more information.

Thank you for your support.

PS: Sorry I forgot to mention that I create Win32 programs.

With best regards
Thomas

Last edited by tbo (2020-07-14 14:47:14)

array81 · 2020-07-14 16:43:21

I have compiled and ran demo 30 (win32) then use browser to browser client pages.
I have only mORMot Framework e VerySimple.Lua installed.
mORMot Framework version used: 1.18.6078

Also my application is win32 and work but I tested it only in debug mode until now.

Last edited by array81 (2020-07-14 21:13:32)

sakura · 2020-07-15 09:50:55

So, I had the time to at least get logging into the MVC demo with 10.4
After building a RELEASE version of it and starting it, it crashes on TInterfaceFactory.Create.

The log can be found at https://pastebin.com/ZxtCTe7m

Currently, I am a bit flabbergasted as to why that happens though.

@Arnaud, as written in the Email, you may get remote access to a VM with Delphi 10.4 and that demo to fiddle arround.

Regards,
Daniel

array81 · 2020-07-15 20:45:15

OK. I can confirm there is a problem with demo 30 and last version of framework.

With version 1.18.6078 the demo work without problem.
With version 1.18.6089 the demo crash after run.

Test on Delphi 10.4 in release mode and win32.

Last edited by array81 (2020-07-15 20:45:31)

sakura · 2020-07-17 08:53:07

@ab - any idea what we could try to solve this problem? Would you like remote access to a 10.4 instance?

Regards,
Daniel

P.S.: Personally, I can wait to see you @EKON24 in November :-D

ab · 2020-07-17 10:12:13

@array81

I don't see any difference between 1.18.6078 and 1.18.6089 which may be the reason of this crash...
Do you have any clue? Can you refine to a single commit?

sakura · 2020-07-17 11:53:45

@ab - I cannot confirm 1.18.6078 working.

I have tried back to version 1.18.5749 and the problem persists. It seems to have to do with the SynCommons.MoveX87 method.

Log from the revision: https://pastebin.com/hWC7d3Lc

tbo · 2020-07-17 12:52:25

sakura wrote:

It seems to have to do with the SynCommons.MoveX87 method.

I have replaced "MoveFast := @MoveX87" by "MoveFast := @System.Move" and the error still remains.

I have tested version 1.18.5999 and there is already the error. The error depends on the constellation shown in the incoming posting.

Bis bald...
Thomas

sakura · 2020-07-17 13:23:28

I have located the problem, and it seems to be a compiler/linker problem of Delphi.

When compiling all, we get a hint:
[dcc32 Hint] SynCommons.pas(17526): H2077 Value assigned to 'TSynAnsiFixedWidth.AnsiBufferToUTF8' never used
Which is the line

  Result := Dest;

and it get's removed by compiler/linker output

I will try - at a later point this weekend or Monday- to get the compiler to simply love the line. That should resolve the problem.

sakura · 2020-07-17 13:26:05

P.S.: Quick Fix - but not compatible with older compilers like Delphi 7:

Exit(Dest);

Replace that line with

ab · 2020-07-17 14:12:53

Buggy Delphi compiler...
Please try https://synopse.info/fossil/info/c2a35d8687

No other compiler hint reported? There are several other identical patterns in the code. Perhaps the "goto" in this method made the compiler especially unhappy.

Thanks a lot for the investigation!
Perhaps worth a QC report to Embarcadero.

sakura · 2020-07-17 17:18:10

ab wrote:

Buggy Delphi compiler...

Yeah, not expected, but sh*t happens.

ab wrote:

Please try https://synopse.info/fossil/info/c2a35d8687

Will do later. :-)

ab wrote:

No other compiler hint reported? There are several other identical patterns in the code. Perhaps the "goto" in this method made the compiler especially unhappy.

Will check, don't think so. A few came from the sample itself.

ab wrote:

Thanks a lot for the investigation!

Helps me too. ;-)

ab wrote:

Perhaps worth a QC report to Embarcadero.

Definitely. Will try to simplify and do just that.

Have a nice weekend,
Daniel

ab · 2020-07-18 10:09:01

Latest 10.4 update 2 is unlikely to solve this issue.

It seems to have introduced some new regressions: https://en.delphipraxis.net/topic/3147- … available/

sakura · 2020-07-19 13:33:06

Nope, problem still exists. However, a nicer fix for your particular problem exists as well.

Replace the very first else with a semicolon --->

  if (self=nil) or (Dest=nil) then begin
    Result := nil;
    Exit;
  end else
  if (Source<>nil) and (SourceChars>0) then begin
...

change to

  if (self=nil) or (Dest=nil) then begin
    Result := nil;
    Exit;
  end;
  if (Source<>nil) and (SourceChars>0) then begin
...

And the problem is gone as well. I am trying to minimize the problem and will submit that to the team on Monday.

Regards,
Daniel

sakura · 2020-07-19 13:56:37

So, registered at the quality portal: https://quality.embarcadero.com/browse/RSP-30088

ab · 2020-07-19 20:06:00

It is an awful Delphi bug which should impact a lot of user code - not only mORMot.
I guess this is safer to wait for a fix (in patch 3) and avoid using 10.4 until stabilized.

It reminds me of https://blog.synopse.info/?post/2017/03 … C64-broken
Not sure the Quality and Testing did increase since that time...

Thanks a lot for creating a QC entry.
Hope they take it into consideration.

sakura · 2020-07-20 05:41:51

I have opened up that bug for discussion at the Delphi-PRAXiS as well, hoping to get some traction: https://en.delphipraxis.net/topic/3155- … elphi-104/

As for your question, whether any hints are left, yes one, at first glance looking alike, but this time, I believe it to be correct.

In mORMotService.pas, method function TService.GetControlHandler: TServiceControlHandler; (line 1143)

It says
[dcc32 Hint] mORMotService.pas(1170): H2077 Value assigned to 'TService.GetControlHandler' never used

However, you have a non-conditional EXIT in line 1154, so the code after that shall never be reached.

Please review ;-)

P.S.: Last, but not least, with the latest changes to mORMot, it works fine under Delphi 10.4 :-) As we are currently just developing, with a release years ahead, I feel okay continuing to use 10.4 ;-)

Last edited by sakura (2020-07-20 05:46:33)

ab · 2020-07-20 07:57:55

Very nice!
Let's hope indeed that a hint is enough to identify potential misbehavior.

I tried to fix Delphi 10.4 unexpected hint with https://synopse.info/fossil/info/7572d7d83c

Thanks a lot for the feedback.

sakura · 2020-07-20 08:12:07

Well, I compiled our project and I get a perfect result now, in Delphi 10.4

Hints: 0
Warnings: 0
Errors: what's that ;-)

ab · 2020-07-20 08:30:13

Great!

I have written a blog article to give some feedback to users, since they are likely to use Delphi 10.4 - our survey stated that most users try to keep in synch with the latest Delphi revision.

https://blog.synopse.info/?post/2020/07 … elphi-10.4

sakura · 2020-07-20 08:44:04

Perfekt and thanks for the flowers - see you in Düsseldorf in November, I hope :-)

ab · 2020-07-20 09:24:32

Prost!

macfly · 2020-07-20 18:27:45

https://en.delphipraxis.net...
"patch release before holidays" syndrome...

@ab

I think you can cancel the twitter account creation.

After this post, Embarcadero will no longer want to sponsor the sponsorship of mORMot.

ab · 2020-07-20 20:38:18

So sad... I won't tweet...

sakura · 2020-07-30 09:34:19

Bug is opened, finally :-)

ab · 2020-09-03 06:35:26

Issue has been closed as "Expected behavior" in the JIRA events...
But in practice, sakura reported that the latest 10.4.1 release was not showing the problem any more!

So the issue has been fixed.
Don't use Delphi 10.4 but the 10.4.1 branch.

sakura · 2020-09-03 07:27:31

ab wrote:

Don't use Delphi 10.4 but the 10.4.1 branch.

I'd say wait for 10.5 (or 10.5.1?) - Anyways, 10.4.1 is way better than 10.4, but as far as I had time to play around, I'd still wait. 10.4 changed so many things - usually for the better - but they all still need to learn to play nicely with each other.

missionhq · 2022-12-09 04:24:24

Just for feedback, it seems this issue may have re-appeared in Delphi v11.1
Using a TDynArray to manage an an array of Int64's which I Pop items from, it worked in Debug but Release gives the "EAccessViolation (c0000005) [] at c4c540 SynCommons.MoveX87"

var
  lId: int64;
begin
  if fMyIDsDynArray.Pop(lID) then ...

but changing to this, works (no FastMove)...

if fMyIDsDynArray.Count > 0 then
begin
  lId:= pInt64(fMyIDsDynArray.ElemPtr(0))^;
  fMyIDsDynArray.Delete(0);
  ...

HTH...

ab · 2022-12-09 11:18:42

So is it a mORMot bug or a Delphi 11.1 problem?

missionhq · 2022-12-11 03:29:54

Hi ab
Did some investigating over the weekend going through the generated code
The problem is occurring in TDynArray.ElemPtr - and only seems to be when the TDynArray.ElemPtr is inlined, optimised, and I'm guessing it's also depending on something like whether parameters are passed on the stack or by register to the procedure using it - not sure but it doesn't happen in all scenarios.

It's the use of the label and the Goto that are confusing the compiler...

function TDynArray.ElemPtr(index: PtrInt): pointer;
label ok;
var c: PtrUInt;
begin // very efficient code on FPC and modern Delphi
  result := pointer(fValue);
  if result=nil then
    exit;
  result := PPointer(result)^;
  if result=nil then
    exit;
  c := PtrUInt(fCountP);
  if c<>0 then begin
    if PtrUInt(index)<PCardinal(c)^ then              { <== only here does 'edi' register get populated with our "index" }
ok:   inc(PByte(result),PtrUInt(index)*ElemSize) else { <== assumes "index" is now in 'edi' register}
      result := nil
  end else
    {$ifdef FPC}
    if PtrUInt(index)<=PPtrUInt(PtrUInt(result)-_DALEN)^ then
    {$else}
    if PtrUInt(index)<PPtrUInt(PtrUInt(result)-_DALEN)^ then
    {$endif FPC}
      goto ok else                                    { <== Goto 'ok' but 'edi' has not been populated with our 'index' }
      result := nil;
end;

I guess you would say this is a Delphi Optimisation problem - I expanded out the 'if' statement to not use the Goto and this fixed the problem - slightly more code but just as efficient I think.

function TDynArray.ElemPtr(index: PtrInt): pointer;
var c: PtrUInt;
begin // very efficient code on FPC and modern Delphi
  result := pointer(fValue);
  if result=nil then
    exit;
  result := PPointer(result)^;
  if result=nil then
    exit;
  c := PtrUInt(fCountP);
  if c<>0 then
  begin
    if PtrUInt(index)<PCardinal(c)^ then
      inc(PByte(result),PtrUInt(index)*ElemSize)
    else
      result := nil
  end
  else
   {$ifdef FPC}
    if PtrUInt(index)<=PPtrUInt(PtrUInt(result)-_DALEN)^ then
   {$else}
    if PtrUInt(index)<PPtrUInt(PtrUInt(result)-_DALEN)^ then
   {$endif FPC}
      inc(PByte(result),PtrUInt(index)*ElemSize)
    else
      result := nil
end;

You could say this is a bit of a 'gotcha' but at the same time I can see the Goto jumping to inside an if statement being a bit of an odd scenario for it

ab · 2022-12-11 10:18:00

What is weird is that I am not able to reproduce the problem with the Delphi 11.1 compiler.
Anyway, it is clearly a Delphi optimizer bug. Using a goto inside an inlined function makes the code shorter and more efficient.
FPC has no troubles inlining it.

I will remove the label on Delphi...
Please try https://synopse.info/fossil/info/c2c13a48bc for mORMOt 1.
and https://github.com/synopse/mORMot2/commit/46b74996 for mORMOt 2.

missionhq · 2022-12-11 23:55:12

Well back in the office now (Australia time ) and am also reproducing the problem in Delphi 11.1 and 11.2 too (was using 10.4 earlier).
Not sure if it's my setup, or just down to the usage case of the ElemPtr in certain situations.
I know the above has fixed things but would be interested to know if this test program reproduces the problem for you in a release build?

program MoveFastx87;
{$APPTYPE CONSOLE}
uses
  SysUtils, SynCommons;
var
  anArray: TInt64DynArray;
  aDynArray: TDynArray;
  aInt64: Int64;
begin
  try
    aDynArray.Init(TypeInfo(TInt64DynArray), anArray);
    aInt64:= 1;
    aDynArray.Add(aInt64);
    if aDynArray.Pop(aInt64) then
      writeln ('Popped ', aInt64);
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
  readln;
end.

flydev · 2022-12-12 08:40:08

I tested your sample and I could not reproduce that in 11.0, 11.1 and 11.2, debug or release mode, in 32bit or 64bit. No exception, and element popped.
=> mormot v1 & v2..

Last edited by flydev (2022-12-12 08:42:54)

missionhq · 2022-12-12 09:21:43

Well that was unexpected!
I've tried this program on 3 different machines running different version of Delphi and all 3 are failing in release build.
Could you confirm you're using the older ver of mormot (before ab's fix), and I'm going to see what my 3x installs all have in common - maybe it's something in my config that's doing this.
BTW, I'm actually running a Debug build but with {$O+} and {$O-} around "procedure TDynArray.ElemMoveTo" implementation - still shows the problem but easier to debug.

missionhq · 2022-12-12 10:06:30

Forget this last post - forgot to turn on optimisation around ElemMoveTo. So the problem did exist up until ab's fix. I'll leave it at one of those quirks I have to live with unless I come across anything more...

Well it seems i have found the solution, if not the problem.
I was using an older version of Mormot1 dated may 2022.

Last edited by missionhq (2022-12-12 10:43:32)

flydev · 2022-12-12 10:48:11

missionhq wrote:

Could you confirm you're using the older ver of mormot (before ab's fix)

Yes, the version was from the commits of Nov 22, 2022. Even with Debug/O+, no exception.

@missionhq: did a reset, cloned from commit 74cfbc4 then with debug O+ the exception is triggered on delphi 11.2, I will try when I am back on 11.0 and 11.1.

EAccessViolation: Access violation at address 00941F00 in module 'Project01.exe'. Read of address 076F5980

Last edited by flydev (2022-12-12 11:02:30)

missionhq · 2022-12-12 10:59:40

Cheers flydev - it's so weird, predictable on 3 machines for me and I can see that the inlined ElemPtr is returning an invalid pointer. It bugs me when I can't get to the root cause of a problem but I think I'm going to have to give up on this one

flydev · 2022-12-12 11:04:03

i edited my last post @missionhq, I confirm the bug at least on 11.2.

missionhq · 2022-12-12 11:16:13

Well at least I feel better knowing it's not just me
I think we can put it down to a compiler bug and know that the fix ab committed is worth having, at leas for Delphi.
Thanks again for your help!

ab · 2022-12-12 15:29:45

I tried to make a full review of the mORMOt 2 source code to ensure no such goto is inlined on Delphi.

https://github.com/synopse/mORMot2/commit/10d9a1b4

Should help stability with such Delphi regressions.

mORMot Open Source

#1 2020-07-11 11:54:59

Problem mORMotMVC with Delphi 10.4

#2 2020-07-11 12:36:20

Re: Problem mORMotMVC with Delphi 10.4

#3 2020-07-11 19:54:55

Re: Problem mORMotMVC with Delphi 10.4

#4 2020-07-12 11:00:56

Re: Problem mORMotMVC with Delphi 10.4

#5 2020-07-12 13:58:08

Re: Problem mORMotMVC with Delphi 10.4

#6 2020-07-13 08:03:26

Re: Problem mORMotMVC with Delphi 10.4

#7 2020-07-13 11:06:08

Re: Problem mORMotMVC with Delphi 10.4

#8 2020-07-13 14:32:42

Re: Problem mORMotMVC with Delphi 10.4

#9 2020-07-14 13:52:00

Re: Problem mORMotMVC with Delphi 10.4

#10 2020-07-14 14:00:15

Re: Problem mORMotMVC with Delphi 10.4

#11 2020-07-14 14:37:20

Re: Problem mORMotMVC with Delphi 10.4

#12 2020-07-14 16:43:21

Re: Problem mORMotMVC with Delphi 10.4

#13 2020-07-15 09:50:55

Re: Problem mORMotMVC with Delphi 10.4

#14 2020-07-15 20:45:15

Re: Problem mORMotMVC with Delphi 10.4

#15 2020-07-17 08:53:07

Re: Problem mORMotMVC with Delphi 10.4

#16 2020-07-17 10:12:13

Re: Problem mORMotMVC with Delphi 10.4

#17 2020-07-17 11:53:45

Re: Problem mORMotMVC with Delphi 10.4

#18 2020-07-17 12:52:25

Re: Problem mORMotMVC with Delphi 10.4

#19 2020-07-17 13:23:28

Re: Problem mORMotMVC with Delphi 10.4

#20 2020-07-17 13:26:05

Re: Problem mORMotMVC with Delphi 10.4

#21 2020-07-17 14:12:53

Re: Problem mORMotMVC with Delphi 10.4

#22 2020-07-17 17:18:10

Re: Problem mORMotMVC with Delphi 10.4

#23 2020-07-18 10:09:01

Re: Problem mORMotMVC with Delphi 10.4

#24 2020-07-19 13:33:06

Re: Problem mORMotMVC with Delphi 10.4

#25 2020-07-19 13:56:37

Re: Problem mORMotMVC with Delphi 10.4

#26 2020-07-19 20:06:00

Re: Problem mORMotMVC with Delphi 10.4

#27 2020-07-20 05:41:51

Re: Problem mORMotMVC with Delphi 10.4

#28 2020-07-20 07:57:55

Re: Problem mORMotMVC with Delphi 10.4

#29 2020-07-20 08:12:07

Re: Problem mORMotMVC with Delphi 10.4

#30 2020-07-20 08:30:13

Re: Problem mORMotMVC with Delphi 10.4

#31 2020-07-20 08:44:04

Re: Problem mORMotMVC with Delphi 10.4

#32 2020-07-20 09:24:32

Re: Problem mORMotMVC with Delphi 10.4

#33 2020-07-20 18:27:45

Re: Problem mORMotMVC with Delphi 10.4

#34 2020-07-20 20:38:18

Re: Problem mORMotMVC with Delphi 10.4

#35 2020-07-30 09:34:19

Re: Problem mORMotMVC with Delphi 10.4

#36 2020-09-03 06:35:26

Re: Problem mORMotMVC with Delphi 10.4

#37 2020-09-03 07:27:31

Re: Problem mORMotMVC with Delphi 10.4

#38 2022-12-09 04:24:24

Re: Problem mORMotMVC with Delphi 10.4

#39 2022-12-09 11:18:42

Re: Problem mORMotMVC with Delphi 10.4

#40 2022-12-11 03:29:54