#1 Re: mORMot 2 » Performance of Mormot Collections vs. Spring v2 Collections » 2023-11-27 10:37:54

I'm aware I'm not speaking for the whole mOMRot community, but in our project we can get without generics. But without mORMot using Delphi/Pascal were impossible.

Thats all what I can say to this topic from side.

#2 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-11-08 10:41:39

In TRttiCustomList.DoRegister Info.KIND has rkBOOL.
But Instance (Result) will not set after GlobalClass.Create and given to AddToPairs(result, Info);

In AddToPairs I can't see from where Instance is getting his KIND (it's just rkUnknown) until inc(Counts[Instance.Kind]);

#3 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-11-07 15:42:06

Yes, with your last patch it works as expected.

Yes, it was rkUnknown.

#4 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-11-07 10:45:54

I've tested and changed code in a very unprofessional way and maybe I'm lost, but I want to share my guess founded in mormon.core.rtti:

The address of the CS will be changed through line 8730 in TRttiCustomList.AddToPairs:

inc(Counts[Instance.Kind]);

After that the RegisterSafe.UnLock; in line 9002 goes with the wrong Pointer address in mutex_unlock_common and therefore it results in a segfault.

If I comment line 8730 the segfault is away and also my real app is running under FreeBSD 13.2 as under Win/Linux.

But I don't know, what's the meaning of line 9002 and the drawbacks of that.

#5 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-11-02 09:07:26

Yes, I do always a clean & build under lazarus. But mormot.core.linux.inc? Do you mean mormot.core.os.posix.inc?

#6 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-11-01 16:52:12

Please look at the stack trace from lazarus. Is #1 und #2 double or normal?
I'm not able to debug constructive (perhaps because of inlining).

#0 mutex_unlock_common(0x9013d1708, false, 0x0) at /usr/src/lib/libthr/thread/thr_mutex.c:976
#1 CTHREADS_$$_CLEAVECRITICALSECTION$formal at :0
#2 SYSTEM_$$_LEAVECRITICALSECTION$POINTER at :0
#3 DOREGISTER(0x801cef6a0, 0xe3d8d8) at ../../mORMot2/src/core/mormot.core.rtti.pas:8630
#4 SETGLOBALCLASS(0x801cef6a0, 0xf8af30) at ../../mORMot2/src/core/mormot.core.rtti.pas:8751
#5 INITIALIZEUNIT at ../../mORMot2/src/core/mormot.core.json.pas:11687
#6 MORMOT.CORE.JSON_$$_init$ at ../../mORMot2/src/core/mormot.core.json.pas:11699
#7 fpc_initializeunits at :0
#8 main at SOneSrv2.dpr:551

#7 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-11-01 14:34:19

Ok, than no progress at all:

mutex_unlock_common (m=0x90134b708, cv=false, mtx_defer=0x0)
    at /usr/src/lib/libthr/thread/thr_mutex.c:976
976        if (__predict_false(PMUTEX_OWNER_ID(m) != id))
(gdb) backtrace 10
#0  mutex_unlock_common (m=0x90134b708, cv=false, mtx_defer=0x0)
    at /usr/src/lib/libthr/thread/thr_mutex.c:976
#1  0x0000000000434b49 in CTHREADS_$$_CLEAVECRITICALSECTION$formal ()
#2  0x00000000004254ca in SYSTEM_$$_LEAVECRITICALSECTION$POINTER ()
#3  0x00000000004dcae3 in DOREGISTER (this=0x801de8370, INFO=0xeb7b68)
    at ../../mORMot2/src/core/mormot.core.rtti.pas:8630
#4  0x00000000004dd357 in SETGLOBALCLASS (this=0x801de8370,
    RTTICLASS=0x1004fa0) at ../../mORMot2/src/core/mormot.core.rtti.pas:8751
#5  0x000000000094f8bd in INITIALIZEUNIT ()
    at ../../mORMot2/src/core/mormot.core.json.pas:11687
#6  0x000000000094f979 in MORMOT.CORE.JSON_$$_init$ ()
    at ../../mORMot2/src/core/mormot.core.json.pas:11699
#7  0x0000000000421652 in fpc_initializeunits ()
#8  0x0000000000403b9f in main () at SOneSrv2.dpr:551

If a do a breakpoint in thr_mutex.c for mutex_lock_common and mutex_unlock_common, than I can see, that this function will be called.
Are you sure, a confusion of libc and libthr functions exists?

#8 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-11-01 08:22:20

I've applied your commit, but
how can I remove the reference to clib in mormot.core.os.posix.inc? There are other functions (e.g. clock_gettime) needing clib.
Do I need cthreads anymore? We have disabled MemoryManager for testing.

#9 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-11-01 07:39:59

@DonAlfredo
This patch is already included in R13.2 and it couldn't be an issue on linker side.

Thanks a lot for your hint!

#10 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-10-31 18:32:20

@DonAlfredo
Thanks for this link. I've alfredy (joke) read about this issue. But the patch is fro 2019 and I've the R13.2 from Apr 2023 and should the backtrace not shown something of libc in such a case? But I will compare the patch with the current source files and perhaps that will help.

#11 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-10-31 17:14:52

with TLightLock app hangs when leaving InizializeUnit in mormot.core.log. After stepping over end; Debugger shows end; of FinalizeUnit.

Assembler show this:
0000000000421472 eb55                     jmp    0x4214c9 <fpc_initializeunits+121>

0000000000421474 48c745f800000000         movq   $0x0,-0x8(%rbp)
000000000042147C 488b45f8                 mov    -0x8(%rbp),%rax
0000000000421480 488d4001                 lea    0x1(%rax),%rax
0000000000421484 488945f8                 mov    %rax,-0x8(%rbp)
0000000000421488 488b45f8                 mov    -0x8(%rbp),%rax
000000000042148C 48c1e004                 shl    $0x4,%rax
0000000000421490 488d15a99ba100           lea    0xa19ba9(%rip),%rdx        # 0xe3b040 <INITFINAL>
0000000000421497 48833c0200               cmpq   $0x0,(%rdx,%rax,1)
000000000042149C 7502                     jne    0x4214a0 <fpc_initializeunits+80>
000000000042149E eb12                     jmp    0x4214b2 <fpc_initializeunits+98>
00000000004214A0 488b45f8                 mov    -0x8(%rbp),%rax
00000000004214A4 48c1e004                 shl    $0x4,%rax
00000000004214A8 488d15919ba100           lea    0xa19b91(%rip),%rdx        # 0xe3b040 <INITFINAL>
00000000004214AF ff1402                   call   *(%rdx,%rax,1)
00000000004214B2 488d05879ba100           lea    0xa19b87(%rip),%rax        # 0xe3b040 <INITFINAL>
00000000004214B9 488b55f8                 mov    -0x8(%rbp),%rdx
00000000004214BD 48895008                 mov    %rdx,0x8(%rax)
00000000004214C1 483b5df8                 cmp    -0x8(%rbp),%rbx
00000000004214C5 7602                     jbe    0x4214c9 <fpc_initializeunits+121>
00000000004214C7 ebb3                     jmp    0x42147c <fpc_initializeunits+44>
00000000004214C9 488d0550aea100           lea    0xa1ae50(%rip),%rax        # 0xe3c320 <TC_$SYSTEM_$$_INITPROC>
00000000004214D0 48833800                 cmpq   $0x0,(%rax)
00000000004214D4 7502                     jne    0x4214d8 <fpc_initializeunits+136>
00000000004214D6 eb09                     jmp    0x4214e1 <fpc_initializeunits+145>
00000000004214D8 488d0541aea100           lea    0xa1ae41(%rip),%rax        # 0xe3c320 <TC_$SYSTEM_$$_INITPROC>
00000000004214DF ff10                     call   *(%rax)
00000000004214E1 488b5df0                 mov    -0x10(%rbp),%rbx
00000000004214E5 4889ec                   mov    %rbp,%rsp
00000000004214E8 5d                       pop    %rbp

#12 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-10-31 16:59:38

Thanks, with TLightLock app hangs again.

With TOSLightLock segfault again:

#0  mutex_unlock_common (m=0x90134b708, cv=false, mtx_defer=0x0)
    at /usr/src/lib/libthr/thread/thr_mutex.c:976
#1  0x00000000004dcae3 in DOREGISTER (this=0x801de8370, INFO=0xeb7b68)
    at ../../mORMot2/src/core/mormot.core.rtti.pas:8630
#2  0x00000000004dd357 in SETGLOBALCLASS (this=0x801de8370, RTTICLASS=0x1004fa0)
    at ../../mORMot2/src/core/mormot.core.rtti.pas:8751
#3  0x000000000094f8bd in INITIALIZEUNIT ()
    at ../../mORMot2/src/core/mormot.core.json.pas:11687
#4  0x000000000094f979 in MORMOT.CORE.JSON_$$_init$ ()
    at ../../mORMot2/src/core/mormot.core.json.pas:11699
#5  0x0000000000421652 in fpc_initializeunits ()
#6  0x0000000000403b9f in main () at SOneSrv2.dpr:551

Registers:
rax            0x7fffffff          2147483647
rbx            0x90134b708         38674937608
rcx            0x801349008         34379960328
rdx            0x0                 0
rsi            0x0                 0
rdi            0x90134b708         38674937608
rbp            0x7fffffffd540      0x7fffffffd540
rsp            0x7fffffffd4e0      0x7fffffffd4e0
r8             0x6c6f6f42          1819242306
r9             0x80000             524288
r10            0x4a7b48            4881224
r11            0x0                 0
r12            0x7fffffffd9c0      140737488345536
r13            0x0                 0
r14            0x189be             100798
r15            0x801612000         34382880768
rip            0x800f06a33         0x800f06a33 <mutex_unlock_common+51>
eflags         0x10206             [ PF IF RF ]
cs             0x43                67
ss             0x3b                59
ds             0x3b                59
es             0x3b                59
fs             0x13                19
gs             0x1b                27
fs_base        0x800ef1120         34375405856

#13 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-10-31 13:43:19

Now with TLightLock my real app hangs at starting and I have to find out where.

#14 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-10-31 08:18:31

Please give me some more time to find out next issues, because the main issue resists and just a first step of initialization of the units seems to work…

#16 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-10-30 20:11:52

Yes, I think that's the same issue like the docker one. But it's just my intension, because the av is released at the same time in the same unit initializations.

I've tested your new commit (thank's a lot for your investigation), but got the same segfault as without it:

mutex_unlock_common (m=0x901398708, cv=false, mtx_defer=0x0)
    at /usr/src/lib/libthr/thread/thr_mutex.c:976
976        if (__predict_false(PMUTEX_OWNER_ID(m) != id))
(gdb) bt 10
#0  mutex_unlock_common (m=0x901398708, cv=false, mtx_defer=0x0)
    at /usr/src/lib/libthr/thread/thr_mutex.c:976
#1  0x0000000000431c19 in CTHREADS_$$_CLEAVECRITICALSECTION$formal ()
#2  0x000000000042560a in SYSTEM_$$_LEAVECRITICALSECTION$POINTER ()
#3  0x00000000004d9ea3 in DOREGISTER (this=0x801637188, INFO=0xeb4b58)
    at ../../mORMot2/src/core/mormot.core.rtti.pas:8629
#4  0x00000000004da717 in SETGLOBALCLASS (this=0x801637188, RTTICLASS=0x1001a30)
    at ../../mORMot2/src/core/mormot.core.rtti.pas:8750
#5  0x000000000094ca5d in INITIALIZEUNIT () at ../../mORMot2/src/core/mormot.core.json.pas:11687
#6  0x000000000094cb19 in MORMOT.CORE.JSON_$$_init$ ()
    at ../../mORMot2/src/core/mormot.core.json.pas:11699
#7  0x0000000000421792 in fpc_initializeunits ()
#8  0x0000000000403cdf in main () at SOneSrv2.dpr:550

#17 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-10-30 17:44:42

the segfault comes from thr_mutex.c when the CS will be unlocked a second time:

/*
* Check if the running thread is not the owner of the mutex.
*/
if (__predict_false(PMUTEX_OWNER_ID(m) != id)) <-- here
  return (EPERM);

This comes from the second step of a for loop in mormot.core.rtti which calls

function TRttiCustomList.DoRegister(Info: PRttiInfo): TRttiCustom;

and here the finally RegisterSafe.UnLock;
calls in thr_mutex.c

mutex_unlock_common(struct pthread_mutex *m, bool cv, int *mtx_defer)

stack trace:
#0  mutex_unlock_common (m=0x90138e708, cv=false, mtx_defer=0x0) at /usr/src/lib/libthr/thread/thr_mutex.c:976
#1  0x0000000000431c19 in CTHREADS_$$_CLEAVECRITICALSECTION$formal ()
#2  0x00000000004ee651 in UNLOCK (this=...) at ../../mORMot2/src/core/mormot.core.os.pas:9003
#3  0x00000000004d9e59 in DOREGISTER (this=0x801637188, INFO=0xea9f78) at ../../mORMot2/src/core/mormot.core.rtti.pas:8629
#4  0x00000000004da677 in SETGLOBALCLASS (this=0x801637188, RTTICLASS=0xff6df0) at ../../mORMot2/src/core/mormot.core.rtti.pas:8750
#5  0x000000000094bffd in INITIALIZEUNIT () at ../../mORMot2/src/core/mormot.core.json.pas:11679
#6  0x000000000094c0b9 in MORMOT.CORE.JSON_$$_init$ () at ../../mORMot2/src/core/mormot.core.json.pas:11691
#7  0x0000000000421792 in fpc_initializeunits ()
#8  0x0000000000403cdf in main () at SOneSrv2.dpr:550

#18 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-10-30 17:14:28

I've testet with standard memory manager from fpc and with cmem.
No reallocations of large buffers of my own.
Now I've tested the two defines, but with no success.
(freebsd 13.2, fpc 3.2.2)

#19 Re: mORMot 2 » Error running docker container from ex\martin-doyle\07-HttpDockerORM » 2023-10-29 16:38:54

Same behavior as under FreeBSD.
The AV occurs after unlocking a CriticalSection. I don't think it has to do with an old libc, but with using libthr (instead of libpthread under linux).
I guess it's not an Docker issue rather than an issue of the using image.

I'm not able to find a solution atm. The only thing I've found is the initialization of the CS will be called serveral time in right same thread (the thread of the initialization of the units.
Perhaps the libthr releases a CS when it will be initialized a second time. But it's just a guess.

#20 mORMot 2 » Can't get mORMot2 running under FreeBSD13 » 2023-10-16 15:48:11

danielkuettner
Replies: 1

Although linux is our production environment I'm testing with FreeBSD perhaps as a replacement in future. My last tests some month ago were successful under Proxmox in a FreeBSD13 VM.

Now I'm testing with a real FreeBSD13 host and here I can first compile successfully my program but it comes a runtime exception after starting when mORMot2 initialize a global mutex between initialization und finalization. I've tested with fpc 3.2.0, 3.2.2 and 3.3.1.

gdb show's:

Program received signal SIGSEGV, Segmentation fault.
Address not mapped to object.
mutex_unlock_common (m=0x90139f708, cv=false, mtx_defer=0x0)
    at /usr/src/lib/libthr/thread/thr_mutex.c:973
973        if (__predict_false(PMUTEX_OWNER_ID(m) != id))

Something seems to be different under FreeBSD with /lib/libthr.so.3 compared to /lib/x86_64-linux-gnu/libpthread.so.0 under Linux.

PS:
I've also tried several versions of FreeBSD13 and I've build FreeBSD 13.2 Release from source with connected Kernel.

#21 Re: mORMot 2 » Variant issues under freebsd13 with fpc 3.2.2 » 2023-04-23 17:20:43

Thanks for your hint (after reading I remember).
After patching ncal.pas it just works with 3.2.2 under freebsd!

#22 Re: mORMot 2 » Variant issues under freebsd13 with fpc 3.2.2 » 2023-04-23 12:34:37

Following late binding calls doesn't work:

v._(0)

where v is an _ARR

This post seems related to the issue I have:
https://synopse.info/forum/viewtopic.php?pid=34034

And I know I can work around with using TDocVariant instead of
last binding.

But not using late binding is a regression.

#23 mORMot 2 » Variant issues under freebsd13 with fpc 3.2.2 » 2023-04-22 10:15:52

danielkuettner
Replies: 3

Hi Arnaud,

I could successfully compile our project, but there are runtime av's relating to variants.
Are there patches needed to run under this constellation or should it just work?

Here are a result from mORMot2tests:

#16 4E59E2A5-E8C1-46AB-90E5-F426EC9DBB1E<>4e59e2a5-e8c1-46ab-90e5-f426ec9dbb1e uuid  !  - Dmi smbios: 1 / 44 FAILED  527us
0000000000000000  ! fail  #16 4E59E2A5-E8C1-46AB-90E5-F426EC9DBB1E<>4e59e2a5-e8c1-46ab-90e5-f426ec9dbb1e uuid
000000000018FA8D  ! EXC   ESynVariant {Message:"TDocVariant.DispInvoke: invalid (1) call"} [Main] at 684f2d

Daniel

#24 Re: mORMot 1 » Fatal Error (Exception External:?) in Interfaced based Services » 2022-08-27 07:48:55

@sakura You should think about an unwanted exception in your sm and the manual inserted raise as an example for that.
Than it's interesting because I often see such external exceptions in my log files and misinterpreted these.

#25 Re: mORMot 1 » TryLoadLibray PostgreSql lib » 2022-08-25 16:07:39

Are you using 32-bit? Postgres14 comes only with a 64-bit client

#26 Re: mORMot 1 » MongoDB 6.0 » 2022-08-13 08:15:16

Nice Blog about new wire protocol! You could be also a salesman or a story-teller. Unusual for a asm optimizer smile

#27 Re: mORMot 1 » MongoDB 6.0 » 2022-08-09 11:01:23

Yes, this is could be useful in case of big inserts, I guess.

#29 Re: mORMot 1 » MongoDB 6.0 » 2022-08-07 13:15:31

Ok, thanks, that sounds better.

I understand your statements totally and I agree with them, but just some thoughts:

1.  also I use your aggregate call it doesn't get the rights count documents. There reply count are 101/34398/16795 (if no BatchSize is set).
But in SendAndGetRepliesAndFree Opaque will override every time OnEachReply is called. Here is the issue!

2. the first 101 batch is just for analyzing of the optimal batch count and I trust the algo at server side here. Because every document could have a special size I don't know the optimal BatchSize of my own database.
The second call brings than 38.000 and the last one 16.000. So there is not so much to optimize. If the batches are to large, than in a client-server architecture with many clients, the first client gets his reply some ms faster, but at costs of waiting all others.

#30 Re: mORMot 1 » MongoDB 6.0 » 2022-08-07 10:21:31

Test with aggregate has same issue as with RunCommand.

From expected DocumentCount of 51294 aggregate returns only 16905.

I think I could find out where the issue is. In SendAndGetCursor first 34398 Documents of reply are replaced by next 16905. But I miss the first 101 before 34398, that comes with the version of dc66841.

Here are my wishes:

1. I want to use RunCommand as under commit dc66841. Because I want to have full control about what is sending and what's the replies are.
2. I need your help because I don't understand all the details of SendAndGetCursor(msg, reply) especially working with BSONVariant and other details (e.g. why has BatchSize a value of 65536 -> I haven't set Batchsize at all)

#31 Re: mORMot 1 » MongoDB 6.0 » 2022-08-07 07:30:49

Sure I'll try it.

But do you want to say, RunCommand isn't supported from mORMot2 any more (after rewriting)? Although all was still working in commit dc66841?

#32 Re: mORMot 1 » MongoDB 6.0 » 2022-08-06 09:19:30

After removing $db params in my Commands I get following error for aggregates:

'{"ok":0,"errmsg":"Requested getMore on namespace ''Rottler.$cmd'', but cursor belongs to a different namespace Rottler.Persons","code":13,"codeName":"Unauthorized"}'

aggregate was:

{aggregate:"Persons",pipeline:[{$match:{PersonTypeID:0,Parents.ParentID:{$in:[903087]},live:{$ne:false}}},{$sort:{LastProcessingDate:-1}},{$project:{_id:0,ID:"$_id",Number:1,Title:1,SecondName:1,FirstName:1,DayOfBirth:1,Sex:1,PersonTypeID:1,Remarks:1,LastProcessingDate:1,CostCenter:{$arrayElemAt:[{$map:{input:{$filter:{input:"$Parents",as:"p",cond:{$eq:["$$p.PersonTypeID",1]}}},as:"cc",in:"$$cc.Number"}},0]},LoginExpiryDate:1,Alive:1,LoginName:1,Parents.ParentID:1,ContactData:1,DSV:1,FirstSalesOrder:1,LastSalesOrder:1}},{$limit:200}],cursor:{},allowDiskUse:true,maxTimeMS:30000}

Problem seems to be in:

SendAndGetRepliesAndFree->call of getMore ($cmd seems to be to needed under new wire protocol (IIRC). It was a virtual collection for OP_QUERY).

When I remove +'.$cmd' in RunCommand Collections is not stetted:
'{getMore:5068573612759429335,collection:"",batchSize:65536,$db:"Rottler"}' -> because Request.CollectionName = '' in line 2786.

Before rewriting every call with RunCommand gets one replay. The getMore request was done by myself, when requested in the reply.
The replay looks like this:
('{"cursor":{"firstBatch":[{"PersonTypeID":4,"Number":"HLD","Alive":true,...

Now replay is:
'[{"PersonTypeID":4,"Number":"HLD","Alive":true,...

If I use your implementation with getMore I tried to set DatabaseName in RunCommand like this:

BsonVariantType.GetItem(command, 'aggregate', v);
  if not VarIsEmpty(v) then
  GetDocumentsAndFree(NewCommand(
    aDatabaseName + '.' + v, command, flags), returnedvalue)

But the replies read with getMore are still not correct this way.

#33 Re: mORMot 1 » MongoDB 6.0 » 2022-08-05 21:12:47

You are right. I had to remove all my $db params.
Sorry for that.

#34 Re: mORMot 1 » MongoDB 6.0 » 2022-08-05 08:27:24

Just a hint: your commit #0ba4230 don't work with JSON-Objects e.g. listCollections with params, because $db have to be the param after listCollections and not the last one.

wrong:
'{listCollections:"Rottler",filter:{name:"revenuesPayments"},$db:"Rottler"}'

right:
'{listCollections:"Rottler",$db:"Rottler",filter:{name:"revenuesPayments"}}'

You would have ti insert the $db param in the command string that comes from the user. That was not necessary in my opinion. I would add $db param only in case of VarIsStr. In all other cases,
the user must deliver the $db param in his command to work with new protocol. So I had to update all my encapsulate methods and added the $db param without the need to change mORMot2 routines.

The AddItem added the $db param also multiple times: '{listCollections:"Rottler",filter:{name:"revenuesPayments"},$db:"Rottler", $db:"Rottler"}'

Daniel

#35 Re: mORMot 1 » MongoDB 6.0 » 2022-08-04 08:24:36

Hello Arnaud,

thanks for implementing new wire protocol. I'm testing your commit and it looks promising.

Daniel

#36 Re: mORMot 1 » MongoDB 6.0 » 2022-08-02 13:17:02

Hello Arnaud,

I've sent you an email with a new version of mormot.db.nosql.mongodb.pas.
But the PR #114 is the better one.

Daniel

#37 Re: mORMot 1 » MongoDB 6.0 » 2022-07-31 07:16:25

Some changes are necessary e.g. replacement of CursorID (not available in > 3.6).
I will send you an other version of mormot.db.nosql.mongodb.pas as soon as I've fixed last errors on my side.

#38 Re: mORMot 1 » MongoDB 6.0 » 2022-07-30 17:55:52

Some errors produced by TMongoConnection.GetReply are not real errors but comes from old handling of replies (e.g. Request.MongoRequestID has wrong value).

Even my statistic request works!

#39 Re: mORMot 1 » MongoDB 6.0 » 2022-07-30 15:53:56

Hi Arnaud,

after making some PRs further changes are necessary to work with new WIRE protocol.
But creating PRs with GitHub is too painful.
If you want, I can send you my version of mormot.db.nosql.mongodb.pas.

Atm my service starts with auth + ssl, list collections and check indexes and insert an array of documents. And all other stuff looks good from my side, because we only used RunCommand and inserting new $db field is the only change we have to do at our side.

Daniel

#40 Re: mORMot 1 » MongoDB 6.0 » 2022-07-24 07:59:05

Hi Arnaud,

my production service uses the 5.0.9 atm and I'm glad to get this issue in a test env. The little mORMot has grown to a bigger one and I wonder how you deal with it.
If I can do something, please let me know.

Daniel

#41 mORMot 1 » MongoDB 6.0 » 2022-07-23 19:13:50

danielkuettner
Replies: 23

Hi Arnaud,

with MongoDB 6.0 I get following error:

"Unsupported OP_QUERY command" -> https://www.mongodb.com/docs/v6.0/relea … es-removed

It has to be OP_MSG instead.

Same with OP_UPDATE and OP_INSERT and OP_REPLY.

Regards,
Daniel

#42 Re: mORMot 1 » Native TLS Support for mORMot 2 REST or WebSockets Servers » 2022-07-11 14:40:32

Now I've debugged in LogViewer and first, procedure THttpServerResp.Execute was called periodically, but after I've added a log entry for it, it won't called for a while and the error 10053 is also away in that time window. After it will be called, error 10053 is coming up.

#43 Re: mORMot 1 » Native TLS Support for mORMot 2 REST or WebSockets Servers » 2022-07-11 07:35:53

The header is always empty (='').

In mormot.net.sock line 3442 F.BufPtr has following value:

'HTTP/1.1 200 OK'#$D#$A'X-Powered-By: mORMot 1.18 synopse.info'#$D#$A'Server: mORMot (Windows)'#$D#$A'Content-Length: 0'#$D#$A'Content-Type: application/json; charset=UTF-8'#$D#$A'Accept-Encoding: synlz,gzip'#$D#$A'Connection: Keep-Alive'#$D#$A#$D#$A'0627055'

and size is 1024 (always).

In error case sock has following value:

($3A8, '127.0.0.1', '8091', '', '', $36EE460, nil, 30000, 91406, 134278, nil, 0, False, nlTcp, 'PUT /LogService/RemoteLog HTTP/1.1'#$D#$A'Host: 127.0.0.1:8091'#$D#$A'Accept: */*'#$D#$A'User-Agent: Mozilla/5.0 (Win; mORMot 2.0.3606 HCS)'#$D#$A'Keep-Alive: 20000'#$D#$A'Connection: Keep-Alive'#$D#$A'Content-Length: 24'#$D#$A'Content-Type: application/json; charset=UTF-8'#$D#$A#$D#$A'20220711 07315309  +    '#9'02.498.812'#$D#$A'20220711 07303732  -    02.498.840Answer Interface=200 out=15 B in 2.49s'#$D#$A'20220711 07303732 ret   '#9#9'mormot.rest.server.TRestServerRoutingRest(0a167520) {"result":[[]]}swer in=75 B'#$D#$A'20220711 07303501 call  '#9#9'uRestServerDB.TSOneRestServerDB(03028dd0) IIntranet.PollWebRTCAnswer{"Params":{"user":"daniel","accesskey":"fc4eb1e37cf603a315a40e13e533c3a6"}} ENetSock {Message:"THttpClientSocket.SockRecvLn error 10053 after 0 chars"} [TRemoteLog LogService] at 6ac915  '#$D#$A'20220711 07241519 srvr  '#9#9' 192.168.1.214 POST root/Intranet.PollWebRTCAnswer Interface=200 out=15 B in 23.03s'#$D#$A'20220711 07241519 ret   '#9#9'mormot.rest.server.TRestServerRoutingRest(0a167520) {"result":[[]]}'#$D#$A'20220711 07241519  -    '#9'23.037.594'#$D#$A'20220711 07241519  -    23.037.624cf603a315a40e13e533c3a6"}}'#$D#$A'20220711 06555037 srvr  '#9#9' 192.168.1.214 POST root/Intranet.PollWebRTCAnswer Interface=200 out=15 B in 4.81ms'#$D#$A'20220711 06555037 ret   '#9#9'mormot.rest.server.TRestServerRoutingRest(0a1670a0) {"result":[[]]}'#$D#$A'20220711 06555037  -    '#9'00.004.826'#$D#$A'20220711 06555037  -    00.004.851'#$D#$A'20220711 06562747 EXC   ENetSock {Message:"THttpClientSocket.SockRecvLn error 10053 after 0 chars"} [TRemoteLog LogService] at 6ac915  [TRemoteLog LogService] at 6ac915  0220711 06512160  -    '#9'00.654.069'#$D#$A'20220711 06512160  -    00.654.089'#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0

In such cases len = -1 (line 1858 mormot.net.sock)

Perhaps it has nothing to do with updates of mORMot2, but with the body of my request. But I don't know how to get it.

#44 Re: mORMot 1 » Native TLS Support for mORMot 2 REST or WebSockets Servers » 2022-07-10 18:58:36

After updating mORMot2 (last version was from April or May) I get errors (10053) when using remote-logger under Win32/Delphi 10.4.
The remote-logger runs on the same Win2019 machine as the mORMot2 service.
Under linux/fpc I can't see this error.
Perhaps the remote-logger is closing the socket because of a wrong header. Let me know if I can test something. The remote-logger is compiled from the samples of mORMot1.

#45 Re: mORMot 1 » regression for Variants in commits after 96c8a5d3 » 2022-03-24 18:45:34

Thanks!

I've tested your changes and it works in my project without issues.

#46 mORMot 1 » regression for Variants in commits after 96c8a5d3 » 2022-03-23 10:42:22

danielkuettner
Replies: 3

Hello Arnaud,

this code doesn't work with a commit later than 96c8a5d3 (2022-02-12):

var
  aCCGID: Integer;

V:= _OBJ([]);
aCCGID:= 1;
V.ID:= aCCGID;

Debugger shows a EJsonException instead of a parsed json.

(Windows11, Delphi 10.4, 32/64bit)

Daniel

#47 Re: mORMot 1 » ERR_CONTENT_LENGTH_MISMATCH using mORMot2 » 2021-08-12 06:34:17

Ctxt.OutContent is perfectly identical (mORMot1 vs. mORMot2)! Not what I expected.

Seems to be a wrong content-length in http header. How can I check this value? In Ctxt I haven't found it?

As a side note, content won't parsed under mORMot2 by

_Json(JSON, Result, [dvoReturnNullForUnknownProperty, dvoAllowDoubleValue, dvoValueCopiedByReference]);
--> var JSON: RawUTF8, Result: Variant)

whereas under mORMot1 it will.

#48 Re: mORMot 1 » ERR_CONTENT_LENGTH_MISMATCH using mORMot2 » 2021-08-12 04:51:34

I've looked for tools for that too and under linux there are also xxd and vimdiff or colordiff.

#49 Re: mORMot 1 » ERR_CONTENT_LENGTH_MISMATCH using mORMot2 » 2021-08-11 14:57:33

at Service level Result is ok, in Http.Request Ctxt.OutContent is wrong often (not always).
Although length of Ctxt.OutContent is correct, you can't convert Ctxt.OutContent to _JSON (json is invalid here).

With mORMot1 length is the same as under mORMot2, but you can convert Ctxt.OutContent to _JSON without error.

In TBaseWriter.AddNoJsonEscape P can't parsed to json with _Json (TDocVariantData(Value).InitJson(Json, Options) gives false), but P parsed with JSON-Lint says "valid json"

I've to compare the P (mORMot1<>2) tomorrow.

#50 Re: mORMot 1 » ERR_CONTENT_LENGTH_MISMATCH using mORMot2 » 2021-08-11 13:45:04

error still occurs after last update (14d8508bf1578e996542c44887e63efe91115972).

Ctxt.OutContent is not smaller, it's bigger, but seems to be wrong (when I can trust the Browsers).

I will follow your advices and give feedback...

Board footer

Powered by FluxBB