Posts by sakura

sakura · mORMot 1

It looks great - and thank YOU, bugs come with software, it is just strange how this came up, rather with that update, actually not related to it...
I'll take a close look at CopyProperties tomorrow :-)

Thanks again,
Daniel

sakura · mORMot 1

Hi,

I believe I finally have made it down to the problem, that is arising.

Now there are basically two possibilities for my problem:

a) (likely) - me using TRttiCustom incorrectly
b) something in the handling of empty UTF8String-properties

The full sample is at: https://pastebin.com/ENbYBjm4

When ran as show, it will throw an exception when using SetValue.
To NOT see the exception, set Source.Field_01 to anything, BUT an empty string.

Fun fact: i have about 50 regression tests against my Domain-ORM-mapper with various data, incl. empty strings, all of them succeeded. However, this simple sample crashed.
Added it to my tests now as well and hoping to learn more about my problem.

Regards
Daniel

P.S: Tested on Delphi 10.4.2 and Delphi 11.0 - Win32

sakura · mORMot 1

Hi Arnaud,

thanks for all your thoughts, as this did not change anything, I have tried to compile a minimal example. Likely, the fault lies with me :-(

See new thread please.

Thank you for all your help,
Daniel

sakura · mORMot 1

It does not change the outcome and playing around a bit, I have not found anything that could solve it, either. Still assigning the names "the classical way" works, which stumps me as to why it should work that way, but not yours. :-|

sakura · mORMot 1

No, I do not anywhere :-)

However, I can confirm to you, that the source of the problem arises from within the AddColumn source. When I change it to:

  len := StrLen(aColName);
  if fExpand then
  begin
    if aColIndex = 0 then // non-expanded mode doesn't use ColNames[]
      SetLength(ColNames, aColCount);
//    collen := len + 1;
// ...
//    P[len] := ':';
      ColNames[aColIndex] := '"' + UTF8String(aColName) + '":';
    end
    else
    begin
....

everything works fine.

Regards,
Daniel

sakura · mORMot 1

ab wrote:

Can you try to find out which exact commit is the culprit?

It is the one, mentioned in the first post. Undoing just this one commit (using git tools), I tried it this morning, and staying otherwise up to the latest commits, as of the time before I have started this thread, everything works as expected...

P.S.: I have also tried, the commits up to that mentioned - all newer fail, the older ones work just fine.

sakura · mORMot 1

ab wrote:

There is a RSP but they marked it as "to be fixed" in a next release.

Thanks, do you have the RSP-ID available?

Thanks,
Daniel

sakura · mORMot 1

The rest of the call stack is completely within my sources.

First, I run the query against the database, to retrieve the current version. I use the ORM-classes directly to check. This is the moment, when the problem is "initiated".

Noticing, that the database is not initialized, I want to create my standard user. I create my user using my KDD-classes for user management and use my own mapping-tools, to write the data into the user ORM-classes. Those tools use the mormot.core.rtti to map between the classes and read/write the properties. This is, where the error occurs.

So, it does not really have to do anything with either the sqlite access, nor with the ORM classes used at this point.

However, yet, when using the rtti methods from mormot.core.rtti, the problem arises, as long, as I do not change the code to the version mentioned above. Debugging into the mapping, it looks like the memory pointers got corrupted for the properties.

sakura · mORMot 1

ab wrote:

Please retry with my latest update.

First off: the problem persists with the latest update.

ab wrote:

What is the generated JSON content?

'{"fieldCount":4,"values":["ID","Version","CreatedAt","HostName"],"rowCount":0}'#$A

Simple table, no content, perfectly valid json, I'd say.

ab wrote:

Are the ColNames[] content correct?

Yes.

Can't really explain what's going on myself.

sakura · mORMot 1

Did you do 64-Bit? I just got a ton of messages there....

Here a list of compiler messages on Delphi 11, when compiling for 64 Bits, Windows:
https://pastebin.com/czS8b46C

Regards,
Daniel

sakura · mORMot 1

Hi Arnaud,

I am currently stumped, as to why a commit from Oct 1st breaks some code for me.
Commit: https://github.com/synopse/mORMot2/comm … 0296d37348

In this commit you have introduced the method TJsonWriter.AddColumn and you use it in TSqlRequest.Execute:

    // directly assign column names from SQlite3 API into W
    for i := 0 to FieldCount - 1 do
      W.AddColumn(sqlite3.column_name(Request, i), i, FieldCount);

When I change that code back to

    // get col names and types
    SetLength(W.ColNames, FieldCount);
    for i := 0 to FieldCount - 1 do
      W.ColNames[i] := sqlite3.column_name(Request, i);
    W.AddColumns; // write or init field names for appropriate Json Expand

my code, using your Rtti-methods for reading/writing properties of my objects works just fine. With the new changes, that code breaks completely. I am not saying, that I might not have made a mistake in my rtti-related code, however, for now, all comes down to that change, whether my object transpose system works, or not :-|

Partial stack trace, when error occurs:

20211008 06535548 0000009233EFCFD0  ! EXCOS EAccessViolation (c0000005) [Main] 
at 51b3d7 mormot.core.unicode.pas TSynAnsiFixedWidth.AnsiBufferToUtf8 (2853)  
@ExceptionHandler mormot.core.unicode.pas TSynAnsiConvert.AnsiBufferToRawUtf8 (2560) 
mormot.core.unicode.pas TSynAnsiConvert.AnsiToUtf8 (2545) 
mormot.core.text.pas VariantToUtf8 (8761) 
mormot.core.text.pas VariantToUtf8 (8839) 
mormot.core.rtti.pas TRttiCustomProp.SetValue (6227)

A little more information, as to the settings used in AddColumn(s):
fExpand evaluates to True
twoForceJsonExtended in CustomOptions evaluates to False
---> routing to ColNames[ i ] := '"' + ColNames[ i ] + '":'; in AddColumns.

Would you have an idea, what might break here, would you like remote access to my machine, to see what's going on?

Regards,
Daniel

P.S.: Yet there is a relation - I have just executed a SQL-request and copy those results into my object.

sakura · mORMot 1

Hi,

you need your own github account, clone the repo to your account, upload the changes to yours and then start a pull-request. This way, a review before merge is possible.

Regards,
Daniel

sakura · mORMot 1

radexpol wrote:

@sakura, I have no more code, nothing to disable. That problem appears when using runtime packages.

Sorry, I missed that part with the runtime packages. Haven't tried that with mORMot 2, but design-time packages for the Delphi IDE never worked with mORMot 1 either. Slightly different, I know, but I believe that will be the case with run-time packages always, as mORMot is going deeply into the VMT and RTTI of the application. But let's see what Arnaud can tell :-)

sakura · mORMot 1

Even though that *seems* to solve your problem, disable portions of your code and watch when that error disappears. As many of us are using that unit as well any many more projects without that problem, I would believe that some part of your code is producing that hiccup.

sakura · mORMot 1

radexpol wrote:

@sakura, Clean before Build to see those errors.

Believe me when I write, I always do ;-)
Actually, one of my pre-build scripts does.

Regards,
Daniel

P.S.: I did a manual clean too, same results. No Hints whatsoever.

sakura · mORMot 1

FYI: Delphi 11 does not show any warnings.

sakura · mORMot 1

Why should it not?

Yes for both 1 and 2.

sakura · mORMot 1

Okay, too bad.
Thx.

sakura · mORMot 1

ab wrote:

On Windows, it makes sense to use http.sys to publish TLS

So it should work, but doesn't - I only do Windows, never have a Linux system around me. ;-)

Remember, as written above, I had not yet used WebSockets on the project, just wanted to prepare for it, step by step.

ab wrote:

If you want to communicate securely between mORMot servers, do not use TLS but our WebSockets encryption layer over binary websockets.

I need WebSockets for Server-Browser-Communication. Not between mORMot-Servers - not yet anyway.

Regards,
Daniel

sakura · mORMot 1

I'll check it, thx. Might just be the right direction. However, just being naughty, as the TRestHttpServer class offers it, why not implement it? :-D

sakura · mORMot 1

But it worked in mORMot1 with no problems. SSL certs I have and installed according to the mormot1 docs.

sakura · mORMot 1

Hi,

before wanting to add web sockets, I would initialize the http rest server as follows:

  TRestHttpServer.Create('80', RestServer, 'localhost', useHttpApiRegisteringURI, nil, 32, secNone);
  TRestHttpServer.Create('443', RestServer, 'localhost', useHttpApiRegisteringURI, nil, 32, secSSL);

It worked just fine, resolving the http headers with and without SSL.

Changing it to

  TRestHttpServer.Create('80', RestServer, 'localhost', useBidirSocket, nil, 32, secNone);
  TRestHttpServer.Create('443', RestServer, 'localhost', useBidirSocket, nil, 32, secSSL);

it continues to work without SSL just fine, but using SSL it will not work anymore.

The error message (later on reads):
Project MYP.exe raised exception class $C0000005 with message 'access violation at 0x006786e1: read of address 0x00000000'.

The source of the problem seems to be in THttpServer.Process - reading values from http gives back either empty strings (for URL, InContent) or garbage for others (Method, InHeaders)

Cannot really look further, as, it this point, I am kindo out of my depth.

@ab: if you like remote access, let me know ;-)

Env: Running Delphi 10.4 on Windows 10.

Regards,
Daniel

sakura · mORMot 1

Hi Arnaud,

in "mormot.core.data" @ TDynArrayHasher.FindOrNew following compiler hint:
[dcc32 Hint] mormot.core.data.pas(8464): H2164 Variable 'cmp' is declared but never used in 'TDynArrayHasher.FindOrNew'

Regards,
Daniel

sakura · mORMot 1

Hi,

will mormot2 support http 2.0 protocols?

I just noticed, that it likely does.
So let's rephrase: how do I enable it. How can I handle multiplex requests?

Regards,
Daniel

sakura · mORMot 1

Not what I'd love to read, but I can live with that ;-)

Thanks

sakura · mORMot 1

Basically I am trying to find a way to check, whether the client has aborted the request in the meantime. In .NET running on http.sys you get something called a CancellationToken. That would allow canceling longer operations, when the client "lost interest".

sakura · mORMot 1

I wouldn't think about callback, rather just a field/property like ClientListening (definitely not the best name), that would be false, once disconnected. This way the methods could stop gathering/sorting/whatever data and abort upon checking that flag.
I guess that could be achieved just before setting ServiceRunningContext^.Request := nil.

Where does that "ServiceRunningContext^.Request := nil" happen? I'd be happy to check out a way for achieving that.

Regards,
Daniel

sakura · mORMot 1

I guess it would be a good idea, if there is a way to signal it to the running context in some way, for longer MVC ops to check on that. I tried studying the source for it, but could not make out a place where I could inject logic on that :-|

Regards,
Daniel

sakura · mORMot 1

Hi,

is there a way, possibly through ServiceRunningContext, to determine, whether the client has terminated the current request?

sakura · mORMot 1

Ping @ab :-)

sakura · mORMot 1

Hi,

I got the following to error messages:

[dcc32 Error] mormot.rest.mvc.pas(1095): E2250 There is no overloaded version of 'UTF8ToString' that can be called with these arguments
[dcc32 Error] mormot.rest.mvc.pas(1853): E2250 There is no overloaded version of 'UTF8ToString' that can be called with these arguments

Both could be fixed similar to

Utf8ToString(fFactory.Methods[m].Uri, MethodName);

->

Utf8ToString(fFactory.Methods[m].Uri, string(MethodName));

Delphi 10.4.2 - all updates

sakura · mORMot 1

ab wrote:

Which exact encryption is required (there are several around)?

The KBV (German association of health insurers) simply states password protected. No further information at this point :-|

sakura · mORMot 1

ab wrote:

ZIP encryption is weak in terms of security,

That I know and understand, it's requirements by the German government forcing me to support it. Silly, I know, but that's the state of out nations IT :-o

ab wrote:

is not supported by mormot.core.zip.pas.

Personally, I fully support that decision, based on my requirements, I regret it though :-D

Thanks,
Daniel

sakura · mORMot 1

Hi Arnaud,

is there a way, using mORMot 2 to created and read password protected ZIP archives. Strolling through the source, I haven't seen anything. Did I miss it?

Regards,
Daniel

sakura · mORMot 1

No problem, looking forward to it :-)

sakura · mORMot 1

Hi,

in mormot.core.json are references to YAML-processing, especially

function TSynJsonFileSettings.LoadFromYaml(var aYaml: RawUtf8): boolean;
begin
  result := YamlSettingsToObject(aYaml, self);
end;

which is not declared and references the undeclared method YamlSettingsToObject.

I guess new, exciting things are comin' along :-)

sakura · mORMot 1

itSDS wrote:

Hi Arnaud, late feedback.

Wow :-D
6 years is late, glad you still use the same framework, shows again how awesome that is!

sakura · mORMot 1

I've seen it myself, you can optimize as much as you like, sometimes you miss something or other and the whole server blocks everything else. Sometimes, only real-life data will show the problems, and often, as a developer you do not have access to such, for reasons of data security.

sakura · mORMot 1

I know the feeling though, thanks for making us feel better ;-)

sakura · mORMot 1

Hi,

as DDD has not yet been made available to mORMot2, ist there a preferred way to get DDD-classes (based upon TSynAutoCreateFields) filled with data retrieved into TOrm classes?

Regards,
Daniel

sakura · mORMot 1

Thanks, yeah, I saw this quite early, but my mind was set to doing it without any SQLite3 support, and this blocked me from seeing the way.

sakura · mORMot 1

I guess I had the wrong idea about how to do that. Am I right, that I still have to use the TRestServerDB implementation, which internally uses SQLite3, but simply register the model table for MSSQL using VirtualTableExternalRegister/VirtualTableExternalRegisterAll?

sakura · mORMot 1

Oh yes, in mORMot2 we already have TSqlDBOleDBMSSQL2012ConnectionProperties, however, I am stumped right now, how to connect that with which TRestServer implementation. As we say in German, I ain't seeing that forest for all the trees here...

sakura · mORMot 1

Okay, all I have done so far was easy access using TRestServerDB and SQLite3. I know, that plugging in MSSQL is supposed to be just as easy, but I am stumped at this moment and can't see how to do it, preferably w/o ZEOS. Which RestServer-Class would I use and how would a connect it to the SQL Server?

Thanks for the quick hint,
Daniel

sakura · mORMot 1

ab wrote:

Please check https://synopse.info/fossil/info/86e1418dc1

Works like a charm <3

  ServiceManagerApp := TMypServiceManagerApplication.Create;
  ServiceManagerApp.Start(RestServer, TypeInfo(IMypServiceManagerApplication));
  RestServer.ServiceMethodRegisterPublishedMethods('svc_', ServiceManagerApp);

and the App-Class has methods like list_session and may now be called as domain/root/svc/list/session :-)

sakura · mORMot 1

keinn wrote:

but if the mormot.core.json and mormot.core.text both imported in the unit , u can't do so. it can NOT be compiled.

Most of my units have both and I am using FormatUtf8 all over the place, no problems at all.

Delphi 10.4.2 W32/W64

sakura · mORMot 1

Just browsed through some old mORMot 1 test project, haven't done it in mORMot 2 yet. I created an override for TSQLRestServerURIContext.URIDecodeSOAByMethod. Retrieve the URI and URIBlobFieldName, replacing all / with _ and then checking like

type
  TSQLRestServerAccess = class(TSQLRestServer);

...

    MethodIndex := TSQLRestServerAccess(Server).fPublishedMethods.FindHashed(TestURI);

    if MethodIndex >= 0 then
    begin
      URI := TestURI;
      URIBlobFieldName := TestURIBlobFieldName;

      Exit;
    end;

Sorry, can't show all the rest here :-|

sakura · mORMot 1

At my old employer's place I wrote a method to replace all / in the URI with _ in the method calls. It was quite easy to do and I am planning on doing that again, as this improves readability for the user.

A call from domain/root/admin/user/add would go to the method admin_user_add.

sakura · mORMot 1

ab wrote:

The source code has already been changed in between your pull request....

Okay

ab wrote:

Please use the forum first to avoid closing the pull request.

It's okay though, doesn't hurt much ;-)

ab wrote:

Which compiler are you using?

Usually always the current one, in this case D10.4.2-W32/W64

Regards,
Daniel

sakura · mORMot 1

Fixed in Pull Request

https://github.com/synopse/mORMot2/pull/28

mORMot Open Source

#101 Re: mORMot 1 » Using RTTI and GetValue/SetValue with empty UTF8String Prop » 2021-10-11 18:01:56

#102 mORMot 1 » Using RTTI and GetValue/SetValue with empty UTF8String Prop » 2021-10-11 10:48:18

#103 Re: mORMot 1 » Commit from Oct 1st breaks seemingly unrelated code » 2021-10-11 10:27:04

#104 Re: mORMot 1 » Commit from Oct 1st breaks seemingly unrelated code » 2021-10-09 11:23:07

#105 Re: mORMot 1 » Commit from Oct 1st breaks seemingly unrelated code » 2021-10-09 09:01:45

#106 Re: mORMot 1 » Commit from Oct 1st breaks seemingly unrelated code » 2021-10-08 12:52:24

#107 Re: mORMot 1 » A lot of hints while using mORMot 2 » 2021-10-08 12:29:39

#108 Re: mORMot 1 » Commit from Oct 1st breaks seemingly unrelated code » 2021-10-08 12:28:08

#109 Re: mORMot 1 » Commit from Oct 1st breaks seemingly unrelated code » 2021-10-08 11:37:43

#110 Re: mORMot 1 » A lot of hints while using mORMot 2 » 2021-10-08 11:25:24

#111 mORMot 1 » Commit from Oct 1st breaks seemingly unrelated code » 2021-10-08 06:47:19

#112 Re: mORMot 1 » BUGFix for mORMot.pas » 2021-10-06 12:42:37

#113 Re: mORMot 1 » Access Violation when using mORMot 2 / DLL / Runtime packages » 2021-09-30 07:49:06

#114 Re: mORMot 1 » Access Violation when using mORMot 2 / DLL / Runtime packages » 2021-09-30 04:17:06

#115 Re: mORMot 1 » A lot of hints while using mORMot 2 » 2021-09-27 15:13:23

#116 Re: mORMot 1 » A lot of hints while using mORMot 2 » 2021-09-27 04:34:04

#117 Re: mORMot 1 » can Delphi 11 compile mORMot? » 2021-09-16 04:43:34

#118 Re: mORMot 1 » Rest HTTP Server, BiDirSocket and SSL » 2021-09-06 19:52:01

#119 Re: mORMot 1 » Rest HTTP Server, BiDirSocket and SSL » 2021-09-06 16:12:00

#120 Re: mORMot 1 » Rest HTTP Server, BiDirSocket and SSL » 2021-09-06 11:45:36

#121 Re: mORMot 1 » Rest HTTP Server, BiDirSocket and SSL » 2021-09-06 11:28:53

#122 mORMot 1 » Rest HTTP Server, BiDirSocket and SSL » 2021-09-06 10:48:36

#123 mORMot 1 » Small Hint at current State » 2021-09-06 06:40:36

#124 mORMot 1 » Support for http 2.0 » 2021-08-30 08:28:46

#125 Re: mORMot 1 » mORMot2 - Request terminated » 2021-07-06 12:55:01

#126 Re: mORMot 1 » mORMot2 - Request terminated » 2021-07-06 08:54:39

#127 Re: mORMot 1 » mORMot2 - Request terminated » 2021-07-05 19:55:26

#128 Re: mORMot 1 » mORMot2 - Request terminated » 2021-07-05 16:13:22

#129 mORMot 1 » mORMot2 - Request terminated » 2021-07-05 11:06:12

#130 Re: mORMot 1 » [dcc32 Error] mormot.rest.mvc.pas(1095): E2250 There is no overloaded » 2021-07-01 15:15:06

#131 mORMot 1 » [dcc32 Error] mormot.rest.mvc.pas(1095): E2250 There is no overloaded » 2021-06-29 08:06:44

#132 Re: mORMot 1 » Password protected ZIP archives » 2021-06-23 14:51:15

#133 Re: mORMot 1 » Password protected ZIP archives » 2021-06-23 06:14:23

#134 mORMot 1 » Password protected ZIP archives » 2021-06-22 07:08:55

#135 Re: mORMot 1 » LoadFromYaml undeclared » 2021-05-10 09:23:54

#136 mORMot 1 » LoadFromYaml undeclared » 2021-05-10 05:59:18

#137 Re: mORMot 1 » Problem: Cookie Handling in MVC Server is Case sensitive » 2021-05-05 08:58:35

#138 Re: mORMot 1 » Getting rid of TSQLRestServerDB giant (read) lock » 2021-05-04 06:23:21

#139 Re: mORMot 1 » JSON De-serialization help needed » 2021-05-04 04:27:24

#140 mORMot 1 » mORMot2 - mapping between ORM and KDD/DDD classes » 2021-05-03 11:20:24

#141 Re: mORMot 1 » MSSQL Server in mORMot2 » 2021-04-29 12:48:36

#142 Re: mORMot 1 » MSSQL Server in mORMot2 » 2021-04-29 11:00:57

#143 Re: mORMot 1 » MSSQL Server in mORMot2 » 2021-04-29 10:07:35

#144 mORMot 1 » MSSQL Server in mORMot2 » 2021-04-29 08:47:05

#145 Re: mORMot 1 » can ServiceMethodRegisterPublishedMethods support prefix contain "/" » 2021-04-28 13:50:15

#146 Re: mORMot 1 » FormatUtf8 in core.json and core.text conflit » 2021-04-28 13:41:11

#147 Re: mORMot 1 » can ServiceMethodRegisterPublishedMethods support prefix contain "/" » 2021-04-28 05:03:29

#148 Re: mORMot 1 » can ServiceMethodRegisterPublishedMethods support prefix contain "/" » 2021-04-28 04:41:32

#149 Re: mORMot 1 » Combined signed-unsigned warnings » 2021-04-23 15:38:37

#150 mORMot 1 » Combined signed-unsigned warnings » 2021-04-23 06:15:58

Board footer