#501 mORMot 1 » What's the correct way to connect a https-enabled mORMot rest server? » 2018-04-15 10:49:10

edwinsn
Replies: 7

Situation
===
I compiled the '04 - HTTP Client-Server' official mORMot sample and have successfully deployed it on a Windows 2016 server behind IIS 10, the server has both http/80 and https/443 ports open, and both are redirected to the background mORMot rest server using the IIS URL Rewrite module.

The IIS-as-the-reverse-proxy part is already working, because I was able to connect to the remote mORMot server using:

TSQLHttpClient.Create('server.mysite.com','80', Form1.Model);

The https part is also working, because open 'https://server.mysite.com/root' with a web browser I can get the correct result from Project04Server.exe, which is:

{
"errorCode": 403,
"errorText": "Authentication Failed: Invalid signature (0)"
}

The problem
===
The problem is that Project04Client.exe cannot connect to the remote Project04Server.exe, I have tried all the following forms but failed with various errors:

  Form1.Database := TSQLHttpClient.Create('server.mysite.com','80', Form1.Model);  // ok, but not https

  Form1.Database := TSQLHttpClient.Create('server.mysite.com','443',Form1.Model); // failed, ServerTimestampSynchronize will fail
  Form1.Database := TSQLHttpClient.Create('https://server.mysite.com','443',Form1.Model); // failed, ServerTimestampSynchronize will fail
  Form1.Database := TSQLHttpClient.Create('https://server.mysite.com','80',Form1.Model); // failed, ServerTimestampSynchronize will fail


  Form1.Database := TSQLHttpsClient.Create('https://server.mysite.com', Form1.Model, 443);  // failed, ServerTimestampSynchronize will fail
  Form1.Database := TSQLHttpClientWinINet.Create('https://server.mysite.com', Form1.Model, 443);  // failed, ServerTimestampSynchronize will fail

  myUri.From('https://server.mysite.com');
  Form1.Database := NewSQLHttpClient(myUri, Form1.Model, False); // Access Violation

  Form1.Database := TSQLHttpClientRequest(TSQLHttpsClient).Create('server.mysite.com', '443',
    Form1.Model, True); // Access Violation

What is the correct way to connect a https-enabled mORMot rest server?
Thanks!

#502 Re: mORMot 1 » [issue] mORMotWrappers.pas compilation error under Lazarus 1.9 trunk » 2018-04-12 09:37:57

@ab,

I tried run it with the debugger, the first hundreds of failures happens in method TTestLowLevelCommon._UTF8.Test() exactly this line:

Check(StringCodePage(W)=1252);

The stack trace in the log file shows hexadecimal source locations like:

stack trace API 005BAB82 005D1B4A 005CFA79 008874BA 0088A5A5 005D0BB5 0098950C 009897FF 00993C09 76C3336A 77C298F2 77C298C5

Is there a way to shows the real call stack with real method names and line numbers?

#503 Re: mORMot 1 » [issue] mORMotWrappers.pas compilation error under Lazarus 1.9 trunk » 2018-04-12 09:04:55

@ab,
I use the very same set of Synopse Framework source code for both XE4 and FPC, so it's not possible some of the files are un-synced (out of date).

#504 Re: mORMot 1 » [issue] mORMotWrappers.pas compilation error under Lazarus 1.9 trunk » 2018-04-12 06:46:47

@mpv, thanks, downloaded it and tried, but TestSql3.exe would still opens another console window... what happened? I'm compiling the official TestSql3.dpr project.

Update: Initially I also tried running testSql3.exe > output.txt, while the output.txt file was generated, it's empty.

#505 Re: mORMot 1 » [issue] mORMotWrappers.pas compilation error under Lazarus 1.9 trunk » 2018-04-12 04:26:41

Update: Now the latest commit can be compiled under xe4.
But it surprises me that the fpc-compiled TestSQL3 program passed all test cases, while 20,293 of 40,427,058 test cases compiled by XE4 failed!

The log file is here: https://send.firefox.com/download/d16be … Ar_b_E93NQ

FPC/Lazarus trunk version 2018/01/25 TestSQL3.exe result:
===
Run on: Windows 7 64bit SP1 (6.1.7601) - codepage=936
    4 x Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz (x86)
Using mORMot 1.18.4518
    TSQLite3LibraryStatic 3.23.0 with internal MM
Generated with: Free Pascal 3.1.1 MOP 32 bit compiler

Time elapsed for all tests: 1m19
Performed 2018-04-12 12:04:37 by edwin on EDWIN-T450

Total assertions failed for all test suits:  0 / 38,960,317
! All tests passed successfully.

Delphi XE4TestSQL3.exe result :
===
Run on: Windows 7 64bit SP1 (6.1.7601) - codepage=936
    4 x Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz (x86)
Using mORMot 1.18.4518
    TSQLite3LibraryStatic 3.23.0 with internal MM
Generated with: Delphi XE4 32 bit compiler
Time elapsed for all tests: 5m50
Performed 2018-04-12 12:00:21 by edwin on EDWIN-T450

Total assertions failed for all test suits:  20,293 / 40,427,058
! Some tests FAILED: please correct the code.


Update 2:
Additional console output:
===
1.6. ECC cryptography:
- Reference vectors: 23 assertions passed  31.39ms
- ecc_make_key: 50 assertions passed  126.61ms
- ecdsa_sign: 50 assertions passed  128.60ms
- ecdsa_verify: 50 assertions passed  156.48ms
- ecdh_shared_secret: 147 assertions passed  271.41ms
  - Certificates and signatures: 6 / 91 FAILED  113.64ms
- ECC command line tool: 118 assertions passed  26.63s
- ECDHE stream protocol: 521,295 assertions passed  1.94s
Total failed: 6 / 521,824  - ECC cryptography FAILED  29.41s

There are more but I cannot scroll up the console window...

#506 Re: mORMot 1 » [issue] mORMotWrappers.pas compilation error under Lazarus 1.9 trunk » 2018-04-11 13:47:04

@ab, I'll stick with whichever FPC trunk version you use, so have you upgraded your Lazarus? wink

#507 Re: mORMot 1 » [issue] mORMotWrappers.pas compilation error under Lazarus 1.9 trunk » 2018-04-11 10:31:33

Unfortunately, I need to try it under Delphi xe4 but it failed to compile tongue

[dcc32 Error] SynCommons.pas(62196): E2441 Inline function declared in interface section must not use local symbol 'COMPRESS_SYNLZ'

#508 Re: mORMot 1 » [issue] mORMotWrappers.pas compilation error under Lazarus 1.9 trunk » 2018-04-11 03:33:07

Update: Using FPC trunk 2018/01/25 (SVN 57146) the latest mORMot TestSQL3.dpr project can be build successfully!

#509 Re: mORMot 1 » [issue] mORMotWrappers.pas compilation error under Lazarus 1.9 trunk » 2018-04-11 03:11:06

Yes, I got my answer from Alred - go to the Setup+ dialog and on the upper right there are edit boxes to enter the rev. numbers.

#510 Re: mORMot 1 » [issue] mORMotWrappers.pas compilation error under Lazarus 1.9 trunk » 2018-04-10 15:26:05

@ab,
Did a check just now, I confirm I'm using the latest revision of mORMotWrappers.pas.

Do you know how to let fpcupdeluxe to retrieve the FPC trunk 2018/01/25 (SVN 57146) instead of the latest rev? I'd like to use that revision!

#511 mORMot 1 » [issue] mORMotWrappers.pas compilation error under Lazarus 1.9 trunk » 2018-04-10 07:20:58

edwinsn
Replies: 17

Error message:
===
mORMotWrappers.pas(724,75) Error: Incompatible type for arg no. 2: Got "Variant", expected "TFileName"

mORMot version:
===
Almost latest, i checked and and the line where the error happened is exactly the same as the one in the latest github repository.

Lazarus version:
===
Lazarus trunk built with fpcupdelux few days ago.

Screenshot:
===
https://screenpresso.com/=5pazd

#512 Re: mORMot 1 » Replication, transactions, consistency » 2018-04-10 01:49:44

@ab, I didn't know about the performance part. I posted these links because I was searching something on the forum and found this post, so I thought I'd post them here. As a matter of fact I didn't study the Raft details, just heard something about it.

#513 Re: mORMot 1 » Replication, transactions, consistency » 2018-04-09 15:11:22

As far as I know, the Raft protocol (https://raft.github.io/) is the current mechanism for doing replication/distribution.

Two open source projects using the raft spec:
dqlite: https://github.com/CanonicalLtd/dqlite
rqlite: https://github.com/rqlite/rqlite

an interactive, graphical illustration of the raft consensus protocol:
http://thesecretlivesofdata.com/raft/

#514 Re: mORMot 1 » roolback transection » 2018-04-08 09:55:54

Hi, I suggest you to use Google Translate to translate your question into English: https://translate.google.com/

#515 Re: mORMot 1 » Benchmark information » 2018-04-08 05:29:28

emk wrote:

EDIT: I read Index» mORMot Framework» Difference between DTO and entity
From that thread, this corresponds to my needs:   smile

I know people must use DDD/DTO for a reason, but so far I feel it's a little over-abstraction for my own purpose, I might be wrong

That's my question! smile
I might should have done some research instead of asking that, so I went ahead today to have read some material, and I found that the wikipedia page explained it very very clear:

A data transfer object (DTO[1][2]) is an object that carries data between processes. The motivation for its use is that communication between processes is usually done resorting to remote interfaces (e.g., web services), where each call is an expensive operation.

And this:

...the whole purpose of DTOs is to shift data in expensive remote calls.

I understand it in this way - if TSQLRecord is corresponding to a database table row, a DTO is corresponding to a "database view row". I can see benefits of such approach, for example assuming we have the following db tables:
- TSQLDepartment (with 10 fields)
- TSQLStaff (with 20 fields)

If the clients needs the following data:
- Staff's first name and last name.
- The name of the department.
It's only 3 fields from the above described two tables. We'll have two ways to transfer the data from the server to the client:
- Non-DTO way: on the client-side, read the TSQLStaff record and the corresponding TSQLDepartment record, and this involves two network round-trip and 30 fields.
- The DTO way: On the server-side we define an interface method, such as IServerApi.GetStaffFullNameAndDeptName, and the client-side needs only one call to the API and this involves one a single network round-trip and 3 fields, moreover, the client-sides code logic can be significantly simplified.

Just my humble conclusion after after reading several wikipedia pages.

#516 Re: mORMot 1 » The new sqlite3_serialize and sqlite3_deserialize functions » 2018-04-05 04:40:49

Another use case would be using the deserialized db as a data cache.

#517 Re: mORMot 1 » The new sqlite3_serialize and sqlite3_deserialize functions » 2018-04-04 16:47:22

Good to hear that. I believe these two new API exist for a reason. At first glance, it achieve 'run in memory entirely' with just two calls, very easy and simple than the backup API.

#518 mORMot 1 » The new sqlite3_serialize and sqlite3_deserialize functions » 2018-04-04 08:01:09

edwinsn
Replies: 4

Hey!

If I understand it correctly, the new The new sqlite3_serialize and sqlite3_deserialize functions allows you to easily run the entire database in memory (deserialize) and when done, serialize it to a disk file!

For small-to-medium sized databases that can fit in the RAM properly, this approach can be a big performance improvement!

So @ab, I'm wondering how can achieve that with the mORMot framework smile

#520 Re: mORMot 1 » Embed SQLite C object files into application » 2018-03-13 09:43:53

What a great addition!

And the diff of this commit must also shows the idea for implementing a **compression** extension smile

#521 Re: mORMot 1 » sqlite - recover a database corrupted (backup/restore) » 2018-03-12 11:48:01

I have experienced a "database disk image is malformed" error because the user forced the computer to shutdown when the program is operating the sqlite db.

#522 Re: mORMot 1 » Does THttpApiServer is single-threaded? » 2018-03-09 12:45:22

No, I didn't mean you need to re-implement/duplicate the code logic from the stored procedure, you just call it from within the service class. What I mean is that, if I understand you correctly you don't have to manually do the following:

"I'm going to convert the stored procedure's result from variant to object for temporary storage and processing, and convert it to JSON for a HTTP response. (I know there is a FetchAllAsJSON but I do sure need an object)"

if you use TSQLRestServer instead.

#523 Re: mORMot 1 » Does THttpApiServer is single-threaded? » 2018-03-09 09:58:34

DDGG wrote:

@edwinsn Although I don’t quite understand you said, I'm still grateful!

If you want to expose from the server-side to the client-side your stored procedure such as GetUserLoginInfo you mentioned, you have two high-level "mORMot approaches":
1 - use method-based service, or
2 - use interface-base service.


For approach #1, you can take a look at https://github.com/synopse/mORMot/blob/ … Server.dpr

Code like the following will be much much more easier and intuitive than doing everything in THttpApiServer.OnRequest.

type
  // TSQLRestServerFullMemory kind of server is light and enough for our purpose
  TServiceServer = class(TSQLRestServerFullMemory)
  published
    procedure Sum(Ctxt: TSQLRestServerURIContext);
  end;

See that? You don't even need the TSQLRecord class or the SQLITE engine.

#524 Re: mORMot 1 » Does THttpApiServer is single-threaded? » 2018-03-09 07:53:53

@DDGG, I believe your situation perfectly fits in the use case of TSQLRestServer, please search for 'method-based service' and 'interface-based serfice' in the mORMot document, that's the correct approach for implementing your GetUserLoginInfo API.

and you can use an in-memory db (use ':memory:' as the db file name) as the database, then just define a single dummy TSQLRecord-derived class in order for the framework to work.

#525 Re: mORMot 1 » Pattern match failed » 2018-02-25 03:27:02

I didn't take into account the unit file size when I was thinking about it, just thought about it logically wink

#526 Re: mORMot 1 » Pattern match failed » 2018-02-24 09:34:08

This sounds like a simplified regular expression, cool! And I didn't know about it before.

PS, not a real issue, but I couldn't resist - I thought it should be placed in SynCommon.pas instead?

#527 Re: mORMot 1 » Is there a XPATH implementation? » 2018-02-20 09:03:13

@mdbs99, yes, you have a valid point smile

@mpv, do we have a compatible Node.Js package list for syNode?

#528 Re: mORMot 1 » Is there a XPATH implementation? » 2018-02-19 04:03:45

XML is too verbose and is such a mess, JSON is better.

Food for thought: http://jmespath.org/

#529 Re: mORMot 1 » Proposal for a new behavior for TObjectDynArrayWrapper/IObjectDynArray » 2018-02-19 03:39:36

Count me in, the OwnsObjects parameter would allow us to control the lifetime of the contained objects.

#530 Re: mORMot 1 » Client-server contract incompatibilty » 2018-02-06 18:25:18

If you have changed the definition of IMyService, you have to **compile** both the server and client programs.
All, I mean all, all of the time I got a "contract" error was the above mentioned reason.

#531 Re: mORMot 1 » TObjProxyVariant - new variant kind » 2018-01-31 14:53:42

Speaking of cache, does the mORMot framework has a generic object pool yet?

#532 Re: mORMot 1 » TObjProxyVariant - new variant kind » 2018-01-31 13:22:19

Very handy!

Usually I use duckduckdelphi in such case, but it uses Extended RTTI so in some cases it's slow.

#533 Re: mORMot 1 » Spell checking » 2018-01-30 15:51:25

Hi ab,

If it makes sense, I'd suggest to include the "load_extension()" in the engine compilation if it doesn't add to much size to the resulting .obj file, you know, allow using those powerful Sqlite loadable extensions is a plus to mORMot smile

#534 Re: mORMot 1 » Spell checking » 2018-01-30 15:26:59

Hi ab, after searching for "sqlite3.c" in the pdf document, I have found the relative info, thanks.

#535 Re: mORMot 1 » Spell checking » 2018-01-30 12:55:01

BTW, I want to use the following SQLITE extensions smile
- the 'dbdump' extension.
  for dumping data to disk and then restoring back to the db.

- the spell check extension: http://www.sqlite.org/spellfix1.html

- the csv extension: https://sqlite.org/csv.html
  For accessing a csv disk file through the SQL interface, wow!

- the carray extension: https://www.sqlite.org/carray.html
  I assume I can pass an delphi array to the sqlite engine?

- the zipfile extension: http://sqlite.org/zipfile.html
  For reading a disk .zip file.

- the appendvfs extension:https://sqlite.org/src/file/ext/misc/appendvfs.c
  For reading a sqlite db appended to the end of a .exe file.

Any instruction for using these extensions will be appreciated.

#536 Re: mORMot 1 » Spell checking » 2018-01-30 12:38:50

Hi ab,

How to load a sqlite loadable extension with mORMot? I tried executing

select load_extension("spellfix1");

but got an error: "Error SQLITE_ERROR (1) [select load_extension("spellfix1");] using 3.22.0 - no such function: load_extension, extended_errcode=1".

And according to the sqlite3 documentation (https://sqlite.org/loadext.html#compili … _extension), an extension is actually a .DLL, does it mean I have to download the sqlite c source code, install Microsoft VC, and compile the extension manually?

Thanks.

#539 mORMot 1 » [not-an-issue]No logs for a wrong 'CREATE INDEX' with expression » 2018-01-29 12:10:44

edwinsn
Replies: 2

Hi ab,

When TSQLRestServerDB executes a wrong SQL like the following, it will failed without any errors being logged to the log file.

CREATE INDEX CustomerRelatedDocList ON Customer (
    JSON_EXTRACT(WrongFieldName, "$.RecordID") ASC,
    JSON_EXTRACT(WrongFieldName, "$.RecordClassName") 
);

For reference, the correct SQL is as following (and can be executed without issue):

CREATE INDEX CustomerRelatedDocList ON Customer (
    JSON_EXTRACT(RelatedDocList, "$.RecordID") ASC,
    JSON_EXTRACT(RelatedDocList, "$.RecordClassName") 
);

RelatedDocList is defined as a TObjectList which is persisted in JSON format.

I'm using this rev: https://github.com/synopse/mORMot/commi … 76b9b42046 (updated SQLite3 engine to latest version 3.22.0)

#540 Re: mORMot 1 » [Problem] Implementation conn pool in unigui ? » 2018-01-29 04:35:06

I don't use any Sencha products, but let's see what will happen to Delphi...

#541 Re: mORMot 1 » [Problem] Implementation conn pool in unigui ? » 2018-01-28 06:29:42

@warleyalex, So it's a win for delphi deveopers?

One question -
- I think Ideara hasn't acquired FMSOFT?

#542 Re: mORMot 1 » How to get the authenticated user when using interface based approach? » 2018-01-27 09:32:13

Since ab has said that:
>>"but note that CurrentServiceContext.Request may be nil if the service is executed in-place (i.e. directly from the server side)",

I guess the more reliable way is to inherit your implementation class from TInjectableObjectRest, which has a Server property auto-assigned?

#543 Re: mORMot 1 » [FIXED]FieldType compilation error under xe4, in file mORMotVCL.pas » 2018-01-24 16:11:53

The issue's gone now.
Thanks for the **always** very quick help!

#544 mORMot 1 » [FIXED]FieldType compilation error under xe4, in file mORMotVCL.pas » 2018-01-24 08:38:00

edwinsn
Replies: 2

Hi ab,

Since the 'issues' page for the mORMot github project is not visible, I'm posting the xe4 compilation error here:

[dcc32 Error] mORMotVCL.pas(330): E2250 There is no overloaded version of 'FieldType' that can be called with these arguments
[dcc32 Error] mORMotVCL.pas(430): E2250 There is no overloaded version of 'FieldType' that can be called with these arguments

This revision:
https://github.com/synopse/mORMot/commi … 55b54b56b8

#545 Re: mORMot 1 » SignUp on HTTP » 2018-01-23 06:14:52

I think the so-called initialization vector or nonce or salt, since it's already being sent to the client in plain text, when posting back to the server, just include that salt in plain text would be ok.

#546 Re: mORMot 1 » http.sys traffic capturing? » 2018-01-21 15:35:50

@mpv,

I can confirm I'm using the TWinHTTP-based clients.

Looks my computer is a bit in a mess - the button for going into the proxy settings UI in th eInternet Options window is grayed out, at the moment I don't have the time to handle it, so at the end I resorted to this approach - specify "127.0.0.1:8888" as the proxy server (which is Fiddler4's listening port) when creating the TSQLHttpClient connection.

Thanks for the help!

#547 Re: mORMot 1 » http.sys traffic capturing? » 2018-01-21 09:24:15

@mpv, oh yes, through a new firewall I installed yesterday, I noticed the 'proxy changed' notification upon fiddle4 starting and closing.

But, how to " configure a mOrmot client to use a default system proxy"?

I noticed the aProxyServer and aProxyByPass parameters for TSQLHttpClient.Create(), as I understand, I can specify a proxy server to use, and specify which hosts to not use the proxy.

Thanks again smile

#548 Re: Language » Pascal Generics are great or we are just following other languages? » 2018-01-21 09:04:17

Yes, we can live without generics, but life is much better with it, especially with a collection library provided by Spring4d smile

#549 Re: mORMot 1 » http.sys traffic capturing? » 2018-01-20 15:07:48

@mpv, thanks for the info. I my previous testing Fiddler  captures only some of the http traffics but not the mORMot client traffics. Not sure why, but will try again next time.

#550 mORMot 1 » http.sys traffic capturing? » 2018-01-15 14:22:02

edwinsn
Replies: 7

Has anybody successfully setup tracking of the mORMot http/WebSockets traffic between the client and the http.sys server, with well-known tools such as fiddle, fiddleCap or WireShark?

Today I tried all the three without luck. it seems that fiddle and fiddleCap (really helpful if it works) captures WinInet only, so  the two work with the tracffic of IE which uses WinINet.

Anybody has any hints? Thanks.

Board footer

Powered by FluxBB