You are not logged in.
Our thoughts and prayers are with you and the Ukraine people. This war should never have started. We hope it ends quickly.
Hi @ab
In your blog post about mORMot 2 performance, you mention that "The official release of mORMot 2 is around the edge". We are very curious (and excited) for that day. Is there a specific date or feature parity that must be reached or are we getting close? ![]()
Thanks Arnaud. This is very exciting and must have taken you a lòt of time!
So much appreciated. It is a very useful expansion of the framework.
Can't wait to get my hands on the examples and start playing with this.
Yes, that is probably it. Maybe its a browser issue. I wont worry too much about that now.
32 threads
Maybe this is a browser issue and not a framework issue. It happens when I test using two tabs of the same browser, but not when using two different browsers at the same time, such as chrome and firefox.
I have done something similar to the documentation, but it looks as if all requests are then executed sequentially. I created a method that executes the functions like this
if Services['Calculator'].Get(svr) then
if SameText(req.FunctionName, 'Add') then
Ctxt.Returns(svr.Add(1, 2))To come to that conclusion, I did something like this in Add:
...
StartTime := now;
Sleep(5000);
StopTime := now;
Result := '{"function": "Add", "StartTime": "' + DateTimeToStr(StartTime) + '", "StopTime": "' + DateTimeToStr(StopTime) + '"}';
..When I open 2 browsers and call the above method at almost the same time, the results indicate that they are executed one after the other:
browser1: {"function": "Add", "StartTime": "2022/05/12 11:26:56", "StopTime": "2022/05/12 11:27:01"}
browser2: {"function": "Add", "StartTime": "2022/05/12 11:27:01", "StopTime": "2022/05/12 11:27:06"}Is this the expected behaviour, or am I doing something wrong? The interface was declared with sicShared, but the same effect is seen when using sicPerThread or sicSingle .
I need to temporarily put something in place of a Datasnap server that third parties integrated to. So what is important, is that I maintain exactly the same uri formats. Unfortunately that means that third party clients over which I have no control, need to be able to make POST requests such as /root/Calculator/Add/1/2 which I need to translate to the registered interface functions, ie to ICalculator.Add(1, 2) and execute it.
How would I then find and execute the correct function on that interface with the correct parameters? I assume I'll have to search for the methodindex from the interfacefactory and then step through and assign the declared parameters?
Have someone done something like this before?
Since ExecuteCommand is private, how would the function be called after the parameters are assigned?
Is it currently possible to do for example
GET 'customers/{CustomerId}/invoices/{InvoiceNumber}'
or
POST /root/Calculator/Add/1/2
using interface based services?
TSQLRestServerFullMemory
It looks as if it always thinks the binary data is json, so maybe that is used as a fallback when it can't determine the data type? Since these are partial files, not full files (which would be too big - up to 1 gb), I assume determining the content type will not succeed. How can I provide / override / force the content type when serving binary data?
Providing the full path of the original file does not resolve the issue - is still sends the content-type as json.
I am attempting to correctly set the content-type when sending parts of binary files using code like this:
AddToCSV('Accept-Ranges: bytes', Ctxt.Call.OutHead, #13#10);
AddToCSV('Content-Range: bytes '+ inttostr(RangeStart) +'-'+inttostr(RangeEnd)+'/'+inttostr(SrcFileSize), Ctxt.Call.OutHead, #13#10);
AddToCSV(GetMimeContentTypeHeader('',SrcFileName), Ctxt.Call.OutHead, #13#10);
Ctxt.ReturnBlob(blob, HTTP_PARTIALCONTENT);In the procedure, the content-type is correctly set as video/H264 in the Ctxt.Call.OutHead, but when the header reaches the client, it is content-type: application/json; charset=UTF-8
Any suggestions where this might be overridden? I have tried stepping through ReturnBlob, but when it returns to my proc, the header still looks correct, so I am not sure where it is happening.
Using v1.18
Hi ab
Streaming the file directly from the server is great. I wish this was possible with mORMot 1. If I may get back to the second half of dexter's question - saving of the file by the client. Is there a way for the client (ie TWinHTTP.Request) to stream the bytes received from the server directly to file/disk without caching it in memory first?
That would work, I'll just have to consider that the functions in the dll will need db access and should not be able to be used by other unauthenticated programs/users
Would the opposite of this be possible?
I have a generic mormot server that is used for authentication and to perform some calculations and database actions. This runs offline directy at customers' sites. For some customers, some of the interface functions must be replaced to do calculations differently.
Since I do not want to recompile the exe for every customer request, I'm hoping to move only those functions to a dll and have the mormot server use the custom versions of the functions in the dll if they are present.
From my understanding of TSQLRestServer.ExportServer, the expectation is that the entire mormot server will live inside the dll. Is that correct? I hope to keep the rest server inside my main exe (since that is its only reason to exist) and just extend it with additional or replacement interface functions from the dll. Would that be possible?
Yes, extracted the latest from that release into the static folder.
It is caused by mormot.db.raw.sqlite3.static
Unfortunately it does not specifiy a specific unit, just mormot2tests.dpr being open.
I assume that still having the mormot 1 paths in my library path would not be an issue?
The issue happens as soon as any of these units are included in the uses clause:
// test.orm.sqlite3 in '.\test.orm.sqlite3.pas';
// test.orm.extdb in '.\test.orm.extdb.pas';
// test.orm.threads in '.\test.orm.threads.pas';
// test.orm.network in '.\test.orm.network.pas';
// test.soa.core in '.\test.soa.core.pas';
// test.soa.network in '.\test.soa.network.pas';So it will be something that is shared by all of them. I'll try to isolate it to a specific cause.
Hi ab
Not sure if this is relevant yet, but reporting incase it can be useful. Downloaded the latest release (172d480) from github's releases page and was able to successfully build and run mormot2tests for win32. No errors reported.
Changed mormot2tests to Win64 and when building using Delphi 10.3.1 on Win10 the compiler returns the following error:
[dcc64 Fatal Error] F2084 Internal Error: L3226This is a Delphi internal error, so its probably an issue with Embarcadero's compiler. I dont see any other useful information to report. Restarting the ide did not remove the error.
Edit: forgot to mention that it also happens with the clone of the repository, not just the release
That works perfectly :-)
Thanks @ab. This is seriously impressive support.
It looks like another GotoNextNotSpace() might be needed in SynCommons.pas in the function TDynArray.LoadFromJSON function before line 50300, since that is where it gets the space value instead of the next [ character. Unfortunately I dont know the code well enough to know the impact of changes there.
Interesting is that in all scenarios, JSONArrayCount return the correct count.
Thanks @ab, I'll try to find it. It looks like the issue is in SynCommons.pas in the function TDynArray.LoadFromJSON function (around line 50270). I'll try to isolate it further and provide a more precise test scenario.
in mORMot.pas, in function TServiceMethodExecute.ExecuteJson at line 60667 the following code is called:
smvDynArray: begin
Par := fDynArrays[IndexVar].Wrapper.LoadFromJSON(Par);for the correct json, Par has a value, and for the incorrect json, Par is nil. In this scenario, the Par value that gets sent to LoadFromJSON looks like this when the function succeeds:
' [["ItemA1", "ItemA2"],["ItemB1","ItemB2"]], "ID'and like this when it fails:
' [ ["ItemA1", "ItemA2"],["ItemB1","ItemB2"]], "ID'A 3rd party logged an issue when consuming one of my rest server's functions. I tested and confirmed it on Mormot 1.18.6261. Not sure how to resolve it. Any suggestions would be welcome.
In my interfaces, I use the following type as an input parameter:
type TStringDataSet = array of TStringDynArray;
...
function Test(const ID: Integer; const Data: TStringDataSet): TServiceCustomAnswer;When a POST is done using the following json body, the call succeeds and the data is correctly parsed and handled:
{"Data": [["ItemA1", "ItemA2"],["ItemB1","ItemB2"]], "ID":12}However, when the json is formatted to include either a line break or spaces between the opening square bracket of Data and its first item, a 406 Not Acceptable error is returned. The json is still valid. Sample json causing the issue:
{"Data": [
["ItemA1", "ItemA2"],["ItemB1","ItemB2"]], "ID":12}
... or ...
{"Data": [ ["ItemA1", "ItemA2"],["ItemB1","ItemB2"]], "ID":12}Is this something that the mormot parser should be able to handle, or is it something that I'll need a custom serializer for?
Moving to Linux can be very frustrating, but is absolutely worth it. I would move the rest of my work to Linux in a heartbeat if Delphi could run natively on it. Ironically Lazarus is the tool keeping me on Windows. There just is not enough patience in the world to make that possible.
What would be the most efficient way to format ISQLDBROWS into json that uses the format below?
{
"fieldCount": 3,
"rowCount": 2,
"fieldNames": ["DateData", "IntData", "StrData"],
"values": [
["2016-03-14T13:23:25", 1, "Hello"],
["2016-03-14T13:23:45", 2, "Bye"]
]
}I basically need to create a custom version of FetchAllToJSON. Should I create a new TextWriter and manually step through all the records and fields, or is there maybe a function that can return the entire row in json?
Thanks Vitaly that looks like an option, but also looks scary, since I'm not too sure what impact that will have.
I suppose that would entail overriding both TSQLRestServerURIContext.ServiceResultStart and TSQLRestServerURIContext.ServiceResultEnd and replacing them with versions that look for a new boolean property which causes them to not make any changes to wr at all?
Thanks ab
It can get very complicated to add/update out parameters on many functions when a standard long/nested record result is to be returned, especially if the expected output is prepopulated as a record by a universal function.
Is there maybe an event on TSQLRestServerFullMemory that can be overridden before the data is sent to manually remove the leading {"Result": and trailing " from the outgoing json string, or another property that can be set to not add it at all?
Thanks ab
It is now up to 6203
Hi ab
I used the TFullname records posted by cypriotcalm and added them to the calculator in the sample project 27 (Project14ServerHttpWrapper), since this declaration is already done in there to test the result:
aServer.ServiceDefine(TServiceCalculator,[ICalculator],sicShared).ResultAsJSONObjectWithoutResult := true;I added a test function like this:
function TServiceCalculator.GetSingleFullname: TFullname;
begin
Result.Firstname := 'First Name';
Result.Lastname := 'Last Name';
end;and expected the result to not contain "Result":
However, it looks like it always encapsulates it in a Result ie
{"Result":{"Firstname":"First Name","Lastname":"Last Name"}}Is that the expected behaviour of this property, or did I misunderstood the reason for it? Is the idea behind this property not to send the data without result, ie:
{"Firstname":"First Name","Lastname":"Last Name"}?
Is it possible to send a record like that without Result being added? It seems that using RawJson also always include Result?
Hi ab
Are the Nightly Builds of 1.18 still being built nightly or should they be downloaded elsewhere? When using the link on the downloads page (https://synopse.info/files/mORMotNightlyBuild.zip) it returns the build from 2020-11-23 up to commit 6171
Is that correct?
Thanks ab
It looks like the code provided by @MCvanderKooij works very well as-is, I just had to add SynTable to the uses clause of uJWTRSA.
Is it possible to create RS256 JWT assertions using mormot?
I need to integrate with 3rd party systems that requires it, and looking for a pain-free way to generate the jwt assertions. Unfortunately rs256 is a requirement from them.
Thanks mpv, that is a good idea and will fit very well within the current process, even better than sms or email 2fa
Thanks for the suggestion ab, I'll check on JWT again.
I'll probably still need to have mormot sessions, since we use a complicated permissions system. So some of the interface functions look up the current session user (and some properties captured when he logged in) to determine which actions may be allowed. From what I understand from https://synopse.info/forum/viewtopic.php?id=4840 that TSQLAuthUser may not be available from the context.
Is anybody using 2 factor authentication together with mormot?
I am thinking of adding (optional) 2fa to a web app (javascript) that currently authenticates against a mormot server and uses its interfaces. There are no Delphi/desktop programs connecting to it, just JS clients.
I already use a custom TSQLRestServerAuthenticationDefault class for authentication, so assume that the overridden Auth and/or GetUser functions will need to be changed to support that. What I'm not so sure about, is when the TSQLAuthUser will be created. Depending on the type of 2fa (sms, email etc) a significant amount of time can pass between the first authentication and when the 2fa value is provided.
Ideally the process would be: authenticate, send 2fa key, get 2fa key from user, activate the login that has already happened. But I'm not sure that will work, because then, either the user will have to log in again, or the activation of the login session will have to be manually checked in every interface to prevent the pre-2fa login session from getting access to the interfaces.
If anybody already added 2fa to their projects, some guidance/pointers will be very helpful.
Thanks Daniel! That saved me a lot of time. Based on the error, I incorrectly assumed that StripSemicolon was the cause, it didn't occur to me that the cause was the actual connection.
I tried adding the option in the linked mysql documentation and still get the same issue, but at least now I know where to look ![]()
Edit: I found this burried deep in the documentation: SQL syntax for prepared statements does not support multi-statements So now I'm back to where I started. How to sucessfully get the last inserted id, without mysql just returning zero.
Does anybody know how to achieve this? Any suggestions will be helpful.
How do I set the StripSemicolon to false when running queries with ODBC Connections (TSQLDBConnectionProperties.Execute) on Mysql?
I need to insert records and get the last inserted id back. Chaining the select to the insert statement works with firedac connections (libmysql.dll) but not with odbc connections. This is needed, because if the SELECT last_insert_id() is run separately after the inserts, Mysql sees it as coming from a different connection and returns zero, which makes it impossible to do inserts on master-detail tables.
The following query works perfectly for Firedac, but not Odbc, because the semicolons are removed from the Odbc statement, but not from the Firedac statement:
INSERT INTO test (fld1, fld2) VALUES (?, ?) ; SELECT last_insert_id()With MS Sql Server its easier, because there I can use a statement like this, which returns correctly:
INSERT INTO test (fld1, fld2) output inserted.id VALUES (?, ?) Any suggestions will be very helpful.
Thanks @mpv. That is exactly the info I was looking for ![]()
Is there a maximum number of parameters that can be passed to a query when using Firedac?
I am able to perform a big insert on Mysql when connected with TodbcConnectionProperties without issues, but when using Firedac (libmysql.dll), I get the following error:
{
"errorCode":500,
"error":
{"ESQLDBException":{
"ClassName":"ESQLDBException",
"Address":"a8fef1 SynDB.ReplaceParamsByNames (8038) ",
"Message": "Parameters :AA to :ZZ"
}}
}It is raised by ReplaceParamsByNames which does not seem to be called when using the odbc connection. Am I correct? If so, what is the maximum number of parameters that can be used in a query, or is this limited to a byte size and not a count?
Would it be possible for me to determine this limit in my code before running the query?
That solves about 99% of the issues I was facing.
Thanks @ab!
We need to add parameters to the interface functions in our Mormot service. Since all parameters for interface functions are required values, adding an extra parameter breaks backwards compatibility. I remember that there is a property that can be set to make parameters not required, but can't remember its name, so can't find it in the documentation (I know it is there somewhere, just don't know how to search for it).
To ensure some form of backward compatibility for future changes to these interface functions, I can see 4 options:
1) Make all parameters for all interface functions optional (using the property mentioned above). This will have a big impact, since all the interface functions will then need to be changed to manually check whether all the parameters have been supplied and are of the correct type.
2) Add default values for new parameters which will apply if they are not supplied (doesn't look possible)
3) Add a versioning scheme (maybe with url rewrites?) which will add missing parameters with default values before calling the real function ie /v1/User/CreateUser will execute /User/CreateUser after adding the new (if not supplied) parameters to the input
4) Basically replace the input parameters for all interface functions with something generic like:
function TUser.CreateUser(const Parameters: RawJson):integer;which has the same impact as 1)
I'm sure I'm missing something obvious, but would anybody have suggestions as to what is the correct route to take here? This service is used by third parties, so ensuring backwards compatibility is all I'm interested in. However, if I am going to break backwards compatibility with a major version bump, I should apply it to every function.
This link describes it better than I can:
https://techcommunity.microsoft.com/t5/ … a-p/324623
"Client Certificate Authentication is a mutual certificate based authentication, where the client provides its Client Certificate to the Server to prove its identity. This happens as a part of the SSL Handshake"
I need to perform secure tcp communications to a 3rd party server where client certificate authentication is used and where the server is configured for this part to not be optional.
When making a TLS tcp connection to a 3rd party server using TCrtSocket, the server requests a client certificate. Valid certificates with client authentication key usage purposes are installed on the client's windows certificate store. How do I provide these to the server over the standard tls tcp connection?
Is it possible to use TLS v1.3 on tcp socket connections with mormot or this wrapper?
Thanks ab. I always forget about _Safe, somehow TDocVariantData(Data) still comes more readable and naturally to me.
This is why politics and programming should never mix. In this instance I unfortunately don't have a say in what the client does. They use an html select dropdown, which can sometimes be a single value and sometimes a multiple select. While it would take them 10 seconds to ensure an array is always sent, its not happening.
My interim solution to keep the project on track, was to keep the grouping as a Variant, but then internally copy it to an array. It might not be elegant, but will do for now. I'm sure there is a more efficient way of doing this.
procedure ParseVariantToArray(const Data: Variant; var Arr: TStringDynArray);
var
I: Integer;
begin
SetLength(Arr, 0);
if (not VarIsEmpty(Data)) and (not VarIsNull(Data)) then
begin
if VarIsNumeric(Data) then
Arr := [FloatToStr(Data)]
else
if VarIsStr(Data) then
Arr := [Data]
else
for I := 0 to TDocVariantData(Data).Count -1 do
Arr := Arr + [TDocVariantData(Data).Value[I]];
end;
end;What would be the best option to overload an interface function? I understand that mormot would not know which function to call, so maybe declaring all the parameters as variant?
I need to provide somthing silimar to this (have unfortunately no say in providing as 2 differently named functions - external requirement):
function CountItems(const grouping: string): integer; overload;
function CountItems(const grouping: TStringDynArray): integer; overload;Would using a plain Variant work? I tried this:
function CountItems(const grouping: Variant): integer;which works fine for the string, but how would I use that as a TStringDynArray?
Currently on Windows, I can access an instance of an interface like this
TMyInterface(aRestServer.Service<IStaff>).NewSetting := 10;Since
function Service<T: IInterface>: T; is only declared for Delphi2010 and up, how would I do this on Linux using fpc?
Would this be the same:?
var tmp: TMyInterface;
...
if aRestServer.Services.Info(TypeInfo(IStaff)).Get(tmp) then
TMyInterface(tmp).NewSetting := 10;I use Delphi 10.3, but this has been a thorn in my side for a few months, so also had the issue with Delphi 10.2 and older versions of the framework. The most recent site where this happened was built with a nightly build downloaded on 2019-02-14. In which file is the build and version number saved, so I can paste it here?
I'll get more up to date logs for the full call stacks and post when it happens again. I even had traces showing the issue happening in ntdll.dll (which is unlikely, probably just bubbled up from odbc drivers). But this caused me to think that maybe there might be a reference to a parameter that gets freed before the query is passed to the driver. Since most of the sites still use the 32bit version, I can not currently specifically rule the 64bit version in or out.
Was just trying to find out if others have experienced something similar which can help me to replicate it, which would make debugging possible. I've never had it appear in the lab while compiled in debug mode, always only the release versions on clients' premises.