You are not logged in.
@flydev @tbo
Sorry for the delay — I had other issues here.
I will study both examples and write feedback to you as soon as possible.
Thank you both for your time.
Instead of using ROOT_NAME_FILE, I simply wrote 'root'.
Since I don't have TFileAuthUser, I replaced it with TAuthUser in my code. However, this led to the errors I mentioned earlier, which is quite confusing.
Without this example, how would a user even know how to implement this correctly? I still don't know how to do it
By the way, my mORMot2 version is up to date as of yesterday.
On the client side I'm using
Client.SetUser(User.UserName, User.Password)
.
Then, I've adapted the example code as this:
var
vAuthUser: TAuthUser;
begin
vAuthUser := TAuthUser(Server.SessionGetUser(ServiceRunningContext.Request.Session));
if Assigned(vAuthUser) then
try
// => vAuthUser is always NIL...
finally
vAuthUser.Free;
end;
// my own code here...
end;
But `vAuthUser` is always NIL.
And if I set
Server.AuthenticationRegister(TRestServerAuthenticationDefault)
, the authentication fails directly, without even enter on the method.
I don't want to use "model", but for testing I added
CreateWithOwnModel([TAuthGroup, TAuthUser], {HandleUserAuthentication=} True);
like the example, but then I got errors:
20241009 16175800 EXC EModelException {Message:"TAuthUser is not part of TOrmModel root=root"} [Main] at 443f23
20241009 16175800 + serv.core.TTaskServer(02472660).Shutdown() root CurrentRequestCount=0
20241009 16175800 - 00.000.837
20241009 16175800 info serv.core.TTaskServer(02472660) TRest.Destroy root
20241009 16180217 EXC EModelException {Message:"TAuthUser is not part of TOrmModel root=root"} [Main] at 443f23
Exception EModelException in module serv.exe at 00043F23.
TAuthUser is not part of TOrmModel root=root.
This part is VERY confusing and I couldn't find any place that explains this — even in ChatGPT.
Can you help me?
Thanks Thomas,
But I couldn't see any call to `Server.AuthenticationRegister(TRestServerAuthenticationDefault)` or so in this example.
If I don't use this line, in my code, there is no authentication and no error — but I need authentication...
I can't compile this example, as it uses another (newer) Delphi version — I'm using D7.
The client side have Client.SetUser(user, passwd) method (TRestHttpClient class).
On the server side, how can I obtain this same information (user, passwd), within the implementation of the class that implements the interface (which is shared between client and server, i.e., interface-based)?
Could you give me an example of how to do this?
My ideas are:
- the service will get the data of all users when it is started — I will get it "manually", with a query and saving it in some object for each thread instantiated
- the client will log in, using a method of the service, to start the application
- then, for each method called (interface-based) the server should read the username sent in the requests "somewhere that I don't know how yet" — because I don't want to have to send the username as a parameter of each method of the interface
PS: I'm using Delphi 7.
I would like to implement user/password authentication. What is the simplest way to achieve this (perhaps using TRestServerAuthenticationDefault)?
Additionally, how can I implement and verify it on the server side without using ORM part?
I mean, I need to check if user/password are Ok and, at the same time, use the username (after it was validate) on server side.
@Marco
You know where it was meant to appear next: in the TIS grid.
Sure
Nice improvements!
Note that those both lines compile and run with the antique Delphi 7 compiler - who said the pascal language was not expressive, even back in the day?
And I still have mine, which still working on production nowadays!
Thanks.
vIndex1 points to vDoc.Value[1] as a direct pointer.
It does not share the data, it is just the same data.
In this case, same data or sharing the data is the same to me. But, Ok, we're at the same page.
If you want to create another TDocVariantData to hold the values, it won't be a direct pointer.
At most, it could be a vtPointer kind of variant.
Which is not the same: if you change the value in the 2nd TDocVariantData, it won't be reflected on the first/main TDocVariantData.
What doesn't work for me
"sharing the same data" means that all (sub)objects will point to the same data of the main object.
Currently, I can do that using arrays, creating new objets for each index and those objects points to the main data.
For example, we can split an array of objects, creating new objects for each item, then changing any value on those subobjects it will change the main array data:
var
vDoc: TDocVariantData;
vIndex1: PDocVariantData;
begin
vDoc.Clear;
if vDoc.InitJson('[{"id":1,"text":"blabla"},{"id":2,"text":"toto"}]', JSON_FAST_FLOAT) then
begin
ShowMessage(vDoc.ToJson);
vIndex1 := _Safe(vDoc.Value[1]);
vIndex1^.S['text'] := 'new value'; // shared data; it will change both object
ShowMessage(vDoc.ToJson);
ShowMessage(vIndex1^.ToJson);
end
else
ShowMessage('invalid json');
end;
Following the same idea, it would be nice if we could split an object, creating new objects for each property of the main one — but sharing the data, of course.
From a TDocVariantData object, I would like to split its properties into new instances.
For example, initiate a DocVariantData (dv for short) using this JSON:
{"obj:"{"id:1,"text":"blabla"}}
name = "obj"
value = {"id":1,"text":"blabla"}
I can iterate the dv using its enumerator:
var
dv: TDocVariantData;
pair: TDocVariantFields;
begin
if dv.InitJson('{"obj:"{"id":1,"text":"blabla"}}', JSON_FAST_FLOAT) then
begin
for pair in dv do
begin
// here pair.Value^ will contain {"id:1,"text":"blabla"}
// now, how can I create two new PDocVariantData objects...
// 1 = {"id":1}
// 2 = {"text":"blabla"}
// ...but both sharing the same data from dv ?
end;
end;
end;
I haven't found any _Obj* function or TDocVariantData.Method to do that.
Maybe a new _ObjFast(PRawUtf8, PValue) or so could be create?
Thanks.
Congratulations on achieving this!
It is already used on production for some projects, and we will use it on our side - for https://www.tranquil.it/en/ - for a huge project in the next months.
One more interesting case for mORMot!
Right, thank you.
Thanks and I'm sorry, I forgot that these appears on documentation:
FPC SVN 45643 (3.2.0)
Lazarus SVN 63526 (2.0.10) << but here isn't the same
Hi,
Which versions of FPC and Lazarus are you currently using — installed by fpcupdeluxe — that are working well with mORMot 1 and 2 versions?
I'm building another version of my Lazarus for new projects.
Thanks.
So, Zeos will require mormot always?
I wouldn't say always, but if Zeos needs mORMot, this code should be provided from a package.
IIRC, by the time, we didn't create a package for Zeos because units were not separated as they are now in version 2.
Thinking out loud:
What if we define a mormot2 package and a mormot2-zeos package (including only mormot.db.sql.zeos.pas) ?
Together with a zeos package and a zeos-mormot2 package.
The zeos-mormot2 package could have zeos and mormot2 as dependency, and mormot2-zeos both mormot2 and zeos as dependencies.Or even just compile zeos with no MORMOT2 flag - and use this plain zeos package from mormot2-zeos.
I suspect the performance optimization may not be very high, since zeos is already well optimized.
If a project need the extra-performance of the MORMOT2 Zeos flag, it could use it explicitly within its build process - but for the development within the IDE, a plain zeos without MORMOT2 could be enough.It could fix the dependency issue, I guess.
We should create a "mormot.base.pack" (or mormot.base.run, following the "suffix pattern" which "run" stands for runtime packages and "dsgn" stands for IDE installed package).
Then, create another one just for Zeos, mormot.db.zeos.pack.
Zeos team should use mormot.base.pack instead .inc, conditionals, or paths for mORMot.
"For example, zcomponent require SynCommons from mormot_base that require zcomponent."
Are you sure that Zeos is using SynCommons on its official repository? I don't update Zeos for a long time...
If yes, Zeos may use mORMot package instead to use SynCommons.pas direclty.
Hi,
You shouldn't add relative path to folder where Zeos.inc is located but, instead, add`zcomponent` package from ZeosLib into it (item 5 from README).
@okoba mORMot 1 has packages. I guess v2 should be released first, than we can add the packages following what was done in v1.
Congratulations for this excellent work!
SynUnicode is either UnicodeString or WideString - the faster on the platform. On FPC, you have UnicodeString <> string by default, and UnicodeString=WideString on POSIX... So our SynUnicode is somewhat more consistent if you want a fast UTF-16 Type. Perhaps UTF16String may be more namingful, and closed to Utf8String convention.
For sure SynUnicode should exist - the faster on the platform, you've said - but I'm talking about the naming, which could be (T)SynUnicodeString following the convention.
About 'T' prefix, it was never the case for simple raw types like integer, string or such.
I know, from Borland yeah, but I think that following a convention is better, even more important in Pascal that is no case-sensitive. A 'T' helps for not having a collision name with a property for example.
RawUTF8 isn't a good name, but better than just Utf8.
Using just Utf8 seems not complete: is it a string, a stream, a char, what?
Every string type has String as suffix so, to keep a consistence a new string type should use it too. Think for example in UTF8Char type, should we rename it too? A char is "less" than a string, so a char could use Utf8 and Utf8String is a-string-of-chars(?)
SynUnicode is inconsistent because "say nothing" about the type, as it doesn't keep the naming using String as suffix—that should be renamed.
If you really want and will change the name anyway, don't think only in RawUTF8, but RawJSON, RawUnicodeString, and so on. The prefix "Raw" might be removed.
Last, you should use "T" as a prefix for all types. PRawUTF8 has "P" because it is a pointer, as TRawUTF8 uses "T" because it is a type—I don't care if Integer and Currency don't have a "T", it is inconsistent anyway.
Finally, the perfect name should be TUtf8String but I guess you won't use it...
Original code is complex (and I cannot post here). I will make an simple example without any access do DB just to know if this issue is something related to privilege to install—even though I'm using Admin, but the machine is not mine. It's part of a network with its own rules, antivirus, AD, etc.
If it works, it means that has something wrong in original code... however, it works perfectly on console mode.
Guys, please, any more ideias?
Your OpenSCManager failed with ERROR_ACCESS_DENIED.
I think you must use "Run as Administrator".
I use from the beginning...
So /install works but /start fails?
If this is the case, then the service may be half-installed.
IT happens when a service installation or execution was broken, due e.g. to the executable still running when uninstalling.
Unstall it, restart the PC, ensure it is actually uninstalled, then reinstall it.
Yes, /install works but /start fails.
I did what you said and I got the same error.
On log, I have 2 Stop() calls when I tried to execute /install for the first time, after reboot.
Yes, I can install without any problem.
If I use "/start", it seems that it's started (I can see on Windows Services that status was changed) doing nothing, as it was a fake start.
Debuggint I can see below that:
1. FHandle=0
2. backupError=5 — I'm not sure why 'OpenSCManager(''%'',''%'') for [%]' wasn't register on the log file, as I'm using "*", though.
constructor TServiceController.CreateOpenService(const TargetComputer,
DataBaseName, Name: String; DesiredAccess: DWORD);
var backupError: cardinal;
begin
inherited Create;
StringToUTF8(Name,FName);
FSCHandle := OpenSCManager(pointer(TargetComputer), pointer(DatabaseName),
GENERIC_READ);
if FSCHandle = 0 then begin
backupError := GetLastError;
ServiceLog.Add.Log(sllLastError,'OpenSCManager(''%'',''%'') for [%]',
[TargetComputer,DatabaseName,FName]);
SetLastError(backupError);
Exit;
end;
FHandle := OpenService(FSCHandle, pointer(Name), DesiredAccess);
if FHandle=0 then begin
backupError := GetLastError;
ServiceLog.Add.Log(sllLastError,'OpenService("%")',[Name]);
SetLastError(backupError);
end;
end;
After that, returning to the TSynDaemon.CommandLine() the function Show() is going to be executed:
cStart:
Show(ctrl.Start([]));
Than, the last error changes for "6", after execution of StartService():
function StartService(hService: SC_HANDLE; dwNumServiceArgs: DWORD;
lpServiceArgVectors: Pointer): BOOL; stdcall; external advapi32
name 'StartService'+{$ifdef UNICODE}'W'{$else}'A'{$endif};
Here you can see the original error, debugging mORMot:
I've created a console program which is a HTTP server - OK
I've created a daemon program that uses the same classes from console program - OK
I can run the daemon using /console and it works - OK
However, if I try to /start the daemon, it "starts" but I got an error on log:
20201217 21340931 ! + ServDaemonCore.TDaemonService(026fb3e8).Stop
20201217 21340931 ! - 00.000.000
20201217 21340933 ! EXC EOleException 80040e4d ("Especificação de autorização inválida") [] at 4fc65e stack trace API 46ae31 404de8 77a171b4 77a03b36 4fc65e 407977 4e34b4 70b9ac 712bb1 713082 4d3c99 4d3601 760d1946 75f5fa29 779f75f4 779f75c4
It is in Portuguese... but debugging the original error is "Error 6 - The handle is invalid when trying to start services".
Running on Windows PRO; user is Admin; it's not antivirus.
Any tips?
i got :
syndbexplorerFrame.pas (line 803) Intenal Error C15309
any idea what should be done
Try delete all .dcu for this project and use Build instead Compile.
I think it is a Delphi 7 bug that happens sometimes...
You still haven't confirmed if you are using http.sys?
Yes, I'm.
From the existing logs, you should be able to find out how much time each request take, and how the thread pool is used.
If indeed the Enter/Leave of TSQLRestServer.URI takes a lot of time, and all threads are used, then you have a thread retention.
Since the problem is that there is a blocking process, and you need to wait, it is difficult to change anything.You could try to increase the number of threads from 32 to 256 (which is the highest value allowed). It would consume a little more memory, but may help preventing timeouts.
I've already increased the number of threads from 32 to 64 and I didn't have complains so far.
This code is the same I've talked in another thread: a rewrite of code from Desktop perspective to Web. Some process take 500ms and that is great in Deskop, but not for Web. I still have a lot work to do...
Are you using a reverse proxy in-between?
Are you running on Windows?On Windows, the http.sys Web server is very stable, so I doubt there is any problem on that level.
If your requests take 100ms to the DB, perhaps there is some contention on server side.
No proxy. Yes, on Windows.
Yes the server is very stable, but it seems that my app is receiving more requests that it can handle.
Unfortunately, responses are taking ~1.5s in average. That is because my app is calling another microservice (Java) to do some stuff and it takes almost 1s to do it — I'm working to remove such dependency...
---
Not atm, but you can add it, follow my instructions here.
Thanks. But as @ab and I has said, I'm using Windows.
---
@mdbs99
Did you check https://docs.microsoft.com/en-us/window … or-httpsys as mpv proposed?
I looked quickly... I didn't know that tool. Sounds interesting... but as I understood, it just allowing me monitoring in real-time but no logging.
My problem is that they are complain about time out. But as I don't have any logs, because the request didn't touch my code, it looks like perfect from my point of view... so, I need (or I guess I need it) to have logs for those requests as a prove that they have happened.
I would like to log all in the app server. Is that any chance to mORMot do that automatically, as it does for all requests?
My issue is that a consumer says that he/she got a time out, but I have nothing in my logs because the request never touch the business rules (my code, not mORMot) at all. I would like to see how many requests have not been process to make changes based on this information.
I'm using the default configuration for my HTTP Server (i.e., 32 threads, etc...) and I got some issues from some consumers in Java: Read Time Out error.
I got nothing on log so, I imagine that all threads are working while a new request came and the server throw away after a while... (?)
My question is: how can I get aware about these requests that weren't process?
Thanks.
From documentation:
16.4.2. Set up the Server factory:
"The code line above will register the TServiceCalculator class to implement the ICalculator service, with a single shared instance life time (specified via the sicShared parameter). An optional time out value can be specified, in order to automatically release a deprecated instance after some inactivity."
Is that option time out value which we're talking about all the time, to finish inactive threads?
However it seems Delphi 7 is having a hard time with huge library files syncommons.pas and mormot.pas. Every other time i invoke code completion the IDE hangs.
I've been using mORMot on Delphi 7 with success, but I feel your pain.
Which I'm doing is:
- do not use code-completion: disable on IDE or increase in 2 or more seconds to call automatically.
- clicking on classes/functions works: you'll use it to see constructors, parameters, documentation, etc on the source.
- on SynCommons, etc, navigation doesn't work: forget Ctrl+Shift+up/down, continue Clicking on tokens.
@ab
I believe, as he is using a gateway, he don't want the gateway manage these callbacks, only "tenants" know what do do with.
Perhaps one potential use of packages for mORMot may be to use them instead of source for the Delphi IDE.
Besides, mORMot already has packages for Lazarus...
Yes, I am making a cross compiler, Win64 -> Linux 64.
I didn't know about these problems.I will try to make this Win32 -> Linux cross compilation and report the result here.
Thanks for the informations.
You can read more about the context and the confirmation from FPC Team.
I know that you don't control clients, I meant that you wrote the service implementation on the server side. Sorry if I didn't make that clear.
Sorry if I didn't understand you too.
I agree with pvn0.
I cannot shutdown the service, even after this period, if the client already was using the service before
If there is information that will only be on instance A and that cannot be continued on instance B, is this not a type of session?
Instead of trying to discovery if mORMot does what I want to implement, you (all) are trying to know "why" I need it...
Guys, maybe the archtecture there is wrong; maybe storing the URL's of services in a database is wrong... but this is already exists and I need to work with.
The thing you call "A/B service" is known as "Blue-Green Deployment" (google for it) and it usually implementing using some kind of load balance.
Announcing a new URL to client is tooooo complex.
Better to set up a load balance in front of your services, so you can start new instance, instruct load balance to proxy new requests to the new instance and when shut down old instance.
I use nginx for such task, but HAPxoxy is also good candidate. Even IIS may be can do such.
I can't choose what I want to use. They are paying me to do a job. That's it. Maybe I can use IIS, but I don't know yet.
I wrote "A/B service" (quoted), as it is just an idea and not the Blue-Green Deployment. Thanks, anyway.
You are writting very contradicting things, you say there is no session but your entire problem seems to be session related, or lack of.
That is because I'm thinking far. I have a job to do now, but new ones in a near future. I'm apologize if it's confusing for you.
Right, you wrote the service implementation, so if a client is using instance-A when you swap to instance-B, YOU should know that already because you wrote the service implementation
I've already said to you that I don't control clients. It's not about a Delphi client-server. Is much more than that... Java, services, website... anyway.
Okay, now I think I understand.
You return the URL at the beginning of the session and it will be the same until the client is finished.In theory a client can stay for hours in the same session and you would not be able to make this replacement within a constant period.
You can end up with hours of delay if the client remains open.
Almost that
Clients should get from a database the URL for each (active) services.
So, you don't need a session (this service doesn't have one). But clients SHOULD refresh its own "list of URL services" in a *period of time*. If some client haven't updated its list and still trying to use the instance-A—AFTER this period of refreshing—I don't care if he/she got a 400 error because he/she should used the active service, which is instance-B.
But I cannot shutdown the service, even after this period, if the client already was using the service before — that could exists in other type of services, which I will implement as well, and that is why I would like to know how to shutdow the service without kill someone unintentialy. I would like to do this properly, clean, without workarounds.
@ab is right as always
As I guy that worked directly with him, I always had this thought... but not this time
Just incase you need this or anyone is wondering :
THttpServer(aHTTPServer.HttpServer).ThreadPool.RunningThreads THttpServer(aHTTPServer.HttpServer).ThreadPool.PendingContextCount
RunningThreads will tell you how many threads in the pool are active and PendingContextCount will tell you how many requests are queued for processing. (don't call PendingContextCount too often because it uses a lock).
THANK you, to give some light on this thread! Now I start to think that is possible to do what I NEED to do.
I don't know about PendingContextCount (I will read documentation) but as I'm answering you: so, I can return this value and if it is zero I can know, for sure, that no resquest still in a list to be process, right?
Either way if you shutdown your server and your client gets an error you should either A.) be sure all clients have been notified of server change prior to shutdown B.) handle the error on client side.
I cannot control the clients, which is other apps (website, other Java services, of course the Delphi client, and anyone that wants to use that service).
But if server A is no longer receiving requests after a while, why would someone still be using it?
"I cannot stop the service even if just one person is using it, because I cannot control if this client got the new URL, from the database, to use the instance-B."
Whatever there are threads still working or not, is not relevant. I am afraid you still don't understand how the mORMot HTTP server thread pool works, and that "killing the threads" doesn't mean anything in our context. The threads are not running, they are sleeping until there is a request incoming.
Arnaud, I know that threads are not running if nobody is send request. Yes, they are sleeping, but I (as an admin of the service, not as a developer that has created) don't know if thread are sleeping or running, before I kill the process.
I cannot drop a user that is using the service. I just can't... and I'm afraid that you still don't understand me. Again, I cannot stop the service even if just one person is using it, because I cannot control if this client got the new URL, from the database, to use the instance-B!
There is already a feature in TSQLRestServer/THttpServer which allows a graceful shutdown of the server process.
If you just stop the mORMot daemon, it will shutdown properly, refusing any new input and finishing the pending requests.
So you can safely shutdown one instance, which won't accept any more request, and another one could take the next requests. If you want 100% up time, you can put a front-end, e.g. nginx reverse proxy with load balancing to the two instances.
Are you talking about mORMot 2.0?
If it already exists in 1.18 let us know, as everybody still using that version.
Another information is that I'm not using a demon (yet).
...refusing any new input and finishing the pending requests.
By "finishing" you meant stop to process in the middle, returning an error to the client? If I understood wrong, please try to explain again, otherwise I cannot do that.
PS: I cannot tell them what I would like to use (nginx, IIS, whatever). By now, I would like to use just mORMot service process.
...a time of 5 minutes after you have redirected requests to server B will be enough so that server A is not doing anything else.
For sure, I don't know. I can just suppose that no one is using...
I cannot guarantee that a service is always online, 100% of time, if I can shutdown someone that are using it because I supposed that no one was using.
Don't you agree?
PS: Thanks for understand me!