How configure the server to finish inactive threads?

mdbs99 · 2020-11-16 11:58:30

How can I configure the server to finish inactive threads in a period of time?
Doing that will prevent some errors — see https://synopse.info/forum/viewtopic.ph … 398#p33398 — and allow me to replace binaries without forcing a shutdown, for example.

--
Original post https://synopse.info/forum/viewtopic.ph … 402#p33402

Last edited by mdbs99 (2020-11-16 11:59:50)

ab · 2020-11-16 13:39:03

The threads are part of the thread pool, so they are by definition allocated at startup, until the process exits.
"Finishing" such threads is not the solution to the problem of DB connections, for instance.
And I don't understand what you mean by "allow me to replace binaries without forcing a shutdown" - I don't understand what it means. You can't replace a binary without shutting it down first. What do you imagine been able to do?

You are taking the problem from the wrong tail, I am afraid.

mdbs99 · 2020-11-16 14:15:51

I wrote with a context in my mind and I might haven't express myself in a good way.
I know that "The threads are part of the thread pool, so they are by definition allocated at startup, until the process exits." and there is nothing wrong with. What I'm asking is if there is a (default) way to a server kill those threads automatically, if they are inactive.

"Finishing" such threads is not the solution to the problem of DB connections, for instance.

It can help. If there are no new requests, threads will die and for the next new request they will be created again, using a new connection.

"allow me to replace binaries without forcing a shutdown" need a better explanation (my mistake, sorry):
I want to implement a "A/B service" which means that, if I need to replace the instance-A, I will first publish an instance-B, change the URI and port (that remains in a database) for all new requests point to the new service instance. Then, when all requests finished in instance-A, I'm able to remove the binary, replacing with the same version in B — for the next version I can do the opposite, starting with instance-B.

If threads are not killed automatically, I need to check the server using another approach only to know if something stay in process.
Just as an example, using IIS + FastCGI + fpWeb, I can setup IIS to kill my threads using the approach that I've explained above. But I'm not using IIS or Apache... just mORMot.

Have you understood now?

pvn0 · 2020-11-16 15:01:19

From what I understand, your own code blocks the threads from exiting gracefully. You should never do that and you should never kill a thread.

macfly · 2020-11-16 15:40:19

Actually, what you want to do is finish the application, right?

Ending threads being executed by the application will not allow the executable to be replaced.

mdbs99 · 2020-11-16 15:57:42

pvn0 wrote:

From what I understand, your own code blocks the threads from exiting gracefully. You should never do that and you should never kill a thread.

My own code do nothing. Threads are there, running forever and I'm not killing them.
I just would like that mORMot kill them, *if* they are doing nothing for a period of time...

macfly wrote:

Actually, what you want to do is finish the application, right?
Ending threads being executed by the application will not allow the executable to be replaced.

No, I can finish the app anytime, but "brutally"! If I call "shutdown" any moment, some thread may not finished its job and the client will receive an error.

But, if there is no thread running, I can call "shutdown" to kill the HTTP server quietly and replace the binary (with a cup of coffee in my hand...), as the new version of the service is already running in a new URL (instance-B), which means that my service is never down.

Guys,

All services are registered in a database. All clients should first get the URL from there to know where is the active instance.
If I want to replace any binary I will first change that register, pointing to a new instance (B), which means that all new requests will use the new version (B).
Then I just wait "the period of time" knowing that mORMot will kill all threads that are in stand by... only the server will stay active, then I call shutdown.

That is what I do to maintain the service online 24h/d. If you, guys, know another way to do that — I mean, always having an instance available for the clients — please, teach me.

Last edited by mdbs99 (2020-11-16 16:08:42)

macfly · 2020-11-16 16:17:09

Yes, I understand what you want to do.

But if server A is no longer receiving requests for a certain period, wouldn't it be safe to terminate it? Why wait for the mORMot service to report that is idle.

For example, a time of 5 minutes after you have redirected requests to server B will be enough so that server A is not doing anything else.

mdbs99 · 2020-11-16 16:29:04

macfly wrote:

...a time of 5 minutes after you have redirected requests to server B will be enough so that server A is not doing anything else.

For sure, I don't know. I can just suppose that no one is using...
I cannot guarantee that a service is always online, 100% of time, if I can shutdown someone that are using it because I supposed that no one was using.
Don't you agree?

PS: Thanks for understand me!

Last edited by mdbs99 (2020-11-16 16:35:01)

ab · 2020-11-16 17:11:35

Whatever there are threads still working or not, is not relevant. I am afraid you still don't understand how the mORMot HTTP server thread pool works, and that "killing the threads" doesn't mean anything in our context. The threads are not running, they are sleeping until there is a request incoming.

There is already a feature in TSQLRestServer/THttpServer which allows a graceful shutdown of the server process.
If you just stop the mORMot daemon, it will shutdown properly, refusing any new input and finishing the pending requests.
So you can safely shutdown one instance, which won't accept any more request, and another one could take the next requests. If you want 100% up time, you can put a front-end, e.g. nginx reverse proxy with load balancing to the two instances.

macfly · 2020-11-16 17:25:24

mdbs99 wrote:

I cannot guarantee that a service is always online, 100% of time, if I can shutdown someone that are using it because I supposed that no one was using.
Don't you agree?

But if server A is no longer receiving requests after a while, why would someone still be using it?

mdbs99 · 2020-11-16 17:40:13

ab wrote:

Whatever there are threads still working or not, is not relevant. I am afraid you still don't understand how the mORMot HTTP server thread pool works, and that "killing the threads" doesn't mean anything in our context. The threads are not running, they are sleeping until there is a request incoming.

Arnaud, I know that threads are not running if nobody is send request. Yes, they are sleeping, but I (as an admin of the service, not as a developer that has created) don't know if thread are sleeping or running, before I kill the process.

I cannot drop a user that is using the service. I just can't... and I'm afraid that you still don't understand me. Again, I cannot stop the service even if just one person is using it, because I cannot control if this client got the new URL, from the database, to use the instance-B!

There is already a feature in TSQLRestServer/THttpServer which allows a graceful shutdown of the server process.
If you just stop the mORMot daemon, it will shutdown properly, refusing any new input and finishing the pending requests.
So you can safely shutdown one instance, which won't accept any more request, and another one could take the next requests. If you want 100% up time, you can put a front-end, e.g. nginx reverse proxy with load balancing to the two instances.

Are you talking about mORMot 2.0?
If it already exists in 1.18 let us know, as everybody still using that version.

Another information is that I'm not using a demon (yet).

...refusing any new input and finishing the pending requests.

By "finishing" you meant stop to process in the middle, returning an error to the client? If I understood wrong, please try to explain again, otherwise I cannot do that.

PS: I cannot tell them what I would like to use (nginx, IIS, whatever). By now, I would like to use just mORMot service process.

mdbs99 · 2020-11-16 17:44:41

macfly wrote:

But if server A is no longer receiving requests after a while, why would someone still be using it?

"I cannot stop the service even if just one person is using it, because I cannot control if this client got the new URL, from the database, to use the instance-B."

mdbs99 · 2020-11-16 17:52:52

pvn0 wrote:

@ab is right as always

As I guy that worked directly with him, I always had this thought... but not this time

Just incase you need this or anyone is wondering :
THttpServer(aHTTPServer.HttpServer).ThreadPool.RunningThreads
THttpServer(aHTTPServer.HttpServer).ThreadPool.PendingContextCount
RunningThreads will tell you how many threads in the pool are active and PendingContextCount will tell you how many requests are queued for processing. (don't call PendingContextCount too often because it uses a lock).

THANK you, to give some light on this thread! Now I start to think that is possible to do what I NEED to do.
I don't know about PendingContextCount (I will read documentation) but as I'm answering you: so, I can return this value and if it is zero I can know, for sure, that no resquest still in a list to be process, right?

Either way if you shutdown your server and your client gets an error you should either A.) be sure all clients have been notified of server change prior to shutdown B.) handle the error on client side.

I cannot control the clients, which is other apps (website, other Java services, of course the Delphi client, and anyone that wants to use that service).

Last edited by mdbs99 (2020-11-16 18:51:39)

macfly · 2020-11-16 18:01:43

mdbs99 wrote:

macfly wrote:
But if server A is no longer receiving requests after a while, why would someone still be using it?
"I cannot stop the service even if just one person is using it, because I cannot control if this client got the new URL, from the database, to use the instance-B."

Okay, now I think I understand.
You return the URL at the beginning of the session and it will be the same until the client is finished.

In theory a client can stay for hours in the same session and you would not be able to make this replacement within a constant period.
You can end up with hours of delay if the client remains open.

mdbs99 · 2020-11-16 18:17:02

macfly wrote:

Okay, now I think I understand.
You return the URL at the beginning of the session and it will be the same until the client is finished.
In theory a client can stay for hours in the same session and you would not be able to make this replacement within a constant period.
You can end up with hours of delay if the client remains open.

Almost that
Clients should get from a database the URL for each (active) services.
So, you don't need a session (this service doesn't have one). But clients SHOULD refresh its own "list of URL services" in a *period of time*. If some client haven't updated its list and still trying to use the instance-A—AFTER this period of refreshing—I don't care if he/she got a 400 error because he/she should used the active service, which is instance-B.

But I cannot shutdown the service, even after this period, if the client already was using the service before — that could exists in other type of services, which I will implement as well, and that is why I would like to know how to shutdow the service without kill someone unintentialy. I would like to do this properly, clean, without workarounds.

ab · 2020-11-16 19:03:52

Check mORMot 1.18 source of the HTTP server, when it shutdowns.

I still don't understand why you cannot shutdown the service.

mpv · 2020-11-16 19:43:16

The thing you call "A/B service" is known as "Blue-Green Deployment" (google for it) and it usually implementing using some kind of load balance.
Announcing a new URL to client is tooooo complex.
Better to set up a load balance in front of your services, so you can start new instance, instruct load balance to proxy new requests to the new instance and when shut down old instance.
I use nginx for such task, but HAPxoxy is also good candidate. Even IIS may be can do such.

Last edited by mpv (2020-11-16 19:48:13)

pvn0 · 2020-11-16 19:55:24

mdbs99 wrote:

I don't know about PendingContextCount (I will read documentation) but as I'm answering you: so, I can return this value and if it is zero I can know, for sure, that no resquest still in a list to be process, right?

After double checking the code you should better use this instead.

 ThttpServer(aHttpServer.HttpServer).ServerConnectionActive

My bad

mdbs99 wrote:

So, you don't need a session (this service doesn't have one). But clients SHOULD refresh its own "list of URL services" in a *period of time*. If some client haven't updated its list and still trying to use the instance-A—AFTER this period of refreshing—I don't care if he/she got a 400 error because he/she should used the active service, which is instance-B.
But I cannot shutdown the service, even after this period, if the client already was using the service before — that could exists in other type of services, which I will implement as well, and that is why I would like to know how to shutdow the service without kill someone unintentialy. I would like to do this properly, clean, without workarounds.

You are writting very contradicting things, you say there is no session but your entire problem seems to be session related, or lack of.

@ab and @mpv are both right, you need load balancing and possibly revamp of your server code.

The biggest confusing thing for me is :

I cannot shutdown the service, even after this period, if the client already was using the service before

Right, you wrote the service implementation, so if a client is using instance-A when you swap to instance-B, YOU should know that already because you wrote the service implementation

macfly · 2020-11-16 20:04:43

I agree with pvn0.

I cannot shutdown the service, even after this period, if the client already was using the service before

If there is information that will only be on instance A and that cannot be continued on instance B, is this not a type of session?

mdbs99 · 2020-11-16 20:07:55

mpv wrote:

The thing you call "A/B service" is known as "Blue-Green Deployment" (google for it) and it usually implementing using some kind of load balance.
Announcing a new URL to client is tooooo complex.
Better to set up a load balance in front of your services, so you can start new instance, instruct load balance to proxy new requests to the new instance and when shut down old instance.
I use nginx for such task, but HAPxoxy is also good candidate. Even IIS may be can do such.

I can't choose what I want to use. They are paying me to do a job. That's it. Maybe I can use IIS, but I don't know yet.
I wrote "A/B service" (quoted), as it is just an idea and not the Blue-Green Deployment. Thanks, anyway.

pvn0 wrote:

You are writting very contradicting things, you say there is no session but your entire problem seems to be session related, or lack of.

That is because I'm thinking far. I have a job to do now, but new ones in a near future. I'm apologize if it's confusing for you.

Right, you wrote the service implementation, so if a client is using instance-A when you swap to instance-B, YOU should know that already because you wrote the service implementation

I've already said to you that I don't control clients. It's not about a Delphi client-server. Is much more than that... Java, services, website... anyway.

mdbs99 · 2020-11-16 20:12:44

macfly wrote:

I agree with pvn0.
I cannot shutdown the service, even after this period, if the client already was using the service before
If there is information that will only be on instance A and that cannot be continued on instance B, is this not a type of session?

Instead of trying to discovery if mORMot does what I want to implement, you (all) are trying to know "why" I need it...

Guys, maybe the archtecture there is wrong; maybe storing the URL's of services in a database is wrong... but this is already exists and I need to work with.

pvn0 · 2020-11-16 20:49:59

mdbs99 wrote:

I've already said to you that I don't control clients. It's not about a Delphi client-server. Is much more than that... Java, services, website... anyway.

I know that you don't control clients, I meant that you wrote the service implementation on the server side. Sorry if I didn't make that clear.

mdbs99 · 2020-11-16 21:38:42

pvn0 wrote:

I know that you don't control clients, I meant that you wrote the service implementation on the server side. Sorry if I didn't make that clear.

Sorry if I didn't understand you too.

mORMot Open Source

#1 2020-11-16 11:58:30

How configure the server to finish inactive threads?

#2 2020-11-16 13:39:03

Re: How configure the server to finish inactive threads?

#3 2020-11-16 14:15:51

Re: How configure the server to finish inactive threads?

#4 2020-11-16 15:01:19

Re: How configure the server to finish inactive threads?

#5 2020-11-16 15:40:19

Re: How configure the server to finish inactive threads?

#6 2020-11-16 15:57:42

Re: How configure the server to finish inactive threads?

#7 2020-11-16 16:17:09

Re: How configure the server to finish inactive threads?

#8 2020-11-16 16:29:04

Re: How configure the server to finish inactive threads?

#9 2020-11-16 17:11:35

Re: How configure the server to finish inactive threads?

#10 2020-11-16 17:25:24

Re: How configure the server to finish inactive threads?

#11 2020-11-16 17:40:13

Re: How configure the server to finish inactive threads?

#12 2020-11-16 17:44:41

Re: How configure the server to finish inactive threads?

#13 2020-11-16 17:52:52

Re: How configure the server to finish inactive threads?

#14 2020-11-16 18:01:43

Re: How configure the server to finish inactive threads?

#15 2020-11-16 18:17:02

Re: How configure the server to finish inactive threads?

#16 2020-11-16 19:03:52

Re: How configure the server to finish inactive threads?

#17 2020-11-16 19:43:16

Re: How configure the server to finish inactive threads?

#18 2020-11-16 19:55:24

Re: How configure the server to finish inactive threads?

#19 2020-11-16 20:04:43

Re: How configure the server to finish inactive threads?

#20 2020-11-16 20:07:55

Re: How configure the server to finish inactive threads?

#21 2020-11-16 20:12:44

Re: How configure the server to finish inactive threads?

#22 2020-11-16 20:49:59

Re: How configure the server to finish inactive threads?

#23 2020-11-16 21:38:42

Re: How configure the server to finish inactive threads?

Board footer