Threads pool for websocket service

dcoun · 2022-06-04 08:59:10

When we have a websocket service that needs to use a thread to finish long time work, we can use the TRestThread to create a thread that will do the job as shown in the example restws_longworkserver.
In a production environment, I face two problems:

a) Many many websocket requests that create a TRestThread will slow down the server. A solution is to have a pool of requests to be completed by a limited number of threads and return the output. TsynDictionary is probably a good solution for that. I do not know if you can propose something else.

b) Creating a thread is time consuming, and to have a pool of threads to be used on request is probably a good solution. What do you propose for that?
- Delphi has a working library for that and a couple of other libraries to help with that. What about FPC? I tried for example Quicklib and it does not work with FPC for threads.
- When does it worth to have a thread pool mechanism to deal with time consuming works in a REST server?
- Does mormot2 has already such a threadpool class that could be used?
- Do you already use a multi-environment FPC library for that and which?

Thank you in advance

ab · 2022-06-04 15:32:35

Use a queue for these long works.
You have for instance TSynBackgroundTimer.Add which is part of each TRestServer.

See also TSynThreadPool but it may be a little more complex.

They are implemented in mormot.core.threads.pas

dcoun · 2022-06-04 16:00:04

I am already using TSynBackgroundTimer.Add and it is working excellent for having a serial list of tasks that they are completed one-by-one

TSynThreadPool is really interesting and can do what I am thinking about parallel processing using a predefined number of threads
Push is to add a procedure to be executed by a thread as I understand
Is there an example that I can play with?
Thank you in advance

dcoun · 2022-06-04 17:22:52

Do you propose something more in the following example?

https://gist.github.com/dkounal/993c8d5 … fe3c4f81bb

In linux I should create it like the following in order to have a queue for more that the running threads request in Push?

ttr:=TmySynThreadPool.Create(4,true);

Also, I noticed in windows, that if destroyed before finishing all iocp queue, the objects in the iocp queue are not freed. How to overcome this?

Last edited by dcoun (2022-06-04 17:33:36)

ab · 2022-06-05 06:19:43

Your sample seems fine at first glace.

I don't understand your concern about Linux. Could you rephrase it?

For IOCP there is no easy way to access the remaining items.
Is it a problem in your case?
Edit: I tried to release all pending tasks at shutdown for Windows/IOCP thread pool.
Please try https://github.com/synopse/mORMot2/commit/9388a43b

dcoun · 2022-06-05 07:57:10

The above commit did not help. The problem is the following in windows:
if I destroy the TSynThreadPool, all items in IOCP are not destroyed. The code in the commit should be in the destroy procedure, or not? Is it needed in the Execute procedure too?

In linux, we do not have the queue of IOCP if more requests are pushed than the available threads. There is a fPendingContext array with a maximum value of 10000 as length and we can use it, if it is created with

ttr:=TmySynThreadPool.Create(4,true);

Am I right?

Also an other question for a perfect use of TSynThreadPool:
I need to initiate a couple of object for use by each thread.
Is it possible to have a derived class in TSynThreadPoolWorkThread to override its create/destroy procedures?
Or, should I use an external array coupled with TSynThreadPool.WorkThread for each TSynThreadPoolWorkThread object's data?

Last edited by dcoun (2022-06-05 08:04:04)

ab · 2022-06-05 12:37:13

The above commit releases the instances in the threads context, because it is where the IOCP queue is read.

The internal queue length of 10,000 is indeed returned by .... TSynThreadPool.QueueLength ...

Create/Destroy are NOT run in the context of each threads.
You have the OnThreadStart/OnThreadTerminate event handlers for this process.

dcoun · 2022-06-05 12:43:13

Too much in one message and I donot help....
For the Linux and the TSynThreadPool.QueueLength everything is ok.

run the example with fastmm4 and report leaks enabled in windows with delphi, press button1, many times button4 and then button2. When everything is finished, close the app. it will report all objects in the IOCP as leaked even with the above commit

I will check OnThreadStart/OnThreadTerminate. Thank you a lot @ab

Last edited by dcoun (2022-06-05 12:50:21)

mORMot Open Source

#1 2022-06-04 08:59:10

Threads pool for websocket service

#2 2022-06-04 15:32:35

Re: Threads pool for websocket service

#3 2022-06-04 16:00:04

Re: Threads pool for websocket service

#4 2022-06-04 17:22:52

Re: Threads pool for websocket service

#5 2022-06-05 06:19:43

Re: Threads pool for websocket service

#6 2022-06-05 07:57:10

Re: Threads pool for websocket service

#7 2022-06-05 12:37:13

Re: Threads pool for websocket service

#8 2022-06-05 12:43:13

Re: Threads pool for websocket service

Board footer