determine whether all Tasks in TSynThreadPool have been executed

zen010101 · 2024-10-14 04:07:43

I know that I can inherit TSynThreadPool and add a TaskCount property in the subclass, increase the Property by 1 after Push, decrement it by 1 when the Task completes, and use TLightLock to make it thread safe.

But I want to know if there is an easier way? I tried the RunningThreads attribute, but it didn’t work. It is always consistent with the parameters of Create(NumberOfThreads) and cannot be used to confirm the number of Tasks being executed.

Here is my code: https://gist.github.com/zen010101/def68 … 0b028d70d2

zen010101 · 2024-10-14 04:52:08

When WINIOCP is not used, I can determine whether all Tasks have been executed by judging whether PendingContextCount is 0.

ab · 2024-10-14 06:56:16

Please try
https://github.com/synopse/mORMot2/commit/e46982c6

Now you should have access to a TSynThreadPool.PendingContextCount property even with IOCP.

Note: InterlockedIncrement/Decrement() are better than a lock if you just need atomic access to an integer.

zen010101 · 2024-10-14 09:23:59

Confirm that the PendingContextCount property can also be used in IOCP mode as the condition for determining the completion of all Task executions.

zen010101 · 2024-10-14 09:47:41

New bug:

When TSynThreadPool is destroyed in the middle of executing the tasks, the TaskAbort is not called, but in Linux, all works well.

see this, compile it for windows and linux to see the differences: https://gist.github.com/zen010101/def68 … 0b028d70d2

The possible issue might be here:

destructor TSynThreadPool.Destroy;
var
  i: PtrInt;
  endtix: Int64;
begin
  fTerminated := true; // fWorkThread[].Execute will check this flag
  try                      
    {$ifdef USE_WINIOCP}
    // notify the threads we are shutting down
    for i := 0 to fWorkThreadCount - 1 do
      IocpPostQueuedStatus(fRequestQueue, 0, nil, {ctxt=}nil);
      // TaskAbort() is done in Execute when fTerminated = true
    {$else}

ab · 2024-10-14 12:51:37

The behavior did not change. Previous code did the same.

So it is not a regression, it is a long-standing bug.

Edit:
Should be fixed with
https://github.com/synopse/mORMot2/commit/3db4b9e4

zen010101 · 2024-10-14 14:23:16

Confirm fixed.

Last edited by zen010101 (2024-10-14 14:23:36)

zen010101 · 2024-10-14 19:17:42

New bug:

When creating a thread pool and passing a larger thread value in IOCP mode, TSynThreadPool.Free will enter an infinite loop and block the process.

Here is the demo: https://gist.github.com/zen010101/def68 … 0b028d70d2

ab · 2024-10-14 20:07:11

In fact, there is no infinite loop but a 30 seconds timeout in TSynThreadPool.Destroy.

Please try with
https://github.com/synopse/mORMot2/commit/d8b5ce76

zen010101 · 2024-10-15 04:12:43

Confirmed fixed. :-)

mORMot Open Source

#1 2024-10-14 04:07:43

determine whether all Tasks in TSynThreadPool have been executed

#2 2024-10-14 04:52:08

Re: determine whether all Tasks in TSynThreadPool have been executed

#3 2024-10-14 06:56:16

Re: determine whether all Tasks in TSynThreadPool have been executed

#4 2024-10-14 09:23:59

Re: determine whether all Tasks in TSynThreadPool have been executed

#5 2024-10-14 09:47:41

Re: determine whether all Tasks in TSynThreadPool have been executed

#6 2024-10-14 12:51:37

Re: determine whether all Tasks in TSynThreadPool have been executed

#7 2024-10-14 14:23:16

Re: determine whether all Tasks in TSynThreadPool have been executed

#8 2024-10-14 19:17:42

Re: determine whether all Tasks in TSynThreadPool have been executed

#9 2024-10-14 20:07:11

Re: determine whether all Tasks in TSynThreadPool have been executed

#10 2024-10-15 04:12:43

Re: determine whether all Tasks in TSynThreadPool have been executed

Board footer