mORMot Open Source

But still doesn't solve the queue length problem. Under a controlled stress test, the server takes 50ms to finish each request. I've set clone thread size to 256. A sudden burst of ~4000 incoming connection still return status 503 error half of the time.

Thx, but I assume the thread pool is for THttpServer but not THttpAPIServer? I've just tried THttpServer and deliberately block OnRequest(), then make 1000 http connections to the server, and the thread count of the server is well over 1000.

The main problem for me is to handle a sudden burst of incoming connection, say > 20000 at the same time. There is no need to process all the request at the same time. Client can wait but should not return any error. One connection per thread is already out of the question. Http API can handle it since it has build-in kernel mode request queue. The queue length can be easily setup in IIS, but it seems there is no easy way to change the value using Http API 1.0 (THttpAPIServer).

Also, afaik, THttpAPIServer can only handle one single OnRequest() at a time. Http.ReceiveHttpRequest() is only called once in the main loop. If OnRequest() is blocked then other incoming connections have to be waited in the request queue until another Http.ReceiveHttpRequest() is called. If the request queue is full then the next client will get an instant 503 error. My idea is to call Http.ReceiveHttpRequest() multiple times and split those requests into different threads.