Changing the http.sys queue length from default 1000

akirabbq · 2013-05-13 06:28:33

Hi, is it possible to change the queue length? It seems that if the api server's OnRequest() took too long to process, and the incoming queue got filled up to 1000, the next incoming connection will get a 503 error. Under IIS there is an option to change the queue length ("kernel request limit" in IIS6) for a particular application pool (default is 1000), but is there any option to change it under http api server? Thx.

ab · 2013-05-13 07:28:27

This option sounds to be available in HTTP API 2.0
http://msdn.microsoft.com/en-us/library … s.85).aspx

And perhaps HTTP API 1.0
http://msdn.microsoft.com/en-us/library … s.85).aspx
and
http://msdn.microsoft.com/en-us/library … s.85).aspx

But both are time out values, in time, not queue length.

I'm not sure those are the same than your IIS parameter.

Note that mORMot server uses a thread pool, so if one request takes long, the next is taken in account by another thread.
You can enhance the number of threads, if you are afraid of a 503 error.

But I did not see such issues in the wild, even with the current settings.

akirabbq · 2013-05-13 09:28:10

Thx, but I assume the thread pool is for THttpServer but not THttpAPIServer? I've just tried THttpServer and deliberately block OnRequest(), then make 1000 http connections to the server, and the thread count of the server is well over 1000.

The main problem for me is to handle a sudden burst of incoming connection, say > 20000 at the same time. There is no need to process all the request at the same time. Client can wait but should not return any error. One connection per thread is already out of the question. Http API can handle it since it has build-in kernel mode request queue. The queue length can be easily setup in IIS, but it seems there is no easy way to change the value using Http API 1.0 (THttpAPIServer).

Also, afaik, THttpAPIServer can only handle one single OnRequest() at a time. Http.ReceiveHttpRequest() is only called once in the main loop. If OnRequest() is blocked then other incoming connections have to be waited in the request queue until another Http.ReceiveHttpRequest() is called. If the request queue is full then the next client will get an instant 503 error. My idea is to call Http.ReceiveHttpRequest() multiple times and split those requests into different threads.

mpv · 2013-05-13 20:17:07

but I assume the thread pool is for THttpServer but not THttpAPIServer

Please, pay attention to THttpApiServer.Clone method. This one creates the thread pool. Each cloned thread registers itself to http.sys and then http.sys manages it in the similar way as HttpServer does. As far as I understand you overflow not length of queue, but time request wait in queue. So AB is right - increase thread pool size.

akirabbq · 2013-05-14 01:56:30

mpv wrote:

but I assume the thread pool is for THttpServer but not THttpAPIServer
Please, pay attention to THttpApiServer.Clone method. This one creates the thread pool. Each cloned thread registers itself to http.sys and then http.sys manages it in the similar way as HttpServer does. As far as I understand you overflow not length of queue, but time request wait in queue. So AB is right - increase thread pool size.

oops.... I've missed the .Clone() function (though it was for something else...). Thx a lot~

akirabbq · 2013-05-14 04:51:31

But still doesn't solve the queue length problem. Under a controlled stress test, the server takes 50ms to finish each request. I've set clone thread size to 256. A sudden burst of ~4000 incoming connection still return status 503 error half of the time.

mpv · 2013-05-14 17:46:35

What tool do you use to create such load? What OS is your server? May be you reached max number of socket? See here for example http://www.catalyst.com/kb/100072

mpv · 2014-03-05 18:00:21

Ups! We found queue problem during try to put into production system with 20 000 users.
It happens in the morning while all 20 000 users try to log-in to server in the nearly same time.

Also we found a way to reproduce it in synthetic test.
Under UNIX exist tool wrk. To reproduce problem we need to increase socket limit first.

# ulimit -n 10000
# ./wrk -c 10000 -t 10 -d 10s --latency http://mormotHTTPAPIserver/timeStamp

In the HTTP.sys error log we can see (normally here C:\Windows\System32\LogFiles\HTTPERR\ ) QueueFull error messages

To solve the problem we add HTTPAPI2.0 support to SynCrtSock.pas (via {$ifdef HTTPAPI20}) and add THttpApiServer.HTTPQueueLength property to change Queue length from default 1000 to any value.
Also we work on adding other HTTPAPI2.0 features (mostly QoS we need).
AB - I send to you version with HTTPQueueLength.
Note - HTTPAPI2.0 requirements:
- Minimum supported client Windows Vista
- Minimum supported server Windows Server 2008

ab · 2014-03-06 17:11:36

Thanks a lot!

I will include it ASAP.
:-)

ab · 2014-03-07 10:48:06

That's it.

I've included your great proposal.
See http://synopse.info/fossil/info/9ddf00210f

Thanks a lot for your feedback and code patch!

We all hope that the situation in your country will stabilize and be as peaceful as possible.

ab · 2014-03-07 15:47:27

http.sys kernel-mode server handles HTTP API 2.0 (available since Windows Vista / Server 2008), or fall back to HTTP API 1.0 (for Windows XP or Server 2003).

Thanks to this fall-back mechanism, there is no need of any conditional definition now: the class will use the highest API available.

We also included several fixes after further testing of the initial implementation (e.g. when serving multi TSQLRestServerDB instance).

See http://synopse.info/fossil/info/84d0c88f6c

array81 · 2014-03-08 15:26:53

I don't understand.
If I update my framework version with this last version, my application will be problem to run on Windows XP by default?
I need to run my application on Windows XP, Vista, 7 and 8, can I update the framework? Do I need change my code?
Thanks

ab · 2014-03-08 22:04:44

Nothing to change.

The correct API will be recognized automatically.
So any existing code will work for XP or VISTA and up, with no modification.

Only the THttpApiServer.ServerQueueLength property will work only for Vista/2003 and up.
You can use this property as such:

if aSQLHttpServer.HttpServer.InheritsFrom(THttpApiServer) then
  THttpApiServer(aSQLHttpServer.HttpServer).HTTPQueueLength := 5000;

mpv · 2014-03-13 08:29:41

AB, in http://synopse.info/fossil/info/1eda390096 SynCrtSock commit there is issue (from my POW) in THttpApiServer.SetHTTPQueueLength.
We have one fReqQueue for all cloned thread, so we do not need to call SetHTTPQueueLength for all cloned thread. So, last circle is overhead....
See ticket http://synopse.info/fossil/tktview?name=51a9c086f3

Last edited by mpv (2014-03-13 08:30:27)

ab · 2014-03-13 13:27:53

Indeed!
Thanks for the input.

See http://synopse.info/fossil/info/8937d37696

mpv · 2014-03-15 11:00:26

AB, from my production statistic I have near 1 million server request during workday. To each response we add 45 bytes ("X-Powered-By: SynCrtSock http://synopse.info") header . This create additional 45MBytes/day traffic . From protocol POW we can use "Server:" header instead. http://www.w3.org/Protocols/rfc2616/rfc … l#sec14.38. I add

const
  respServerHeader: PAnsiChar = 'mORMot';
....

procedure THttpApiServer.Execute;
......
// send response
Resp^.Version := Req^.Version;
with Resp^.Headers.KnownHeaders[respServer] do begin
   pRawValue := respServerHeader;
  RawValueLength := length(respServerHeader);
end;

and remove adding of X-Powered-By in HTTP_RESPONSE.SetHeaders
After this I got headers in response:

Server: mORMot Microsoft-HTTPAPI/2.0

(I can't found how to disable adding of 'Microsoft-HTTPAPI/2.0' - HTTP.sys add it automatically)
instead of

Server: Microsoft-HTTPAPI/2.0
X-Powered-By: SynCrtSock http://synopse.info

minus 40 bytes

If You decide to apply this patch it will be VERY good to give other developer ability to redefine respServerHeader. May be by add property ServerSoftwareName to THttpApiServer?

I create feature request http://synopse.info/fossil/tktview?name=f4a68b1431

Last edited by mpv (2014-03-15 11:23:08)

ab · 2014-03-15 14:49:45

Please see my comments in the ticket.

mpv · 2014-03-15 15:02:33

Yes, I understand your POW. You are right.

ab · 2014-03-15 15:26:19

You are welcome!

Thanks for taking the time to close all fixed tickets.
This is nice seeing that mORMot's quality is always increasing, and able to serve million requests per days, with high number of concurrent connections!
Mainly thanks to feedback like yours!

We also hope that peace will stay in your country.

mpv · 2014-03-18 11:52:05

AB, I patch constructor THttpApiServer.Create a little bit - now it allow Queue monitoring via Perfmon "HTTP Services Requests Queue" counter. In prev. version we create queue without name, and such queue not visible in counters. Please, modify:

constructor THttpApiServer.Create(CreateSuspended: Boolean);
var bindInfo: HTTP_BINDING_INFO;
    serverSessionStr: WideString;
....
    serverSessionStr := UIntToStr(fServerSessionID);
    EHttpApiServer.RaiseOnError(hCreateRequestQueue,Http.CreateRequestQueue(
      Http.Version,pointer(serverSessionStr),nil,0,fReqQueue));

P.S. Already forgot how to work with fossill

ab · 2014-03-19 16:01:22

Good idea.

Thanks for the feedback.

See http://synopse.info/fossil/info/60d96b225c

mpv · 2014-03-20 07:18:53

AB, ServerSessionID actually is Int64. In my implementation I use UIntToStr(ServerSessionID) since in XE2 I have overload UIntToStr(Value: UInt64). In your implementation we lost some bytes (direct convertion to Cardinal) and as result it is possible to have collision in QueueName. HTTPAPI don't allow to create two queue with the same name - so it is possible situation I can't create two instance of THttpApiServer. May be add StrInt64 from SynCommons to SynCtrlSock and use it?

Last edited by mpv (2014-03-20 07:21:03)

ab · 2014-03-20 14:44:39

You are right.

Identifier to the HTTP API 2 session is now encoded as hexadecimal text (to avoid any collision error).
See http://synopse.info/fossil/info/05e0031069

mpv · 2014-03-29 14:11:50

We add some QoS ability to HTTP API 2 based server:

MaxConnections - The maximum number of HTTP connections allowed
MaxBandwidth - The maximum allowed bandwidth rate in bytes per second

MaxConnections is tested and work perfect. MaxBandwidth is actually change HTTP API value, but download speed remain the same May be this is my OS limitation (Windows 8 Home)..
AB, I send you patched sources.

ab · 2014-03-29 15:59:10

We have added THttpApiServer.MaxBandwidth and THttpApiServer.MaxConnections properties (for HTTP API 2.0 only)...
See http://synopse.info/fossil/info/4e61734d33

Thanks mpv for the proposal and code patch!

cheemeng · 2014-04-07 17:48:40

Arnaud/mpv, would it be possible to publish fServerSessionID and fUrlGroupID as public read-only properties? I have patched my code, but would be great if it is in the official source to make it easier for me to update sources.

The reason the properties are useful is that I need to query the performance stats within the programme itself... or did I miss something?

Thanks!

ab · 2014-04-07 18:07:57

Good idea.

I've added THttpApiServer.ServerSessionID and UrlGroupID read-only properties.
See http://synopse.info/fossil/info/1101f033bb

Thanks for the feedback.
If you have any code to propose, feel free!

cheemeng · 2014-04-07 18:43:00

A quick one... ULONGLONG should be UInt64 and not Int64?

cheemeng · 2014-04-07 18:44:38

And thanks for the prompt response Arnaud!

ab · 2014-04-07 20:01:12

cheemeng wrote:

A quick one... ULONGLONG should be UInt64 and not Int64?

AFAIK yes, it should.

"U" stands for "U"nsigned... But it won't matter much, in all cases (on older versions of Delphi, there is no QWord/UInt64, just plain signed Int64, and it works!)...

cheemeng · 2014-04-08 04:48:29

Line 844 of SynCrtSock.pas

  ULONGLONG = Int64;

Should it be

  ULONGLONG = UInt64;

instead?

Update: Oops sorry, did not notice your reply. Yea it works if we convert it to Hex. It only fails to work if we print out the value as integer.

Last edited by cheemeng (2014-04-08 04:50:13)

mORMot Open Source

#1 2013-05-13 06:28:33

Changing the http.sys queue length from default 1000

#2 2013-05-13 07:28:27

Re: Changing the http.sys queue length from default 1000

#3 2013-05-13 09:28:10

Re: Changing the http.sys queue length from default 1000

#4 2013-05-13 20:17:07

Re: Changing the http.sys queue length from default 1000

#5 2013-05-14 01:56:30

Re: Changing the http.sys queue length from default 1000

#6 2013-05-14 04:51:31

Re: Changing the http.sys queue length from default 1000

#7 2013-05-14 17:46:35

Re: Changing the http.sys queue length from default 1000

#8 2014-03-05 18:00:21

Re: Changing the http.sys queue length from default 1000

#9 2014-03-06 17:11:36

Re: Changing the http.sys queue length from default 1000

#10 2014-03-07 10:48:06

Re: Changing the http.sys queue length from default 1000

#11 2014-03-07 15:47:27

Re: Changing the http.sys queue length from default 1000

#12 2014-03-08 15:26:53

Re: Changing the http.sys queue length from default 1000

#13 2014-03-08 22:04:44

Re: Changing the http.sys queue length from default 1000

#14 2014-03-13 08:29:41

Re: Changing the http.sys queue length from default 1000

#15 2014-03-13 13:27:53

Re: Changing the http.sys queue length from default 1000

#16 2014-03-15 11:00:26

Re: Changing the http.sys queue length from default 1000

#17 2014-03-15 14:49:45

Re: Changing the http.sys queue length from default 1000

#18 2014-03-15 15:02:33

Re: Changing the http.sys queue length from default 1000

#19 2014-03-15 15:26:19

Re: Changing the http.sys queue length from default 1000

#20 2014-03-18 11:52:05

Re: Changing the http.sys queue length from default 1000

#21 2014-03-19 16:01:22

Re: Changing the http.sys queue length from default 1000

#22 2014-03-20 07:18:53

Re: Changing the http.sys queue length from default 1000

#23 2014-03-20 14:44:39

Re: Changing the http.sys queue length from default 1000

#24 2014-03-29 14:11:50

Re: Changing the http.sys queue length from default 1000

#25 2014-03-29 15:59:10

Re: Changing the http.sys queue length from default 1000

#26 2014-04-07 17:48:40

Re: Changing the http.sys queue length from default 1000

#27 2014-04-07 18:07:57

Re: Changing the http.sys queue length from default 1000

#28 2014-04-07 18:43:00

Re: Changing the http.sys queue length from default 1000

#29 2014-04-07 18:44:38

Re: Changing the http.sys queue length from default 1000

#30 2014-04-07 20:01:12

Re: Changing the http.sys queue length from default 1000

#31 2014-04-08 04:48:29

Re: Changing the http.sys queue length from default 1000

Board footer