THttpServer serial processing?

darianmiller · 2011-02-12 02:12:53

I saw your post on StackOverflow about your Delphi socket server component. I downloaded a copy just now and took a quick look. At first it seems to be using IOCP and is scalable with a thread pool, but the implementation has a large bottleneck.

I started a new standard VCL app to verify and subclassed THttpServer:
TmyIOCP = Class(THttpServer)
public
function Request(var ClientSock: THttpServerSocket; var Data: TSockData): cardinal; override;
end;

in the form create:
fIOCPServer := TmyIOCP.Create('80');

custom client routine:

function TmyIOCP.Request(var ClientSock: THttpServerSocket; var Data: TSockData): cardinal;
begin
result := 200;
Data := '<html><head><title>test</title></head><body>Hello ' + FormatDateTime('hh:mm:ss:zz', Now) + '</body></html>';
ClientSock.ContentType := 'text/html';
Sleep(1000 * 30); //30sec pause to process this single client request... simulate database lookup, server side delay, etc.
end;

That's it.. start the app and then launch your webbrowser. Navigate to http://localhost in 3 separate tabs (or 3 separate browsers)

This was my response

Hello 19:57:39:366
Hello 19:58:09:366
Hello 19:58:39:369

So the requests were all serialized...one processed at a time. This is obviously due to
procedure THttpServer.Process
..
// calc answer (unique thread)
try
EnterCriticalSection(ProcessCS); // calculation thread protection
Code := Request(ClientSock,Data);
finally
LeaveCriticalSection(ProcessCS);
end;

Why have all this framework and power and then limit yourself to processing one request at a time?

Darian

ab · 2011-02-12 12:20:13

This was done on purpose.

In fact, what is slow in such a server is not the database handling, but the receiving/sending process to the remote clients, using a limited bandwidth.

The core database framework is able to handle 25,000 requests of 4 KB of JSON data per second (see the test benchmarks in direct access mode) - that is more than 1 Gb/s of row data transmission - client and server on the same core. For the server part, timing is about 10 Gb/s (what takes time is the JSON parsing from the client side). So 10 Gb/s is bandwidth produced by the Request method you quote above, on a single core.

The bottleneck (i.e/ what must be parallelized) is the Client/Server transmission part. The threadpool and I/O Completion ports will speed up this last part.

Using this architecture, one CPU Core will produce more than 10 Gb/s of data, then the other CPU/threads will handle the transmission from/to several clients.

Processing one request at a time makes the DB part of the framework (especially SQLite3) more robust. And still very fast. A pure parallelized architecture wouldn't make it faster, because it will be stopped by the network connections (or perhaps you have a computer with 10 Ethernet 1 GB adapter cards).

darianmiller · 2011-02-12 15:46:33

If you have a bunch of updates to a DB, the sending/receiving from the clients can be quite small in comparison to the server side work.

I didn't think of this last night but if your target DB server is SQLite, which serializes updates anyway, then the architecture makes a lot more sense to me. But, if you are trying to use this server architecture for much else, then I don't think it does. A 'real-life' example would be the typical shopping cart checkout process. Part of the process is to contact UPS to get shipping rates for the order...no one else can add anything to their shopping cart, and no other static page (or even image) is even served, until that one customer gets their rate quote from UPS. It would lead to quite a jittery system. Typically, the vast majority of all server tasks are reads which should be parallized.

Regardless, you have a lot of very nice code and I applaud your efforts to help the Delphi community with your projects. I have my IOCP implementation so it was more of a curiousity item for me.

Thanks,

Darian

ab · 2011-02-12 17:17:27

You are perfectly right.

It was targeting SQLite, and we didn't have to scale a lot with our current clients.

I think the architecture is still modular, and ready for much parallelism.

Our SynScaleMM unit was tuned in this direction.

Thanks in all cases for your feedback and encouragement!

mORMot Open Source

#1 2011-02-12 02:12:53

THttpServer serial processing?

#2 2011-02-12 12:20:13

Re: THttpServer serial processing?

#3 2011-02-12 15:46:33

Re: THttpServer serial processing?

#4 2011-02-12 17:17:27

Re: THttpServer serial processing?

Board footer