mORMot Open Source

At the moment I'm experimenting with a "repackaged" Process method, the strings moved to records:

  TSynHttpServerRequest = record
      InURL, InMethod, InHeaders, InContent, InContentType: RawByteString;
      ConnectionID : Int64;
  end;

  TSynHttpServerResponse = record
     OutContent, OutContentType, OutCustomHeader: RawByteString;
  end;

  TOnHttpServerRequest = function(
      const inRequest : TSynHttpServerRequest;
      var outResponse : TSynHttpServerResponse ) : cardinal of object;

The call overhead is non-measurably lower, but having records will open up adding methods for common stuff (f.i. add standard expiration/last-modified headers, etc.)

Also, currently the compression is set for the whole server, but a lot of data is already compressed (such as png or jpg images), so maybe it could be a boolean flag in the response instead?
Other data could benefit from the deflate being cached on the server-side (thus returning pre-compressed content, rather than re-compressing each time).

You can use Negotiate as well (for Kerberos), but that requires setting up SPN & other site prefixes, which complicates testing the code.

SSPI supports both, I've tried boths, and have the same issue of connection randomly failing in the same conditions, ie. they (AFAICT) never fail for local connections or when a breakpoint is set in the IDE, but have about 50% chance of succeeding when connecting from other machines on the LAN and there is no debugger breakpoints set.

This is so very weird (especially the debugger breakpoint thing), I'm wondering if the issue isn't with the domain controllers here: if one of the controllers were misconfigured in some fashion, it would explain the 50% success rate. Though it wouldn't explain the breakpoints thing or why we don't see the issue from Windows... (unless the OS were to silently fallback).

I guess if someone else tests and can't reproduce my issues, then that'll definitely point the finger at the domain here.

Unless I'm mistaken this is an URI-based scheme? ie. the user info needs to be entered or saved on the client-side in some fashion.

WWW-Authenticate allow that to be resolved by the browser from OS user session info, besides single-sign-on convenience, it can also be made a bit more secure in a LAN since OS logons can be more richly administered, can be tied to biometric systems, etc. all things which would be problematic (or time consuming) to reproduce in a web browser.
Besides, it also means there is no need to have user administration in the web app.

I've added that to the SynopseWebServer demo in DWS

uses dwsUtils;

type

   TNTLMAuth = class
      SecContext : TSecContext;
      User : RawUTF8;
   end;

var
   vPendingAuth : TFastCompareStringList;
   vPendingAuthCS : TFixedCriticalSection;

// and in Process method

   vPendingAuthCS.Enter;
   try
      p:=vPendingAuth.IndexOf(IntToHex(connectionId, 16));
      if p>=0 then begin
         auth:=TNTLMAuth(vPendingAuth.Objects[p]);
      end else begin
         auth:=TNTLMAuth.Create;
         InvalidateSecContext(auth.SecContext);
         vPendingAuth.AddObject(IntToHex(connectionId, 16), auth);
      end;
   finally
      vPendingAuthCS.Leave;
   end;

   if auth.User='' then begin

      auth64:=request.Header('Authorization');
      if auth64='' then begin
         OutCustomHeader:='WWW-Authenticate: NTLM'#13#10;
         OutContentType:=HTML_CONTENT_TYPE;
         OutContent:='<HTML><BODY>401 Unauthorized.</BODY></HTML>';
         Result:=401;
         Exit;
      end else if ServerSSPIAuth(auth.SecContext, RawDecode64(RawByteString(StrAfterChar(auth64, ' '))), OutContent) then begin
         OutCustomHeader:='WWW-Authenticate: NTLM '+BinToBase64(OutContent)+#13#10;
         Result:=401;
         Exit;
      end;

      ServerSSPIAuthUser(auth.SecContext, auth.User);

   end;

   OutContent:=auth.User;

To the Process method I just added a connectionId parameter, which takes the Req^.ConnectionId for the http.sys server, and a Int64(Socket) for the thread/socket-based one.

RawDecode64 is just an alternate base64 decoder I tried, but results are the same as with the Synopse one.

The missing elements are the ConnectionId field of the HTTP_REQUEST structure, and to be notified of a disconnect, one needs to call HttpWaitForDisconnect.

Also would it be possible in a future version of the SynCrtSock to use a record to pass the In/Out fields?

This would allow attaching extra fields (like ConnectionId or the pre-parsed headers), as well as simplify the call (only one pointer on the stack rather than the 8 ones currently required)

At the moment it seems the only way to access ConnectionID is to reimplement THttpApiServer.Execute in a subclass, but of course, that means any future fixes/improvements to THttpApiServer wouldn't be carried over.

Got some of it working, however the Process event in THttpApiServer doesn't have connection information.

How do you know when a connection is lost or if a connection has already been authenticated?

AFAICT in mORMot the above is based on an ID and expiring cache, does it mean there is no way to track the connection at the HTTP level?

If so, does it mean NTLM should only be used to establish a connection cookie or something like that?

Indeed!

IME (wrapping classes for use at work), a lot of the time spent in exposing classes and function to scripts actually goes to the securing more than the wrapping, to ensure there are no leaks, that erroneous parameters don't result in AVs, etc.

Also basic things WebResponse/WebRequest in the demo currently use RTTI, but this is essentially a shortcut for the demo, as RTTI can quickly end up exposing things that are not safe at all.

I haven't really investigated what is available in mORMot so far, but I assume a lot of the work is already done!

Extending the demo to integrate as a service instead, or to have managed sessions, Windows-based authentications, etc. would be nice.

That limitation is now gone from the demo (added basic support for FindFirstChangeNotification to automatically flush the cache)

Added SynopseWebServer demo to DWScript SVN.
Still pretty basic and missing tons of stuff, but works quite well with good perf already
I'll try to jazz it up in future versions to something usable as a "reference" for small production servers (ie. add some security, file I/O from DWS, stuff like that), so if you find mistakes & misuses, let me know.

Btw, is there any boiler plate comment header you want me to add in the headers of the units mentioning Synopse? a link to http://synopse.info or somewhere else? Or should it refer to mORMot instead?

Yes, but that's not enough for instance you could have:

  var insert : TWhatever;
  ...
  request := insert.GetTheSQL();

or

   insert : sql;
   ...
   request := insert;

If the language extension registers "insert" as a new keyword, the above would fail.

And while select isn't too common in Pascal (IME), insert, create, update & delete are quite common.

Had look at HttpApiServer demo, now I've a couple questions to ask:

1) If I understood correctly, the Request method is invoked from multiple threads at the same time, and those threads are from ServerThreadPoolCount ones? What does happen when the thread pool is exhausted? are new threads created on the fly or is there a backlog?

2) the Request methods takes raw http headers, however at the THttpSocket level, headers seem to have been already parsed. If there a standard way to get those pre-parsed headers instead? (to avoid re-parsing them)

3) Are there some url-encoded encoding/decoding routines? (not too complex, but might as well not replicate the effort) oops, saw them

4) Is there already multi-part/content-encoding parser for the request? (thinking of posted forms, might as well not replicate the effort for that)

Also, but that's beyond the scope of the demo, but is there already some form of URL rewriting and/or request dispatch API?
(thinking of being able to have a single virtual, url-rewritten web server serve DWScript, static files, and other mORMot services on the same port)

Yep, but sometimes good old fashionned SQL beats OO in terms of expressiveness. I find OO is practical for browsing & updating data, but SQL is better for joining/filtering "en masse" or spitting out reports/statistics.

I don't find LINQ f.i. to be much neater than the corresponding SQL would be IMHO. I've seen several articles caliming the opposite, but the examples were typically twisted and the SQL looked like auto-generated SQL (like f.i. http://www.linqpad.net/WhyLINQBeatsSQL.aspx)

There is another thing I would like to give a try to: exposing an array of objects or records (in memory) as transient SQLite virtual tables.
This would allow things like (using TPoint just for illustration, the query is meaningless)

  a1 : array of TPoint;
  a2 : array of TPoint;
  ...
  result := (select a1.x, a2.y
               from a1 left join a2 on (a1.x=a2.x)
               where a1.y<a2.y).ToArray(TPoint);

performance might not always be extraordinary, but you could express wicked things, and the work required to make the above would be reasonnable given that SQLite would be doing all the hard stuff.