mORMot Open Source

I found very interesting site http://www.techempower.com/benchmarks/
I think it's a good idea to participate (for framework popularization).
It seems we must be in top of list for Windows platform http://www.techempower.com/benchmarks/# … est=update

Yes, I rewrite my code in this manner - everything work good and now I'm compatible with main mORMot branch
Just one remark - when I send static files in this manner I get incorrect statistics:

TSQLRestServer.URI()
.....
  inc(fStats.fOutcomingBytes,length(Call.OutHead)+length(Call.OutBody)+16);

we actually increase out bytes by fileName length
As for me is good to add fStats.fileTransfered counter and count number of files we send (calculating real file length is slow?)

Oh. Thanks! I miss what I can use HTTP service name. But in all case this operation required Domain Admin right and as you sad  domain sometimes not configured properly. Chaa, do you apply this patch? (SSPI is your's module)

Small proposal:
we need to initialize TSQLRestServer.fSessionCounter in some not 0 value. In current realization if we use non-signed auth scheme ( TSQLRestServerAuthenticationNone for example) we have such collision:
user A got session ID = 1
user B got session ID = 2
I restart server
user B call auth - got session ID = 1
user A call RetrieveSession with his old session ID = 1 and got credential of user B

my proposial is:

constructor TSQLRestServer.Create;
......
with SystemTime do // set start session counter = miliseconds from midnight
    fSessionCounter := (wHour*60*60 + wMinute * 60 + wSecond)*1000 +  wMilliSeconds;

not good also but partially help.

I'm little confused with ID field in ORM. Actually on database level mORMot create INT64 field type, but in source used integer. I remember this problem is already discussed in this forum, but not found topic. I once again try to move my soft from Syn* level to mORMot* level and this is big blocker for me (I use one database-wide ID generator  for all table with first 4 digit reserved for clientID (so I got something like GUID) for easy replication between different database. So I easy overflow int32 ID in real application. Is it possible (may be in future) to change  ID from int32 to Int64 ?

Little question: is PCRE thread safe?
I mean if I compile regexp (Compiled) in main thread and when use it from concurrent threads in this way:
function TStaticRule.Mutch(const Location: RawUTF8): boolean;
begin
  Result := (Compiled <> nil) and (pcre_exec(Compiled, extra, pointer(Location), Length(Location), 0, PCRE_NO_UTF8_CHECK, nil, 0)>=0);
end;
Or I must compile it for every thread separately?

created ticket http://synopse.info/fossil/tktview/dc64 … 080dec862d

2peak.tibor try to generate insert statement without params and run it via sqlplus\toad
Something like.
insert into insdemo values(`b001`,`b002`,..........);

Is it work for 95 column table?

As for me mORMot currently is not ready for server-side use in non-Windows platform because of dependents of http.sys or, in case plain socket - IOCP. So even if we compile some part under Linux it is only possible to use mORMot as CGI.
I think many time about alternative for IOCP and found good lib https://github.com/joyent/libuv. It currently used in node.js for cross-platform async IO and supported by some M$ contributors. So may be this is a way....

If you use one of sicClientDriven, sicPerSession, sicPerUser or sicPerGroup modes - look on TServiceFactoryServer property TimeoutSec

Please, pay attention to THttpApiServer.Clone method. This one creates the thread pool. Each cloned  thread registers itself to http.sys and then http.sys manages it in the similar way as HttpServer does. As far as I understand you overflow not length of queue, but time request wait in queue. So AB is right - increase thread pool size.

You must regisre server first

 THttpApiServer.AddUrlAuthorize('root','888',false,'+')); 

Please see this topic http://synopse.info/forum/viewtopic.php?id=1059 and blog post http://blog.synopse.info/post/2011/03/1 … ode-server
See also http://synopse.info/forum/search.php

Form my POV implementing SOAP client is possible if using existing mORMot classes SynCrtSock.TWinHTTP for communication layer and customJSONSerializer's to serialize/deserialize SOAP messages to delphi classes.
it certainly will require some effort, but..
Or use SynCrtSock.TWinHTTP for communication and this project http://www.philo.de/xml/ to parse XML

Magnifiquement, as always

I do not want a new routing scheme - I just want to expand a little two existing  by adding "default" behavior and one "magic" parameter retrieved from URI and combine it with existing rmREST and rmJSON_RPC   

This give me ability to rebuild my server using interface-based services and be compatible with framework not on Syn* level as today, but on mORMot* level. The next step(SpiderMonkey integration to mORMot) will be much easier for me after that..

Some additional ideas....
I have a "static file" transfer METHOD - just transfer files from server to client (something like "Samples\09 - HttpApi web server") . Now it is implemented as a published method of TSQLRestServer descendant

TMyServer = class(TSQLRestServer)
...
published
  procedure static(var Ctxt: TSQLRestServerCallBackParams);
  procedure YUI(var Ctxt: TSQLRestServerCallBackParams);
  .......
end;

I want to rewrite it using interface-based services. This give me ability to connect "static file transfer" functionality to any server (TSQLRestServerFullMemory  for example). In fact, I do want to give up all the published methods and rewrite all via interface-based service.
In current implementation I call TMyServer.static method by put
<script charset="utf-8" src="http://localhost:8080/root/static/app.js"></script>
in HTML file.
This is not interface-based service URL format. Yes, I can change this to:
<script charset="utf-8" src="http://localhost:8080/root/static.get?file=app.js"></script>  but this is not KISS from HTML POV.

Another example - I have YUI method - realization of YUI interface using my ORM as background.
I want to rewrite it as a method of interface-based service IYUI to.

To solve my tasks I need something like this:

  IStatic = interface(IInvokable)
    ['{84A6479B-4DE8-4B7A-93BB-4CE8935FE03C}']
    function default(const URI: RawUTF8; const asMIME: RawUTF8): TServiceCustomAnswer;
  end;
  Server.ServiceRegister(TStaticFileService,[TypeInfo(IStatic)], sicShared);

"Magic", I want to offer is:
1. "default" method - this is the method that should be called if no ".method" passed in URL but for ServicesRouting=rmREST. This give me possibility to write GET /root/ServiceName.... instead of GET /root/ServiceName.MethodName....
2. "URI" parameter - if parameter with name "URI": RawUTF8 exist in method parameter list value of this parameter retrieved from URL tail. This give me possibility to write  GET /root/static/pictures/myimage.gif?asMIME=application\/pdf and server call my function
TStaticFileService.default("/pictures/myimage.gif", "application\/pdf");

What do you think - is it make sense?

optExecInMainThread absolutely impossible - in this case I got single-thread application. But I need multi-thread... And I don't want to free created instance each time - just create it during first call to service and free when server die.

I try to rewrite my application to use interface-based services. In my case I need per-thread service instance. Is it possible to add TServiceInstanceImplementation.sicPerThread instance implementation pattern? From from the POV of the client it will look like sicShared  as I understand.

Very good! I make some remarks in tickets:
-  in Authentication classes - http://synopse.info/fossil/info/8c8a2a880c  seems "this presentation" link is broken...
- in Authorization classes - http://synopse.info/fossil/info/7c079c49d6 - As for me it will be good to combine onUserAuth + authorization class hierarchy. For example in my case in some services I do not use TSQLAuthUser/Group ( some of my services not use any DB at all - only LDAP for auth - for example cache service in case of ACTIVE-ACTIVE load balancing on main server)

I'm not sure how you want to combine 1. and 2. in the same time
3.4.5. is OK.
For Digest - yes - digest store md5(Username + ':' + realm + ':' + Password) as a password hash. So to use Digest combined with other methods the only way is to add 2 password column in user table - for mORMot and for Digest.

I also

I'm not sure that I understood correctly. If you mean:

 
TSQLRestServer.RegisterAuthenticationScheme('mORMot', TMoromAuth);  // here I mean mORMot algorithm using HEADER
TSQLRestServer.RegisterAuthenticationScheme('Negotiate', TNegotiateAuth);  
TSQLRestServer.RegisterAuthenticationScheme('Basic', TBasicAuth);  

then yes - it make sense. In this case Delphi client choose mORMot, browser for domain user choose Negotiate (as more strong auth method)  and for non-domain user - Basic

if you mean use mORMot-style (session_signature in URI) and header style auth in the same time - no, it not make sense as for me.

And from my POV here we need:
6. add connectionID attribute to TSQLRestServerURIParams ( for the http.sys server =  Req^.ConnectionId, for socket-based = Int64(Socket), for namedPaper = Int64(GetNamedPipeClientProcessId/GetNamedPipeClientSessionId), for messages = Int64(window handle)

7. add event onUserAuth in witch I can check user credential and fill user GroupRights (I actually get a user groups from LDAP and associate it with group binary mask). See also http://synopse.info/forum/viewtopic.php?id=1218

I propose  add support for multiple AUTH schema in mORMot.
I understand what it require some breaking changes, but as a result we got auth, what compatible with  standards and give for mORMot some additional benefits, (we can implement OAuth for example, very easy to use mORMot from browser, we can implement cache according to HTTP standards).
So, the idea is:
1) implement something like TSQLRestServer.RegisterAUTHScheme(const schemeName RawUTF8; onAuthentificate: function(Ctxt), onGetAuthentificationParams: function(Ctxt), onAuthorize: function(Ctxt)).

2) in TSQLRestServer.URI instead of

  URI.Error('',HTML_FORBIDDEN); // 403 in case of authentication failure

return

  
URI.Error('', 401);
  for each AuthScheme in registeredAuthScheme
     URI.Call.OutHead.Add('WWW-Authenticate:' + AuthScheme.SchemeName + AuthScheme.onGetAuthentificationParams(Ctxt));
  end;

HTTP protocol allow to add multiple WWW-Authenticate headers. The task of the client to choose the strongest of the supported schemes.

In case we register schemas [mORMot, Negotiate, NTLM, Digest, Basic] we got something like:

-> GET /root/something
<- 401 
  WWW-Authenticate: mORMot nonce="nonce_value"  
  WWW-Authenticate: Negotiate ...
  WWW-Authenticate: NTLM ...
  WWW-Authenticate: Digest realm="mORMot digest", qop="auth,auth-int", nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093", opaque="5ccc069c403ebaf9f0171e9517f40e41"
  WWW-Authenticate: Basic realm= "mORMot basic"'

in case of Delphi client or other client what support mORMot auth scheme next step is:

-> GET /root/something
  Authorization: mORMot UserName=..., PassWord=..., ClientNonce=... 
<- 401 
  WWW-Authenticate: mORMot session number and a private key

a) second step
-> GET /root/something
  Authorization: mORMot session_signature="zzzzzzzzzzzzzzzzzzzzzzzzzzzz"
<- 200 Ok
  content here

all other client request simple add  header

   Authorization: mORMot session_signature="...."

Browser or client are not familiar with  mORMot auth scheme just choose another. For example if I do request to intranet zone IE ignore WWW-Authenticate: mORMot and choose Negitiate, if request go to "not trusted" site - Digest and so on.

3) In TSQLRestServer.URI if HandeAuthentification=true check "Authorization: schemaName" header, find Auth scheme with schemaName from registered and call AuthScheme.onAuthirize(..)
for mORMot schema, for example, onAuthirize do the same as  in TSQLRestServer.URI in current implemetation.

4) for Neginiare and NTLM schema we can use chaa imlementation given here: http://synopse.info/forum/viewtopic.php?pid=5809#p5809
  We just must add ConnectionId to the Call structure instead of Remote-IP from chaa's code.   ConnectionId is retrived as Eric wtite in this post:
http://synopse.info/forum/viewtopic.php?pid=5816#p5816
"To the Process method I just added a connectionId parameter, which takes the Req^.ConnectionId for the http.sys server, and a Int64(Socket) for the thread/socket-based one"

Negotiate is a connection authentication scheme - not http authentication scheme, so client must be keep-alive, ConnectionId must be used instead of sessionCardinal. onAuthirize(..) handler for Negotiate just look session with ID = ConnectionId exist in sessionList.

5) Also I propose to add custom handler for  user password checking for each schema (it allow not use TAuthUser/TAuthGroup but, for example, LDAP) and   the same handler to calculate Group's for user ( In my case, for examlpe, I need to check LDAP groups for user and assotiate it with TAuthGroup

sorry for my english

To implement http level cache (not only etag but "not modified since" and so on) we must first change auth scheme used in mormot.
Now we put different session_signature in every url, so browser decide
root\blabla?session_signarute=1 and
root\blabla?session_signarute=2
is different url and cashe is therefore impossible.
So my suggestion is to change auth scheme. As minimum - move session signature to request header section, as optimal - use standart HTTP algoritm for auth(we can add out algorotm to 'WWW-Authenticate: Synopse, NTLM, digest` - in this case delphi client can use synopse algoritm, browser NTLM or digest).
I know all PRO of current imlpementation, but the contra is -
1) hard to work from browser,
2) HTTP level cache is impossible
3) impossible to work with 3party software - for example I have idea to implement WebDAV access to my server, but webDAV client know nothing about  current auth algoritm.
What do you think?

O! This is very good.

Please, look at comments in SynCommons

const
  /// maximum number of fields in a database Table
  // - is included in SynCommons so that all DB-related work will be able to
  // share the same low-level types and functions (e.g. TSQLFieldBits,
  // TJSONWriter, TSynTableStatement, TSynTable)
  // - default is 64, but can be set to any value (64, 128, 192 and 256 optimized)
  // - this constant is used internaly to optimize memory usage in the
  // generated asm code, and statically allocate some arrays for better speed
  MAX_SQLFIELDS = 64;

so just set this const to 128 and recompile.

2AB - maybe change to 128 in main trunc - I'm unfortunately have objects with large number of fields to.