mORMot Open Source

Yes, they should really only be used to pass commands, rather than data.
For instance for a mail app, you wouldn't pass the new mails, but a command like "check new mails", and the app would then fetch the mails through regular means.

I guess it helps to consider platform notifications as being barely improved remote keypress events

But mostly the whole point is that on mobile devices, either the application is active and it's okay to poll like a madman, or it's not active, and you can use neither websockets nor long polling. Desktops and laptops are a different story of course.

Another advantage of polling when the app is active is that you can report connection lag to the user, which is quite important for a chat app, multi-player game or trading views!
With long poll or websockets, you can't tell connection lag apart from a lack of updates... unless you "ping" frequently, but then you're no better than polling.

They allow coupling, but don't dictate it. DWS has server-side compilation and strong-typing, so it's not like PHP where you can never now exactly what access what and how or when.

A downside of logic-less data-driven templates is that they're runtime-evaluated, with no strong typing, so they require more effort to maintain to maintain, and are error-prone (a weakness shared by untyped languages like PHP).

I disagree, the XML portion of AJAX is pretty much dead, but the asynchronous, javascript-driven aspects have become ubiquitous.

Pretty much all of google and twitter webpages are JavaScript applications, which load not just the data asynchronously, but the presentation and the code itself, and arguably google "webpages" are architectured like traditional fat clients, with dynamic libraries, data access drivers, UI drivers & APIs, etc. they just happen to run in a web browser.

If you look at trends for gmail, charts, maps, google+, analytics, adsense, drive etc. they're all ever more heavy on the client-side logic, and they're all relying on javascript-driven asynchronous loading (many even have loading progress bar or pacifiers, which I don't consider a progress btw)

While mustache and logic-less templates was interesting as a reaction to PHP (and JSP), I'm not convinced it still is very relevant: you can built simple stuff easily with mustache, but you can't build modern web UIs with it, not without throwing a lot of JS or going through extra hoops. Popular template engines these days seem to all have logic.

Great, thanks!

Thanks!

Thanks! A bit out of budget as it's for experimenting (don't know if you caught my Mandel experiment why it was online).
GPU is fast, but servers with GPU are very expensive.

Currently my best solution is libjpeg-turbo (http://libjpeg-turbo.virtualgl.org/) using the headers from http://www.opengl24.de/header/libjpeg

Compression speed is 7 to 8 times faster than TJPegImage, I'm not entirely sure vs IPP (they claim to be in the same ballpark).

I was under the impression that a known or predictable nonce / salt was as good as no salt from a brute-force hacking point of view, for the same reason that a salted password hash with SHA256 won't be enough to secure password shorter than 8-9 characters from a GPU brute-force crack if the salt is known and SHA256 is not applied thousands of times.

I've read that given the GPU perfs, an ASIC chip could likeley brute-force a salted hash password up to 10-12 characters passwords (numbers, upper and lower case characters and the common punctuation symbols) in a short time frame, and that this was the reason why even salted password databases were hacked these days.

But that may not apply to the situation here

I've observed the opposite (http://delphitools.info/2011/12/13/pimp … -xorshift/)

What about using RDTSC? It's wildly varying, especially on machines with multiple processors.

Wasn't the randomness of it quite low? There were only 86400 different hash values possible per day, and they are predictable long in advance.

I'm a bit wary of pseudo-random generated from time-based seeds at the millisecond level, since the machine time is usually known at that level of precision (these days, most machines are time-synchronized so they're typically within a second of the reference time).

While GetTickCount is a bit less predictable since it depends on the time the machine was last rebooted, it doesn't evolve very fast, and for Windows servers, you can make a good guess based on the latest critical windows updates release dates.

Anyway, my old quad-core CPU can f.i. compute 6 million SHA256 hashes per second with SynCrypto, that's enough to brute-force the whole 42 days range of GetTickCount-based salted hashes in about 10 minutes, and a GPU brute-forcers could scan the whole range in a matter of seconds.

Given the above, since I.FromNow will be guessable, GetTickCount range is limited, so the PtrUInt(Sender) is the only portion that strengthens the nonce, but the range isn't that huge and similar values will occur between runs of the executable (since the TSQLRestClientURI instance size is rather large, there is a good change it'll be allocated from a highly-aligned sub-allocator block in the memory manager)

I see, even though the first AES block of a message is always 100% new random, and that random carries over the rest of the message through CFB, a message is "correct" in isolation and thus can be replayed.

However, that raises another question, to avoid the replay, it means that the nonce must be provided by the other side of the communication channel? Ie. for a client to server message, the client would have to use a nonce provided by the server, and vice-versa? (otherwise unless the server caches recent nonces, the client could replay by reusing the same nonce?)

Is anyone maintaining a standard public repository for mORMot?

git, svn, or anything else would do, or a bridge from fossil (like git-svn does between svn and git).

Currently I'm manually synchronizing with an internal SVN, but that's labor-intensive, and of course all the check-in history is lost (I end up with just "updated from mORMot" and similar comments of mine).
Also for open-source projects, it's a little bit complicated to have dependencies to mORMot, as the zip downloads are impractical, and fossil requires login/password, so everyone ends up with more or less outdated forks...

Impressive stuff!

Following this from afar at the moment (spare time goes into improvements & fixes for 1.1).

With 1.1 there is some RTTI capability in SMS (class attributes, reading & writing published fields, invoking published properties getters & setters), it's rather raw and not wrapped up, but it's there

What other compilers produce Delphi-compatible obj? I quickly looked a while ago, but without much success, if one of the top grade C compilers could be used, that could indeed be convenient.

FWIW the bundling of the dll in the exe is what I've been using for a while, there are two ways to do, one is to simply extract the resource to a temporary file (with a random name, that windows cleans up automatically), this works fine IME, never had any issues with anti-virus with it.
The other is to map the dll directly from the resource. I haven't tried it, this avoid the need to extract anything but requires some low-level wizardry.

By stateful I mean when you want to use the websocket not to push notifications, but to establish a connection to the server in which the actions related to a frame sent through the websocket aren't fully defined by that frame, but also defined by previous frames, and where that state isn't shared with other websockets (like in FTP or SMTP f.i.).

That makes a it a special case where the group consists of a single websocket (and then you only have to generate a unique name for the group, using Time+Random or whatever)

The concept of "client" wouldn't be part of the mechanism, it would be something you would add on top, and same goes for multiplexing on the same websocket.

XE3 uses could be problematic due to the dotted namespaces, though as long as the aliases work, that should be okay (ie. not looking forwards to ifdef's in the uses clauses...)

Eventually yes, though the relevant functionality was added in Win8/Win2012, and I have access to neither at the moment...

Updated & simplified the wrappers.

I've added UIB driver, it's a bit heavier in terms of code size and dependencies, but full-featured and shows it's simple to wrap just any db layer.

It was with SynDBODBC.

Compiler bug confirmed: issue comes and goes when building/rebuilding... and that's an "object" as in the other bug you linked.

Though since it uses inheritance and virtual methods, it can't be replaced by a record, but has to become a class, which means various other changes in the rest of the code to create/destroy manually, but that's probably the only option as compiler support for "object" is becoming weaker and weaker.

Firebird usually, Oracle & MS SQL for larger uses or when the customer already has licenses/admins, and SQLite for everything else.

For Firebird we used to use IBX a long time ago, but due to various bugs and bloat, we've fallen down to the FreeIB layer that IBX was built upon, with a few fixes, it's still running smoothly. There doesn't seem to be any official place for FreeIB anymore, or I would have published the fixes (even though these aren't extensive, and are essentially related to adding Unicode support and fixing minor blob issues), though I'm not sure what the Borland license headers mean (and so if those files are really redistributable from the Borland version, if the original open-source version should be used, or if headers should be remade from scratch from the C headers).

Okay, good to know, I'll go directly for SynDB then!

We've been evolving our own DB connection layers over the years @work, so I haven't used any of the bundled Delphi DB components in years (except to test occasionally... and shake head in disbelief). Having a new source of fast & tested drivers is good!

Also you mentioned ORM, I've not looked into how you're tying mORMot to the servers yet, f.i. to have a ".dws" handler cooperate with a native Delphi mORMot handler.
I guess you're supposed to subclass, check the request, and then call inherited if the request is not handled? (if so, there would be nothing special required for me to make the dws http server compatible with other Delphi-based mORMot handlers)

First version is out there

You'll need trunk latest DWScript & Synopse files, Win2008 or Vista or better (http.sys 2.0 requirement)

I made precompiled binary with mini demo site for those that want to test directly:
http://code.google.com/p/dwscript/downl … Server.zip
For config options and server doc, see http://code.google.com/p/dwscript/wiki/WebServer

At the moment telling notepad++ that .dws files are highlighted as html works "almost" well enough.

Demo includes a basic memory + cpu usage monitor, a minimal DB (SQLite) exemple, and some special cases.
Throttling (CPU and Bandwidth), SSL and NCSA logging are supported too.

Btw, any chance of supporting the WideChar variants in SQLite? ie sqlite3_bind_text16, sqlite3_column_text16, etc. or are they missing from the .obj?

1) good point, eliminated

2) yes, one of the key reason behind the move to http.sys 2.0 is that authentication is supported in the kernel driver (not just faster and less code required, but also less trouble with service/application access rights)

I've got it working now, with logs & bandwidth throttling too (haven't tried authentication yet, nor any of the other properties). Most trouble I had was with a MS documentation bug on MSDN.

If you want to have a look, you can find the relevant files in DWScript SVN under Libraries\SimpleServer, though the rest of the demo in the SVN is WIP and doesn't use them yet, once I get everything in working order, I'll commit the rest.
Btw, I made a SetCompressHeader optimization that should be easy to backport (eliminates the temporary string).

Yes, though the code above currently makes no attempt at distinguishing between v1 & v2.
The snags with supporting both will be a change in the request/response record (they added a couple fields), and having all the v2 functions optional from v1.

That said, the changes from v1 to v2 are rather simple, with some code not requiring any change at all.

I've found a partial (but dirty) workaround by importing HttpCancelHttpRequest from http 2.0, and aborting the request when the headers have been received.
F.i. this will forbid any posts larger than 1kB

          if InContentLength<>0 then begin
            if InContentLength>1024 then begin
               //SendError(412, 'Content-Length too large');
               Http.CancelHttpRequest(fReqQueue, Req^.RequestId, nil);
               continue;
            end;

It's a bit dirty because the request is canceled, so you can't send a response.
The alternative is to just fetch all the content in a rotating buffer and discard everything as it comes in, this allows sending the error response, but then that means your incoming bandwidth will have been eaten, so the cancel may not be that dirty (and you can catch the error on the client side if it's a legit post attempt).

Delving into it, I found Http 2.0 API includes a lot of useful stuff, like kernel-level authentication, static cache, logging, etc.
It ups the requirements to Vista or Windows 2008 though, but looks worth it if you're going to front the server on the wild web (otherwise, it's not too hard to DDOS an HttpAPI 1.0 server by sending in large simultaneous posts that'll keep the THttpApiServer.Execute threads busy eating all the RAM).

Are there any plans to up HttpAPI to 2.0 level?
If so, maybe HttpAPI support it should be taken out of the SynCrtSock unit, so it becomes more easy to pick one or the other by just using the relevant unit.

At a minimum, having an OnHeaderReceived event would be nice, so the above header-related processing could be handled in our code.

Yep, noticed that.

Also it is single-threaded, and in my tests it reached 100% CPU utilisation of its core (ie. 25% on my quad-core), without having the Synopse web server maximized... so it was also measuring its own performance as a client rather than that of the server.

I tried to alleviate it by running it from different client machines at the same time, but that's not very practical.

Yep!

Problem is, it's so fast we can't claim it's faster than the rest, because test clients aren't fast enough to differentiate...
Making a custom test client could alleviate that, but then it would raise issues of fairness of the test...

Initially it was using events & signaling, got changed a couple years ago when I introduced IOCP-based worker threads and futures (aka promises).
Currently our DB layer is asynchronous too, and a few background caching and UI tasks have been converted too.

I tested here, and indeed it is! well done!
The compression is also better in all scenarios I tested (sometimes not by much, but always better).
I've just replaced LZRW1 with SynLZ for the internal test middle-ware here, it should be seeing a few gigabytes of LAN traffic in the next days

Unless I misunderstood SynLZ is LZab? (lempel-ziv-arnaud-bouchez i.e designed by you?)

What does that mean exactly? That it will crash or corrupt, or just that the compression ratio won't be good?

Thinking on it, maybe the current THttpApiServer class could remain in a similar or "low level no bells nor whistles" fashion, and a new subclass be introduced with the extra http response/request class?

This would also have the benefit of "opening up" THttpApiServer a bit for subclassing (currently it can't really be sub-classed, as all the logic is private or in SynCrtSock's implementation)
And with the above in mind, it would be nice if the Process event weren't called directly, but rather called from a DoProcess virtual method, with would facilitate subclassing and adding custom processing before/after/in-place-of the request (such as standard http logs)