mORMot Open Source

@zen010101 could you please run a second test?

* Done in 92470c90 https://github.com/landrix/mORMot2/comm … 54d9abce28 :
  both hooks now live in virtual sub-methods
  (THttpAsyncServerConnection.DoBodyDownload and
  THttpServerSocket.DoBodyDownload), and the managed locals are out of the
  OnRead / GetRequest hot paths.

* Regarding THttpSocket.Destroy calling Http.ProcessDone: I checked every
  place that sets rfContentStreamNeedFree.

  In the THttpSocket hierarchy, the flag is only set for streams created by
  the server:

  * ContentFromFile responses
  * the progressive static-file path
  * the new OnBodyDownload spool

  The first two are released through ProcessDone in the
  THttpServer.Process finally block. The body-download spool is released in
  DownloadBody.

  THttpClientSocket never assigns Http.ContentStream; client downloads use
  the GetBody(DestStream) parameter, and ownership stays with the caller.

  So the new call should be a no-op everywhere except for the case it is meant
  to cover: a connection being closed after the event in GetRequest, but
  before deferred body retrieval by the thread pool or THttpServerResp.

  I would still appreciate a second check in case I missed an ownership path.

* On stream lifetime, nginx as a reverse proxy spools request bodies to a
  temporary file under client_body_temp_path. proxy_request_buffering is
  enabled by default.

  client_body_in_file_only has two relevant modes:

  * clean: nginx deletes the file when the request is complete
  * on: nginx leaves the file in place, and the application becomes
    responsible for it

  In the latter case, the application receives the path through
  $request_body_file.

  The current branch effectively implements clean while exposing the file
  name through InContent, similar to $request_body_file.

  For customizable lifetime handling, I prefer the
  TFileStreamEventuallyDelete approach because it would remove the need for
  ContentInputName entirely.

  TFileStreamEventuallyDelete = class(TFileStreamEx)

  It would delete the file on destruction unless DeleteFileOnDestroy is set
  to false.

  The event would return this stream, or any other TStream as it does today.
  The server would free it through the existing rfContentStreamNeedFree paths,
  while the deletion policy would stay with the stream class.

  The default behavior would match nginx's clean mode. A handler that wants
  to keep the file could set DeleteFileOnDestroy to false or rename the file
  first, which would correspond to nginx's on mode.

  It would also be useful to let the handler reuse the already-open stream
  instead of reopening the file by name. Once the body is complete, the
  server could rewind the stream and expose it on the request, for example
  through a new property on THttpServerRequestAbstract:

  InContentStream: TStream

  The connection would remain responsible for freeing it after the request.
  InContent could still contain the file name with the STATICFILE marker,
  serving as the $request_body_file equivalent.

  This would also remove the current free-before-Prepare sequence in
  DoRequest, because the input stream would move from Http.ContentStream to
  the request instance before response processing begins.

If this design looks right, I will implement it this way. You can choose the
class name and location; mormot.core.os, next to TFileStreamEx, seems like the
natural place.

Thanks for the quick review and merge.

I can take the server-side implementation next, based on the approach outlined earlier in this thread:

* Add a `THttpServerGeneric` callback that is invoked after the request headers have been parsed, but before the body is read. The callback would return the destination `TStream` for the request body; returning `nil` would preserve the existing in-memory behavior. Most of the required plumbing is already in place, since `THttpRequestContext.ProcessRead` writes to `ContentStream` when one is assigned.
* Use a spooled temporary file as the default stream implementation. This should also work safely with `THttpAsyncServer` and its relatively small worker pool. `TPipeStream` could remain available for blocking server implementations.
* Expose the uploaded body to the handler using `InContentType = STATICFILE_CONTENT_TYPE` and `InContent` set to the temporary file path, mirroring the existing response-side mechanism.
* Wrap this at the REST layer through a `TRestUriContext.InStream` property.
* Extend SOA method handling so that an interface method with a single `TStream` parameter can consume the request body directly.
* In streaming mode, reject compressed request bodies with HTTP `415 Unsupported Media Type` and return `Accept-Encoding: identity`.
* Feed the resulting stream into `THttpMultiPartDecoder`, allowing multipart/form-data uploads of arbitrary size with constant memory usage. This should also address issue #292.

Does this match the architecture you had in mind?

I would follow the same workflow as before: agree on the public interface here first, then implement it on a branch together with the relevant tests.

@flydev could you please add a license at https://github.com/flydev-fr/mormot.ai/ … mot-ai-sdk?

Done as suggested: I added a new TTestCoreBase.MultiPartDecoder method and moved the multipart tests there from MimeTypes. MimeTypes now only covers MIME recognition, while the MIM test data array is shared by both.

The existing encoder round-trips are now also decoded incrementally by THttpMultiPartDecoder. This includes Rfc2388NestedFiles=true, where a second decoder reads the nested multipart/mixed directly from the outer Current.Content stream.

I also added decoder-specific tests for a 300 KB section containing many CRLF--boundary near-matches, several work-buffer sizes, a source returning only 1–7 bytes per Read(), delimiters split at every buffer position, token and quoted-pair parameters, padding, preamble and epilogue, the header-line limit, truncated input raising EHttpMultiPart, and edge cases for MultiPartFormDataBoundary().

1,310 assertions, green with FPC aarch64-linux and Delphi Win32/Win64.
the whole TTestCoreBase suite still passes on both.

One issue I ran into: Current.Content.Size reports the number of bytes read so far because the section length is not known in advance. Therefore Delphi's CopyFrom(src, 0) silently copies nothing. The documentation recommends StreamCopyUntilEnd() or a normal Read() loop instead, as already done in mormot.crypt.core and mormot.core.zip.

Two questions before I open the PR:

* Current is a record property, so each access copies its four RawUtf8 fields. The documentation recommends caching Current.Content in a local variable. Would a pointer-returning getter, similar to PHttpMultiPartStreamSection, be preferable?

* MultiPartFormDataBoundary() currently sits in mormot.net.client next to the decoder. Would it fit better in mormot.core.buffers next to the other MultiPartFormData* functions, so it could also be reused by MultiPartFormDataDecode?

https://github.com/landrix/mORMot2/tree … rt-decoder

Thanks for the detailed analysis.

A few thoughts:

• Dedicated event:  If it receives the parsed headers and Content-Length and returns the destination TStream (or nil for the current in-memory behavior), the application can choose between a temporary TFileStreamEx and a TPipeStream per request. The main DoS consideration could then be documented: pipes should be fine with THttpServer, where each connection has its own thread, or with non-blocking/Pending-style consumption; a temporary file would be the safer default for THttpAsyncServer.

• Using STATICFILE_CONTENT_TYPE for the input side makes sense and mirrors the existing output logic. Exposing it through TRestUriContext.InStream, together with detecting a single TStream parameter at the SOA level, should also avoid exposing two competing content types to users.

• 415 with "Accept-Encoding: identity" for compressed input sounds good. Explicit gzip support through TSynZipDecompressor could always be added later if there is actual demand.

The server-side foundation still depends on the event and stream design, but THttpMultiPartDecoder seems fairly self-contained. I could prepare a PR for an incremental boundary scanner operating on any TStream, processing one part at a time, roughly like Go's mime/multipart.Reader. This would let handlers consume a spooled or piped request body without loading the whole multipart payload into memory.

I would include round-trip tests against THttpMultiPartStream, covering preamble and epilogue handling, CRLF edge cases, nested content-type parameters, and boundaries split across read buffers.

Would you prefer it in mormot.net.http, or in mormot.core.buffers next to MultiPartFormDataDecode?

For the server part, I can then adapt to whichever event/stream design you choose.

@flydev Have you been able to publish the code on GitHub yet? I’d share mine with you after that.

By the way, I also developed a sqlite-vec extension using lembed, available at https://github.com/landrix/sqlite-vec-for-Delphi. It builds upon two extensions.
I'm exclusively using FreePascal now—no more Delphi. It's just because of the repo names.

Hi flydev,

thanks a lot — I didn't know about mormot2-extensions, and your
mormot-mcp-server is genuinely the closest thing to "MCP done the mORMot way"
I've seen. I went through it in detail;

- It reads like a mORMot core unit, not a Delphi port: IInvokable tools,
  RTTI-based schema generation, TDocVariant envelopes, the tri-license header
  and * section style.
- The transport layer is excellent — stdio, HTTP, SSE and a real Streamable
  HTTP with token-by-token streaming (incl. the Claude CLI demo). That directly
  answers the streaming question from my post.
- The TSynTestCase coverage for the transports (origin validation, sessions,
  batching, notifications, bad-JSON, 405 paths) is exactly the rigor I'd want.

Where my use case still diverges — and why I think there's room for both:

1. Spec revision. Our agents target the current MCP revision
   (2025-11-25); your server is on 2024-11-05 + Streamable HTTP 2025-03-26.
   For us the newer revision isn't optional.
2. Auth is our product core. We're a SaaS backend with JWT, HttpOnly refresh
   cookies, __Host-csrf double-submit, a role/permission model and owner-capped
   scopes. MCP tool execution has to run *inside* that context on our existing
   mORMot host (TUriRouter + guards), not as a separate server. Your
   TMcpAuthContext is a nice foundation, but the wiring is fairly specific to us.

So I'll keep building our mormot.ai.* clean-room layer (current spec + auth bound
to our context), but I'd love to align the Streamable HTTP transport with your
approach rather than reinvent it.

Which loops back to my question for Arnaud: with community MCP servers now existing,
is there appetite to converge any of this under an official namespace, or do you
prefer it stays as companion packages? Happy either way.

Thanks again for pointing me to it.

Best,
Sven

Well, I just can't call the function sqlite3.db_config(sql.DB, SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION,1);
under Delphi Win64. There is an access violation directly in the sqlite library.
Both in the static version and in the DLL. I can't provide a more detailed analysis because I can't debug sqlite.

It works with Delphi Win32. But the extension is only Win64.

(I have tested the extension with DiSQLite3, there they works)

---------------------------
GExperts Debugger Exception Notification
---------------------------
Project Project1.exe raised exception class EAccessViolation with message 'Access violation at address 0000000000D1EE5F. Read of address FFFFFFFFFFFFFFFF'.
---------------------------
[&Filter ...] [Ignore &All this Session] [&Break] [Additional &Info] [&Continue]
---------------------------
ThreadId=24592
ProcessId=1
ThreadName="Main"
ExceptionMessage="Access violation at address 0000000000D1EE5F. Read of address FFFFFFFFFFFFFFFF"
ExceptionName="EAccessViolation"
ExceptionDisplayName="$C0000005"
ExceptionAddress=00D1EE5F
FileName=<not available>
LineNumber=<not available>

---------------------------

Somehow none of it works. Or it doesn't work with my extension. Does anyone know more about this? I want to download the following extension.

https://github.com/asg017/sqlite-vec/re … .2-alpha.6

Can anyone help?

with wxsqlite it works but i need a DB Explorer

Thanks

For the 64bit variant I have to take the SQLite DLL from https://github.com/utelle/wxsqlite3/releases, right?

Is there a database viewer that supports this encryption?

thank you @ab, for your explanations. I would have to check that again. But I think we exceed the 16 MB limit also not. And the files are rarely changed.

Any news on this project? Win64 Support,...Does it make sense to update to the latest version of SpiderMonkey?

according to my first post,

This code should work, but it does not.
Can someone take a look on this, please.
thanks

var
  sql : TSQLDatabase;
  blob : TSQLBlobStream;
  memory : TMemoryStream;
begin
  sql := TSQLDatabase.Create('d:\temp\test.db');
  try                
     sql.Execute('CREATE TABLE IF NOT EXISTS "files" ("UID"    INTEGER PRIMARY KEY AUTOINCREMENT, "Filename"    TEXT NOT NULL, "File"    BLOB);');
    memory := TMemoryStream.Create;
    memory.LoadFromFile('d:\temp\v\DOCUMENT_2013-20280_2013_05_14_11_14_31.XML');
    memory.Position := 0;
  
    sql.Execute('INSERT INTO files (Filename) VALUES ("'+StringToUTF8('DOCUMENT_2013-20280_2013_05_14_11_14_31.XML')+'")');
    blob := sql.Blob('main','files','File',0,true);     <------------ cannot open value of type null extended_errcode=1
    blob.CopyFrom(memory,memory.Size);
    memory.Free;
    blob.Free;
  finally
    sql.Destroy;
  end;

i do my best.

I have to look also for PDF/A-3 support.