mORMot Open Source

Thanks for bisecting. I can't reproduce it with your test on FPC 3.3.1 aarch64-win64 (-O1/-O2/-O3) or FPC 3.2.2 aarch64-linux (-O1/-O3), all print 200ok. So it looks like an i386 optimizer problem in FPC trunk rather than a logic bug. Does it go away with -O2 for mormot.core.variants.pas only? And which FPC revision is it (fpc -iW)?

@flydev - thank you, that was quick. The qwen3-tts-lm dataset is now in mormot2-ai
(tests/data/infer-ref/qwen3-tts-lm, unchanged from your commit 8f5a846, with your two
generator scripts under tools/infer-ref/qwen3-tts and the provenance in UPSTREAM_BASE and
NOTICE). It will be the regression test for the embeddings-input forward: both stacks
(talker and code predictor), layer states and logits, on CPU, Vulkan and CUDA.

You are right about the weight cache: both GPU backends look weights up by host address,
type and size, so a buffer freed and reallocated at the same address is served stale. The
unit header documents it as a contract, but it is a trap as soon as several stacks or
changing weights share one backend - exactly your case. It is now in the backlog next to
the embeddings input: a way to forget a weight or an address range when its owner frees
it, instead of ClearWeights for everything, on Vulkan and CUDA alike, with a test that
reuses the same address.

And thanks for publishing mormot2-qwen-tts. I will report back here when it is pushed.

Thanks a lot, Arnaud!

SQLite3: both commits replace exactly the two workarounds we carried, and we already switched to them.

• Our RollbackIfOpen existed because a failed Commit (e.g. SQLITE_BUSY) cleared the flag, so the RollBack afterwards did nothing. It is now a plain non-raising RollBack, and a new regression test checks it: a COMMIT fails against another connection's read lock, then RollBack closes the transaction.

• Our own db_config/load_extension helpers are gone. Your LoadExtension is better, since it restores the previous state and never enables the SQL function.

One small observation: LoadExtension takes a TFileName. On non-Unicode compilers (Delphi 7, FPC on Windows), a path with characters outside the ANSI code page is lost before it reaches UTF-8, so a RawUtf8 overload would avoid that. It is not urgent for us, as those extensions will go once our native engine fully replaces them.

GPU: Vulkan first, CUDA second. Vulkan runs on NVIDIA, AMD, Intel (your B580), Adreno and, through MoltenVK, Apple.

• On speed, llama.cpp's community benchmarks (indicative only) show token generation on NVIDIA about equal to CUDA, and prompt processing at about 70-95 % of CUDA. For example, an RTX 5090 does 11.8k vs 15.0k tokens/s pp512 and 274 vs 300 tokens/s tg128. So we keep CUDA as a second backend for maximum prompt throughput on NVIDIA.

• For the binding we would use only PasVulkan's stand-alone Vulkan.pas header (MIT-style licence, Delphi 7 and FPC). It loads the system Vulkan loader dynamically, so we ship no DLL, and without a GPU the executable still runs on the CPU.

• The kernels would be compute shaders compiled offline to SPIR-V and embedded in the executable; the MIT-licensed ggml-vulkan shaders are a good source.

Following your mormot.lib.*.pas remark, I could write mormot.lib.freetype and mormot.lib.harfbuzz as thin TSynLibrary bindings for what the backends load today. That doesn't touch Martin's code.

For the reader: in XRechnung for Delphi I have a small native parser that extracts the embedded invoice XML from ZUGFeRD/Factur-X files. It follows the xref chain including incremental updates, handles the usual stream filters and RC4 with an empty user password. Not a full reader, but maybe a starting point. It is my code, so I can contribute it under the mORMot licence.

How to split the units is Martin's call, of course. Martin, just tell me where I can help.

A dream is coming true. I'm going to take a look at it right away today.

Okay, I’ll try to remain Delphi-compatible for as long as possible and remove any FPC-only references for now. The reason is that I’ll be using https://github.com/joaopauloschuler/neural-api in places, and that is FPC-only—but we’ll see.

It seems to be working for the time being. However, I will no longer make allowances for Delphi or implement any special handling.

This will be my new browser start page. :-)

The issue https://github.com/synopse/mORMot2/issues/292 is effectively resolved with the latest PR.

Is there any interest in me also dealing with the other side?

https://github.com/synopse/mORMot2/issues/509

The hash was only in the test (test.net.proto.pas) - changed to
crc32cHash(body, crc32cHash(body, 0)), tests still pass.

About the lifetime: TFileStreamEventuallyDelete alone can't work with the
current flow, because we free the stream as soon as the body is complete,
i.e. before OnRequest runs - the spooled file has to survive until the
request is done. It fits perfectly once the stream stays alive until the
end of the request (the InContentStream idea). Then we can have both:
the class for the "delete it for me" case, and rfContentFileNameNeedDelete
for any other stream kind. I can implement it that way if you agree.

TPipeStream: it can work, but not out of the box.
- Write() blocks until the whole buffer is consumed (default timeouts are
  INFINITE), so in THttpAsyncServer a slow consumer stalls an event loop
  thread. With WriteTimeout or psoWritePartial a partial Write() raises
  EWriteError in WriteBuffer, which our state machine already turns into 507.
- Read() only returns a clean 0/EOF if ExpectedSize was set and reached -
  otherwise the consumer sees -1 (abort) instead of a normal end of body.
  We know the size for Content-Length, but not for chunked.
- Freeing the stream releases the buffer, so the consumer must be gone by
  then: in practice TBackgroundPipeStream, and that free blocks the server
  thread until the consumer thread finished.
I'd add a /pipe test case to pin all three down.

You were right about the exception frames. 78f283fe drops the
try..except around the OnRead loop; TStream write failures are trapped at
the two ContentStream.WriteBuffer() calls in ProcessRead (only reached
when a stream is assigned) and reported as a new hrsErrorContentStreamWrite
state, which OnRead maps to 507, next to hrsErrorPayloadTooLarge -> 413.

One more pass over the whole branch found two things worth mentioning
(a2f23504): the async family started counting a chunked body from
ContentLength = -1 - GetBody normalises it to 0, ProcessRead never did -
so the limit was one byte off and a bogus Content-Length header could
skew it further. And Prepare now clears an InContentType the client sent
as '!STATICFILE' by itself ('!' is a valid token char), so a callback can
trust that marker to mean a spooled body.

Branch @ a2f23504: FPC aarch64-linux and Delphi Win32/Win64, 179
assertions in TNetworkProtocols.BodyDownload. Still open before the PR is
the stream lifetime - happy to rework it along the
TFileStreamEventuallyDelete idea whenever you've settled on the shape.

@zen010101 could you please run a second test?

* Done in 92470c90 https://github.com/landrix/mORMot2/comm … 54d9abce28 :
  both hooks now live in virtual sub-methods
  (THttpAsyncServerConnection.DoBodyDownload and
  THttpServerSocket.DoBodyDownload), and the managed locals are out of the
  OnRead / GetRequest hot paths.

* Regarding THttpSocket.Destroy calling Http.ProcessDone: I checked every
  place that sets rfContentStreamNeedFree.

  In the THttpSocket hierarchy, the flag is only set for streams created by
  the server:

  * ContentFromFile responses
  * the progressive static-file path
  * the new OnBodyDownload spool

  The first two are released through ProcessDone in the
  THttpServer.Process finally block. The body-download spool is released in
  DownloadBody.

  THttpClientSocket never assigns Http.ContentStream; client downloads use
  the GetBody(DestStream) parameter, and ownership stays with the caller.

  So the new call should be a no-op everywhere except for the case it is meant
  to cover: a connection being closed after the event in GetRequest, but
  before deferred body retrieval by the thread pool or THttpServerResp.

  I would still appreciate a second check in case I missed an ownership path.

* On stream lifetime, nginx as a reverse proxy spools request bodies to a
  temporary file under client_body_temp_path. proxy_request_buffering is
  enabled by default.

  client_body_in_file_only has two relevant modes:

  * clean: nginx deletes the file when the request is complete
  * on: nginx leaves the file in place, and the application becomes
    responsible for it

  In the latter case, the application receives the path through
  $request_body_file.

  The current branch effectively implements clean while exposing the file
  name through InContent, similar to $request_body_file.

  For customizable lifetime handling, I prefer the
  TFileStreamEventuallyDelete approach because it would remove the need for
  ContentInputName entirely.

  TFileStreamEventuallyDelete = class(TFileStreamEx)

  It would delete the file on destruction unless DeleteFileOnDestroy is set
  to false.

  The event would return this stream, or any other TStream as it does today.
  The server would free it through the existing rfContentStreamNeedFree paths,
  while the deletion policy would stay with the stream class.

  The default behavior would match nginx's clean mode. A handler that wants
  to keep the file could set DeleteFileOnDestroy to false or rename the file
  first, which would correspond to nginx's on mode.

  It would also be useful to let the handler reuse the already-open stream
  instead of reopening the file by name. Once the body is complete, the
  server could rewind the stream and expose it on the request, for example
  through a new property on THttpServerRequestAbstract:

  InContentStream: TStream

  The connection would remain responsible for freeing it after the request.
  InContent could still contain the file name with the STATICFILE marker,
  serving as the $request_body_file equivalent.

  This would also remove the current free-before-Prepare sequence in
  DoRequest, because the input stream would move from Http.ContentStream to
  the request instance before response processing begins.

If this design looks right, I will implement it this way. You can choose the
class name and location; mormot.core.os, next to TFileStreamEx, seems like the
natural place.

Thanks for the quick review and merge.

I can take the server-side implementation next, based on the approach outlined earlier in this thread:

* Add a `THttpServerGeneric` callback that is invoked after the request headers have been parsed, but before the body is read. The callback would return the destination `TStream` for the request body; returning `nil` would preserve the existing in-memory behavior. Most of the required plumbing is already in place, since `THttpRequestContext.ProcessRead` writes to `ContentStream` when one is assigned.
* Use a spooled temporary file as the default stream implementation. This should also work safely with `THttpAsyncServer` and its relatively small worker pool. `TPipeStream` could remain available for blocking server implementations.
* Expose the uploaded body to the handler using `InContentType = STATICFILE_CONTENT_TYPE` and `InContent` set to the temporary file path, mirroring the existing response-side mechanism.
* Wrap this at the REST layer through a `TRestUriContext.InStream` property.
* Extend SOA method handling so that an interface method with a single `TStream` parameter can consume the request body directly.
* In streaming mode, reject compressed request bodies with HTTP `415 Unsupported Media Type` and return `Accept-Encoding: identity`.
* Feed the resulting stream into `THttpMultiPartDecoder`, allowing multipart/form-data uploads of arbitrary size with constant memory usage. This should also address issue #292.

Does this match the architecture you had in mind?

I would follow the same workflow as before: agree on the public interface here first, then implement it on a branch together with the relevant tests.

@flydev could you please add a license at https://github.com/flydev-fr/mormot.ai/ … mot-ai-sdk?

Done as suggested: I added a new TTestCoreBase.MultiPartDecoder method and moved the multipart tests there from MimeTypes. MimeTypes now only covers MIME recognition, while the MIM test data array is shared by both.

The existing encoder round-trips are now also decoded incrementally by THttpMultiPartDecoder. This includes Rfc2388NestedFiles=true, where a second decoder reads the nested multipart/mixed directly from the outer Current.Content stream.

I also added decoder-specific tests for a 300 KB section containing many CRLF--boundary near-matches, several work-buffer sizes, a source returning only 1–7 bytes per Read(), delimiters split at every buffer position, token and quoted-pair parameters, padding, preamble and epilogue, the header-line limit, truncated input raising EHttpMultiPart, and edge cases for MultiPartFormDataBoundary().

1,310 assertions, green with FPC aarch64-linux and Delphi Win32/Win64.
the whole TTestCoreBase suite still passes on both.

One issue I ran into: Current.Content.Size reports the number of bytes read so far because the section length is not known in advance. Therefore Delphi's CopyFrom(src, 0) silently copies nothing. The documentation recommends StreamCopyUntilEnd() or a normal Read() loop instead, as already done in mormot.crypt.core and mormot.core.zip.

Two questions before I open the PR:

* Current is a record property, so each access copies its four RawUtf8 fields. The documentation recommends caching Current.Content in a local variable. Would a pointer-returning getter, similar to PHttpMultiPartStreamSection, be preferable?

* MultiPartFormDataBoundary() currently sits in mormot.net.client next to the decoder. Would it fit better in mormot.core.buffers next to the other MultiPartFormData* functions, so it could also be reused by MultiPartFormDataDecode?

https://github.com/landrix/mORMot2/tree … rt-decoder

Thanks for the detailed analysis.

A few thoughts:

• Dedicated event:  If it receives the parsed headers and Content-Length and returns the destination TStream (or nil for the current in-memory behavior), the application can choose between a temporary TFileStreamEx and a TPipeStream per request. The main DoS consideration could then be documented: pipes should be fine with THttpServer, where each connection has its own thread, or with non-blocking/Pending-style consumption; a temporary file would be the safer default for THttpAsyncServer.

• Using STATICFILE_CONTENT_TYPE for the input side makes sense and mirrors the existing output logic. Exposing it through TRestUriContext.InStream, together with detecting a single TStream parameter at the SOA level, should also avoid exposing two competing content types to users.

• 415 with "Accept-Encoding: identity" for compressed input sounds good. Explicit gzip support through TSynZipDecompressor could always be added later if there is actual demand.

The server-side foundation still depends on the event and stream design, but THttpMultiPartDecoder seems fairly self-contained. I could prepare a PR for an incremental boundary scanner operating on any TStream, processing one part at a time, roughly like Go's mime/multipart.Reader. This would let handlers consume a spooled or piped request body without loading the whole multipart payload into memory.

I would include round-trip tests against THttpMultiPartStream, covering preamble and epilogue handling, CRLF edge cases, nested content-type parameters, and boundaries split across read buffers.

Would you prefer it in mormot.net.http, or in mormot.core.buffers next to MultiPartFormDataDecode?

For the server part, I can then adapt to whichever event/stream design you choose.

@flydev Have you been able to publish the code on GitHub yet? I’d share mine with you after that.

By the way, I also developed a sqlite-vec extension using lembed, available at https://github.com/landrix/sqlite-vec-for-Delphi. It builds upon two extensions.
I'm exclusively using FreePascal now—no more Delphi. It's just because of the repo names.

Hi flydev,

thanks a lot — I didn't know about mormot2-extensions, and your
mormot-mcp-server is genuinely the closest thing to "MCP done the mORMot way"
I've seen. I went through it in detail;

- It reads like a mORMot core unit, not a Delphi port: IInvokable tools,
  RTTI-based schema generation, TDocVariant envelopes, the tri-license header
  and * section style.
- The transport layer is excellent — stdio, HTTP, SSE and a real Streamable
  HTTP with token-by-token streaming (incl. the Claude CLI demo). That directly
  answers the streaming question from my post.
- The TSynTestCase coverage for the transports (origin validation, sessions,
  batching, notifications, bad-JSON, 405 paths) is exactly the rigor I'd want.

Where my use case still diverges — and why I think there's room for both:

1. Spec revision. Our agents target the current MCP revision
   (2025-11-25); your server is on 2024-11-05 + Streamable HTTP 2025-03-26.
   For us the newer revision isn't optional.
2. Auth is our product core. We're a SaaS backend with JWT, HttpOnly refresh
   cookies, __Host-csrf double-submit, a role/permission model and owner-capped
   scopes. MCP tool execution has to run *inside* that context on our existing
   mORMot host (TUriRouter + guards), not as a separate server. Your
   TMcpAuthContext is a nice foundation, but the wiring is fairly specific to us.

So I'll keep building our mormot.ai.* clean-room layer (current spec + auth bound
to our context), but I'd love to align the Streamable HTTP transport with your
approach rather than reinvent it.

Which loops back to my question for Arnaud: with community MCP servers now existing,
is there appetite to converge any of this under an official namespace, or do you
prefer it stays as companion packages? Happy either way.

Thanks again for pointing me to it.

Best,
Sven

Well, I just can't call the function sqlite3.db_config(sql.DB, SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION,1);
under Delphi Win64. There is an access violation directly in the sqlite library.
Both in the static version and in the DLL. I can't provide a more detailed analysis because I can't debug sqlite.

It works with Delphi Win32. But the extension is only Win64.

(I have tested the extension with DiSQLite3, there they works)

---------------------------
GExperts Debugger Exception Notification
---------------------------
Project Project1.exe raised exception class EAccessViolation with message 'Access violation at address 0000000000D1EE5F. Read of address FFFFFFFFFFFFFFFF'.
---------------------------
[&Filter ...] [Ignore &All this Session] [&Break] [Additional &Info] [&Continue]
---------------------------
ThreadId=24592
ProcessId=1
ThreadName="Main"
ExceptionMessage="Access violation at address 0000000000D1EE5F. Read of address FFFFFFFFFFFFFFFF"
ExceptionName="EAccessViolation"
ExceptionDisplayName="$C0000005"
ExceptionAddress=00D1EE5F
FileName=<not available>
LineNumber=<not available>

---------------------------

Somehow none of it works. Or it doesn't work with my extension. Does anyone know more about this? I want to download the following extension.

https://github.com/asg017/sqlite-vec/re … .2-alpha.6

Can anyone help?

with wxsqlite it works but i need a DB Explorer

Thanks