mORMot Open Source

vs

Member

Registered: 2019-10-21

Posts: 61

Feature request — let a handler opt out of server-side Range handling

Hi Arnaud,

  I'm building a small reverse proxy in front of several mORMot HTTP servers — a
  single gateway that forwards /<prefix>/... to internal backends (one public URL,
  one TLS cert). Normal requests work fine, but I cannot transparently pass through
  HTTP Range responses for media (e.g. a large MP4 a backend serves via STATICFILE).

  The blocker is that the server applies its OWN Range handling to my handler's
  response:

  * Input side: the incoming `Range:` header is filtered out of Ctxt.InHeaders, so
    my proxy handler can't see/forward it. I can solve this with hsoHeadersUnfiltered.

  * Output side (the real blocker): when rfWantRange is set,
    THttpRequestContext.CompressContentAndFinalizeHead calls ValidateRange, which
    re-slices my OutContent by the absolute RangeOffset. So when my handler already
    returns a correct partial 206 + Content-Range (the bytes I fetched from the
    backend for `bytes=N-`), the server applies the offset a SECOND time:
    RangeOffset >= length(OutContent) -> ContentLength := 0 -> empty body. The
    client gets a 206 with ~0 bytes.

  I couldn't find a THttpServerOption to disable this output-side re-processing —
  hsoHeadersUnfiltered keeps the header on input, but rfWantRange is still set, so
  ValidateRange still runs.

  Minimal repro (useHttpSocket):
    - Handler: on a request whose Range is `bytes=N-` (N>0), set RespStatus := 206,
      add `Content-Range: bytes N-(N+chunk-1)/total` to OutCustomHeaders, and set
      OutContent to the `chunk` bytes for that range.
    - Client: GET with `Range: bytes=N-`.
    - Expected: 206 with `chunk` bytes. Actual: 206 with 0 bytes (the server
      re-ranges OutContent by the absolute N).

  Request: a per-response way for a handler to say "this response is final / I
  manage the range myself — ship my status + headers + OutContent verbatim; do not
  set rfWantRange, do not run ValidateRange, do not re-slice OutContent." For
  example, a response flag the handler can set.

  I'm aware the usual recommendation here is nginx in front + X-Accel-Redirect
  (OnSendFile / NginxSendFileFrom). In this project I specifically want mORMot
  itself to be the single gateway binary (one URL, one TLS cert, no nginx), so I'm
  not looking for an external proxy — only for the minimal primitive that lets
  mORMot's own HTTP server pass a Range/206 response through transparently from a
  handler. This looks like the concrete missing piece under the "reverse proxy"
  scope of #387 / #389.

  With that primitive I can implement a lightweight transparent reverse proxy and
  cap the forwarded range myself to keep memory bounded — no full streaming wrapper
  needed.

  Version: mORMot 2.4, FPC Win64.

  Thanks for the great work!

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 15,487

Website

Re: Feature request — let a handler opt out of server-side Range handling

AFAIR a large local file as STATICFILE will use the built-in range support properly. Nothing special to do: just return the file name, and the server will handle the range content.
There is no need of custom range support.

But this is clearly a limitation in some other cases, which should be handled.

vs

Member

Registered: 2019-10-21

Posts: 61

Re: Feature request — let a handler opt out of server-side Range handling

Thanks, Arnaud!

Yes — the local-file STATICFILE path is exactly what we use on the backend side, and it works flawlessly (verified on a 1.2 GB MP4: seek across the whole length is fast, no issues). So we don't need anything for STATICFILE.

Our concrete case is exactly the "other case" you mention: a reverse-proxy handler in front of the backend. The handler has no local file — it forwards the browser's Range to an upstream HTTP server, gets back a partial 206 + Content-Range + bytes, and tries to ship that response verbatim through Ctxt.OutContent + OutCustomHeaders + RespStatus=206. Because the server then runs ValidateRange on OutContent by the absolute RangeOffset, the already-correct partial body gets re-sliced to empty.

A per-response opt-out (e.g. a response flag the handler can set to say "I have produced the correct 206 + Content-Range + body — ship verbatim, skip ValidateRange") would unblock this without affecting STATICFILE semantics. Any preference on the API shape would let us prototype against your tree.

Thanks again for the great work!