You are not logged in.
Pages: 1
THttpRequestContext.Reset` does not clear `AcceptEncoding` — compression state leaks between requests on a recycled connection.
I think I have found a small but nasty state-leak in `THttpRequestContext`.
Version: mORMot 2, commit `b082ba107` (2026-09-15), Delphi 10.3 Rio, Win64,
`TRestHttpServer` created with `useBidirAsync`, options `HTTPSERVER_DEFAULT_OPTIONS`
(i.e. `rsoCompressGZip` + `rsoCompressSynLZ`).
Symptom
A client that sends no`Accept-Encoding` header at all can receive a response
with `Content-Encoding: gzip`. It obviously cannot decode it, and since it never
asked for compression it does not even try — it just sees garbage where JSON was
expected.
This is intermittent and load dependent: it only happens shortly after *another*
client has sent `Accept-Encoding: gzip` to the same server.
Root cause
`THttpRequestContext.AcceptEncoding` (declared at `mormot.net.http.pas:440`) is
only ever assigned when the incoming request actually carries the header
(`ParseHeader`, `hhAcceptEncoding` branch, line 3813):
hhAcceptEncoding:
begin
// 'ACCEPT-ENCODING:'
GetTrimmed(P + 17, P2, PLen, AcceptEncoding);`THttpRequestContext.Reset` (line 3621) clears the sibling header fields —
`Upgrade`, `BearerToken`, `ResponseHeaders`, `UserAgent`, `Referer`,
`ContentType`, `Headers` — and it does reset the derived bitset at line 3652:
integer(CompressAcceptHeader) := 0;but it never clears `AcceptEncoding` itself.
So on a recycled context, `ParseHeaderFinalize` (line 3934) re-derives the bitset
from the **previous** request's value, because the guard only tests for non-empty:
if (CompressList <> nil) and
(AcceptEncoding <> '') then // <-- stale value passes this test
CompressList^.DecodeAcceptEncoding(pointer(AcceptEncoding), CompressAcceptHeader);`CompressAcceptHeader` is therefore repopulated for a request that never asked for
any content coding, and `CompressContentAndFinalizeHead` duly compresses the body.
With the async server this is easy to hit, because `fHttp: THttpRequestContext` is
a value field of the connection object (`mormot.net.async.pas:912`) and those
objects are pooled and reused across *different* client connections — so the leak
is not limited to keep-alive requests from the same client.
Reproduction
Start a `TRestHttpServer` with the default options (gzip + SynLZ registered), then
alternate two requests to the same endpoint, from separate connections:
1. one **with** `Accept-Encoding: gzip`
2. one with **no** `Accept-Encoding` header at all
Request (2) comes back with `Content-Encoding: gzip` roughly every second or third
iteration. On my box, using a ~24 KB JSON response:
```
baseline (clean) 24514 bytes Content-Encoding: none
after gzip load #1 24514 bytes none
after gzip load #2 18766 bytes gzip <-- never requested
after gzip load #3 24514 bytes none
after gzip load #4 18766 bytes gzip <-- never requested
after gzip load #5 24514 bytes none
```
Note that `curl` does **not** send `Accept-Encoding` unless you pass `--compressed`,
which makes curl a convenient client for step (2).
Suggested fix
Clear the field in `Reset`, alongside the other header fields it already clears
(after the `Referer` block at line 3642):
if Referer <> '' then
FastAssignNew(Referer);
if AcceptEncoding <> '' then
FastAssignNew(AcceptEncoding);
RangeOffset := 0;With this applied the same test gives 0 leaks out of 10 iterations, and explicitly
requested compression still works normally (`Accept-Encoding: gzip` -> 18766 bytes
`Content-Encoding: gzip`; `Accept-Encoding: synlz` -> 27078 bytes
`Content-Encoding: synlz`).
An alternative would be to make `ParseHeaderFinalize` unconditional, or to have
`ParseHeader` always assign `AcceptEncoding` (to `''` when the header is absent),
but clearing it in `Reset` keeps it consistent with how `UserAgent`, `Referer`,
`Upgrade` and `BearerToken` are already handled.
`Host` and possibly other optional fields look like they have the same shape of
problem; `Host` happens to be harmless in practice because virtually every client
sends it on every request, but you may want to audit that list.
Disclosure: I used an AI assistant to help investigate this and to draft the
report. The diagnosis came from instrumenting our own server and client and
reading the mORMot sources; every number quoted above is from an actual run on
our deployment, not from the model.
Although, it can create and register custom authenticator but still wondering is it possible to keep NTLM auth in framework for local testing purpose.
Hi AB,
There seems have a one line code in TRestServerAuthenticationSspi.Auth need to adjust,
could you help to confirm it? Thank you.
// Current git version
result := AuthSessionRelease(Ctxt, aUserName);
if result or
(aUserName = '') or
not Ctxt.InputExists['Data']) then
exit;
//Adjusted version
result := AuthSessionRelease(Ctxt, aUserName);
if result or (
(aUserName = '') and
(not Ctxt.InputExists['Data'])) then
exit;JienYuan
I have two external SQL Table. Everything works fine when I use ORM add/update directly.
However, when I try using Batch to control transaction, exception occurs after BatchSend.
I am wondering whether is Batch control suitable for external tables?
Below is the code and log capture. Any idea is appreciated.
Thank you.
var
aUId: RawUtf8;
aBatch: TRestBatch;
aLog: TOrmMyLog;
aUser: TOrmUTbl;
begin
aUId:= TSQLMyAuthUser(Session.User).UId;
aBatch:=TRestBatch.Create(Sender.Orm,nil,10000);
aLog:=TOrmMyLog.New(Ctxt,'SESSION');
with aLog do begin
DATA_1:='True';
Data_2:='STANDARD';
end;
//Sender.Add(aLog,True);
aBatch.Add(aLog,True);
aUser:=TOrmUTbl.Create;
Sender.Retrieve(csFilter,[],[aUId],aUser);
with aUser do
begin
Updated:=20240604;
Invalidates:=0;
LoggedIn:=True;
LastLogin:=Now;
end;
//Sender.Update(aUser,aFields);
aBatch.Update(aUser,aFields);
Sender.BatchSend(aBatch);
end;2024/6/4 11:34:53.624 Trace MyRestServer.TMyRestServerDB(04fff520) BatchSend {"TRestBatch(04bd3de0)":{Count:2,SizeBytes:212}}
2024/6/4 11:34:53.624 Enter mormot.orm.server.TRestOrmServerBatchSend(04b647e0).EngineBatchSend inlen=212
2024/6/4 11:34:53.624 Enter mormot.db.sql.zeos.TSqlDBZeosConnection(04be4a10).StartTransaction
2024/6/4 11:34:53.624 Leave 00.001.169
2024/6/4 11:34:53.624 SQL mormot.db.sql.zeos.TSqlDBZeosStatement(0509b080) Execute t=1.99ms wr=1 q=insert into dbo.MyLOG (ID,LOGTIME,OPERATOR,ADDR,LOGACTION,DATA_1,DATA_2,MACHINEID,HOSTNAME) values ([32...],['2024-06-04T19:34:47.267'...],['USR'...],['127.0.0.1'...],['SESSION'...],['True'...],['STANDARD'...],['0'...],[''...])
2024/6/4 11:34:53.624 Cache mormot.db.raw.sqlite3.TSqlDatabase(04be4668) cache flushed
2024/6/4 11:34:53.624 DB mormot.db.sql.zeos.TSqlDBZeosStatement(0509b350) Prepare t=530us q=update dbo.UTBL set UPDATED=?,LOGGEDIN=?,INVALIDATES=?,LASTLOGIN=? where ID=?
2024/6/4 11:34:53.624 SQL mormot.db.sql.zeos.TSqlDBZeosStatement(0509b350) Execute t=5.87ms wr=1 q=update dbo.UTBL set UPDATED=['20240604'...],LOGGEDIN=[1...],INVALIDATES=[0...],LASTLOGIN=['2024-06-04T19:34:47'...] where ID=[2...]
2024/6/4 11:34:53.768 Exception ESqlDBException {Message:"Unexpected TSqlDBZeosConnectionProperties.SharedTransaction(1,2)",Statement:null} [R0:MyServices] at 6852ff mormot.core.text.pas ESynException.RaiseUtf8 (9240)
2024/6/4 11:34:59.880 Exception ESqlDBException {Message:"Unexpected TSqlDBZeosConnectionProperties.SharedTransaction(1,3)",Statement:null} [R0:MyServices] at 6852ff mormot.core.text.pas ESynException.RaiseUtf8 (9240) Hi Thomas,
Thanks for the promptly reply.
I did the more testing and noticed my code is working fine when not used WebSocketsEnable() function.
I activated WebSocketsEnable() and noticed there will have logs in below:
EXC ENetSock {LastError:"nrFatalError",Message:"THttpClientSocket.SockSendFlush(localhost) len=539 [Fatal Error - #6]"} [TRemoteLog LogService] at 637c26
EXC ESynLogException {Message:"Missing TSynLog.DisableRotemoteLog(true)"} [R1:root] at 59826e
trace mormot.net.ws.async.TWebSocketAsyncProcess(025bd5c0) ProcessStop: callbacks
debug mormot.net.ws.async.TWebSocketAsyncProcess(025bd5c0) ProcessStop {"TWebSocketProtocolBinary(025a7530)":{Name:"synopsebin",URI:"root",RemoteIP:"172.17.112.1",UpgradeUri:"root",Encrypted:true,FramesInCount:1,FramesInBytes:33,Options:["pboSynLzCompress"],FramesInBytesSocket:66,FramesInCompression:-100,FramesOutCompression:100}}
trace mormot.net.ws.async.TWebSocketAsyncProcess(025bd5c0) SendFrame 172.17.112.1 * focConnectionClose len=0
warn mormot.net.ws.async.TWebSocketAsyncProcess(025bd5c0) Destroy: no focConnectionClose SendFrame
However, this is too far for me.
Best Regards,
JienYuan
After I added
TRestHttpClient.CreateForRemoteLogging('localhost',TSynLog,8091) in restws_chatserver program;
with TSynlog.Family do
....
WebSocketLog := TSynLog; // verbose log of all WebSockets activity
TRestHttpClient.CreateForRemoteLogging('localhost',TSynLog,8091);
try
...
end;to redirect logs to Logview in rest-websockets sample.
The Client.ServerTimeStampSynchronize will hang and raised the exception in Logview.
[bold]EXC ESynLogException {Message:"Missing TSynLog.DisableRotemoteLog(true)"} [R1:root] at 59826e [/bold]
Have any idea? Thank you.
Hi AB,
Thanks for the reply.
I do read the thread and topic in the document.
But it noticed
If your application is developped on any older revision (e.g. Delphi 7, Delphi 2007 or Delphi 2009), you won't be able to automatically serialize records as plain JSON objects directly.
You have several paths available:
By default, the record will be serialized as binary, and encoded as Base64 text
So, I was wondering might Delphi 7 record will be serialized as binary, and encoded as Base64 text then pass to client.
Now, I see it.
Thanks again.
I tried Record type Dynamic Array service result in Delphi 7.
However, the result lost information after 50th item in Delphi 7 but seems correct in Berlin.
Is it a limitation in mORMot 1.18 & Delphi 7 ?
The code is:
Type
TCodeValue=packed record
Code: string;
Value: string;
end;
TCodeValueArray = array of TCodeValue;
IServiceClient = Interface(IInvokable)
['{E1779C4E-C7E3-4F83-BF94-91E907774384}']
function getLis: TCodeValueArray ;
end;
TServiceClient = class(TInjectableObjectRest,IServiceClient)
public
function getList: TCodeValueArray ;
end;
function TServiceClient.getList: TCodeValueArray;
var
x: integer;
begin
SetLength(result,100);
for x:=Low(result) to high(result) do begin
result[x].Code:=Format('User%0:.3d',[x]);
result[x].Value:=Format('Name%0:.3d',[x]);
end;
end;
test:
var
d: TCodeValueArray;
x: integer;
begin
d:=c.getList;
for x:=Low(d) to High(d) do
writeln(d[x].code+'|'+d[x].value);
end;
I only got this in Delphi 7.
User000|Name000 (first)
.
.
User049|Name049
|
|
.
.
| (100th)
Pages: 1