You are not logged in.
I have no way to run the reverse proxy on a separate server, I only have one Windows server.
Arnaud, do you think this error is due to someone "illegally" connecting to this websocket port from the internet?
I have a server based on mORMot that has been running on a VPS at OVH for several years. I am now moving it to another provider, Contabo.
The server is using websockets to internal communicate with the control GUI which is on the same machine (it connects locally), the websocket server is running on port 9000.
At the moment my server is tested in Contabo and I see messages in the log that were never there in OVH:
EXC ECrtSocket ("SockRecvLn after 19 chars [10053 WSAECONNABORTED An established connection was
aborted by the software in your host machine]") [HttpSvr 9000/logservice PoolWork]
at c07dc4 SynCrtSock.TCrtSocket.SockRecvLn (5599) stack trace API
OVH is running Windows Server 2012, Contabo is running Server 2016.
What could be the cause and what can I do about it?
ps. I use the latest version of mORMot from the repository and Delphi 10.3
JSON and access by Variant (TDocVariant)
In the case of an array, we can use .Count() and .Values[] methods to get to individual items.
I have an JSON with object structure like this (but I don't have object declaration):
{
"ONE": {
"ABC": {
"somethinA": {
"value1": "30313233343536373839303132333435"
},
"SomethingB": {
"value1": "303132333435363738393031323334353637383930313233"
},
"SomethingC": {
"value4": "abcdefghijklmnopqrstuv0123456789"
}
},
"DEF": {
"SomethingD": {
"value1": "3E94C6E319ABBB47403FACD6CA68A5AB36D32E6DDA6C58BF6A019F45E90DCCA0"
},
"SomethingE": {
"value1": "303132333435363738393031323334353637383930313233"
},
"SomethingC": {
"value1": "3CD2321C665DB1CA51014D65CC9D9AF46845010AD80018B1E256374558D79DB1"
}
},
"GHI": {
[...]
}
},
"TWO": {
[...]
},
[...]
Is it possible to get what properties are available and iterate over them?
Hi Arnaud,
a small challenge for the mORMot ;-)
https://www.techempower.com/benchmarks/ … ph&test=db
and:
https://github.com/TechEmpower/Framewor … s-Overview
quote:
"We invite fans of frameworks, and especially authors or maintainers of frameworks, to join us in expanding the coverage of this project by implementing tests and contributing to the GitHub repository. The following are specifications for each of the test types we have included to-date in this project. Do not be alarmed; the specifications read quite verbose, but that's because they are specifications. The implementations tend to be quite easy in practice."
I use simply code to decode memory address to method name:
s := TSynMapFile.FromCurrentExecutable.FindLocation(lMethodPtr);
Decoding works a little bit wrong, the given example returns this result:
Client.AccountsManager.Impl.AccountsManager.Impl.TAccountsManagerClient.GetAnyAccountLicenses (218)
while correctly it should be so:
Client.AccountsManager.Impl.TAccountsManagerClient.GetAnyAccountLicenses (218)
the underlined fragment of text is inserted twice, because unit is: Client.AccountsManager.Impl.pas and class name is TAccountsManagerClient
I use Delphi 10.3 and latest mORMot sources.
Arnaud, I prepare very simple example (dpr with 65 lines), please check this code and callstack readed from OnBeforeException event level:
I get this result (first part is result od event, second part is standard SynLog catching exsception log):
--- mormotLogOnBeforeException -------- begin --------
Exception class: Exception
line #0: onBeforeExceptionTest.InsideProc (38)
--- mormotLogOnBeforeException --------- end ----------
20200623 16272444 EXC Exception ("test exception from inside procedure") [] at
54cc89 onBeforeExceptionTest.InsideProc (38) stack trace API 54cc89 onBeforeEx
ceptionTest.InsideProc (38) 54ccec onBeforeExceptionTest.OutsideProc (43) 54ccf8
onBeforeExceptionTest.VeryOutsideProc (48) 54ce24 onBeforeExceptionTest.Main (5
8)
Why in OnBeforeException (where I use new introduced TSynMapFile.FindStackTrace() method) I can't get full stack?
Stack frames are active of course.
When I set result to True in OnBeforeException (so, SynLog catches and log an exception) then I get in log all data from call stack:
20200617 18310722 EXC EPathNotFoundException ("some example exception") [] at
6629ca Main.TfrmMain.btnMakeExceptionClick (96) stack trace API
6629ca Main.TfrmMain.btnMakeExceptionClick (96)
544d65 Vcl.Controls.TControl.Click (7537)
5492c0 Vcl.Controls.TWinControl.WndProc (10280)
55dec5 Vcl.StdCtrls.TButtonControl.WndProc (5308)
[...]
Thanks Arnaud!
I still have a problem with the callstack, with the OnBeforeException event (from TSynLogFamily), it is almost empty, there is only a line calling the exception. The problem is probably not in the callstack extraction itself, because the EStackCount value of the aExceptionContext object is zero.
See:
Arnaud, I know you have been very busy lately with changes in SQLite, but please read my above posts and think about making available in the interface section (public) of the methods to get/convert call stack... please :-)
Are you suggesting that I should read and parse the whole log to get a potential exception? Really?
I need to catch an exception, decode the call stack and send it somewhere... what you're recommending is a lot of over-the-top ideas.
I want SynLog (or other part of mormot) to let me convert the callstack (from OnBeforeException level) to thong with its internal tricks.
So, I just want Arnaud to refactor the code a bit and give external access to callstack conversion (using a private TSynMapFile instance)
@EMartin
Once again... I need to get and pass a call stack to my code as a text/string, how does your example solve my problem?
I don't want to avoid TSynMapFile, I want to avoid having to duplicate code with SynLog (and problems with two TSynMapFile instances!). I just want the SynLog module to have a public function to read the callstack. This function could be used by TSynLog and external code (as I need it now).
Besides, TSynMapFile class should have one global instance (it is currently private).
A good solution would be to add a static class function to the TSynMapFile class, eg.:
class function TSynMapFile.GetCallStackAsStr(const CallStack: PPtrUInt) : TStrings;
or global function in a module, like this:
function GetCallStackAsStr(const CallStack: PPtrUInt) : TStrings;
This functionality would GREATLY extend the usability of SynLog! Even without a rest of mORMot!
This is a very complex code, and also uses non-public methods and values such as TSynMapFile:
unit SynLog;
[...]
implementation
[...]
var
ExeInstanceMapFile: TSynMapFile;
function GetInstanceMapFile: TSynMapFile;
begin
[...]
end;
Should I duplicate this code and these objects?
I would have to use the only available class method that forces a TSynMapFile instance in the background:
class procedure TSynMapFile.Log(W: TTextWriter; aAddressAbsolute: PtrUInt; AllowNotCodeAddr: boolean);
...but this method requires TTextWriter, which I don't want to use in this case.
Wouldn't it be possible to make new, public method converting the callstack into text form in mORMot itself?
Please consider this solution.
I use OnBeforeException event:
TSynLog.Family.OnBeforeException := mormotLogOnBeforeException;
How to get call stack in this event method? Is possible?
How to force a creating MAB file immediately after execute program?
Sometimes I have an impression that it is rather the mORMot that sponsors Delphi and it is Delphi that is mORMot powered ;-)
I try, but without sucess... too many commits :-)
I debug source, in SynCRTSock.pas method RetrieveHeaders() fill RemoteIP param as ermpty string when IP = localhost:
Is this the intended change or should it return '127.0.0.1' or '::1'?
A few days ago I updated mORMot to the latest version and it turned out that the read client's IP stopped working. I'm using interface-based services.
The code I was using to read the client's IP:
sIP := FindIniNameValue(Pointer(ServiceContext.Request.Call.InHead), 'REMOTEIP: ');
or
sIP := FindNameValue(Pointer(ServiceContext.Request.Call.InHead), HEADER_REMOTEIP_UPPER);
Now sIP is empty. How repair this behaviour?
Maybe uncompressed size is coded in first bytes of file? Try this.
I found error in OnBeforeBody event that results from using WITH and the name of a local variable such as object field.
The RemoteIP parameter is passed to the OnBeforeBody event as an empty string (this is the value of Context.RemoteIP field) even though the local variable contains the correct value.
procedure THttpApiServer.Execute;
[...]
var
[...]
RemoteIP: SockString;
Context : THttpServerRequest;
[...]
[...]
begin
[...]
if Assigned(OnBeforeBody) then begin
with Context do <-------------------- this 'with' cause a problem
Err := OnBeforeBody(URL,Method,InHeaders,InContentType,RemoteIP,InContentLength,fUseSSL); <--- RemoteIP is taken from Context object, not from local variable
THttpServerRequest = class
[...]
public
[...]
property RemoteIP: SockString read fRemoteIP write fRemoteIP;
end;
I wrote simple test procedure:
uses PasZip;
procedure Test;
var
i: Integer;
j: Integer;
InBuffer : array of AnsiChar;
OutBuffer : array of AnsiChar;
OutDecompress : array of AnsiChar;
OutSize: Integer;
begin
// init data and structures
SetLength(InBuffer, 1000);
SetLength(OutBuffer, 1000);
SetLength(OutDecompress, 1000);
//fill InBuffer = ('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'A', 'B', 'C', ....etc.)
for i := 0 to 99 do for j := 0 to 9 do InBuffer[i * 10 + j] := AnsiChar(65 + j);
//compress InBuffer => OutBuffer
OutSize := CompressMem(@InBuffer[0], @OutBuffer[0], Length(InBuffer), 1000); // OutSize after compress has value 23
//decompress OutBuffer => OutDecompress
OutSize := UnCompressMem(@OutBuffer[0], @OutDecompress[0], OutSize, 1000)
end;
After last line OutSize has value 1000 (properly) BUT OutDecompress buffer is filled by zeroes!!! not by the original values ABCD...
What i'm doing wrong?
This result look like a base64 coded value, but after decode the value is still not correct.
About enums serialization as string, it is not possible yet for method parameter - but it is for record serialization IIRC.
@ab, yes, for record serialization enums can be serialized as string, but please consider adding options to the server/service configuration so that the parameters of the enumerated type (including complex types such as records or arrays) are serialized to a string
I try, like you suggest, enumerated type with assigned integer value:
TTest = (ttOne = 1, ttTen = 10, ttHund = 100);
then I set two field to ttTen and ttHund value, now I get strange values in transport JSON (coded by mORMot):
{"Test":"0A00000000000000"},{"Test":"6400000000000000"}
So this way is blind :-)
Thanks for replies...
I don't think that the transfer of parameters of the enumerated type is noticeably slower, although it is to be tested.
But I personally see three advantages of this enumeration type encoding:
- makes debugging services easier
- makes it easier to connect foreign clients to the server (in other languages, e.g. JS)
- eliminates the problem of extending the type in the future by new values (then e.g. ftTable = 5 soon becomes ftTable = 10 and all client code needs to be corrected)
@Vitaly for enumerated types, see Spring4D and TEnum type :-)
I use interface based service. The methods has parameters in the form of an array of records, some of the record fields are an enumerated type. When the client communicates with the server, the call parameters are converted to JSON, and the enumerated type are converted/casted to Int32.
Is it possible to set the server or service so that the parameters of the enumerated types are converted to a text string?
So e.g. instead of ftString => 1 to be ftString => 'ftString'
Ehhh, I forgot that TDynArray is just a wrapper! Thx!
types:
type
TFieldData = record
FieldName : UTF8String;
FieldType : TFieldType;
FieldValue : Variant;
end;
TFieldDataArray = TArray<TFieldData>;
code:
var
s: RawUTF8;
Tab : TFieldDataArray;
DynArray : TDynArray;
OtherDynArray: TDynArray;
begin
//prepare array
SetLength(Tab, 2);
//prepare records
Tab[0].FieldName := 'name';
Tab[0].FieldType := ftString;
Tab[0].FieldValue := 'igor';
Tab[1].FieldName := 'count';
Tab[1].FieldType := ftInteger;
Tab[1].FieldValue := 15;
// prepare DynArray
DynArray.Init(TypeInfo(TArray<TFieldData>), Tab);
// serialize array of records to JSON
s := DynArray.SaveToJSON(False); // s = [{"FieldName":"name","FieldType":1,"FieldValue":"igor"},{"FieldName":"count","FieldType":3,"FieldValue":15}]
//trying to deserialize and AV
OtherDynArray.LoadFromJSON(pointer(s)); <======== exception
I get exception:
Access violation at address 00697DF3 in module 'demo.exe''. Write of address 75174F70
How do properly deserialize JSON to array od records?
Is possible to extract method input parameters from ServiceContext on service method call level?
I try ServiceContext.Request.Parameters and ServiceContext.Request.Call.InBody but I get binary or incomplete data.
Try custom record serialization:
SynCommons.TTextWriter.RegisterCustomJSONSerializer(TypeInfo(TSomeRecordType), TSomeRecordCustomWriter.CustomReader, TSomeRecordCustomWriter.CustomWriter);
Please try https://synopse.info/files/html/Synopse … l#TITL_204
And ensure you re-load the documentation page in your browser to get its latest content.
Thanks, this link works properly, previous not.
@ab
the link does not scroll the page to the appropriate chapter of the documentation, is it only happening for me?
Yes, I'm also waiting for the official repo.
Arnaud, please review and merge these changes.
You are right! Thx macfly :-)
I found in SynCommons function:
function UnixMSTimePeriodToString(const UnixTime: TUnixTime; FirstTimeChar: AnsiChar='T'): RawUTF8;
The first parameter of this function should be of TUnixMSTime type and not of TUnixTime.
I know that these types are internally compatible, but in the future this may change and there will be trouble.
I know, but I do it consciously.
What you have proposed is a full date and time from 3 hours ago, e.g. 2019-06-18 13:30
And what I mean is only 3 hours, a time segment (period/interval), not a point on the timeline.
For example, total working time in the last month.
How do works UnixTimePeriodToString() and UnixMSTimePeriodToString() functions from SynCommons unit?
I need convert some interval (period) of time to string. I try these functions, but I get strange results:
var
u : TUnixTime;
ums : TUnixMSTime;
dt: TDateTime;
s: string;
begin
dt := Now() - IncHour(Now(), -3); // three hours period, I mean
ums := DateTimeToUnixMSTime(dt);
s := UnixMSTimePeriodToString(ums, ' ');
Writeln('UnixMSTimePeriodToString: ' + s);
u := DateTimeToUnixTime(dt);
s := UnixTimePeriodToString(u, ' ');
Writeln('UnixTimePeriodToString: ' + s);
and I get:
UnixMSTimePeriodToString: 21:00:00.000
UnixTimePeriodToString: 21:00:00
why 21 not 3 h?
Very, very big thanks for your effort and time pvn0!
Now all is right! :-D
I create simple example apps based od Example 31 from mORMot demos.
See screencast how the second close the connection does not result in an OnWebSocketsClosed event on client side:
client source: https://pastebin.com/4teDqTJG
server source: https://pastebin.com/QKLFUKZF
My event OnWebSocketsClosed looks like this:
procedure TRemoteLogClientWebsockets.OnWebSocketsClosed(sender: TObject);
begin
TThread.Queue(nil, procedure
begin
fService.StartLogSession(fCallback);
end);
end;
After brute-force close connection (by using this tool: https://docs.microsoft.com/en-us/sysint … s/tcpview) first time this event is called and websocket is connected again, but after second disconnection event OnWebSocketsClosed event is never called again.
I got the next problem.
If after unexpected closing the websocket connection I will resume this connection again (start websocket from client side) then next connection break (simulated by TCPView -> Close connection) no longer triggers the OnWebSocketsClosed event.
my custom record is:
TIndicatorExport = record
Index : Integer;
Value : Int64;
// s : string; <================= this field changes the behavior
end;
custom writer is:
class procedure TIndicatorExportCustomWriter.CustomWriter(const aWriter: SynCommons.TTextWriter; const aValue);
var
V: TIndicatorExport absolute aValue;
begin
aWriter.AddJSONEscape(['I', V.Index, 'V', V.Value]);
end;
I register my custom writer by:
SynCommons.TTextWriter.RegisterCustomJSONSerializer(TypeInfo(TIndicatorExport),
nil,
TIndicatorExportCustomWriter.CustomWriter);
And I try use of it:
DynArray: TDynArray;
Data: TArray<TIndicatorExport>;
[..]
DynArray.Init(TypeInfo(TArray<TIndicatorExport>), Data);
Str := DynArray.SaveToJSON(False);
after executed last line the Str has '["00000000000000010000000000000000"]' string,
not my customized, because my custom writer is not called.
BUT... when I add additional field (string) to record then custom serialization works properly!
Where is my mistake?
Correct the order of these lines:
finally
ffactory.free; //first
ffactory := nil; //second
end;
I can't ignore events, because the problem is that the event method is part of the freed server object and at the moment of calling the object is already released which ends with a series of AV...
For now, I did a workaround, I additionally support the OnClientConnected() event and in it I remember the "self" object, which is TWebSocketProcess, thanks to which it later has access to its settings. But it is ugly :-|
But I will wait for a better solution and thanks for the comments :-)
It partially works, but there are further problems.
On the server side, I assign the OnClientDisconnected event:
procedure TWSLogServer.StartRemoteLogService();
begin
[...]
fHTTPServer := TSQLHttpServer.Create(IntToStr(fHTTPPort), [fRestServer], '+', useBidirSocket);
fWSServer := fHTTPServer.WebSocketsEnable(fRestServer, cWebSocketEncryptionKey);
fWSServer.Settings.OnClientDisconnected := OnClientDisconnected; <========== event assigned
end;
When I release the server, I want to release the event, so I execute the code:
procedure TWSLogServer.StopRemoteLogService();
begin
if Assigned(fHTTPServer) then
begin
fWSServer.Settings.OnClientDisconnected := nil; <======== event released, but only on TWebSocketServer class side!
FreeAndNil(fHTTPServer);
FreeAndNil(fRestServer);
end;
end;
However, the event is still called!
Code review and debugging led me to the conclusion that there is an error in the settings, because they are saved in the record:
/// parameters to be used for WebSockets process
TWebSocketProcessSettings = record
[...]
And in working classes there are separate instances of those records that have copies of settings data, so that changing Settings in the TWebSocketServer class instance does not affect the TWebSocketProcess class instance.
TWebSocketServer = class(THttpServer)
protected
fSettings: TWebSocketProcessSettings; <==== first settings
TWebSocketProcess = class(TSynPersistent)
protected
fSettings: TWebSocketProcessSettings; <===== next settings
How do I release a OnClientDisconnected event from a TWebSocketProcess?
Maybe worth change TWebSocketProcessSettings from record to class?
Thanks pvn0 and Arnaud! I will test it today...
In the code that I showed in the example, there are two events:
procedure TLogClientWebsockets.OnClientDisconnected(sender: TObject);
begin
// disconnect event handling is never called!!
end;
procedure TLogClientWebsockets.OnWebSocketsClosed(sender: TObject);
begin
// disconnect event handling is never called!!
end;
None of these events is called, so I ask what I am doing wrong or how to do it correctly?
So there is no way to know from the client side (callback implementation side) that the connection has died?
My problem is protection against network errors and the possibility of resuming the connection. How to do it?
Based on example 31, I wrote a simple service for uploading logs using websockets.
However, I would like the client side (which implements the callback class) to receive an event in the event of a lost connection so that it can re-connect to the server and pass the callback interface.
I made this code for client:
procedure TLogClientWebsockets.OnClientDisconnected(sender: TObject);
begin
// disconnect event handling is never called!!
end;
procedure TLogClientWebsockets.OnWebSocketsClosed(sender: TObject);
begin
// disconnect event handling is never called!!
end;
procedure TLogClientWebsockets.Start();
begin
fModel := TSQLModel.Create([], cRemoteLogRootName);
fRestClient := TSQLHttpClientWebsockets.Create('127.0.0.1', cRemoteLogHTTPPort.ToString, fModel);
fRestClient.Model.Owner := fRestClient;
fRestClient.OnWebSocketsClosed := OnWebSocketsClosed;
fRestClient.WebSockets.OnWebSocketsClosed := OnWebSocketsClosed;
fRestClient.WebSockets.Settings.OnClientDisconnected := OnClientDisconnected;
fRestClient.WebSockets.Settings.HeartbeatDelay := 3000;
fRestClient.WebSockets.Settings.DisconnectAfterInvalidHeartbeatCount := 3;
fRestClient.WebSocketLoopDelay := 100;
fRestClient.WebSocketsUpgrade(cWebSocketEncryptionKey);
fRestClient.ServiceDefine([ILogService], sicShared);
if not fRestClient.Services.Resolve(ILogService, fService) then
raise EServiceException.Create('Service ILogService unavailable');
fLogManager := TLogManager.Create(TPanel(fLogPanel), 0, nil);
fCallback := TRemoteLogCallback.Create(fLogManager, fRestClient, IRemoteLogCallback, fLogPanel.Handle);
fService.StartLogService(fCallback);
end;
Then I run the application and during transmission using TCPView (from Sysinternals) I close the socket that supports the websockets connection. On the server side, the onClientDisconnect event fires. But nothing is happening on the client's side!
How to detect connection loss from the client side?