You are not logged in.
Pages: 1
@franfj
wepoll demo?
How to use it
Sorry, I don't have any full demo now.
I don't remember very well, but it should be easy to replicate the code from other poll classes (have a look at wsapoll or select) and implement the wepoll.
Anyway, the performance of the select implementation worked well enough for my requirements, so I left away this approach.
Are you suggesting that I should read and parse the whole log to get a potential exception? Really?
I need to catch an exception, decode the call stack and send it somewhere... what you're recommending is a lot of over-the-top ideas.
I want SynLog (or other part of mormot) to let me convert the callstack (from OnBeforeException level) to thong with its internal tricks.
So, I just want Arnaud to refactor the code a bit and give external access to callstack conversion (using a private TSynMapFile instance)
You don't need to parse the whole log, just handle the levels that report exceptions (sllExceptionOS and sllException).
If you need another implementation, is not hard to extract all internal exception stack processing from
// this is the main entry point for all intercepted exceptions
procedure SynLogException(const Ctxt: TSynLogExceptionContext);
Anyway, since SynLog does not get callstack for delphi win64 I used JCLDebug (give it a look, although it has another disadvantages).
The simplest solution is using the TSynLogFamiliy.EchoCustom:
/// can be set to a callback which will be called for each log line
// - could be used with a third-party logging system
// - EchoToConsole or EchoCustom can be activated separately
// - you may even disable the integrated file output, via NoFile := true
property EchoCustom: TOnTextWriterEcho read fEchoCustom write SetEchoCustom;
The EchoCustom receives the Level and the formated text, so you could use that text to do whatever you want.
Thanks!
Hi,
I use the revision number on SynopseCommit.inc to track the version I'm using. Today I have tried to update, but found that the latest revision is smaller than the one I have.
The version that git reports today is "1.18.5514", which seems wrong.
The problem origin seems this commit https://synopse.info/fossil/info/c53e884c8efe2a1c and https://github.com/synopse/mORMot/commi … 0b1402f9bb
Modified SynopseCommit.inc from [f06672080f] to [3a76a71004].
1 '1.18.5592' | 1 '1.18.5392'
Could you check this?
Thanks!
Yes, as this library uses internally IOCP it seems an easy way to add that functionality into windows.
I know that sample, but unfortunately on my configuration it does not give consistent results. I've done some tests and results sometimes seems 3x faster than Select, others it shows near the same performance... but even two consecutive tests using Select shows a lot of variation on times, so I can't trust the results.
Maybe it will be interesting to improve the performance of the servers on windows...
https://github.com/piscisaureus/wepoll
This library implements the epoll API for Windows applications. It is fast and scalable, and it closely resembles the API and behavior of Linux' epoll.
Rationale
Unlike Linux, OS X, and many other operating systems, Windows doesn't have a good API for receiving socket state notifications. It only supports the select and WSAPoll APIs, but they don't scale and suffer from other issues.Using I/O completion ports isn't always practical when software is designed to be cross-platform. Wepoll offers an alternative that is much closer to a drop-in replacement for software that was designed to run on Linux.
Compiling as a dll and using it is easy, just a copy&paste the types definition from SynFPCSock.pas, some small modifications, and load the functions from the external dll.
unit wepolldll;
interface
const
// associated file is available for read operations
EPOLLIN = $01;
// urgent data available for read operations
EPOLLPRI = $02;
// associated file is available for write operations
EPOLLOUT = $04;
// error condition happened on the associated file descriptor
EPOLLERR = $08;
// hang up happened on the associated file descriptor
EPOLLHUP = $10;
// sets the One-Shot behaviour for the associated file descriptor
// - i.e. after an event is pulled out, the file descriptor is disabled
EPOLLONESHOT = $40000000;
// sets the Edge-Triggered (ET) behaviour for the associated file descriptor
EPOLLET = $80000000;
EPOLL_CTL_ADD = 1;
EPOLL_CTL_DEL = 2;
EPOLL_CTL_MOD = 3;
type
/// application-level data structure for epoll
TEPollData = record
case integer of
0: (ptr: pointer);
1: (fd: integer);
2: (u32: cardinal);
3: (u64: Int64);
end;
PEPollData = ^TEPollData;
/// epoll descriptor data structure
TEPollEvent = packed record
events: cardinal;
__padding: cardinal; ///////// don't know why, but data returned by epoll_wait is aligned to next 4bytes...
data: TEpollData;
end;
PEPollEvent = ^TEPollEvent;
TEPollEventDynArray = array of TEPollEvent;
function epoll_create(size: integer): integer; cdecl; external 'wepoll.dll';
function epoll_close(epfd: integer): Integer; cdecl; external 'wepoll.dll';
function epoll_ctl(epfd, op, fd: integer; event: PEPollEvent): Integer; cdecl; external 'wepoll.dll';
function epoll_wait(epfd: THandle; events: PEPollEvent; maxevents, timeout: integer): Integer; cdecl; external 'wepoll.dll';
implementation
end.
Static linking will need more work, but should be possible.
I haven't done extensive performance or stability tests, but seems to work fine...
Thanks @edwinsn but I think that does not match what I'm looking for, because I wanted to use my own logging to log the output of some Synopse mORMot classes (at the moment TAsynchServer from SynBidirSock, but could be anything in the future), so I can't change the way that current implementation works (they expect a TSynLogClass and use the .Add method).
Anyway thanks for the code, the linked https://synopse.info/fossil/info/d3f5197f3c shows that it's a "limitation" of the design pattern as I supposed.
Hi ab,
Is there any way to create a SynLog for each instance of a class. I've one class that is used on several parts of the application, and I want to be able to log each one on a different file.
One way of doing it will be creating a TSynLog descendant for each instance.
unit MyServer;
TMyAsynchServer = class (TAsyncServer)
...
constructor Create(..., aLog: TSynLogClass);
And on the application that uses the unit:
unit MyMain;
TMyLog1 = class (TSynLog)
end;
TMyLog2 = class (TSynLog)
end;
...
TMyLogN = class (TSynLog)
end;
...
with TMyLog1.Family do
begin
// I want to log on my custom function
EchoCustom := MySynLogTrace1;
NoFile := True;
end;
FMyServer1 := TMyAsyncServer.Create(.... , TMyLog1);
with TMyLog2.Family do
begin
// I want to log on my custom function
EchoCustom := MySynLogTrace2;
NoFile := True;
end;
FMyServer2 := TMyAsyncServer.Create(.... , TMyLog2);
...
But I'm looking for a way to do it within the unit that defines the component (TMyAsynchServer), to be able to define the "EchoCustom" functions in the same class/unit, as it will echo to a custom logger that is already created by the application.
unit MyServer;
TMyAsynchServer = class (TAsyncServer)
...
constructor Create(...); // no need to define a TSynLogClass since is managed inside
function MySynLogEcho(Sender: TTextWriter; Level: TSynLogInfo;
const Text: RawUTF8): boolean;
...
constructor TMyAsynchServer.Create(....)
begin
// I need a way to create here a TSynLogClass per instance, not shared between other instances of TMyAsychServer
with XXXX.Family do // ! If I use one TSynLog class, all the instances point to the latest instance MySynLogEcho fucntion
begin
// I want to log on my custom function
EchoCustom := MySynLogEcho;
NoFile := True;
end;
inherited Create(..., MySynLogClassWithEcho, ...)
end;
I'm not sure if I've explained it well... it's a bit complex, maybe is not designed to work this way and is not possible, hope that could be understood
Thanks!
I'm developing a server which has the singularity that must close all the connections (the clients stay connected until the server closes the connection, it's an Asterisk FastAGI server).
If I close a connection using "ConnectionRemove" I get random errors (AV, memory corruption, etc.) and analyzing the code, it seems that the "lock" performed on the connection only locks "connections" array but not the internal connection that is going to be removed:
function TAsynchConnections.ConnectionRemove(aHandle: TAsynchConnectionHandle): boolean;
var i: integer;
conn: TAsynchConnection;
begin
result := false;
if (self=nil) or Terminated or (aHandle<=0) then
exit;
conn := ConnectionFindLocked(aHandle,@i);
if conn<>nil then
try
if not fClients.Stop(conn) then
fLog.Add.Log(sllDebug,'ConnectionRemove: Stop=false for %',[conn],self);
result := ConnectionDelete(conn,i);
finally
fConnectionLock.UnLock;
end;
if not result then
fLog.Add.Log(sllTrace,'ConnectionRemove(%)=false',[aHandle],self);
end;
Since the connection may be processing by the "pseRead" thread, the ConnectionDelete call will destroy it, and then the "Read" thread may cause AccessViolations.
Having that in mind, I tried adding a "slot.Lock" to avoid destroying the connection while is being read:
function TAsynchConnections.ConnectionRemove(aHandle: TAsynchConnectionHandle): boolean;
var i: integer;
conn: TAsynchConnection;
locked: boolean;
begin
result := false;
if (self=nil) or Terminated or (aHandle<=0) then
exit;
conn := ConnectionFindLocked(aHandle,@i);
if conn<>nil then
try
locked := conn.fSlot.TryLock(100);
if not locked then begin
fLog.Add.Log(sllDebug,'ConnectionRemove: fSlot.TryLock=false for %',[conn],self);
result := false;
Exit;
end;
if not fClients.Stop(conn) then
fLog.Add.Log(sllDebug,'ConnectionRemove: Stop=false for %',[conn],self);
result := ConnectionDelete(conn,i);
finally
fConnectionLock.UnLock;
if locked then
conn.fSlot.UnLock;
end;
if not result then
fLog.Add.Log(sllTrace,'ConnectionRemove(%)=false',[aHandle],self);
end;
But this has two disadvantages:
* It breaks the ability to perform "ConnectionRemove" inside "OnRead" thread, because connection "lock" is not a mutex/criticalsection, just an Integer which increments on each Lock.
* It still gives random Access Violations and Memory corruption errors (which I haven't found why).
The final solution I've applied is simple, just use a Boolean value inside connection to indicate that I want to close that connection, and let "OnRead" close the connection returning "sorClose".
TMyConnection=class(TAsynchConnection)
private
FDisconnectOnNextRead: boolean;
protected
function OnRead(Sender: TAsynchConnections): TPollAsynchSocketOnRead; override;
...
end;
---------
procedure TMyConnection.Disconnect;
begin
FDisconnectOnNextRead := True;
// I usually send some message to the client here, that causes a OnRead later, then it gets disconnected
end;
function TMyConnection.OnRead(Sender: TAsynchConnections):TPollAsynchSocketOnRead;
begin
if FDisconnectOnNextRead then
begin
result := sorClose; // could be ConnectionRemove, but it's easier to do this way
Exit;
end;
// do the Read then return sorContinue
end;
It's not a "nice" way to do, but it's simple, and works.
Having that in mind, I think that the documentation of the function must say that only MUST be executed from a method inside the "read" thread (like TAsynchConnection.OnRead )
Remove an handle from the internal list, and close its connection
- could be executed e.g. from a TAsynchConnection.OnRead method
https://synopse.info/files/html/api-1.1 … TIONREMOVE
Anyone has any suggestion on how to improve this?
Thanks!
Same issue here.
Also I've found that with the current implementation is impossible to know the disconnection reason of a client. I've tried with "WSAGetLastError" on my connection "TAsynchConnection.BeforeDestroy" and always returns '0' because the ProcessRead function CloseConnection is only performed on a gracefully connection close.
But with the solution proposed this will be possible, however I'd change the code to the following:
{
if res<0 then // error - probably "may block"
break;
if res=0 then begin // socket closed -> abort
CloseConnection;
exit;
end;
}
if (res<0) and (WSAGetLastError() = WSAEWOULDBLOCK) then // error - probably "may block"
break;
if res<=0 then begin // socket closed or error -> abort
CloseConnection;
exit;
end;
As the documentation suggest that only WSAEWOULDBLOCK is the only error that can be resolved trying again later...
Same issue here.
Write works if called anywhere outside of OnRead, but if used inside as documentation suggest it hangs and no write is performed.
I'm trying a simple "echo" program:
function TMyTCPConnection.OnRead(Sender: TAsynchConnections): TPollAsynchSocketOnRead;
var
aLineEnd : Boolean;
sendStr: SockString;
begin
result := sorContinue;
aLineEnd := Pos(#10,fSlot.readbuf)>0;
fLastFullLine := fLastFullLine + fSlot.readbuf;
fSlot.readbuf := '';
if aLineEnd then
begin
sendStr :='Received: '+ fLastFullLine;
fLastFullLine := '';
Sender.Write(self,sendStr);
end;
end;
Pages: 1