You are not logged in.
Sadly, for me, still, it does not return from the Free call. MsgWaitForMultipleObjects still times out...
P.S.: I've tried it 10 times, only one it returned, all other times, it waited for (whatever)...
I do hope we see each other at the EKON, I'll have a drink or two at the ready for you!
I guess, this could relate to https://synopse.info/forum/viewtopic.php?id=5327 as well?
Okay, np.
What is wrong with doing the purpose check just before SetCookie() and not call the method?
Basically nothing, but you have repetitive code all over the place, rather than one location to check it. It's helped me a lot already.
In the end, it stops within the repeat loop:
function TThread.WaitFor: LongWord;
{$IF Defined(MSWINDOWS)}
var
H: array[0..1] of THandle;
WaitResult: Cardinal;
{$IF not Declared(System.Embedded)}
Msg: TMsg;
{$ENDIF}
begin
if FExternalThread then
raise EThread.CreateRes(@SThreadExternalWait);
H[0] := FHandle;
if CurrentThread.ThreadID = MainThreadID then
begin
{$IF not Declared(System.Embedded)}
WaitResult := 0;
{$ENDIF}
H[1] := SyncEvent;
repeat
{$IF Defined(NEXTGEN) and Declared(System.Embedded)}
WaitResult := WaitForMultipleObjects(2, @H, False, 1000);
{$ELSE}
{ This prevents a potential deadlock if the background thread
does a SendMessage to the foreground thread }
if WaitResult = WAIT_OBJECT_0 + 2 then
PeekMessage(Msg, 0, 0, 0, PM_NOREMOVE);
WaitResult := MsgWaitForMultipleObjects(2, H, False, 1000, QS_SENDMESSAGE);
{$ENDIF}
CheckThreadError(WaitResult <> WAIT_FAILED);
if WaitResult = WAIT_OBJECT_0 + 1 then
CheckSynchronize;
until WaitResult = WAIT_OBJECT_0;
end else WaitForSingleObject(H[0], INFINITE);
CheckThreadError(GetExitCodeThread(H[0], Result));
end;MsgWaitForMultipleObjects(2, H, False, 1000, QS_SENDMESSAGE); always returns 258...
P.S.: Delphi 10.3.3
P.S.: Rather than an event handler, a virtual method should suffice, I'd guess.
Hi Arnaud,
how about extending the class TSQLRestServerURIContext for basic implementation of cookie consent for the privacy (GDPR). I would suggest a method, like SetCookie(cookie: RawUTF8; level: TCookiePurpose): boolean. Internally, it'll write to OutSetCookie, if user consent has been given.
If enabled, the context can read a cookie (e.q. cookieconsent=....), that can be analysed with each call to the method. An event handler would allow more specific handling, if desired, otherwise it'll implement the basic 4 levels defined by the GDPR (Strictly necessary cookies [always allowed], Preferences cookies, Statistics cookies, Marketing cookies).
If you like that idea, I'll add that and create a pull request. Otherwise I'll keep it in my child class.
This way, no existing code breaks, but future development could automate cookie consent handling fairly well.
I am just commenting that this goes against the new specification that will be adopted.
Call it 2.0.0.x Alpha-Release and breaks are okay, until you come to the release, I would say ;-)
Hi,
I have a THttpApiWebSocketServer, and when freeing it, at the end of the app, the call will not return. Worked last week Friday (I think) without a problem.
program Project3;
{$APPTYPE CONSOLE}
{$R *.res}
uses
System.SysUtils,
SynCrtSock;
var
WebSocketServer: THttpApiWebSocketServer;
begin
try
WebSocketServer := THttpApiWebSocketServer.Create(False);
WebSocketServer.AddUrlWebSocket('A', '8081', False, '+', True);
Writeln('Freeing');
WebSocketServer.Free;
Writeln('You''ll never see me :-(');
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
Writeln('Done');
Readln;
end.Gets stuck in function TThread.WaitFor: LongWord;
Regards,
Daniel
Okay, thx.
Still feels a bit funny, but I understand. Have a real problem now, will check a bit further and likely will open another thread in a minute.
So Safe.Lock allows me to "lock" multiple times from within the same thread without deadlocking? It only blocks other threads at the same time? Didn't know/test that before. Would be cool.
Still leaves, imo, the big problem, that some methods do locking automatically (like Add/Remove/Clear), while others like Delete do not, which is not obvious, when using the list.
Hi,
so far, it was safe to do following:
LockedList.Safe.Lock;
try
for ItemIdx := 0 to LockedList.Count - 1 do
ItemClear(TMypSession(LockedList[ItemIdx]));
LockedList.Clear;
finally
LockedList.Safe.UnLock;
end;with the recent change, having seen that in many places, it will dead lock at the clear method.
Further, having methods like Add/Remove/... automatically thread safe, and others like Delete not being thread safe is, imho, not a good idea, as one ALWAYS has to look at the documentation, to be sure what's happining. And as the documemantion does not work within the IDE, that means jumping around in the sources.
THe old SafeAdd/Safe... methods where more self-identifying and certainly easy to use.
Please, roll back that change. :-)
Hi Arnaud,
function GetCardinalDef(P: PUTF8Char; Default: PtrUInt): PtrUInt;
var c: byte;
begin
result := Default;
if P=nil then
exit;
c := byte(P^);
repeat
if c=0 then
exit;
if c>ord(' ') then
break;
inc(P);
c := byte(P^);
until false;
dec(c,48); // <--------------------------------------
c := byte(P^)-48;
if c>9 then
exit;
result := c;
repeat
inc(P);
c := byte(P^)-48;
if c>9 then
break;
result := result*10+PtrUInt(c);
until false;
end;I guess, that one may be just removed and fine it is.
It is time to change how the framework is versioned.
Aye :-)
2. mORMot 2.0.0 would induce incompatible API changes.
Ouch, but happens from time to time.
2.1. To remove all reminicence of the "SQLite3 framework", e.g. the SQLite3 sub-folder.
As long as SQLite stays supported, I think no one would ever be sad about a folder going away.
2.2. Try to split SynCommons.pas and mORMot.pas units into smaller units. Putting e.g. the raw JSON process or RTTI in some external unit.
That certainly will be interessting, as we all have to start finding things again. But I hear you, those units are awfully big and likely hard to maintain. Certainly, giving you awesome JSON stuff its own realm to live in is a great step.
2.3. Introduce some high-level classes and units, perhaps as (why-not?) some non-visual TComponent version of our classes. Keeping existing low-level classes, which are stable and full of features - but difficult to adopt.
Not sure what the components would provide really, and likely would never use them, but I think almost no one is using everything of the framework, so they would not hurt. Just can't really see the benefit of those. Then again, I promote using web interfaces, which would not need them anyways.
2.4. Get rid of Delphi 5 official compatibility, and focus on FPC as main target - but perhaps officially maintaining only a few version of Delphi (2007/XE7/10.3?) - with other version maintained by contributors.
Not using FPC, but gladly keep testing on D10.3 and coming versions. As you use FPC for your work since a few years, I am sure, it's been your focus anyways, so getting rid of a couple old versions should be fine, I'd say.
And as your documentation is better than almost any other out there, I am not scared of the coming changes.
Regards,
Daniel
P.S.: Do you have a time frame on that?
Will not quite work, as different clients need to be connected to different host servers, depending on their credentials. With your web sockets relay classes I have a great blueprint for the same setup for our needs. Will get that done this week.
Have tested it and it works. Now I'll have to extend it for our scenario.
I would be happy to try to implement HTTP/HTTPS relay. However, do you have a pointer, as to what could be the most performant way?
Thanks,
Daniel
Works, thanks. My problem was sending the frames, forgot to surround my string with StringToUTF8() in the test app and it never got sent. Works perfectly fine, now. ![]()
Your interfaces are clearly breaking the Liskov substitution principle: the same name is breaking the contract, since it would change the currency.
I just used that as a simplified sample. The real world problem we have is way more complex and as we just have to "run" after federal changes every quarter of the year, and have to be backwards compatible with older requirements for at least a year, I was looking for something easier.
What we really need, are changed implementations, as the services of the other parties change every so often, however, I would like to keep the client unaware of that. The "tax sample" was really a totally over-simplified sample.
The client shall simply say, I need to generate the tally for a certain period and submit it. Depending on the period, we have to implement totally different algorithms. And this is only one of many areas that change quarterly, therefore an abstract factory, that is returning the correct implementation would be best. Honestly, I can't see how that would break the Liskov substitution principle, quite contrary to that. It is using it. I just don't want the client t decide which derived implementations of each service to use, but let the server decide which child class would be the correct one.
Okay, that I should have noticed, when it showed the place, where it failed. Interesting though, that it worked so far, always. Thanks.
Worked with prior releases (from about a week ago), just pulled the most recent updates from GIT and it's not working as expected.
Hi,
a quiet simple call to UrlEncodeJsonObject will fail in an exception (Delphi 10.3.3).
Sample project:
program Project2;
{$APPTYPE CONSOLE}
{$R *.res}
uses
SynCommons,
System.SysUtils;
procedure Test;
var
SendDataJSON, SendDataEncoded: RawUTF8;
begin
SendDataJSON := '{"b":30,"a":12}';
SendDataEncoded := UrlEncodeJsonObject('', PUTF8Char(SendDataJSON), []);
Writeln(SendDataEncoded);
end;
begin
try
Test;
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
Readln;
end.Any hints, anyone?
Not having received any directions, I tried to implement it the most direct way and created a simple patch and pull request: https://github.com/synopse/mORMot/pull/271
Hi,
using
aWebSocketServer := THttpApiWebSocketServer.Create(False);
aWebSocketServer.AddUrlWebSocket(aServerRoot, WebSocketsPort, ServerSecurity = secSSL, '*', True);
aWebSocketServer.RegisterProtocol(aProtocolName, False, OnAccept, OnMessage, OnConnect, OnDisconnect);and connecting to it from a browser client like
socket = new WebSocket(`wss://domain:8081/Test/`, 'protocol')
socket.onopen = _message => {
console.log(_message)
socket.send(JSON.stringify(['mediator', { register: true, service: 'xyz', channels: 'xyz', id:' blubb' }])
}
socket.onmessage = _message => { console.log(_message) }
socket.onerror = _message => { console.log(_message) }
socket.onclose = _message => { console.log(_message) }works perfectly.
However, how could I connect from a Delphi client to the web sockets (used for push notifications), as I do from the browser? Which mORMot-class should I use. All samples I've found directly use interfaces, and similar.
Hi,
is there a designed way, to implement an interface multiple times and have the resolver return an implementation based upon some information.
Simple sample:
type
ITaxCalcService = interface
['{9B1FA8CB-9552-4CE7-B1C7-8D11D72291BC}']
function CalcTaxes(const aNetto: Currency; out aTaxes: Currency; out aTaxPerc: Double): TCQRSResult;
end;
TTaxCalcServiceDE = class(TDDDRepositoryRestCommand, ITaxCalcService)
protected
function CalcTaxes(const aNetto: Currency; out aTaxes: Currency; out aTaxPerc: Double): TCQRSResult;
end;
TTaxCalcServiceAT = class(TDDDRepositoryRestCommand, ITaxCalcService)
protected
function CalcTaxes(const aNetto: Currency; out aTaxes: Currency; out aTaxPerc: Double): TCQRSResult;
end;Perfect would be Rest.Services.Resolve(ITaxCalcService, Srv, 'DE') - but this, I know, does not exist.
I could now create Specific interfaces like
type
ITaxCalcService = interface
['{9B1FA8CB-9552-4CE7-B1C7-8D11D72291BC}']
function CalcTaxes(const aNetto: Currency; out aTaxes: Currency; out aTaxPerc: Double): TCQRSResult;
end;
ITaxCalcServiceDE = interface(ITaxCalcService)
['{08C30244-CD53-42B0-9CB6-DEB043719CAE}']
end;
ITaxCalcServiceAT = interface(ITaxCalcService)
['{7D0CDA1E-C44B-448B-9E28-1364994D6054}']
end;
TTaxCalcServiceDE = class(TDDDRepositoryRestCommand, ITaxCalcServiceDE, ITaxCalcService)
protected
function CalcTaxes(const aNetto: Currency; out aTaxes: Currency; out aTaxPerc: Double): TCQRSResult;
end;
TTaxCalcServiceAT = class(TDDDRepositoryRestCommand, ITaxCalcServiceAT, ITaxCalcService)
protected
function CalcTaxes(const aNetto: Currency; out aTaxes: Currency; out aTaxPerc: Double): TCQRSResult;
end;and have an [GoF] Abstract Factory decide upon the country code which implementation to load. No big problem, however, what would you do? Maybe I am missing something obvious.
I simply want to ensure not to have to create some if-then-else-if... all the time ;-)
Kind regards,
Daniel
Hello Arnaud,
further diving into mORMot for a new project, I have hit a small roadblock.
On the server, it is fairly easy to create custom URI patterns, which allows us to nicely create multiple modules with their unique roots in the server.
From the client, that is harder to manage, as far as I have seen. So, my solution would be following:
Overriding TSQLRestClient.ServiceContainer to return my own implementation. Works out of the box.
Problem. both instances of TServiceContainer.AddInterface should be either virtual, so I can implement my own class of TServiceFactoryClient,
or
TServiceContainer should have a property returning the TServiceFactoryClientClass to b used, to implement custom solutions. Which would your preferred way?
TServiceFactoryClient contains virtual methods, so it seems to be designed to be overriden. :-)
Further, in TServiceFactory.Create, you directly define fInterfaceURI to be ToUTF8(aInterface^.Name) (excluding a possibly leading "I" from the name. How about moving that into a virtual method too, so it can easily be adapted?
Thanks for the input, I would feel free to change as advised and create a pull request.
Kind regards,
Daniel Wischnewski
Why not simply create one or more records to pass your data? (just askin')
I can't really see a problem here. Clean Coding usually would limit you to about 3 parameters anyway and more than six is akin to code smell...
Oh yes, it will. May it be, that you forgot to commit sample 38, you mentioned it in the comments, but it does not show in the rep.
Thx
Hi, this week Bernd Ua is visiting us, but next week I'll have time again to look at it. Thanks a bunch already, you are simply awesome.
lso because I believe this will be an arduous task.
Oh yeah, it will be and I am not sure we would like to do it. But for now, this is a small research project, so I am fine with this.
@ab : that is my wish too, however, in the environment of most customers (medical networks) those possibilities are not within their power. All we can arrange for, is a connection to our server. Installing other VPN solutions is simply not possible. Firewalls are not controlled by our customers in this network. And we are talking about a lot of customers...
Hi Arnaud,
right now I cannot see, how this would solve the problem. Let me try to explain in other words.
Simple work model is easy. Client App (possibly a web interface, possibly a GUI client) connects to the mORMot server, as just about any of the demo apps to demonstrate. Easy, no problem.
The mORMot server is the *only* server having direct access to the data, no replication involved. Now, for various reasons and restraints, when out of office, the client app needs to access the server. The client cannot use a VPN, nor connect to the mORMot server directly. There is simply no way inbound connections into the server are available. Outbound are though. Think of it, like TeamViewer can map sessions around most firewalls, by providing an HTTPS server to route all traffic.
When running the mORMot server will connect with our HTTPS server and announce, that it is available.
When being out of office, the users client will connect to an HTTPS server provided by us, identify and get the correct destination server assigned.
From here on all requests made by the client are sent to our HTTPS server and this one must forward it to the correct mORMot server.
The real problem for me is, that mORMot usually receives request via POST/GET/DELETE/..., however, as there are no inbound connections allowed, the server has to pick up those requests (HTTP GET) and unwrap the original request and process it as it would, when the client connects directly.
So my question here would be, do you have an idea where best to hook into the framework to make that work. Yes, the process would be slower, than connecting directly, however, as of now, we do not see a better approach. However, a better approach is just as welcome. However, we must keep in mind, while data must be transfered, the database must not be replicated.
Due to the hardware setup, many of our customers cannot open ports to the outside (special security hardware), however, it would be possible to connect via HTTPS to our servers, which we could announce through the company, that setups the hardware.
I am not fond of this approach myself, however, this would be the only way for us to provide that possibility at all. Due to all those, currently, customers have two installations and manually import and export their database onto thumb drives to work at home and in the office. :-o
In theory, companies uses VPN to resolve this, as many companies allow remote work from home to its salaries, they only use vpn to allow secure connection from outside to local server !
Yes, but many customers here do not like the burden of a VPN setup and rather look for another, for them, simple solution. VPN was my first though too, but this is considered undesirable here.
Hi,
imagine the following setup. You have your mORMot server running at the office. And you want to access it with a client from another place, like home or a coffee shop. The user cannot connect directly into the office, so we need a relay server to send to requests into the office server and the answers back to the client.
Now, one idea here is, to make this into an add-on feature, and yes, let the customer pay for the service.
Communication shall be encrypted end-to-end. However, data packets going through our server must be identified, eg: who sends them, to which customers server do they need to be routed. Servers and clients connect to our relay server alike. Now down to the question:
Where would be the best point to hook into the communication chain.
The relay server cannot open TCP/IP to the server, so the server could open a web socket into the relay server to be informed when a request arrives. Pick it up and process it.
On the client, the whole call should be wrapped, so its request can be identified by the relay server and routed to the correct office server.
Those is a very early concept and any ideas leading me the right way are welcome. No line of code done yet. First a proof of concept. :-D
Regards,
Daniel
Hello Arnaud,
thx for your service, this fixed it for me too.
Kind regards,
Daniel
At some point, following function gets called (Unit SynSQLite3Static):
function log(const val: extended): extended;
asm
fld val
fldln2
fxch
fyl2x
fwait
end;Upon coming to fwait, the exception is thrown.
Well, as I said, I did not get very far. From
function TSQLDBStatement.FetchAllToJSON(JSON: TStream; Expanded: boolean;
RewindToFirst: boolean): PtrInt;
...
while Step(RewindToFirst) do beginit goes into
function TSQLDBSQLite3Statement.Step(SeekFirst: boolean): boolean;
...
result := fStatement.Step=SQLITE_ROW;to
function TSQLRequest.Step: integer;
...
result := sqlite3_check(RequestDB,sqlite3.step(Request),'Step');Which seems to be a function within the SQLite driver. (sqlite3.step(Request))
Then it jumps into
function memset(P: Pointer; B: Integer; count: Integer): pointer; cdecl; { always cdecl }
// a fast full pascal version of the standard C library function
begin
FillCharFast(P^, count, B);
result := P;
end;and upon return into the CPU view... :-(
Regards,
Daniel
I have created a simple sample, that will reproduce the problem.
Start SynDB Explorer with a new, empty SQLITE3 DB.
Execute each of the following statement separately, the last on will reproduce the error:
CREATE VIRTUAL TABLE Idx USING fts5(A, tokenize='unicode61');
INSERT INTO Idx (RowID, A) VALUES (1, 'hello world');
INSERT INTO Idx (RowID, A) VALUES (2, 'hello max');
INSERT INTO Idx (RowID, A) VALUES (3, 'hello frank');
SELECT RowID, A FROM Idx WHERE Idx MATCH 'max';
SELECT RowID, A FROM Idx WHERE Idx MATCH 'max' ORDER BY Rank;Regards,
Daniel
Hi,
we have manually created a FTS5 Fulltext-Index, where the following SQL code will crash, when using TSQLRequest:
SELECT GlobalID, Module, Service, DefaultDisplayText FROM ukiFtsMap JOIN (SELECT RowID, Rank FROM ukiFtsIndex WHERE ukiFtsIndex MATCH 'max') ON ID = RowID ORDER BY Rank LIMIT 15 OFFSET 0When removing the Rank from the SELECT statement, all will execute just fine, however, the important match ranking is lost:
SELECT GlobalID, Module, Service, DefaultDisplayText FROM ukiFtsMap JOIN (SELECT RowID FROM ukiFtsIndex WHERE ukiFtsIndex MATCH 'max') ON ID = RowID LIMIT 15 OFFSET 0First time, I try to execute this statement in the SynDB Explorer, I get the following error message:
[Window Title]
SynDB Explorer 1.18.5030 (compiled with Delphi 10.3 Rio 32 bit) - FTS
[Main Instruction]
Error
[Content]
Invalid floating point operation
[Expanded Information]
Exception class: EInvalidOp
[^] Hide details [OK]Subsequent request will fail with following error:
[Window Title]
SynDB Explorer 1.18.5030 (compiled with Delphi 10.3 Rio 32 bit) - FTS
[Main Instruction]
Error
[Content]
Access violation at address 0084C1CC in module 'SynDBExplorer.exe'. Read of address 00000008
[Expanded Information]
Exception class: EAccessViolation
[^] Hide details [OK]Please note, that the same SQL will run fine with other SQLite Browsers, so I believe the problem is with the mORMot implementation. However, debugging did not get me very far. :-(
The problem can be recreated with the latest nightly build.
Kind regards,
Daniel
@Arnaud: I can send you a copy of the DB3 file for checking into this.
Hi,
I have the following simple SQL Records:
TSQLBook = class;
TSQLAuthor = class;
TSQLLinkBookAuthor = class(TSQLRecordMany)
protected
FSource: TSQLBook;
FDest: TSQLAuthor;
published
property Source: TSQLBook read FSource write FSource;
property Dest: TSQLAuthor read FDest write FDest;
end;
TSQLBook = class(TSQLRecord)
protected
FTitle: RawUTF8;
FDesc: RawUTF8;
FAuthorList: TSQLLinkBookAuthor;
published
property Title: RawUTF8 read FTitle write FTitle;
property Desc: RawUTF8 read FDesc write FDesc;
property AuthorList: TSQLLinkBookAuthor read FAuthorList;
end;
TSQLAuthor = class(TSQLRecord)
protected
FName: RawUTF8;
FBookList: TSQLLinkBookAuthor;
published
property Name: RawUTF8 read FName write FName;
property BookList: TSQLLinkBookAuthor read FBookList;
end;In my Domain-Classes, what type do I need to use for the links? See -->
type
TBook = class;
TAuthor = class;
TLinkBookAuthor = class(TUkiSynAutoCreateFields)
private
FSource: TBook;
FDest: TAuthor;
protected
procedure AssignTo(aDest: TSynPersistent); override;
public
constructor Create; override;
destructor Destroy; override;
published
property Source: TBook read FSource write FSource;
property Dest: TAuthor read FDest write FDest;
end;
TLinkBookAuthorObjArray = array of TLinkBookAuthor;
TBook = class(TUkiSynAutoCreateFields)
private
FTitle: RawUTF8;
FDesc: RawUTF8;
--> FAuthorList: TLinkBookAuthorObjArray;
published
property Title: RawUTF8 read FTitle write FTitle;
property Desc: RawUTF8 read FDesc write FDesc;
--> property AuthorList: TLinkBookAuthorObjArray read FAuthorList;
end;
...Kind regards,
Daniel