#1 Re: mORMot 1 » Single quote and JSON in MVCApplication » 2019-02-05 10:33:54

Oh great I didn't knew there was such helper. It work as expected thanks for your help !

This is what I ended-up with:

Delphi:
procedure TWebApp.Test(var Scope: Variant);
var
  aSomeText: RawUTF8;
  aVarRes: Variant;
begin
  aSomeText := 'Some text with single quote here '' and that''s it.';
  TDocVariant.New(aVarRes);
  TDocVariantData(aVarRes).AddValue('Display', aSomeText);
  _ObjAddProps(['test', aVarRes], Scope);
end;

JS:
<script type="text/javascript">
	var aTest = JSON.parse({{{JSONQuote Scope.test}}});
	alert(JSON.stringify(aTest));
</script>

#2 mORMot 1 » Single quote and JSON in MVCApplication » 2019-02-04 12:43:25

StxLog
Replies: 2

Hi,

In my web application (TMVCApplication), I'd like to send a JSON object to my view, who contains a string with a single quote:

procedure TWebApp.Test(var Scope: Variant);
var
  aSomeText: RawUTF8;
  aVarRes: Variant;
begin
  aSomeText := 'Some text with single quote here '' and that''s it.';
  TDocVariant.New(aVarRes);
  TDocVariantData(aVarRes).AddValue('Display', aSomeText);
  _ObjAddProps(['test', aVarRes], Scope);
end;

The value of aVarRes in the debugger is:

'{"Display":"Some text with single quote here '' and that''s it."}'

In the view I have this in my script:

<script type="text/javascript">
	var aTest = '{{{Scope.test}}}';
	alert(aTest);
</script>

and I get:

<script type="text/javascript">
	var aTest = '{"Display":"Some text with single quote here ' and that's it."}';
	alert(aTest);
</script>

Which obviously raise an error.
If I try a string replace to replace all '' by \'' (the escape char in JS) like this before passing it to my variant:

aSomeText := StringReplace(aSomeText, '''', '\''', [rfReplaceAll]);

It seem that the char \ is escaped automatically like this:

value of "aSomeText" is: 'Some text with single quote here \'' and that\''s it.'
value of "aVarRes" is: '{"Display":"Some text with single quote here \\'' and that\\''s it."}'

Which obviously fail too..

edit:
and if I do this it work as expected: (but I can't in my use case build all the JSON by hand)

  aSomeText := '{"Display":"Some text with single quote here \'' and that\''s it."}';
  _ObjAddProps(['test', aSomeText], Scope);

I've tried a few others way but couldn't get it to work.
What would be the best way for me to achieve this ?

Thanks for your time !

#3 mORMot 1 » Virtual table » 2017-09-28 09:40:53

StxLog
Replies: 1

Hi mORMot community,

We'd like to have a table on which we can use the ORM (SELECT and UPDATE), but part of the structure of the table need to be changed from the code (there is a fixed structure, TSQLFamily, in which the end-user can add/remove fields) so using a TSQLRecord is excluded (I guess?).
(to be simple, we actually have a TSQLFamily declared, and a TSQLFamilyUser with "FamilyID", "FieldName" and "FieldValue" <- very simplified)
As of right now, we are able to consolidate all those data in the client side, but the process is quite long and we can't SELECT or JOIN on it.

So I'm starting to look at TSQLVirtualTable, and I'd like to know if I'm going in the right direction?


IF I've correctly understood the doc, I need:
-Inherit a TSQLVirtualTable or TSQLRecordVirtualTableAutoID (let say TSQLVirtualFamily) and override GetTableModuleProperties and Structure (to be able to not use a TSQLRecord) and I guess
-Inherit a TSQLVirtualTableCursorIndex (TSQLVirtualFamilyCursor) and override at least Column and Search

and then register the table with VirtualTableRegister

How far am I from the truth? And more important, is this the right choice for what I'm trying to do?

#4 Re: mORMot 1 » Query on lot of records » 2017-09-15 08:47:52

@oz I totaly agree with you for a normal paging functionality, but the way I'm using it it wasn't a true problem.

BUT as far as I understand the EXPLAIN command, the LIMIT way seem more fast or at least equal so yes, go for the LIMIT as it would avoid the headaches as stated by @oz.

No advices on a proper-(mORMot)-way to append two JSON array or two TSQLTableJSON?

#5 Re: mORMot 1 » Query on lot of records » 2017-09-14 14:48:16

tongue I'm slowly switching my mind to ORM but it take time

I've been able to introduce the paging functionnality based on your answer and it works great, thanks!

As for optimization, what should I prefer?

FormatUTF8('ID >= ? AND ID < ? ORDER BY ID', [],
          [aLastID, aLastID+C_MAX_LOADING]));

or

FormatUTF8('ID >= ? ORDER BY ID LIMIT ?', [],
          [aLastID, C_MAX_LOADING]));

What would be the fastest (if there is any difference) in your opinion? (I know it isn't exactly the same result I'm going to have but that's not a problem).

And as of right now, I'm using RetrieveListJSON to execute my SELECT query, and consolidating the resulting JSON in a way you wouldn't aprove (for testing purpose). Do you have any advice for consolidating all the resulting JSON array together?

Thanks for your fast reply, as usual.

#6 mORMot 1 » Query on lot of records » 2017-09-13 12:06:10

StxLog
Replies: 6

Hi mORMot community,

First of, I'm sorry if this as already been answered, I didn't knew what to type to find such topic.

I have a table with around 800.000 record in it, and between 5 to 8 fields. What would be the best way to query all those records at once?
We are currently experiencing timeout in a local configuration with a simple ExecuteList(aTable, 'SELECT * FROM MyTable');
The timeout fire before the server can even end processing the request.

We haven't cached anything explicitly for now, and we haven't done any optimization or else as it is the first time we are experiencing slowdown with this great framework.

Thanks for you help.

#7 mORMot 1 » Unique constraint » 2017-07-11 14:52:22

StxLog
Replies: 1

Hi mORMot community,

It seems that if I define a field as "stored AS_UNIQUE" in my model, and then later I decide it should'nt be Unique, if I remove the "stored AS_UNIQUE" the constraint remain in the DB.
I understand it is normal and it's not a bug from mORMot, but what is the best way to remove this constraint? Should I use plain SQL on the database directly? Is there a "mormot way" I haven't seen?

Thanks for your help

#8 mORMot 1 » Return in callback » 2017-05-31 14:52:46

StxLog
Replies: 1

Hi mORMot community,

I'd like to call a service similar to this with CallbackGetResult

procedure TSQLRestServerDBCustomLI.SimpleButLong(Ctxt: TSQLRestServerURIContext);
begin
  if UrlDecodeNeedParameters(Ctxt.Parameters, 'param1') then
  begin
    DoSomethingShort();

    Ctxt.Returns('something blabla ok');  //return there, the user only need this and don't want to wait
    
    DoSomethingLong(); //Should be executed, but user shouldn't be waiting
  end else
    Ctxt.Error('ERROR: params missing');
end;

and get the result (client side) when it hit "Ctxt.Returns" and not at the end of the procedure, is this possible? I havn't been able to find it in the doc or forum.

#10 mORMot 1 » TSQLRestRoutingREST.FillInput limted to 48 parameters » 2017-05-24 08:26:42

StxLog
Replies: 3

Hi mORMot users!

I'm using a very big HTML form with my TMVCApplication, and I have this error:
"Security Policy: Accept up to 48 parameters for TSQLRestRoutingREST.FillInput"

From what I can see in the sources, there is no parameters allowing me to tick this, isn't it?

So, does someone have a possible solution for me to bypass this limitation?

Thanks for your help!

#11 mORMot 1 » SetUser ignore case » 2017-03-24 08:13:27

StxLog
Replies: 0

Hi mORMot users,

I've just noticed that SetUser is case sensitive for the LogonName (I don't know how I've missed that until now..).
It seems weird as in the DB the LogonName field is declared as SYSTEMNOCASE.
Is this on purpose or it's only on my side ?

If it is by design, is there a way to ignore case for SetUser, or should I lowercase all my LogonName before add/edit and before SetUser?

Thanks for your advices!

#12 Re: mORMot 1 » Synchronize in service » 2017-03-20 16:11:14

Thanks all for reply and trying to help !

@igors233: I needed to pass some information contained within my thread yes.

Yes indeed, I wanted to avoid using OTL as what I wanted to do was quite "simple" (and a lot of work with OTL just for this).

So what i've done in the end is override my DoTerminate method from my TThread like that:

procedure TMyThread.DoTerminate;
begin
  if fByPassSynchronize then
  begin
    if Assigned(OnTerminate) then OnTerminate(Self);
  end else
    inherited;
end;

So when i'm within a service I juste have to set BypassSynchronize := true and my OnTerminate event is well fired at the end.
I've just added a locker to protect shared ressource in the onterminate method just in case,

I will run some more tests as I didn't had time today, but I think this will be enough for my purpose. If not I will make a server method and initiate a client to call it.

Thanks for taking the time even if this isn't directly related to mORMot ; )

#13 Re: mORMot 1 » Recursive nested array pb » 2017-03-17 10:01:47

I think dynamic array are saved as BLOB in DB, so you need either to force the retreive by record:

LTag := TTag.Create(FServer, LID);
FServer.RetrieveBlobFields(LTag);
//Access your LTag.SubCat[1] here

or enable forceblobfield at table or DB level with TSQLRestClientURI.ForceBlobTransfert or TSQLRestClientURI.ForceBlobTransfertTable[]

By default, Blob fields aren't retrieved to save bandwith.

#14 Re: mORMot 1 » Synchronize in service » 2017-03-17 08:23:38

Right now, simply with the OnTerminate event of the thread. They do not talk to each other while thread execution. I just need to briefly reenter in the main thread when execution is over.

#15 Re: mORMot 1 » Synchronize in service » 2017-03-16 15:11:39

Thanks for reply.

Maybe there is something I do not understand (it is very likely, i'm not used to services), but my service isn't waiting while my secondary thread is running, I just want to execute one method when thread is done.

My purpose is to execute some long procedure in background, and when it's finished notify users by emails, but I can't send the email from my background task for various reason. Maybe I could achieve this with TSynBackgroundThread* classes? Haven't looked at it yet, as my threaded class is already tested.

#16 mORMot 1 » Synchronize in service » 2017-03-16 13:15:51

StxLog
Replies: 12

Hello,
Within my server I have to launch one custom TThread for doing some non-critical action in background. Its execution is generally around 2 min, then stop and call DoTerminate to notify server he can do some other action. In a console app, everything is working fine with the SynCommons.ConsoleWaitForEnterKey procedure which call CheckSynchronize. But when i'm using my server as Windows service, with TServiceSingle, there is no CheckSynchronize call.

What would be the best way to implement a similar logic to call CheckSynchronize in services please?

#17 Re: mORMot 1 » Synchronize master/slave » 2017-02-28 13:51:32

Oh I didn't understood this in the doc, now I get why we are stuck.
Sadly not deeply enough, we wanted to know if it was coming from us or the framework before going deeper, we are kind of late for our project.
We will try to investigate and keep you informed in this topic as we progress.

#18 Re: mORMot 1 » Synchronize master/slave » 2017-02-28 11:48:40

Hello again,
I'm sorry to insist, but we would like to know if we are doing something wrong here and if yes, where/how? Everything is kind of working in our project except this hmm

Does anyone is using "real-time synchronization" on multiple table? Any advice?

And i'll add a question: is there a way of re-evaluate the 'version' field on a whole DB or table?

Thank you to all.

#19 Re: mORMot 1 » Delphi XE2 - E2251 Ambiguous overloaded call to 'HexToBin' » 2017-02-15 12:37:53

The "E2251 Ambiguous overloaded call to 'HexToBin'" was only the first part of the error message, it was also telling me that it needed boolean value too because Classes.HexToBin needed it I guess.

#20 mORMot 1 » Delphi XE2 - E2251 Ambiguous overloaded call to 'HexToBin' » 2017-02-15 11:35:39

StxLog
Replies: 3

Hi AB,

I've updated today to try your fix 232b1769f8 but i ran into one problem:
When I compile TestSQL3.dpr, with Delphi XE2 on Windows 10, it gave me error "E2251 Ambiguous overloaded call to 'HexToBin'" on line 2299 in SynLog.pas. If I put SynCommons.HexToBin it's ok.

Thanks.

edit: Haven't seen it was already posted https://synopse.info/forum/viewtopic.php?id=3810 sorry for double post.

edit2: It seems also that the Sqlite3Obj haven't been updated. I have error message "[...] Linked version is 3.16.2 whereas the current/expectred is 3.17.0 [...]"

#21 mORMot 1 » Synchronize master/slave » 2017-02-14 12:53:09

StxLog
Replies: 4

Hello AB,

We'are adding synchronization to our project.
Here is a sample project i've made for this topic https://drive.google.com/open?id=0B0BLh … zZsUVV2UTA (you will probably need to update the .dcu folder for both project)

In this sample, if you connect to local server and start synchronization, then add on both table some data, the synchronization process is well executed and data are replicated in the slave DB.
But if you just connect to local server without starting synchronization, add some data on both table, and then enable synchronization, it will only synchronize on slave side the first table to start.

We have noticed this when we have started to add synchronization on existing database. We had only few tables replicated, depending on the first we were synchronizing.

Is this expected, or are we doing something wrong somewhere? Maybe it isn't allowed to replicate multiple table?

I hope you'll understand me well, and thanks for your help.


edit: edited link to add SynRestMidasVCL.pas and SynRestVCL.pas to project

#23 mORMot 1 » DateTime and Currency for GetFieldData in TSynVirtualDataSet » 2017-02-13 10:22:38

StxLog
Replies: 3

Hi AB,

We've added in GetFieldData for TSynVirtualDataSet the handle for Currency field (same as ftFloat) and corrected DateTime with which we had problems in conversions. Handle 30/12/1899 date as NULL.
Here is our overriden code integrated within getFieldData in SynVirtualDataSet.

begin
  result := false;
  OnlyTestForNull := (Buffer=nil);
  RowIndex := PRecInfo(ActiveBuffer).RowIndentifier;
  Data := GetRowFieldData(Field,RowIndex,DataLen,OnlyTestForNull);
  if Data=nil then // on success, points to Int64,Double,Blob,UTF8
    exit;
  result := true;
  if OnlyTestForNull then
    exit;
  Dest := pointer(Buffer); // works also if Buffer is [var] TValueBuffer
  case Field.DataType of
  ftBoolean:
    PWORDBOOL(Dest)^ := PBoolean(Data)^;
  ftInteger:
    PInteger(Dest)^ := PInteger(Data)^;
  ftLargeint, ftFloat, ftCurrency:
    PInt64(Dest)^ := PInt64(Data)^;
  ftDate,ftTime,ftDateTime: begin
    if PDateTime(Data)^ = 0 then begin
      result := false;
      exit;
    end;
    DataConvert(Field,Data,Dest,true);
  end;
  ftString: begin
    if DataLen<>0 then begin
      CurrentAnsiConvert.UTF8BufferToAnsi(Data,DataLen,Temp);
      DataLen := length(Temp);
      MaxLen := Field.DataSize-1; // without trailing #0
      if DataLen>MaxLen then
        DataLen := MaxLen;
      move(pointer(Temp)^,Dest^,DataLen);
    end;
    PAnsiChar(Dest)[DataLen] := #0;
  end;
  ftWideString: begin
    {$ifdef ISDELPHI2007ANDUP} // here Dest = PWideChar[] of DataSize bytes
    if DataLen=0 then
      PWideChar(Dest)^ := #0 else
      UTF8ToWideChar(Dest,Data,(Field.DataSize-2)shr 1,DataLen);
    {$else}          // here Dest is PWideString
    UTF8ToWideString(Data,DataLen,WideString(Dest^));
    {$endif}
  end;
  // ftBlob,ftMemo,ftWideMemo should be retrieved by CreateBlobStream()
  else raise EDatabaseError.CreateFmt('%s.GetFieldData DataType=%d',
         [ClassName,ord(Field.DataType)]);
  end;
end;

We would be glad if you think this is relevant.
Thanks.

edit: If you use this patch, I think DateTimeToNative inside SynVirtualDataset isn't used anymore.

#24 Re: mORMot 1 » SessionUser.Data not Loading on SetUser » 2017-02-01 12:55:09

As usual, thanks for your fast reply and your help, i've updated the library and everything is working as expected ; )

#25 Re: mORMot 1 » SessionUser.Data not Loading on SetUser » 2017-01-31 16:46:44

Well I have a TSQLAuthUser in my model trough my TSQLUser class:

TSQLUser = class(TSQLAuthUser)

So yes, if I directly use TSQLAuthUser as my user table, RetrieveBlobFields() work well, but if i'm using my own inherited class it doesn't work, as GetTableIndex doesn't find TSQLAuthUser.

It seemed to me that mORMot only allowed one TSQLAuthUser per model, so I would think GetTableIndex should return my inherited class. Am i misunderstanding something?

#26 Re: mORMot 1 » SessionUser.Data not Loading on SetUser » 2017-01-31 12:56:00

And what is the best way to get the Data field client side? I have a TSQLUser class overriding TSQLAuthUser, and I need to have the Data field filled in the SessionUser. I've tried to do fClient.RetrieveBlobFields(fClient.SessionUser); as stated in the doc but it tell me I need TSQLAuthUser in the model, and if i do fClient.RetrieveBlob(TSQLUser, fClient.SessionUser.GetID, 'Data', fClient.SessionUser.Data); it tells me i'm not calling RetrieveBlob with the correct argument...
I can still do
  fClient.RetrieveBlob(TSQLUser, fClient.SessionUser.GetID, 'Data', aTmpData);
  fClient.SessionUser.Data := aTmpData;

but it seems weird

Am I missing something here?

#27 Re: mORMot 1 » access rights per record/columns » 2016-12-15 08:32:27

Thanks for your fast and clear answer, we'll try this way then.

#28 mORMot 1 » access rights per record/columns » 2016-12-14 10:03:22

StxLog
Replies: 2

I'd like to define some more tuned access rights than per table. Depending on the TSQLAuthGroup the user should only be able to select or edit X rows or X cols depending on a value of one cols or any other condition.

From what I've seen, you said in others threads and in the doc that I should use an interface based service for doing so.
Could i also do it by defining a custom authentification schema? Does this would be a good way for my purpose?

Thanks.

#29 Re: mORMot 1 » Custom authentication » 2016-10-03 11:27:10

Hi,

I still cannot figure out how to make RemoteDataCreate to work. But i'm moving on and try with TSQLRestServerRemoteDB:
I have succesfully made it work, i can log in and log out from my client, but if I try MyClient.RetrieveList<TSQLSomething>('...', [...]); he throw me an EAbstractError for the MainEngineList function and indeed she isn't overriden in the TSQLRestServerRemoteDB.
Is this on purpose? Should I make my own implementation? Can i have some advices?

Thanks again

#30 Re: mORMot 1 » Custom authentication » 2016-09-28 15:22:33

Thanks,

but I have a problem with it:
from what I have understood, I have my SRV1 TSQLRestServerDB on localhost:8082 with in his model TSQLUser inheriting from TSQLAuthUser among others TSQLRecord. Everything is working fine the server is running and I can connect to it to query records.
My SRV2 is a TSQLRestServerFullMemory with an empty model, he only handle a few thing. He is the one who need to get TSQLUser from SRV1.
So:

//Init client
fTEST := TSQLHttpClient.Create('localhost', '8084', TSQLModel.Create([TSQLExample1, TSQLUser, TSQLExample2, ...], 'SRV1'));
fTEST.Model.Owner := fTEST;
if not fTEST.ServerTimeStampSynchronize then
      raise Exception.Create('Erreur: Contact');
if not fTEST.SetUser('user', TSQLUser.ComputeHashedPassword('pass'), true) then
      raise Exception.Create('Erreur: Auth');

//Init server
fSRV2 := TSQLRestServerFullMemory.CreateWithOwnModel([], true, 'SRV2');
fSRV2.RemoteDataCreate(TSQLUser, fTEST);
fSRV2.CreateMissingTables;

Now if i execute this, when it come to RemoteDataCreate I have an EModelException "TSQLUser should be part of the TSQLModel 'SRV2'"
And if i set TSQLUser to be part of the model of fSRV2, i have an EORMException "Duplicate TSQLRestServerFullMemory.RemoteDataCreate(TSQLUser)", which make sense.

What did i do wrong? Any hint?


EDIT: In the doc you wrote "The only prerequirement is that all TSQLRecord classes in the main model do exist in the redirected database model."  -> In this context my main model is the empty one (SRV2), and the redirected database model is the one with TSQLUser (SRV1)? So technically they do exists in the redirected database model, doesn't they?

#31 mORMot 1 » Custom authentication » 2016-09-28 11:08:28

StxLog
Replies: 3

Hi,

I have two mormot servers running currently (let's call them srv1 and srv2), with each having his own User table. I'd like to change that and make Srv2 ask if pair username&password is correct on Srv1.
Right now they both use TSQLRestServerAuthenticationSignedURI as default authenticate scheme.
What is the best (mORMot) way to implement this logic? Should I Inherit from TSQLRestServerAuthenticationSignedURI and just override one or two function, inherit from TSQLRestServerAuthentication as stated in the doc, or other?

Thanks for your time,

#32 Re: mORMot 1 » Contribution: TSynRestDataset » 2016-09-12 11:31:20

We have been working with your units for a project and we found these very usefull. We have added a few things in it, and decided to share them with you:
https://drive.google.com/open?id=0B0BLh … XM1amVzdlE

SynRestVCL:
    - cast String<=>RawUTF8
    - Params null handled
    - Params string quoted

SynRestMidasVCL:
    - cast String<=>RawUTF8
    - DoBeforeDelete override instead of DoOnBeforeDelete
    - SetCommandText ifdef ISDELPHIXE instead of XE2


If you think these small edits are relevant please update the files.

We'll keep following this thread for later update.

#34 mORMot 1 » How to copy a row » 2016-08-29 12:03:26

StxLog
Replies: 3

Hi Arnaud and mORMot community,

I'd like to make a copy of an entry in my db, is there a way to do it in the "mORMot way" without having to first create the object and then re-post it with a new ID?

Thanks in advance!

#35 Re: mORMot 1 » Returning file within a TMVCApplication » 2016-05-13 14:05:13

Thanks for your reply and your time ! It worked as expected.

#36 mORMot 1 » Returning file within a TMVCApplication » 2016-05-13 10:25:59

StxLog
Replies: 2

Hi! I already have asked similar question but didn't had the time to try everything out at this time, so here i am again.

In my TMVCApplication, I'd like a function (or procedure doesn't matter) to return a pdf from stream.
So after search and try, I'm with

function Test(const id: Integer): TServiceCustomAnswer;
begin
    [...]
    Result.Header := BINARY_CONTENT_TYPE_HEADER;
    Result.Content := StreamToRawByteString(TMemoryStream(pdfExport.Stream));    
end;

From what I understand, doesn't this should be enough? But when I link to Test?id=22 well it redirect me to Test.html... (and create a mustache template)

I also have tried with CurrentServiceContext() and is Request member, couldn't figure out how to do it.

If someone could point me in the right direction I would greatly appreciate.

#37 Re: mORMot 1 » MVC /json /mvc-info » 2016-05-09 07:48:40

Thanks for your fast reply, I was able to deactivate /mvc-info with TMVCPublishOption  but couldn't find for /json?

#38 mORMot 1 » MVC /json /mvc-info » 2016-05-06 10:26:53

StxLog
Replies: 2

Hello ab,

I couldn't find if there is a way to disable the /json and /mvc-info commands? I'd like to not have those available in production.

Thanks for your help.

#39 Re: mORMot 1 » Memory leak when connected with 2 client » 2016-04-01 14:52:00

Oh okay, this makes more sense, you should REST ; ) thanks again

Also, should I do

MyRestServer.Model.Owner := MyRestServer;

or is this nonsense?

#40 Re: mORMot 1 » Memory leak when connected with 2 client » 2016-04-01 13:38:27

Great, thanks you! I've tried with direct database file name and no leak anymore.

But to remind, I was constructing my TSQLRestServerDB like this

TSQLRestServerDB.Create(
          TSQLModel.Create([], ref),
          TSQLDataBase.Create(ExeVersion.ProgramFilePath+refUniqueServeur+'.s3db', '', SQLITE_OPEN_CREATE or SQLITE_OPEN_READWRITE),
          True)

So aHandleUserAuthentication is already set to true. And I was cleaning it like this

        with fRestTab.Items[ref] do begin
          DB.DBClose;
          DB.Free;
          Model.Free;
        end;
        fRestTab.Remove(ref);

DB.DBClose + DB.Free shouldn't be freeing my Database instance?


Anyway, thanks a lot, problem solved! ; )

#41 Re: mORMot 1 » Memory leak when connected with 2 client » 2016-04-01 12:57:46

I don't know if that relevant, but i've noticed something: if I delete the second .s3db file (the one from the mounted REST), and then launch the server app and client and connect to first REST, mount second REST, unmount it (without connecting to it) and disconnect, when closing server app I also have a memory leak, but not if the .s3db is already created.
Here is a link to the report for this: http://pastebin.com/0ntcVirL

And a link to the report when I connect the client to the mounted REST, my initial problem: http://pastebin.com/ZFLcDcZn


Everything point to the Mount and Unmount methods, but i can't see what am I doing wrong for freeing the REST instance?

Or maybe is there a better way to mount database at demand with your framework?
Thanks.

#42 Re: mORMot 1 » Memory leak when connected with 2 client » 2016-04-01 10:55:56

Thanks for your reply and sorry for the lack of information.
The memory leak is on server side.

There is my full Unmount procedure

procedure TMainServer.UnmountREST(Ctxt: TSQLRestServerURIContext);
var ref: RawUTF8;
begin
  if UrlDecodeNeedParameters(Ctxt.Parameters, 'ref') then begin
    ref := Ctxt['ref'];
    fLog.Enter();
    fLog.Add.Log(sllInfo, 'Ref: '+ref);


    if fHTTPServer.RemoveServer(fRestTab.Items[ref]) then begin
      with fRestTab.Items[ref] do begin
        DB.DBClose;
        DB.Free;
        Model.Free;
      end;
      fRestTab.Remove(ref);  //autoFree because of [doOwnsValues]
      fLog.Add.Log(sllInfo, ' -- '''+ref+''' unmounted');
    end else begin
      fLog.Add.Log(sllError, ' xx '''+ref+''' couldn''t be unmounted');
    end;

  end else begin
    fLog.Add.Log(sllError, 'Unmout called without parameter');
    Ctxt.Results(['ERREUR: ref missing']);
  end;
end;

I was just using System.ReportMemoryLeaksOnShutdown := true,
but now I have added FastMM4 with FullDebugMode, and the objects who leaks are still marked as "unknow". Should I enable other options from FastMM?

The thing is, I only have memory leaks when i'm connecting client to the mounted REST, but none if I just mount then unmount REST.


Thanks for your advices, this is juste a sample server to show you my problem, I'm going to add some lock to MountREST and UnmountREST. And I will change the access method for my REST instance, thanks!

#43 mORMot 1 » Memory leak when connected with 2 client » 2016-04-01 07:56:13

StxLog
Replies: 9

Hi,



I have a TSQLRestServerDB declared as such:

  TSQLTESTRestServerDB = class(TSQLRestServerDB)
    public
      onMount: TSQLRestServerCallBack;
      onUnmount: TSQLRestServerCallBack;
    published
      procedure Mount(Ctxt: TSQLRestServerURIContext);
      procedure Unmount(Ctxt: TSQLRestServerURIContext);
  end;

And my main server class as such:

  TMainServer = class(TObject)
  protected
    procedure SetLog;
    procedure Connect;
    function OnConnect(Sender: TSQLRestServer; Session: TAuthSession; Ctxt: TSQLRestServerURIContext): boolean;
    function OnDisconnect(Sender: TSQLRestServer; Session: TAuthSession; Ctxt: TSQLRestServerURIContext): boolean;
    function OnUpdate(Sender: TSQLRestServer; Event: TSQLEvent; aTable: TSQLRecordClass; const aID: TID; const aSentData: RawUTF8): boolean;
  public
    fLog: TSynLog;
    fMODELBDD: TSQLModel;
    fBDD: TSQLDataBase;
    fREST: TSQLTESTRestServerDB;
    fRestTab: TObjectDictionary<RawUTF8, TSQLRestServerDB>;
    fHTTPServer: TSQLHttpServer;

    constructor Create;
    destructor Destroy; override;
    procedure MountREST(Ctxt: TSQLRestServerURIContext);
    procedure UnmountREST(Ctxt: TSQLRestServerURIContext);
  end;

So I set up my connection like this:

  fMODELBDD := TSQLModel.Create([]);
  try
    fBDD := TSQLDataBase.Create(ExeVersion.ProgramFilePath+'MainTest.s3db', '', SQLITE_OPEN_CREATE or SQLITE_OPEN_READWRITE);
    try
      fREST := TSQLTESTRestServerDB.Create(fMODELBDD, fBDD, True);
      try
        fREST.CreateMissingTables();

        fREST.OnSessionCreate := OnConnect;
        fREST.OnSessionClosed := OnDisconnect;
        fREST.OnUpdateEvent := OnUpdate;

        //My events
        fREST.onMount := MountREST;
        fREST.onUnmount := UnmountREST;
        // /

          fHTTPServer := TSQLHttpServer.Create(AnsiString('8080'), [fREST], '+', useHttpApiRegisteringURI);
          try
            fRestTab := TObjectDictionary<RawUTF8, TSQLRestServerDB>.Create([doOwnsValues]);

            fLog.Add.Log(sllInfo, ' - Init ok');
            fLog.Add.Log(sllInfo, ' - Server set on port ::8080::');
          except
            FreeAndNil(fHTTPServer);
          end;
      except
        FreeAndNil(fREST);
      end;
    except
      FreeAndNil(fBDD);
    end;
  except
    FreeAndNil(fMODELBDD);
  end;

MountREST:

procedure TMainServer.MountREST(Ctxt: TSQLRestServerURIContext);
var ref: RawUTF8;
begin
  if UrlDecodeNeedParameters(Ctxt.Parameters, 'ref') then begin
    ref := Ctxt['ref'];
    fLog.Enter();
    fLog.Add.Log(sllInfo, 'Ref: '+ref);

    if fRestTab.ContainsKey(ref) then begin
      //If already mounted
      Ctxt.Results([ref]);
    end else begin
      //if not
      fRestTab.Add(ref,
        TSQLRestServerDB.Create(
          TSQLModel.Create([], ref),
          TSQLDataBase.Create(ExeVersion.ProgramFilePath+ref+'.s3db', '', SQLITE_OPEN_CREATE or SQLITE_OPEN_READWRITE), True)
        );
      try
        //fRestTab.Items[refUniqueServeur].Model.Owner := fRestTab.Items[refUniqueServeur];    //Is this usefull?
        fRestTab.Items[ref].CreateMissingTables;

        if fHTTPServer.AddServer(fRestTab.Items[ref]) then begin
          Ctxt.Results([ref]);
          fLog.Add.Log(sllInfo, ' ++ '''+ref+''' mounted');
        end else begin
          fLog.Add.Log(sllError, ' xx '''+ref+''' couldn''t be mounted');
        end;
      except
        fRestTab.Items[ref].DB.DBClose;
        fRestTab.Items[ref].DB.Free;
        fRestTab.Items[ref].Model.Free;
        fRestTab.Remove(ref);
        fLog.Add.Log(sllError, 'failure while mounting: '+ref);
      end;
    end;

  end else begin
    fLog.Add.Log(sllError, 'Mount called without parameter');
    Ctxt.Results(['ERREUR: ref missing']);
  end;
end;

and very quickly how i clean my REST item when calling UnmountREST:

    if fHTTPServer.RemoveServer(fRestTab.Items[ref]) then begin
      fRestTab.Items[ref].DB.DBClose;
      fRestTab.Items[ref].DB.Free;
      fRestTab.Items[ref].Model.Free;
      fRestTab.Remove(ref);  //autoFree because of [doOwnsValues]
      fLog.Add.Log(sllInfo, ' -- '''+ref+''' unmounted');
    end else begin
      fLog.Add.Log(sllError, ' xx '''+ref+''' couldn''t be unmounted');
    end;

And now for my memory leak:
If I do with my client something like this:

//Connection to the first REST
  fClient1 := TSQLHttpClient.Create('localhost', '8080', TSQLModel.Create([]), false);
  if not fClient1.ServerTimeStampSynchronize then
    raise Exception.Create('Fail while timestamp synchro');
  if not fClient1.SetUser('Admin', 'synopse') then
    raise Exception.Create('Fail while SetUser');
//Call MountREST without connecting to it
  fClient1.CallBackGetResult('Mount', ['ref', 'TEST002']);
//Then unmount and disconnect
  fClient1.CallBackGetResult('Unmount', ['ref', 'TEST002']);
  FreeAndNil(fClient1);

I have absolutely no problem when exiting the server app, no memory leak.
But if I init a connection with the second mounted REST like this

//Connection to the first REST
  fClient1 := TSQLHttpClient.Create('localhost', '8080', TSQLModel.Create([]), false);
  if not fClient1.ServerTimeStampSynchronize then
    raise Exception.Create('Fail while timestamp synchro');
  if not fClient1.SetUser('Admin', 'synopse') then
    raise Exception.Create('Fail while SetUser');
//Call MountREST and connect to it
  fClient1.CallBackGetResult('Mount', ['ref', 'TEST002']);
  fClient2 := TSQLHttpClient.Create('localhost', '8080', TSQLModel.Create([], 'TEST002'), false);
  if not fClient2.ServerTimeStampSynchronize then
    raise Exception.Create('CLIENT2: Fail while timestamp synchro');
  if not fClient2.SetUser('Admin', 'synopse') then
    raise Exception.Create('CLIENT2: Fail while SetUser');
//Then disconnect, unmount and disconnect again
  FreeAndNil(fClient2);
  fClient1.CallBackGetResult('Unmount', ['ref', 'TEST002']);
  FreeAndNil(fClient1);

I have memory leak of type unknow.

Why is it only when i'm connecting client to it? Am I doing something wrong?

If you need sample app or the memory leak file just tell me.


Thanks all for your help

#44 Re: mORMot 1 » Services CallbackReleased » 2016-03-04 10:24:23

ok I understand, thanks for your help and your time!

#45 Re: mORMot 1 » Services CallbackReleased » 2016-03-03 14:02:04

When I do F7 it does this

TInterfacedCallback.Destroy
TInterfacedCallback.CallbackRestUnregister
TServiceContainerClient.CallBackUnRegister
TSQLRestClientCallbacks.UnRegister
...

And then FCallback is = nil


But its really erratic, sometimes server side I can see the log 'disconnect' but almost never. Mostly I can see it when i'm doing some step to step, or if I put a sleep(100) after the FCallback := nil; but not necessary all the time

I'm sorry I can't give more precise info, i'm really not an expert in Delphi, but if you need something I'll do necessary

#46 Re: mORMot 1 » Services CallbackReleased » 2016-03-03 13:05:11

Indeed, so i've extracted the service to do this:

  TMainServerService = class(TInterfacedObject, IMainServService)
  protected
    fConnected: array of IMainServCallback;
  public
    procedure Connect(const callback: IMainServCallback);
    procedure CallbackReleased(const callback: IInvokable; const interfaceName: RawUTF8);
    function TotalConnected: Integer;
  end;

  TMainServer = class(TObject)
  protected

    [ ... ]

Changed my service definition to this

fRestServer.ServiceDefine(TMainServerService, [IMainServService], sicShared).
            SetOptions([],[optExecLockedPerInterface]);

And I launch it like this

    try
      server := TMainServer.Create;
      WriteLn('-- running... Press key to leave');
      readln;
    finally
      server.Free;
      server := nil;
    end;

Is this good?
But I still have the same problem, i can see client connect, but not when they disconnect from the service. (nb: when they close their session i can see it in TSQLRestServerDB.OnSessionClosed)

#47 mORMot 1 » Services CallbackReleased » 2016-03-02 14:06:50

StxLog
Replies: 6

Hi,
I'm trying the websocket connection since a few days, and even if everything's working fine, i've bumped into one problem:

Here my interfaces

unit IMainServerCS;
interface
uses
  SysUtils, SynCommons, mORMot;

type
  IMainServCallback = interface(IInvokable)
    ['{7E7CA6D9-DA5A-4524-8257-41B14F2DBE8A}']
    procedure sendMsg(const ID: integer; const msg: string);
  end;

  IMainServService = interface(IServiceWithCallbackReleased)
    ['{397EBE90-AAA0-4ED2-BDEA-ED566585565A}']
    procedure Connect(const callback: IMainServCallback);
    function TotalConnected: Integer;
  end;

const
  TRANSMISSION_KEY = 'SECRET';

implementation

initialization
  TInterfaceFactory.RegisterInterfaces(
    [TypeInfo(IMainServService), TypeInfo(IMainServCallback)]);
end.

Here how i setup my server

type
  TMainServer = class(TInterfacedObject, IMainServService)
  protected
    fConnected: array of IMainServCallback;

    procedure SetLog;
    procedure SetConnection;
    procedure FreeAll;
  public
    fLog: TSynLog;
    fMODELBDD: TSQLModel;
    fBDD: TSQLDataBase;
    fRestServer: TSQLRestServerDB;
    fHTTPServer: TSQLHttpServer;
    fWebApp: TOtherWebApp;

    constructor Create;
    destructor Destroy; override;

    // INTERFACE SERVICE
    procedure Connect(const callback: IMainServCallback);
    procedure CallbackReleased(const callback: IInvokable; const interfaceName: RawUTF8);
    function TotalConnected: Integer;
  end;

implementation

{ TMainServeur }
constructor TMainServer.Create;
begin
  inherited;
  Self.SetLog;            //Init logs
  Self.SetConnection;   //Init connection
end;

destructor TMainServer.Destroy;
begin
  fLog.Enter(self, '::Destroy::');
  Self.FreeAll;
  inherited;
end;

procedure TMainServer.SetLog;
begin
  fLog := TSynLog.Add;
  with fLog.Family do begin
    {$IFDEF CONSOLE}
      EchoToConsole := LOG_VERBOSE;
    {$ENDIF}
    Level := LOG_VERBOSE;
    TSynLogTestLog := TSQLLog;
    RotateFileCount := 5;
    OnArchive := EventArchiveSynLZ;
    ArchiveAfterDays := 1;
    ArchivePath := ExeVersion.ProgramFilePath+'log\archive';
    PerThreadLog := ptIdentifiedInOnFile;
    DestinationPath := ExeVersion.ProgramFilePath+'log\';
    EndOfLineCRLF := true;
    AutoFlushTimeOut := 6;
  end;
  WebSocketLog := TSQLLog;
end;

procedure TMainServer.FreeAll;
begin
  fHTTPServer.Shutdown;
  if Assigned(fHTTPServer) then
    FreeAndNil(fHTTPServer);
  if Assigned(fWebApp) then
    FreeAndNil(fWebApp);
  if fRestServer <> nil then
    fRestServer := nil;
  if Assigned(fBDD) then
    FreeAndNil(fBDD);
  if Assigned(fMODELBDD) then
    FreeAndNil(fMODELBDD);
end;


procedure TMainServer.SetConnection;
begin
  fLog.Enter(self, ' - Init server...');

  fMODELBDD := TSQLModel.Create([TSQLUser, TSQLOther]);
  try
    fBDD := TSQLDataBase.Create(ExeVersion.ProgramFilePath+DB, '', SQLITE_OPEN_CREATE or SQLITE_OPEN_READWRITE);
    try
      fRestServer := TSQLRestServerDB.Create(fMODELBDD, fBDD, True);
      try
        fRestServer.AuthenticationUnregister(TSQLRestServerAuthenticationSSPI);
        fRestServer.CreateMissingTables(); 
        fRestServer.DB.LockingMode := lmExclusive;
        fRestServer.ServiceDefine(TMainServeur, [IMainServService], sicShared).
            SetOptions([],[optExecLockedPerInterface]);

        fWebApp := TOtherWebApp.Create;
        try
          fWebApp.parent := nil;
          fWebApp.Start(fRestServer);

          fHTTPServer := TSQLHttpServer.Create(AnsiString(PORT_CONNECT), [fRestServer], '+', useBidirSocket);
          try
            fHTTPServer.WebSocketsEnable(fRestServer, TRANSMISSION_KEY).
                Settings.SetFullLog;
            fHTTPServer.RootRedirectToURI('root/Default');
            fHTTPServer.AccessControlAllowOrigin := '*';
          except
            FreeAndNil(fHTTPServer);
          end;
        except
          FreeAndNil(fWebApp);
        end;
      except
        fRestServer := nil;
      end;
    except
      FreeAndNil(fBDD);
    end;
  except
    FreeAndNil(fMODELBDD);
  end;
end;



procedure TMainServer.Connect(const callback: IMainServCallback);
begin
  InterfaceArrayAdd(fConnected, callback);

  TSynLog.Add.Log(sllInfo, ' ++ Connect. '+IntToStr(self.TotalConnected+1)+' user(s).');
end;

procedure TMainServer.CallbackReleased(const callback: IInvokable; const interfaceName: RawUTF8);
begin
  if interfaceName='IMainServCallback' then begin
    InterfaceArrayDelete(fConnected, callback);
    TSynLog.Add.Log(sllInfo, ' -- Disconnect. Still '+IntToStr(self.TotalConnected+1)+' user(s).');
  end;
end;

function TMainServer.TotalConnected: Integer;
begin
  result := high(fConnected);
end;

end.

Then in another unit I have my callback implementation:

type
  TMainServCallback = class(TInterfacedCallback, IMainServCallback)
    protected
      procedure sendMsg(const ID: integer; const msg: string);
  end;

[...]

procedure TMainServCallback.sendMsg(const ID: integer; const msg: string);
begin
  //do something
end;
end.

And how I connect (in a DUnit procedure):

procedure TTestCaseConnection.TestConnection;
var
  FClient: TSQLHttpClientWebsockets;
  FService: IMainServService;
  FCallback: IMainServCallback;
begin
  FClient := TSQLHttpClientWebsockets.Create(AnsiString(SYN_ADRESSE), AnsiString(SYN_PORT), TSQLModel.Create([TSQLUser, TSQLOther]));
  try
    FClient.Model.Owner := FClient;
    FClient.WebSocketsUpgrade(TRANSMISSION_KEY);

    FClient.ServerTimeStampSynchronize;

    FClient.SetUser(SYN_USER_NAME, TSQLUser.ComputeHashedPassword(SYN_USER_PASS), true);

    //callback/service
    FClient.ServiceDefine([IMainServService], sicShared);
    FClient.Services.Resolve(IMainServService, FService);

    FCallback := TMainServCallback.Create(FClient, IMainServCallback);
    try
      FService.Connect(FCallback, 'XX');

      [.. do something .. ]

    finally
      FCallback := nil;
      FService := nil;
    end;
  finally
    FClient.SessionClose;
    FreeAndNil(FClient);
  end;
end;

My problem is:
When I connect, on the server console i can see the message '++connect...' but when i disconnect it doesnt call the CallbackReleased(..)
After a few try, I've noticed that if i do some step by step starting from "[.. do something .. ]" in the client, CallbackReleased() is called and the message '--disconnect' appear server side as i was expecting at the beginning.

It seems like the client is disconnecting too fast? But i'm no expert and can't really tell.

Server is in localhost, and when i compile and run Project31ChatServer/Client in same condition i dont have this issue.

Thanks all!

#48 Re: Other components » SynZip directory » 2016-01-26 14:28:33

I'm truly impressed by your reactivity, great job and many thanks ! I'll spread the word, this is the best framework i've ever worked with ; )


Edit: Oh, and of course AddFolder(..) works perfectly.

#49 Other components » SynZip directory » 2016-01-26 10:58:40

StxLog
Replies: 2

Hi!

Is there a way to directly zip a folder with all data in it with TZipWrite ?

Thanks for your time.

#50 Re: mORMot 1 » Change header from TMVCApplication method » 2015-11-13 13:09:11

Thanks, indeed I havn't seen I could access it from here.

So i've been trying to implement it, with Request.ReturnBlob or Request.Call.OutHead ... But couldn't figure out how to do, so if someone could point me in the right direction it would be greatly appreciated.
Should i send my stream from a procedure? A function? MVC command?

I have my interface and TMVCApp declared as such:

ICustWebApp = interface(IMVCApplication)
  [...]
  function GetReportDoc(const ID: RawUTF8): TMVCAction;
  procedure Reports(const link: RawUTF8; var Scope: Variant);
end;

TCustWebApp = class(TMVCApplication, ICustWebApp)
  [...]
  function GetReportDoc(const ID: RawUTF8): TMVCAction;
  procedure Reports(const link: RawUTF8; var Scope: Variant);
end;

When I click in a link in the Reports.html page, it call GetReportDoc which generate then, theorically, send the report.


I've been trying to replace the return value of my GetReportDoc by TServiceCustomAnswer, but by doing so I need a GetReportDoc.html (dont want to) file and i can see my header and content, but as json when I go to root/GetReportDoc/json?ID=1, not in the HTTP header

Board footer

Powered by FluxBB