#1 mORMot Framework » Inconsistency between receive timeout param and actual HTTP timeout ? » 2020-09-14 12:31:32

ComingNine
Replies: 1

It seems that TSQLHttpClient descendant will try to retry a service call after certain amount of time has lapsed. Furthermore, the client exists with an error code of 505 after another amount of time. However, there seems to be inconsistency between the receive timeout parameter when HttpClient is created and the timing of the second service call and client exit. Therefore, it seems to be confusing concerning setting up the receive timeout parameter.

The compilable programs can be viewed at viewed at gist. The server program is a simple shell wrapper. The client program tries to run a long-running shell command " date ; sleep 600 ".

parameter     second service call  client exit505
------------- ---------------------   ----------------------                   
default(30s)    180s (3m)                    480s (8m)
receiv + 13    258s (4m18s)                 558s (9m18s)
receiv + 15    270s (4m30s)                 510s (8m30s)
receiv + 17    282s (4m42s)                 522s (8m42s)
receiv + 70    300s (5m)                    600s (10m)
receiv + 100   450s (7m30s)                 690s (11m30s)

: data collected for http time out & retry

As seen in the table above, using the default parameters for creating HttpClient, the second service call occurs after 180s has elapsed. The 1st to 4th rows indicate that there is a multiple of 6 between the constructor receive timeout parameter and the timing of the second service call. However, the 5th and 6th rows indicate that there are no consistency.

Could you help to comment how to set up the receive timeout properly ? yikes
Many thanks !

#2 Re: mORMot Framework » Cannot compile last timeline version » 2020-08-17 13:13:41

Thank you for your efforts ! IIRC, to realize a pointer to external function in Delphi (and FPC as well), one does not use external keyword but SafeLoadLibrary followed by GetProcAddress. I have to try and return.

#3 Re: mORMot Framework » Cannot compile last timeline version » 2020-08-17 11:19:56

If the following two code blocks are equivalent to each other, the regex posted above could be used to automatically transform the hundreds of declarations in SpiderMonkey.pas big_smile

current

type TJS_Initialize = function : Boolean; cdecl;
var JS_Init: TJS_Initialize external SpiderMonkeyLib name 'SM_Initialize';

Delphi compatible

function SM_Initialize: Boolean; cdecl; external SpiderMonkeyLib name 'SM_Initialize';
type TJS_Initialize = function : Boolean; cdecl;
var JS_Init: TJS_Initialize = SM_Initialize; // external SpiderMonkeyLib name 'SM_Initialize';

#4 Re: mORMot Framework » Cannot compile last timeline version » 2020-08-16 17:22:42

The following regex could be helpful to alleviate the burden to support Delphi compatibility of SpiderMonkey.pas

(?gs)type\s([_A-Za-z]+)\s*=\s*(function|procedure)(.*?cdecl;)\s*var(\s[_A-Za-z]+):\s*([_A-Za-z]+)\s(external.*?;)

See also the online tester

#5 Re: mORMot Framework » Cannot compile last timeline version » 2020-08-16 14:10:37

Is there any news on this ? As @array81 reported, latest mORMot and SyNode are incompatible with each other when Delphi is used.

#6 Re: mORMot Framework » AV raised in TestSQL3 cross compiled by FPC. » 2019-07-25 12:50:56

Many thanks !

PS: I have just tried "FPC trunk SVN 40491; Lazarus trunk SVN 59757", as suggested from the documentation, and the same AV exists. big_smile

#7 mORMot Framework » AV raised in TestSQL3 cross compiled by FPC. » 2019-07-25 04:30:47

ComingNine
Replies: 2

The latest mORMot is cross compiled to linux64 by FPC on Windows. The console log is shown below.

 1.2. Low level types:
  - RTTI: 443 assertions passed  363us
  - Url encoding: 200 assertions passed  659us
  - Encode decode JSON: 396,397 assertions passed  132.40ms
  - Variants: 64 assertions passed  212us
  - Mustache renderer: 147 assertions passed  1.75ms
! Low level types - TDocVariant
! Exception EAccessViolation raised with messsage:
!  Access violation
  - TDecimal128: 17,446 assertions passed  1.89ms
  - BSON: 245,068 assertions passed  50.87ms
     100000 TBSONObjectID.ComputeNew in 49.84ms i.e. 2,006,179/s, aver. 0us
  - TSynTableStatement: 221 assertions passed  213us
  - TSynMonitorUsage: 1,202 assertions passed  212us
  Total failed: 0 / 732,939  - Low level types PASSED  714.93ms

The line indicated from the file log is shown below.

          TSynTestEvent(C.TestMethod[t])(); // run tests + Check() and TestFailed()

The version of fpc is

C:\fpcupdeluxe\fpc\bin\i386-win32 > fpc
Free Pascal Compiler version 3.3.1-r39711 [2018/09/08] for i386

Could you check this ? Many thanks !

PS: The same mORMot compiled by FPC on Windows as a Windows executable runs flawlessly.

#8 mORMot Framework » Cannot compile SynLog under Delphi 7 » 2019-07-25 04:18:33

ComingNine
Replies: 1

SynLog in the latest version cannot compile under Delphi 7. It seems that anonymous cannot create ticket any more. The two lines are thus shown below. Could you fix this ? smile

                    code                                      line no
before  R.Seek(PtrUInt(P)-PtrUInt(R.MappedBuffer));           
now     R.Seek(P-PByte(R.MappedBuffer));                      1736

before  W.WriteDirectEnd(PtrUInt(P)-Beg);
now     W.DirectWriteFlush(P-Beg,tmp);                        2044

#9 Re: Language » Working with windows logs (tutorial) » 2019-05-04 04:14:25

mpv wrote:

Since many of us use mORMot as a service I decided to provide small tutorial how to say Windows to format log messages correctly during Windows log output
...
Hope it help for somebody...

Can you help to inform how to work with linux logs, i.e., append to /var/log/messages or /var/log/the_subject_app ? Many thanks !

#10 Re: Delphi » Delphi new generation VCL » 2018-11-03 11:57:50

MSEide+MSEgui has been updated to 4.6.2. Could you help to comment whether it is possible to use mORMot with the current MSEide+MSEgui  ?

https://sourceforge.net/projects/mseide … gui/4.6.2/

#12 mORMot Framework » Latest mORMot.pas does not compile under latest FPC. » 2018-09-12 05:31:51

ComingNine
Replies: 24

Latest mORMot.pas does not compile using latest FPC for both i386-win32 and x86_64-linux. The line and msg shown are below.

Line 56264:             ParamName := @VMP^.Name;  
mORMot.pas(56264,30) Error: Variable identifier expected

The reason is in FPC 3.3.1-r39711 (installed using fpcupdeluxe), Name becomes a property. NamePtr should be used.

ParamName := VMP^.NamePtr;  

PS: It seems that anonymous cannot create ticket now ? There is no create ticket on the tickets link.

#13 Re: mORMot Framework » ExtendedToStr behaves inconsistently between FPC and Delphi » 2018-09-08 15:07:07

mpv wrote:

Dealing with floating with big precision is always a problem. In our applications we try to avoid it by limiting precision part to maximum 4 digits and adding division(factor). For example if we need to store 8 digits precision (0.66599631)  we store 2 field {value: 6659.9631, factor: 10000}. Hope it's help

Many thanks for your clever experience ! Is it because a number with maximum 4 digits can be exactly represented as Currency type ?

#14 Re: mORMot Framework » ExtendedToStr behaves inconsistently between FPC and Delphi » 2018-09-08 10:13:32

ab wrote:

Yes, this is a known inconsistency, since they have two diverse implementations.

Sad to hear. It should be noted that ExtendedToStr behaves also inconsistently between Delphi25Tokyo's Win32 and Win64.

#15 mORMot Framework » ExtendedToStr behaves inconsistently between FPC and Delphi » 2018-09-08 04:05:57

ComingNine
Replies: 5

As shown in the sample code on gist , the ExtendedToStr behaves inconsistently between FPC and Delphi. Could you help to suggest the workaround ?

PS: FloatToStrF and Format seem to behave inconsistent within FPC, and between FPC and Delphi.

#16 mORMot Framework » Float in a variant field of TSQLRecord becomes string when got back fr » 2018-09-03 14:40:13

ComingNine
Replies: 0

As shown in the gist, I am trying to store a float field into a variant field of a TSQLRecord. When the Variant field is assigned by calling ObjectToJSON, it has the correct internals.

   {"Val":1234.56789} 

Furthermore, the DB file has the correct float internals.

However, when the TSQLRecord is retrieved back from DB, its variant field contains string-based internals. As a result, the float field cannot be parsed back by calling ObjectLoadJSON.

   {"Val":"1234.56789"} 

Could you help to comment whether it is possible to prevent the transformation of the float internal into string internal ?

#17 mORMot Framework » Could you share the build env for static\i386 and x86_64 win64 .a file » 2018-08-13 14:40:04

ComingNine
Replies: 1

Could you share the build environment for static\i386-win and static\x86_64-win64 .a files ? For example, the OS and the toolchain versions and the gcc switches ? Many thanks ! smile

#18 Re: mORMot Framework » How to serialize record with enum property ? » 2018-07-30 17:21:05

Dear ab, many thanks !


Could you be kind enough to comment where these text settings you mentioned, i.e., woEnumSetsAsText or twoEnumSetsAsTextInRecord, can be applied to force enum serialization as text, for example, in the gist sample ? I mean, switching from

 (RecordSaveJSON(Rec, TypeInfo(TRec))); 

to

 (RecordSaveJSON(Rec, TypeInfo(TRec), True)); 

does not work.

#19 Re: mORMot Framework » How to serialize record with enum property ? » 2018-07-30 15:53:54

From your documentation , it seems that the solution is to switch from text-based serialization and deserialization to callback-based ones. big_smile Sorry for the trouble.

#20 Re: mORMot Framework » How to serialize record with enum property ? » 2018-07-30 14:54:04

From your documentation , it is suggested that "For other types (like enumerations or sets), you can simply use the unsigned integer types corresponding to the binary value, e.g. byte word cardinal Int64 (depending on the sizeof() of the initial value)." With such treatment, the updated gist code serializes the enum property within the record as unsigned integer.

Nevertheless, the enum property within the TObject can be serialized to human readable text, could you help to comment whether it is possible to serialize enum property within the record also as human readable text ?

{
        "E": 0,
        "K": "",
        "V": 0
}
{
        "ClassName":"TObj",
        "E": "meFirst",
        "K": "",
        "V": 0
}

#21 mORMot Framework » How to serialize record with enum property ? » 2018-07-30 14:28:54

ComingNine
Replies: 4

As shown in the gist code , the call to TTextWriter.RegisterCustomJSONSerializerFromText fails if the record definition contains enum type: ESynException 'Unregistered ptCustom for TJSONRecordTextDefinition.AddItem(E: TMYENUM)'.

Could you help to comment the correct way to deal with record type with enum property in JSON serialization ?
Many thanks !

#22 Re: mORMot Framework » Problem when deserializing class with array prop backed by methods. » 2018-07-17 02:39:58

Dear ab, it seems to me that the deserialization of the class fails because GetDynArray of an array prop backed by methods will set DynArray.fValue to nil, and consequently "From := DynArray.LoadFromJSON()" concludes all of the remaining JSON is invalid and jumps to the end of the JSON. Could you help to comment ?

Furthermore, the gist sample has been clarified. As shown in DeserializeClassWithPropertyBackedByMethods2.dpr, there is no two properties pointing the actual storage.

#23 mORMot Framework » Problem when deserializing class with array prop backed by methods. » 2018-07-16 09:06:39

ComingNine
Replies: 2

As shown in the sample, a class with an array prop backed by methods is serialized and then deserialized. The deserialization of the class fails because, since GetDynArray of that prop sets DynArray.fValue to nil, DynArray.LoadFromJSON concludes all of the remaining JSON is invalid and jumps to the end of the JSON. It would seem more proper if DynArray.LoadFromJSON only skips the very array property backed by methods instead of all the remaining JSON. Could you help to comment ?

#24 Re: mORMot Framework » Regression of ManyAdd/Many.../SourceGet/DestGet with "current" omitted » 2018-03-05 09:04:17

As discussed in https://synopse.info/forum/viewtopic.php?id=164, ManyAdd/ManyDelete/ManySelect/SourceGet/DestGet can be used with one less parameter - the "current" fSourceID/fDestID.

That is to say, in about 184th line of the "mORMot and Open Source friends-d573e7358a/SQLite3/Samples/ThirdPartyDemos/Migajek/synopse-sqlite-demo/Unit1.pas", both lines below should work. Nevertheless, only the second one works in the latest source.

      cust.Tasks.ManyAdd(globalClient, task.ID, true)
      cust.Tasks.ManyAdd(globalClient, cust.ID, task.ID, true)

Could you help to comment whether the situation is made clearer ? big_smile

#25 Re: mORMot Framework » Regression of ManyAdd/Many.../SourceGet/DestGet with "current" omitted » 2018-03-05 08:47:40

Sorry for the confusion.

I did not manage to find the fix... 

The diff was intended to show how to reproduce the regression . Could you help to check and fix ?

#26 mORMot Framework » Regression of ManyAdd/Many.../SourceGet/DestGet with "current" omitted » 2018-03-05 06:08:50

ComingNine
Replies: 4

As discussed in https://synopse.info/forum/viewtopic.php?id=164, ManyAdd/ManyDelete/ManySelect/SourceGet/DestGet can be used with one less parameter - the "current" fSourceID/fDestID.

However, in the latest sample code "mORMot and Open Source friends-d573e7358a\SQLite3\Samples\ThirdPartyDemos\Migajek\synopse-sqlite-demo", the overloaded methods with one less parameter does not work, as tested with the following modifications

HiWin10@MICROSO-8C7VFP4 /cygdrive/d/mORMot and Open Source friends-d573e7358a/SQLite3/Samples/ThirdPartyDemos/Migajek/synopse-sqlite-demo
$ diff Unit1.pas ../synopse-sqlite-demo.original/
184,185c184
<   ACustomer.Tasks.DestGet(globalClient, fIds);
<   // ACustomer.Tasks.DestGet(globalClient, ACustomer.ID, fIds);
---
>   ACustomer.Tasks.DestGet(globalClient, ACustomer.ID, fIds);
371,372c370
<                     cust.Tasks.ManyAdd(globalClient, task.ID, true)
<                     // cust.Tasks.ManyAdd(globalClient, cust.ID, task.ID, true)
---
>                     cust.Tasks.ManyAdd(globalClient, cust.ID, task.ID, true)
374,375c372
<                     cust.Tasks.ManyDelete(globalClient, task.ID);
<                     // cust.Tasks.ManyDelete(globalClient, cust.ID, task.ID);
---
>                     cust.Tasks.ManyDelete(globalClient, cust.ID, task.ID);

Could you help to fix the behavior ? Many thanks for your efforts ! smile

#27 Re: SyNode » Complaining invalid arrow-function arguments in latest source. » 2018-02-13 17:40:17

Thank you very much for your helpful suggestions and comments !

PS: SM52 + Linux sounds quite interesting ! Great contributions from you !

#28 Re: SyNode » Complaining invalid arrow-function arguments in latest source. » 2018-02-13 12:16:04

The exception is seen with Win XE8/XE Tokyo, x32, SM45.

When SM52 is defined, the exception is not seen. Many thanks for your instructive test. Nevertheless, could you help to comment whether SM45 is deprecated in your company ?

#29 SyNode » Complaining invalid arrow-function arguments in latest source. » 2018-02-13 03:36:48

ComingNine
Replies: 5

With the latest mORMot & SyNode, the sample "02 - Binding" raises exception of "invalid arrow-function arguments (parentheses around the arrow-function may help)" during startup. Could you help to fix it ?

#30 Re: mORMot Framework » Could you add UnRegisterClassForJSON methods into mORMot.pas ? » 2017-12-22 16:58:57

In addition to the post above, fLastClass needs to be reset to nil after removal... smile

procedure TJSONSerializerRegisteredClass.RemoveOnce(aItemClass: TClass);
begin
  fSafe.Lock;
  try
    if PtrUIntScanExists(pointer(List),Count,PtrUInt(aItemClass)) then begin
      Remove(aItemClass);
      if fLastClass = aItemClass then
        fLastClass := nil;
    end;  
  finally
    fSafe.UnLock;
  end;
end;

#31 mORMot Framework » Could you add UnRegisterClassForJSON methods into mORMot.pas ? » 2017-12-22 09:25:20

ComingNine
Replies: 1

When utilizing JSON serialization, one would try and test different versions of a specific class (isolated in different units). Under such circumstances, UnRegisterClassForJSON methods could be useful big_smile

....

    procedure RemoveOnce(aItemClass: TClass);

....

    class procedure UnRegisterClassForJSON(aItemClass: TClass); overload;
    class procedure UnRegisterClassForJSON(const aItemClass: array of TClass); overload;

....

procedure TJSONSerializerRegisteredClass.RemoveOnce(aItemClass: TClass);
begin
  fSafe.Lock;
  try
    if PtrUIntScanExists(pointer(List),Count,PtrUInt(aItemClass)) then
      Remove(aItemClass);
  finally
    fSafe.UnLock;
  end;
end;

....

class procedure TJSONSerializer.UnRegisterClassForJSON(aItemClass: TClass);
begin
  if JSONSerializerRegisteredClass=nil then
    GarbageCollectorFreeAndNil(JSONSerializerRegisteredClass,
      TJSONSerializerRegisteredClass.Create);
  JSONSerializerRegisteredClass.RemoveOnce(aItemClass);
end;

class procedure TJSONSerializer.UnRegisterClassForJSON(const aItemClass: array of TClass);
var i: integer;
begin
  for i := 0 to high(aItemClass) do
    UnRegisterClassForJSON(aItemClass[i]);
end;

#32 Re: mORMot Framework » Could you add to SynTests.pas the generation of dunit-report.xml ? » 2017-12-22 05:42:53

In the source code shown, SynTests is slightly modified to generate dunit-report.xml. The generated file can be useful for automating the testing of multiple executable files.


Can you help to review and include the changes  ? big_smile

#33 mORMot Framework » Could you add to SynTests.pas the generation of dunit-report.xml ? » 2017-12-05 07:08:05

ComingNine
Replies: 2

When using DUnit, a dunit-report.xml containing test statistics will be generated. This can be useful for automating the testing of multiple executable files, because one can use tools (e.g., FinalBuilder) to check whether there are failures during automated tests. 

Therefore, could you help to add the functionality of generating dunit-report.xml to SynTests.pas ? smile

#34 Re: mORMot Framework » Record property serialization in Delphi < 2010, no type info? » 2017-12-03 16:32:09

ab wrote:

...
And the root cause of the problem is that a record published property has no RTTI in the property field of the class, anyway. So adding a ref-counted field won't be enough...

Many thanks for your knowledgeable comments !

However, for the "won't be enough" part, as shown in a trivial example which is compilable under D7 and a SO post, it seems that adding a ref-counted field will be enough to generate RTTI for the record, for other records that contain it, and for the class that somehow publishes the record. Could you comment about or a possible counter-example ? big_smile

#35 Re: mORMot Framework » Record property serialization in Delphi < 2010, no type info? » 2017-12-03 13:36:17

In D7, you could try to define an additional ref-counted field within the record to force rtti generation for your simple record types.
See http://codeverge.com/embarcadero.delphi … pe/1040908

#36 Re: mORMot Framework » Record property serialization in Delphi < 2010, no type info? » 2017-12-03 02:22:14

See example of serializing record in "10.1.3.2.4. Text-based definition" smile

#37 Re: Source Code repository » SQLite3: "Many to Many" relationship updates » 2017-11-26 08:21:56

It seems that TSQLRecordMany.FillMany and TSQLRecordMany.DestGetJoined provide rather similar functionality: retrieving dest items related to certain source item. Could you help to comment the difference between them ? For example, what are the different situations where either FillMany or DestGetJoined fits better ?  yikes

After playing around a bit, now the difference becomes more clear to me... Sorry for the trouble! big_smile

...ThirdPartyDemos\Migajek\synopse-sqlite-demo...

procedure TForm1.LoadTasksForCustomer(const ACustomer: TSQLCustomer; const AList: TStrings);
var
 tasks: TSQLTasks;
 MsgUTF8: RawUTF8;
 Msg: string;
 task: TSQLTask;
 fIds: TIDDynArray;
begin
  // Can be created directly
  tasks := TSQLTasks.Create;
  try
    // How to get Dest IDs
    // tasks.DestGet(globalClient, fIds); // Does not work
    tasks.DestGet(globalClient, ACustomer.ID, fIds); // Works
    // ACustomer.Tasks.DestGet(globalClient, ACustomer.ID, fIds); // Original
    task:= TSQLTask.CreateAndFillPrepare(globalClient, TInt64DynArray(fIds));
    AList.BeginUpdate();
    AList.Clear();
    try
      while task.FillOne do
        AList.AddObject(Format('%s', [UTF8ToString(task.Text)]), Pointer(task.id));
    finally
      AList.EndUpdate();
      FreeAndNil(task);
    end;

    // How to get Pivot instances, and Dest instances later
    tasks.FillMany(globalClient, ACustomer.ID);
    while tasks.FillOne do begin
      // MsgUTF8 := tasks.Dest.Text;
      task := TSQLTask.Create(globalClient, tasks.Dest.ID);
      try
        MsgUTF8 := task.Text;
        Msg := UTF8ToString(MsgUTF8);
        Msg := 'Get Dest through Pivot: ' + UTF8ToString(MsgUTF8);
        OutputDebugString(PChar(Msg));
      finally
        task.Free;
      end;
    end;

    // How to get Dest instances directly
    task := tasks.DestGetJoined(globalClient, '', ACustomer.ID) as TSQLTask;
    try
      while task.FillOne do begin
        MsgUTF8 := task.Text;
        Msg := 'Directly get Dest: ' + UTF8ToString(MsgUTF8);
        OutputDebugString(PChar(Msg));
      end;
    finally
      task.Free;
    end;

  finally
    tasks.Free;
  end;
end;

procedure TForm1.lbTasksClick(Sender: TObject);
var
  tasks: TSQLTasks;
  task: TSQLTask;
  cust: TSQLCustomer;
  clientsIds: TIDDynArray;
  i, j: integer;
  MsgUTF8: RawUTF8;
  Msg: string;
begin
  gbEditTask.Visible:= lbTasks.ItemIndex <> -1;
  if not gbEditTask.Visible then
    exit;

  gbEditTask.Visible:= lbTasks.Items.Objects[lbTasks.ItemIndex] <> nil;
  if not gbEditTask.Visible then
    exit;

  task:= LoadTask(integer(lbTasks.Items.Objects[lbTasks.ItemIndex]));
  cbTaskPriority.ItemIndex:= Ord(task.Priority);
  cbTaskPriority.Tag:= task.ID;

  FillCustomersList(CheckListBox1.Items, true);
  CheckListBox1.Tag:= task.ID;
  // load list of customers assigned to the given task
  cust:= TSQLCustomer.Create();
  try
    cust.Tasks.SourceGet(globalClient, task.ID, clientsIds);
    for i:= low(clientsIds) to high(clientsIds) do
      begin

        // find the client on the list (by ID)
        for j:= 0 to CheckListBox1.Count -1 do
          if Integer(CheckListBox1.Items.Objects[j]) = clientsIds[i] then
              CheckListBox1.Checked[j]:= true;
      end;
  finally
    cust.Free();
    FreeAndNil(task);
  end;

  // Learn about FillManyFromDest
  tasks := TSQLTasks.Create;
  try
    tasks.FillManyFromDest(globalClient, integer(lbTasks.Items.Objects[lbTasks.ItemIndex]));
    while tasks.FillOne do begin
      cust := TSQLCustomer.Create(globalClient, tasks.Source.ID);
      try
        MsgUTF8 := cust.FirstName;
        Msg := UTF8ToString(MsgUTF8);
        Msg := 'Get Source through Pivot: ' + UTF8ToString(MsgUTF8);
        OutputDebugString(PChar(Msg));
      finally
        cust.Free;
      end;
    end;
  finally
    tasks.Free;
  end;
end;

#38 Re: mORMot Framework » Typo in example code of many-to-many relationship in the documentation » 2017-11-26 07:59:03

Another typo

    /// retrieve all records associated to a particular Dest record, which
    // has a TSQLRecordMany property
    // - returns the Count of records corresponding to this aSource record    <<<<<<<<<<<<<<<<<<<<<<< "this aSource record" should be "this aDest record" ?
    // - use a "for .." loop or a "while FillOne do ..." loop to iterate
    // through all Dest items, getting also any additional 'through' columns    <<<<<<<<<<<<<<<<<<<<<<< "through all Dest items" should be "through all Source items" ?
    // - the optional aAndWhereSQL parameter can be used to add any additional
    // condition to the WHERE statement (e.g. 'Salary>:(1000): AND Salary<:(2000):')
    // according to TSQLRecordMany properties - note that you should better use
    // inlined parameters for faster processing on server, so you may call e.g.
    // ! aRec.FillManyFromDest(Client,DestID,FormatUTF8('Salary>? AND Salary<?',[],[1000,2000]));
    function FillManyFromDest(aClient: TSQLRest; aDestID: TID;
      const aAndWhereSQL: RawUTF8=''): integer;

Moreover, would you also include test for FillManyFromDest in SynSelfTests.pas ? big_smile

#39 mORMot Framework » Typo in example code of many-to-many relationship in the documentation » 2017-11-26 04:16:06

ComingNine
Replies: 1

https://synopse.info/files/html/Synopse … 01.18.html

...
5.5.2.2.1. Introducing TSQLRecordMany
...

    for i := 1 to high(dID) do
    begin
      Check(MS.DestList.SourceGet(aClient,dID[i],res));
      if not Check(length(res)=1) then   <<<<<<<<<<<<<<<<<<<<<<<<<<< should be CheckFailed as shown in SynSelfTests.pas
        Check(res[0]=sID[i]);
      Check(MS.DestList.ManySelect(aClient,sID[i],dID[i]));
      Check(MS.DestList.AssociationTime=i);
    end;
for i := 1 to high(sID) do
begin
  Check(MS.DestList.DestGet(aClient,sID[i],res));
  if Check(length(res)=1) then   <<<<<<<<<<<<<<<<<<<<<<<<<<< should be CheckFailed as shown in SynSelfTests.pas
    continue; // avoid GPF
  Check(res[0]=dID[i]);

big_smile

#40 Re: mORMot Framework » Display wrong characters even though data are intact after AES enc/dec » 2017-11-20 09:51:52

Many thanks for your help ! tongue
It works after the following change:

...
  {$IFDEF FPC}
  plain := mmoPlain.Text;
  {$ELSE ~FPC}
  plain := StringToUTF8(mmoPlain.Text);
  {$ENDIF ~FPC}
...
  {$IFDEF FPC}
  mmoPlain.Text := plain;
  {$ELSE ~FPC}
  mmoPlain.Text := UTF8ToString(plain);
  {$ENDIF ~FPC}
...

#41 Re: mORMot Framework » Why cannot deserialize TStrings prop if class inherits TPersistent ? » 2017-11-20 01:17:22

ab wrote:

IIRC there is an explicit check in SynCommons.pas which check for the TSynPersistent class type.

Many thanks for your helpful comments !
Indeed the explicit check for TSynPersistent class type is the root reason ! If SynCommons.TSynPersistent is found in the inheritance chain, the TSynPersistentClass(ItemClass).Create call ensures that the correct constructor is called. For TPersistent for example, the ItemClass.Create call goes to TObject.Create, in which case the deserialization will not work unless the field is initialized in the overridden AfterConstruction procedure.


procedure TClassInstance.Init(C: TClass);
begin
  ItemClass := C;
  if C<>nil then
  repeat // this unrolled loop is faster than cascaded if C.InheritsFrom()
    ...
    if C<>TSynPersistent then
    ...
      ItemCreate := cicTSynPersistent;
      exit;
    ...   
function TClassInstance.CreateNew: TObject;
begin
  ...
  case ItemCreate of
    ...
    cicTSynPersistent: begin
      result := TSynPersistentClass(ItemClass).Create;
      exit;
    end;
    ...
    cicTObject: begin
      result := ItemClass.Create;
      exit;
    end;
  ...
end;

#42 mORMot Framework » Why cannot deserialize TStrings prop if class inherits TPersistent ? » 2017-11-19 13:33:05

ComingNine
Replies: 2

I am trying to learn about the JSON deserialization inside mORMot. As shown in the code below (and in the compilable code in gist), I am trying to deserialize a class with a published TStrings property from JSON. The deserialization only works if the class inherits from SynCommons.TSynPersistent, and does not work if the class inherits from Classes.TPersistent. Interestingly, the deserialization also does not work if the TSynPersistent code is copied exactly into the .dpr and the class is made to inherit from this copy.

type
{$M+}

  // Exactly the same as in SynCommons.pas
  TSynPersistent = class(TObject)
  ...

//  TSample = class(TPersistent) // Does not work
  TSample = class(TSynPersistent) // Does not work
//  TSample = class(SynCommons.TSynPersistent) // Works
  private
    FContent: TStrings;
  published
    property Content: TStrings read FContent;
  public
//    constructor Create; // if inherits from TPersistent
    constructor Create; override; // if inherits from TSynPersistent or SynCommons.TSynPersistent
    destructor Destroy; override;

  end;
var
  Sample: TSample;
  JsonContentRaw: RawByteString;
  JsonContentUTF8: RawUTF8;
  JsonContentUTF8Buf: PUTF8Char;
  JsonContentValid: Boolean;
begin
    TJSONSerializer.RegisterClassForJSON([TSynPersistent, TSample]);

    Sample := TSample.Create;
    Sample.Content.Add('new line 3');
    SynCommons.FileFromString(ObjectToJSON(Sample, [woHumanReadable, woStoreClassName]), ChangeFileExt(ParamStr(0), '.Sample.json'));
    FreeAndNil(Sample);

    JsonContentRaw := SynCommons.StringFromFile(ChangeFileExt(ParamStr(0), '.Sample.json'));
    JsonContentUTF8 := CurrentAnsiConvert.AnsiToUTF8(JsonContentRaw);
    JsonContentUTF8Buf := @JsonContentUTF8[1];
    Sample := JSONToNewObject(JsonContentUTF8Buf, JsonContentValid) as TSample;
    Writeln(Format('%s', [BoolToStr(JsonContentValid, True)]));
    if JsonContentValid then
      Writeln(Format('%s', [Sample.Content.Text]));
end.

After following into mORMot.pas, it seems that the obvious reason could be that the GetterAddr(Instance) call returns the pointer to the underlying field if the class inherits from SynCommons.TSynPersistent, but returns a pointer to nil if the class inherits from TPersistent or, interestingly, the exact copy of TSynPersistent. Could you help to comment why a pointer to nil is returned if the class inherits from TPersistent or, more interestingly, the exact copy of TSynPersistent ? yikes

function TPropInfo.ClassFromJSON(Instance: TObject; From: PUTF8Char;
  var Valid: boolean; Options: TJSONToObjectOptions): PUTF8Char;
...  
  if GetterIsField then
    // no setter -> use direct in-memory access from getter (if available)
    Field := GetterAddr(Instance) else
    // no setter, nor direct field offset -> impossible to set the instance
    exit;
  result := JSONToObject(Field^,From,Valid,nil,Options);
...  

#43 Re: mORMot Framework » Display wrong characters even though data are intact after AES enc/dec » 2017-11-18 16:42:42

For Delphi, TMemo now shows correct characters after calls of CurrentAnsiConvert routines are modified to be "symmetric".  yikes

procedure TMainForm.btnEncryptWrongClick(Sender: TObject);
var
  Key: TSHA256Digest;
  plain, cipher: RawByteString;
begin
  SHA256Weak(CurrentAnsiConvert.UTF8ToAnsi(StringToUTF8(AES_KEY)), Key);
  plain := StringToUTF8(mmoPlain.Text);
  cipher := SynCrypto.AES(Key, 256, plain, true);
//  cipher := SynCrypto.AES(Key, 256, CurrentAnsiConvert.UTF8ToAnsi(plain), true);
  mmoCipher.Text :=  UTF8ToString(BinToBase64(cipher));
end;

procedure TMainForm.btnDecryptWrongClick(Sender: TObject);
var
  Key: TSHA256Digest;
  plain, cipher: RawByteString;
begin
  SHA256Weak(CurrentAnsiConvert.UTF8ToAnsi(StringToUTF8(AES_KEY)), Key);
  cipher := Base64ToBin(StringToUTF8(mmoCipher.Text));
  plain := SynCrypto.AES(Key, 256, cipher, false);
//  plain := CurrentAnsiConvert.AnsiToUTF8(SynCrypto.AES(Key, 256, cipher, false)); // Calling of CurrentAnsiConvert routines has to be "symmetric"
  mmoPlain.Text := UTF8ToString(plain);
end;

However, for FPC (on Windows), TMemo displays some characters correctly but question marks for others... O_O Could you help to comment about the reason of this behavior and possible workaround ?

The updated test files can be viewed at https://gist.github.com/anonymous/1995e … 1bad7f2974

#44 mORMot Framework » Display wrong characters even though data are intact after AES enc/dec » 2017-11-18 14:24:46

ComingNine
Replies: 3

I am trying to encrypt the content from the TMemo (Chinese characters), and then decrypt and show them. The data (bytes) seems to be the same after AES encryption/decryption, as can be seen by going to the address "plain[1]" in the Memory debug window. However, the TMemo cannot display the original/correct characters. Could you help to comment what is the correct way out here ? Many thanks ! O_O

PS: The situation is tested with Delphi 7 ( comment out type ChineseString = type Ansistring(936) ) and Delphi Tokyo.
PS: Under Delphi Tokyo, the line calling SHA256Weak generates complaint "W1058 Implicit string cast with potential data loss from 'string' to 'RawByteString'." I thought methods with RawByteString parameter are ready for any string argument... O_O Could you help to comment the correct way to call SHA256Weak ?

uses
  SynCommons, SynLog, SynCrypto;

const
  TEST_KEY: string = 'TEST_KEY';

procedure TMainForm.btnEncryptClick(Sender: TObject);
var
  Key: TSHA256Digest;
  plain, cipher: RawByteString;
begin
  SHA256Weak(TEST_KEY, Key);
  plain := StringToUTF8(mmoPlain.Text);
//  mmoPlain.Text := UTF8ToString(plain);
//  mmoPlain.Text := UTF8ToString(AnyAnsiToUTF8(plain));
//  Exit;
  cipher := SynCrypto.AES(Key, 256, plain, true);
  mmoCipher.Text :=  UTF8ToString(BinToBase64(cipher));
end;

type
  ChineseString = type Ansistring(936); // https://stackoverflow.com/questions/7222615/how-can-i-convert-string-encoded-with-windows-codepage-1251-to-a-unicode-string

procedure TMainForm.btnDecryptClick(Sender: TObject);
var
  Key: TSHA256Digest;
  plain, cipher: RawByteString;
  ChineseStr: ChineseString;
begin
  SHA256Weak(TEST_KEY, Key);
  cipher := Base64ToBin(StringToUTF8(mmoCipher.Text));
  plain := SynCrypto.AES(Key, 256, cipher, false);

  mmoPlain.Text := UTF8ToString(AnyAnsiToUTF8(plain)); // Does not work
//  mmoPlain.Text := UTF8ToString(CurrentAnsiConvert.AnsiToUTF8(plain)); // Does not work
//  mmoPlain.Text := UTF8ToUnicodeString(CurrentAnsiConvert.AnsiToUTF8(plain)); // Does not work

//  ChineseStr := UTF8ToString(AnyAnsiToUTF8(plain)); // Does not work
//  mmoPlain.Text := ChineseStr;

// mmoPlain.Text := TEncoding.Default.GetString(TBytes(@plain[1])); // Does not work
end;

The test files can be viewed at http://gist.github.com/anonymous/fa917e … 447137bcbb

#45 Re: mORMot Framework » Reset color to ccBlack instead of ccLightGray in TSynLog.ConsoleEcho ? » 2017-11-12 13:45:41

yikes Sorry. It seems that linking the Crt unit will drastically change how SynLog colorize the echoed log. For example, sllInfo level log will be bold instead of being ccWhite.

For me, it looks more natural to reset color to ccBlack instead of ccLightGray at the end of TSynLog.ConsoleEcho with Linux.

#46 Re: mORMot Framework » Reset color to ccBlack instead of ccLightGray in TSynLog.ConsoleEcho ? » 2017-11-12 13:10:02

It seems NormVideo is not void ?

Procedure NormVideo;
{
  Set normal back and foregroundcolors.
}
Begin
  TextColor(7);
  TextBackGround(0);
End; 

yikes I just tried to add "Crt.NormVideo" at the end of the above .lpr under FPC and it worked.

#47 Re: mORMot Framework » Reset color to ccBlack instead of ccLightGray in TSynLog.ConsoleEcho ? » 2017-11-12 11:10:43

Many thanks for your effort ! I will insert "Crt.NormVideo" at the end of .lpr under FPC smile

#48 Re: mORMot Framework » Reset color to ccBlack instead of ccLightGray in TSynLog.ConsoleEcho ? » 2017-11-11 11:15:34

After searching the internet I am still not sure. For example, on some distributions the default terminal color is dependent on the "theme". Perhaps it is better to leave it as is...

#49 mORMot Framework » Reset color to ccBlack instead of ccLightGray in TSynLog.ConsoleEcho ? » 2017-11-10 16:45:25

ComingNine
Replies: 7

Since normally the console color is black, could you consider to reset color to ccBlack instead of ccLightGray at the end of TSynLog.ConsoleEcho ?
Currently, if EchoToConsole is set to LOG_VERBOSE, the color of the prompt for the next command becomes from black to light gray, which seems odd... big_smile

Example:

program Project1;

{$IFNDEF FPC}
{$APP CONSOLE}
{$ENDIF}

uses
  SysUtils,
  SynCommons, SynLog;

procedure xx;
var
 ILog: ISynLog;
 SMethodName: string;
 SMethodNameUTF8: RawUTF8;
begin
  SMethodName := 'enter xx';
  SMethodNameUTF8 := StringToUTF8(SMethodName);
  (*
    [url]https://synopse.info/forum/viewtopic.php?pid=25540[/url]
    @ returns a pointer to a variable: @str is a pointer to the string pointer, not to the string content
    @str[1] is a pointer to the first char, i.e., the pointer to the string content
    Pointer() type-cast the variable to another type: hard-cast of a string variable as Pointer returns the pointer to the string content
  *)
  // Does not work.
  // ILog := TSynLog.Enter(nil, @SMethodNameUTF8);
  // Either way works.
  ILog := TSynLog.Enter(nil, @SMethodNameUTF8[1]);
  // ILog := TSynLog.Enter(nil, Pointer(SMethodNameUTF8));
  ILog.Log(sllWarning, 'hi xx');
end;

begin
  with TSynLog.Family do
  begin
    {$IFDEF MSWINDOWS}
      AutoFlushTimeOut := 1;
    {$ENDIF ~MSWINDOWS}
    Level := LOG_VERBOSE;
    EchoToConsole := LOG_VERBOSE;
    PerThreadLog := ptIdentifiedInOnFile;
    RotateFileCount := 50;
    RotateFileSizeKB := 20 * 1024; // rotate by 20 MB logs
  end;

  xx;
end.

rh4c5i.png

Board footer

Powered by FluxBB