#1 Re: mORMot Framework » mORMot 2 Release » 2022-06-10 06:41:20

trx

@AOG
Sure, I have commited the relevant files here: https://github.com/transmogrifix/sx-ddd

But I would not use them for new mORMot2 projects and rather wait for whatever ab is planning DDD/KDD-wise.

#2 Re: mORMot Framework » mORMot 2 Release » 2022-06-09 17:45:50

trx

I have several projects in production which use mORMot2. As with all new code, a few issues came up (in production of course smile ) occasionally but were fixed and things are running smoothly now.
The last one being a locking issue in TSynCache which was introduced when RW locks were added.
I was looking forward to RW locks for a long time, it is a great addition. Although this is an example of a feature which can add hard to detect bugs with unit tests, which got me thinking if mORMot2 could use some kind of branching/versioning/whatever strategy for stable and development/bleeding edge?

About DDD:
The projects were all upgraded from mORMot 1 from 6 months to a few months ago and they all used DDD units, it was pretty easy to port mORMot1's DDD units to mORMot2 though (at least the parts I was using relating to persistence, authentication and emails).

#3 Re: mORMot Framework » [mORMot2] EAccessViolation in TServiceFactoryServer.InstanceFreeGC » 2022-02-02 16:44:59

trx

I am following your work on that for some time, very nice!
I would like to use it, is it now the recommended server for production?

Thank you for your help and for fixing the issue so fast!

#4 Re: mORMot Framework » [mORMot2] EAccessViolation in TServiceFactoryServer.InstanceFreeGC » 2022-02-02 14:05:20

trx

I think it is happening when they expire, my interface services are used by an Angular app only.
It only starts happening after some time after the server is started.
See the following pastebin with the log:
https://pastebin.com/3jpjcpU0

I don't use any callbacks.
I register the services like so:
FMainRest.ServiceRegister(TOrderService, [TypeInfo(IOrderServiceCommand)], sicClientDriven).ResultAsJSONObject := True;

I haven't made any changes to the Angular app or the server in some time, I have only upgraded mORMot2.

#5 mORMot Framework » [mORMot2] EAccessViolation in TServiceFactoryServer.InstanceFreeGC » 2022-02-02 11:47:10

trx
Replies: 5

Hi ab,

My server (Linux, FPC ) logs are getting filled with:

20220202 11342032 EXCOS EAccessViolation (0f) [Pool31adminauthpublicapi] at 737e00 ../mORMot2/src/soa/mormot.soa.server.pas tservicefactoryserver.instancefree (821) ../mORMot2/src/soa/mormot.soa.server.pas tservicefactoryserver.instancefreegc (842) ../mORMot2/src/soa/mormot.soa.server.pas tservicefactoryserver.retrieveinstance (1045) ../mORMot2/src/soa/mormot.soa.server.pas tservicefactoryserver.executemethod (1354) ../mORMot2/src/rest/mormot.rest.server.pas trestserveruricontext.internalexecutesoabyinterfacecomputeresult (3245) ../mORMot2/src/rest/mormot.rest.server.pas trestserveruricontext.internalexecutesoabyinterface (3298) ../mORMot2/src/rest/mormot.rest.server.pas trestserverroutingrest.executesoabyinterface (4455) ../mORMot2/src/rest/mormot.rest.server.pas trestserveruricontext.executecommand (2941) ../mORMot2/src/rest/mormot.rest.server.pas trestserver.uri (6880) ../mORMot2/src/rest/mormot.rest.http.server.pas tresthttpserver.request (1088) ../mORMot2/src/net/mormot.net.server.pas thttpservergeneric.request (1416) ../mORMot2/src/net/mormot.net.server.pas thttpserver.process (2008) ../mORMot2/src/net/mormot.net.server.pas thttpserversocket.taskprocessbody (2106) ../mORMot2/src/net/mormot.net.server.pas thttpserversocket.taskprocess (2062) ../mORMot2/src/net/mormot.net.server.pas tsynthreadpoolthttpserver.task (2474) ../mORMot2/src/core/mormot.core.threads.pas tsynthreadpoolworkthread.dotask (2834) ../mORMot2/src/core/mormot.core.threads.pas tsynthreadpoolworkthread.execute (2873)

I am on git commit fcc29c07aa0b3585a131bf6c30a2b4681927543b (reviewed all singleton locks), I am not sure when it started but it was also happening with commit f2fd54a8d92a4711acfc6802b25efad2282ffd17 (minor OpenSSL library refactoring)
After a while, the server becomes unusable.

Any idea what the culprit is?

Thanks!

#6 Re: mORMot Framework » Will mORMOT/Delphi work together on Linux? » 2021-11-16 12:16:41

trx

FpDebug does not use gdb, it is a dwarf debugger written in pascal.
See https://wiki.freepascal.org/FpDebug for more info.

In any case, it is currently the best debugger backend for Lazarus (it works the best for me that is).

#7 Re: mORMot Framework » Error code: 406 fail on aggregate (ORM.add) » 2021-09-10 14:08:38

trx

Hi,

You haven't initialized fLogType. Try using TSynAutoCreateFields as the base class of TMyLog and remove the write part of the logType property.

TMyLog = class(TSynAutoCreateFields)
    private
      fLogdate: TDate;
      fComment: RawUTF8;
      fLogType: TLogType;
    published
      property logDate: TDate read fLogDate write fLogDate;
      property comment: RawUTF8 read fComment write fComment;
      property logType: TLogType read fLogType;
  end;

For the TSQLRecord properties, you have flattened them propertly, but make sure to use the same letter casing.
For example logType.name should be logType_name and not LogType_Name.

#8 Re: mORMot Framework » mORMot 2 in Good Shape » 2021-01-19 09:15:14

trx

The new mORMot is a masterpiece.
Thank you ab!

#9 Re: mORMot Framework » DDD object flattening » 2020-11-15 19:45:53

trx

Hi,

You can exclude the pilSubClassesFlattening flag by overriding TDDDRepositoryRestFactory.GetAggregateRTTIOptions.
I wanted to do the same, so a virtual method was introduced for that purpose. See: https://github.com/synopse/mORMot/pull/179

#10 Re: mORMot Framework » Renaming TSQLRecord to TORM » 2020-10-29 13:17:27

trx

The new naming scheme seems fine to me.
Prefixing everything with Syn just because someone might have naming conflicts seems a bit of overkill.

#11 Re: mORMot Framework » TObjectListLocked changed, creating dead locks » 2020-07-08 08:10:38

trx
George wrote:

Is there an option to allow multiple read single write logic?
Like TMultiReadExclusiveWriteSynchronizer do.
Useful when object must be thread safe with big amount of read operations and rare write operations.

I usually use TPasMPMultipleReaderSingleWriterLock from PasMP which works with both Delphi and FPC (Windows and Linux).

#12 Re: mORMot Framework » Default values for new fields in external DB » 2020-06-02 20:21:35

trx

That is currently not the case for external DB.
I have created a pull request on GitHub (317) with a potential fix.

EDIT:
It seems that my fix is not a proper solution. InitializeTable must be called somewhere after because for example TSQLRest.UpdateField() is not possible at that point:

Error SQLITE_LOCKED (6) [UPDATE CISInfo SET Test8=? WHERE Test8=?] using 3.31.0 - vtable constructor called recursively: CISInfo, extended_errcode=6
 In file '..\..\fpc\mORMot\SynSQLite3.pas' at line 5392

#13 Re: mORMot Framework » Default values for new fields in external DB » 2020-06-02 13:33:35

trx

Ab,

When I override TSQLRecord.InitializeTable, it gets called when the table is created or the table is empty.
But it does not get called when the table already contains rows and I have added a new field.

Am I missing something?
Thank you.

#15 mORMot Framework » Default values for new fields in external DB » 2020-06-02 11:28:14

trx
Replies: 6

Hi,

I have a TSQLRecord descendant (TMyRecord) which is stored in a an external database (MariaDB via ZEOS).
After table is created by the ORM (CreateMissingTables) and after I insert some rows, if I add another eg. RawUTF8 published property the field is created in the DB automatically.

My issue is that the default value for all existing rows will be NULL.
So if I then try to select an existing row (TMyRecord.CreateAndFillPrepare and FillOne) I get no results.

Currently the only way I have to make those rows available again is to manually connect to the database and change NULL to empty string for all rows.

I haven't been able to find a solution for this in the documentation and the sources.
Any suggestions?

Thank you!
Svetozar Belic.

#16 mORMot Framework » AV in TSynLog.Destroy » 2020-04-11 12:45:56

trx
Replies: 1

Hi ab,

I am not sure if you receive notifications for comments on your commits in GitHub.
If not, please see my comment https://github.com/synopse/mORMot/commi … #r38431461

Thanks,
Svetozar Belic.

#17 Re: mORMot Framework » AV in TSQLRecord.Validate » 2020-03-31 08:31:25

trx

Hi ab,

My TSQLRecord derived class has one TID property with stored AS_UNIQUE. I don't have any other filters or validators.

The clients call a method of an interface based service IOrderApiService.Add, it is registered like so:
FApiRest.ServiceRegister(TOrderService, [TypeInfo(IOrderApiService)], sicClientDriven);

That method prepares the aggregate and then adds it to the DB via a TDDDRepositoryRestCommand descendant which is registered like this:
FMainRest.ServiceDefine(TInfraRepoOrder, [IDomOrderCommand, IDomOrderQuery], sicClientDriven);

I was under the assumption that I would not need any locks in my code if I registered the services like that.

Does this mean I have to stop using stored AS_UNIQUE and add my own thread-safe TSynValidateUniqueField-like class?

Thanks,
Svetozar Belic.

#18 mORMot Framework » AV in TSQLRecord.Validate » 2020-03-30 20:51:09

trx
Replies: 2

Hi,

I occasionally get an AV from TSynValidateUniqueField.Process which is called from TSQLRecord.Validate. This happens when I have multiple threads adding/updating records.
Here is a stack trace leading up to the AV : https://pastebin.com/MaxfK1mG

The cause seems to be the use of filters in TSQLRecordProperties.Filters concurrently. More specifically, multiple threads are using and overwriting TSynValidateRest.ProcessRec:

  if wasTSynValidateRest then begin // set additional parameters
    ValidateRest.fProcessRec := self;
    ValidateRest.fProcessRest := aRest;
  end;         

Am I missing something, or should those two properties be passed to TSynValidateRest.Process as arguments instead to avoid this?

Thanks,
Svetozar Belic.

#19 Re: mORMot Framework » fields of type currency is not persisted » 2020-03-16 12:54:03

trx

I use Currency type and have no such problems.
Have you tried using directly just Currency everywhere (ie. don't use TPrice anywhere)? It should work that way.

Alternatively, don't create a new type for your Currency fields and just create an alias:

type 
  TPrice = Currency;

For the framework to process the property as you would expect, the property type needs to equal TypeInfo(Currency).
By using TPrice = type Currency; you have created a new type, so it is no longer the same type.

#21 Re: mORMot Framework » Inline assembler syntax error » 2020-02-19 15:13:34

trx

I have the same error. I haven't had time to investigate so I've just made a quick fix in SynCommons.pas at line ~60429:

procedure TRawUTF8ListHashed.RehashOnChanged;
...
  collisions := fHash.Rehash({forced=}false); // I've set this to false.
...

I can provide a stack trace if needed.

#22 Re: mORMot Framework » ObjectToJSON and stored AS_UNIQUE » 2020-02-07 18:28:18

trx

Hi,

1. AS_UNIQUE is meant to be used with TSQLRecord descendants only, it does not have the same meaning elsewhere.
Its value is False, so in your case for an ordinary published property it means "Do not store".

2. Its a code completion bug, I haven't found a solution yet.

#23 Re: mORMot Framework » ISQLDBRows and memory » 2020-01-28 22:12:17

trx
macfly wrote:
trx wrote:

Note that if your TSQLRestServer is not registered with your TSQLHttpServer, TSQLRestServer.EndCurrentThread will not be called so you must call it yourself.

I have this scenario and there are no leaks that were reported by FastMM4 or noticed.

By doing this you are not forcing the creation of connections per thread?
Depending on the provider, is this not necessary as mentioned by ab above?

In my case some requests were not processed in the thread pool because of keep-alive, so a dedicated background thread was created.
See the implementation of TSynThreadPoolTHttpServer.Task(aCaller: TSynThread; aContext: Pointer) in SynCrtSock.pas for details.

In that case, when a DB connection (for me via ZEOS to MySQL) is created for each background thread, it is not automatically released when the thread ends.
I have only noticed this because MySQL started complaining about too many connections.

#24 Re: mORMot Framework » ISQLDBRows and memory » 2020-01-28 14:37:16

trx

Note that if your TSQLRestServer is not registered with your TSQLHttpServer, TSQLRestServer.EndCurrentThread will not be called so you must call it yourself.

For example, I have two TSQLRestServers:
1. One for external DB which I don't register with TSQLHttpServer.
2. One in memory which is registered with TSQLHttpServer which provides interface based services. Those services then use the first REST server to get the required data and a DB connection is created per thread.
When the thread ends, TSQLHttpServer will call EndCurrentThread of the server 2. but not 1. so the DB connection will not be released.

To solve this I use a TSQLHttpServer descendant:

TMyHTTPServer = class(TSQLHttpServer)
  private
    FDBRest: TSQLRestServer;
  protected
    procedure HttpThreadTerminate(Sender: TThread); override;
    begin
      inherited HttpThreadTerminate(Sender);
      if Assigned(FDBRest) then
        FDBRest.EndCurrentThread(Sender); 
    end;
end;

#25 Re: mORMot Framework » [Solved] Set initial values of a record » 2020-01-10 16:02:38

trx

If your compiler supports it, you can do:

myVar := Default(TMyRecord)

#27 Re: mORMot Framework » A bug in TSQLPropInfoRTTIDynArray.SetValue » 2018-01-21 20:44:37

trx

Yeah, I see you fixed the source of the problem.

Thank you!

#28 Re: mORMot Framework » A bug in TSQLPropInfoRTTIDynArray.SetValue » 2018-01-21 19:17:07

trx

When commiting the batch, in TSQLRestServerDB.InternalBatchStop that T*DynArray field is determined to be ftBlob at line 2008 (latest master branch) :

Types[f] := Props.Fields.List[prop].SQLDBFieldType;

Later in that same method, a statement is prepared and field values are bound using the determined types:

          for f := 0 to valuesCount-1 do begin
            if GetBit(ValuesNull[0],f) then
              fStatement^.BindNull(f+1) else
              case Types[prop] of
              ftInt64:
                fStatement^.Bind(f+1,GetInt64(pointer(Values[f])));
              ftDouble, ftCurrency:
                fStatement^.Bind(f+1,GetExtended(pointer(Values[f])));
              ftDate, ftUTF8:
                fStatement^.Bind(f+1,Values[f]);
              ftBlob:
                fStatement^.BindBlob(f+1,Values[f]);
              end;

This field's data (a json string) is therefore saved as a ftBlob. Later when you retrieve the row in unit SynSQLite3, TSQLRequest.FieldsToJSON the field is a ftBlob which gets encoded to base64. When this base64 encoded value ends up in TSQLPropInfoRTTIDynArray.SetValue, it is not decoded first and fails to be processed.

I am not sure where this should be fixed, you are more familiar with the code. Correcting this in TSQLPropInfoRTTIDynArray.SetValue seems the like cleanest approach to me because I don't see why it wouldn't accept base64 and be a T*ObjArray.

Thanks ab,

Svetozar Belic.

#29 mORMot Framework » A bug in TSQLPropInfoRTTIDynArray.SetValue » 2018-01-11 03:50:07

trx
Replies: 6

Hi,

I was trying out mORMot's DDD features and have noticed a bug which i believe is in the implementation of TSQLPropInfoRTTIDynArray.SetValue.
For example, using a class like the following (with TSQLRestServerDB and SQLite3):

  TTestSubObjectName = type RawUTF8;
  TTestSubObject = class(TSynPersistent)
  private
    FName: TTestSubObjectName;
  published
    property Name : TTestSubObjectName read FName write FName;
  end;
  TTestSubObjectObjArray = array of TTestSubObject;

  TTestObject = class(TSynAutoCreateFields)
  private
    FSubObjects: TTestSubObjectObjArray;
  published
    property SubObjects : TTestSubObjectObjArray read FSubObjects write FSubObjects;
  end;
  TTestObjectObjArray = array of TTestObject;	

  TJSONSerializer.RegisterObjArrayForJSON([
    TypeInfo(TTestSubObjectObjArray), TTestSubObject,
    TypeInfo(TTestObjectObjArray), TTestObject
  ]);	
	
  // Persistence object.
  TSQLTestObject = class(TSQLRecord)
  protected
    FSubObjects: TTestSubObjectObjArray;
  published
    property SubObjects : TTestSubObjectObjArray read FSubObjects write FSubObjects;
  end;

   
If you create one test object with several subobjects and save (commit) it, everything works as expected, when you select and get the object again, the subobjects are there.
The problem arises when you create, add and commit several such test objects at once. If you select and get them, you will notice that there are no subobjects.

I did some digging around and I see that if you commit one object, its SubObjects value ends up in the database as TEXT whereas if you commit multiple objects, their SubObjects value ends up as BLOB.
I think this is because batch insert uses prepared statements and explicitly sets the field type to BLOB because SubObjects property is a DynArray. Single insert on the other hand saves everything as text.

When retrieving the objects from the DB, the implementation of TSQLPropInfoRTTIDynArray.SetValue expects JSON text.
But because the field was saved as a BLOB, it gets a base64 encoded value instead which does not get handled properly.

procedure TSQLPropInfoRTTIDynArray.SetValue(Instance: TObject; Value: PUTF8Char; wasString: boolean);
var tmp: TSynTempBuffer;
    da: TDynArray;
begin
  GetDynArray(Instance,da);
  if Value=nil then
    da.Clear else
    try
      // fObjArray is not nil, so it will attempt jump to da.LoadFromJSON which does not expect a base64 encoded input.
      if (fObjArray=nil) and Base64MagicCheckAndDecode(Value,tmp) then
        da.LoadFrom(tmp.buf) else 
        da.LoadFromJSON(tmp.Init(Value));
    finally
      tmp.Done;
    end;
end;

The following quick fix solves the problem:

procedure TSQLPropInfoRTTIDynArray.SetValue(Instance: TObject; Value: PUTF8Char; wasString: boolean);
var
  tmp: TSynTempBuffer;
  da: TDynArray;
begin
  GetDynArray(Instance,da);
  if Value = nil then
    da.Clear else
    try
      if not Base64MagicCheckAndDecode(Value,tmp) then
        tmp.Init(Value);
      if fObjArray = nil then
        da.LoadFrom(tmp.Buf) else
        da.LoadFromJSON(tmp.buf)
    finally
      tmp.Done;
    end;
end;

The above problem has another side effect.
For example, you commit one object and after that you commit two in a batch.
You then retrieve all inserted objects, and see that the first object is correct, but the other two are not.
Their SubObjects field has the value of the first object. This happens because when they get filled, the same record (TSQLRecordFill.fTableMap.Dest) is reused.
So the first object gets filled properly, the other two attempt to TSQLPropInfoRTTIDynArray.SetValue but fail, so they end up with the first one's data.

This problem does not occur with eg. TSQLRestServerFullMemory.
I have created a quick test application which reproduces the problem : http://txlab.org/objArrayPublishedPropTest.zip

Thanks,
Svetozar Belic.

#30 mORMot Framework » TSQLRecord with an object that has a TObjectList » 2015-08-30 09:35:07

trx
Replies: 1

Hi,

I have a TSQLRecord like so :

  TMzItem = class(TSQLRecordWithNames)
  protected
    FDescriptions : TMzNameList;
    FImages       : TMzImageList;
  public
    constructor Create; override;
    destructor Destroy; override;
  published
    property Descriptions : TMzNameList read FDescriptions write FDescriptions;
    property Images       : TMzImageList read FImages write FImages;
  end;

constructor TMzItem.Create;
begin
  inherited;
  FDescriptions := TMzNameList.Create;
  FImages := TMzImageList.Create;
end;

destructor TMzItem.Destroy;
begin
  FImages.Free;
  FDescriptions.Free;
  inherited;
end;

TMzNameList and TMzName look like :

  TMzName = class(TPersistentWithCustomCreate)
  private
    FName: RawUTF8;
    FLanguageID: TID;
  public
    constructor Create; overload; override;
    constructor Create(const AName : RawUTF8; const ALangID : TID); overload;
    function GetLanguage(const AClient : TSQLRest): TMzLanguage;
  published
    property LanguageID : TID read FLanguageID write FLanguageID;
    property Name : RawUTF8 read FName write FName;
  end;

  TMzNameList = class(TObjectList)
  protected
    function GetItem(Index: Integer): TMzName;
    procedure SetItem(Index: Integer; const Value: TMzName);
  public
    function Add(AObject: TMzName): Integer; inline;
    property Items[Index: Integer]: TMzName read GetItem write SetItem; default;
  end;

constructor TMzName.Create(const AName: RawUTF8; const ALangID: TID);
begin
  Create;
  FName := AName;
  FLanguageID := ALangID;
end;

constructor TMzName.Create;
begin
  inherited Create;
end;

function TMzName.GetLanguage(const AClient: TSQLRest): TMzLanguage;
begin
  Result := TMzLanguage.Create(AClient, FLanguageID);
end;

function TMzNameList.Add(AObject: TMzName): Integer;
begin
  inherited Add(TObject(AObject));
end;

function TMzNameList.GetItem(Index: Integer): TMzName;
begin
  Result := TMzName(inherited Items[Index]);
end;

procedure TMzNameList.SetItem(Index: Integer; const Value: TMzName);
begin
  inherited Items[Index] := TObject(Value);
end;

initialization

  TJSONSerializer.RegisterClassForJSON([TMzName]);

When i create a TMzItem with some Descriptions they are saved just fine. The problem is with TMzImageList :

  TMzImage = class(TPersistentWithCustomCreate)
  protected
    FId : integer;
    FNames : TMzNameList;
    FIsDefault : boolean;
    FIsVisible : boolean;
  public
    constructor Create; overload; override;
    constructor Create(const AId : integer; const AIsDefault : Boolean; const AIsVisible : Boolean); reintroduce; overload;
    destructor Destroy; override;
  published
    property Names     : TMzNameList read FNames write FNames;
    property ID        : integer read FId write FId;
    property IsDefault : boolean read FIsDefault write FIsDefault;
    property IsVisible : boolean read FIsVisible write FIsVisible;
  end;

  TMzImageList = class(TObjectList)
  protected
    function GetItem(Index: Integer): TMzImage;
    procedure SetItem(Index: Integer; const Value: TMzImage);
  public
    function Add(AObject: TMzImage): Integer; inline;
    property Items[Index: Integer]: TMzImage read GetItem write SetItem; default;
  end;

function TMzImageList.Add(AObject: TMzImage): Integer;
begin
  inherited Add(TObject(AObject));
end;

function TMzImageList.GetItem(Index: Integer): TMzImage;
begin
  Result := TMzImage(inherited Items[Index]);
end;

procedure TMzImageList.SetItem(Index: Integer; const Value: TMzImage);
begin
  inherited Items[Index] := TObject(Value);
end;

constructor TMzImage.Create(const AId: integer; const AIsDefault, AIsVisible: Boolean);
begin
  Create;
  FNames := TMzNameList.Create;
  FId := AId;
  FIsDefault := AIsDefault;
  FIsVisible := AIsVisible;
end;

constructor TMzImage.Create;
begin
  inherited Create;
end;

destructor TMzImage.Destroy;
begin
  FNames.Free;
  inherited;
end;

initialization

  TJSONSerializer.RegisterClassForJSON([TMzImage]);

When i add some TMzImage to Images of TMzItem they are not saved. Instead that property is empty ('[]'). If i remove Names from TMzImage, then Images are saved properly. I have tried to debug Client.Add(item, True) and i can see that the TMzItem is properly converted to JSON along with its images and their names but not saved as such. Images is [].

Have you any suggestions?

Thank you.

trx.

Board footer

Powered by FluxBB