Posts by Roberto Schneiders

Roberto Schneiders · mORMot 1

I agree. But the tests I did with DBExpress and Aurelius also used ODBC.

Is not a problem with the mORMot ODBC?

If I find something I notice.

Roberto Schneiders · mORMot 1

I'm doing a some basics performance tests. My first test is basically insert 10.000 records.

I tested using ClientDataSet (DBExpress), Aurelius and mORMot.

I expected the ORM's were slower than DBExpress in this test. This is confirmed. But I did not expect so weak performance of mORMot (Usually the mORMot destroys its competitors in these tests ).

I must have done something wrong or there must be some problem (perhaps related to ODBC).

This is the code I use.

var
  oPeople: TPeople;
  i:Integer;
  results: TIntegerDynArray;
  StopWatch:TStopWatch;
begin
  StopWatch := TStopwatch.StartNew;
  StopWatch.start;

  fClient.BatchStart(TPeople);
  fclient.TransactionBegin(TPeople);
  for i:=0 to 9999 do
  begin
    oPeople := TPeople.Create;
    oPeople.Name := 'People '+IntToStr(i);
    oPeople.Age  := i;
    fClient.BatchAdd(oPeople, true);
    oPeople.free;

    if (i mod 1000) = 0 then
    begin
      fClient.BatchSend(results);
      fClient.BatchStart(TPeople);
    end;
  end;
  fClient.BatchSend(results);
  fClient.Commit();
  StopWatch.Stop;
  showmessage(IntTOStr(StopWatch.ElapsedMilliseconds));
end;

I read this topic
http://synopse.info/forum/viewtopic.php?id=769
I tried to use transactions and BATCH, but none brought me gain than I expected.

DBExpress: 12.1 s
Aurelius: 44.3 s (Not exactly the same table as used in other tests, but is similar. I'll do more tests tomorrow.)
mORMot: 140s (Using BATCH and Transactions)

NOTE: All tests are using ODBC.

Roberto Schneiders · mORMot 1

I changed the TPeople class definition. I added a Birthdate property.

  TPeople = class (TSQLRecord)
  private
    FName: RawUTF8;
    FAge: Integer;
    FBirthDate: TDateTime;
  published
    property Name:RawUTF8 read FName Write FName;
    property Age:Integer read FAge Write FAge;
    property BirthDate:TDateTime Read FBirthDate Write FBirthDate;
  end;

The framework did not create the new field on the firebird database.

The DB.GetFieldNames method (in TSQLRestServerDB.CreateMissingTables) returns these names:

('Name', 'Age', 'BirthDate')

But BirthDate field does not exist in the database.

Roberto Schneiders · mORMot 1

it worked!

Roberto Schneiders · mORMot 1

Not exactly. Does not appears to be empty.

After the ReHash (TSQLRestServerStaticExternal.Create) method fHashs (fFieldsDynArray.fHashs) property has this value:

((0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), ( 0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0) , (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), ( 0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0) , (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), ( 0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0) , (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), ( 0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0) , (3930815333} {$ EA4B7B65, 2), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0) , (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), ( 0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0) , (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), ( 0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (1307410844 4DED819C} {$, 0), (0, 0) , (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), ( 0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0) , (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), ( 0, 0), (0, 0), (3472868801 CEFFC5C1} {$, 1), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0) , (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), ( 0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0) , (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), ( 0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0) , (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0))

Apparently have 3 values.

Roberto Schneiders · mORMot 1

I still can not identify why the fFieldsDynArray.FindHashed not find the fields of the class / table.

Any tips?

Roberto Schneiders · mORMot 1

Yes, Connection.DBMS = dFirebird;

I was wrong again. I did not realize that this line was the SQL parser.

The Create method is correct.

Roberto Schneiders · mORMot 1

Absolutely. I had seen this in the documentation. Not applied yet because it is the first test.

Thanks for the tip.

Roberto Schneiders · mORMot 1

Let me make a suggestion Arnaud?

Use Trello to organize the Roadmap.

I am using Trello for some time and is a fantastic tool . I think it would be interesting you create a table with items of the roadmap. So the whole community will know that you are working on. If more people to join you in the development, also serves as a tool to coordinate the work. You can create a board for every Synopse project.

The community could vote on features they would most like to see implemented (of course this does not mean that you will strictly follow this priority).

For each card on board you can add a checklist with all items that you need to implement to meet that functionality. So everyone, including you, can follow the progress of a task.

The community can comment on the functionality.

You can put an end prediction for a given task, and thus the whole community know that you are working to complete this task by a certain date.

I'm not saying you need to give satisfaction to the community about what is working or when it will end. But it would be something interesting to bring the community even further of development of the framework.

Joel talks a bit about the tool in this post. (http://www.joelonsoftware.com/items/2012/07/09.html)

https://trello.com/
https://trello.com/about
https://trello.com/guide

An example is the Development Board of Trello software, you can see it here https://trello.com/board/trello-develop … 136000000c

Roberto Schneiders · mORMot 1

After solving the problem in the creation of tables in firebird I found another problem.
http://synopse.info/forum/viewtopic.php?pid=6279#p6279

I can not add any records. I tried to debug the EngineAdd method of TSQLRestServerStaticExternal class. The problem with this method is that the field fFieldsDynArray apparently does not have the table fields.

I found a problem in the Create method of this class.
This code:

  // compute the SQL statements used internaly for external DB requests
  fSelectOneDirectSQL := StoredClassProps.SQL.TableSimpleFields[true,false];
  if IdemPChar(pointer(fSelectOneDirectSQL),'ROWID') then
    System.Delete(fSelectOneDirectSQL,1,3); // RowID -> ID when executed in direct
  fSelectOneDirectSQL := FormatUTF8('select % from %',[fSelectOneDirectSQL,fTableName]);
  if fOwner<>nil then // override only if was not retrieved  from fModel[0]
    fStoredClassProps.SQL.SelectAll[true] := fSelectOneDirectSQL;
  fSelectOneDirectSQL := fSelectOneDirectSQL+' where ID=?';
  fSelectAllDirectSQL := FormatUTF8('select %,ID from %',
    [StoredClassProps.SQL.InsertSet,fTableName]);
  fSelectTableHasRowsSQL := 'SELECT ID FROM '+fStoredClassRecordProps.SQLTableName+' LIMIT 1';

generates this select:

SELECT ID FROM People LIMIT 1

This select is not compatible with firebird. The right way is:

SELECT FIRST 1 ID FROM People

I changed the code to see if the cause of the problem is only that, but it is not.

Apparently the methods "Init" or "ReHash" are not working as expected in this part of the code (TSQLRestServerStaticExternal.Create):

  fFieldsDynArray.Init(TypeInfo(TSQLDBColumnDefineDynArray),fFields,nil,nil,nil,nil,true);
  fFieldsDynArray.ReHash;

The fFields has this value:

(('ID                             ', '160', ftInt64, 8, 8, 0, True), ('NAME                           ', '2611', ftUTF8, 8, 8, 0, True), ('AGE                            ', '160', ftInt64, 8, 8, 0, True))

The FindHashed method always returns -1 in this code (TSQLRestServerStaticExternal.ExecuteFromJSON):

    for F := 0 to Decoder.FieldCount-1 do begin
      k := fFieldsDynArray.FindHashed(Decoder.FieldNames[F]);
      if k<0 then
        exit; // JSON field does not exist in the external DB table
      Types[F] := fFields[k].ColumnType;
    end;

Decoder has this value:

(('ID', 'Name', 'Age', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', ''), ('1', 'Roberto', '25', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', ''), [], 3, 19, False, $18EED8, 1)

Roberto Schneiders · mORMot 1

I figured when I saw the documentation.

Thank you.

Roberto Schneiders · mORMot 1

Here we go.

The SQL generated by the framework is this:

CREATE TABLE People (ID  INT64 PRIMARY KEY, Name BLOB SUB_TYPE 1 SEGMENT SIZE 2000 CHARACTER SET UTF8,Age INT64)

But the firebird does not support INT64. Actually it is declared as BIGINT.

/*******************************************************************************
The next statement causes the following error:

Dynamic SQL Error.
SQL error code = -607.
Invalid command.
Specified domain or source column INT64 does not exist.
*******************************************************************************/

The ExecuteDirect command returns nil.

  if fFields=nil then begin
    // table is not yet existing -> try to create it
    SQL := fProperties.SQLCreate(fTableName,TSQLRecordClassToExternalFields(aClass));
    if SQL<>'' then
      if ExecuteDirect(pointer(SQL),[],[],false)<>nil then
        fProperties.GetFields(fTableName,fFields); // fields from DB after create
  end;

This SQL command works fine:

CREATE TABLE People (ID  BIGINT PRIMARY KEY, Name BLOB SUB_TYPE 1 SEGMENT SIZE 2000 CHARACTER SET UTF8,Age BIGINT)

http://www.firebirdsql.org/manual/migra … types.html

Roberto Schneiders · mORMot 1

I'm debugging CreateMissingTables method trying to identify the problem.

Apparently is behaving as expected. In line 725 we have this code:

DB.Execute(Model.GetSQLCreate(t)); // don't catch exception in constructor

the method Model.GetSQLCreate(t) returns the following SQL:

CREATE VIRTUAL TABLE People USING External(Name TEXT COLLATE SYSTEMNOCASE, Age INTEGER);

The method db.Execute apparently runs successfully.

When attempting to open the SQLite file using SQLiteExpert an error occurs when loading the schema of the table People.
"Error refreshing schema for table "People":_no such module:_External."

Roberto Schneiders · mORMot 1

Yes, the database exists.
No, it is not empty.

I'll try to debug. If I get any information I notice.

Native driver for firebird is excellent news. We work with Firebird and PostgreSQL. I was already satisfied with the performance using ODBC with native driver think it will be excellent. Thanks Arnaud.

Roberto Schneiders · mORMot 1

I published a new post with the new tests (Using XE3 Update 1).

http://robertocschneiders.wordpress.com … ts-part-2/

Roberto Schneiders · mORMot 1

I do not know if I understand correctly.

From what I know the TSQLTable is readonly, right?
I do not understand exactly what you mean by "update the PUTF8Char pointers in the TSQLTable array". You mean simply overwrite the existing records pointers? If I need to add a new record?

Had some example?

Thanks Arnaud.

Roberto Schneiders · mORMot 1

If I use BATCH. When running the UpdateFromServer command he will return the changed data? Because the grid must contain the changes.

Roberto Schneiders · mORMot 1

Please do not crucify me for write the ClientDataSet word. I know that it is prohibited here . I have not found a better way to ask that.

I am creating a class (TcxCustomDataSource) to use DevExpress grids using mORMot ORM.

I'm using a TSQLTable (TSQLRecord.FillTable) to provide the data to the grid.

function TmORMotCustomDataSource.GetValue(ARecordHandle: TcxDataRecordHandle;
  AItemHandle: TcxDataItemHandle): Variant;
begin
  Result := Table.GetVariant(Integer(ARecordHandle)+1, Integer(AItemHandle), self);
end;

When information is changed on the grid I make changes in the database (Client.Update) and then I use the command Client.UpdateFromServer to update the grid.
This is the code:

procedure TmORMotCustomDataSource.SetValue(ARecordHandle: TcxDataRecordHandle; AItemHandle: TcxDataItemHandle;
  const AValue: Variant);
var
  ARecord: TSQLRecord;
  Changed: Boolean;
begin
  inherited;

  ARecord := FRecord.Create;
  FRecord.FillRow(Integer(ARecordHandle)+1, ARecord); // FRecord is a TSQLRecord (a list of People) created by CreateAndFillPrepare
  aRecord.SetFieldVariant(Table.GetS(0, Integer(AItemHandle)) ,AValue);
  FClient.Update(ARecord);

  fClient.UpdateFromServer([FRecord.FillTable],Changed, Pointer(Integer(ARecordHandle)+1));

  Self.DataChanged; // This make the grid reload all information
end;

This works perfectly, but I need to work differently in some cases.

This way, when some information is changed in the Grid, it is immediately applied to the database.

I need to have the option to only apply changes to the database later. Trough a Save button, usually.

With DBExpress did this by placing the ApplyUpdates on the Save button. How I do this using mORMot?

Roberto Schneiders · mORMot 1

Ok I get it.

I've got a problem. I put the properties into published. The error no longer occurs. but ...

The method "CreateMissingTables" is not creating the table in my database.

The Add method (Client.add) always returns 0.

Any idea?

Roberto Schneiders · mORMot 1

I do not know.

As I understand it is a tool primarily used by WCF clients (which in theory is SOA. I am no expert on this, forgive me if I'm wrong. Maybe wikipedia is misleading me http://en.wikipedia.org/wiki/Windows_Co … Foundation) .
http://msdn.microsoft.com/en-us/library/aa751905.aspx

Apparently I misunderstood your answer on StackOverflow.
In your answer you put "It is ready to be Consumed by AJAX clients or WCF (after the latest custom marshaling of the interfaces)". So I thought that maybe there was some implementation for WCF tools.
In that case, what do you mean by "after custom marshaling of the interfaces"? Translate the object pascal code to C #?

Thank you.

Roberto Schneiders · mORMot 1

Thanks Arnaud.

Can you explain the third parameter of the constructor of the class TSQLRestClientDB? What 'test.db3' mean?

When I use the TSQLDataBase and when I use TFileName? how to get the object of type TSQLDatabase

Roberto Schneiders · mORMot 1

Can I use the ServiceModel Metadata Utility Tool (http://msdn.microsoft.com/en-us/library/aa347733.aspx) on a mORMot server ?

I saw your response on this link but i not sure
http://stackoverflow.com/questions/5650 … and-delphi

Roberto Schneiders · mORMot 1

I'm doing tests aiming learn better the framework. I read all the documentation but still do not understand everything. There are many things and the documentation is quite large. Very different from what I was used (Embarcadero libraries)

I tried to look at the code examples that exist in the documentation but I have not found how to do this.

what I want to do is a simple example:
- Connect to a database ODBC (This is ok).
- Test the command "CreateMissingTables" (This is not working yet)
- Write something using ORM.

Then, of course, want to go further in ORM.

First, I created an object to represent a table that does not yet exist in the database:

  TPeople = class (TSQLRecord)
  private
    FName: RawUTF8;
    FAge: Integer;
  public
    property Name:RawUTF8 read FName Write FName;
    property Age:Integer read FAge Write FAge;
  end;

I'm trying to do so, and certainly not the right way, because it does not work.

  fModel := TSQLModel.Create([TPeople]);
  fConnection := TODBCConnectionProperties.Create('teste', 'teste', 'user', 'password');
  VirtualTableExternalRegister(fModel,TPeople,fConnection,'People');
  fConnection.MainConnection.Connect;
  if fConnection.MainConnection.IsConnected then
  begin
    fClient := TSQLRestClientDB.Create(fModel,nil,'test.db3',TSQLRestServerDB);
    fClient.Server.CreateMissingTables;
  end;

Until line 6 all ok. I think the problem starts at line 7. The method TSQLRestClientDB.Create asks for the third parameter an "ADB: TSQLDataBase" or "aDBFileName: TFileName;". I have no idea how to get the object of type TSQLDatabase. In the examples of the documentation is always passed a string (TFileName) and I have no idea what it is.

Anyway. the line 7 works, gives no error.
The command fClient.Server.CreateMissingTables returns an access violation. This error occurs when calling the method "GetVirtualTableSQLCreate (RecordProps)" at line 17516 of mOTMot.pas unit.

I appreciate if someone can help me.

Roberto Schneiders · mORMot 1

I also do not know much C#, but have three C# developers working here. Maybe we can develop it.

I'll talk to those developers to see if they have any idea how to do this.

Roberto Schneiders · mORMot 1

Yes, this makes perfect sense to me.

I still know little bit of the framework, so I apologize if I'm doing stupid questions.

I still have doubts about this C# code that will be generated.
Basically these codes are the public interfaces? Or in the case of C# would be only DTO's classes?

In delphi I do not need to worry about REST requests or JSON parsers. In C# I have to do this manually?

Is there the possibility of having the service factory in C#? So I could use it in C# in the same way that I use in Delphi. Example: fClient.Services.Info (typeinfo (ICalculator).) Get (I)

Obviously there had to be a part of the framework implemented in C#. But that would be possible? Do you think that is feasible?

Roberto Schneiders · mORMot 1

We currently have clients Delphi and C # consuming a DataSnap server. My idea is to create a new one using the mORMot framework.

On the roadmap are these two items:
- Direct connection of interface-driven services from a rich AJAX client built with Smart Mobile Studio;
- Direct connection of interface-driven services from a rich C# client, with a code generation tool;

In DataSnap we have the proxy classes and MobileConnector's helping with C#. But the way we work just with interfaces in mORMot is fantastic, much better, but only for Delphi clients. I imagine that these items of roadmap reference it. Could spend more information on how this would work?

The way of working with server classes (A service for example) in C# would be similar to Delphi? Or the communication with the server must be implemented manually, as it is today.

You plan to work on these items soon or they will take time to be implemented?

mORMot Open Source

#101 Re: mORMot 1 » Performance problem with Firebird » 2013-01-22 10:24:42

#102 mORMot 1 » Performance problem with Firebird » 2013-01-21 19:44:20

#103 Re: mORMot 1 » Problem with ODBC and Firebird » 2013-01-21 18:22:00

#104 Re: mORMot 1 » Problem with ODBC and Firebird » 2013-01-21 16:35:20

#105 Re: mORMot 1 » Problem with ODBC and Firebird » 2013-01-21 15:49:38

#106 Re: mORMot 1 » Problem with ODBC and Firebird » 2013-01-21 12:52:17

#107 Re: mORMot 1 » Problem with ODBC and Firebird » 2013-01-18 19:15:21

#108 Re: mORMot 1 » I need help on a simple example using ODBC. » 2013-01-18 19:08:49

#109 mORMot 1 » [Suggestion] Use Trello to organize the Roadmap » 2013-01-18 19:07:26

#110 mORMot 1 » Problem with ODBC and Firebird » 2013-01-18 18:45:14

#111 Re: mORMot 1 » I need help on a simple example using ODBC. » 2013-01-18 18:08:59

#112 Re: mORMot 1 » I need help on a simple example using ODBC. » 2013-01-18 16:19:24

#113 Re: mORMot 1 » I need help on a simple example using ODBC. » 2013-01-18 12:52:12

#114 Re: mORMot 1 » I need help on a simple example using ODBC. » 2013-01-18 10:30:18

#115 Re: mORMot 1 » DataSnap analysis based on Speed & Stability tests(Compared to mORMot) » 2013-01-18 10:25:37

#116 Re: mORMot 1 » Is there something like the ClientDataSet.ApplyUpdates in mORMot? » 2013-01-17 19:29:22

#117 Re: mORMot 1 » Is there something like the ClientDataSet.ApplyUpdates in mORMot? » 2013-01-17 19:16:05

#118 mORMot 1 » Is there something like the ClientDataSet.ApplyUpdates in mORMot? » 2013-01-17 18:56:18

#119 Re: mORMot 1 » I need help on a simple example using ODBC. » 2013-01-14 17:52:28

#120 Re: mORMot 1 » Direct connection of interface-driven services from a rich C# client » 2013-01-14 10:45:27

#121 Re: mORMot 1 » I need help on a simple example using ODBC. » 2013-01-14 10:12:29

#122 Re: mORMot 1 » Direct connection of interface-driven services from a rich C# client » 2013-01-11 19:46:49

#123 mORMot 1 » I need help on a simple example using ODBC. » 2013-01-11 17:50:13

#124 Re: mORMot 1 » Direct connection of interface-driven services from a rich C# client » 2013-01-11 17:32:33

#125 Re: mORMot 1 » Direct connection of interface-driven services from a rich C# client » 2013-01-11 16:02:01

#126 mORMot 1 » Direct connection of interface-driven services from a rich C# client » 2013-01-11 13:08:12

Board footer