mORMot Open Source

Found another place that also eats the sqlite errors silently:

TSQLRestServerDB.EngineExecute();
Begin
...
      {$ifdef WITHLOG}
      DB.Log.Log(sllError,'% for %',[E,aSQL],self);
      {$else}
      LogToTextFile('TSQLRestServerDB.EngineExecute: '+RawUTF8(E.Message)+#13#10+aSQL);
      {$endif}

...
end;

I think when withlog is not defined, just call raise, give our programs the chance to get the exact error and handle accordingly. Handling errors with exceptions, instead of IF statements, are a commonly accepted concept, IIRC.

Arnaud, this is not a real client/server app, but a single EXE USING TSQLRestClientDB, so it really need to handle the actual sqlite server. Do you think if I switch to TSQLRestServerDB will fix the problem?

additional info, My derived TransactionBegin() is protected with a TOmniMREW (mulit-reader-exclusive-writer) lock, like this:

function TDataStoreClient.TransactionBegin(aTable: TSQLRecordClass; SessionID:
    cardinal): boolean;
begin
  //if the db server is in the same process, we protect the writing operation from
  //being executed by multi threads.
  if IsServerInSameProcess then
    gAppScopeDbLock.EnterWriteLock; //this is an global lock

  result := inherited TransactionBegin(aTable, SessionID);
end;

procedure TDataStoreClient.Commit(SessionID: cardinal);
begin
  try
    inherited Commit(SessionID);
  finally
    if IsServerInSameProcess then
      gAppScopeDbLock.ExitWriteLock;
  end;
end;

Any logical problems in my derived TSQLRestClientDB class?

Maybe execute a raw "insert into...select" SQL statement using EngineExecute() is faster and less of a hassle. Just an idea, I don't remember the details, please google yourself.

If you use SQLite as the backend databae, SQLite itself doesn't support renaming a field.

Assume you have TSQLOldTable, now create a new class called TSQLNewTable with the new field definitions, call CreateMissingTables(); Then read old records from TSQLOldTable and add to TSQLNewTable;

drop tale  TSQLOldTable.

Then change all references to TSQLOldTable in your code to TSQLNewTable.

Thanks Arnaud, the problem's fixed. I guess fmShareDenyNone doesn't ensure thread-safety, I guess the problem was that, Thread B trying to open the same .mab file while thread A hasn't done its opening of the very same file.

Yes, it should be sufficient in most cases...

@ab, could you give a very simple example of your idea? I couldn't understand how to define a TSQLRecordFTS4  child class utilizing the  RecordRef  you are talking about, especially how to set the rowid value for a new FTS record...

My current record definition:

  TSQLFTSDoc = class(TSQLRecordFTS4Porter)
  private
    FKind: String;
    FSrcRecId: Integer;
    FContent: String;
    FDate: String;
  published
    ///  the kind of data, eg 'order', 'contact', etc. usually matches the
    ///  corresponding table name.
    property Kind: String read FKind write FKind;

    ///  the ID of the original record in the original table in the origiinal DB.
    ///  When we have 'Kind' and 'SrcRecId', we'll be able to locate the original record.
    property SrcRecId: Integer read FSrcRecId write FSrcRecId;

    property Content: String read FContent write FContent;

    ///  Human readable date, such as "2014 may 1", please note the spaces in the date string,
    ///  because we want to utilize the sqlite FTS search query language, so that the user can enter
    ///  'date:2014' and search all data within the year of 2014.
    property Date: String read FDate write FDate;
  end;

Note: With is approach, with the explicit Kind (tableName) and SrcRecId fields, I can write joined SQL easily...

I haven't tried, but since sqlite document says a FTS table can only contain string fields except for the rowid, I thought I can have string fields in the TSQLRecordFTS4 decedents. But it's great to know I can also have Integers! How about TDateTime?

For the 'source table', I'll use table names (strings), in order to take advantage of the ":" FTS search operation, for example:

table:order

Will search only orders, but not other kind of data.

Thanks for the help. You are right, in additional to the source record ID,  I also have to store the table ID in order to be able to retrieve the source record at a later time.

I just checked the document, looks like it has some restriction on using TRecordReference, I'd better off store 'source record ID' and 'source table ID' (define as enum) as strings separately in the FTS4 record (TSQLRecordFTS4).

Thank you, things are clarified now.

Hi MPV,

It's great! Thanks for the help. I'm not trying to build the dll in the near future, just wanted to get things clarified

Thanks Ab.

Just set Scaled := False for all the forms, then VCL will not do its **incomplete and inaccurate** scaling.

If I do improve SynDbExplroer  I'll send you the source.

@mpv, welcome back! And great to hear that. And yes I already know Ternjs, which is written by an amazing developer!

When FPC support is fully implemented, I believe mORMot will bring FPC/Lazurus to the next level of web development, maybe like RoR to Ruby

@ab, would you contact mpv via e-mail ask for the source code of the DLL which he promised to publish it? I think it's a key  aspect to keep the JavaScript part of the mORMot framework live and going?

great! Performance is not a big problem in my specific situation. Thank you.

Oh! My bad! Sorry!

As per our conversatin, since I changed to TSQLRestServerDB from TSQLRestClientURINamedPipe, and didn't comment out all calls to .Free(), thus causing the global TSQLRestServerDB instanced got freed unexpectedly, and the error I described above is due to accessing a already destroyed object...

Sorry for have wasted a little time of your, Arnaud

Anyways, I still want my try...except...end block to catch all internal exceptions of TSQLRestServerDB.

ab, one more question, it seems that my try...except...end block can't catch the exception - the exception raises only in the DELPHI IDE?  And since TSQLRestServerDB doesn't have the LastErrorException properly like TSQLRestClientURI does, is there a way to catch those exceptions? Thanks.

@ab, there is something I don't understand, why amUnlocked is still thread-safe? I checked the source code, in amUnlocked  mode, there is no criticalSection protection?

Re named pipe improvements, I'm afraid I've not got a level that I can have any good suggestion for it

Yes, the clients are in the same process, I didn't know we can directly access the same TSQLRestServerDB instance from multiple threads!

Just to be sure, do you mean, if I have a global var:

unit1
interface
...
var
  gRestSvr: TSQLRestServerDB;//this is a global variable declared in a unit.

...
  //make the writing to the sqlite3 db runs in a single background thread.
  gRestSvr.AcquireWriteMode := amBackgroundThread;

And in a TThread.Execute() I can call cany methods of gRestSvr?

gRestSvr.anyRestServerMethod();// this code is in a thread other than the main thread.

If yes, that'll be awesome! Can you confirm?

Thanks Arnaud, I did full text search the mORMot source code, but forgot the manual