#1 Re: mORMot 1 » TWinHttp vs HttpGet, why this side effect (and how to solve it) ? » 2019-03-03 15:43:54

Ooooh ! Thanks again @mpv !
(BTW I already said that somewhere on this forum, but we really need a "thumbs up" button ! No obvious way to thankfully mention the great help and the incredible work of ab :-p)

#2 mORMot 1 » TWinHttp vs HttpGet, why this side effect (and how to solve it) ? » 2019-03-03 09:35:36

jbroussia
Replies: 3

Hi,

I'm trying to use an inherited TWinHttp (from THttpRequest) to download the content from an URL.
It *seems* to work... except that the result is not what's expected ! The content is retrieved (status code = 200), but is different from the (expected) one I get when using the HttpGet (from SynCtrlSock too) !?

Sample ex.:

- With WinHttp...

function blabla.GetContent1: SockString;
var
	Http: {$IFDEF MSWINDOWS}TWinHTTP{$ELSE}TCurlHTTP{$ENDIF};
	sInHeaders, sInData, sOutHeaders: SockString;
	sUrl: string;
	iResponseStatus: Integer;
begin
	Result := '';

	sUrl := 'https://www.betexplorer.com/soccer/';
	Http := {$IFDEF MSWINDOWS}TWinHTTP{$ELSE}TCurlHTTP{$ENDIF}.Create(Url);
	try
		iResponseStatus := Http.Request(sUrl, 'GET', 0, sInHeaders, sInData, '', sOutHeaders, Result);
	finally
		Http.Free;
	end;
end;

- With HttpGet

function blabla.GetContent2: SockString;
var
	sUrl: string;
begin
	sUrl := 'https://www.betexplorer.com/soccer/';
	Result := HttpGet(sUrl);
end;

So, the URL is specific, it may work in a different way on other URLs but that's the one I need to get. On the first case the content seems to be from the "main" page, while the second comes from the requested page ("soccer" page).

Did I miss some parameter when using TWinHttp ? Is something happening on the external server that detects the origin from the request to send it a different content ? I can't understand what's happening between the two cases :-\

Thanks.

#4 mORMot 1 » Big newbie, how to retrieve STATUS_CODE when using TWinHttp ? » 2019-02-24 10:43:02

jbroussia
Replies: 3

Hi,

Sorry, I'm going back to Delphi after a quite long break and I lost some obvious knowledge :-\ (and I crashed a few months ago, spent a couple of weeks in coma, it seems that some things are now damaged in my brain now... not complaining, just using this as an argument to get mercy for an answer :-p)

So, I'm trying to use TWinHttp to download the content of some web pages, the class is working as expected but I'd like 1- to have more control on the User-Agent  ("myClass(TWinHttp).UserAgent := 'blabla'" should be OK, right ?) and 2- to retrieve the status code from the server (200, 404, 2xx, 3xx, ...): how to do it with TWinHttp ?

Thanks !

#5 Re: mORMot 1 » Error: TSQLDBSQLite3Statement.Step(SeekFirst=true) not implemented » 2017-09-10 09:20:13

What I did is simply modify a couple lines in SynDBSQLite3:

// Around lines 775-780...
  if SeekFirst then begin
    (* if fCurrentRow>0 then
      raise ESQLDBException.CreateUTF8('%.Step(SeekFirst=true) not implemented',[self]); *)
    fCurrentRow := 0;
    //fStatement.Reset;
  end;

It's been working fine for me.

#6 Re: mORMot 1 » Bugs with TSQLRestClientURI.OnAuthentificationFailed » 2017-03-10 16:49:00

OK, I did. I don't know the rules on how to create a ticket, I hope it will be accepted.

#8 Re: mORMot 1 » Bugs with TSQLRestClientURI.OnAuthentificationFailed » 2017-03-09 20:08:35

Nope, still can't reproduce it :-\ I can disconnect the server as much as I want, and when I restart it and reconnect the client, the authentication process works as expected.
What if you comment out the last DoRetry (as in original code) ?

if OnAuthentificationFailed(Retry, aUserName, aPassword, aPasswordHashed)
and SetUser(StringToUTF8(aUserName), StringToUTF8(aPassword), aPasswordHashed) then begin
  // DoRetry;
  fInAuthenticationLoop := FALSE;
  Break;
end;

#9 Re: mORMot 1 » Bugs with TSQLRestClientURI.OnAuthentificationFailed » 2017-03-09 19:25:13

I can't reproduce it, can you describe the steps required ?

#10 Re: mORMot 1 » Bugs with TSQLRestClientURI.OnAuthentificationFailed » 2017-03-09 10:33:12

So I now have it working as expected, plus the calling query will be executed once the user is re-authenticated, with code modified as follow:

Modifications in TSQLRestClientURI:
- Added a variable to check if we're in authentication loop

TSQLRestClientURI = class(TSQLRestClient)
  protected
    fInAuthenticationLoop: Boolean;
    // ...

- Modified the default value of property MaximumAuthentificationRetry to 1 (as a value of 0 to retry once doesn't make sense to me ?)

property MaximumAuthentificationRetry: Integer
  read fMaximumAuthentificationRetry write fMaximumAuthentificationRetry default 1;

- And TSQLRestClientURI.URI modified to this:

function TSQLRestClientURI.URI(const url, method: RawUTF8; Resp, Head, SendData: PRawUTF8): Int64Rec;
var
  Retry: Integer;
  aUserName, aPassword: string;
  StatusMsg: RawUTF8;
  Call: TSQLRestURIParams;
  aPasswordHashed: Boolean;

  procedure DoRetry;
  begin
    Call.Url := url;
    if fSessionAuthentication <> nil then
      fSessionAuthentication.ClientSessionSign(Self, Call);
    Call.Method := method;
    if SendData <> nil then
      Call.InBody := SendData^;

    {$ifndef LVCL}
    if Assigned(fOnIdle) then begin
      if fBackgroundThread = nil then
        fBackgroundThread := TSynBackgroundThreadEvent.Create(OnBackgroundProcess, OnIdle, FormatUTF8('% "%" background', [Self, Model.Root]));
      if not fBackgroundThread.RunAndWait(@Call) then
        Call.OutStatus := HTTP_UNAVAILABLE;
    end
    else
    {$endif}
    begin
      InternalURI(Call);
      if not(ioNoOpen in fInternalOpen) then
        if (Call.OutStatus = HTTP_NOTIMPLEMENTED) and (ioOpened in fInternalOpen) then begin
          InternalClose;     // force recreate connection
          Exclude(fInternalOpen, ioOpened);
          InternalURI(Call); // try request again
        end
        else
          Include(fInternalOpen, ioOpened);
    end;

    Result.Lo := Call.OutStatus;
    Result.Hi := Call.OutInternalState;
    if Head <> nil then
      Head^ := Call.OutHead;
    if Resp <> nil then
      Resp^ := Call.OutBody;
    fLastErrorCode := Call.OutStatus;
  end;

begin
  if Self = nil then begin
    Int64(Result) := HTTP_UNAVAILABLE;
    SetLastException(nil, HTTP_UNAVAILABLE);
    Exit;
  end;

  fLastErrorMessage := '';
  fLastErrorException := nil;
  if fServerTimeStampOffset = 0 then begin
    if not ServerTimeStampSynchronize then begin
      Int64(Result) := HTTP_UNAVAILABLE;
      Exit; // if TimeStamp is not available, server is down!
    end;
  end;

  Call.Init;
  if (Head <> nil) and (Head^ <> '') then
    Call.InHead := Head^;
  if fSessionHttpHeader <> '' then
    Call.InHead := Trim(Call.InHead + #13#10 + fSessionHttpHeader);

  try
    DoRetry;
    if (Call.OutStatus = HTTP_TIMEOUT) and RetryOnceOnTimeout then begin
      InternalLog('% % returned "408 Request Timeout" -> RETRY', [method, url], sllError);
      DoRetry;
    end
    else if (Call.OutStatus = HTTP_FORBIDDEN) and Assigned(OnAuthentificationFailed) and not fInAuthenticationLoop then begin
      fInAuthenticationLoop := TRUE;
      Retry := 1;
      while Retry <= MaximumAuthentificationRetry do begin
        // "403 Forbidden" in case of authentication failure -> try relog
        if OnAuthentificationFailed(Retry, aUserName, aPassword, aPasswordHashed)
        and SetUser(StringToUTF8(aUserName), StringToUTF8(aPassword), aPasswordHashed) then begin
          DoRetry;
          fInAuthenticationLoop := FALSE;
          Break;
        end;
        Inc(Retry);
      end;
    end;

    if not StatusCodeIsSuccess(Call.OutStatus) then begin
      StatusCodeToErrorMsg(Call.OutStatus, StatusMsg);
      if Call.OutBody = '' then
        fLastErrorMessage := StatusMsg
      else
        fLastErrorMessage := Call.OutBody;
      InternalLog('% % returned % (%) with message  %', 
        [method, url, Call.OutStatus, StatusMsg, fLastErrorMessage], sllError);
      if Assigned(fOnFailed) then
        fOnFailed(Self, nil, @Call);
    end;

  except
    on E: Exception do begin
      Int64(Result) := HTTP_NOTIMPLEMENTED; // 501
      SetLastException(E, HTTP_NOTIMPLEMENTED, @Call);
      Exit;
    end;
  end;
end;

Maybe not the cleanest code (hey, not worse than using goto labels ! :-P), but it works for me.

Can anyone have a look and give me some feedback ?
Thanks

#11 Re: mORMot 1 » Bugs with TSQLRestClientURI.OnAuthentificationFailed » 2017-03-06 18:08:52

After playing with the code in mORMot TSQLRestClientURI.URI function, I have authentication retries "almost" working:

function TSQLRestClientURI.URI(const url, method: RawUTF8; Resp, Head, SendData: PRawUTF8): Int64Rec;
var
  Retry: Integer;
  aUserName, aPassword: string;
  StatusMsg: RawUTF8;
  Call: TSQLRestURIParams;
  aPasswordHashed: Boolean;

  procedure DoRetry;
  begin
    Call.Url := url;
    if fSessionAuthentication <> nil then
      fSessionAuthentication.ClientSessionSign(Self, Call);
    Call.Method := method;
    if SendData <> nil then
      Call.InBody := SendData^;

    {$ifndef LVCL}
    if Assigned(fOnIdle) then begin
      if fBackgroundThread = nil then
        fBackgroundThread := TSynBackgroundThreadEvent.Create(OnBackgroundProcess, OnIdle, FormatUTF8('% "%" background', [Self, Model.Root]));
      if not fBackgroundThread.RunAndWait(@Call) then
        Call.OutStatus := HTTP_UNAVAILABLE;
    end
    else
    {$endif}
    begin
      InternalURI(Call);
      if not(ioNoOpen in fInternalOpen) then
        if (Call.OutStatus = HTTP_NOTIMPLEMENTED) and (ioOpened in fInternalOpen) then begin
          InternalClose;     // force recreate connection
          Exclude(fInternalOpen, ioOpened);
          InternalURI(Call); // try request again
        end
        else
          Include(fInternalOpen, ioOpened);
    end;

    Result.Lo := Call.OutStatus;
    Result.Hi := Call.OutInternalState;
    if Head <> nil then
      Head^ := Call.OutHead;
    if Resp <> nil then
      Resp^ := Call.OutBody;
    fLastErrorCode := Call.OutStatus;
  end;

begin
  if Self = nil then begin
    Int64(Result) := HTTP_UNAVAILABLE;
    SetLastException(nil, HTTP_UNAVAILABLE);
    Exit;
  end;

  fLastErrorMessage := '';
  fLastErrorException := nil;
  if fServerTimeStampOffset = 0 then begin
    if not ServerTimeStampSynchronize then begin
      Int64(Result) := HTTP_UNAVAILABLE;
      Exit; // if TimeStamp is not available, server is down!
    end;
  end;

  Call.Init;
  if (Head <> nil) and (Head^ <> '') then
    Call.InHead := Head^;
  if fSessionHttpHeader <> '' then
    Call.InHead := Trim(Call.InHead + #13#10 + fSessionHttpHeader);

  try
    DoRetry;
    if (Call.OutStatus = HTTP_TIMEOUT) and RetryOnceOnTimeout then begin
      InternalLog('% % returned "408 Request Timeout" -> RETRY', [method, url], sllError);
      DoRetry;
    end
    else if (Call.OutStatus = HTTP_FORBIDDEN) and Assigned(OnAuthentificationFailed) then begin
      Retry := 1;
      while Retry <= MaximumAuthentificationRetry do begin
        // "403 Forbidden" in case of authentication failure -> try relog
        if OnAuthentificationFailed(Retry, aUserName, aPassword, aPasswordHashed)
        and SetUser(StringToUTF8(aUserName), StringToUTF8(aPassword), aPasswordHashed) then begin
          DoRetry;
          Break;
        end;
        Inc(Retry);
      end;
    end;

    if not StatusCodeIsSuccess(Call.OutStatus) then begin
      StatusCodeToErrorMsg(Call.OutStatus, StatusMsg);
      if Call.OutBody = '' then
        fLastErrorMessage := StatusMsg
      else
        fLastErrorMessage := Call.OutBody;
      InternalLog('% % returned % (%) with message  %', 
        [method, url, Call.OutStatus, StatusMsg, fLastErrorMessage], sllError);
      if Assigned(fOnFailed) then
        fOnFailed(Self, nil, @Call);
    end;

  except
    on E: Exception do begin
      Int64(Result) := HTTP_NOTIMPLEMENTED; // 501
      SetLastException(E, HTTP_NOTIMPLEMENTED, @Call);
      Exit;
    end;
  end;
end;

But there is still that problem with recursivity, because SetUser calls back TSQLRestClientURI.URI so we're caught in a loop.

What would be the best, clean way to get out of it ?

Thanks !

Edit: for this code to "work" I also set MaximumAuthentificationRetry default value to 1.

#12 Re: mORMot 1 » Bugs with TSQLRestClientURI.OnAuthentificationFailed » 2017-03-05 08:01:53

I think the first bug in TSQLRestClientURI.URI is that:

if not OnAuthentificationFailed(Retry+2,aUserName,aPassword,aPasswordHashed) or
  not SetUser(StringToUTF8(aUserName),StringToUTF8(aPassword),aPasswordHashed) then
  break;

should be:

if not OnAuthentificationFailed(Retry+2,aUserName,aPassword,aPasswordHashed) or
  SetUser(StringToUTF8(aUserName),StringToUTF8(aPassword),aPasswordHashed) then
  break;

???

After modifying this line, then remain at least one bug and one "problem":
* it seems there is a bug of "recursivity" with counting/incrementing the Retry variable:
    - if wrong credentials are input, Retry doesn't increase; if correct credentials are input, Retry does increase ???
    - if wrong credentials are input before the correct credentials, then you have to input the correct credentials "X+1" times, where X is the number of times you input wrong credentials. So if my first try is incorrect, then I need to input the correct credentials 2 times to be authenticated. If my first 2 tries are incorrect then I need to input the correct credentials 3 times !?

* then there is a problem, in that, I think, once the credentials are correct and validated, the request should not return an empty result ? It seems there should be a call to "InternalURI(Call);" somewhere after authentication worked ?

#13 Re: mORMot 1 » Bugs with TSQLRestClientURI.OnAuthentificationFailed » 2017-03-04 07:44:29

Hum, no it doesn't work :-( The behavior is even stranger ! For ex. after a second server restart, the event is not fired anymore so the user is not asked for credentials anymore. Also, same as before, it does only work if username/password are correct at first try, not for subsequent tries.

You shouldn't call SetUser anyways, as it is called in TSQLRestClientURI.URI if any credentials were input:

if not OnAuthentificationFailed(Retry+2,aUserName,aPassword,aPasswordHashed) or
  not SetUser(StringToUTF8(aUserName),StringToUTF8(aPassword),aPasswordHashed) then
  break;

About the comment: I agree, but not everybody is able to tell if they really know enough about XYZ. Most people believe they know it all. So many times I've seen people discover a new passion and start spreading their "knowledge" as experts after just a couple of weeks or months: it takes years if not decades to become an expert in something ! Anyways, if you are aware that you are not an expert , that you still have a lot to learn, then you can still talk about subject XYZ, if you explain your audience that what you say is not definitive and has to be taken with an open mind.
Looking forward for your next videos :-)

#14 Re: mORMot 1 » Bugs with TSQLRestClientURI.OnAuthentificationFailed » 2017-03-03 19:31:47

Thanks.
By the way, while you are here: great videos, but quite difficult to watch as you often type/show code on screen at the same time that comments are displayed below... and then there is extra music ! :-D

#15 Re: mORMot 1 » Bugs with TSQLRestClientURI.OnAuthentificationFailed » 2017-03-03 18:07:40

Hi warleyalex,

I'm not sure if you confirm the problem or if you're saying that my code fails because "Result := TLoginForm.Login..." is always true ?
According to the comments in mORMot code, the function assigned to OnAuthentificationFailed:

// - should return TRUE if aUserName and aPassword both contain some entered
// values to be sent for remote secure authentication
// - should return FALSE if the user pressed cancel or the number of Retry
// reached a defined limit

For my testing purpose, I don't check if aUsername and aPassword are empty (as I manually enter some text), so the result is TRUE if ModalResult is mrOK, FALSE if mrCancel.

#16 mORMot 1 » Bugs with TSQLRestClientURI.OnAuthentificationFailed » 2017-03-03 10:21:06

jbroussia
Replies: 18

Hi ab,

There are a couple bugs in TSQLRestClientURI.URI in regard to the re-authentification process, for ex. when the server had to be restarted and lost information about opened sessions.

First, it seems that MaximumAuthentificationRetry is ignored, the event will keep firing as long as the authentification fails ?

Then, and it's quite difficult to explain, the algorithm is acting weird: in case the first try fails, then it is not possible anymore to authentificate with the correct credentials on subsequent tries.

Shortened code:

uses mORMotUILogin;

procedure TfrmClient.FormShow(Sender: TObject);
begin
  // ...
  // Client.MaximumAuthentificationRetry := 3;
  // Client.RetryOnceOnTimeout := FALSE;
  Client.OnAuthentificationFailed := TryLogin;
  // ...
end;

function TfrmClient.TryLogin(Retry: Integer; var aUserName, aPassword: string; out aPasswordHashed: Boolean): Boolean;
begin
  Result := TLoginForm.Login('Authentification required', 'Please enter your credentials', aUserName, aPassword, TRUE, '');
end;

Steps to reproduce:
- start server, start and connect client
- stop/restart server
- query the server from the client (re-authentification required)
- enter a wrong password at first try
- enter the correct username/password at subsequent tries: -> authentification keeps failing.

#17 Re: mORMot 1 » Check user timeout » 2017-03-01 11:35:51

I'm trying to learn how to walk, but each step I make, I fall :'-(

Has anyone had any experience in 5 years with OnAuthentificationFailed ? The documentation is sparse and outdated (https://synopse.info/files/html/Synopse … #TITLE_547), there is no example in Samples, and little reference on the forum (almost everything is in this current topic) :-\

Thanks !

#18 Re: mORMot 1 » Check user timeout » 2017-02-28 20:21:07

Hi,

Sorry, I'm late to the party !
I'm trying to reconnect after authentication failed (after server restart) using the OnAuthentificationFailed event.
I set the MaximumAuthentificationRetry value to 3. Nothing fancy in my code:

function TfrmClient.evtAuthentificationFailed(Retry: Integer; var aUserName, aPassword: string; out aPasswordHashed: Boolean): Boolean;
begin
  Result := FALSE;

  with TfrmLogin.Create(Self) do begin
    Caption := IntToStr(Retry);  // TEST
    try
      if ShowModal = mrOk then begin
        aUserName := edtUsername.Text;
        aPassword := edtPassword.Text;
        aPasswordHashed := FALSE;
        Result := TRUE;
      end;
    finally
      Free;
    end;
  end;
end;

Yet the algorithm seems quite crazy:
- if I input the correct username/password on the *FIRST* try, authentification works and everything is fine.
- BUT if I first input a wrong username/password combination, then things go "crazy": even if I input the correct combination on following tries, the OnAuthentificationFailed event keeps firing
  * if I keep inserting wrong combinations, "Retry" won't increase ?
  * if I keep inserting the correct combination, "Retry" will increase (up to 5, though I set the max to 3 ?) but authentification will keep failing ? After the 5 try, the event is not fired anymore but authentification still fails ?
  * if I insert a wrong combination after I inserted the correct combination, "Retry" will be reset to 1 ?

#19 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-22 10:57:19

ab wrote:

Perhaps you may reuse the SQlite3 instance of Props for the DB (i.e. use Props.MainSQLite3DB instead of 'mydb.db' for TSQLRestServerDB.Create).
There should be a single exclusive low-level TSQLDataBase instance to access the shared mydb.db file.

Yes, thanks, I was wondering why I had two references to the DB file...

And call DB.CreateMissingTables.

Forgot to put it in the pseudo-code, but indeed I'm calling it in my actual code.

After trying many many things, I think the main problem I have is with the way I declare/use external tables. The field name mapping doesn't work, most probably as a result of me doing something wrong with external table declaration hmm Truth be told, for now I don't need this feature (mapping), and I can use an internal DB, but as I'm learning how to use the mORMot framework, I like to understand what I'm doing incorrectly.
Then there is another little problem in accessing SQLite Views: DB.Retrieve and DB.Create(TSQLRecord, ID) don't work, but this is easily circumvented by using CreateAndFillPrepare instead. I think this is because DB.Retrieve/Create internally use the "RowID" column but my View only has an ID field, whicjh I can query with CreateAndFillPrepare.

#20 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-21 12:05:24

Is that pseudo code correct to declare my sqlite3 db as external, and map the "AlbumId" field of the album_artist view to "ID" ?

Props := TSQLDBSQLite3ConnectionProperties.Create('mydb.db', '', '', '');
Model := TSQLModel.Create([TSQLalbum_artist], ROOT_NAME);
VirtualTableExternalRegister(Model, TSQLalbum_artist, Props);
Model.Props[TSQLalbum_artist].NoCreateMissingTable := TRUE;         // album_artist is a View
Model.Props[TSQLalbum_artist].ExternalDB.MapField('ID', 'AlbumId'); // Map field AlbumId to ID
DB := TSQLRestServerDB.Create(Model, 'mydb.db', TRUE);

#21 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-21 10:32:29

Well, that's the reason why I think I'm not doing things as intended: the debugger never reaches the breakpoint I set in TSQLRestStorageExternal.AdaptSQLForEngineList hmm

(I get an exception in SynSQLite3.sqlite3_check where the resulting SQL is: 'SELECT ID,Title,Name FROM album_artist WHERE ID < ?')

#22 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-19 10:58:34

Hi ab,

Yes I tried mapping the ID of the view with...

Model.Props[TSQLalbum_artist].ExternalDB.MapField('ID', 'AlbumId');

Could it be that the way I define my DB as external, or map the field names, is incorrect (see code a couple messages above) ?
When accessing an external DB with mapped field names, even querying "normal" tables (not Views) doesn't work for me:

aAlbum := TSQLalbums.Create(Database, 1);
// dosomethingwith(aAlbum) // OK

aAlbum := TSQLalbums.CreateAndFillPrepare(Database, 'ID<?', [10]); // AlbumID field is mapped to ID
// while aAlbum.FillOne do
  // dosomethingwith(aAlbum)  // EMPTY

Very probable that I'm doing something wrong but can't spot what !

#23 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-19 08:57:57

Sorry to insist but I've been stuck on this for 2 weeks :'-(

ab: is it a limitation of the framework ?

Thanks.

#24 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-17 09:22:19

Abstract:

I want to query some SQLite3 View from ORM:
- I defined a TSQLRecord class matching the View, the first column of the View is an ID column
- I defined my DB as external by using TSQLDBSQLite3ConnectionProperties
- I registered my tables with VirtualTableExternalRegisterAll
- and I used NoCreateMissingTable for my View

Now I still can't query my View (only "SELECT" statements obviously):
- If the ID column is aliased to RowID:
  * I can use  TSQLRest.Retrieve(aClass, aID)
  * But "complex" queries don't work (CreateAndFillPrepare...) -> no error, but empty result
- Else (no alias or different than RowID):
  * Queries don't work at all -> no error, empty result

Does anybody have any experience with SQLite3 Views and accessing them with mORMot ? I'd like to know if I'm doing something wrong, or if it's a limitation of SQLite3, a limitation of the framework, or even a bug ? :-\
(@razielucas: did you have any success with your Views on MSSQL ?)

Thanks !

#25 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-16 09:38:01

That's what I'm doing (I think ?)

Here is my code (I hope it's not too long and won't break the forum)...

Server:

program Server;

{$APPTYPE CONSOLE}

uses
  {$I SynDprUses.inc} // use FastMM4 on older Delphi, or set FPC threads
  mORMot, mORMotHttpServer, mORMotSQLite3,
  SynCommons,
  SynDB,
  SynDBSQLite3, SynSQLite3, SynSQLite3Static, mORMotDB,
  chinook;

var
  DBConnProps: TSQLDBSQLite3ConnectionProperties;
  Model: TSQLModel;
  DBServer: TSQLRestServerDB;
  HttpServer: TSQLHttpServer;
begin
  DBConnProps := TSQLDBSQLite3ConnectionProperties.Create('chinook.db', '', '', '');
  try
    Model := CreateModel;
    VirtualTableExternalRegisterAll(Model, DBConnProps);
    Model.Props[TSQLalbum_artist].NoCreateMissingTable := TRUE;
    Model.Props[TSQLartists].ExternalDB.MapField('ID', 'ArtistId');
    Model.Props[TSQLalbums].ExternalDB.MapField('ID', 'AlbumId');
    try
      DBServer := TSQLRestServerDB.Create(Model, 'chinook.db', TRUE);
      try
        DBServer.CreateMissingTables;
        HttpServer := TSQLHttpServer.Create('8080', [DBServer], '+', HTTP_DEFAULT_MODE);
        try
          HttpServer.AccessControlAllowOrigin := '*';
          Writeln(#10'Background HTTP server is running.'#10);
          Writeln('Press [Enter] to close the server.'#10);
          readln;
        finally
          HttpServer.Free;
        end;
      finally
        DBServer.Free;
      end;
    finally
      Model.Free;
    end;
  finally
    DBConnProps.Free;
  end;
end.

Model:

unit chinook;

interface

uses
  SynCommons, mORMot;

type
  TSQLartists = class(TSQLRecord)
  private
    fName: RawUTF8;
  published
    property Name: RawUTF8 read fName write fName;
  end;

  TSQLalbums = class(TSQLRecord)
  private
    fTitle: RawUTF8;
    fArtistId: TSQLartists;
  published
    property Title: RawUTF8 read fTitle write fTitle;
    property ArtistId: TSQLartists read fArtistId write fArtistId;
  end;

  // VIEW
  TSQLalbum_artist = class(TSQLRecord)
  private
    fTitle: RawUTF8;
    fName: RawUTF8;
  published
    property Title: RawUTF8 read fTitle write fTitle;
    property Name: RawUTF8 read fName write fName;
  end;

const
  ROOT_NAME = 'root';

function CreateModel: TSQLModel;
  
implementation

function CreateModel: TSQLModel;
begin
  Result := TSQLModel.Create([TSQLartists,
                              TSQLalbums,
                              TSQLalbum_artist],
                              ROOT_NAME);
end;

end.

And on the client side:

procedure TfrmClient.btnTestClick(Sender: TObject);
var
  aAlbumArtist: TSQLalbum_artist;
begin
  // Test 1: OK
  aAlbumArtist := TSQLalbum_artist.Create(Database, 1);
  // aAlbumArtist := TSQLalbum_artist.Create;
  // Database.Retrieve(1, aAlbumArtist);
  try
    mmoResult.Lines.Add(UTF8ToString(ObjectToJSON(aAlbumArtist)));
  finally
    aAlbumArtist.Free;
  end;

  // Test 2: NOT OK
  aAlbumArtist := TSQLalbum_artist.CreateAndFillPrepare(Database, 'RowID < ?', [10]);
  // aAlbumArtist := TSQLalbum_artist.CreateAndFillPrepare(Database, 'ID < ?', [10]);
  try
    with mmoResult.Lines do begin
      BeginUpdate;
      while aAlbumArtist.FillOne do
        Add(UTF8ToString(ObjectToJSON(aAlbumArtist)));
      EndUpdate;
    end;
  finally
    aAlbumArtist.Free;
  end;
end;

#26 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-15 19:00:46

Update on queries with mORMot on SQLite3 Views...

The above workaround only works with a simple DB.Retrieve(ID, aSQLRecord).
It doesn't work with more advanced queries (CreateAndFillPrepare...)

:-\

I don't know what to do, is it better to use SQL queries after all ?

#27 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-13 10:15:33

I'm really confused ab, how do you define a record as external ? The only reference I could find in the documentation is this: https://synopse.info/files/html/Synopse … l#TITL_145
And the only difference I see with internal records is the use of VirtualTableExternalRegister ? I used VirtualTableExternalRegisterAll in my code and it still wouldn't work without aliasing the ID column to RowId (SQLite3).

Also, proper mapping ? This View:

CREATE VIEW "album_artist" AS 
SELECT
albums.AlbumId AS RowID,
albums.Title,
artists.Name
FROM
artists
INNER JOIN albums ON albums.ArtistId = artists.Artistid

...is mapped to:

TSQLalbum_artist = class(TSQLRecord)
private
  fTitle: RawUTF8;
  fName: RawUTF8;
published
  property Title: RawUTF8 read fTitle write fTitle;
  property Name: RawUTF8 read fName write fName;
end;

Is that wrong ?

#28 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-12 19:41:05

For anyone interested in the topic, I finally solved my problem by aliasing the ID column name to"RowID".
It wouldn't work  with "ID", "id" or anything else.

So a view may be defined like this for example:

SELECT
albums.AlbumId AS RowID, /* <--- IMPORTANT !!! Doesn't work if named or aliased to ID */
albums.Title,
artists.Name
FROM
artists
INNER JOIN albums ON albums.ArtistId = artists.Artistid

#29 Re: mORMot 1 » SQLite3: how to map properties to column names ? » 2017-02-12 19:04:50

OooooK, I think I have it working now... I was doing...

Model.Props[TSQLartists].ExternalDB.MapField('ID', 'ArtistId');
VirtualTableExternalRegisterAll(Model, DBConnProps);

instead of...

VirtualTableExternalRegisterAll(Model, DBConnProps);
Model.Props[TSQLartists].ExternalDB.MapField('ID', 'ArtistId');

#30 mORMot 1 » SQLite3: how to map properties to column names ? » 2017-02-12 10:28:13

jbroussia
Replies: 2

Hi,

I have a SQLite3 table "Artists" with the following definition:

CREATE TABLE "artists" (
"ArtistId"  INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
"Name"  TEXT
);

That DB is statically linked (using SynSQLite3Static) to my server app.

How do I map the "ArtistId" field to the "ID" field required by mORMot so that I can use this model to query the DB ?

TSQLartists = class(TSQLRecord)
  private
  	fName: RawUTF8;
  published
  	property Name: RawUTF8 read fName write fName;
  end;

I don't think this is supposed to work ? (gives me an access violation)

Model.Props[TSQLartists].ExternalDB.MapField('ID', 'ArtistId');

#31 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-10 09:31:48

OK, to better illustrate my question, I created a really really simple project (still based on sample 04). Here is the archive containing the code and db file:
https://drive.google.com/file/d/0B1Ycio … sp=sharing

In my sqlite3 db I created a very simple view with an id column. I can query tables, but not the view.
Is it a limitation of the mORMot framework?

#32 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-07 14:55:06

I'm working on a legacy DB of about 500MB so it's not easy to produce a sample that I could post here.
So I'll try to explain what I'm doing with a couple screenshots:

- the DB relational structure (partial):
mini_375103capt01.png
- and the view I want to query (as you can see , the view has an ID column):
mini_355775capt02.png

This is the class mapping to that view:

TSQLmatch_detail = class(TSQLRecord)
  private
    // fMatchId: TID;
    fCompetitionName: RawUTF8;
    fArea1Name: RawUTF8;
    fTeam1Name: RawUTF8;
    fArea2Name: RawUTF8;
    fTeam2Name: RawUTF8;
    fDate: TDateTime;
  published
    // property match_id: TID read fMatchId write fMatchId;
    property competition_name: RawUTF8 read fCompetitionName write fCompetitionName;
    property area1_name: RawUTF8 read fArea1Name write fArea1Name;
    property team1_name: RawUTF8 read fTeam1Name write fTeam1Name;
    property area2_name: RawUTF8 read fArea2Name write fArea2Name;
    property team2_name: RawUTF8 read fTeam2Name write fTeam2Name;
    property date: TDateTime read fDate write fDate;
  end;

But queries to the view don't return anything (obviously it's working fine on regular tables).

For ex...

  aMatchDetail := TSQLmatch_detail.Create(Database, 1010055);

...will return an empty object:
{"ID":0,"competition_name":"","area1_name":"","team1_name":"","area2_name":"","team2_name":"","date":""}

1- So, how do I access SQLite3 Views from mORMot ?

...and...

2- How do you map SQLRecord properties to statically linked SQLite3 database column names ?

Thanks

#33 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-07 09:29:44

Can someone please confirm that accessing SQLite3 Views works from the ORM, and that it is just me doing something wrong ? I'm stuck :'-(

Thanks.

#34 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-05 19:01:14

Thank you ab, that solves the "table already exists" problem.

Now I can't seem to be able to retrieve data from my view:
- I don't think TSQLRecordName.Create(Client, someID) is a possibility as, from my limited knowledge, views don't have a ID column ? Well, I tried anyway, but it will return an empty record (with id=0).
- queries like TSQLRecordName.CreateAndFillPrepare don't return anything.

#35 Re: mORMot 1 » Using Database View's instead of Database Table's » 2017-02-05 15:47:49

Hi ab and all,

I'm trying to define a model for a SQLite3 database (using Sample 04 as a starting point).
My database contains views which I want to access from ORM.

I'm following the instructions from https://synopse.info/files/html/Synopse … #TITLE_162
but
1- I'm not sure if I should inherit my class from TSQLRecord or TSQLRecordVirtual ?
2- On Model.CreateMissingTables I get the following error:
Exception class ESQLite3Exception with message 'Error SQLITE_ERROR (1) [CREATE VIRTUAL TABLE match_detail USING Binary(match_id INTEGER, competition_name TEXT COLLATE SYSTEMNOCASE, area1_name TEXT COLLATE SYSTEMNOCASE, team1_name TEXT COLLATE SYSTEMNOCASE, area2_name TEXT COLLATE SYSTEMNOCASE, team2_name TEXT COLLATE SYSTEMNOCASE, date TEXT COLLATE ISO8601);] using 3.16.2 - table match_detail already exists, extended_errcode=1'.

Simplified code is like this:

Model := TSQLModel.Create([TSQLRecordClass1, TSQLRecordClass2], ROOT_NAME);
Model.VirtualTableRegister(TSQLRecordClass2, TSQLVirtualTableBinary); // TSQLRecordClass2 models the view
Server := TSQLRestServerDB.Create(Model, 'mydb.sqlite', TRUE);
Server.CreateMissingTables;
...

What am I dong wrong ?

Thanks.

#36 Re: mORMot 1 » 2nd Edition of mORMot book » 2017-02-04 19:04:02

erick wrote:
jbroussia wrote:

Can I login into createspace with my Amazon credentials or do I need to create a new account ?

EDIT: tried with my Amazon credentials, didn't work, so new account needed...

EDIT2: finally ordered from Amazon instead ! Book not available right now, need to wait for restocking. Got it from Amazon.fr as Amazon.de and Amazon.co.uk don't list the book.

Sorry it didn't work.  I don't know what failed - if I can't figure it out I might get rid of that option.  I know Amazon's sites work, though it might take a few days for the book to be available on all of them. 

Erick

It didn't fail. I created an account on createspace, but afterwards decided it was just easier to order from Amazon. Now I hope it won't take too long to restock...

#37 Re: mORMot 1 » 2nd Edition of mORMot book » 2017-02-03 09:29:21

Can I login into createspace with my Amazon credentials or do I need to create a new account ?

EDIT: tried with my Amazon credentials, didn't work, so new account needed...

EDIT2: finally ordered from Amazon instead ! Book not available right now, need to wait for restocking. Got it from Amazon.fr as Amazon.de and Amazon.co.uk don't list the book.

#38 Re: mORMot 1 » Problem with TSQLTableJSON.step » 2017-01-25 11:23:28

That's what I was using before, and what got me confused when I tried MultiFieldValues: I forgot to change the variable type ! (so ashamed)

#39 Re: mORMot 1 » Problem with TSQLTableJSON.step » 2017-01-23 09:10:27

Should not work on Sundays. My mind was so blurry... neutral

#40 mORMot 1 » Problem with TSQLTableJSON.step » 2017-01-22 15:41:15

jbroussia
Replies: 4

Hi ab,

While trying to use step through MultiFieldsValues records, I get an exception I can't make my mind around.
For ex. just playing with Sample 04, the following code raises an exception after the Step line...

procedure TForm1.btnTestClick(Sender: TObject);
var
  aSampleRecord: TSQLSampleRecord;
  s: RawUTF8;
begin
  with Database.MultiFieldValues(TSQLSampleRecord, 'Name,Question', 'ID >= ?', [1]) do begin
    try
      while Step(FALSE, @aSampleRecord) do begin
        s := Format('Name: %s; Question: %s', [aSampleRecord.Name, aSampleRecord.Question]);
        QuestionMemo.Lines.Add(s);
      end;
    finally
      Free;
    end;
  end;
end;

-> Project Project04Client.exe raised exception class EVariantBadVarTypeError with message 'Invalid variant type'.

I have no idea what I'm doing wrong, it's just a copy/paste from the documentation :-\
(Delphi 2010)

#41 mORMot 1 » Columns defined as float instead of real, does it matter ? » 2017-01-08 13:38:53

jbroussia
Replies: 1

Hi,

So after defining my data classes and having mORMot create the tables for me (SQLite3), I see that columns that should contain floating point values (in my case, properties defined as "double" in Delphi) end up being defined as "FLOAT" in SQLite3. Shouldn't it be "REAL" instead ? It probably doesn't matter but I'm curious about the reason behind.

Thanks.

#42 Re: mORMot 1 » My mORMot Videos » 2016-12-29 12:31:11

Have you tried with Opera browser, the last versions have an integrated VPN, could help ?

#43 Re: mORMot 1 » My mORMot Videos » 2016-12-27 20:00:44

There are extensions for your browser that allow to download Youtube videos for offline viewing.

#44 Re: mORMot 1 » TDocVariant, problem (bug ?) assigning property from another object » 2016-12-22 10:56:29

Creating a copy of the object is not a solution because in the end I need to modify the properties of the object in the array.
And I can't put back the modified object in the array using:

a._(0) := o1;	// NOPE

And I still don't understand why this works (example 2):

s := o2.Val1;
o1.Val1 := s;

And this does as well:

o1.Val1 := string(o2.Val1);

But this does not (example 3):

o1.Val1 := o2.Val1;

#45 mORMot 1 » TDocVariant, problem (bug ?) assigning property from another object » 2016-12-21 20:37:18

jbroussia
Replies: 3

Hi ab,

I'm encountering an annoying problem when trying to assign some properties from one object to another object. A few lines of code make it easier to describe it:

var
	a, o1, o2: Variant;
	s: string;
begin
	// THIS WORKS-------------------
	o1 := _Obj([]);
	o1.Val1 := 'blabla';
	o1.Val2 := 'bleble';
	
	o2 := _Obj([]);
	o2.Val1 := 'blibli';
	o2.Val2 := 'bloblo';

	o1.Val1 := o2.Val1;	// < --- OK
	o1.Val2 := o2.Val2;	// < --- OK

	Caption := o1;	// '{"Val1":"blibli","Val2":"bloblo"}'
	// -----------------------------


	// THIS WORKS-------------------
	s := '[{"Val1":"blabla","Val2":"bleble"},{"Val1":"blibli","Val2":"bloblo"}]';
	a := _Json(s);

	o1 := a._(0);
		
	o2 := _Obj([]);
	o2.Val1 := 'blublu';
	o2.Val2 := 'blybly';

	s := o2.Val1;
	o1.Val1 := s;	// < --- OK
	s := o2.Val2;
	o1.Val2 := s;	// < --- OK

	Caption := a;	// '[{"Val1":"blublu","Val2":"blybly"},{"Val1":"blibli","Val2":"bloblo"}]'
	// -----------------------------


	// THIS DOES NOT WORK-----------
	s := '[{"Val1":"blabla","Val2":"bleble"},{"Val1":"blibli","Val2":"bloblo"}]';
	a := _Json(s);

	o1 := a._(0);
		
	o2 := _Obj([]);
	o2.Val1 := 'blublu';
	o2.Val2 := 'blybly';

	o1.Val1 := o2.Val1;	// < --- NOT OK: raised exception class EVariantBadVarTypeError with message 'Invalid variant type'
	o1.Val2 := o2.Val2;	// < --- NOT OK

	Caption := a;	// Expecting '[{"Val1":"blublu","Val2":"blybly"},{"Val1":"blibli","Val2":"bloblo"}]'
	// -----------------------------

As you see, the first two examples work as expected but the third, which is a mix of the two others, raises an exception in SynCommons (function TSynInvokeableVariantType.SetProperty).
Is it normal behavior (meaning that my code is bad), or is it a bug ?

Thanks !

#46 Re: mORMot 1 » new book about mORMot » 2016-12-20 09:38:37

Thanks. Strange that it is still out of stock on Amazon FR but available on DE. And 81 cents more on DE ! :-P

#47 Re: mORMot 1 » Re-use of TQuery does not work » 2016-12-17 17:13:46

I'm using a global TQuery for all my queries (SQLite) and don't have that problem. Don't forget to Close your query before each use.

#48 Re: mORMot 1 » new book about mORMot » 2016-12-14 13:43:19

Any idea when the book will be available in Europe ? Amazon FR tells me it's not in stock.

#50 Re: mORMot 1 » Sample REST API with GET request » 2016-11-28 11:41:02

Yeah, I'm there too. Also trying to get some code from warleyalex who put videos on Youtube doing just that, but he seems to have disappeared ?

Board footer

Powered by FluxBB