#1 Re: mORMot 1 » Problem with cursors and Step » 2014-02-06 14:26:07

Hello

I'm really sorry, it works ok.

The problem is that I'm not working anymore with Mormot directly, I made the server and gave it to coworkers to finish it and they where who tell me about this issue, but the real issue was that the database wasn't returning any data, so, the cursor, rightly, was returning early. I tried it this morning on my own computer and the code works well. I have to try things out before posting here and not trust blindly in my coworkers.

Best regards and long live Mormot...

#2 mORMot 1 » Problem with cursors and Step » 2014-02-05 19:07:03

jvillasantegomez
Replies: 3

Hello all

I'm having a problem when trying to Step from a cursor. Here's some code

function TMyClass.GetCursor: ISQLDBRows;
var
  query: RawUTF8;
  stmt : ISQLDBStatement;
  cursor: ISQLDBRows;
begin
  query := 'BEGIN SOME_PACKAGE.SOME_METHOD(P_cursor => ?); END;';
  stmt := Props.NewThreadSafeStatementPrepared(query, false);
  stmt.BindCursor(1);
  stmt.ExecutePrepared;
  cursor := stmt.BoundCursor(1);
  Result := cursor;
end;

// here's the calling code
cursor :=  TMyClass.GetCursor;
while cursor.step do begin
  writeln(cursor['column1'], ' ', cursor['column2']);
end;

The problem here is that, even when cursor.FetchAllAsJSON(true); returns ok with all the data on the database, when I try to step a cursor with code it returns fast, meaning that it doesn't write any line on the console. The problem seems to be on unit SynDBOracle on function TSQLDBOracleStatement.Step(SeekFirst: boolean): boolean;

function TSQLDBOracleStatement.Step(SeekFirst: boolean): boolean;
var sav, status: integer;
begin
  if not Assigned(fStatement) then
    raise ESQLDBOracle.CreateFmt('%s.Execute should be called before Step',[fStatementClassName]);
  result := false;
  if (fCurrentRow<0) or (fRowCount=0) then  // <<---- here's the problem
    exit; // no data available at all
  sav := fCurrentRow;
  fCurrentRow := -1;
  if fColumnCount=0 then
    exit; // no row available at all (e.g. for SQL UPDATE) -> return false

when I call Step on a cursor, fCurrentRow is always -1 so the function exists, but I know that the cursor is not empty 'cause when I call cursor.fetchAllAsJson it returns all the data.

Any help??

Best Regards!

#3 Re: mORMot 1 » Filtering, sorting and paginating data » 2014-01-24 14:56:11

It works great... Thks for the quick response.

One question thought. Do you think that records are better/faster than TCollectionItem/TInterfacedCollection for deserializing JSON objects?

Best regards...

#4 Re: mORMot 1 » Filtering, sorting and paginating data » 2014-01-23 20:40:45

It seems like a good addition, but I don't really want to pass my filter params on the url. We're writing big services here and the url scheme will allways have the limitation of the number of characters you can send. This is, after all, an internal api so we can get away with passing the parameters with POST. But, how do I deserialize that JSON (passed as a POST parameter) on the server side?

JSONDecode, JSONToObject, any other way?

I think that something like this would be very much appreciated:
https://github.com/thomaserlang/delphi-json

best regards and thks...

#5 mORMot 1 » Filtering, sorting and paginating data » 2014-01-23 17:02:29

jvillasantegomez
Replies: 5

Hello all

Working with mormot i want to filter my data on the server, I'm not using the ORM or anything however, the sql stuff it's not the problem, the problem is that i don't find an easy way to pass to an Interface service this data structure por sorting, filtering and pagination:

Here's the JSONi want to pass to mormot from the client

{
  // this is for pagination
  "startrow": 0,
  "endrow": 10,
  // this is for sorting
  "sortcolumn": "somecolumn",
  "sortorder": "asc",
  // this is for filtering (it is an array of objects)
  [
    {"property": "someProperty", "operator": "LIKE", "value": "someValue"},
    {"property": "someProperty", "operator": "<=", "value": "someValue"}
  ]
}

I want to pass this json to an interface service and have the json decoded on the server. Does anyone knows how to do this?

It could be a better JSON like this, but I suppose that mormot would have a hard time parsing it:

{
  "pagination": {
    "startrow": 0,
    "endrow": 10,
  },
  "sorting": {
    "sortcolumn": "somecolumn",
    "sortorder": "asc",
  },
  "filtering": [
    {"property": "someProperty", "operator": "LIKE", "value": "someValue"},
    {"property": "someProperty", "operator": "<=", "value": "someValue"}
  ]
}

And then I want an easy way to parse that and get access to the parsed JSON from delphi

Regards...

#6 Re: mORMot 1 » OFF-TOPIC: Merry christmas... » 2013-12-26 15:00:22

Feliz Navidad a todos y próspero año nuevo...

#7 mORMot 1 » How to set custom status code » 2013-12-20 16:46:19

jvillasantegomez
Replies: 1

How would I set a custom status code on mormot?

I want, inside my services, to be able to set a custom http status code (say 500)

If possible, it would be nice to add a property to TServiceCustomAnswer for setting the status code, now you can set headers and body, but it would be nice to set the status code as well.

Regards...

#8 Re: mORMot 1 » Client monitoring session timeout or disconnection » 2013-12-17 20:51:34

I think the timeout stuff is a little ugly.

I do see the case for it, but it would be nice if the server remembers the timed out session and returns some message to the client, like:
403
Your session timed out

Or even another custom error code, because on the client we can capture this and later reconnect.

#9 Re: mORMot 1 » How to get the current user? » 2013-12-11 21:44:27

I think that sicPerSession it's the best fit for me, done some testing and everything is working so far...

best regards...

#10 Re: mORMot 1 » How to get the current user? » 2013-12-11 13:34:02

Here's how I register my services:

aServer.ServiceRegister(TMyService, [TypeInfo(IMyService)], sicPerSession)
      .DenyAll.AllowAllByName(['User']);

and the problems that i'm having are the ones stated above. I use a javascript client (written in angularjs) to access my server.

Now, if I change my servers with this:

aServer.ServiceRegister(TMyService, [TypeInfo(IMyService)], sicPerUser)
      .DenyAll.AllowAllByName(['User']);

At least the logout works ok, i'll debug the other error soon.

So, it seems that sicPerSession. I'll read the SAD to figure out what "aInstanceCreation" it's better for my services.

Best regards.

#11 Re: mORMot 1 » How to get the current user? » 2013-12-10 18:42:08

On the other hand, when I let my server running and come back after some time elapsed it would give me the same error as above. This time i debugged the process and ended up here:
Line: 33923 of mormot unit

   // first release any deprecated instances
    if fInstanceTimeout<>0 then
    for i := fInstancesCount-1 downto 0 do
      with fInstances[i] do
      if InstanceID<>0 then
      if (Inst.LastAccess<LastAccess) or
         (QWord(Inst.LastAccess)>QWord(LastAccess)+QWord(fInstanceTimeout)) then begin
        // mark this entry is empty
        {$ifdef WITHLOG}
        SQLite3Log.Add.Log(sllServiceCall,
          'Deleted % instance (id=%) after % ms timeout (max % ms)',
          [fInterfaceURI,InstanceID,Inst.LastAccess-LastAccess,fInstanceTimeOut],self);
        {$endif}
        SafeFreeInstance({$ifndef LVCL}fImplementationFreeInMainThread{$endif});   // <-- this line raises the error
      end;

I suspect that it is because the session has expired, but it mus return a 403 response and don't raise exception... or better yet, reconnect when asked.

This problem first raises on line

if not Ctxt.Authenticate then

of procedure TSQLRestServer.URI on mormot unit, so, it is effectively related to authentication.

It is worth noting that, after that, i must restart my server for it to work again, so, this is a serious error killing out my server.

Any help???
best regards

#12 Re: mORMot 1 » How to get the current user? » 2013-12-10 17:22:54

Unfortunately, a get request to this URI:
http://localhost:8080/api/auth?UserName=User&session=13165683&session_signature=00c8e473dd7c1cc0e83631c6

fails with:
{"ErrorCode":500,"ErrorText":"ExceptionEInvalidPointer:Invalidpointeroperation"}

The error raises on Line: 33941 of function TServiceFactoryServer.InternalInstanceRetrieve of MormotUnit

     // search the instance corresponding to Inst.InstanceID
      for i := 0 to fInstancesCount-1 do
        with fInstances[i] do
        if InstanceID=Inst.InstanceID then begin
          if aMethodIndex<0 then begin
            // aMethodIndex=-1 for {"method":"_free_", "params":[], "id":1234}
            SafeFreeInstance({$ifndef LVCL}fImplementationFreeInMainThread{$endif});  // <-- this line raises the error
            result := true; // successfully released instance
            exit;
          end;
          LastAccess := Inst.LastAccess;
          Inst.Instance := Instance;
          exit;
        end;

That line, going deeper, calls this function (an stdcall) on System unit wich produces the error:
function InterlockedDecrement(var Addend: Integer): Integer; stdcall;
  external kernel name 'InterlockedDecrement';

Any help?
Best regards...

#14 Re: mORMot 1 » How to get the current user? » 2013-12-09 21:46:58

Now remains the issue of how to log out, this url always returns an error: 500 (Internal Server Error)

GET http://localhost:8080/api/auth?UserName=User&session=29886586&session_signature=01c8087ad952578cf587857c

best regards

#15 Re: mORMot 1 » How to get the current user? » 2013-12-09 21:40:39

I've solved the issue. The Par variable content are the parameters. It seems that when you do a POST even when you don't need to pass any parameters you need to pass and empty array like this:

requestCurrentUser: function() {
      var url = Settings.mainUrl + '/Users.GetCurrent';
      var token = getSessionSignature(url.substr(Settings.serverUrl.length + 1));
      $.ajax({
      type: "POST",
      dataType: "json",
      url: url + '?session_signature=' + token,
      data: [],
      contenttype: 'application/json; charset=utf-8',
      timeout:2000,
      success: function(response, textStatus, jqXHR) {
        return currentUser;  // As saved on local storage
      },
      error: function (jqXHR, textStatus, errorThrown) {
        console.log(jqXHR);
      }
    });

so data: [] would solve the issue right away... Thks for the responses...

#16 Re: mORMot 1 » How to get the current user? » 2013-12-09 20:25:36

My TResponse type is simply a record that i return to the client:

TResponse = record
    count: integer;
    data: RawUTF8;
  end;

with that record i use this global function to return the response to the client:

function TBase.WrapResponseSuccess(response: TResponse; code, status: string): RawUTF8;
begin
  Result := JSONEncode([
    'code', code,
    'status', status,
    'count', response.count,
    'data', response.data
  ]);
end;

in my services I always return a TServiceCustom Answer like this:

function TService.MyService(someparam: RawUTF8): TServiceCustomAnswer;
var
  response: TResponse;
begin
  try
    try
      response := Self.controller.SomeFunction(someparam);

      Result.Header := JSON_CONTENT_TYPE_HEADER;
      Result.Content := WrapResponseSuccess(response);
    finally
      // ...
    end;
  except
    On E: Exception do begin
      Result.Header := JSON_CONTENT_TYPE_HEADER;
      Result.Content := WrapResponseError(E.Message, E.ClassName);
    end;
  end;
end;

#17 Re: mORMot 1 » How to get the current user? » 2013-12-09 20:06:20

I understand that.

For now, here's the line that raises the exception on the mORmot unit: Ctxt.ExecuteSOAByInterface

procedure TSQLRestServer.URI(var Call: TSQLRestURIParams);
var Ctxt: TSQLRestServerURIContext;
{$ifdef WITHSTATPROCESS}
    timeStart,timeEnd: Int64;
begin
  QueryPerformanceCounter(timeStart);
{$else}
begin
{$endif}
  inc(fStats.fIncomingBytes,length(Call.url)+length(call.method)+length(call.InHead)+length(call.InBody)+12);
  Call.OutInternalState := InternalState; // other threads may change it
  Call.OutStatus := HTML_BADREQUEST; // default error code is 400 BAD REQUEST
  Ctxt := ServicesRouting.Create(self,Call);
  try
    {$ifdef WITHLOG}
    Ctxt.Log := SQLite3Log.Add;
    Ctxt.Log.Enter(Self,pointer(Ctxt.URIWithoutSignature),true);
    {$endif}
    if Ctxt.Method=mNone then
      Ctxt.Error('Unknown VERB') else
    // 1. decode URI
    if not Ctxt.URIDecodeREST then
      Ctxt.Error('Invalid Root',HTML_NOTFOUND) else begin
      Ctxt.URIDecodeSOAByMethod;
      if (Ctxt.MethodIndex<0) and (Ctxt.URI<>'') and
         (reService in Call.RestAccessRights^.AllowRemoteExecute) then
        Ctxt.URIDecodeSOAByInterface;
      // 2. handle security
      if not Ctxt.Authenticate then // 403 in case of authentication failure
        // 401 Unauthorized response MUST include a WWW-Authenticate header,
        // which is not what we used, so we won't send 401 error code but 403
        Call.OutStatus := HTML_FORBIDDEN else
      // 3. call appropriate ORM / SOA commands
      try
        if Ctxt.MethodIndex>=0 then
          Ctxt.ExecuteSOAByMethod else
        if Ctxt.Service<>nil then
          Ctxt.ExecuteSOAByInterface else     // <<- this line raises 500 (Internal Server Error) 
        if  Ctxt.Method in [mLOCK,mGET,mUNLOCK,mSTATE] then
          // handle read methods
          Ctxt.ExecuteORMGet else
        // write methods (mPOST, mPUT, mDELETE...) are locked
        if AcquireWrite(Ctxt) then // make it thread-safe and transaction-safe
        try
          Ctxt.ExecuteORMWrite;
        finally
          ReleaseWrite;
        end else
          // AcquireWrite(SessionID) returned false (e.g. endless transaction)
          Call.OutStatus := HTML_TIMEOUT; // 408 Request Time-out
      except
        on E: Exception do
          Ctxt.Error('Exception %: %',
           [PShortString(PPointer(PPtrInt(E)^+vmtClassName)^)^,E.Message],
           HTML_SERVERERROR); // 500 internal server error
      end;
    end;
    // 4. returns expected result to the client and update Server statistics
    if (Call.OutStatus in [HTML_SUCCESS,HTML_CREATED]) or
       (Call.OutStatus=HTML_NOTMODIFIED) then begin
      inc(fStats.fResponses);
      {$ifdef WITHLOG}
      Ctxt.Log.Log(sllServer,'% % -> %',[Call.Method,Ctxt.URI,Call.OutStatus],self);
      {$endif}
    end else begin
      inc(fStats.fInvalid);
      if Call.OutBody='' then // if no custom error message, compute it now as JSON
        Ctxt.Error(Ctxt.CustomErrorMsg,Call.OutStatus);
    end;
    inc(fStats.fOutcomingBytes,length(Call.OutHead)+length(Call.OutBody)+16);
    if (Call.OutBody<>'') and
       (length(Call.OutHead)=Length(STATICFILE_CONTENT_TYPE_HEADER)) and
       (Call.OutHead[Length(HEADER_CONTENT_TYPE)+1]='!') and
       IdemPChar(pointer(Call.OutHead),STATICFILE_CONTENT_TYPE_HEADER) then
      inc(fStats.fOutcomingFiles);
    if (Ctxt.Static<>nil) and Ctxt.Static.fOutInternalStateForcedRefresh then
      // force always refresh for Static table which demands it
      Call.OutInternalState := cardinal(-1) else
      // database state may have changed above
      Call.OutInternalState := InternalState;
    {$ifdef WITHSTATPROCESS}
    QueryPerformanceCounter(timeEnd);
    inc(fStats.ProcessTimeCounter,timeEnd-timeStart);
    {$endif}
  finally
    Ctxt.Free;
  end;
end;

It seems that the authentication works, but the problem is while calling my service implementation...

Going deeper, this line is the one that raises the exception on mORmot Unit:

if not fInterface.fMethods[Ctxt.ServiceMethodIndex].InternalExecute(
          [PAnsiChar(Inst.Instance)+entry^.IOffset],Ctxt.ServiceParameters,WR,
           Ctxt.Call.OutHead,fExecution[Ctxt.ServiceMethodIndex].Options,
           Ctxt.ForceServiceResultAsJSONObject)

And here's the exit command that raises the exception on mORmot unit method: TServiceMethod.InternalExecute

if Par^ in [#1..' '] then repeat inc(Par) until not(Par^ in [#1..' ']);
    if Par^<>'[' then
      Exit;

That conditions and exit are very cryptic to me... sad
Any help??

#18 Re: mORMot 1 » How to get the current user? » 2013-12-09 19:28:22

The problem is that i don't know how to debbug it.

Mormot is like an onion (or a route middleware for people comming from standard frameworks), it seems that the problem is on the authentication routine so, the request never hits my service implementation. I'm using TSQLRestServerAuthenticationDefault and i don't know where to place the breakpoint to debug this, can you hint me on this?

Also, i don't know how to effectively log out a user and for long running servers it start giving "Invalid pointer operation" exception when I try to access my server. If you hint me about where to start debbugging the authentication stuff i will debug this out and write here.

As a side note, it would be great to support another kind of authentication, HMAC authentication without any server side session (as seen here: http://www.thebuzzmedia.com/designing-a … tication/), this is great for people writting an api, after i debug the problem with TSQLRestServerAuthenticationDefault  and solve it I would give a go at writing an hmac authentication for mormot, i've done it dozens of times on php so, it wouldn't be so hard to write on delphi.

Best regards.

#19 mORMot 1 » How to get the current user? » 2013-12-09 16:10:18

jvillasantegomez
Replies: 17

When writing a client with javascript I want to get the current logged in user whenever the user reloads the page. This is a SPA but the user can reload the page on the browser and I don't wanna him to authenticate again.

I wrote this Service on mormot:

function TUsers.GetCurrent: TResponse;
var
  context: TServiceRunningContext;
begin
  context := CurrentServiceContext;
  Result.data := IntToStr(context.Request.Session);
  Result.count := 1;
end;

That work well after authentication, but when the user reloads the page, after authentication, it returns a 500 (Internal Server Error).

Here's what i want to do in javascript:

    requestCurrentUser: function() {
      var url = Settings.mainUrl + '/Users.GetCurrent';
      var token = getSessionSignature(url.substr(Settings.serverUrl.length + 1));
      $.ajax({
      type: "POST",
      dataType: "json",
      url: url + '?session_signature=' + token,
      contenttype: 'application/json; charset=utf-8',
      timeout:2000,
      success: function(response, textStatus, jqXHR) {
        return currentUser;  // As saved on local storage
      },
      error: function (jqXHR, textStatus, errorThrown) {
        console.log(jqXHR);
      }
    });

Here's the use case. A users effectively logs in on our application, mormot accepts his credentials and start a session for him, this session lasts about 60minutes.
The user closes his browers and reopen it, he must be authenticated by default because on the server the session is still valid but when I call the functions above it returns 500 (Internal Server Error)

Best regards

Also, how to effectively do a logout from javascript... this url always fails:

  var url = Settings.mainUrl + '/auth' + '?session=' + sessionId + '&UserName=' + userName;

#20 mORMot 1 » Exception when returning NULL from oracle database » 2013-12-03 21:37:53

jvillasantegomez
Replies: 15

It happens that sometimes you want to return null from a select, this we do it to later have a unified interface on the client, so, you could easily do something like this:

SELECT one, two, NULL AS three FROM sometable;

But the problem here is that mORmot raises an exception (understandable) when you do stuff like that. So, i changed the following lines to let mormot return some null values from a db query:

Line: 2686 
    (original) => if ColumnValueDBSize=0 then    
    (change) => // if ColumnValueDBSize=0 then
  Line: 2687 
    (original) => raise ESQLDBOracle.CreateFmt('Column "%s": size=0',[ColumnName]);
    (change) => // raise ESQLDBOracle.CreateFmt('Column "%s": size=0',[ColumnName]);

only commented out those (2686 and 2687 on latest commit) two lines smile

best regards...

#21 Re: mORMot 1 » Problem with Oracle Cursors » 2013-12-03 20:47:35

IT IS WORKIN GREAT NOW!!

Thanks for your commit, I knew that you would do a much better job than me.

//==================================

A question: Any reason why I'm receiving this error when i try to post a new thread?

An error was encountered
Error: Could not connect to smtp host "mail.gandi.net" (110) (Connection timed out).

I found another thing about oracle (it will drive me mad), made some changes and it is working now, i wanted to post the solution (hack until you review it) for future users of the framework.

best regards...

#22 Re: mORMot 1 » Problem with Oracle Cursors » 2013-12-03 18:10:34

Ok, this is a hack.

I changed line 2792 of SynDBOracle from:

fRowCount := (fInternalBufferSize - ColCount shl 4) div RowSize;

to

fRowCount := 0;

Now cursors are returning (number of actual records in DB) + 1 and that one empty record i slice it on the client end.
This is a super hack until anyone finds a better solution.

The problem seems to be with the fRowCount variable, since we're returning cursors, but i'm not really sure.
I'm really trying to push mormot forward at work, they won't buy into it (yet) but today i found a time to debug this a little, if anyone finds a better solution please write here...

I'll keep on this
until then... best regards.

#23 Re: mORMot 1 » Problem with Oracle Cursors » 2013-12-02 18:38:09

I really want to help here and i been debugging this when my time permits it, but as i said before, i'm not very good at delphi, just trying to publish some rest services on delphi to work on a javascript client at work.

Anyways, i'll keep trying debuggin it to see if i can find the culprit. In the meantime, if i use ZEOS do you think that it would handle cursors?

For what is worth i'm not blaming anyone here, i just think that if you don't support oracle cursors you just can't call your framework as working with oracle, because it crearly doesn't. The SAD clearle said that oracle is a supported database, but as you can see mormot doesn't support oracle well, cursors, functions, are standard oracle stuff that you simply don't support.

On the other hand, i think that it is very important to know, from a user standpoint, if any lib/framework really works on production... academic discussions are great, but some people just need to get shit done fast. Any framework out there worth would have a sesion on it's documentation pointing out production uses, you can find things like http://builtwithlaravel.com/, http://builtwithbootstrap.com/, http://backbonejs.org/#examples, builtwith.angularjs.org/... etc... This is an important point for deciding if using the framework is worth the time invested learning it, as important for working people as good documentation and good community support.

Being from Cuba i don't really have a chance to work on the open source community, but as you said, it is better to contribute than to blame... I'm really doing my best, i'm posting what would be my own lack of knowledge with the hope that others could help too.

Anyways, great framework...

#24 Re: mORMot 1 » Problem with Oracle Cursors » 2013-12-02 16:05:25

Is anyone really using mORmot on a production environment??

I (sadly) find it so buggy that i don't think anyone is using it for a real production server with a real database, but i could (hope) be wrong.

#25 Re: mORMot 1 » Problem with Oracle Cursors » 2013-11-29 16:04:47

I'm posting some code to get some context;

Here's a simple table with only one record:

CREATE TABLE MyTable
(
  TEXTO  VARCHAR2(4000 BYTE)
)

Here's the oracle procedure

PROCEDURE DUMMY(
    P_cursor OUT SYS_REFCURSOR
) IS
  BEGIN
    OPEN P_cursor FOR
	SELECT TEXTO FROM MyTable;
END dummy;

And here's the delphi calling code

var
  query: RawUTF8;
  stmt: ISQLDBStatement;
  cursor: ISQLDBRows;
begin
  query :=  'BEGIN SOME_PKG.DUMMY(P_cursor => ?); END;';
  stmt := Props.NewThreadSafeStatementPrepared(query, false);
  stmt.BindCursor(1);
  stmt.ExecutePrepared;
  cursor := stmt.BoundCursor(1);     
  Result := cursor.FetchAllAsJSON(true);
end;

Even with only one record on the table called MyTable, this is what cursor.FetchAllAsJSON(true) is returning:

[
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "" },
  { "TEXTO": "Some texto" }
]

As can be seen, the only record that should be returned was the last one ({"TEXTO": "Some texto"} as it is the only record on the MyTable table, but somehow mormot is returning lots of empty records.

Any help would be much appreciated!!!

#26 Re: mORMot 1 » Calling Oracle stored procedures and functions » 2013-11-28 16:35:55

Sorry

After some tests i found out that cursors are returning some empty records that don't belong to my dataset as stored in the db.
Whe you do Result := cursor.FetchAllAsJSON(true); it donsn't return my db records as expected, but it returns them mixin in some empty records.

best regards...

#27 mORMot 1 » Problem with Oracle Cursors » 2013-11-28 16:31:06

jvillasantegomez
Replies: 11

Hi

The implementation of oracle cursors seems to be broken.

I don't know the cause, but when you do Result := cursor.FetchAllAsJSON(true); it returns empty records that are not on my dataset.

I I do a select only, it works great, but with cursors it returns a json with some records empty.

Any help??

#28 Re: mORMot 1 » mORmot Patterns: Database Connection. » 2013-11-14 17:48:34

Yeah, you are right, buth the question here is where/when to call TSQLDBOracleConnectionProperties.CreateWithCodePage(...). After that I always use TSQLDBConnectionProperties.NewThreadSafeStatementPrepared() all around my code.

It seems that you can go away with calling TSQLDBOracleConnectionProperties.CreateWithCodePage(...) once on server startup, but i'm not really sure. I come from a dynamic language background (php, node.js, ruby) and don't know how to really do things with delphi lasting running proccesses.

So, here's the question again.
If you just need to call TSQLDBOracleConnectionProperties.CreateWithCodePage(...) once on server startup, then a singleton does a good job, but, is that a persistent connection against oracle database?.

But if that is not a good solution then how to do it right?

Best regards...

#29 Re: mORMot 1 » mORmot Patterns: Database Connection. » 2013-11-14 13:40:03

I'm posting the code for my DBSingleton class, I'm new to delphi and don't really know if this is the right way to code a singleton, but forget about implementation, the important thing is that this class creates a TSQLDBOracleConnectionProperties once, then, whenever I need a connection i just reference that one connection like: Global_DBSingleton.GetConnection;

feedback is welcome!

unit uDBSingleton;

interface

uses
  SysUtils, SynDbOracle;

type
  TDBSingleton = class(TObject)
  public
    constructor Create;
    destructor Destroy; override;
    function GetConnection: TSQLDBOracleConnectionProperties;
  protected
    fProps: TSQLDBOracleConnectionProperties;
  end;

var
  Global_DBSingleton: TDBSingleton;

implementation

uses
  uCommons;

constructor TDBSingleton.Create;
begin
  if Global_DBSingleton <> nil then
    raise Exception.Create('Global DB Singleton already created!')
  else
    Global_DBSingleton := Self;
end;

destructor TDBSingleton.Destroy;
begin
  if Assigned(fProps) then
    fProps.Free;
  Global_DBSingleton := nil;

  inherited Destroy;
end;

function TDBSingleton.GetConnection: TSQLDBOracleConnectionProperties;
begin
  if not Assigned(Global_DBSingleton) then
    raise Exception.Create('Global DB Singleton not created!. You must call Create once.');

  if not Assigned(fProps) then
    fProps := TSQLDBOracleConnectionProperties.CreateWithCodePage(SERVER_NAME, USER_ID, USER_PASSWORD, 0);

  Result := fProps;
end;

end.

#30 Re: mORMot 1 » mORmot Patterns: Database Connection. » 2013-11-13 19:16:36

Well, I ended up creating a singleton db class. So whenever I need a TSQLDBOracleConnectionProperties i just call DB.getConnection() to get it. The connection is initialized just once i i let TSQLDBConnectionProperties.NewThreadSafeStatementPrepared() to handle threads issues.

Another approach is to create the same singleton, and on every service request do a DB.getConnection.NewConnection but it could be an overhead.

What do you think about it?

#31 mORMot 1 » mORmot Patterns: Database Connection. » 2013-11-13 15:20:21

jvillasantegomez
Replies: 6

Hello

It is time to start writing and discussing about good patterns in mORmot. Here's one of them for database connections, all samples are with Oracle but I think that the pattern will stand any database server.

Sqlite is a very neat piece of software, but the reality is that we always use something else as our backend RDBMS, so, at some point the question comes: I'm having too many connections against my database?

Here's how i'm approaching the subject, note that i'm using mORmot authentication and I want it to be on oracle.

When I create my server I have this lines:

aProps := TSQLDBOracleConnectionProperties.Create(SERVER_NAME, DATABASE_NAME, USER_ID, USER_PASSWORD);
aModel := TSQLModel.Create([TSQLAuthGroup, TSQLAuthUser], WA_ROOT_NAME);
VirtualTableExternalRegisterAll(aModel, aProps, false);

aServer := TSQLRestServerDB.Create(aModel, GetRealPath(WA_DB_NAME), true);
aServer.AuthenticationRegister(TSQLRestServerAuthenticationDefault);
aServer.CreateMissingTables(0);
aServer.NoAJAXJSON := False;

aServer.ServiceRegister(TService, [TypeInfo(IService)], sicPerSession).DenyAll.AllowAllByName(['User']);

aHTTPServer := TSQLHttpServer.Create(PORT_NAME, [aServer], '+', useHttpApiRegisteringURI);
aHTTPServer.OnlyJSONRequests := False;
aHTTPServer.AccessControlAllowOrigin := '*'; // for AJAX requests to work

With that I ensure that both AuthGroup and AuthUser tables will be created on Oracle and that the authentication will check that tables, meaning that the ORM will use Oracle from this point on. I don't really use the ORM in my code (i like SQL very much) but the authentication mecanism seems to use it. All this goes on my server creation code (think the main method that starts everything).

After that i organize the rest of my code (interface base service) with three discrete pieces. I have services (of course), controllers and repositories. Of them, the repositories will need an oracle connection (TSQLDBOracleConnectionProperties), and here comes the first question. Do i can reuse the connection created on my server initialization code or do i open new connections everytime.

What i ended up doing was to to create a new connection for every interface i have, so, every interface base service will have it's own
fProps := TSQLDBOracleConnectionProperties.Create(SERVER_NAME,DATABASE_NAME,USER_ID, USER_PASSWORD);, and since i defined my services as sicPerSession i assure that every user will have it's own connection to oracle, but this will be for every service i define, and I mean to define a lot of them.

That approach works great, until i wanted to test my server endpoint with apache jmeter, oracle could'nt get the charge... effectively "too many connections"

So, how is everyone here approaching this pattern?. I think that i could create a singleton (DbConnection) open the connection once and share it for everyone that needs it, but it would be better to create a pool of connections. The problem here is that Delphi runs as a lasting process, I think that one connection per request would be a nice compromise, but i'm not sure how to do it.

Comments on this thread will be much appreciated. I will come up with more patterns of this great framework over time.

Best regards...

#32 Re: mORMot 1 » Calling Oracle stored procedures and functions » 2013-11-12 19:23:20

It now works like a charm.

Now ISQLDBStatement has the ParamToVariant() method and you can bind to oracle cursor.
For the sake of completness it would be great if one could call oracle functions, but I can live without it.

Thks for your quick commits to the great mORmot framework.

#33 Re: mORMot 1 » Calling Oracle stored procedures and functions » 2013-11-12 14:35:10

Hello

First of all, thaks very much for your contributions to the community. You are doing a great work with mORmot.

Now, i'm trying the cursor functionality but it doesn't work so far.

here's a simple procedure on oracle:

PROCEDURE dummy (
    P_cursor OUT Globales.T_c_weak_cursor,
    P_id     IN Number
  ) IS
  BEGIN
    OPEN P_cursor FOR
		    SELECT * FROM some_table T WHERE T.Id = P_id
  END dummy;

And here's the calling delphi code:

var
  id: RawUTF8;
  query: RawUTF8;
  stmt: ISQLDBStatement;
  cursor: ISQLDBRows;
begin
  id := '2000';
	
  query :=  'BEGIN SOME_PKG.DUMMY(P_cursor => ?, P_id => ?); END;';
  stmt := Props.NewThreadSafeStatementPrepared(query, false);
  stmt.BindCursor(1);
  stmt.BindTextU(2, id, paramIn);
  stmt.ExecutePrepared;
  cursor := stmt.BoundCursor(1);     
  Result := cursor.FetchAllAsJSON(true);
end;

If Your execute that, the line cursor := stmt.BoundCursor(1); will throw this error: 'No BindCursor() on Param #2'.
Now, if you change the line cursor := stmt.BoundCursor(1); to cursor := stmt.BoundCursor(0); it will show this error: 'Prepare called with wrong ExpectResults', wich is odd because as the cursor is an out parameter is ok to call prepare with false.
Now, if you then change the line stmt := Props.NewThreadSafeStatementPrepared(query, false); to stmt := Props.NewThreadSafeStatementPrepared(query, true); Now that same line gives the error: Prepare called with wrong ExpectResults', wich is odder.

So, what gives??
The error raises on line 2603 on unit SynDBOracle because this line is true: if fExpectResults<>(StatementType=OCI_STMT_SELECT);
I haven't worked with OCI before, but it seems that is waiting for a select when in reality i'm just returning a cursor as an OUT parameter, so fExpectResults is false and different from OCI_STMT_SELECT.

Best regards...

#34 Re: mORMot 1 » Calling Oracle stored procedures and functions » 2013-11-11 21:29:00

I have a lot of work to do, but as soon as i get a chance i will try the cursor functionality. Maybe tomorrow or the day after.

Thaks for contributing that much to the community.

#35 Re: mORMot 1 » Calling Oracle stored procedures and functions » 2013-11-11 14:19:18

Never Mind

Here's how to doit:

  stmt := Self.Props.NewThreadSafeStatement;
  stmt.Prepare(query, false);
  stmt.Bind(1, ..., paramIn);
  stmt.BindTextU(2, ..., paramIn);
  stmt.BindTextU(3, ..., paramIn);
  stmt.BindTextS(4, '', paramOut);
  stmt.Bind(5, 0, paramOut);
  stmt.ExecutePrepared;
  stmt.ParamToVariant(4, out1, true);
  stmt.ParamToVariant(5, out2, true);

You can't use Props.NewThreadSafeStatementPrepared(query, false); because that doesn't returns a TSQLDBStatement.

Thks for your response.

#36 Re: mORMot 1 » Calling Oracle stored procedures and functions » 2013-11-11 13:13:16

Hello

The problem is that the statement Self.Props.NewThreadSafeStatementPrepared returns a ISQLDBStatement that doesn't have any method called ParamToVariant, so, do i have to cast or something??

Regards...

#37 mORMot 1 » Calling Oracle stored procedures and functions » 2013-11-07 19:44:58

jvillasantegomez
Replies: 10

Hello

Can anyone post some code or example of calling a stored procedure and/or function from oracle pl/sql.

I can't make it work and this is basic stuff... sad

I want to get values by out parameters or by return function of oracle.

Any Help would be much appreciated!!!

Here's some code that sadly doesn't work AT ALL!!!

query :=  'BEGIN TEST_PKG.DUMMY(?, ?, ?, ?, ?); END;';
stmt := Props.NewThreadSafeStatementPrepared(query, false);

stmt.Bind(1, param1, paramIn);
stmt.BindTextU(2, param2, paramIn);
stmt.BindTextU(3, param3, paramIn);
stmt.BindTextS(4, param4, paramOut);
stmt.Bind(5, param5, paramOut);
stmt.ExecutePrepared;

In here param1, param2 and param3 are parameters that go to the procedure for it to work. This are effectively IN bound parameters.
param4 and param5 are the response for the procedure call. They are effectively OUT bound parameters.
Inside the procedure the last 2 params (param4 and param5) are filled up with some data, but back in Delphi i doesn't work at all, I always get back a wrong answer in param4 and param5.

The blog is full of benchmarks but no so many code samples.... sad

PLEASE HELP!!!

#39 Re: mORMot 1 » How to bind to a cursor on oracle » 2013-11-07 16:58:33

WoW

I will do that, wrap all the functions into procedures with out parameters, but again, it is a shame that SynDBOracle doesn't supports calling functions from oracle.

Thks for your response.

#40 Re: mORMot 1 » How to bind to a cursor on oracle » 2013-11-07 14:27:55

I've addedd a feature request for cursor support, it seems like a very good addition to the mormot framework.

Now, I'm still having problems trying to call a funtion from oracle... How do you bind the return value of the function?
Here's what i've got so far:

FUNCTION dummy (
  param1 IN integer, 
  param2 IN integer,
  param3 IN string,
  param4 OUT string
) RETURN INTEGER;

query :=  'BEGIN ? := dummy(?, ?, ?, ?); END;';
stmt := Props.NewThreadSafeStatementPrepared(query, false);
stmt.Bind(1, value, paramOut);
stmt.Bind(2, param1, paramIn);
stmt.BindTextU(3, param2, paramIn);
stmt.BindTextU(4, param3, paramIn);
stmt.BindTextS(5, param4, paramOut);
stmt.ExecutePrepared;

                     

In here, param4 is a variable declared as string and will be filled by the function call.
value is declared as an integer and will be the return value of the function itself.

after the call, i want this two variables to be filled by the stored proc call
value equals the return value of the funcion
param4 equals the OUT param of the function wich is filled inside the function.

Saddly i don't get it to work yet.

My problem with oracle and all this is that i'm evolving a big codebase and I want to reuse the functions/procedures written in oracle, but so far i'm having trouble with it. Can anyone please help??

#41 Re: mORMot 1 » How to create a custom json wrapper » 2013-11-07 13:37:46

Thks for your reply

I will end up using

/// encode the supplied data as an UTF-8 valid JSON object content
// - data must be supplied two by two, as Name,Value pairs, e.g.
// ! JSONEncode(['name','John','year',1972]) = '{"name":"John","year":1972}'
// - note that cardinal values should be type-casted to Int64() (otherwise
// the integer mapped value will be transmitted, therefore wrongly)
function JSONEncode(const NameValuePairs: array of const): RawUTF8;

It seems to be very fast and good enough for what i'm trying to do.

#42 Re: mORMot 1 » How to create a custom json wrapper » 2013-11-06 21:28:01

Well, I've found a solution to this problem

It seems that the custom object must be declared between {$M+} ... {$M-} to have RTTI information at serialization time. Here's the declaration of the TResponse custom object for it to work:

{$M+}
  TResponse = class
  private
    fData, fMensaje: RawUTF8;
  published
    property Data: RawUTF8 read fData write fData;
    property Mensaje: RawUTF8 read fMensaje write fMensaje;
  end;
{$M-}

With that i can get the json with javascript, but i have to do a JSON.parse(...) for it to work, with the default it worked out of the box. Anyways, in the jquery success callback you can easily do this:

success: function(data, textStatus, jqXHR) {
  var m = JSON.parse(data.Mensaje);
  var d = JSON.parse(data.Data);
  // here m equals 'some message' and d equals the result of the db query...
}

I have lost the id passed to the client, but since i'm using a browser (javascript) as the client i guess it is not a big deal, as far as i know sicClientDriven is just used when you have a Delphi client for your mormot server.

Now I'm wondering about the performance hit of this kind of response, I'm thinking about make this kind of wrapped response the default for all my services, does it has an impact on performance this kind of Object to JSON serialization for every request??

#43 Re: mORMot 1 » How to create a custom json wrapper » 2013-11-06 20:42:32

Now, If I add to my services this line:

Result.Header := JSON_CONTENT_TYPE_HEADER;

then here's the result i'm getting:

{}

Yep, an empty json... what's going on here??

#44 Re: mORMot 1 » How to create a custom json wrapper » 2013-11-06 20:24:22

It dosn't work

I defined this class:

TResponse = class
  public
    fData, fMensaje: RawUTF8;
  published
    property Data: RawUTF8 read fData write fData;
    property Mensaje: RawUTF8 read fMensaje write fMensaje;
  end;

Then in my service i did this:

begin
  repository := TRepository.Create(fProps);
  data := repository.GetById(id);  // This returns data according to Result := stmt.FetchAllAsJSON(true);

  response := TResponse.Create;
  response.Data := data;
  response.Mensaje := 'some message';
  Result.Content := ObjectToJson(response);
  response.Free; // do not forget to release memory!
end;

After that here's the json i get back:

{"result":["￰AAJ7fQ=="],"id":25443627}

Some unreadeable json when i'm expecting some result set from the database call in response.Data and the message i set in response.Mensaje.
Of course, I defined the service with a return type of TServiceCustomAnswer.

#45 Re: mORMot 1 » How to bind to a cursor on oracle » 2013-11-06 20:01:07

You're right, Oracle dosn't use standard stuff, it is the Microsoft of the database universe....

Now, Another related question...
Is it posible to call an oracle stored proc with 3 params in and one param out but also returning something... here's an example function spec on oracle:

FUNCTION dummy (
  param1 IN integer, 
  param2 IN integer,
  param3 IN string,
  param4 OUT string
) RETURN INTEGER;

In here i want to call this procedure and get back an integer result, also get back some string on param4 wich is an out param
Is that posible on mormot?? and if so who would i do it??

Here's what i got so far but it doesn't work

query :=  'call dummy (?, ?, ?, ?)';
  stmt := Props.NewThreadSafeStatementPrepared(query, false);  // it is false here??, remember, it is a function with returns some integer
  stmt.Bind(1, param1, paramIn);
  stmt.BindTextU(2, param2, paramIn);
  stmt.BindTextU(3, param3, paramIn);
  stmt.BindTextS(4, param4, paramOut);
  stmt.ExecutePrepared;                        // How do i get the returned integer from the function??

In here, param4 is a variable declared as string and will be filled by the function call, but i also want to get a handle to the function return variable.

Is this possible at all??

This is really important for me, it is ok if i can't call a procedure or function returning a cursor, after all it is not standard sql, but i must be able to call other kinds of procedures and/or functions from oracle. Please, any help would be much appreciated.

#46 Re: mORMot 1 » How to bind to a cursor on oracle » 2013-11-06 17:13:57

The use of stored procs is not just about to speed up the inserts, i'm using them to get data back from oracle with pretty convulated queries that would be a headache to write in a delphi string and because i'm working with an existing codebase that has a lot of the domain logic on oracle pl/sql. All the data retrieve procedures return a cursor and is a shame that i can't use that with mormot.

There seems not to be a solution for this wich, as i said, is a shame because every oracle developer out there knows that they has to return a cursor for any client (java, .NET, php) to work with the result set.

Anyways, i'm investigating converting the cursor to JSON on oracle and returning the json string to the caller. If i can get it working i'll post my solution here.

Mormot is indeed a great framework but it has its shortcomings.

#47 Re: mORMot 1 » How to create a custom json wrapper » 2013-11-06 16:48:47

Here is what I thinking:

TRespond = class
  private
    fData: RawUTF8;
    fMessage: RawUTF8;
  published
    property Data: RawUTF8 read fData write fData;
    property Message: RawUTF8 read fMessagewrite fMessage;
  end;

Then on any of my services:

function TService.GetById(id: RawUTF8): RawJSON;
var
  repository: TRepository;
  data: RawUTF8;
  respond: TRespond;
begin
  repository := TRepository.Create(fProps);
  data := repository.GetById(id);

  respond := TRespond.Create;
  respond.fData := data;
  respond.fMessage := 'some message';
  Result := ObjectToJson(respond);
end;

But sadly it doesn't work. Also, how could i get a handle to any errors for a custom json error response. I think it would be a good idea to describe on the docs the request/response cycle of an interface based service in mormot.

any suggestions???

#48 mORMot 1 » How to create a custom json wrapper » 2013-11-06 16:42:55

jvillasantegomez
Replies: 9

Ok

Mormot seems like a nice solution at first when you have a large codebase in delphi for evolving that codebase to a restfull representation, but sometimes i can't even find an easy way of doing things.

Here is my question: How can i create a custom json wrapper and return that from an interface based service??

For example, i want that all my methods in an interface based service to return stuff like this

A successful response:
{
  "code":200,
  "status":"success",
  "data": "some data from the service"
}

An error response:
{
  "code":500,
  "status":"error",
  "developerMessage":"description of the problem for the app developer",
  "userMessage": "Pass this message to the app user",
  "morInfo": "some url to get more info"
}

This is pretty normal when using php or any other server side technology, but, how can i make this kind of wrapped response to work with mormot?

#49 Re: mORMot 1 » How to bind to a cursor on oracle » 2013-11-06 14:49:12

It is sad indeed.

Oracles diverges in this respect to any other major database systems, so, by now we can't just use a cursor nor in an out parameter nor in a return value from a function. It is sad indeed, i will end up sending the sql string from delphi to oracle.

Thaks for your response.

#50 Re: mORMot 1 » How to bind to a cursor on oracle » 2013-11-06 14:00:43

Hello

It seems that you can't have an oracle stored proc that just returns a select. You must always return something, by a cursor out variable or doing a select ... into. Here's some code:

procedure dummy(p_id integer in) is
begin
  Select * from whatever;
end;

You can't make this to work on oracle, you must return a cursor for it to work.

My question then is, if I have to declare this types of procedures like this:

procedure dummy(P_cursor IN OUT Globales.T_c_weak_cursor, p_id IN integer) IS
  BEGIN
    OPEN P_cursor FOR
      Select * from whatever;		  
END

How can i use that with mormot, meaning, how can I call that last stored proc inside mormot?

Any help would be much appreciated.

Board footer

Powered by FluxBB