You are not logged in.
Hello
I'm really sorry, it works ok.
The problem is that I'm not working anymore with Mormot directly, I made the server and gave it to coworkers to finish it and they where who tell me about this issue, but the real issue was that the database wasn't returning any data, so, the cursor, rightly, was returning early. I tried it this morning on my own computer and the code works well. I have to try things out before posting here and not trust blindly in my coworkers.
Best regards and long live Mormot...
Hello all
I'm having a problem when trying to Step from a cursor. Here's some code
function TMyClass.GetCursor: ISQLDBRows;
var
query: RawUTF8;
stmt : ISQLDBStatement;
cursor: ISQLDBRows;
begin
query := 'BEGIN SOME_PACKAGE.SOME_METHOD(P_cursor => ?); END;';
stmt := Props.NewThreadSafeStatementPrepared(query, false);
stmt.BindCursor(1);
stmt.ExecutePrepared;
cursor := stmt.BoundCursor(1);
Result := cursor;
end;
// here's the calling code
cursor := TMyClass.GetCursor;
while cursor.step do begin
writeln(cursor['column1'], ' ', cursor['column2']);
end;
The problem here is that, even when cursor.FetchAllAsJSON(true); returns ok with all the data on the database, when I try to step a cursor with code it returns fast, meaning that it doesn't write any line on the console. The problem seems to be on unit SynDBOracle on function TSQLDBOracleStatement.Step(SeekFirst: boolean): boolean;
function TSQLDBOracleStatement.Step(SeekFirst: boolean): boolean;
var sav, status: integer;
begin
if not Assigned(fStatement) then
raise ESQLDBOracle.CreateFmt('%s.Execute should be called before Step',[fStatementClassName]);
result := false;
if (fCurrentRow<0) or (fRowCount=0) then // <<---- here's the problem
exit; // no data available at all
sav := fCurrentRow;
fCurrentRow := -1;
if fColumnCount=0 then
exit; // no row available at all (e.g. for SQL UPDATE) -> return false
when I call Step on a cursor, fCurrentRow is always -1 so the function exists, but I know that the cursor is not empty 'cause when I call cursor.fetchAllAsJson it returns all the data.
Any help??
Best Regards!
It works great... Thks for the quick response.
One question thought. Do you think that records are better/faster than TCollectionItem/TInterfacedCollection for deserializing JSON objects?
Best regards...
It seems like a good addition, but I don't really want to pass my filter params on the url. We're writing big services here and the url scheme will allways have the limitation of the number of characters you can send. This is, after all, an internal api so we can get away with passing the parameters with POST. But, how do I deserialize that JSON (passed as a POST parameter) on the server side?
JSONDecode, JSONToObject, any other way?
I think that something like this would be very much appreciated:
https://github.com/thomaserlang/delphi-json
best regards and thks...
Hello all
Working with mormot i want to filter my data on the server, I'm not using the ORM or anything however, the sql stuff it's not the problem, the problem is that i don't find an easy way to pass to an Interface service this data structure por sorting, filtering and pagination:
Here's the JSONi want to pass to mormot from the client
{
// this is for pagination
"startrow": 0,
"endrow": 10,
// this is for sorting
"sortcolumn": "somecolumn",
"sortorder": "asc",
// this is for filtering (it is an array of objects)
[
{"property": "someProperty", "operator": "LIKE", "value": "someValue"},
{"property": "someProperty", "operator": "<=", "value": "someValue"}
]
}
I want to pass this json to an interface service and have the json decoded on the server. Does anyone knows how to do this?
It could be a better JSON like this, but I suppose that mormot would have a hard time parsing it:
{
"pagination": {
"startrow": 0,
"endrow": 10,
},
"sorting": {
"sortcolumn": "somecolumn",
"sortorder": "asc",
},
"filtering": [
{"property": "someProperty", "operator": "LIKE", "value": "someValue"},
{"property": "someProperty", "operator": "<=", "value": "someValue"}
]
}
And then I want an easy way to parse that and get access to the parsed JSON from delphi
Regards...
Feliz Navidad a todos y próspero año nuevo...
How would I set a custom status code on mormot?
I want, inside my services, to be able to set a custom http status code (say 500)
If possible, it would be nice to add a property to TServiceCustomAnswer for setting the status code, now you can set headers and body, but it would be nice to set the status code as well.
Regards...
I think the timeout stuff is a little ugly.
I do see the case for it, but it would be nice if the server remembers the timed out session and returns some message to the client, like:
403
Your session timed out
Or even another custom error code, because on the client we can capture this and later reconnect.
I think that sicPerSession it's the best fit for me, done some testing and everything is working so far...
best regards...
Here's how I register my services:
aServer.ServiceRegister(TMyService, [TypeInfo(IMyService)], sicPerSession)
.DenyAll.AllowAllByName(['User']);
and the problems that i'm having are the ones stated above. I use a javascript client (written in angularjs) to access my server.
Now, if I change my servers with this:
aServer.ServiceRegister(TMyService, [TypeInfo(IMyService)], sicPerUser)
.DenyAll.AllowAllByName(['User']);
At least the logout works ok, i'll debug the other error soon.
So, it seems that sicPerSession. I'll read the SAD to figure out what "aInstanceCreation" it's better for my services.
Best regards.
On the other hand, when I let my server running and come back after some time elapsed it would give me the same error as above. This time i debugged the process and ended up here:
Line: 33923 of mormot unit
// first release any deprecated instances
if fInstanceTimeout<>0 then
for i := fInstancesCount-1 downto 0 do
with fInstances[i] do
if InstanceID<>0 then
if (Inst.LastAccess<LastAccess) or
(QWord(Inst.LastAccess)>QWord(LastAccess)+QWord(fInstanceTimeout)) then begin
// mark this entry is empty
{$ifdef WITHLOG}
SQLite3Log.Add.Log(sllServiceCall,
'Deleted % instance (id=%) after % ms timeout (max % ms)',
[fInterfaceURI,InstanceID,Inst.LastAccess-LastAccess,fInstanceTimeOut],self);
{$endif}
SafeFreeInstance({$ifndef LVCL}fImplementationFreeInMainThread{$endif}); // <-- this line raises the error
end;
I suspect that it is because the session has expired, but it mus return a 403 response and don't raise exception... or better yet, reconnect when asked.
This problem first raises on line
if not Ctxt.Authenticate then
of procedure TSQLRestServer.URI on mormot unit, so, it is effectively related to authentication.
It is worth noting that, after that, i must restart my server for it to work again, so, this is a serious error killing out my server.
Any help???
best regards
Unfortunately, a get request to this URI:
http://localhost:8080/api/auth?UserName=User&session=13165683&session_signature=00c8e473dd7c1cc0e83631c6
fails with:
{"ErrorCode":500,"ErrorText":"ExceptionEInvalidPointer:Invalidpointeroperation"}
The error raises on Line: 33941 of function TServiceFactoryServer.InternalInstanceRetrieve of MormotUnit
// search the instance corresponding to Inst.InstanceID
for i := 0 to fInstancesCount-1 do
with fInstances[i] do
if InstanceID=Inst.InstanceID then begin
if aMethodIndex<0 then begin
// aMethodIndex=-1 for {"method":"_free_", "params":[], "id":1234}
SafeFreeInstance({$ifndef LVCL}fImplementationFreeInMainThread{$endif}); // <-- this line raises the error
result := true; // successfully released instance
exit;
end;
LastAccess := Inst.LastAccess;
Inst.Instance := Instance;
exit;
end;
That line, going deeper, calls this function (an stdcall) on System unit wich produces the error:
function InterlockedDecrement(var Addend: Integer): Integer; stdcall;
external kernel name 'InterlockedDecrement';
Any help?
Best regards...
Thank you!!
Now remains the issue of how to log out, this url always returns an error: 500 (Internal Server Error)
GET http://localhost:8080/api/auth?UserName=User&session=29886586&session_signature=01c8087ad952578cf587857c
best regards
I've solved the issue. The Par variable content are the parameters. It seems that when you do a POST even when you don't need to pass any parameters you need to pass and empty array like this:
requestCurrentUser: function() {
var url = Settings.mainUrl + '/Users.GetCurrent';
var token = getSessionSignature(url.substr(Settings.serverUrl.length + 1));
$.ajax({
type: "POST",
dataType: "json",
url: url + '?session_signature=' + token,
data: [],
contenttype: 'application/json; charset=utf-8',
timeout:2000,
success: function(response, textStatus, jqXHR) {
return currentUser; // As saved on local storage
},
error: function (jqXHR, textStatus, errorThrown) {
console.log(jqXHR);
}
});
so data: [] would solve the issue right away... Thks for the responses...
My TResponse type is simply a record that i return to the client:
TResponse = record
count: integer;
data: RawUTF8;
end;
with that record i use this global function to return the response to the client:
function TBase.WrapResponseSuccess(response: TResponse; code, status: string): RawUTF8;
begin
Result := JSONEncode([
'code', code,
'status', status,
'count', response.count,
'data', response.data
]);
end;
in my services I always return a TServiceCustom Answer like this:
function TService.MyService(someparam: RawUTF8): TServiceCustomAnswer;
var
response: TResponse;
begin
try
try
response := Self.controller.SomeFunction(someparam);
Result.Header := JSON_CONTENT_TYPE_HEADER;
Result.Content := WrapResponseSuccess(response);
finally
// ...
end;
except
On E: Exception do begin
Result.Header := JSON_CONTENT_TYPE_HEADER;
Result.Content := WrapResponseError(E.Message, E.ClassName);
end;
end;
end;
I understand that.
For now, here's the line that raises the exception on the mORmot unit: Ctxt.ExecuteSOAByInterface
procedure TSQLRestServer.URI(var Call: TSQLRestURIParams);
var Ctxt: TSQLRestServerURIContext;
{$ifdef WITHSTATPROCESS}
timeStart,timeEnd: Int64;
begin
QueryPerformanceCounter(timeStart);
{$else}
begin
{$endif}
inc(fStats.fIncomingBytes,length(Call.url)+length(call.method)+length(call.InHead)+length(call.InBody)+12);
Call.OutInternalState := InternalState; // other threads may change it
Call.OutStatus := HTML_BADREQUEST; // default error code is 400 BAD REQUEST
Ctxt := ServicesRouting.Create(self,Call);
try
{$ifdef WITHLOG}
Ctxt.Log := SQLite3Log.Add;
Ctxt.Log.Enter(Self,pointer(Ctxt.URIWithoutSignature),true);
{$endif}
if Ctxt.Method=mNone then
Ctxt.Error('Unknown VERB') else
// 1. decode URI
if not Ctxt.URIDecodeREST then
Ctxt.Error('Invalid Root',HTML_NOTFOUND) else begin
Ctxt.URIDecodeSOAByMethod;
if (Ctxt.MethodIndex<0) and (Ctxt.URI<>'') and
(reService in Call.RestAccessRights^.AllowRemoteExecute) then
Ctxt.URIDecodeSOAByInterface;
// 2. handle security
if not Ctxt.Authenticate then // 403 in case of authentication failure
// 401 Unauthorized response MUST include a WWW-Authenticate header,
// which is not what we used, so we won't send 401 error code but 403
Call.OutStatus := HTML_FORBIDDEN else
// 3. call appropriate ORM / SOA commands
try
if Ctxt.MethodIndex>=0 then
Ctxt.ExecuteSOAByMethod else
if Ctxt.Service<>nil then
Ctxt.ExecuteSOAByInterface else // <<- this line raises 500 (Internal Server Error)
if Ctxt.Method in [mLOCK,mGET,mUNLOCK,mSTATE] then
// handle read methods
Ctxt.ExecuteORMGet else
// write methods (mPOST, mPUT, mDELETE...) are locked
if AcquireWrite(Ctxt) then // make it thread-safe and transaction-safe
try
Ctxt.ExecuteORMWrite;
finally
ReleaseWrite;
end else
// AcquireWrite(SessionID) returned false (e.g. endless transaction)
Call.OutStatus := HTML_TIMEOUT; // 408 Request Time-out
except
on E: Exception do
Ctxt.Error('Exception %: %',
[PShortString(PPointer(PPtrInt(E)^+vmtClassName)^)^,E.Message],
HTML_SERVERERROR); // 500 internal server error
end;
end;
// 4. returns expected result to the client and update Server statistics
if (Call.OutStatus in [HTML_SUCCESS,HTML_CREATED]) or
(Call.OutStatus=HTML_NOTMODIFIED) then begin
inc(fStats.fResponses);
{$ifdef WITHLOG}
Ctxt.Log.Log(sllServer,'% % -> %',[Call.Method,Ctxt.URI,Call.OutStatus],self);
{$endif}
end else begin
inc(fStats.fInvalid);
if Call.OutBody='' then // if no custom error message, compute it now as JSON
Ctxt.Error(Ctxt.CustomErrorMsg,Call.OutStatus);
end;
inc(fStats.fOutcomingBytes,length(Call.OutHead)+length(Call.OutBody)+16);
if (Call.OutBody<>'') and
(length(Call.OutHead)=Length(STATICFILE_CONTENT_TYPE_HEADER)) and
(Call.OutHead[Length(HEADER_CONTENT_TYPE)+1]='!') and
IdemPChar(pointer(Call.OutHead),STATICFILE_CONTENT_TYPE_HEADER) then
inc(fStats.fOutcomingFiles);
if (Ctxt.Static<>nil) and Ctxt.Static.fOutInternalStateForcedRefresh then
// force always refresh for Static table which demands it
Call.OutInternalState := cardinal(-1) else
// database state may have changed above
Call.OutInternalState := InternalState;
{$ifdef WITHSTATPROCESS}
QueryPerformanceCounter(timeEnd);
inc(fStats.ProcessTimeCounter,timeEnd-timeStart);
{$endif}
finally
Ctxt.Free;
end;
end;
It seems that the authentication works, but the problem is while calling my service implementation...
Going deeper, this line is the one that raises the exception on mORmot Unit:
if not fInterface.fMethods[Ctxt.ServiceMethodIndex].InternalExecute(
[PAnsiChar(Inst.Instance)+entry^.IOffset],Ctxt.ServiceParameters,WR,
Ctxt.Call.OutHead,fExecution[Ctxt.ServiceMethodIndex].Options,
Ctxt.ForceServiceResultAsJSONObject)
And here's the exit command that raises the exception on mORmot unit method: TServiceMethod.InternalExecute
if Par^ in [#1..' '] then repeat inc(Par) until not(Par^ in [#1..' ']);
if Par^<>'[' then
Exit;
That conditions and exit are very cryptic to me...
Any help??
The problem is that i don't know how to debbug it.
Mormot is like an onion (or a route middleware for people comming from standard frameworks), it seems that the problem is on the authentication routine so, the request never hits my service implementation. I'm using TSQLRestServerAuthenticationDefault and i don't know where to place the breakpoint to debug this, can you hint me on this?
Also, i don't know how to effectively log out a user and for long running servers it start giving "Invalid pointer operation" exception when I try to access my server. If you hint me about where to start debbugging the authentication stuff i will debug this out and write here.
As a side note, it would be great to support another kind of authentication, HMAC authentication without any server side session (as seen here: http://www.thebuzzmedia.com/designing-a … tication/), this is great for people writting an api, after i debug the problem with TSQLRestServerAuthenticationDefault and solve it I would give a go at writing an hmac authentication for mormot, i've done it dozens of times on php so, it wouldn't be so hard to write on delphi.
Best regards.
When writing a client with javascript I want to get the current logged in user whenever the user reloads the page. This is a SPA but the user can reload the page on the browser and I don't wanna him to authenticate again.
I wrote this Service on mormot:
function TUsers.GetCurrent: TResponse;
var
context: TServiceRunningContext;
begin
context := CurrentServiceContext;
Result.data := IntToStr(context.Request.Session);
Result.count := 1;
end;
That work well after authentication, but when the user reloads the page, after authentication, it returns a 500 (Internal Server Error).
Here's what i want to do in javascript:
requestCurrentUser: function() {
var url = Settings.mainUrl + '/Users.GetCurrent';
var token = getSessionSignature(url.substr(Settings.serverUrl.length + 1));
$.ajax({
type: "POST",
dataType: "json",
url: url + '?session_signature=' + token,
contenttype: 'application/json; charset=utf-8',
timeout:2000,
success: function(response, textStatus, jqXHR) {
return currentUser; // As saved on local storage
},
error: function (jqXHR, textStatus, errorThrown) {
console.log(jqXHR);
}
});
Here's the use case. A users effectively logs in on our application, mormot accepts his credentials and start a session for him, this session lasts about 60minutes.
The user closes his browers and reopen it, he must be authenticated by default because on the server the session is still valid but when I call the functions above it returns 500 (Internal Server Error)
Best regards
Also, how to effectively do a logout from javascript... this url always fails:
var url = Settings.mainUrl + '/auth' + '?session=' + sessionId + '&UserName=' + userName;
It happens that sometimes you want to return null from a select, this we do it to later have a unified interface on the client, so, you could easily do something like this:
SELECT one, two, NULL AS three FROM sometable;
But the problem here is that mORmot raises an exception (understandable) when you do stuff like that. So, i changed the following lines to let mormot return some null values from a db query:
Line: 2686
(original) => if ColumnValueDBSize=0 then
(change) => // if ColumnValueDBSize=0 then
Line: 2687
(original) => raise ESQLDBOracle.CreateFmt('Column "%s": size=0',[ColumnName]);
(change) => // raise ESQLDBOracle.CreateFmt('Column "%s": size=0',[ColumnName]);
only commented out those (2686 and 2687 on latest commit) two lines
best regards...
IT IS WORKIN GREAT NOW!!
Thanks for your commit, I knew that you would do a much better job than me.
//==================================
A question: Any reason why I'm receiving this error when i try to post a new thread?
An error was encountered
Error: Could not connect to smtp host "mail.gandi.net" (110) (Connection timed out).
I found another thing about oracle (it will drive me mad), made some changes and it is working now, i wanted to post the solution (hack until you review it) for future users of the framework.
best regards...
Ok, this is a hack.
I changed line 2792 of SynDBOracle from:
fRowCount := (fInternalBufferSize - ColCount shl 4) div RowSize;
to
fRowCount := 0;
Now cursors are returning (number of actual records in DB) + 1 and that one empty record i slice it on the client end.
This is a super hack until anyone finds a better solution.
The problem seems to be with the fRowCount variable, since we're returning cursors, but i'm not really sure.
I'm really trying to push mormot forward at work, they won't buy into it (yet) but today i found a time to debug this a little, if anyone finds a better solution please write here...
I'll keep on this
until then... best regards.
I really want to help here and i been debugging this when my time permits it, but as i said before, i'm not very good at delphi, just trying to publish some rest services on delphi to work on a javascript client at work.
Anyways, i'll keep trying debuggin it to see if i can find the culprit. In the meantime, if i use ZEOS do you think that it would handle cursors?
For what is worth i'm not blaming anyone here, i just think that if you don't support oracle cursors you just can't call your framework as working with oracle, because it crearly doesn't. The SAD clearle said that oracle is a supported database, but as you can see mormot doesn't support oracle well, cursors, functions, are standard oracle stuff that you simply don't support.
On the other hand, i think that it is very important to know, from a user standpoint, if any lib/framework really works on production... academic discussions are great, but some people just need to get shit done fast. Any framework out there worth would have a sesion on it's documentation pointing out production uses, you can find things like http://builtwithlaravel.com/, http://builtwithbootstrap.com/, http://backbonejs.org/#examples, builtwith.angularjs.org/... etc... This is an important point for deciding if using the framework is worth the time invested learning it, as important for working people as good documentation and good community support.
Being from Cuba i don't really have a chance to work on the open source community, but as you said, it is better to contribute than to blame... I'm really doing my best, i'm posting what would be my own lack of knowledge with the hope that others could help too.
Anyways, great framework...
Is anyone really using mORmot on a production environment??
I (sadly) find it so buggy that i don't think anyone is using it for a real production server with a real database, but i could (hope) be wrong.
I'm posting some code to get some context;
Here's a simple table with only one record:
CREATE TABLE MyTable
(
TEXTO VARCHAR2(4000 BYTE)
)
Here's the oracle procedure
PROCEDURE DUMMY(
P_cursor OUT SYS_REFCURSOR
) IS
BEGIN
OPEN P_cursor FOR
SELECT TEXTO FROM MyTable;
END dummy;
And here's the delphi calling code
var
query: RawUTF8;
stmt: ISQLDBStatement;
cursor: ISQLDBRows;
begin
query := 'BEGIN SOME_PKG.DUMMY(P_cursor => ?); END;';
stmt := Props.NewThreadSafeStatementPrepared(query, false);
stmt.BindCursor(1);
stmt.ExecutePrepared;
cursor := stmt.BoundCursor(1);
Result := cursor.FetchAllAsJSON(true);
end;
Even with only one record on the table called MyTable, this is what cursor.FetchAllAsJSON(true) is returning:
[
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "" },
{ "TEXTO": "Some texto" }
]
As can be seen, the only record that should be returned was the last one ({"TEXTO": "Some texto"} as it is the only record on the MyTable table, but somehow mormot is returning lots of empty records.
Any help would be much appreciated!!!
Sorry
After some tests i found out that cursors are returning some empty records that don't belong to my dataset as stored in the db.
Whe you do Result := cursor.FetchAllAsJSON(true); it donsn't return my db records as expected, but it returns them mixin in some empty records.
best regards...
Hi
The implementation of oracle cursors seems to be broken.
I don't know the cause, but when you do Result := cursor.FetchAllAsJSON(true); it returns empty records that are not on my dataset.
I I do a select only, it works great, but with cursors it returns a json with some records empty.
Any help??
Yeah, you are right, buth the question here is where/when to call TSQLDBOracleConnectionProperties.CreateWithCodePage(...). After that I always use TSQLDBConnectionProperties.NewThreadSafeStatementPrepared() all around my code.
It seems that you can go away with calling TSQLDBOracleConnectionProperties.CreateWithCodePage(...) once on server startup, but i'm not really sure. I come from a dynamic language background (php, node.js, ruby) and don't know how to really do things with delphi lasting running proccesses.
So, here's the question again.
If you just need to call TSQLDBOracleConnectionProperties.CreateWithCodePage(...) once on server startup, then a singleton does a good job, but, is that a persistent connection against oracle database?.
But if that is not a good solution then how to do it right?
Best regards...
I'm posting the code for my DBSingleton class, I'm new to delphi and don't really know if this is the right way to code a singleton, but forget about implementation, the important thing is that this class creates a TSQLDBOracleConnectionProperties once, then, whenever I need a connection i just reference that one connection like: Global_DBSingleton.GetConnection;
feedback is welcome!
unit uDBSingleton;
interface
uses
SysUtils, SynDbOracle;
type
TDBSingleton = class(TObject)
public
constructor Create;
destructor Destroy; override;
function GetConnection: TSQLDBOracleConnectionProperties;
protected
fProps: TSQLDBOracleConnectionProperties;
end;
var
Global_DBSingleton: TDBSingleton;
implementation
uses
uCommons;
constructor TDBSingleton.Create;
begin
if Global_DBSingleton <> nil then
raise Exception.Create('Global DB Singleton already created!')
else
Global_DBSingleton := Self;
end;
destructor TDBSingleton.Destroy;
begin
if Assigned(fProps) then
fProps.Free;
Global_DBSingleton := nil;
inherited Destroy;
end;
function TDBSingleton.GetConnection: TSQLDBOracleConnectionProperties;
begin
if not Assigned(Global_DBSingleton) then
raise Exception.Create('Global DB Singleton not created!. You must call Create once.');
if not Assigned(fProps) then
fProps := TSQLDBOracleConnectionProperties.CreateWithCodePage(SERVER_NAME, USER_ID, USER_PASSWORD, 0);
Result := fProps;
end;
end.
Well, I ended up creating a singleton db class. So whenever I need a TSQLDBOracleConnectionProperties i just call DB.getConnection() to get it. The connection is initialized just once i i let TSQLDBConnectionProperties.NewThreadSafeStatementPrepared() to handle threads issues.
Another approach is to create the same singleton, and on every service request do a DB.getConnection.NewConnection but it could be an overhead.
What do you think about it?
Hello
It is time to start writing and discussing about good patterns in mORmot. Here's one of them for database connections, all samples are with Oracle but I think that the pattern will stand any database server.
Sqlite is a very neat piece of software, but the reality is that we always use something else as our backend RDBMS, so, at some point the question comes: I'm having too many connections against my database?
Here's how i'm approaching the subject, note that i'm using mORmot authentication and I want it to be on oracle.
When I create my server I have this lines:
aProps := TSQLDBOracleConnectionProperties.Create(SERVER_NAME, DATABASE_NAME, USER_ID, USER_PASSWORD);
aModel := TSQLModel.Create([TSQLAuthGroup, TSQLAuthUser], WA_ROOT_NAME);
VirtualTableExternalRegisterAll(aModel, aProps, false);
aServer := TSQLRestServerDB.Create(aModel, GetRealPath(WA_DB_NAME), true);
aServer.AuthenticationRegister(TSQLRestServerAuthenticationDefault);
aServer.CreateMissingTables(0);
aServer.NoAJAXJSON := False;
aServer.ServiceRegister(TService, [TypeInfo(IService)], sicPerSession).DenyAll.AllowAllByName(['User']);
aHTTPServer := TSQLHttpServer.Create(PORT_NAME, [aServer], '+', useHttpApiRegisteringURI);
aHTTPServer.OnlyJSONRequests := False;
aHTTPServer.AccessControlAllowOrigin := '*'; // for AJAX requests to work
With that I ensure that both AuthGroup and AuthUser tables will be created on Oracle and that the authentication will check that tables, meaning that the ORM will use Oracle from this point on. I don't really use the ORM in my code (i like SQL very much) but the authentication mecanism seems to use it. All this goes on my server creation code (think the main method that starts everything).
After that i organize the rest of my code (interface base service) with three discrete pieces. I have services (of course), controllers and repositories. Of them, the repositories will need an oracle connection (TSQLDBOracleConnectionProperties), and here comes the first question. Do i can reuse the connection created on my server initialization code or do i open new connections everytime.
What i ended up doing was to to create a new connection for every interface i have, so, every interface base service will have it's own
fProps := TSQLDBOracleConnectionProperties.Create(SERVER_NAME,DATABASE_NAME,USER_ID, USER_PASSWORD);, and since i defined my services as sicPerSession i assure that every user will have it's own connection to oracle, but this will be for every service i define, and I mean to define a lot of them.
That approach works great, until i wanted to test my server endpoint with apache jmeter, oracle could'nt get the charge... effectively "too many connections"
So, how is everyone here approaching this pattern?. I think that i could create a singleton (DbConnection) open the connection once and share it for everyone that needs it, but it would be better to create a pool of connections. The problem here is that Delphi runs as a lasting process, I think that one connection per request would be a nice compromise, but i'm not sure how to do it.
Comments on this thread will be much appreciated. I will come up with more patterns of this great framework over time.
Best regards...
It now works like a charm.
Now ISQLDBStatement has the ParamToVariant() method and you can bind to oracle cursor.
For the sake of completness it would be great if one could call oracle functions, but I can live without it.
Thks for your quick commits to the great mORmot framework.
Hello
First of all, thaks very much for your contributions to the community. You are doing a great work with mORmot.
Now, i'm trying the cursor functionality but it doesn't work so far.
here's a simple procedure on oracle:
PROCEDURE dummy (
P_cursor OUT Globales.T_c_weak_cursor,
P_id IN Number
) IS
BEGIN
OPEN P_cursor FOR
SELECT * FROM some_table T WHERE T.Id = P_id
END dummy;
And here's the calling delphi code:
var
id: RawUTF8;
query: RawUTF8;
stmt: ISQLDBStatement;
cursor: ISQLDBRows;
begin
id := '2000';
query := 'BEGIN SOME_PKG.DUMMY(P_cursor => ?, P_id => ?); END;';
stmt := Props.NewThreadSafeStatementPrepared(query, false);
stmt.BindCursor(1);
stmt.BindTextU(2, id, paramIn);
stmt.ExecutePrepared;
cursor := stmt.BoundCursor(1);
Result := cursor.FetchAllAsJSON(true);
end;
If Your execute that, the line cursor := stmt.BoundCursor(1); will throw this error: 'No BindCursor() on Param #2'.
Now, if you change the line cursor := stmt.BoundCursor(1); to cursor := stmt.BoundCursor(0); it will show this error: 'Prepare called with wrong ExpectResults', wich is odd because as the cursor is an out parameter is ok to call prepare with false.
Now, if you then change the line stmt := Props.NewThreadSafeStatementPrepared(query, false); to stmt := Props.NewThreadSafeStatementPrepared(query, true); Now that same line gives the error: Prepare called with wrong ExpectResults', wich is odder.
So, what gives??
The error raises on line 2603 on unit SynDBOracle because this line is true: if fExpectResults<>(StatementType=OCI_STMT_SELECT);
I haven't worked with OCI before, but it seems that is waiting for a select when in reality i'm just returning a cursor as an OUT parameter, so fExpectResults is false and different from OCI_STMT_SELECT.
Best regards...
I have a lot of work to do, but as soon as i get a chance i will try the cursor functionality. Maybe tomorrow or the day after.
Thaks for contributing that much to the community.
Never Mind
Here's how to doit:
stmt := Self.Props.NewThreadSafeStatement;
stmt.Prepare(query, false);
stmt.Bind(1, ..., paramIn);
stmt.BindTextU(2, ..., paramIn);
stmt.BindTextU(3, ..., paramIn);
stmt.BindTextS(4, '', paramOut);
stmt.Bind(5, 0, paramOut);
stmt.ExecutePrepared;
stmt.ParamToVariant(4, out1, true);
stmt.ParamToVariant(5, out2, true);
You can't use Props.NewThreadSafeStatementPrepared(query, false); because that doesn't returns a TSQLDBStatement.
Thks for your response.
Hello
The problem is that the statement Self.Props.NewThreadSafeStatementPrepared returns a ISQLDBStatement that doesn't have any method called ParamToVariant, so, do i have to cast or something??
Regards...
Hello
Can anyone post some code or example of calling a stored procedure and/or function from oracle pl/sql.
I can't make it work and this is basic stuff...
I want to get values by out parameters or by return function of oracle.
Any Help would be much appreciated!!!
Here's some code that sadly doesn't work AT ALL!!!
query := 'BEGIN TEST_PKG.DUMMY(?, ?, ?, ?, ?); END;';
stmt := Props.NewThreadSafeStatementPrepared(query, false);
stmt.Bind(1, param1, paramIn);
stmt.BindTextU(2, param2, paramIn);
stmt.BindTextU(3, param3, paramIn);
stmt.BindTextS(4, param4, paramOut);
stmt.Bind(5, param5, paramOut);
stmt.ExecutePrepared;
In here param1, param2 and param3 are parameters that go to the procedure for it to work. This are effectively IN bound parameters.
param4 and param5 are the response for the procedure call. They are effectively OUT bound parameters.
Inside the procedure the last 2 params (param4 and param5) are filled up with some data, but back in Delphi i doesn't work at all, I always get back a wrong answer in param4 and param5.
The blog is full of benchmarks but no so many code samples....
PLEASE HELP!!!
How would i use this new functionality??
WoW
I will do that, wrap all the functions into procedures with out parameters, but again, it is a shame that SynDBOracle doesn't supports calling functions from oracle.
Thks for your response.
I've addedd a feature request for cursor support, it seems like a very good addition to the mormot framework.
Now, I'm still having problems trying to call a funtion from oracle... How do you bind the return value of the function?
Here's what i've got so far:
FUNCTION dummy (
param1 IN integer,
param2 IN integer,
param3 IN string,
param4 OUT string
) RETURN INTEGER;
query := 'BEGIN ? := dummy(?, ?, ?, ?); END;';
stmt := Props.NewThreadSafeStatementPrepared(query, false);
stmt.Bind(1, value, paramOut);
stmt.Bind(2, param1, paramIn);
stmt.BindTextU(3, param2, paramIn);
stmt.BindTextU(4, param3, paramIn);
stmt.BindTextS(5, param4, paramOut);
stmt.ExecutePrepared;
In here, param4 is a variable declared as string and will be filled by the function call.
value is declared as an integer and will be the return value of the function itself.
after the call, i want this two variables to be filled by the stored proc call
value equals the return value of the funcion
param4 equals the OUT param of the function wich is filled inside the function.
Saddly i don't get it to work yet.
My problem with oracle and all this is that i'm evolving a big codebase and I want to reuse the functions/procedures written in oracle, but so far i'm having trouble with it. Can anyone please help??
Thks for your reply
I will end up using
/// encode the supplied data as an UTF-8 valid JSON object content
// - data must be supplied two by two, as Name,Value pairs, e.g.
// ! JSONEncode(['name','John','year',1972]) = '{"name":"John","year":1972}'
// - note that cardinal values should be type-casted to Int64() (otherwise
// the integer mapped value will be transmitted, therefore wrongly)
function JSONEncode(const NameValuePairs: array of const): RawUTF8;
It seems to be very fast and good enough for what i'm trying to do.
Well, I've found a solution to this problem
It seems that the custom object must be declared between {$M+} ... {$M-} to have RTTI information at serialization time. Here's the declaration of the TResponse custom object for it to work:
{$M+}
TResponse = class
private
fData, fMensaje: RawUTF8;
published
property Data: RawUTF8 read fData write fData;
property Mensaje: RawUTF8 read fMensaje write fMensaje;
end;
{$M-}
With that i can get the json with javascript, but i have to do a JSON.parse(...) for it to work, with the default it worked out of the box. Anyways, in the jquery success callback you can easily do this:
success: function(data, textStatus, jqXHR) {
var m = JSON.parse(data.Mensaje);
var d = JSON.parse(data.Data);
// here m equals 'some message' and d equals the result of the db query...
}
I have lost the id passed to the client, but since i'm using a browser (javascript) as the client i guess it is not a big deal, as far as i know sicClientDriven is just used when you have a Delphi client for your mormot server.
Now I'm wondering about the performance hit of this kind of response, I'm thinking about make this kind of wrapped response the default for all my services, does it has an impact on performance this kind of Object to JSON serialization for every request??
Now, If I add to my services this line:
Result.Header := JSON_CONTENT_TYPE_HEADER;
then here's the result i'm getting:
{}
Yep, an empty json... what's going on here??
It dosn't work
I defined this class:
TResponse = class
public
fData, fMensaje: RawUTF8;
published
property Data: RawUTF8 read fData write fData;
property Mensaje: RawUTF8 read fMensaje write fMensaje;
end;
Then in my service i did this:
begin
repository := TRepository.Create(fProps);
data := repository.GetById(id); // This returns data according to Result := stmt.FetchAllAsJSON(true);
response := TResponse.Create;
response.Data := data;
response.Mensaje := 'some message';
Result.Content := ObjectToJson(response);
response.Free; // do not forget to release memory!
end;
After that here's the json i get back:
{"result":["AAJ7fQ=="],"id":25443627}
Some unreadeable json when i'm expecting some result set from the database call in response.Data and the message i set in response.Mensaje.
Of course, I defined the service with a return type of TServiceCustomAnswer.
You're right, Oracle dosn't use standard stuff, it is the Microsoft of the database universe....
Now, Another related question...
Is it posible to call an oracle stored proc with 3 params in and one param out but also returning something... here's an example function spec on oracle:
FUNCTION dummy (
param1 IN integer,
param2 IN integer,
param3 IN string,
param4 OUT string
) RETURN INTEGER;
In here i want to call this procedure and get back an integer result, also get back some string on param4 wich is an out param
Is that posible on mormot?? and if so who would i do it??
Here's what i got so far but it doesn't work
query := 'call dummy (?, ?, ?, ?)';
stmt := Props.NewThreadSafeStatementPrepared(query, false); // it is false here??, remember, it is a function with returns some integer
stmt.Bind(1, param1, paramIn);
stmt.BindTextU(2, param2, paramIn);
stmt.BindTextU(3, param3, paramIn);
stmt.BindTextS(4, param4, paramOut);
stmt.ExecutePrepared; // How do i get the returned integer from the function??
In here, param4 is a variable declared as string and will be filled by the function call, but i also want to get a handle to the function return variable.
Is this possible at all??
This is really important for me, it is ok if i can't call a procedure or function returning a cursor, after all it is not standard sql, but i must be able to call other kinds of procedures and/or functions from oracle. Please, any help would be much appreciated.
The use of stored procs is not just about to speed up the inserts, i'm using them to get data back from oracle with pretty convulated queries that would be a headache to write in a delphi string and because i'm working with an existing codebase that has a lot of the domain logic on oracle pl/sql. All the data retrieve procedures return a cursor and is a shame that i can't use that with mormot.
There seems not to be a solution for this wich, as i said, is a shame because every oracle developer out there knows that they has to return a cursor for any client (java, .NET, php) to work with the result set.
Anyways, i'm investigating converting the cursor to JSON on oracle and returning the json string to the caller. If i can get it working i'll post my solution here.
Mormot is indeed a great framework but it has its shortcomings.
Here is what I thinking:
TRespond = class
private
fData: RawUTF8;
fMessage: RawUTF8;
published
property Data: RawUTF8 read fData write fData;
property Message: RawUTF8 read fMessagewrite fMessage;
end;
Then on any of my services:
function TService.GetById(id: RawUTF8): RawJSON;
var
repository: TRepository;
data: RawUTF8;
respond: TRespond;
begin
repository := TRepository.Create(fProps);
data := repository.GetById(id);
respond := TRespond.Create;
respond.fData := data;
respond.fMessage := 'some message';
Result := ObjectToJson(respond);
end;
But sadly it doesn't work. Also, how could i get a handle to any errors for a custom json error response. I think it would be a good idea to describe on the docs the request/response cycle of an interface based service in mormot.
any suggestions???
Ok
Mormot seems like a nice solution at first when you have a large codebase in delphi for evolving that codebase to a restfull representation, but sometimes i can't even find an easy way of doing things.
Here is my question: How can i create a custom json wrapper and return that from an interface based service??
For example, i want that all my methods in an interface based service to return stuff like this
A successful response:
{
"code":200,
"status":"success",
"data": "some data from the service"
}
An error response:
{
"code":500,
"status":"error",
"developerMessage":"description of the problem for the app developer",
"userMessage": "Pass this message to the app user",
"morInfo": "some url to get more info"
}
This is pretty normal when using php or any other server side technology, but, how can i make this kind of wrapped response to work with mormot?
It is sad indeed.
Oracles diverges in this respect to any other major database systems, so, by now we can't just use a cursor nor in an out parameter nor in a return value from a function. It is sad indeed, i will end up sending the sql string from delphi to oracle.
Thaks for your response.
Hello
It seems that you can't have an oracle stored proc that just returns a select. You must always return something, by a cursor out variable or doing a select ... into. Here's some code:
procedure dummy(p_id integer in) is
begin
Select * from whatever;
end;
You can't make this to work on oracle, you must return a cursor for it to work.
My question then is, if I have to declare this types of procedures like this:
procedure dummy(P_cursor IN OUT Globales.T_c_weak_cursor, p_id IN integer) IS
BEGIN
OPEN P_cursor FOR
Select * from whatever;
END
How can i use that with mormot, meaning, how can I call that last stored proc inside mormot?
Any help would be much appreciated.