You are not logged in.
Pages: 1
I am using 'Delphi 10.1 Berlin' and the latest version of 'mORMot' to access a 'mySQL' database via a 'TSQLRestClientHTTP' component.
Suddenly all my calls to 'CreateAndFillPrepare' are very slow. They used to be fairly fast. Now they take more than 15 seconds to execute.
For example, the call below takes 21 seconds.
iMyProductRecord := TMyProducts.CreateAndFillPrepare(iClient, 'ExpireOn', 'ID = ?', [aProductID]);
I have not changed my code for the past couple of weeks.
Has anybody else experienced this problem and/or knows how to address it?
You are correct ID=6 already exist. However, programmatically, no one outside of the ORM scope has access to the records. Nevertheless, I did find out that a colleague had manually deleted some records. Could that explain the corruption occurring with the ID internal sequence maintained within TSQLRestServer?
Is there a way to re-initialize the ID internal sequence?
Using the latest version of the Synopse mORMot Framework, I have successfully created a Delphi 10.1 Berlin ReST server.
Within my 'Model,' I have the following defined:
TMyCustomerProducts = class(TSQLRecord)
private
fCustID: Integer;
fProductName: RawUTF8;
published
/// ORM will create the table columns
property CustID: Integer read fCustID write fCustID;
property ProductName: RawUTF8 index 12 read fProductName write fProductName stored AS_UNIQUE;
end;
When my 'Client' tries to add a new 'TMyCustomerProducts' record, sometimes the call to the 'Add' function fails:
var
iNewID: Integer;
iClient: TSQLRestClientHTTP;
iMyCustomerProductRecord: TMyCustomerProducts;
begin
...
iNewID := iClient.Add(iDesktopRegistrationRecord, True); // when it fails: iNewID = 0
...
end;
I checked the 'log' file and found the following entries:
20160907 16100446 * SQL TSQLDBZEOSStatement(0091E640) insert into tblMyCustomerProducts (ID,CustID,ProductName) values (6,2181,'Product #1')
20160907 16100446 * EXC EZSQLException ("SQL Error: Duplicate entry '6' for key 'PRIMARY'") at 0061550C stack trace API
Table 'tblMyCustomerProducts' was created by the Server itself. It's "sqlite3" definition is as follow:
CREATE TABLE `tblMyCustomerProducts` (
`ID` BIGINT(20) NOT NULL,
`CustID` INT(11) NOT NULL,
`ProductName` VARCHAR(12) NOT NULL,
PRIMARY KEY (`ID`),
UNIQUE INDEX `ProductName` (`ProductName`)
)
COLLATE='latin1_swedish_ci'
ENGINE=InnoDB
;
How is it possible (what am I doing wrong) that I am getting a duplicate entry for key 'Primary'?
Using Delphi XE8 and the latest nightly build of mORMot, I have created a 'stand-alone' ReST Server (as shown below) that runs on a Windows Server hosted by Amazon.
iConnection := TSQLDBZEOSConnectionProperties.Create(TSQLDBZEOSConnectionProperties.URI(dMySQL, c_mySQLDBServer + ':3306'), c_mySQLDB, c_mySQLDBUsername, c_mySQlDBPassword);
try
iModel := OurDataModel;
VirtualTableExternalRegister(iModel, TCustomersProducts, iConnection, 'TableCustomersProducts');
try
iRestServer := TSQLRestServerDB.Create(iModel, ':memory:', false);
try
iRestServer.CreateMissingTables;
iHttpServer := TSQLHttpServer.Create(SERVER_PORT, [iRestServer], '+', useHttpApiRegisteringURI);
try
writeln('Background server is running.'#10);
write('Press [Enter] to close the server.');
readln;
finally
iHttpServer.Free;
end;
finally
iRestServer.Free;
end;
finally
iModel.Free;
end;
finally
iConnection.Free;
end;
Everything works as expected during the day. Indeed, my server is running in the background and my Client is able to access it without any problems. However, if the server has been 'running' all night long, the next morning, my Client app is no longer able to access it. At that point, I have to stop and re-start the server.
I suppose that, because the server was not called during the night, the connection to the database was lost.
What should I do in order to keep my server running functionally for long period of time? Any suggestions, pointers would be greatly appreciated.
Using Delphi XE8, the latest nightly build of mORMot and trying to follow the 'SUM' example of Chapter 14 of the documentation in addition to accessing a mySQL database, I have created the following ReST Server:
uses
...
ReST_ModelTest in 'ReST_ModelTest.pas';
const
...
type
TMyHttpServer = class(TSQLHttpServer)
published
procedure Sum(Ctxt: TSQLRestServerURIContext);
end;
procedure TMyHttpServer.Sum(Ctxt: TSQLRestServerURIContext);
begin
with Ctxt do
Results([InputDouble['a'] + InputDouble['b']]);
end;
var
iModel: TSQLModel;
iHttpServer: TMyHttpServer;
iRestServer: TSQLRestServerDB;
iConnection: TSQLDBConnectionProperties;
begin
SQLite3Log.Family.Level := LOG_VERBOSE;
SQLite3Log.Family.PerThreadLog := ptIdentifiedInOnFile;
iConnection := TSQLDBZEOSConnectionProperties.Create(TSQLDBZEOSConnectionProperties.URI(dMySQL, c_mySQLDBServer+ ':3306'), c_mySQLDB, c_mySQLDBUsername, c_mySQlDBPassword);
try
iModel := MyDataModel;
VirtualTableExternalRegister(iModel, TMyCustomers, iConnection, 'tblMyCustomers');
try
iRestServer := TSQLRestServerDB.Create(iModel, ':memory:', false); // authentication = false
try
iRestServer.AcquireExecutionMode[execORMGet] := amBackgroundORMSharedThread;
iRestServer.AcquireExecutionMode[execORMWrite] := amBackgroundORMSharedThread;
iRestServer.CreateMissingTables;
iHttpServer := TMyHttpServer.Create(SERVER_PORT, [iRestServer], '+', useHttpApiRegisteringURI);
try
iHttpServer.AccessControlAllowOrigin := '*'; // allow cross-site AJAX queries
writeln('Background server is running.'#10);
write('Press [Enter] to close the server.');
readln;
finally
iHttpServer.Free;
end;
finally
iRestServer.Free;
end;
finally
iModel.Free;
end;
finally
iConnection.Free;
end;
end.
My 'ReST_ModelTest' is defined as follow:
unit ReST_ModelTest;
interface
uses
SynCommons,
mORMot;
type
TMyCustomers = class(TSQLRecord) // TSQLRecord has a built-in ID: integer primary key
private
fCustomerID: RawUTF8;
fCustomerName: RawUTF8;
published
/// ORM will create the table columns
property CustomerID: RawUTF8 index 15 read fCustomerID write fCustomerID;
property CustomerName: RawUTF8 index 16 read fCustomerName write fCustomerName;
end;
function MyDataModel: TSQLModel;
const
SERVER_ROOT = 'root';
SERVER_PORT = '888';
implementation
function MyDataModel: TSQLModel;
begin
Result := TSQLModel.Create([TMyCustomers], SERVER_ROOT);
TMyCustomers.AddFilterOrValidate('CustomerID', TSynValidateText.Create); // ensure field exists
TMyCustomers.AddFilterOrValidate('CustomerName', TSynValidateText.Create); // ensure field exists
end;
end.
After generating my 'mORMotClient' unit, I would like to create a REST client that would be using the both the 'TMyCustomers' type defined above as well as the 'SUM' function. However, I am not sure on how to do that. I have tried the following but it crashes my Client app.
function do_Sum(aVal1: Double; aVal2: Double): Double;
var
iErr: Integer;
iResults: TSQLRecordClass;
iClient: TSQLRestClientHTTP;
begin
iClient := GetClient('localhost');
try
Val(iClient.CallBackGetResult('sum', ['a', aVal1, 'b', aVal2], iResults), Result, iErr);
finally
iClient.Free;
end;
end;
I must be doing something wrong but I cannot figure out what it is. Any pointers to a solution to my problem would be greatly appreciated.
Thanks for your input.
After reading your recommendations, I realized that my original post was not complete and accurate. Indeed, I had forgotten to mention that my database is a 'mysql' one. I understand that the type of database used might not be relevant to my problem but I should not have forgotten to mention it in the first place.
I had also forgotten to mention that I had enabled the logs. However, at this point in time, I have not added a '.map' file to my application as I am still reading the documentation concerning these types of files.
Finally, when dealing with the "not successful" statement, the 'CreateAndFillPrepare' should have read:
iCustomerSQLRecord := TCustomers.CreateAndFillPrepare(iClient, 'ID, CustomerID, CustomerStatus, CustomerName, CustomerDate', 'ID = ?', [aCustomerID]);
It is when I noticed the missing 'CustomerStatus' from my supposedly unsuccessful 'CreateAndFillPrepare' statement that I found a solution to my problem. In my 'mysql' database 'CustomerStatus' was defined as a 'bit' field. As soon as I changed its type to 'tinyint,' everything worked as expected.
Using Delphi XE8 and the latest nightly build of mORMot, I have created a REST server in which I have the following TSQLRecord class defined:
TCustomers = class(TSQLRecord)
protected
fCustomerID: Integer;
fCustomerStatus: Boolean;
fCustomerName: String;
fCustomerDate: TDateTime;
published
property CustomerID: Integer read fCustomerID write fCustomerID;
property CustomerStatus: Boolean read fCustomerStatus write fCustomerStatus;
property CustomerName: String index 255 read fCustomerName write fCustomerName;
property CustomerDate: TDateTime read fCustomerDate write fCustomerDate;
end;
After generating my 'mORMotClient' unit, I created a REST client that will be using the 'TCustomers' type defined above.
If I run the following from within my client, the 'FillOne' is successful:
var
iClient: TSQLRestClientHTTP;
iCustomerSQLRecord: TCustomers;
begin
iClient := GetClient('localhost');
try
// Let's locate the 'customer' record
iCustomerSQLRecord := TCustomers.CreateAndFillPrepare(iClient, 'ID, CustomerID', 'ID = ?', [aCustomerID]);
try
if (iCustomerSQLRecord.FillOne) then
begin
...
end;
finally
iCustomerSQLRecord.Free;
end;
finally
iClient.Free;
end;
But if I run the following from within my client, the 'FillOne' is UNSUCCESSFUL:
var
iClient: TSQLRestClientHTTP;
iCustomerSQLRecord: TCustomers;
begin
iClient := GetClient('localhost');
try
// Let's locate the 'customer' record
iCustomerSQLRecord := TCustomers.CreateAndFillPrepare(iClient, 'ID, CustomerID, CustomerName, CustomerDate', 'ID = ?', [aCustomerID]);
try
if (iCustomerSQLRecord.FillOne) then
begin
...
end;
finally
iCustomerSQLRecord.Free;
end;
finally
iClient.Free;
end;
I must be doing something wrong. But I am unable to figure out what I am doing wrong. Any inputs would be greatly appreciated. Thanks.
Yes, I was/am using the latest source code (1.18.628).
Per your suggestions, I used 'HttpServiceSetup /install' instead of 'HttpService /install' and I was able to successfully install the service.
@ab
I am running my 'client' as a VCL application for testing purposes only (it is faster than deploying to my mobile device each time I made a change). At the end, it will be running on a mobile device. That is why I am developing a cross platform client. For my own curiosity, why is not advised to use a cross platform client within a VCL app?
I am not sure if I understand your response in terms of use the same model as the server.
On my local machine, I have created a ReST Server and Client (code bewlow). When I run the Server locally, I am able via the Client to retrieve a list of products stored in a mySQL database hosted externally. My end goal is to run (at first as a console) my ReSt Server on a Windows instance hosted by Amazon.
I am able to 'install' and run my 'console' server on the Windows instance. However, for the life of me, I am unable to figure out how to modify the client so that it might access my 'mORMot' server. Any pointers and/or advices would be greatly appreciated.
My ReST Server is defined as follow:
uses
...,
OurReST_Model;
var
iModel: TSQLModel;
iHttpServer: TSQLHttpServer;
iRestServer: TSQLRestServerDB;
iConnection: TSQLDBConnectionProperties;
begin
SQLite3Log.Family.Level := LOG_VERBOSE;
SQLite3Log.Family.PerThreadLog := ptIdentifiedInOnFile;
iConnection := TSQLDBZEOSConnectionProperties.Create(TSQLDBZEOSConnectionProperties.URI(dMySQL, c_mySQLDBServer + ':3306'), c_mySQLDB, c_mySQLDBUsername, c_mySQlDBPassword);
try
iModel := OurDataModel;
VirtualTableExternalRegister(iModel, TCustomersProducts, iConnection, 'TableCustomersProducts');
try
iRestServer := TSQLRestServerDB.Create(iModel, ':memory:', false);
try
iRestServer.CreateMissingTables;
iHttpServer := TSQLHttpServer.Create(SERVER_PORT, [iRestServer], '+', useHttpApiRegisteringURI);
try
writeln('Background server is running.'#10);
write('Press [Enter] to close the server.');
readln;
finally
iHttpServer.Free;
end;
finally
iRestServer.Free;
end;
finally
iModel.Free;
end;
finally
iConnection.Free;
end;
end.
'OurReST_Model' is defined as follow:
unit OurReST_Model;
interface
uses
SynCommons,
mORMot;
type
TCustomersProducts = class(TSQLRecord) // TSQLRecord has a built-in ID: integer primary key
private
fProductName: RawUTF8;
fProductAddedOn: TDateTime;
published
/// ORM will create the table columns
property ProductName: RawUTF8 index 12 read fProductName write fProductName;
property ProductAddedOn: TDateTime read fProductAddedOn write fProductAddedOn;
end;
function OurDataModel: TSQLModel;
const
SERVER_ROOT = 'root';
SERVER_PORT = '888';
implementation
function OurDataModel: TSQLModel;
begin
Result := TSQLModel.Create([TCustomersProducts], SERVER_ROOT);
end;
end.
Within my 'client' VCL application, if I want to retrieve a list of products that belongs to a specific customer, I do the following:
function get_UserProducts(aUserID: Integer; var aClientDataSet: TClientDataSet): Boolean;
var
iModel: TSQLModel;
iClient: TSQLRestClientHTTP;
iUserRecord: TCustomersProducts;
begin
iModel := GetModel;
iClient := GetClient('localhost');
try
iUserRecord := TCustomersProducts.CreateAndFillPrepare(iClient, '', 'ID = ?', [aUserID]);
try
aClientDataSet.Close;
aClientDataSet.Open;
while iUserRecord.FillOne do begin
aClientDataSet.Append;
aClientDataSet.FieldByName('ProductName').AsString := iUserRecord.ProductName;
aClientDataSet.FieldByName('ProductAddedOn').AsDateTime := iUserRecord.ProductAddedOn;
aClientDataSet.Post;
end;
Result := True;
finally
iUserRecord.Free;
end;
finally
iClient.Free;
iModel.Free;
end;
end;
@ab
In regards to:
2.8. External database:
- TQuery: 2,003 assertions passed 13.71ms
- SynDBRemote: 30,095 assertions passed 1.34s
- External records: 2 assertions passed 182us
- Auto adapt SQL: 543 assertions passed 54.21ms
- Crypted database: 253,275 assertions passed 254.20ms
! - External via REST: 21,893 / 170,325 FAILED 1.83s
! - External via virtual table: 21,893 / 170,325 FAILED 2.67s
! - External via REST with change tracking: 22,904 / 180,425 FAILED 3.84s
- JET database: 7,007 assertions passed 3.30s
Total failed: 66,690 / 814,000 - External database FAILED 13.31s
using the v. 1.18.610 of the Synopse framework, here is what I found (so far) while trying to debug the problem:
It is as if the value of variable 'Start' in 'procedure TTestExternalDatabase.Test(StaticVirtualTableDirect, TrackChanges: boolean);' (file: 'SynSelfTests.pas') is always less than the values of 'RExt.LastChange' and 'RExt.CreatedAt.'
@ab
A 'pdf' is generated. Is it the right one? I am not certain but I believe that it is. Here is a copy of it if you want to double check.
@ab
In regards to:
1.4. Synopse PDF:
! - TPdfDocument: 1 / 4 FAILED 6.11ms
- TPdfDocumentGDI: 8 assertions passed 365.98ms
Total failed: 1 / 12 - Synopse PDF FAILED 372.98ms
using the v. 1.18.610 of the Synopse framework, here is what I found (so far) while trying to debug the problem:
The problem occurs within 'procedure TTestSynopsePDF._TPdfDocument;' (file: 'SynSelfTests.pas') when executing statement 'Check(Hash32(MS.Memory,MS.Position)=Hash[embed]);' (with 'embed' set to False).
On my Windows 8 (64 bit) Pro N machine, 'Hash32(MS.Memory,MS.Position)' returns '2317878866' whereas 'Hash[embed]' returns '2451964498'.
Using Delphi XE7 Update 1 and the latest 'unstable' version of 'mORMot', I have compiled project 'httpservice.'
When I run the 'httpservice' executable with the '-c' parameter, everything works as expected.
When I try to install the service (via a call, in administrator mode, to the command line 'httpservice /install'), I do not get the usual 'Service installed' confirmation message. In addition, the service does not show up in the list of installed services.
Am I missing something important?
No, it is not Azure. It will be Amazon.
I will keep you pdated on my progress as my project moves along.
At this point in time, I am running the server locally as a console (on my own Windows 8 64-bit Virtual Machine). However, the server is 'connected' to an external mySQL database that is hosted in a different state.
Thank you for your help.
You were right. My program was not pointing to the right folder (in terms of the location of the mustache templates). Once I understood the nature of my problem, the fix was easy.
On a Windows 8 64-bit machine, using Delphi XE7 Update 1, I adapted the '28 - Simple RESTful ORM Server' example, so that the server 'links' to a mySQL database stored in the cloud. Everything works as expected.
My question is 'how to I implement the client side?' As suggested in '16.2. Generating client wrappers,' I tried to generate the source code needed for client access. But, so far, all my attempts have failed. Am I going at it the wrong way?
Thanks to both of you for your clear and consice answer. It is conforting to know that I am following the right path.
Thanks for pointing me in the right direction. I had never used/knew about SVN until now. It took me a little bit of time to grasp the concept. Then once I discovered 'TortoiseSVN,' I was up and running.
The example entiled 'Simple RESTful ORM Server' included with 'mORMot' is a console application.
Would a 'real' ReSTful ORM server be a service instead (on a Windows server)?
If I understand the 'mORMot' documentation properly, using 'TSQLDBZEOSConnectionProperties' in conjunction with 'libmysql.dll' is the prefered method to connect to a mySQL database. I also understand that the recommended version of Zeos to used is: 7.2. I have downloaded Zeos v. 7.2 (dated October 21, 2014).
I now have a stupid question. Do I install Zeos into my Delphi XE7 in order to use it with 'mORmot' or do I simply add the path to the 'src' folder to my application?
After reading 'Leander007' and 'ab' suggestions regarding my URI problem, I compiled and run the 'TestSQL3Register.' This solved my original URI problem. Even though this problem was solved, I now have new ones:
2.8. External database:
- TQuery: 2,003 assertions passed 9.69ms
- SynDBRemote: 20,036 assertions passed 525.28ms
- External records: 2 assertions passed 262us
- Auto adapt SQL: 633 assertions passed 48.43ms
- Crypted database: 253,275 assertions passed 284.52ms
! - External via REST: 21,981 / 170,325 FAILED 2.36s
! - External via virtual table: 21,893 / 170,325 FAILED 2.96s
! - External via REST with change tracking: 22,904 / 180,425 FAILED 4.22s
In regard of the '1.4. Synopse PDF: ! - TPdfDocument: 1 / 4 FAILED 50.88ms' error, I did switch my system code page from 437 to 1252 and rebooted my machine. This did not solve my original problem.
FYI:
----
Synopse framework used: 1.18.533
Running on wEight_64 with code page 1252
SQlite3 engine used: 3.8.7.1
Generated with: Delphi XE7 compiler
By the way, I got the exact same results when compiling and running 'TestSQL3' with Delphi 2010 (on an XP 32-bit machine).
I am using Delphi XE7 Update 1 on a Windows 8 virtual machine. I downloaded and uncompressed the snapshot of the latest source code repository into "C:\Dev\Lib\". Idownloaded and uncompressed the static 32 bit SQLite3 .obj files into "C:\Dev\Lib\SQLite3\". I opened the TestSQL3.dpr program from the SQLite3 sub-folder. I compiled the program. When I run it, I get the following fails:
1.4. Synopse PDF:
! - TPdfDocument: 1 / 4 FAILED 50.88ms
- TPdfDocumentGDI: 8 assertions passed 46.90ms
Total failed: 1 / 12 - Synopse PDF FAILED 98.96ms
2.8. External database:
- TQuery: 2,003 assertions passed 8.43ms
! Exception ESQLDBRemote raised with messsage:
! TSQLDBServerHttpApi.Create: administrator rights needed to register URI root on port 888
What am I doing wrong?
To 'ab:' thanks for your input. I will be following your advices and access/read all the links you mentioned in your kind response.
To ' Leander007:' thanks for your input. I might have been unclear in my description of what we would like to implement. We would like to use a SQLite database (outside of Amazon S3) for user login purposes. When a user logs in (either via the desktop app or the mobile one), the database would be used to verify their eligibility and return (if authorized) the 'path' to their S3 bucket. Amazon S3 would only be used as a data depositary for data transfer between the desktop app and the mobile one.
I am a Delphi programmer that, until this month, has only developed desktop applications. The company I worked for has decided to create a mobile app to be used in conjunction with our desktop app. As a result of this decision, we have upgraded to Delphi XE7 and they have asked me to implement the following features:
- both the desktop and the mobile app should be able to login onto a database in the Cloud. The purpose of the login is to verify the user membership as well as retrieve the path to the user data storage;
- both the desktop and the mobile app should be able to upload and/or download data located under the path retrieved during the login process.
We are thinking of using Amazon S3 for our data storage and a SQLite database. Both the Amazon S3 account and the SQLite database would be accessed via a web server (ReST server) for security purposes.
As a neophyte, I have never dealt with this type of problem before. As a result, I have been spending some time researching the subject online. As I was doing so, I discovered the mORMot Framework. Not really knowing where to start and being rather overwhelmed by the complexity of the task, I would like your expert opinions on the feasibility of using mORMot Framework for our project. If you do not think that we should use mORMot Framework, would you recommend another framework and/or a different approach?
Thank you, in advance, for your inputs.
Pages: 1