mORMot and Open Source friends
Check-in [e1e58505f3]
Not logged in

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
SHA1:e1e58505f35d7dbb4b72399916a5885ad6008d9b
Date: 2014-05-09 15:02:20
User: User
Comment:BREAKING CHANGE: TSQLRestServerStatic* classes are now renamed as TSQLRestStorage* and do not inherit from TSQLRestServer any more (which was a design issue) but plain TSQLRest
  • this (long-time waiting) huge code refactoring results in a much cleaner design
  • documentation has been updated to reflect the changes
  • it won't change anything when using the framework (but the new names): it is an implementation detail
Tags And Properties
Context
2014-05-09
15:08
[977b023d4b] fixed small compiler warning with Unicode Delphi compilers (user: User, tags: trunk)
15:02
[e1e58505f3] BREAKING CHANGE: TSQLRestServerStatic* classes are now renamed as TSQLRestStorage* and do not inherit from TSQLRestServer any more (which was a design issue) but plain TSQLRest
  • this (long-time waiting) huge code refactoring results in a much cleaner design
  • documentation has been updated to reflect the changes
  • it won't change anything when using the framework (but the new names): it is an implementation detail
(user: User, tags: trunk)
14:48
[a7e77ec859] let sample "23 - JavaScript Tests" be compiled under Win64 (without SM support, by design) (user: User, tags: trunk)
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to SQLite3/Documentation/Synopse SQLite3 Framework.pro.

3012
3013
3014
3015
3016
3017
3018
3019
3020
3021
3022
3023
3024
3025
3026
....
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
....
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
....
3632
3633
3634
3635
3636
3637
3638
3639
3640
3641
3642
3643
3644
3645
3646
....
3768
3769
3770
3771
3772
3773
3774
3775
3776
3777
3778
3779
3780
3781
3782
....
4191
4192
4193
4194
4195
4196
4197
4198
4199
4200
4201
4202
4203
4204
4205
....
4231
4232
4233
4234
4235
4236
4237
4238
4239
4240
4241
4242
4243
4244
4245
....
4340
4341
4342
4343
4344
4345
4346
4347
4348
4349
4350
4351
4352
4353
4354
4355
4356
4357
4358
4359
4360
4361
4362
4363
4364
4365
4366
....
4415
4416
4417
4418
4419
4420
4421
4422
4423
4424
4425
4426
4427
4428
4429
4430
4431
4432
4433
4434
4435
4436
4437
4438
4439
4440
4441
4442
4443
4444
4445
4446
....
5148
5149
5150
5151
5152
5153
5154
5155
5156
5157
5158
5159
5160
5161
5162
....
5163
5164
5165
5166
5167
5168
5169
5170
5171
5172
5173
5174
5175
5176
5177
5178
5179
5180
5181
5182
5183
5184
5185
5186
5187
5188
5189
5190
5191
5192
5193
5194
5195
5196
....
5201
5202
5203
5204
5205
5206
5207
5208
5209
5210
5211
5212
5213
5214
5215
5216
5217
5218
5219
....
5562
5563
5564
5565
5566
5567
5568
5569
5570
5571
5572
5573
5574
5575
5576
....
6299
6300
6301
6302
6303
6304
6305

6306
6307
6308
6309
6310
6311
6312
6313
6314
6315
6316
6317
6318
6319
6320

6321
6322
6323
6324
6325
6326
6327

6328




6329
6330
6331
6332
6333
6334
6335
6336
....
6514
6515
6516
6517
6518
6519
6520
6521
6522
6523
6524
6525
6526
6527
6528
6529
6530
6531
6532
6533
6534
6535
....
6737
6738
6739
6740
6741
6742
6743
6744
6745
6746
6747
6748
6749
6750
6751
6752
6753
6754
6755
6756
6757
6758
6759
6760
6761
6762
6763
6764
6765
6766
6767
6768
6769
6770
6771
6772
6773
6774
6775
....
6778
6779
6780
6781
6782
6783
6784
6785
6786
6787
6788
6789
6790
6791
6792
6793
6794
6795
6796
6797
6798
6799
6800
6801
6802
6803
6804
.....
11715
11716
11717
11718
11719
11720
11721
11722
11723
11724
11725
11726
11727
11728
11729
The database is just one way of your objects persistence:
- Don't think about tables with simple types (text/number...), but objects with high level types;
- Don't think about @*Master/Detail@, but logical units;
- Don't think "@*SQL@", think about classes;
- Don't wonder "How will I store it?", but "Which data do I need?".
For instance, don't be tempted to always create a pivot table (via a {\f1\fs20 @*TSQLRecordMany@} property), but consider using a {\i @*dynamic array@}, {\f1\fs20 @*TPersistent@, @*TStrings@} or {\f1\fs20 @*TCollection@} @*published properties@ instead.
Or consider that you can use a {\f1\fs20 TRecordReference} property pointing to any registered class of the {\f1\fs20 @*TSQLModel@}, instead of creating one {\f1\fs20 @*TSQLRecord@} property per potential table.
The {\i mORMot} framework is even able to persist the object without any SQL database, e.g. via {\f1\fs20 TSQLRestServerStaticInMemory}. In fact, its ORM core is optimized but not tied to SQL.
:  Objects, not tables
With an @*ORM@, you should usually define fewer tables than in a "regular" relational database, because you can use the high-level type of the {\f1\fs20 @*TSQLRecord@} properties to handle some per-row data.
The first point, which may be shocking for a database architect, is that you should better {\ul not} create @*Master/Detail@ tables, but just one "master" object with the details stored within, as @*JSON@, via {\i @*dynamic array@}, {\f1\fs20 @*TPersistent@, @*TStrings@} or {\f1\fs20 @*TCollection@} properties.
Another point is that a table is not to be created for every aspect of your software configuration. Let's confess that some DB architects design one configuration table per module or per data table. In an ORM, you could design a configuration class, then use the unique corresponding table to store all configuration encoded as some JSON data, or some DFM-like data. And do not hesitate to separate the configuration from the data, for all not data-related configuration - see e.g. how the {\f1\fs20 mORMotOptions} unit works. With our framework, you can serialize directly any {\f1\fs20 @*TSQLRecord@} or {\f1\fs20 TPersistent} instance into JSON, without the need of adding this {\f1\fs20 TSQLRecord} to the {\f1\fs20 @*TSQLModel@} list. Since revision 1.13 of the framework, you can even define {\f1\fs20 TPersistent} @published properties@ in your {\f1\fs20 TSQLRecord} class, and it will be automatically serialized as TEXT in the database.
:  Methods, not SQL
At first, you should be tempted to write code as such (this code sample was posted on our forum, and is not bad code, just not using the @*ORM@ orientation of the framework):
!  DrivesModel := CreateDrivesModel();
................................................................................
The {\i mORMot} point of view (which is not the only one), is to let the DB persist the data, as safe and efficient as possible, but rely on higher levels layers to implement the business logic. The framework favors {\i convention over configuration}, which is known to save a lot of time (if you use WCF on a daily basis, as I do, you and your support team know about the {\f1\fs20 .config} syndrome). It will make it pretty database-agnostic (you can even not use a SQL database at all), and will make the framework code easier to debug and maintain, since we don't have to deal with all the DB engine particularities. In short, this is the @*REST@ point of view, and main cause of success: @*CRUD@ is enough in our KISS-friendly design.
:  Persist TSQLRecord, not any class
About the fact that you need to inherit from {\f1\fs20 @*TSQLRecord@}, and can't persist anything, our purpose was in fact very similar to the "@**Layer Supertype@" pattern of @*Domain-Driven@-Design, as explained by Martin Fowler:\line {\i It is not uncommon for all the objects in a layer to have methods you don't want to have duplicated throughout the system. You can move all of this behavior into a common Layer Supertype.}
In fact, for {\f1\fs20 TSQLRecord} / {\f1\fs20 TSQLRest} / @*ORM@ remote access, you have already all @*Client-Server@ @*CRUD@ operations available. Those classes are abstract common Supertypes, ready to be used in your projects. It has been optimized a lot (e.g. with a cache and other nice features), so I do not think reinventing a CRUD / database service is worth the prize. You have secure access to the ORM classes, with user/group attributes. Almost everything is created by code, just from the {\f1\fs20 TSQLRecord} class definition, via @*RTTI@. So it may be faster (and safer) to rely on it, than defining all your class hierarchy by hand.
Having a dedicated class help the layer separation, and therefore a cleaner design. See @68@ for more details about DDD and how {\i mORMot}'s {\f1\fs20 TSQLRecord} can help you reuse existing code, write less and safer.
:  Several ORMs at once
To be clear, {\i mORMot} offers three kind of table definitions:
- Via {\f1\fs20 @*TSQLRecord@} / {\f1\fs20 @*TSQLRecordVirtual@} "native ORM" classes: data storage is using either fast in-memory lists via {\f1\fs20 @*TSQLRestServerStaticInMemory@}, or {\i @*SQLite3@} tables (in memory, on file, or virtual). In this case, we do not use {\f1\fs20 index} for strings (column length is not used by any of those engines).
- Via {\f1\fs20 @*TSQLRecord@} "external ORM-managed" classes: after registration via a call to the {\f1\fs20 @*VirtualTableExternalRegister@()} function, external DB tables are created and managed by the ORM, via SQL - see @27@. These classes will allow creation of tables in any supported database engine - currently {\i SQLite3, @*Oracle@, @*Jet@/MSAccess, @*MS SQL@, @*Firebird@, @*DB2@, @*PostgreSQL@, @*MySQL@} and {\i @*NexusDB@} - via whatever {\i OleDB, ODBC} / ZDBC provider, or any {\f1\fs20 DB.pas} unit). For the "external ORM-managed" {\f1\fs20 TSQLRecord} type definitions, the ORM expects to find an {\f1\fs20 index} attribute for any text column length (i.e. {\f1\fs20 RawUTF8} or {\f1\fs20 string} published properties). This is the only needed parameter to be defined for such a basic implementation, in regard to {\f1\fs20 TSQLRecord} kind of classes.
- Via {\f1\fs20 @*TSQLRecordMappedAutoID@} / {\f1\fs20 @*TSQLRecordMappedForcedID@} "external mapped" classes: DB tables are not created by the ORM, but already existing in the DB, with sometimes a very complex layout. This feature is not yet implemented, but on the road-map. For this kind of classes we won't probably use attributes, nor even external files, but we will rely on definition from code, either with a fluent definition, or with dedicated classes (or interface).
The concern of not being able to persist any class (it needs to inherit from {\f1\fs20 TSQLRecord}) does perfectly make sense.
On the other hand, from the implementation point of view, it is very powerful, since you have a lot of methods included within this class definition. It does also make sense to have a common ancestor able to identify all three kind of {\i mORMot}'s table definitions: the same abstract ancestor is used, and clients won't even need to know that they are implemented in-memory, using a {\i SQLite3} engine, or any external database. Another benefit of using a parent {\f1\fs20 class} is to enforce code safety using Delphi's {\i @*strong type@} abilities: you won't be able to pass a non-persistent type to methods which expect one.
From the {\i @*Domain-Driven@} / @*SOA@ point of view, it is now an established rule to make a distinction between {\i Data Transfer Objects} (@*DTO@) and @*Domain Values@ ({\i Entity objects} or {\i Aggregates}). In most implementations, persistence objects (aka ORM objects) should be either the aggregate roots themselves (you do not store Entity objects and even worse DTOs), or dedicated classes. Do not mix layers, unless you like your software to be a maintenance nightmare!
Some @*Event Sourcing@ architectures even implement {\i several DB back-end at once}:
- It will store the status on one database (e.g. high-performance in-memory) for most common requests to be immediate;
................................................................................
- And even sometimes fill some dedicated consolidation DBs for further analysis.
AFAIK it could be possible to directly access ORM objects remotely (e.g. the consolidation DB), mostly in a read-only way, for dedicated reporting, e.g. from consolidated data - this is one potential @*CQRS@ implementation pattern with {\i mORMot.} Thanks to the framework security, remote access will be safe: your clients won't be able to change the consolidation DB content!
As can be easily guessed, such design models are far away from a basic ORM built only for class persistence.
:  The best ORM is the one you need
Therefore, we may sum up some potential use of ORM, depending of your intent:
- If your understanding of ORM is just to persist some {\i existing} objects with a lot of existing business code, {\i mORMot} won't help you directly, since it expects objects to inherit from {\f1\fs20 TSQLRecord} - but note that most ORMs, even those allowing to persist any {\f1\fs20 class}, would not be so easy to work with;
- If you want a very fast low-level Client-Server layer, {\i mORMot} is a first class candidate: we identified that some users are using the built-in JSON serialization and HTTP server features to create their application;
- If you want to persist some data objects (not tied to complex business logic), the framework's ORM will be a light and fast candidate, targetting {\i SQLite3, @*Oracle@, @*Jet@/MSAccess, @*MS SQL@, @*Firebird@, @*DB2@, @*PostgreSQL@, @*MySQL@, @*NexusDB@} databases, or even with no SQL engine, using {\f1\fs20 @*TSQLRestServerStaticInMemory@} class which is able to persist its content with small files - see @57@;
- If you need (perhaps not now, but probably in the future) to create some kind of scalable {\i @*Domain-Driven@} design architecture, you'll have all needed features at hand with {\i mORMot};
- If your expectation is to map an existing complex DB, {\i mORMot} will handle it soon (it is planned and prepared within the framework architecture).
Therefore, {\i mORMot} is not just an ORM, nor just a "classic" ORM.
About how to use {\i mORMot} with existing code, see @66@.
:MVC pattern
%cartoon04.png
: Creating a Model
According to the {\i @*Model@-View-Controller} (@*MVC@) pattern - see @10@ - the database schema should be handled separately from the User Interface.
The {\f1\fs20 @**TSQLModel@} class centralizes all {\f1\fs20 @*TSQLRecord@} inherited classes used by an application, both database-related and business-logic related.
In order to follow the @*MVC@ pattern, the {\f1\fs20 TSQLModel} instance is to be used when you have to deal at table level. For instance, do not try to use low-level {\f1\fs20 TSQLDataBase.GetTableNames} or {\f1\fs20 TSQLDataBase.GetFieldNames} methods in your code. In fact, the tables declared in the Model may not be available in the {\i @*SQLite3@} database schema, but may have been defined as {\f1\fs20 @*TSQLRestServerStaticInMemory@} instance via the {\f1\fs20 TSQLRestServer.StaticDataCreate} method, or being external tables - see @27@. You could even have a {\f1\fs20 mORMot} server running without any {\i @*SQLite3@} engine at all, but pure in-memory tables!
Each {\f1\fs20 TSQLModel} instance is in fact associated with a {\f1\fs20 TSQLRest} instance. An {\f1\fs20 Owner} property gives access to the current running client or server {\f1\fs20 @*TSQLRest@} instance associated with this model.
By design, models are used on both Client and Server sides. It is therefore a good practice to use a common unit to define all {\f1\fs20 TSQLRecord} types, and have a common function to create the related {\f1\fs20 TSQLModel} class.
For instance, here is the corresponding function as defined in the first samples available in the source code repository (unit {\f1\fs20 SampleData.pas}):
!function CreateSampleModel: TSQLModel;
!begin
!  result := TSQLModel.Create([TSQLSampleRecord]);
!end;
................................................................................
Difficult to find a faster ORM, I suspect.
:   Software and hardware configuration
The following tables try to sum up all available possibilities, and give some benchmark (average objects/second for writing or reading).
In these tables:
- 'SQLite3 (file full/off/exc)' indicates use of the internal {\i @*SQLite3@} engine, with or without {\f1\fs20 Synchronous := smOff} and/or {\f1\fs20 DB.LockingMode := lmExclusive} - see @60@;
- 'SQLite3 (mem)' stands for the internal {\i SQLite3} engine running in memory;
- 'SQLite3 (ext ...)' is about access to a {\i SQLite3} engine as external database - see @27@, either as file or memory;
- '{\f1\fs20 TObjectList}' indicates a {\f1\fs20 TSQLRestServerStaticInMemory} instance - see @57@ - either static (with no SQL support) or virtual (i.e. SQL featured via {\i SQLite3} virtual table mechanism) which may persist the data on disk as JSON or compressed binary;
- '@*NexusDB@' is the free embedded edition, available from official site;
- '@*Jet@' stands for a {\i MSAccess} database engine, accessed via OleDB.
- 'Oracle' shows the results of our direct OCI access layer ({\f1\fs20 SynDBOracle.pas});
- '@*Zeos@ *' indicates that the database was accessed directly via the @*ZDBC@ layer;
- 'FireDAC *' stands for {\i @*FireDAC@} library;
- 'UniDAC *' stands for {\i @*UniDAC@} library;
- 'BDE *' when using a {\i @*BDE@} connection;
................................................................................
|{\b ODBC MySQL}|10143|65538|82447
|{\b ZEOS MySQL}|2052|171803|245772
|{\b FireDAC MySQL}|3636|75081|105028
|{\b UniDAC MySQL}|4798|99940|146968
|%
The {\i @*SQLite3@} layer gives amazing reading results, which makes it a perfect fit for most typical ORM use. When running with {\f1\fs20 DB.LockingMode := lmExclusive} defined (i.e. "off exc" rows), reading speed is very high, and benefits from exclusive access to the database file - see @60@. External database access is only required when data is expected to be shared with other processes, or for better scaling: e.g. for physical n-Tier installation with dedicated database server(s).
In the above table, it appears that all libraries based on {\f1\fs20 DB.pas} are slower than the others for reading speed. In fact, {\f1\fs20 TDataSet} sounds to be a real bottleneck, due to its internal data marshalling. Even {\i @*FireDAC@}, which is known to be very optimized for speed, is limited by the {\f1\fs20 TDataSet} structure. Our direct classes, or even ZEOS/ZDBC performs better, since they are able to output JSON content with no additional marshalling.
For both writing and reading, {\f1\fs20 TObjectList} / {\f1\fs20 TSQLRestServerStaticInMemory} engine gives impressive results, but has the weakness of being in-memory, so it is not ACID by design, and the data has to fit in memory. Note that indexes are available for IDs and {\f1\fs20 stored AS_UNIQUE} properties.
As a consequence, search of non-unique values may be slow: the engine has to loop through all rows of data. But for unique values (defined as {\f1\fs20 stored AS_UNIQUE}), both insertion and search speed is awesome, due to its optimized O(1) hash algorithm - see the following benchmark, especially the "{\i By name}" row for "{\i TObjectList}" columns, which correspond to a search of an unique {\f1\fs20 RawUTF8} property value via this hashing method.
|%9%10%10%10%10%10%10%10%10%9%9
| |\b SQLite3 (file full)|SQLite3 (file off)|SQLite3 (mem)|TObjectList (static) |TObjectList (virt.)|SQLite3 (ext file full)|SQLite3 (ext file off)|SQLite3 (ext mem)|Oracle|Jet\b0
|{\b By one}|10461|10549|44737|103577|103553|43367|44099|45220|901|1074
|{\b By name}|9694|9651|32350|70534|60153|22785|22240|23055|889|1071
|{\b All Virt.}|167095|162956|168651|253292|118203|97083|90592|94688|56639|52764
|{\b All Direct}|167123|144250|168577|254284|256383|170794|165601|168856|88342|75999
................................................................................
\TSQLVirtualTableJSON\TSQLVirtualTable
\TSQLVirtualTableBinary\TSQLVirtualTableJSON
\TSQLVirtualTableLog\TSQLVirtualTable
\TSQLVirtualTableCursorIndex\TSQLVirtualTableCursor
\TSQLVirtualTableCursorJSON\TSQLVirtualTableCursorIndex
\TSQLVirtualTableCursorLog\TSQLVirtualTableCursorIndex
\
{\f1\fs20 @*TSQLVirtualTableJSON@, @*TSQLVirtualTableBinary@} and {\f1\fs20 TSQLVirtualTableCursorJSON} classes will implement a Virtual Table using a {\f1\fs20 @*TSQLRestServerStaticInMemory@} instance to handle fast in-memory @*static@ databases. Disk storage will be encoded either as UTF-8 @*JSON@ (for the {\f1\fs20 TSQLVirtualTableJSON} class, i.e. the '{\f1\fs20 JSON}' module), or in a proprietary @*SynLZ@ compressed format (for the {\f1\fs20 TSQLVirtualTableBinary} class, i.e. the '{\f1\fs20 Binary}' module). File extension on disk will be simply {\f1\fs20 .json} for the '{\f1\fs20 JSON}' module, and {\f1\fs20 .data} for the '{\f1\fs20 Binary}' module. Just to mention the size on disk difference, the 502 KB {\f1\fs20 People.json} content (as created by included regression tests) is stored into a 92 KB {\f1\fs20 People.data} file, in our proprietary optimized format.
Note that the virtual table module name is retrieved from the class name. For instance, the {\f1\fs20 TSQLVirtualTableJSON} class will have its module named as 'JSON' in the SQL code.
To handle external databases, two dedicated classes, named {\f1\fs20 TSQLVirtualTableExternal} and {\f1\fs20 TSQLVirtualTableCursorExternal} will be defined in a similar manner - see @%%HierExternalTables@ @30@.
As you probably have already stated, all those Virtual Table mechanism is implemented in {\f1\fs20 mORMot.pas}. Therefore, it is independent from the {\i @*SQLite3@} engine, even if, to my knowledge, there is no other SQL database engine around able to implement this pretty nice feature.
:  Defining a Virtual Table module
Here is how the {\f1\fs20 TSQLVirtualTableLog} class type is defined, which will implement a @*Virtual Table@ module named "{\f1\fs20 Log}". Adding a new module is just made by overriding some Delphi methods:
!  TSQLVirtualTableLog = class(TSQLVirtualTable)
!  protected
................................................................................
!    property DateTime: TDateTime read fDateTime;
!    /// the log event level
!    property Level: TSynLogInfo read fLevel;
!    /// the textual message associated to the log event
!    property Content: RawUTF8 read fContent;
!  end;
You could have overridden the {\f1\fs20 Structure} method in order to provide the {\f1\fs20 CREATE TABLE} SQL statement expected. But using Delphi class RTTI allows the construction of this SQL statement with the appropriate column type and @*collation@, common to what the rest of the @*ORM@ will expect.
Of course, this {\f1\fs20 RecordClass} property is not mandatory. For instance, the {\f1\fs20 TSQLVirtualTableJSON.GetTableModuleProperties} method won't return any associated {\f1\fs20 TSQLRecordClass}, since it will depend on the table it is implementing, i.e. the running {\f1\fs20 @*TSQLRestServerStaticInMemory@} instance. Instead, the {\f1\fs20 Structure} method is overridden, and will return the corresponding field layout of each associated table.
Here is how the {\f1\fs20 Prepare} method is implemented, and will handle the {\f1\fs20 vtWhereIDPrepared} feature:
!function TSQLVirtualTable.Prepare(var Prepared: TSQLVirtualTablePrepared): boolean;
!begin
!  result := Self<>nil;
!  if result then
!!    if (vtWhereIDPrepared in fModule.Features) and
!!       Prepared.IsWhereIDEquals(true) then
................................................................................
{\f1\fs20 TSQLRecordLogFile} was defined to map the column name as retrieved by the {\f1\fs20 TSQLVirtualTableLog} ('{\f1\fs20 log}') module, and should not to be used for any other purpose.
The Virtual Table module associated from such classes is retrieved from an association made to the server {\f1\fs20 @*TSQLModel@}. In a @*Client-Server@ application, the association is not needed (nor to be used, since it may increase code size) on the Client side. But on the server side, the {\f1\fs20 TSQLModel. VirtualTableRegister} method must be called to associate a {\f1\fs20 TSQLVirtualTableClass} (i.e. a Virtual Table module implementation) to a {\f1\fs20 TSQLRecordVirtualClass} (i.e. its ORM representation).
For instance, the following code will register two {\f1\fs20 TSQLRecord} classes, the first using the '{\f1\fs20 JSON}' virtual table module, the second using the '{\f1\fs20 Binary}' module:
!  Model.VirtualTableRegister(TSQLRecordDali1,TSQLVirtualTableJSON);
!  Model.VirtualTableRegister(TSQLRecordDali2,TSQLVirtualTableBinary);
This registration should be done on the Server side only, {\i before} calling {\f1\fs20 TSQLRestServer.Create} (or {\f1\fs20 TSQLRestClientDB.Create}, for a @*stand-alone@ application). Otherwise, an exception is raised at virtual table creation.
:57  In-Memory "static" process
We have seen that the {\f1\fs20 @*TSQLVirtualTableJSON@, @*TSQLVirtualTableBinary@} and {\f1\fs20 TSQLVirtualTableCursorJSON} classes implement a @*Virtual Table@ module using a {\f1\fs20 @**TSQLRestServerStaticInMemory@} instance to handle fast @**static@ in-memory database.
Why use such a database type, when you can create a {\i @*SQLite3@} in-memory table, using the {\f1\fs20 :memory:} file name? That is the question...
- {\i SQlite3} in-memory tables are not persistent, whereas our {\f1\fs20 JSON} or {\f1\fs20 Binary} virtual table modules can be written on disk on purpose, if the {\f1\fs20 aServer.StaticVirtualTable[aClass].CommitShouldNotUpdateFile} property is set to {\f1\fs20 true} - in this case, file writing should be made by calling explicitly the {\f1\fs20 aServer.StaticVirtualTable[aClass].UpdateToFile} method;
- {\i SQlite3} in-memory tables will need two database connections, or call to the {\f1\fs20 @*ATTACH DATABASE@} SQL statement - both of them are not handled natively by our @*Client-Server@ framework;
- {\i SQlite3} in-memory tables are only accessed via SQL statements, whereas {\f1\fs20 @*TSQLRestServerStaticInMemory@} tables can have faster direct access for most common @*REST@ful commands ({\f1\fs20 GET / POST / PUT / DELETE} individual rows) - this could make a difference in server CPU load, especially with the @*Batch@ feature of the framework;
- On the server side, it could be very convenient to have a direct list of in-memory {\f1\fs20 @*TSQLRecord@} instances to work with in pure Delphi code; this is exactly what {\f1\fs20 TSQLRestServerStaticInMemory} allows, and definitively makes sense for an @*ORM@ framework;
- On the client or server side, you could create calculated fields easily with {\f1\fs20 TSQLRestServerStaticInMemory} dedicated "getter" methods written in Delphi, whereas {\i SQlite3} in-memory tables would need additional SQL coding;
- {\i SQLite3} tables are stored in the main database file - in some cases, it could be much convenient to provide some additional table content in some separated database file (for a round robin table, a configuration table written in JSON, some content to be shared among users...): this is made possible using our {\f1\fs20 JSON} or {\f1\fs20 Binary} virtual table modules (but, to be honest, the {\f1\fs20 @*ATTACH DATABASE@} statement could provide a similar feature);
- The {\f1\fs20 TSQLRestServerStaticInMemory} class can be used stand-alone, i.e. without the {\i SQLite3} engine so it could be used to produce small efficient server software - see the "{\f1\fs20 SQLite3\\Samples\\01 - In Memory ORM}" folder.
:   In-Memory tables
A first way of using @*static@ tables, independently from the {\i SQLite3} engine, is to call the {\f1\fs20 TSQLRestServer. StaticDataCreate} method.
This method is only to be called server-side, of course. For the Client, there is no difference between a regular and a static table.
The in-memory {\f1\fs20 @*TSQLRestServerStaticInMemory@} instance handling the storage can be accessed later via the {\f1\fs20 StaticDataServer[]} property array of {\f1\fs20 TSQLRestServer}.
As we just stated, this primitive but efficient database engine can be used without need of the {\i SQLite3} database engine to be linked to the executable, saving some KB of code if necessary. It will be enough to handle most basic @*REST@ful requests.
:76   In-Memory virtual tables
A more advanced and powerful way of using @*static@ tables is to define some classes inheriting from {\f1\fs20 @*TSQLRecordVirtualTableAutoID@}, and associate them with some {\f1\fs20 TSQLVirtualTable} classes. The {\f1\fs20 TSQLRecordVirtualTableAutoID} parent class will specify that associated @*virtual table@ modules will behave like normal {\i SQLite3} tables, so will have their {\f1\fs20 RowID} property computed at {\f1\fs20 INSERT}).
For instance, the supplied regression tests define such two tables with three columns, named {\f1\fs20 FirstName}, {\f1\fs20 YearOfBirth} and {\f1\fs20 YearOfDeath}, after the @*published properties@ definition:
!  TSQLRecordDali1 = class(TSQLRecordVirtualTableAutoID)
!  private
!    fYearOfBirth: integer;
................................................................................
!    end;
!!    Check(aClient.TableRowCount(TSQLRecordDali1)=1001);
!!    aClient.Commit;
!  except
!!    aClient.RollBack;
!  end;
A {\f1\fs20 Commit} is needed from the Client side to write anything on disk. From the Server side, in order to create disk content, you'll have to explicitly call such code on purpose:
As we already noticed, data will be written by default on disk with our {\f1\fs20 @*TSQLRestServerStaticInMemory@}-based virtual tables. In fact, the {\f1\fs20 Commit} method in the above code will call {\f1\fs20 TSQLRestServerStaticInMemory.UpdateFile}.
Please note that the {\i @*SQlite3@} engine will handle any Virtual Table just like regular {\i SQLite3} tables, concerning the @*atomic@ity of the data. That is, if no explicit @*transaction@ is defined (via {\f1\fs20 TransactionBegin / Commit} methods), such a transaction will be performed for every database modification (i.e. all @*CRUD@ operations, as {\f1\fs20 INSERT / UPDATE / DELETE}). The {\f1\fs20 TSQLRestServerStaticInMemory. UpdateToFile} method is not immediate, because it will write all table data each time on disk. It is therefore mandatory, for performance reasons, to nest multiple modification to a Virtual Table with such a transaction, for better performance. And in all cases, it is the standard way of using the ORM. If for some reason, you later change your mind and e.g. move your table from the {\f1\fs20 TSQLVirtualTableJSON / TSQLVirtualTableBinary} engine to the default {\i SQlite3} engine, your code could remain untouched.
It is possible to force the In-Memory virtual table data to stay in memory, and the {\f1\fs20 COMMIT} statement to write nothing on disk, using the following property:
! Server.StaticVirtualTable[TSQLRecordDali1].CommitShouldNotUpdateFile := true;
In order to create disk content, you'll then have to explicitly call the corresponding method on purpose:
! Server.StaticVirtualTable[TSQLRecordDali1].UpdateToFile;
Since {\f1\fs20 StaticVirtualTable} property is only available on the Server side, you are the one to blame if your client updates the table data and this update never reaches the disk!
:   In-Memory and ACID
For data stored in memory, the {\f1\fs20 TSQLRestServerStaticInMemory} table is @*ACID@.\line It means that concurrent access will be consistent and work safely, as expected.
On disk, this kind of table is ACID only when its content is written to the file.\line I mean, the whole file which will be written in an ACID way. The file will always be consistent.
The exact process of these in-memory tables is that each time you write some new data to a {\f1\fs20 TSQLRestServerStaticInMemory} table:
- It will be ACID in memory (i.e. work safely in concurrent mode);
- Individual writes (INSERT/UPDATE/DELETE) won't automatically be written to file;
- COMMIT will by default write the whole table to file (either as JSON or compressed binary);
- COMMIT won't write the data to file if the {\f1\fs20 CommitShouldNotUpdateFile} property is set to TRUE;
- ROLLBACK process won't do anything, so won't be ACID - but since your code may later use a real RDBMS, it is a good habit to always write the command, like in the sample code above, as {\f1\fs20 except aClient.RollBack}.
When you write the data to file, the whole file is rewritten: it seems not feasible to write the data to disk at every write - in this case, SQLite3 in exclusive mode will be faster, since it will write only the new data, not the whole table content.
This may sound like a limitation, but on our eyes, it could be seen more like a feature. For a particular table, we do not need nor want to have a whole RDBMS/SQL engine, just direct and fast access to a {\f1\fs20 TObjectList}. The feature is to integrate it with our @*REST@ engine, and still be able to store your data in a regular database later ({\i SQLite3} or external), if it appears that {\f1\fs20 TSQLRestServerStaticInMemory} storage is too limited for your process.
:  Virtual Tables to access external databases
As will be stated @27@, some external databases may be accessed by our ORM.
The @*Virtual Table@ feature of {\i @*SQLite3@} will allow those remote tables to be accessed just like "native" {\i SQLite3} tables - in fact, you may be able e.g. to write a valid SQL query with a {\f1\fs20 @*JOIN@} between {\i SQlite3} tables, {\i @*MS SQL@ Server, @*MySQL@, @*FireBird@, @*PostgreSQL@, @*MySQL@, @*DB2@} and {\i @*Oracle@} databases, even with multiple connections and several remote servers. Think as an ORM-based {\i Business Intelligence} from any database source. Added to our code-based reporting engine (able to generate @*pdf@), it could be a very powerful way of consolidating any kind of data.
In order to define such {\i external} tables, you define your regular {\f1\fs20 @*TSQLRecord@} classes as usual, then a call to the {\f1\fs20 @**VirtualTableExternalRegister@()} function will define this class to be managed as a virtual table, from an external database engine. Using a dedicated external database server may allow better response time or additional features (like data sharing with other applications or languages). Server-side may omit a call to {\f1\fs20 VirtualTableExternalRegister()} if the need of an internal database is expected: it will allow custom database configuration at runtime, depending on the customer's expectations (or license).
:  Virtual tables from the client side
For external databases - see @27@ - the SQL conversion will be done on the fly in a more advanced way, so you should be able to work with such virtual tables from the client side without any specific model notification. In this case, you can safely define your tables as {\f1\fs20 TSQLValue1 = class(TSQLRecord)}, with no further code on client side.
When working with {\i static} (in-memory / {\f1\fs20 TObjectList}) storage, if you expect all ORM features to work remotely, you need to notify the Client-side model that a table is implemented as virtual. Otherwise you may encounter some SQL errors when executing requests, like "{\i no such column: ID}".
................................................................................
struct1:yob -> struct2:yob;
struct1:yod -> struct2:yod;
\
Note that only the {\f1\fs20 ID} and {\f1\fs20 YearOfDeath} column names were customized.
Due to the design of {\i SQLite3} virtual tables, and mORMot internals in its current state, the database primary key must be an {\f1\fs20 INTEGER} field to be mapped as expected by the ORM.
:30  External database ORM internals
The {\f1\fs20 mORMotDB.pas} unit implements @*Virtual Table@s access for any {\f1\fs20 @*SynDB@}-based external database for the framework.
In fact, the {\f1\fs20 TSQLRestServerStaticExternal, TSQLVirtualTableCursorExternal} and {\f1\fs20 TSQLVirtualTableExternal} classes will implement this feature:
\graph HierExternalTables External Databases classes hierarchy
\TSQLRecordVirtual\TSQLRecord
\TSQLRecordVirtualTableAutoID\TSQLRecordVirtual
\TSQLVirtualTableCursorExternal\TSQLVirtualTableCursor
\TSQLVirtualTableExternal\TSQLVirtualTable
\
The registration of the class is done by a call to the following new global procedure:
................................................................................
!procedure VirtualTableExternalRegister(aModel: TSQLModel; aClass: TSQLRecordClass;
!  aExternalDB: TSQLDBConnectionProperties; const aExternalTableName: RawUTF8);
This procedure will register on the Server-side an external database for an ORM class:
- It will define the supplied class to behave like a {\f1\fs20 TSQLRecordVirtualTableAutoID} class (i.e. its {\f1\fs20 @*TSQLModelRecordProperties@.Kind} property will be ovewritten to {\f1\fs20 rCustomAutoID} in this ORM model);
- It will associate the supplied class with a {\f1\fs20 TSQLVirtualTableExternal} module;
- The {\f1\fs20 TSQLDBConnectionProperties} instance should be shared by all classes, and released globally when the ORM is no longer needed;
- The full table name, as expected by the external database, should be provided here ({\f1\fs20 SQLTableName} will be used internally as table name when called via the associated {\i SQLite3} Virtual Table) - if no table name is specified (''), will use {\f1\fs20 SQLTableName} (e.g. 'Customer' for a class named {\f1\fs20 TSQLCustomer});
- Internal adjustments will be made to convert SQL on the fly from internal ORM representation into the expected external SQL format (e.g. table name or {\f1\fs20 ID} property) - see {\f1\fs20 TSQLRestServerStatic. AdaptSQLForEngineList} method.
Typical usage may be for instance:
!aProps := TOleDBMSSQLConnectionProperties.Create('.\SQLEXPRESS','AdventureWorks2008R2','','');
!aModel := TSQLModel.Create([TSQLCustomer],'root');
!VirtualTableExternalRegister(aModel,TSQLCustomer,aProps,'Sales.Customer');
!aServer := TSQLRestServerDB.Create(aModel,'application.db'),true)
All the rest of the code will use the "regular" ORM classes, methods and functions, as stated by @3@.
In order to be stored in an external database, the ORM records can inherit from any {\f1\fs20 TSQLRecord} class. Even if this class does not inherit from {\f1\fs20 TSQLRecordVirtualTableAutoID}, it will behave as such, once {\f1\fs20 @*VirtualTableExternalRegister@} function has been called for the given class.
As with any regular {\f1\fs20 TSQLRecord} classes, the ORM core will expect external tables to map an {\f1\fs20 Integer ID} published property, auto-incremented at every record insertion. Since not all databases handle such fields - e.g. {\i @*Oracle@} - auto-increment will be handled via a {\f1\fs20 select max(id) from tablename} statement run at initialization, then computed on the fly via a thread-safe cache of the latest inserted {\i RowID}.
You do not have to know where and how the data persistence is stored. The framework will do all the low-level DB work for you. And thanks to the {\i Virtual Table} feature of {\i SQlite3}, internal and external tables can be mixed within SQL statements. Depending on the implementation needs, classes could be persistent either via the internal {\i SQLite3} engine, or via external databases, just via a call to {\f1\fs20 VirtualTableExternalRegister()} before server initialization.
In fact, {\f1\fs20 TSQLVirtualTableCursorExternal} will convert any query on the external table into a proper optimized SQL query, according to the indexes existing on the external database. {\f1\fs20 TSQLVirtualTableExternal} will also convert individual SQL modification statements (like insert / update / delete) at the {\i SQLite3} level into remote SQL statements to the external database.
Most of the time, all @*REST@ful methods ({\f1\fs20 GET/POST/PUT/DELETE}) will be handled directly by the {\f1\fs20 TSQLRestServerStaticExternal} class, and won't use the virtual table mechanism. In practice, most access to the external database will be as fast as direct access, but the virtual table will always be ready to interpret any cross-database complex request or statement.
Direct REST access will be processed as following - when adding an object, for instance:
\graph TSQLRecordPeopleExtVirtualRest ORM Access Via REST
\TSQLRestServerDB.Add\TSQLRestServerDB.EngineAdd\internal川able
\TSQLRestServerDB.EngineAdd\TSQLRequest\INSERT INTO...
\TSQLRequest\SQlite3 engine\internal engine
\SQlite3 engine\SQLite3 file
\TSQLRestServerDB.Add\TSQLRestServerStaticExternal.EngineAdd\external川able卜EST
\TSQLRestServerStaticExternal.EngineAdd\ISQLDBStatement\INSERT INTO...
\ISQLDBStatement\External DB client\ODBC/ZDBC/OleDB...
\External DB client\External DB server
\
\page
Indirect access via virtual tables will be processed as following:
\graph TSQLRecordPeopleExtVirtualVirtual ORM Access Via Virtual Table
\TSQLRestServerDB.Add\TSQLRestServerDB.EngineAdd\internal or叉xternal table
................................................................................
\TSQLVirtualTableExternal.Insert\ISQLDBStatement\INSERT INTO...
\ISQLDBStatement\External DB client\ODBC/ZDBC/OleDB...
\External DB client\External DB server
\
About speed, here is an extract of the test regression log file (see code above, in previous paragraph), which shows the difference between RESTful call and virtual table call, working with more than 11,000 rows of data:
$  - External via REST: 133,666 assertions passed  409.82ms
$  - External via virtual table: 133,666 assertions passed  1.12s
The first run is made with {\f1\fs20 TSQLRestServer.StaticVirtualTableDirect} set to TRUE (which is the default) - i.e. it will call directly {\f1\fs20 TSQLRestServerStaticExternal} for RESTful commands, and the second will set this property to FALSE - i.e. it will call the {\i SQLite3} engine and let its virtual table mechanism convert it into another SQL calls.
It is worth saying that this test is using an in-memory {\i SQLite3} database (i.e. instantiated via {\f1\fs20 SQLITE_MEMORY_DATABASE_NAME} as pseudo-file name) as its external DB, so what we test here is mostly the ORM overhead, not the external database speed. With real file-based or remote databases (like @*MS SQL@), the overhead of remote connection won't make noticeable the use of Virtual Tables.
In all cases, letting the default {\f1\fs20 StaticVirtualTableDirect=true} will ensure the best possible performance. As stated by @59@, using a virtual or direct call won't affect the CRUD operation speed: it will by-pass the virtual engine whenever possible.
\page
: MongoDB database access
{\i @**MongoDB@} (from "humongous") is a cross-platform document-oriented database system, and certainly the best known @*NoSQL@ database.\line According to @http://db-engines.com in April 2014, {\i MongoDB} is in 5th place of the most popular types of database management systems, and first place for NoSQL database management systems.\line Our {\i mORMot} gives premium access to this database, featuring full @82@ abilities to the framework.
Integration is made at two levels:
- Direct low-level access to the {\i MongoDB} server, in the {\f1\fs20 SynMongoDB.pas} unit;
- Close integration with our ORM (which becomes {\i defacto} an ODM), in the {\f1\fs20 mORMotMongoDB.pas} unit.
{\i MongoDB} eschews the traditional table-based relational database structure in favor of @*JSON@-like documents with dynamic schemas ({\i MongoDB} calls the format @*BSON@), which matches perfectly {\i mORMot}'s @*REST@ful approach.
:  MongoDB client
The {\f1\fs20 SynMongoDB.pas} unit features direct optimized access to a {\i MongoDB} server.
................................................................................
And... that's all!
You can then use any ORM command, as usual:
!  writeln(Client.TableRowCount(TSQLORM)=0);
As with external databases, you can specify the field names mapping between the objects and the {\i MongoDB} collection.\line By default, the {\f1\fs20 TSQLRecord.ID} property is mapped to the {\i MongoDB}'s {\f1\fs20 _id} field, and the ORM will populate this {\f1\fs20 _id} field with a sequence of integer values, just like any {\f1\fs20 TSQLRecord} table.\line You can specify your own mapping, using for instance:
! aModel.Props[aClass].ExternalDB.MapField(..)
Since the field names are stored within the document itself, it may be a good idea to use shorter naming for the {\i MongoDB} collection. It may save some storage space, when working with a huge number of documents.
Once the {\f1\fs20 TSQLRecord} is mapped to a {\i MongoDB} collection, you can always have direct access to the {\f1\fs20 TMongoCollection} instance later on, by calling:
! (aServer.StaticDataServer[aClass] as TSQLRestServerStaticMongoDB).Collection
This may allow any specific task, including any tuned query or process.
:   ORM/ODM CRUD methods
You can add documents with the standard CRUD methods of the ORM, as usual:
!  R := TSQLORM.Create;
!  try
!    for i := 1 to COLL_COUNT do begin
!      R.Name := 'Name '+Int32ToUTF8(i);
................................................................................
|%
Note that you can have {\i several} protocols exposing the same {\f1\fs20 TSQLRestServer} instance. You may expose the same server over HTTP and over named pipes, at the same time, depending on your speed requirements.
: TSQLRest classes
This architecture is implemented by a hierarchy of classes, implementing the @*REST@ful pattern - see @9@ - for either stand-alone, client or server side, all inheriting from a {\f1\fs20 @*TSQLRest@} common ancestor, as two main branches:
\graph ClientServerRESTClasses RESTful Client-Server classes
\TSQLRestServer\TSQLRest
\TSQLRestClient\TSQLRest

\
All ORM operations (aka @*CRUD@ process) are available from the abstract {\f1\fs20 TSQLRest} class definition, which is overridden to implement either a Server (via {\f1\fs20 TSQLRestServer} classes), or a Client (via {\f1\fs20 TSQLRestClientURI} classes) access to the data.
You should instantiate the classes corresponding to the needed transmission protocol, but should better rely on abstraction, i.e. implement your whole code logic relying on abstract {\f1\fs20 TSQLRestClient / TSQLRestServer} classes. It will then help changing from one protocol or configuration at runtime, depending on your customer's expectations.
:  Server classes
The following classes are available to implement a {\i Server} instance:
\graph ServerRESTClasses RESTful Server classes
\TSQLRestServerDB\TSQLRestServer
\TSQLRestServerRemoteDB\TSQLRestServer
\TSQLRestServerFullMemory\TSQLRestServer
\TSQLRestServer\TSQLRest
\
In practice, in order to implement the business logic, you should better create a new {\f1\fs20 class}, inheriting from one of the above {\f1\fs20 TSQLRestServer} classes. Having your own inherited class does make sense, especially for implementing your own method-based services - see @49@, or {\f1\fs20 override} internal methods.
The {\f1\fs20 TSQLRestServerDB} class is the main kingn of Server of the framework. It will host a {\i @*SQLite3@} engine, as its core @42@.
If your purpose is not to have a full {\i SQLite3} engine available, you may create your server from a {\f1\fs20 @*TSQLRestServerFullMemory@} class instead of {\f1\fs20 TSQLRestServerDB}: this will implement a fast in-memory engine (using {\f1\fs20 TSQLRestServerStaticInMemory} instances), with basic CRUD features (for ORM), and persistence on disk as JSON or optimized binary files - this kind of server is enough to handle authentication, and host @*service@s in a stand-alone way.
If your services need to have access to a remote ORM server, it may use a {\f1\fs20 @*TSQLRestServerRemoteDB@} class instead: this server will use an internal {\f1\fs20 TSQLRestClient} instance to handle all ORM operations - it can be used e.g. to host some services on a stand-alone server, with all ORM and data access retrieved from another server: it will allow to easily implement a proxy architecture (for instance, as a DMZ for publishing services, but letting ORM process stay out of scope). See @75@ for some hosting scenarios.

In the {\i mORMot} units, you may also find those classes also inheriting from {\f1\fs20 TSQLRestServer}:
\graph ServerRESTFakeClasses RESTful Server static classes
\TSQLRestServerStaticInMemory\TSQLRestServerStaticRecordBased
\TSQLRestServerStaticInMemoryExternal\TSQLRestServerStaticInMemory
\TSQLRestServerStaticRecordBased\TSQLRestServerStatic
\TSQLRestServerStaticExternal\TSQLRestServerStatic
\TSQLRestServerStatic\TSQLRestServer

\




In the above class hierarchy, the {\f1\fs20 TSQLRestServerStatic[InMemory][External]} classes are in fact "fake servers". They are used within a main {\f1\fs20 TSQLRestServer} to host some given {\f1\fs20 TSQLRecord} classes, either in-memory, or on external databases. They do not enter in account in our Client-Server presentation, but are implementation details, on the server side.
:  Client classes
A full set of {\i client} classes will implement a RESTful access to a remote database, with associated services and business logic:
\graph ClientRESTClasses RESTful Client classes
rankdir=LR;
\TSQLRestClientURINamedPipe\TSQLRestClientURI
\TSQLRestClientURIMessage\TSQLRestClientURI
\TSQLRestClientURIDll\TSQLRestClientURI
................................................................................
On typical production use, the {\i mORMot} HTTP server - see @6@ - will run on its own optimized thread pool, then call the {\f1\fs20 TSQLRestServer.URI} method. This method is therefore expected to be thread-safe, e.g. from the {\f1\fs20 TSQLHttpServer. Request} method. Thanks to the @*REST@ful approach of our framework, this method is the only one which is expected to be thread-safe, since it is the single entry point of the whole server. This KISS design ensure better test coverage.
On the Client side, all {\f1\fs20 TSQLRestClientURI} classes are protected by a global mutex (critical section), so are thread-safe. As a result, a single {\f1\fs20 TSQLHttpClient} instance can be shared among several threads, even if you may also use one client per thread, as is done with sample 21 - see below, for better responsiveness.
:  By design
We will now focus on the server side, which is the main strategic point (and potential bottleneck or point of failure) of any {\i Client-Server} architecture.
In order to achieve this thread-safety without sacrificing performance, the following rules were applied in {\f1\fs20 TSQLRestServer.URI}:
- Most of this method's logic is to process the URI and parameters of the incoming request (in {\f1\fs20 TSQLRestServerURIContext.URIDecode*} methods), so is thread-safe by design (e.g. {\f1\fs20 Model} and {\f1\fs20 RecordProps} access do not change during process);
- At @*REST@ful / @*CRUD@ level, {\f1\fs20 Add/Update/Delete/TransactionBegin/Commit/Rollback} methods are locked by default (with a 2 seconds timeout), and {\f1\fs20 Retrieve*} methods are not;
- {\f1\fs20 TSQLRestServerStatic} main methods ({\f1\fs20 EngineList, EngineRetrieve, EngineAdd, EngineUpdate, EngineDelete, EngineRetrieveBlob, EngineUpdateBlob}) are thread-safe: e.g. {\f1\fs20 @*TSQLRestServerStaticInMemory@} uses a per-Table Critical Section;
- {\f1\fs20 TSQLRestServerCallBack} method-based services - i.e. @*published method@s of the inherited {\f1\fs20 TSQLRestServer} class as stated @49@ - must be implemented to be thread-safe by default;
- {\f1\fs20 Interface}-based services - see @63@ - have several execution modes, including thread safe automated options (see {\f1\fs20 TServiceMethodOption}) or manual thread safety expectation, for better scaling - see @72@;
- A protected {\f1\fs20 fSessionCriticalSection} is used to protect shared {\f1\fs20 fSession[]} access between clients;
- The {\i @*SQLite3@} engine access is protected at SQL/JSON @*cache@ level, via {\f1\fs20 DB.LockJSON()} calls in {\f1\fs20 @*TSQLRestServerDB@} methods;
- Remote external tables - see @27@ - use thread-safe connections and statements when accessing the databases via SQL;
- Access to {\f1\fs20 fStats} was not made thread-safe, since this data is indicative only: a {\i mutex} was not used to protect this resource.
We tried to make the internal Critical Sections as short as possible, or relative to a table only (e.g. for {\f1\fs20 TSQLRestServerStaticInMemory}).
This default behavior can be tuned, using {\f1\fs20 TSQLRestServerURI.AcquireExecutionMode[]} property and {\f1\fs20 AcquireExecutionLockedTimeOut[]} when {\f1\fs20 amLocked} is set:
|%25%50%25
|\b Command|Description|Default\b0
|{\f1\fs20 execSOAByMethod}|for method-based services|{\f1\fs20 amUnlocked}
|{\f1\fs20 execSOAByInterface}|for interface-based services|{\f1\fs20 amUnlocked}
|{\f1\fs20 execORMGet}|for ORM reads i.e. {\i Retrieve*} methods|{\f1\fs20 amUnlocked}
|{\f1\fs20 execORMWrite}|for ORM writes i.e. {\i Add Update Delete TransactionBegin Commit Rollback} methods|{\f1\fs20 amLocked} +\line timeout of 2000 ms
................................................................................
\
In fact, the above function correspond to a database @*model@ with only external virtual tables, and with {\f1\fs20 StaticVirtualTableDirect=false}, i.e. calling the Virtual Table mechanism of {\i SQlite3} for each request.
But most of the time, i.e. for RESTful / @*CRUD@ commands, the execution is more direct:
\graph ArchVirtualDirect Client-Server implementation - Server side with "static" Virtual Tables
subgraph cluster {
\Http Server\TSQLRestServer.URI\dispatch
\TSQLRestServer.URI\TSQLRestServerDB.EngineAdd\Is a SQLite3 table?
\TSQLRestServer.URI\TSQLRestServerStaticExternal.EngineAdd\Is a static table?
\TSQLRestServerDB.EngineAdd\SQLite3 SQL\SQL insert
\SQLite3 SQL\SQLite3 engine\Is a SQLite3 table?
\SQLite3 engine\SQLite3 BTREE\prepare + execute
\SQLite3 BTREE\Database file\atomic write
\SQLite3 BTREE\TSQLRestServer.URI\return new ID
\TSQLRestServerStaticExternal.EngineAdd\TSQLDBConnectionProperties\external database
\TSQLDBConnectionProperties\TSQLDBStatement\compute SQL
\TSQLDBStatement\OleDB/ODBC or other\execute SQL
\OleDB/ODBC or other\Database Client\store data
\Database Client\OleDB/ODBC or other
\OleDB/ODBC or other\TSQLDBStatement
\TSQLDBStatement\TSQLRestServerStaticExternal.EngineAdd
\TSQLRestServerStaticExternal.EngineAdd\TSQLRestServer.URI\return new ID
\TSQLRestServer.URI\Http Server\return 200 OK + ID
label = "Server";
}
\
As stated in @27@, the @*static@ {\f1\fs20 TSQLRestServerStaticExternal} instance is called for most RESTful access. In practice, this design will induce no speed penalty, when compared to a direct database access. It could be even faster, if the server is located on the same computer than the database: in this case, use of JSON and REST could be faster - even faster when using @28@.
In order to be exhaustive, here is a more complete diagram, showing how native {\i @*SQLite3@}, in-memory or external tables are handled on the server side. You'll find out how CRUD statements are handled directly for better speed, whereas any SQL @*JOIN@ query can also be processed among all kind of tables.
\graph ArchServerFull Client-Server implementation - Server side
subgraph cluster {
\Http Server\TSQLRestServer.URI\dispatch
\TSQLRestServer.URI\TSQLRestServerDB.Engine*\Refers to兀 SQLite3 table or兀 JOINed query?
\TSQLRestServer.URI\TSQLRestServerStatic.九ngine*\CRUD over兀 static table?
\TSQLRestServerDB.Engine*\SQLite3 SQL\decode into SQL
\SQLite3 SQL\SQLite3 engine\prepare + execute
\SQLite3 engine\SQLite3 BTREE\For any又QLite3川able
\SQLite3 engine\TSQLRestServer.URI
\SQLite3 BTREE\Database file\atomic尸ead/write
\SQLite3 BTREE\SQLite3 engine
\SQLite3 engine\TSQLVirtualTableJSON三SQLVirtualTableBinary\For any in-memory丈irtual Table
................................................................................
\TSQLDBConnectionProperties\TSQLDBStatement\compute SQL
\TSQLDBStatement\OleDB/ODBC or other\execute SQL
\OleDB/ODBC or other\Database Client\handle data
\Database Client\OleDB/ODBC or other
\OleDB/ODBC or other\TSQLDBStatement
\TSQLDBStatement\SQLite3 engine
\TSQLRestServer.URI\Http Server\return 200 OK +尸esult (JSON)
\TSQLRestServerStatic.九ngine*\TSQLRestServerStaticExternal.九ngine*\Is an external丈irtual Table?
\TSQLRestServerStatic.九ngine*\TSQLRestServerStaticInMemory.九ngine*\Is an in-memory table?(Virtual or static)
\TSQLRestServerStatic.九ngine*\TSQLRestServerStaticMongoDB.九ngine*\Is a MongoDB table?
\TSQLRestServerStaticExternal.九ngine*\TSQLDBConnectionProperties三SQLDBStatement\compute SQL
\TSQLDBConnectionProperties三SQLDBStatement\OleDB/ODBC寸r other\execute SQL
\OleDB/ODBC寸r other\TSQLRestServer.URI
\OleDB/ODBC寸r other\Database七lient\handle data
\Database七lient\OleDB/ODBC寸r other
\TSQLRestServerStaticInMemory.九ngine*\CRUD夕n-memory三SQLRecord
\CRUD夕n-memory三SQLRecord\TSQLRestServer.URI
\TSQLVirtualTableJSON三SQLVirtualTableBinary\Process夕n-memory三SQLRecord
\Process夕n-memory三SQLRecord\SQLite3 engine
\TSQLRestServerStaticMongoDB.九ngine*\TMongoCollection刀ongoDB Client
\TMongoCollection刀ongoDB Client\TSQLRestServer.URI
label = "Server";
}
\
You will find out some speed numbers resulting from this unique architecture in the supplied @59@.
: Stateless design
:  Server side synchronization
................................................................................
!  end;
Just for fun... I could not resist posting this code here; if you are curious, take a look at the "official" {\f1\fs20 RTTI.pas} or {\f1\fs20 RIO.pas} units as provided by Embarcadero, and you will probably find out that the {\i mORMot} implementation is much easier to follow, and also faster (it does not recreate all the stubs or virtual tables for each wrapper, for instance). :)

[SDD-DI-2.2.1]
; SRS-DI-2.2.1 - The SQLite3 engine must be embedded to the framework
:Implementation
It's worth noting that the {\i Synopse mORMot framework}, whatever its previous name stated ("Synopse SQLite3 framework"), is not bound to {\i SQLite3}.
You can use another database engine for its internal data storage, for example we provide a {\f1\fs20 TSQLRestServerStaticInMemory} class which implements a fast but limited in-memory database engine.
Therefore, the {\i SQLite3} engine itself is not implemented in the @!TSQLRestServer!Lib\SQLite3\mORMot.pas@ unit, but in dedicated units.
The {\i SQLite3} engine is accessed at two levels:
- A low-level direct access to the {\i SQLite3} library, implemented in @!TSQLite3LibraryDynamic,TSQLite3Library,TSQLRequest.Execute,TSQLDataBase,TSQLTableDB.Create!Lib\SynSQLite3.pas@;
- A low-level statically linked library @!TSQLite3LibraryStatic!Lib\SynSQLite3Static.pas@, embedding the engine within the project executable;
- A possible use of external {\f1\fs20 sqlite3.dll} library, via the {\f1\fs20 TSQLite3LibraryDynamic} class;
- A high-level access, implementing a Client-Side or Server-Side native {\f1\fs20 TSQLRest} descendant using the {\i SQLite3} library for ORM data persistence, in @!TSQLRestServerDB,TSQLRestClientDB!Lib\SQLite3\mORMotSQLite3.pas@.
In addition to those two units, the @!TSQLDBSQLite3Connection,TSQLDBSQLite3Statement,TSQLDBSQLite3ConnectionProperties!Lib\SynDBSQLite3.pas@ unit publishes all features of the {\i SQlite3} database engine to its internal {\f1\fs20 SynDB} fast database access classes, which can be used uncoupled from the rest of the framework (i.e. without ORM).







|







 







|







 







|









|







 







|







 







|







 







|







 







|







 







|



|
|
|

|



|







 







|
|






|

|






|







 







|







 







|










|






|
|







 







|



|







 







|







 







>













|

>
|
<
|
|
|
|
|
>

>
>
>
>
|







 







|






|







 







|





|





|
|




|





|







 







|
|
|
|




|



|







 







|







3012
3013
3014
3015
3016
3017
3018
3019
3020
3021
3022
3023
3024
3025
3026
....
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
....
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
....
3632
3633
3634
3635
3636
3637
3638
3639
3640
3641
3642
3643
3644
3645
3646
....
3768
3769
3770
3771
3772
3773
3774
3775
3776
3777
3778
3779
3780
3781
3782
....
4191
4192
4193
4194
4195
4196
4197
4198
4199
4200
4201
4202
4203
4204
4205
....
4231
4232
4233
4234
4235
4236
4237
4238
4239
4240
4241
4242
4243
4244
4245
....
4340
4341
4342
4343
4344
4345
4346
4347
4348
4349
4350
4351
4352
4353
4354
4355
4356
4357
4358
4359
4360
4361
4362
4363
4364
4365
4366
....
4415
4416
4417
4418
4419
4420
4421
4422
4423
4424
4425
4426
4427
4428
4429
4430
4431
4432
4433
4434
4435
4436
4437
4438
4439
4440
4441
4442
4443
4444
4445
4446
....
5148
5149
5150
5151
5152
5153
5154
5155
5156
5157
5158
5159
5160
5161
5162
....
5163
5164
5165
5166
5167
5168
5169
5170
5171
5172
5173
5174
5175
5176
5177
5178
5179
5180
5181
5182
5183
5184
5185
5186
5187
5188
5189
5190
5191
5192
5193
5194
5195
5196
....
5201
5202
5203
5204
5205
5206
5207
5208
5209
5210
5211
5212
5213
5214
5215
5216
5217
5218
5219
....
5562
5563
5564
5565
5566
5567
5568
5569
5570
5571
5572
5573
5574
5575
5576
....
6299
6300
6301
6302
6303
6304
6305
6306
6307
6308
6309
6310
6311
6312
6313
6314
6315
6316
6317
6318
6319
6320
6321
6322
6323

6324
6325
6326
6327
6328
6329
6330
6331
6332
6333
6334
6335
6336
6337
6338
6339
6340
6341
6342
....
6520
6521
6522
6523
6524
6525
6526
6527
6528
6529
6530
6531
6532
6533
6534
6535
6536
6537
6538
6539
6540
6541
....
6743
6744
6745
6746
6747
6748
6749
6750
6751
6752
6753
6754
6755
6756
6757
6758
6759
6760
6761
6762
6763
6764
6765
6766
6767
6768
6769
6770
6771
6772
6773
6774
6775
6776
6777
6778
6779
6780
6781
....
6784
6785
6786
6787
6788
6789
6790
6791
6792
6793
6794
6795
6796
6797
6798
6799
6800
6801
6802
6803
6804
6805
6806
6807
6808
6809
6810
.....
11721
11722
11723
11724
11725
11726
11727
11728
11729
11730
11731
11732
11733
11734
11735
The database is just one way of your objects persistence:
- Don't think about tables with simple types (text/number...), but objects with high level types;
- Don't think about @*Master/Detail@, but logical units;
- Don't think "@*SQL@", think about classes;
- Don't wonder "How will I store it?", but "Which data do I need?".
For instance, don't be tempted to always create a pivot table (via a {\f1\fs20 @*TSQLRecordMany@} property), but consider using a {\i @*dynamic array@}, {\f1\fs20 @*TPersistent@, @*TStrings@} or {\f1\fs20 @*TCollection@} @*published properties@ instead.
Or consider that you can use a {\f1\fs20 TRecordReference} property pointing to any registered class of the {\f1\fs20 @*TSQLModel@}, instead of creating one {\f1\fs20 @*TSQLRecord@} property per potential table.
The {\i mORMot} framework is even able to persist the object without any SQL database, e.g. via {\f1\fs20 TSQLRestStorageInMemory}. In fact, its ORM core is optimized but not tied to SQL.
:  Objects, not tables
With an @*ORM@, you should usually define fewer tables than in a "regular" relational database, because you can use the high-level type of the {\f1\fs20 @*TSQLRecord@} properties to handle some per-row data.
The first point, which may be shocking for a database architect, is that you should better {\ul not} create @*Master/Detail@ tables, but just one "master" object with the details stored within, as @*JSON@, via {\i @*dynamic array@}, {\f1\fs20 @*TPersistent@, @*TStrings@} or {\f1\fs20 @*TCollection@} properties.
Another point is that a table is not to be created for every aspect of your software configuration. Let's confess that some DB architects design one configuration table per module or per data table. In an ORM, you could design a configuration class, then use the unique corresponding table to store all configuration encoded as some JSON data, or some DFM-like data. And do not hesitate to separate the configuration from the data, for all not data-related configuration - see e.g. how the {\f1\fs20 mORMotOptions} unit works. With our framework, you can serialize directly any {\f1\fs20 @*TSQLRecord@} or {\f1\fs20 TPersistent} instance into JSON, without the need of adding this {\f1\fs20 TSQLRecord} to the {\f1\fs20 @*TSQLModel@} list. Since revision 1.13 of the framework, you can even define {\f1\fs20 TPersistent} @published properties@ in your {\f1\fs20 TSQLRecord} class, and it will be automatically serialized as TEXT in the database.
:  Methods, not SQL
At first, you should be tempted to write code as such (this code sample was posted on our forum, and is not bad code, just not using the @*ORM@ orientation of the framework):
!  DrivesModel := CreateDrivesModel();
................................................................................
The {\i mORMot} point of view (which is not the only one), is to let the DB persist the data, as safe and efficient as possible, but rely on higher levels layers to implement the business logic. The framework favors {\i convention over configuration}, which is known to save a lot of time (if you use WCF on a daily basis, as I do, you and your support team know about the {\f1\fs20 .config} syndrome). It will make it pretty database-agnostic (you can even not use a SQL database at all), and will make the framework code easier to debug and maintain, since we don't have to deal with all the DB engine particularities. In short, this is the @*REST@ point of view, and main cause of success: @*CRUD@ is enough in our KISS-friendly design.
:  Persist TSQLRecord, not any class
About the fact that you need to inherit from {\f1\fs20 @*TSQLRecord@}, and can't persist anything, our purpose was in fact very similar to the "@**Layer Supertype@" pattern of @*Domain-Driven@-Design, as explained by Martin Fowler:\line {\i It is not uncommon for all the objects in a layer to have methods you don't want to have duplicated throughout the system. You can move all of this behavior into a common Layer Supertype.}
In fact, for {\f1\fs20 TSQLRecord} / {\f1\fs20 TSQLRest} / @*ORM@ remote access, you have already all @*Client-Server@ @*CRUD@ operations available. Those classes are abstract common Supertypes, ready to be used in your projects. It has been optimized a lot (e.g. with a cache and other nice features), so I do not think reinventing a CRUD / database service is worth the prize. You have secure access to the ORM classes, with user/group attributes. Almost everything is created by code, just from the {\f1\fs20 TSQLRecord} class definition, via @*RTTI@. So it may be faster (and safer) to rely on it, than defining all your class hierarchy by hand.
Having a dedicated class help the layer separation, and therefore a cleaner design. See @68@ for more details about DDD and how {\i mORMot}'s {\f1\fs20 TSQLRecord} can help you reuse existing code, write less and safer.
:  Several ORMs at once
To be clear, {\i mORMot} offers three kind of table definitions:
- Via {\f1\fs20 @*TSQLRecord@} / {\f1\fs20 @*TSQLRecordVirtual@} "native ORM" classes: data storage is using either fast in-memory lists via {\f1\fs20 @*TSQLRestStorageInMemory@}, or {\i @*SQLite3@} tables (in memory, on file, or virtual). In this case, we do not use {\f1\fs20 index} for strings (column length is not used by any of those engines).
- Via {\f1\fs20 @*TSQLRecord@} "external ORM-managed" classes: after registration via a call to the {\f1\fs20 @*VirtualTableExternalRegister@()} function, external DB tables are created and managed by the ORM, via SQL - see @27@. These classes will allow creation of tables in any supported database engine - currently {\i SQLite3, @*Oracle@, @*Jet@/MSAccess, @*MS SQL@, @*Firebird@, @*DB2@, @*PostgreSQL@, @*MySQL@} and {\i @*NexusDB@} - via whatever {\i OleDB, ODBC} / ZDBC provider, or any {\f1\fs20 DB.pas} unit). For the "external ORM-managed" {\f1\fs20 TSQLRecord} type definitions, the ORM expects to find an {\f1\fs20 index} attribute for any text column length (i.e. {\f1\fs20 RawUTF8} or {\f1\fs20 string} published properties). This is the only needed parameter to be defined for such a basic implementation, in regard to {\f1\fs20 TSQLRecord} kind of classes.
- Via {\f1\fs20 @*TSQLRecordMappedAutoID@} / {\f1\fs20 @*TSQLRecordMappedForcedID@} "external mapped" classes: DB tables are not created by the ORM, but already existing in the DB, with sometimes a very complex layout. This feature is not yet implemented, but on the road-map. For this kind of classes we won't probably use attributes, nor even external files, but we will rely on definition from code, either with a fluent definition, or with dedicated classes (or interface).
The concern of not being able to persist any class (it needs to inherit from {\f1\fs20 TSQLRecord}) does perfectly make sense.
On the other hand, from the implementation point of view, it is very powerful, since you have a lot of methods included within this class definition. It does also make sense to have a common ancestor able to identify all three kind of {\i mORMot}'s table definitions: the same abstract ancestor is used, and clients won't even need to know that they are implemented in-memory, using a {\i SQLite3} engine, or any external database. Another benefit of using a parent {\f1\fs20 class} is to enforce code safety using Delphi's {\i @*strong type@} abilities: you won't be able to pass a non-persistent type to methods which expect one.
From the {\i @*Domain-Driven@} / @*SOA@ point of view, it is now an established rule to make a distinction between {\i Data Transfer Objects} (@*DTO@) and @*Domain Values@ ({\i Entity objects} or {\i Aggregates}). In most implementations, persistence objects (aka ORM objects) should be either the aggregate roots themselves (you do not store Entity objects and even worse DTOs), or dedicated classes. Do not mix layers, unless you like your software to be a maintenance nightmare!
Some @*Event Sourcing@ architectures even implement {\i several DB back-end at once}:
- It will store the status on one database (e.g. high-performance in-memory) for most common requests to be immediate;
................................................................................
- And even sometimes fill some dedicated consolidation DBs for further analysis.
AFAIK it could be possible to directly access ORM objects remotely (e.g. the consolidation DB), mostly in a read-only way, for dedicated reporting, e.g. from consolidated data - this is one potential @*CQRS@ implementation pattern with {\i mORMot.} Thanks to the framework security, remote access will be safe: your clients won't be able to change the consolidation DB content!
As can be easily guessed, such design models are far away from a basic ORM built only for class persistence.
:  The best ORM is the one you need
Therefore, we may sum up some potential use of ORM, depending of your intent:
- If your understanding of ORM is just to persist some {\i existing} objects with a lot of existing business code, {\i mORMot} won't help you directly, since it expects objects to inherit from {\f1\fs20 TSQLRecord} - but note that most ORMs, even those allowing to persist any {\f1\fs20 class}, would not be so easy to work with;
- If you want a very fast low-level Client-Server layer, {\i mORMot} is a first class candidate: we identified that some users are using the built-in JSON serialization and HTTP server features to create their application;
- If you want to persist some data objects (not tied to complex business logic), the framework's ORM will be a light and fast candidate, targetting {\i SQLite3, @*Oracle@, @*Jet@/MSAccess, @*MS SQL@, @*Firebird@, @*DB2@, @*PostgreSQL@, @*MySQL@, @*NexusDB@} databases, or even with no SQL engine, using {\f1\fs20 @*TSQLRestStorageInMemory@} class which is able to persist its content with small files - see @57@;
- If you need (perhaps not now, but probably in the future) to create some kind of scalable {\i @*Domain-Driven@} design architecture, you'll have all needed features at hand with {\i mORMot};
- If your expectation is to map an existing complex DB, {\i mORMot} will handle it soon (it is planned and prepared within the framework architecture).
Therefore, {\i mORMot} is not just an ORM, nor just a "classic" ORM.
About how to use {\i mORMot} with existing code, see @66@.
:MVC pattern
%cartoon04.png
: Creating a Model
According to the {\i @*Model@-View-Controller} (@*MVC@) pattern - see @10@ - the database schema should be handled separately from the User Interface.
The {\f1\fs20 @**TSQLModel@} class centralizes all {\f1\fs20 @*TSQLRecord@} inherited classes used by an application, both database-related and business-logic related.
In order to follow the @*MVC@ pattern, the {\f1\fs20 TSQLModel} instance is to be used when you have to deal at table level. For instance, do not try to use low-level {\f1\fs20 TSQLDataBase.GetTableNames} or {\f1\fs20 TSQLDataBase.GetFieldNames} methods in your code. In fact, the tables declared in the Model may not be available in the {\i @*SQLite3@} database schema, but may have been defined as {\f1\fs20 @*TSQLRestStorageInMemory@} instance via the {\f1\fs20 TSQLRestServer.StaticDataCreate} method, or being external tables - see @27@. You could even have a {\f1\fs20 mORMot} server running without any {\i @*SQLite3@} engine at all, but pure in-memory tables!
Each {\f1\fs20 TSQLModel} instance is in fact associated with a {\f1\fs20 TSQLRest} instance. An {\f1\fs20 Owner} property gives access to the current running client or server {\f1\fs20 @*TSQLRest@} instance associated with this model.
By design, models are used on both Client and Server sides. It is therefore a good practice to use a common unit to define all {\f1\fs20 TSQLRecord} types, and have a common function to create the related {\f1\fs20 TSQLModel} class.
For instance, here is the corresponding function as defined in the first samples available in the source code repository (unit {\f1\fs20 SampleData.pas}):
!function CreateSampleModel: TSQLModel;
!begin
!  result := TSQLModel.Create([TSQLSampleRecord]);
!end;
................................................................................
Difficult to find a faster ORM, I suspect.
:   Software and hardware configuration
The following tables try to sum up all available possibilities, and give some benchmark (average objects/second for writing or reading).
In these tables:
- 'SQLite3 (file full/off/exc)' indicates use of the internal {\i @*SQLite3@} engine, with or without {\f1\fs20 Synchronous := smOff} and/or {\f1\fs20 DB.LockingMode := lmExclusive} - see @60@;
- 'SQLite3 (mem)' stands for the internal {\i SQLite3} engine running in memory;
- 'SQLite3 (ext ...)' is about access to a {\i SQLite3} engine as external database - see @27@, either as file or memory;
- '{\f1\fs20 TObjectList}' indicates a {\f1\fs20 TSQLRestStorageInMemory} instance - see @57@ - either static (with no SQL support) or virtual (i.e. SQL featured via {\i SQLite3} virtual table mechanism) which may persist the data on disk as JSON or compressed binary;
- '@*NexusDB@' is the free embedded edition, available from official site;
- '@*Jet@' stands for a {\i MSAccess} database engine, accessed via OleDB.
- 'Oracle' shows the results of our direct OCI access layer ({\f1\fs20 SynDBOracle.pas});
- '@*Zeos@ *' indicates that the database was accessed directly via the @*ZDBC@ layer;
- 'FireDAC *' stands for {\i @*FireDAC@} library;
- 'UniDAC *' stands for {\i @*UniDAC@} library;
- 'BDE *' when using a {\i @*BDE@} connection;
................................................................................
|{\b ODBC MySQL}|10143|65538|82447
|{\b ZEOS MySQL}|2052|171803|245772
|{\b FireDAC MySQL}|3636|75081|105028
|{\b UniDAC MySQL}|4798|99940|146968
|%
The {\i @*SQLite3@} layer gives amazing reading results, which makes it a perfect fit for most typical ORM use. When running with {\f1\fs20 DB.LockingMode := lmExclusive} defined (i.e. "off exc" rows), reading speed is very high, and benefits from exclusive access to the database file - see @60@. External database access is only required when data is expected to be shared with other processes, or for better scaling: e.g. for physical n-Tier installation with dedicated database server(s).
In the above table, it appears that all libraries based on {\f1\fs20 DB.pas} are slower than the others for reading speed. In fact, {\f1\fs20 TDataSet} sounds to be a real bottleneck, due to its internal data marshalling. Even {\i @*FireDAC@}, which is known to be very optimized for speed, is limited by the {\f1\fs20 TDataSet} structure. Our direct classes, or even ZEOS/ZDBC performs better, since they are able to output JSON content with no additional marshalling.
For both writing and reading, {\f1\fs20 TObjectList} / {\f1\fs20 TSQLRestStorageInMemory} engine gives impressive results, but has the weakness of being in-memory, so it is not ACID by design, and the data has to fit in memory. Note that indexes are available for IDs and {\f1\fs20 stored AS_UNIQUE} properties.
As a consequence, search of non-unique values may be slow: the engine has to loop through all rows of data. But for unique values (defined as {\f1\fs20 stored AS_UNIQUE}), both insertion and search speed is awesome, due to its optimized O(1) hash algorithm - see the following benchmark, especially the "{\i By name}" row for "{\i TObjectList}" columns, which correspond to a search of an unique {\f1\fs20 RawUTF8} property value via this hashing method.
|%9%10%10%10%10%10%10%10%10%9%9
| |\b SQLite3 (file full)|SQLite3 (file off)|SQLite3 (mem)|TObjectList (static) |TObjectList (virt.)|SQLite3 (ext file full)|SQLite3 (ext file off)|SQLite3 (ext mem)|Oracle|Jet\b0
|{\b By one}|10461|10549|44737|103577|103553|43367|44099|45220|901|1074
|{\b By name}|9694|9651|32350|70534|60153|22785|22240|23055|889|1071
|{\b All Virt.}|167095|162956|168651|253292|118203|97083|90592|94688|56639|52764
|{\b All Direct}|167123|144250|168577|254284|256383|170794|165601|168856|88342|75999
................................................................................
\TSQLVirtualTableJSON\TSQLVirtualTable
\TSQLVirtualTableBinary\TSQLVirtualTableJSON
\TSQLVirtualTableLog\TSQLVirtualTable
\TSQLVirtualTableCursorIndex\TSQLVirtualTableCursor
\TSQLVirtualTableCursorJSON\TSQLVirtualTableCursorIndex
\TSQLVirtualTableCursorLog\TSQLVirtualTableCursorIndex
\
{\f1\fs20 @*TSQLVirtualTableJSON@, @*TSQLVirtualTableBinary@} and {\f1\fs20 TSQLVirtualTableCursorJSON} classes will implement a Virtual Table using a {\f1\fs20 @*TSQLRestStorageInMemory@} instance to handle fast in-memory @*static@ databases. Disk storage will be encoded either as UTF-8 @*JSON@ (for the {\f1\fs20 TSQLVirtualTableJSON} class, i.e. the '{\f1\fs20 JSON}' module), or in a proprietary @*SynLZ@ compressed format (for the {\f1\fs20 TSQLVirtualTableBinary} class, i.e. the '{\f1\fs20 Binary}' module). File extension on disk will be simply {\f1\fs20 .json} for the '{\f1\fs20 JSON}' module, and {\f1\fs20 .data} for the '{\f1\fs20 Binary}' module. Just to mention the size on disk difference, the 502 KB {\f1\fs20 People.json} content (as created by included regression tests) is stored into a 92 KB {\f1\fs20 People.data} file, in our proprietary optimized format.
Note that the virtual table module name is retrieved from the class name. For instance, the {\f1\fs20 TSQLVirtualTableJSON} class will have its module named as 'JSON' in the SQL code.
To handle external databases, two dedicated classes, named {\f1\fs20 TSQLVirtualTableExternal} and {\f1\fs20 TSQLVirtualTableCursorExternal} will be defined in a similar manner - see @%%HierExternalTables@ @30@.
As you probably have already stated, all those Virtual Table mechanism is implemented in {\f1\fs20 mORMot.pas}. Therefore, it is independent from the {\i @*SQLite3@} engine, even if, to my knowledge, there is no other SQL database engine around able to implement this pretty nice feature.
:  Defining a Virtual Table module
Here is how the {\f1\fs20 TSQLVirtualTableLog} class type is defined, which will implement a @*Virtual Table@ module named "{\f1\fs20 Log}". Adding a new module is just made by overriding some Delphi methods:
!  TSQLVirtualTableLog = class(TSQLVirtualTable)
!  protected
................................................................................
!    property DateTime: TDateTime read fDateTime;
!    /// the log event level
!    property Level: TSynLogInfo read fLevel;
!    /// the textual message associated to the log event
!    property Content: RawUTF8 read fContent;
!  end;
You could have overridden the {\f1\fs20 Structure} method in order to provide the {\f1\fs20 CREATE TABLE} SQL statement expected. But using Delphi class RTTI allows the construction of this SQL statement with the appropriate column type and @*collation@, common to what the rest of the @*ORM@ will expect.
Of course, this {\f1\fs20 RecordClass} property is not mandatory. For instance, the {\f1\fs20 TSQLVirtualTableJSON.GetTableModuleProperties} method won't return any associated {\f1\fs20 TSQLRecordClass}, since it will depend on the table it is implementing, i.e. the running {\f1\fs20 @*TSQLRestStorageInMemory@} instance. Instead, the {\f1\fs20 Structure} method is overridden, and will return the corresponding field layout of each associated table.
Here is how the {\f1\fs20 Prepare} method is implemented, and will handle the {\f1\fs20 vtWhereIDPrepared} feature:
!function TSQLVirtualTable.Prepare(var Prepared: TSQLVirtualTablePrepared): boolean;
!begin
!  result := Self<>nil;
!  if result then
!!    if (vtWhereIDPrepared in fModule.Features) and
!!       Prepared.IsWhereIDEquals(true) then
................................................................................
{\f1\fs20 TSQLRecordLogFile} was defined to map the column name as retrieved by the {\f1\fs20 TSQLVirtualTableLog} ('{\f1\fs20 log}') module, and should not to be used for any other purpose.
The Virtual Table module associated from such classes is retrieved from an association made to the server {\f1\fs20 @*TSQLModel@}. In a @*Client-Server@ application, the association is not needed (nor to be used, since it may increase code size) on the Client side. But on the server side, the {\f1\fs20 TSQLModel. VirtualTableRegister} method must be called to associate a {\f1\fs20 TSQLVirtualTableClass} (i.e. a Virtual Table module implementation) to a {\f1\fs20 TSQLRecordVirtualClass} (i.e. its ORM representation).
For instance, the following code will register two {\f1\fs20 TSQLRecord} classes, the first using the '{\f1\fs20 JSON}' virtual table module, the second using the '{\f1\fs20 Binary}' module:
!  Model.VirtualTableRegister(TSQLRecordDali1,TSQLVirtualTableJSON);
!  Model.VirtualTableRegister(TSQLRecordDali2,TSQLVirtualTableBinary);
This registration should be done on the Server side only, {\i before} calling {\f1\fs20 TSQLRestServer.Create} (or {\f1\fs20 TSQLRestClientDB.Create}, for a @*stand-alone@ application). Otherwise, an exception is raised at virtual table creation.
:57  In-Memory "static" process
We have seen that the {\f1\fs20 @*TSQLVirtualTableJSON@, @*TSQLVirtualTableBinary@} and {\f1\fs20 TSQLVirtualTableCursorJSON} classes implement a @*Virtual Table@ module using a {\f1\fs20 @**TSQLRestStorageInMemory@} instance to handle fast @**static@ in-memory database.
Why use such a database type, when you can create a {\i @*SQLite3@} in-memory table, using the {\f1\fs20 :memory:} file name? That is the question...
- {\i SQlite3} in-memory tables are not persistent, whereas our {\f1\fs20 JSON} or {\f1\fs20 Binary} virtual table modules can be written on disk on purpose, if the {\f1\fs20 aServer.StaticVirtualTable[aClass].CommitShouldNotUpdateFile} property is set to {\f1\fs20 true} - in this case, file writing should be made by calling explicitly the {\f1\fs20 aServer.StaticVirtualTable[aClass].UpdateToFile} method;
- {\i SQlite3} in-memory tables will need two database connections, or call to the {\f1\fs20 @*ATTACH DATABASE@} SQL statement - both of them are not handled natively by our @*Client-Server@ framework;
- {\i SQlite3} in-memory tables are only accessed via SQL statements, whereas {\f1\fs20 @*TSQLRestStorageInMemory@} tables can have faster direct access for most common @*REST@ful commands ({\f1\fs20 GET / POST / PUT / DELETE} individual rows) - this could make a difference in server CPU load, especially with the @*Batch@ feature of the framework;
- On the server side, it could be very convenient to have a direct list of in-memory {\f1\fs20 @*TSQLRecord@} instances to work with in pure Delphi code; this is exactly what {\f1\fs20 TSQLRestStorageInMemory} allows, and definitively makes sense for an @*ORM@ framework;
- On the client or server side, you could create calculated fields easily with {\f1\fs20 TSQLRestStorageInMemory} dedicated "getter" methods written in Delphi, whereas {\i SQlite3} in-memory tables would need additional SQL coding;
- {\i SQLite3} tables are stored in the main database file - in some cases, it could be much convenient to provide some additional table content in some separated database file (for a round robin table, a configuration table written in JSON, some content to be shared among users...): this is made possible using our {\f1\fs20 JSON} or {\f1\fs20 Binary} virtual table modules (but, to be honest, the {\f1\fs20 @*ATTACH DATABASE@} statement could provide a similar feature);
- The {\f1\fs20 TSQLRestStorageInMemory} class can be used stand-alone, i.e. without the {\i SQLite3} engine so it could be used to produce small efficient server software - see the "{\f1\fs20 SQLite3\\Samples\\01 - In Memory ORM}" folder.
:   In-Memory tables
A first way of using @*static@ tables, independently from the {\i SQLite3} engine, is to call the {\f1\fs20 TSQLRestServer. StaticDataCreate} method.
This method is only to be called server-side, of course. For the Client, there is no difference between a regular and a static table.
The in-memory {\f1\fs20 @*TSQLRestStorageInMemory@} instance handling the storage can be accessed later via the {\f1\fs20 StaticDataServer[]} property array of {\f1\fs20 TSQLRestServer}.
As we just stated, this primitive but efficient database engine can be used without need of the {\i SQLite3} database engine to be linked to the executable, saving some KB of code if necessary. It will be enough to handle most basic @*REST@ful requests.
:76   In-Memory virtual tables
A more advanced and powerful way of using @*static@ tables is to define some classes inheriting from {\f1\fs20 @*TSQLRecordVirtualTableAutoID@}, and associate them with some {\f1\fs20 TSQLVirtualTable} classes. The {\f1\fs20 TSQLRecordVirtualTableAutoID} parent class will specify that associated @*virtual table@ modules will behave like normal {\i SQLite3} tables, so will have their {\f1\fs20 RowID} property computed at {\f1\fs20 INSERT}).
For instance, the supplied regression tests define such two tables with three columns, named {\f1\fs20 FirstName}, {\f1\fs20 YearOfBirth} and {\f1\fs20 YearOfDeath}, after the @*published properties@ definition:
!  TSQLRecordDali1 = class(TSQLRecordVirtualTableAutoID)
!  private
!    fYearOfBirth: integer;
................................................................................
!    end;
!!    Check(aClient.TableRowCount(TSQLRecordDali1)=1001);
!!    aClient.Commit;
!  except
!!    aClient.RollBack;
!  end;
A {\f1\fs20 Commit} is needed from the Client side to write anything on disk. From the Server side, in order to create disk content, you'll have to explicitly call such code on purpose:
As we already noticed, data will be written by default on disk with our {\f1\fs20 @*TSQLRestStorageInMemory@}-based virtual tables. In fact, the {\f1\fs20 Commit} method in the above code will call {\f1\fs20 TSQLRestStorageInMemory.UpdateFile}.
Please note that the {\i @*SQlite3@} engine will handle any Virtual Table just like regular {\i SQLite3} tables, concerning the @*atomic@ity of the data. That is, if no explicit @*transaction@ is defined (via {\f1\fs20 TransactionBegin / Commit} methods), such a transaction will be performed for every database modification (i.e. all @*CRUD@ operations, as {\f1\fs20 INSERT / UPDATE / DELETE}). The {\f1\fs20 TSQLRestStorageInMemory. UpdateToFile} method is not immediate, because it will write all table data each time on disk. It is therefore mandatory, for performance reasons, to nest multiple modification to a Virtual Table with such a transaction, for better performance. And in all cases, it is the standard way of using the ORM. If for some reason, you later change your mind and e.g. move your table from the {\f1\fs20 TSQLVirtualTableJSON / TSQLVirtualTableBinary} engine to the default {\i SQlite3} engine, your code could remain untouched.
It is possible to force the In-Memory virtual table data to stay in memory, and the {\f1\fs20 COMMIT} statement to write nothing on disk, using the following property:
! Server.StaticVirtualTable[TSQLRecordDali1].CommitShouldNotUpdateFile := true;
In order to create disk content, you'll then have to explicitly call the corresponding method on purpose:
! Server.StaticVirtualTable[TSQLRecordDali1].UpdateToFile;
Since {\f1\fs20 StaticVirtualTable} property is only available on the Server side, you are the one to blame if your client updates the table data and this update never reaches the disk!
:   In-Memory and ACID
For data stored in memory, the {\f1\fs20 TSQLRestStorageInMemory} table is @*ACID@.\line It means that concurrent access will be consistent and work safely, as expected.
On disk, this kind of table is ACID only when its content is written to the file.\line I mean, the whole file which will be written in an ACID way. The file will always be consistent.
The exact process of these in-memory tables is that each time you write some new data to a {\f1\fs20 TSQLRestStorageInMemory} table:
- It will be ACID in memory (i.e. work safely in concurrent mode);
- Individual writes (INSERT/UPDATE/DELETE) won't automatically be written to file;
- COMMIT will by default write the whole table to file (either as JSON or compressed binary);
- COMMIT won't write the data to file if the {\f1\fs20 CommitShouldNotUpdateFile} property is set to TRUE;
- ROLLBACK process won't do anything, so won't be ACID - but since your code may later use a real RDBMS, it is a good habit to always write the command, like in the sample code above, as {\f1\fs20 except aClient.RollBack}.
When you write the data to file, the whole file is rewritten: it seems not feasible to write the data to disk at every write - in this case, SQLite3 in exclusive mode will be faster, since it will write only the new data, not the whole table content.
This may sound like a limitation, but on our eyes, it could be seen more like a feature. For a particular table, we do not need nor want to have a whole RDBMS/SQL engine, just direct and fast access to a {\f1\fs20 TObjectList}. The feature is to integrate it with our @*REST@ engine, and still be able to store your data in a regular database later ({\i SQLite3} or external), if it appears that {\f1\fs20 TSQLRestStorageInMemory} storage is too limited for your process.
:  Virtual Tables to access external databases
As will be stated @27@, some external databases may be accessed by our ORM.
The @*Virtual Table@ feature of {\i @*SQLite3@} will allow those remote tables to be accessed just like "native" {\i SQLite3} tables - in fact, you may be able e.g. to write a valid SQL query with a {\f1\fs20 @*JOIN@} between {\i SQlite3} tables, {\i @*MS SQL@ Server, @*MySQL@, @*FireBird@, @*PostgreSQL@, @*MySQL@, @*DB2@} and {\i @*Oracle@} databases, even with multiple connections and several remote servers. Think as an ORM-based {\i Business Intelligence} from any database source. Added to our code-based reporting engine (able to generate @*pdf@), it could be a very powerful way of consolidating any kind of data.
In order to define such {\i external} tables, you define your regular {\f1\fs20 @*TSQLRecord@} classes as usual, then a call to the {\f1\fs20 @**VirtualTableExternalRegister@()} function will define this class to be managed as a virtual table, from an external database engine. Using a dedicated external database server may allow better response time or additional features (like data sharing with other applications or languages). Server-side may omit a call to {\f1\fs20 VirtualTableExternalRegister()} if the need of an internal database is expected: it will allow custom database configuration at runtime, depending on the customer's expectations (or license).
:  Virtual tables from the client side
For external databases - see @27@ - the SQL conversion will be done on the fly in a more advanced way, so you should be able to work with such virtual tables from the client side without any specific model notification. In this case, you can safely define your tables as {\f1\fs20 TSQLValue1 = class(TSQLRecord)}, with no further code on client side.
When working with {\i static} (in-memory / {\f1\fs20 TObjectList}) storage, if you expect all ORM features to work remotely, you need to notify the Client-side model that a table is implemented as virtual. Otherwise you may encounter some SQL errors when executing requests, like "{\i no such column: ID}".
................................................................................
struct1:yob -> struct2:yob;
struct1:yod -> struct2:yod;
\
Note that only the {\f1\fs20 ID} and {\f1\fs20 YearOfDeath} column names were customized.
Due to the design of {\i SQLite3} virtual tables, and mORMot internals in its current state, the database primary key must be an {\f1\fs20 INTEGER} field to be mapped as expected by the ORM.
:30  External database ORM internals
The {\f1\fs20 mORMotDB.pas} unit implements @*Virtual Table@s access for any {\f1\fs20 @*SynDB@}-based external database for the framework.
In fact, the {\f1\fs20 TSQLRestStorageExternal, TSQLVirtualTableCursorExternal} and {\f1\fs20 TSQLVirtualTableExternal} classes will implement this feature:
\graph HierExternalTables External Databases classes hierarchy
\TSQLRecordVirtual\TSQLRecord
\TSQLRecordVirtualTableAutoID\TSQLRecordVirtual
\TSQLVirtualTableCursorExternal\TSQLVirtualTableCursor
\TSQLVirtualTableExternal\TSQLVirtualTable
\
The registration of the class is done by a call to the following new global procedure:
................................................................................
!procedure VirtualTableExternalRegister(aModel: TSQLModel; aClass: TSQLRecordClass;
!  aExternalDB: TSQLDBConnectionProperties; const aExternalTableName: RawUTF8);
This procedure will register on the Server-side an external database for an ORM class:
- It will define the supplied class to behave like a {\f1\fs20 TSQLRecordVirtualTableAutoID} class (i.e. its {\f1\fs20 @*TSQLModelRecordProperties@.Kind} property will be ovewritten to {\f1\fs20 rCustomAutoID} in this ORM model);
- It will associate the supplied class with a {\f1\fs20 TSQLVirtualTableExternal} module;
- The {\f1\fs20 TSQLDBConnectionProperties} instance should be shared by all classes, and released globally when the ORM is no longer needed;
- The full table name, as expected by the external database, should be provided here ({\f1\fs20 SQLTableName} will be used internally as table name when called via the associated {\i SQLite3} Virtual Table) - if no table name is specified (''), will use {\f1\fs20 SQLTableName} (e.g. 'Customer' for a class named {\f1\fs20 TSQLCustomer});
- Internal adjustments will be made to convert SQL on the fly from internal ORM representation into the expected external SQL format (e.g. table name or {\f1\fs20 ID} property) - see {\f1\fs20 TSQLRestStorage. AdaptSQLForEngineList} method.
Typical usage may be for instance:
!aProps := TOleDBMSSQLConnectionProperties.Create('.\SQLEXPRESS','AdventureWorks2008R2','','');
!aModel := TSQLModel.Create([TSQLCustomer],'root');
!VirtualTableExternalRegister(aModel,TSQLCustomer,aProps,'Sales.Customer');
!aServer := TSQLRestServerDB.Create(aModel,'application.db'),true)
All the rest of the code will use the "regular" ORM classes, methods and functions, as stated by @3@.
In order to be stored in an external database, the ORM records can inherit from any {\f1\fs20 TSQLRecord} class. Even if this class does not inherit from {\f1\fs20 TSQLRecordVirtualTableAutoID}, it will behave as such, once {\f1\fs20 @*VirtualTableExternalRegister@} function has been called for the given class.
As with any regular {\f1\fs20 TSQLRecord} classes, the ORM core will expect external tables to map an {\f1\fs20 Integer ID} published property, auto-incremented at every record insertion. Since not all databases handle such fields - e.g. {\i @*Oracle@} - auto-increment will be handled via a {\f1\fs20 select max(id) from tablename} statement run at initialization, then computed on the fly via a thread-safe cache of the latest inserted {\i RowID}.
You do not have to know where and how the data persistence is stored. The framework will do all the low-level DB work for you. And thanks to the {\i Virtual Table} feature of {\i SQlite3}, internal and external tables can be mixed within SQL statements. Depending on the implementation needs, classes could be persistent either via the internal {\i SQLite3} engine, or via external databases, just via a call to {\f1\fs20 VirtualTableExternalRegister()} before server initialization.
In fact, {\f1\fs20 TSQLVirtualTableCursorExternal} will convert any query on the external table into a proper optimized SQL query, according to the indexes existing on the external database. {\f1\fs20 TSQLVirtualTableExternal} will also convert individual SQL modification statements (like insert / update / delete) at the {\i SQLite3} level into remote SQL statements to the external database.
Most of the time, all @*REST@ful methods ({\f1\fs20 GET/POST/PUT/DELETE}) will be handled directly by the {\f1\fs20 TSQLRestStorageExternal} class, and won't use the virtual table mechanism. In practice, most access to the external database will be as fast as direct access, but the virtual table will always be ready to interpret any cross-database complex request or statement.
Direct REST access will be processed as following - when adding an object, for instance:
\graph TSQLRecordPeopleExtVirtualRest ORM Access Via REST
\TSQLRestServerDB.Add\TSQLRestServerDB.EngineAdd\internal川able
\TSQLRestServerDB.EngineAdd\TSQLRequest\INSERT INTO...
\TSQLRequest\SQlite3 engine\internal engine
\SQlite3 engine\SQLite3 file
\TSQLRestServerDB.Add\TSQLRestStorageExternal.EngineAdd\external川able卜EST
\TSQLRestStorageExternal.EngineAdd\ISQLDBStatement\INSERT INTO...
\ISQLDBStatement\External DB client\ODBC/ZDBC/OleDB...
\External DB client\External DB server
\
\page
Indirect access via virtual tables will be processed as following:
\graph TSQLRecordPeopleExtVirtualVirtual ORM Access Via Virtual Table
\TSQLRestServerDB.Add\TSQLRestServerDB.EngineAdd\internal or叉xternal table
................................................................................
\TSQLVirtualTableExternal.Insert\ISQLDBStatement\INSERT INTO...
\ISQLDBStatement\External DB client\ODBC/ZDBC/OleDB...
\External DB client\External DB server
\
About speed, here is an extract of the test regression log file (see code above, in previous paragraph), which shows the difference between RESTful call and virtual table call, working with more than 11,000 rows of data:
$  - External via REST: 133,666 assertions passed  409.82ms
$  - External via virtual table: 133,666 assertions passed  1.12s
The first run is made with {\f1\fs20 TSQLRestServer.StaticVirtualTableDirect} set to TRUE (which is the default) - i.e. it will call directly {\f1\fs20 TSQLRestStorageExternal} for RESTful commands, and the second will set this property to FALSE - i.e. it will call the {\i SQLite3} engine and let its virtual table mechanism convert it into another SQL calls.
It is worth saying that this test is using an in-memory {\i SQLite3} database (i.e. instantiated via {\f1\fs20 SQLITE_MEMORY_DATABASE_NAME} as pseudo-file name) as its external DB, so what we test here is mostly the ORM overhead, not the external database speed. With real file-based or remote databases (like @*MS SQL@), the overhead of remote connection won't make noticeable the use of Virtual Tables.
In all cases, letting the default {\f1\fs20 StaticVirtualTableDirect=true} will ensure the best possible performance. As stated by @59@, using a virtual or direct call won't affect the CRUD operation speed: it will by-pass the virtual engine whenever possible.
\page
:83 MongoDB database access
{\i @**MongoDB@} (from "humongous") is a cross-platform document-oriented database system, and certainly the best known @*NoSQL@ database.\line According to @http://db-engines.com in April 2014, {\i MongoDB} is in 5th place of the most popular types of database management systems, and first place for NoSQL database management systems.\line Our {\i mORMot} gives premium access to this database, featuring full @82@ abilities to the framework.
Integration is made at two levels:
- Direct low-level access to the {\i MongoDB} server, in the {\f1\fs20 SynMongoDB.pas} unit;
- Close integration with our ORM (which becomes {\i defacto} an ODM), in the {\f1\fs20 mORMotMongoDB.pas} unit.
{\i MongoDB} eschews the traditional table-based relational database structure in favor of @*JSON@-like documents with dynamic schemas ({\i MongoDB} calls the format @*BSON@), which matches perfectly {\i mORMot}'s @*REST@ful approach.
:  MongoDB client
The {\f1\fs20 SynMongoDB.pas} unit features direct optimized access to a {\i MongoDB} server.
................................................................................
And... that's all!
You can then use any ORM command, as usual:
!  writeln(Client.TableRowCount(TSQLORM)=0);
As with external databases, you can specify the field names mapping between the objects and the {\i MongoDB} collection.\line By default, the {\f1\fs20 TSQLRecord.ID} property is mapped to the {\i MongoDB}'s {\f1\fs20 _id} field, and the ORM will populate this {\f1\fs20 _id} field with a sequence of integer values, just like any {\f1\fs20 TSQLRecord} table.\line You can specify your own mapping, using for instance:
! aModel.Props[aClass].ExternalDB.MapField(..)
Since the field names are stored within the document itself, it may be a good idea to use shorter naming for the {\i MongoDB} collection. It may save some storage space, when working with a huge number of documents.
Once the {\f1\fs20 TSQLRecord} is mapped to a {\i MongoDB} collection, you can always have direct access to the {\f1\fs20 TMongoCollection} instance later on, by calling:
! (aServer.StaticDataServer[aClass] as TSQLRestStorageMongoDB).Collection
This may allow any specific task, including any tuned query or process.
:   ORM/ODM CRUD methods
You can add documents with the standard CRUD methods of the ORM, as usual:
!  R := TSQLORM.Create;
!  try
!    for i := 1 to COLL_COUNT do begin
!      R.Name := 'Name '+Int32ToUTF8(i);
................................................................................
|%
Note that you can have {\i several} protocols exposing the same {\f1\fs20 TSQLRestServer} instance. You may expose the same server over HTTP and over named pipes, at the same time, depending on your speed requirements.
: TSQLRest classes
This architecture is implemented by a hierarchy of classes, implementing the @*REST@ful pattern - see @9@ - for either stand-alone, client or server side, all inheriting from a {\f1\fs20 @*TSQLRest@} common ancestor, as two main branches:
\graph ClientServerRESTClasses RESTful Client-Server classes
\TSQLRestServer\TSQLRest
\TSQLRestClient\TSQLRest
\TSQLRestStorage\TSQLRest
\
All ORM operations (aka @*CRUD@ process) are available from the abstract {\f1\fs20 TSQLRest} class definition, which is overridden to implement either a Server (via {\f1\fs20 TSQLRestServer} classes), or a Client (via {\f1\fs20 TSQLRestClientURI} classes) access to the data.
You should instantiate the classes corresponding to the needed transmission protocol, but should better rely on abstraction, i.e. implement your whole code logic relying on abstract {\f1\fs20 TSQLRestClient / TSQLRestServer} classes. It will then help changing from one protocol or configuration at runtime, depending on your customer's expectations.
:  Server classes
The following classes are available to implement a {\i Server} instance:
\graph ServerRESTClasses RESTful Server classes
\TSQLRestServerDB\TSQLRestServer
\TSQLRestServerRemoteDB\TSQLRestServer
\TSQLRestServerFullMemory\TSQLRestServer
\TSQLRestServer\TSQLRest
\
In practice, in order to implement the business logic, you should better create a new {\f1\fs20 class}, inheriting from one of the above {\f1\fs20 TSQLRestServer} classes. Having your own inherited class does make sense, especially for implementing your own method-based services - see @49@, or {\f1\fs20 override} internal methods.
The {\f1\fs20 TSQLRestServerDB} class is the main kingn of Server of the framework. It will host a {\i @*SQLite3@} engine, as its core @42@.
If your purpose is not to have a full {\i SQLite3} engine available, you may create your server from a {\f1\fs20 @*TSQLRestServerFullMemory@} class instead of {\f1\fs20 TSQLRestServerDB}: this will implement a fast in-memory engine (using {\f1\fs20 TSQLRestStorageInMemory} instances), with basic CRUD features (for ORM), and persistence on disk as JSON or optimized binary files - this kind of server is enough to handle authentication, and host @*service@s in a stand-alone way.
If your services need to have access to a remote ORM server, it may use a {\f1\fs20 @*TSQLRestServerRemoteDB@} class instead: this server will use an internal {\f1\fs20 TSQLRestClient} instance to handle all ORM operations - it can be used e.g. to host some services on a stand-alone server, with all ORM and data access retrieved from another server: it will allow to easily implement a proxy architecture (for instance, as a DMZ for publishing services, but letting ORM process stay out of scope). See @75@ for some hosting scenarios.
:  Storage classes
In the {\i mORMot} units, you may also find those classes also inheriting from {\f1\fs20 TSQLRestStorage}:

\graph StorageRESTClasses RESTful storage classes
\TSQLRestStorageExternal\TSQLRestStorage
\TSQLRestStorageMongoDB\TSQLRestStorage
\TSQLRestStorageRecordBased\TSQLRestStorage
\TSQLRestStorageInMemory\TSQLRestStorageRecordBased
\TSQLRestStorageInMemoryExternal\TSQLRestStorageInMemory
\
In the above class hierarchy, the {\f1\fs20 TSQLRestStorage[InMemory][External]} classes are in fact used to store some {\f1\fs20 TSQLRecord} tables in any non-SQL backend:
- {\f1\fs20 TSQLRestStorageExternal} maps tables stored in an external database - see @27@;
- {\f1\fs20 TSQLRestStorageInMemory} stores the data in a {\f1\fs20 TObjectList} - see @57@;
- {\f1\fs20 TSQLRestStorageMongoDB} will connect to a remote {\i @*MongoDB@} server to store the tables as a @*NoSQL@ collection of documents - see @83@.
Those classes are used within a main {\f1\fs20 TSQLRestServer} to host some given {\f1\fs20 TSQLRecord} classes, either in-memory, or on external databases. They do not enter in account in our Client-Server presentation, but are implementation details, on the server side.
:  Client classes
A full set of {\i client} classes will implement a RESTful access to a remote database, with associated services and business logic:
\graph ClientRESTClasses RESTful Client classes
rankdir=LR;
\TSQLRestClientURINamedPipe\TSQLRestClientURI
\TSQLRestClientURIMessage\TSQLRestClientURI
\TSQLRestClientURIDll\TSQLRestClientURI
................................................................................
On typical production use, the {\i mORMot} HTTP server - see @6@ - will run on its own optimized thread pool, then call the {\f1\fs20 TSQLRestServer.URI} method. This method is therefore expected to be thread-safe, e.g. from the {\f1\fs20 TSQLHttpServer. Request} method. Thanks to the @*REST@ful approach of our framework, this method is the only one which is expected to be thread-safe, since it is the single entry point of the whole server. This KISS design ensure better test coverage.
On the Client side, all {\f1\fs20 TSQLRestClientURI} classes are protected by a global mutex (critical section), so are thread-safe. As a result, a single {\f1\fs20 TSQLHttpClient} instance can be shared among several threads, even if you may also use one client per thread, as is done with sample 21 - see below, for better responsiveness.
:  By design
We will now focus on the server side, which is the main strategic point (and potential bottleneck or point of failure) of any {\i Client-Server} architecture.
In order to achieve this thread-safety without sacrificing performance, the following rules were applied in {\f1\fs20 TSQLRestServer.URI}:
- Most of this method's logic is to process the URI and parameters of the incoming request (in {\f1\fs20 TSQLRestServerURIContext.URIDecode*} methods), so is thread-safe by design (e.g. {\f1\fs20 Model} and {\f1\fs20 RecordProps} access do not change during process);
- At @*REST@ful / @*CRUD@ level, {\f1\fs20 Add/Update/Delete/TransactionBegin/Commit/Rollback} methods are locked by default (with a 2 seconds timeout), and {\f1\fs20 Retrieve*} methods are not;
- {\f1\fs20 TSQLRestStorage} main methods ({\f1\fs20 EngineList, EngineRetrieve, EngineAdd, EngineUpdate, EngineDelete, EngineRetrieveBlob, EngineUpdateBlob}) are thread-safe: e.g. {\f1\fs20 @*TSQLRestStorageInMemory@} uses a per-Table Critical Section;
- {\f1\fs20 TSQLRestServerCallBack} method-based services - i.e. @*published method@s of the inherited {\f1\fs20 TSQLRestServer} class as stated @49@ - must be implemented to be thread-safe by default;
- {\f1\fs20 Interface}-based services - see @63@ - have several execution modes, including thread safe automated options (see {\f1\fs20 TServiceMethodOption}) or manual thread safety expectation, for better scaling - see @72@;
- A protected {\f1\fs20 fSessionCriticalSection} is used to protect shared {\f1\fs20 fSession[]} access between clients;
- The {\i @*SQLite3@} engine access is protected at SQL/JSON @*cache@ level, via {\f1\fs20 DB.LockJSON()} calls in {\f1\fs20 @*TSQLRestServerDB@} methods;
- Remote external tables - see @27@ - use thread-safe connections and statements when accessing the databases via SQL;
- Access to {\f1\fs20 fStats} was not made thread-safe, since this data is indicative only: a {\i mutex} was not used to protect this resource.
We tried to make the internal Critical Sections as short as possible, or relative to a table only (e.g. for {\f1\fs20 TSQLRestStorageInMemory}).
This default behavior can be tuned, using {\f1\fs20 TSQLRestServerURI.AcquireExecutionMode[]} property and {\f1\fs20 AcquireExecutionLockedTimeOut[]} when {\f1\fs20 amLocked} is set:
|%25%50%25
|\b Command|Description|Default\b0
|{\f1\fs20 execSOAByMethod}|for method-based services|{\f1\fs20 amUnlocked}
|{\f1\fs20 execSOAByInterface}|for interface-based services|{\f1\fs20 amUnlocked}
|{\f1\fs20 execORMGet}|for ORM reads i.e. {\i Retrieve*} methods|{\f1\fs20 amUnlocked}
|{\f1\fs20 execORMWrite}|for ORM writes i.e. {\i Add Update Delete TransactionBegin Commit Rollback} methods|{\f1\fs20 amLocked} +\line timeout of 2000 ms
................................................................................
\
In fact, the above function correspond to a database @*model@ with only external virtual tables, and with {\f1\fs20 StaticVirtualTableDirect=false}, i.e. calling the Virtual Table mechanism of {\i SQlite3} for each request.
But most of the time, i.e. for RESTful / @*CRUD@ commands, the execution is more direct:
\graph ArchVirtualDirect Client-Server implementation - Server side with "static" Virtual Tables
subgraph cluster {
\Http Server\TSQLRestServer.URI\dispatch
\TSQLRestServer.URI\TSQLRestServerDB.EngineAdd\Is a SQLite3 table?
\TSQLRestServer.URI\TSQLRestStorageExternal.EngineAdd\Is a static table?
\TSQLRestServerDB.EngineAdd\SQLite3 SQL\SQL insert
\SQLite3 SQL\SQLite3 engine\Is a SQLite3 table?
\SQLite3 engine\SQLite3 BTREE\prepare + execute
\SQLite3 BTREE\Database file\atomic write
\SQLite3 BTREE\TSQLRestServer.URI\return new ID
\TSQLRestStorageExternal.EngineAdd\TSQLDBConnectionProperties\external database
\TSQLDBConnectionProperties\TSQLDBStatement\compute SQL
\TSQLDBStatement\OleDB/ODBC or other\execute SQL
\OleDB/ODBC or other\Database Client\store data
\Database Client\OleDB/ODBC or other
\OleDB/ODBC or other\TSQLDBStatement
\TSQLDBStatement\TSQLRestStorageExternal.EngineAdd
\TSQLRestStorageExternal.EngineAdd\TSQLRestServer.URI\return new ID
\TSQLRestServer.URI\Http Server\return 200 OK + ID
label = "Server";
}
\
As stated in @27@, the @*static@ {\f1\fs20 TSQLRestStorageExternal} instance is called for most RESTful access. In practice, this design will induce no speed penalty, when compared to a direct database access. It could be even faster, if the server is located on the same computer than the database: in this case, use of JSON and REST could be faster - even faster when using @28@.
In order to be exhaustive, here is a more complete diagram, showing how native {\i @*SQLite3@}, in-memory or external tables are handled on the server side. You'll find out how CRUD statements are handled directly for better speed, whereas any SQL @*JOIN@ query can also be processed among all kind of tables.
\graph ArchServerFull Client-Server implementation - Server side
subgraph cluster {
\Http Server\TSQLRestServer.URI\dispatch
\TSQLRestServer.URI\TSQLRestServerDB.Engine*\Refers to兀 SQLite3 table or兀 JOINed query?
\TSQLRestServer.URI\TSQLRestStorage.九ngine*\CRUD over兀 static table?
\TSQLRestServerDB.Engine*\SQLite3 SQL\decode into SQL
\SQLite3 SQL\SQLite3 engine\prepare + execute
\SQLite3 engine\SQLite3 BTREE\For any又QLite3川able
\SQLite3 engine\TSQLRestServer.URI
\SQLite3 BTREE\Database file\atomic尸ead/write
\SQLite3 BTREE\SQLite3 engine
\SQLite3 engine\TSQLVirtualTableJSON三SQLVirtualTableBinary\For any in-memory丈irtual Table
................................................................................
\TSQLDBConnectionProperties\TSQLDBStatement\compute SQL
\TSQLDBStatement\OleDB/ODBC or other\execute SQL
\OleDB/ODBC or other\Database Client\handle data
\Database Client\OleDB/ODBC or other
\OleDB/ODBC or other\TSQLDBStatement
\TSQLDBStatement\SQLite3 engine
\TSQLRestServer.URI\Http Server\return 200 OK +尸esult (JSON)
\TSQLRestStorage.九ngine*\TSQLRestStorageExternal.九ngine*\Is an external丈irtual Table?
\TSQLRestStorage.九ngine*\TSQLRestStorageInMemory.九ngine*\Is an in-memory table?(Virtual or static)
\TSQLRestStorage.九ngine*\TSQLRestStorageMongoDB.九ngine*\Is a MongoDB table?
\TSQLRestStorageExternal.九ngine*\TSQLDBConnectionProperties三SQLDBStatement\compute SQL
\TSQLDBConnectionProperties三SQLDBStatement\OleDB/ODBC寸r other\execute SQL
\OleDB/ODBC寸r other\TSQLRestServer.URI
\OleDB/ODBC寸r other\Database七lient\handle data
\Database七lient\OleDB/ODBC寸r other
\TSQLRestStorageInMemory.九ngine*\CRUD夕n-memory三SQLRecord
\CRUD夕n-memory三SQLRecord\TSQLRestServer.URI
\TSQLVirtualTableJSON三SQLVirtualTableBinary\Process夕n-memory三SQLRecord
\Process夕n-memory三SQLRecord\SQLite3 engine
\TSQLRestStorageMongoDB.九ngine*\TMongoCollection刀ongoDB Client
\TMongoCollection刀ongoDB Client\TSQLRestServer.URI
label = "Server";
}
\
You will find out some speed numbers resulting from this unique architecture in the supplied @59@.
: Stateless design
:  Server side synchronization
................................................................................
!  end;
Just for fun... I could not resist posting this code here; if you are curious, take a look at the "official" {\f1\fs20 RTTI.pas} or {\f1\fs20 RIO.pas} units as provided by Embarcadero, and you will probably find out that the {\i mORMot} implementation is much easier to follow, and also faster (it does not recreate all the stubs or virtual tables for each wrapper, for instance). :)

[SDD-DI-2.2.1]
; SRS-DI-2.2.1 - The SQLite3 engine must be embedded to the framework
:Implementation
It's worth noting that the {\i Synopse mORMot framework}, whatever its previous name stated ("Synopse SQLite3 framework"), is not bound to {\i SQLite3}.
You can use another database engine for its internal data storage, for example we provide a {\f1\fs20 TSQLRestStorageInMemory} class which implements a fast but limited in-memory database engine.
Therefore, the {\i SQLite3} engine itself is not implemented in the @!TSQLRestServer!Lib\SQLite3\mORMot.pas@ unit, but in dedicated units.
The {\i SQLite3} engine is accessed at two levels:
- A low-level direct access to the {\i SQLite3} library, implemented in @!TSQLite3LibraryDynamic,TSQLite3Library,TSQLRequest.Execute,TSQLDataBase,TSQLTableDB.Create!Lib\SynSQLite3.pas@;
- A low-level statically linked library @!TSQLite3LibraryStatic!Lib\SynSQLite3Static.pas@, embedding the engine within the project executable;
- A possible use of external {\f1\fs20 sqlite3.dll} library, via the {\f1\fs20 TSQLite3LibraryDynamic} class;
- A high-level access, implementing a Client-Side or Server-Side native {\f1\fs20 TSQLRest} descendant using the {\i SQLite3} library for ORM data persistence, in @!TSQLRestServerDB,TSQLRestClientDB!Lib\SQLite3\mORMotSQLite3.pas@.
In addition to those two units, the @!TSQLDBSQLite3Connection,TSQLDBSQLite3Statement,TSQLDBSQLite3ConnectionProperties!Lib\SynDBSQLite3.pas@ unit publishes all features of the {\i SQlite3} database engine to its internal {\f1\fs20 SynDB} fast database access classes, which can be used uncoupled from the rest of the framework (i.e. without ORM).

Changes to SQLite3/Samples/01 - In Memory ORM/Project01.dpr.

2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
..
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
..
39
40
41
42
43
44
45
46
47
48
49
   Synopse mORMot framework

   Sample 01 - In Memory ORM
     purpose of this sample is to show the basic ORM usage of the framework:

   - a TRecord class is defined in Unit1.pas
   - a static server (i.e. in-memory database) is initialized (see
     TSQLRestServerStatic.Create below); 
     it will store the data in a JSON file in the disk and won't require
     the SQLite3 database engine
   - the purpose of the form in Unit1.pas is to add a record to the
     database; the Time field is filled with the current date and time
   - the 'Find a previous message' button show how to perform a basic query
   - on application quit, the Database.Destroy will update the JSON file
   - since the framework use UTF-8 encoding, we use some basic functions for
................................................................................
   - note the tiny size of the EXE (since we don't use SQLite3), less than
     80KB with LVCL :)

  Version 1.0 - January 24, 2010
    - Initial Release

  Version 1.1 - April 14, 2011
    - use TSQLRestServerStaticInMemory instead of abstract TSQLRestServerStatic

}

program Project01;

uses
  Forms,
................................................................................

{$R *.res}

begin
  Application.Initialize;
  Application.CreateForm(TForm1, Form1);
  Form1.Caption := ' Sample 01 - In Memory ORM';
  Form1.Database := TSQLRestServerStaticInMemory.Create(TSQLSampleRecord,nil,
    ChangeFileExt(paramstr(0),'.db'));
  Application.Run;
end.







|







 







|







 







|



2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
..
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
..
39
40
41
42
43
44
45
46
47
48
49
   Synopse mORMot framework

   Sample 01 - In Memory ORM
     purpose of this sample is to show the basic ORM usage of the framework:

   - a TRecord class is defined in Unit1.pas
   - a static server (i.e. in-memory database) is initialized (see
     TSQLRestStorage.Create below); 
     it will store the data in a JSON file in the disk and won't require
     the SQLite3 database engine
   - the purpose of the form in Unit1.pas is to add a record to the
     database; the Time field is filled with the current date and time
   - the 'Find a previous message' button show how to perform a basic query
   - on application quit, the Database.Destroy will update the JSON file
   - since the framework use UTF-8 encoding, we use some basic functions for
................................................................................
   - note the tiny size of the EXE (since we don't use SQLite3), less than
     80KB with LVCL :)

  Version 1.0 - January 24, 2010
    - Initial Release

  Version 1.1 - April 14, 2011
    - use TSQLRestStorageInMemory instead of abstract TSQLRestStorage

}

program Project01;

uses
  Forms,
................................................................................

{$R *.res}

begin
  Application.Initialize;
  Application.CreateForm(TForm1, Form1);
  Form1.Caption := ' Sample 01 - In Memory ORM';
  Form1.Database := TSQLRestStorageInMemory.Create(TSQLSampleRecord,nil,
    ChangeFileExt(paramstr(0),'.db'));
  Application.Run;
end.

Changes to SQLite3/Samples/01 - In Memory ORM/Unit1.pas.

13
14
15
16
17
18
19
20
21

22
23
24
25
26
27
28
29
30
31
32
33
34
35
36











37
38
39
40
41
42
43
..
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
..
75
76
77
78
79
80
81
82
83
84
85
86
87
    QuestionMemo: TMemo;
    NameEdit: TEdit;
    Label1: TLabel;
    Label2: TLabel;
    AddButton: TButton;
    QuitButton: TButton;
    FindButton: TButton;
    procedure AddButtonClick(Sender: TObject);
    procedure FormDestroy(Sender: TObject);

    procedure QuitButtonClick(Sender: TObject);
    procedure FindButtonClick(Sender: TObject);
    procedure FormCreate(Sender: TObject);
  private
  public
    Database: TSQLRest;
    Model: TSQLModel;
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}












procedure TForm1.AddButtonClick(Sender: TObject);
var Rec: TSQLSampleRecord;
begin
  Rec := TSQLSampleRecord.Create;
  try
    // we use explicit StringToUTF8() for conversion below
................................................................................
      NameEdit.SetFocus;
    end;
  finally
    Rec.Free;
  end;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
  Database.Free;
  Model.Free;
end;

procedure TForm1.QuitButtonClick(Sender: TObject);
begin
  Close;
end;

procedure TForm1.FindButtonClick(Sender: TObject);
var Rec: TSQLSampleRecord;
................................................................................
      QuestionMemo.Text := 'Not found' else
      QuestionMemo.Text := UTF8ToString(Rec.Question);
  finally
    Rec.Free;
  end;
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
  Model := CreateSampleModel; // from SampleData unit
end;

end.







|

>


<












>
>
>
>
>
>
>
>
>
>
>







 







<
<
<
<
<
<







 







<
<
<
<
<

13
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
..
62
63
64
65
66
67
68






69
70
71
72
73
74
75
..
80
81
82
83
84
85
86





87
    QuestionMemo: TMemo;
    NameEdit: TEdit;
    Label1: TLabel;
    Label2: TLabel;
    AddButton: TButton;
    QuitButton: TButton;
    FindButton: TButton;
    procedure FormCreate(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
    procedure AddButtonClick(Sender: TObject);
    procedure QuitButtonClick(Sender: TObject);
    procedure FindButtonClick(Sender: TObject);

  private
  public
    Database: TSQLRest;
    Model: TSQLModel;
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

procedure TForm1.FormCreate(Sender: TObject);
begin
  Model := CreateSampleModel; // from SampleData unit
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
  Database.Free;
  Model.Free;
end;

procedure TForm1.AddButtonClick(Sender: TObject);
var Rec: TSQLSampleRecord;
begin
  Rec := TSQLSampleRecord.Create;
  try
    // we use explicit StringToUTF8() for conversion below
................................................................................
      NameEdit.SetFocus;
    end;
  finally
    Rec.Free;
  end;
end;







procedure TForm1.QuitButtonClick(Sender: TObject);
begin
  Close;
end;

procedure TForm1.FindButtonClick(Sender: TObject);
var Rec: TSQLSampleRecord;
................................................................................
      QuestionMemo.Text := 'Not found' else
      QuestionMemo.Text := UTF8ToString(Rec.Question);
  finally
    Rec.Free;
  end;
end;






end.

Changes to SQLite3/Samples/12 - SynDB Explorer/SynDBExplorerMain.pas.

75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
...
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
begin // just not to be written in plain ascii in .config file
  SetLength(result,length(s));
  for i := 0 to length(s)-1 do
    PByteArray(result)[i] := PByteArray(s)[i] xor (i+137);
end;

procedure TDbExplorerMain.FormCreate(Sender: TObject);
var Conns: TSQLRestServerStaticInMemory;
function TryConnect(C: TSQLConnection; LoadTableNames: boolean): boolean;
const CONN_CLASSES: array[TExpConnectionType] of TSQLDBConnectionPropertiesClass =
  (TSQLDBOracleConnectionProperties,
   {$ifdef WIN64}
   nil,nil,nil,nil,
   {$else}
   TOleDBOracleConnectionProperties,TOleDBMSOracleConnectionProperties,
................................................................................
      C.Ident := S2U(FN);
      C.Server := C.Ident;
      TryConnect(C,True);
    finally
      C.Free;
    end;
  end else begin
    Conns := TSQLRestServerStaticInMemory.Create(
      TSQLConnection,nil,ChangeFileExt(paramstr(0),'.config'),false);
    try
      Conns.ExpandedJSON := true; // for better human reading and modification
      Task.Title := MainCaption;
      Task.Inst := sSelectAConnection;
      Task.Content := sSelectOrCreateAConnection;
      if Conns.Count=0 then







|







 







|







75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
...
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
begin // just not to be written in plain ascii in .config file
  SetLength(result,length(s));
  for i := 0 to length(s)-1 do
    PByteArray(result)[i] := PByteArray(s)[i] xor (i+137);
end;

procedure TDbExplorerMain.FormCreate(Sender: TObject);
var Conns: TSQLRestStorageInMemory;
function TryConnect(C: TSQLConnection; LoadTableNames: boolean): boolean;
const CONN_CLASSES: array[TExpConnectionType] of TSQLDBConnectionPropertiesClass =
  (TSQLDBOracleConnectionProperties,
   {$ifdef WIN64}
   nil,nil,nil,nil,
   {$else}
   TOleDBOracleConnectionProperties,TOleDBMSOracleConnectionProperties,
................................................................................
      C.Ident := S2U(FN);
      C.Server := C.Ident;
      TryConnect(C,True);
    finally
      C.Free;
    end;
  end else begin
    Conns := TSQLRestStorageInMemory.Create(
      TSQLConnection,nil,ChangeFileExt(paramstr(0),'.config'),false);
    try
      Conns.ExpandedJSON := true; // for better human reading and modification
      Task.Title := MainCaption;
      Task.Inst := sSelectAConnection;
      Task.Content := sSelectOrCreateAConnection;
      if Conns.Count=0 then

Changes to SQLite3/Samples/15 - External DB performance/PerfMain.pas.

678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
        end;
      finally
        Timer.Start;
        try
          {$ifdef USEMONGODB}
          if Server='MongoDB' then
            (Client.Server.StaticDataServer[TSQLRecordSample] as
              TSQLRestServerStaticMongoDB).Drop else
          {$endif}
          if not DBIsFile then
            Client.Server.EngineExecuteAll('drop table '+Value.SQLTableName);
        finally
          Client.Free;
        end;
        Stat.fClientCloseTime := Timer.Stop;







|







678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
        end;
      finally
        Timer.Start;
        try
          {$ifdef USEMONGODB}
          if Server='MongoDB' then
            (Client.Server.StaticDataServer[TSQLRecordSample] as
              TSQLRestStorageMongoDB).Drop else
          {$endif}
          if not DBIsFile then
            Client.Server.EngineExecuteAll('drop table '+Value.SQLTableName);
        finally
          Client.Free;
        end;
        Stat.fClientCloseTime := Timer.Stop;

Changes to SQLite3/Samples/24 - MongoDB/MongoDBTestCases.pas.

328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
...
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
...
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
  Check(fDB<>nil);
  Check(fDB.Name=DB_NAME);
  Check(fMongoClient.ServerBuildInfoNumber<>0);
  fModel := TSQLModel.Create([TSQLORM]);
  fClient := TSQLRestClientDB.Create(fModel,nil,':memory:',TSQLRestServerDB);
  Check(StaticMongoDBRegister(TSQLORM,fClient.Server,fDB,'mORMot')<>nil);
  fClient.Server.CreateMissingTables;
  (fClient.Server.StaticDataServer[TSQLORM] as TSQLRestServerStaticMongoDB).Drop;
  Check(fClient.TableRowCount(TSQLORM)=0);
  fStartTimeStamp := fClient.ServerTimeStamp;
  Check(fStartTimeStamp>10000);
  fClient.Server.NoAJAXJSON := true;
end;

procedure TTestORM.CleanUp;
................................................................................
      Check(fClient.Add(R,True)=i);
    end;
  finally
    R.Free;
  end;
  NotifyTestSpeed('rows inserted',COLL_COUNT,fMongoClient.BytesTransmitted-bytes);
  Check(fClient.TableRowCount(TSQLORM)=COLL_COUNT);
  (fClient.Server.StaticDataServer[TSQLORM] as TSQLRestServerStaticMongoDB).Drop;
end;

procedure TTestORM.InsertInBatchMode;
var R: TSQLORM;
    i: integer;
    bytes: Int64;
    IDs: TIntegerDynArray;
................................................................................
      TestOne(R,i);
    end;
    Check(n=ExpectedCount);
  finally
    R.Free;
  end;
  temp := fRunConsole;
  (fClient.Server.StaticDataServer[TSQLORM] as TSQLRestServerStaticMongoDB).Drop;
  fUpdateOffset := 0;
  InsertInBatchMode;
  fRunConsole := temp;
end;

procedure TTestORM.DeleteInBatchMode;
var i,n: integer;







|







 







|







 







|







328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
...
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
...
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
  Check(fDB<>nil);
  Check(fDB.Name=DB_NAME);
  Check(fMongoClient.ServerBuildInfoNumber<>0);
  fModel := TSQLModel.Create([TSQLORM]);
  fClient := TSQLRestClientDB.Create(fModel,nil,':memory:',TSQLRestServerDB);
  Check(StaticMongoDBRegister(TSQLORM,fClient.Server,fDB,'mORMot')<>nil);
  fClient.Server.CreateMissingTables;
  (fClient.Server.StaticDataServer[TSQLORM] as TSQLRestStorageMongoDB).Drop;
  Check(fClient.TableRowCount(TSQLORM)=0);
  fStartTimeStamp := fClient.ServerTimeStamp;
  Check(fStartTimeStamp>10000);
  fClient.Server.NoAJAXJSON := true;
end;

procedure TTestORM.CleanUp;
................................................................................
      Check(fClient.Add(R,True)=i);
    end;
  finally
    R.Free;
  end;
  NotifyTestSpeed('rows inserted',COLL_COUNT,fMongoClient.BytesTransmitted-bytes);
  Check(fClient.TableRowCount(TSQLORM)=COLL_COUNT);
  (fClient.Server.StaticDataServer[TSQLORM] as TSQLRestStorageMongoDB).Drop;
end;

procedure TTestORM.InsertInBatchMode;
var R: TSQLORM;
    i: integer;
    bytes: Int64;
    IDs: TIntegerDynArray;
................................................................................
      TestOne(R,i);
    end;
    Check(n=ExpectedCount);
  finally
    R.Free;
  end;
  temp := fRunConsole;
  (fClient.Server.StaticDataServer[TSQLORM] as TSQLRestStorageMongoDB).Drop;
  fUpdateOffset := 0;
  InsertInBatchMode;
  fRunConsole := temp;
end;

procedure TTestORM.DeleteInBatchMode;
var i,n: integer;

Changes to SQLite3/mORMot.pas.

679
680
681
682
683
684
685



686
687
688
689
690
691
692
...
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
...
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
...
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
....
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
....
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
....
3089
3090
3091
3092
3093
3094
3095
3096
3097
3098
3099
3100
3101
3102
3103
....
3185
3186
3187
3188
3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199
3200
3201
....
3624
3625
3626
3627
3628
3629
3630
3631

3632
3633
3634
3635
3636
3637
3638
3639
....
3673
3674
3675
3676
3677
3678
3679
3680
3681
3682
3683
3684
3685
3686
3687
....
8300
8301
8302
8303
8304
8305
8306
8307
8308
8309
8310
8311
8312
8313
8314
8315
8316
8317
8318
....
8323
8324
8325
8326
8327
8328
8329
8330
8331
8332
8333
8334
8335
8336
8337
....
8355
8356
8357
8358
8359
8360
8361
8362







8363
8364
8365
8366
8367
8368
8369
8370
8371
8372
8373
8374
8375
8376
8377
8378
8379
8380
8381
8382
8383
8384
8385
8386
8387
8388
8389
8390
8391
8392
8393
8394
8395
8396
8397
8398
8399
8400
8401
8402
8403
8404
8405
8406
8407
8408
8409
8410
8411
8412
8413
8414
8415
8416
8417
8418
8419
8420
8421
8422
....
8621
8622
8623
8624
8625
8626
8627
8628
8629
8630
8631
8632
8633
8634
8635
8636
8637
8638
8639
8640
8641
8642
8643
8644
8645
8646
8647
....
8664
8665
8666
8667
8668
8669
8670

8671
8672


8673
8674
8675
8676


8677
8678
8679
8680
8681
8682
8683
....
8686
8687
8688
8689
8690
8691
8692

8693
8694
8695
8696
8697
8698
8699
8700
....
8746
8747
8748
8749
8750
8751
8752
8753
8754
8755
8756
8757
8758
8759
8760
8761
8762
8763
8764
8765
8766
8767

8768
8769
8770
8771
8772
8773
8774
8775
8776
8777
8778
8779
8780
8781
8782

8783
8784
8785
8786
8787
8788
8789
8790
8791
....
9234
9235
9236
9237
9238
9239
9240
9241
9242
9243
9244
9245
9246
9247
9248
9249
....
9319
9320
9321
9322
9323
9324
9325
9326
9327
9328
9329
9330
9331
9332
9333
....
9664
9665
9666
9667
9668
9669
9670
9671
9672
9673
9674
9675
9676
9677
9678
9679
9680
9681
9682
9683
....
9691
9692
9693
9694
9695
9696
9697
9698
9699
9700
9701
9702
9703
9704
9705
9706
9707
9708
9709
9710
9711
9712
9713
9714
9715
9716
9717
9718
9719
9720
9721
9722
9723
9724
9725
9726
9727
9728
9729
9730
9731
....
9735
9736
9737
9738
9739
9740
9741
9742
9743
9744
9745
9746
9747
9748
9749


9750
9751
9752
9753
9754
9755
9756
9757








9758






9759
9760
9761
9762
9763
9764
9765
....
9787
9788
9789
9790
9791
9792
9793
9794
9795
9796
9797
9798
9799
9800
9801
9802
9803
9804
9805
9806
9807
9808
9809
9810
9811
9812
9813
9814
9815
9816
9817
9818
9819
9820
9821
9822
9823
....
9824
9825
9826
9827
9828
9829
9830









9831
9832
9833
9834
9835
9836
9837
9838
9839
9840
9841
9842
9843
9844
9845
9846
9847
9848
9849
9850
9851
9852
9853
9854
9855
9856
9857
9858
9859
9860

9861
9862
9863
9864
9865
9866
9867
9868
....
9946
9947
9948
9949
9950
9951
9952
9953
9954
9955
9956
9957
9958
9959
9960
9961
9962
9963
9964
9965
9966
9967
9968
9969
9970
9971
9972
9973
9974
9975
9976
9977
9978
9979
9980
9981
9982
9983
9984
9985
9986
9987
9988
9989
9990
9991
9992
.....
10022
10023
10024
10025
10026
10027
10028
10029
10030
10031
10032
10033
10034
10035
10036
.....
10123
10124
10125
10126
10127
10128
10129
10130
10131
10132
10133
10134
10135
10136
10137
10138
10139
10140
10141
10142
10143
10144
10145
10146
10147
10148
10149
10150
10151
10152
10153
10154
10155
10156
10157
10158
.....
10200
10201
10202
10203
10204
10205
10206
10207
10208
10209
10210
10211
10212
10213
10214
10215
10216
10217
10218
10219


10220
10221
10222
10223
10224
10225
10226
10227
10228
10229
10230
10231
10232
10233
10234
10235
10236
.....
10253
10254
10255
10256
10257
10258
10259
10260
10261
10262
10263
10264
10265
10266
10267
10268
10269


10270
10271
10272
10273
10274
10275
10276
10277
10278
10279
10280
10281
10282
10283
10284
10285
10286
10287
10288

10289
10290
10291
10292
10293
10294




10295
10296
10297
10298
10299
10300
10301
.....
10309
10310
10311
10312
10313
10314
10315
10316

10317
10318
10319
10320
10321
10322
10323
10324
10325
10326
10327
10328
10329
10330
10331
10332
10333
10334
10335
10336
10337
10338
10339
10340
10341
.....
10342
10343
10344
10345
10346
10347
10348
10349
10350
10351
10352
10353
10354
10355
10356
10357
10358
10359
10360
10361
10362
10363
10364
10365
10366
10367
10368
10369
10370
10371
10372
10373
10374
10375
10376
10377
.....
10407
10408
10409
10410
10411
10412
10413
10414
10415
10416
10417
10418
10419
10420
10421
10422
10423
10424
10425
10426
10427
10428
.....
10448
10449
10450
10451
10452
10453
10454
10455
10456
10457
10458
10459
10460
10461
10462
10463
10464
10465
10466
10467
10468
10469
.....
10493
10494
10495
10496
10497
10498
10499
10500
10501
10502
10503
10504
10505
10506
10507
.....
10517
10518
10519
10520
10521
10522
10523
10524
10525
10526
10527
10528
10529
10530
10531
10532
10533
10534
10535
10536
10537
10538
10539
10540
10541
10542
10543
10544
10545
10546
10547
10548
10549
10550
10551
10552
10553
10554
.....
10583
10584
10585
10586
10587
10588
10589
10590
10591
10592
10593
10594

10595
10596
10597
10598
10599
10600
10601
10602
10603
10604
10605
10606
10607
10608
10609
10610
10611
10612
10613
10614
10615
10616
10617
10618
10619
10620
10621
10622
10623
10624
10625
10626
10627
10628
10629
10630
10631
10632
10633
10634
10635
10636
10637
10638
10639
10640
10641
.....
10655
10656
10657
10658
10659
10660
10661
10662
10663
10664
10665
10666
10667
10668
10669
.....
10672
10673
10674
10675
10676
10677
10678
10679
10680
10681
10682
10683
10684
10685
10686
10687
10688
10689
10690
10691
10692
10693
10694
10695
.....
10737
10738
10739
10740
10741
10742
10743
10744
10745
10746
10747
10748
10749
10750
10751


10752
10753
10754
10755
10756
10757
10758
.....
10761
10762
10763
10764
10765
10766
10767
10768
10769
10770
10771
10772
10773
10774
10775
10776
10777
10778
10779
10780
10781
10782
10783
10784
10785
10786
10787
10788
10789
10790
10791
10792
10793
10794
10795
10796
10797
10798
10799
.....
10954
10955
10956
10957
10958
10959
10960
10961
10962
10963
10964
10965

10966
10967
10968
10969
10970
10971
10972
10973
10974
10975
10976
10977
10978
10979
10980
10981
10982
10983
10984
.....
11579
11580
11581
11582
11583
11584
11585
11586
11587
11588
11589
11590
11591
11592
11593
11594
11595
11596
11597
.....
11621
11622
11623
11624
11625
11626
11627
11628
11629
11630
11631
11632
11633
11634
11635
11636
11637
11638
11639
.....
11660
11661
11662
11663
11664
11665
11666
11667

11668
11669
11670
11671
11672
11673
11674
11675
11676
11677
11678
11679
11680
11681
11682
11683
.....
11739
11740
11741
11742
11743
11744
11745
11746
11747
11748
11749


11750
11751
11752
11753
11754
11755
11756
.....
11814
11815
11816
11817
11818
11819
11820
11821
11822
11823
11824
11825
11826
11827
11828
.....
11836
11837
11838
11839
11840
11841
11842
11843
11844
11845
11846
11847
11848
11849
11850
.....
11879
11880
11881
11882
11883
11884
11885
11886
11887
11888
11889
11890
11891
11892
11893
.....
21654
21655
21656
21657
21658
21659
21660
21661
21662
21663
21664
21665
21666
21667
21668
.....
21755
21756
21757
21758
21759
21760
21761
21762
21763
21764
21765
21766
21767
21768
21769
.....
21877
21878
21879
21880
21881
21882
21883
21884
21885
21886
21887
21888
21889
21890
21891
21892
21893
21894
21895
21896
21897
21898
21899
21900
.....
21907
21908
21909
21910
21911
21912
21913
21914
21915
21916
21917
21918
21919
21920
21921
.....
21941
21942
21943
21944
21945
21946
21947
21948
21949
21950
21951
21952
21953
21954
21955
.....
21961
21962
21963
21964
21965
21966
21967
21968
21969
21970
21971
21972
21973
21974
21975
21976
.....
21996
21997
21998
21999
22000
22001
22002





























22003
22004
22005
22006
22007
22008
22009
.....
22094
22095
22096
22097
22098
22099
22100

22101

22102



22103

22104
22105
22106
22107
22108
22109
22110
22111
22112
.....
22117
22118
22119
22120
22121
22122
22123
22124
22125
22126
22127
22128
22129
22130
22131
22132
22133
22134
22135


22136
22137

22138








22139
22140
22141
22142
22143
22144
22145
.....
22151
22152
22153
22154
22155
22156
22157


























22158
22159
22160
22161
22162
22163
22164
.....
22238
22239
22240
22241
22242
22243
22244
22245
22246
22247
22248
22249
22250
22251
22252
22253
.....
22276
22277
22278
22279
22280
22281
22282
22283




























22284
22285
22286
22287
22288
22289
22290
22291
22292

22293
22294
22295
22296
22297

22298
22299
22300
22301
22302
22303
22304
22305
22306
22307
22308
22309

22310
22311
22312
22313

22314
22315
22316
22317
22318
22319
22320
22321

22322
22323
22324
22325
22326
22327
22328
22329
.....
22330
22331
22332
22333
22334
22335
22336
22337
22338
22339
22340
22341
22342
22343
22344
22345
22346









22347
22348
22349
22350
22351
22352
22353
.....
22885
22886
22887
22888
22889
22890
22891
22892
22893
22894
22895
22896
22897
22898
22899
.....
22966
22967
22968
22969
22970
22971
22972
22973
22974
22975
22976
22977
22978
22979
22980
22981
22982
22983
22984
.....
23543
23544
23545
23546
23547
23548
23549
23550
23551
23552
23553
23554
23555

23556
23557
23558
23559
23560
23561
23562
23563
.....
23570
23571
23572
23573
23574
23575
23576
23577

23578

23579
23580
23581
23582
23583

23584

23585
23586
23587
23588
23589
23590
23591
23592
23593
23594
23595
23596
23597
23598
23599
23600
23601
23602
23603
23604
23605
23606
23607
23608
23609
23610

23611
23612
23613
23614
23615
23616
23617

23618
23619
23620
23621

23622
23623
23624
23625
23626
23627
23628
23629
23630
23631
23632
23633
23634
23635
23636

23637
23638
23639
23640
23641
23642
23643
23644

23645
23646
23647
23648
23649
23650
23651
23652
23653
.....
23934
23935
23936
23937
23938
23939
23940
23941
23942
23943
23944
23945
23946
23947
23948
23949
23950
23951
23952
23953
23954
23955
23956
23957
23958
23959
23960
23961
23962
23963
23964
23965
23966
23967
23968
23969
23970
23971
23972
23973
23974
.....
23980
23981
23982
23983
23984
23985
23986
23987
23988
23989
23990
23991
23992
23993
23994
23995
23996
23997
23998
23999
24000
24001
24002
24003
24004
24005
24006
24007
24008
24009
24010
24011
24012
24013
24014
24015
24016
24017
24018
24019
.....
24038
24039
24040
24041
24042
24043
24044
24045
24046
24047
24048
24049
24050
24051
24052
24053
24054
24055
24056
24057
24058
24059
24060
.....
24066
24067
24068
24069
24070
24071
24072
24073
24074
24075
24076
24077
24078
24079
24080
24081
24082
24083
24084
24085
24086
24087
24088
24089
24090
24091
24092
24093
24094
24095
24096
24097
24098
24099
24100
24101
24102
24103
24104
24105
24106
24107
24108
24109
24110
24111
24112
24113
24114
24115
24116
24117
24118
24119
24120
24121
24122
24123
24124
24125
24126
24127
24128
24129
24130
24131
24132
24133
24134
24135
24136
24137
24138
24139
24140
24141
24142
24143
24144
24145
24146
24147
24148
24149
24150
24151
24152
24153
24154
24155
24156
24157
24158
24159
24160
24161
24162
24163
24164
24165
24166
24167
24168
24169
24170
24171
24172
24173
24174
24175
24176
24177
24178
24179
24180
24181
24182
24183
24184
24185
24186
24187
24188
24189
24190
24191
24192
24193
24194
24195
24196
24197
24198
24199
24200
24201
24202
24203
24204
24205
24206
24207
24208
24209
24210
24211
24212
24213
24214
24215
24216
24217
24218
24219
24220
24221
24222
24223
24224
24225
24226
24227
24228
24229
24230
24231
24232
24233
24234
24235
24236
24237
24238
24239
24240
24241
24242
24243
24244
24245
24246
24247
24248
24249
24250
24251
24252
24253
24254
24255
24256
24257
24258
24259
24260
24261
24262
24263
24264
24265
24266
24267
24268
24269
24270
24271
24272
24273
24274
24275
24276
24277
24278
24279
24280
24281
24282
24283
24284
24285
24286
24287
24288

24289
24290
24291
24292
24293
24294
24295
24296
24297
24298
24299
24300
24301
24302
24303
24304
24305
24306
24307
.....
24309
24310
24311
24312
24313
24314
24315
24316
24317
24318
24319
24320
24321
24322
24323
.....
24334
24335
24336
24337
24338
24339
24340

24341

24342
24343
24344
24345
24346
24347
24348
24349
24350
24351
24352
24353
24354
24355
24356
24357
24358
24359
24360
24361
24362
24363
24364
24365
24366
24367
.....
24872
24873
24874
24875
24876
24877
24878
24879
24880
24881


24882
24883
24884
24885
24886
24887
24888
.....
24900
24901
24902
24903
24904
24905
24906

24907
24908
24909
24910
24911
24912
24913
24914
24915
24916
24917


24918
24919
24920
24921
24922
24923
24924
24925
.....
24954
24955
24956
24957
24958
24959
24960
24961
24962
24963
24964
24965
24966
24967
24968
.....
25042
25043
25044
25045
25046
25047
25048
25049
25050
25051
25052
25053
25054
25055
25056
.....
25065
25066
25067
25068
25069
25070
25071
25072
25073
25074
25075
25076
25077
25078
25079
25080
25081
25082
.....
25089
25090
25091
25092
25093
25094
25095

25096
25097
25098
25099
25100
25101
25102
25103
25104
25105
25106
25107
25108
25109
25110
25111
25112
25113
25114
25115
25116
25117
.....
25130
25131
25132
25133
25134
25135
25136
25137
25138
25139
25140
25141
25142
25143
25144
.....
25629
25630
25631
25632
25633
25634
25635
25636
25637
25638
25639
25640
25641

25642
25643
25644
25645
25646
25647
25648
25649
.....
25693
25694
25695
25696
25697
25698
25699

25700
25701
25702
25703
25704
25705
25706
25707
25708
25709
25710
25711
25712
25713
25714
25715
.....
25740
25741
25742
25743
25744
25745
25746
25747
25748
25749
25750
25751
25752
25753
25754
25755
25756
.....
25757
25758
25759
25760
25761
25762
25763
25764
25765
25766
25767
25768
25769
25770
25771
25772
25773
25774
25775
25776
25777
25778
25779
25780
25781
25782
25783
.....
25938
25939
25940
25941
25942
25943
25944
25945
25946
25947
25948
25949
25950
25951
25952
.....
26015
26016
26017
26018
26019
26020
26021



























































































26022
26023
26024
26025
26026
26027
26028
.....
26639
26640
26641
26642
26643
26644
26645
26646
26647
26648
26649
26650
26651
26652
26653
26654
26655
26656
26657
26658
26659
26660
26661
26662
26663
26664
26665
26666
26667
26668
26669
26670
26671
26672
26673
26674
26675

26676
26677
26678
26679
26680
26681
26682
26683
26684

26685
26686
26687
26688
26689
26690
26691
26692
26693
26694
26695
26696
26697
26698
26699
26700
26701
26702
26703
26704
26705
26706
26707
26708
26709
26710
26711
26712
26713
26714
26715
26716
26717
26718
26719
26720
26721
26722
26723
26724
26725
26726
26727
.....
26743
26744
26745
26746
26747
26748
26749
26750
26751
26752
26753
26754
26755
26756
26757
.....
26758
26759
26760
26761
26762
26763
26764
26765
26766
26767
26768
26769
26770
26771
26772
26773
26774
26775
26776
26777
26778
26779
26780
26781
26782
26783
26784
26785
26786
26787
26788
26789
26790
26791
26792
26793
26794
26795
26796
26797
26798
26799
26800
.....
26821
26822
26823
26824
26825
26826
26827
26828
26829
26830
26831
26832
26833
26834
26835
26836
26837
26838
26839
26840
26841
26842
26843
26844
26845
26846
26847
26848
26849
26850
26851
26852
26853
26854
26855
26856
26857
26858
26859
26860
26861
26862
26863
26864
26865
26866
26867
26868
26869
26870
26871
26872
26873
26874
26875
26876
26877
26878
26879
26880
26881
26882
26883
26884
26885
26886
26887
26888
26889
26890
26891
26892
26893
26894
26895
26896
26897
26898
26899
26900
26901
26902
26903
26904
26905
26906
26907
26908
26909
26910
26911
26912
26913
26914
26915
26916
26917
26918
26919
26920
26921
26922
26923
26924
26925
26926
26927
26928
26929
26930
26931
26932
26933
26934
26935
26936
26937
26938
26939
26940
26941
26942
26943
26944
26945
26946
26947
.....
26982
26983
26984
26985
26986
26987
26988
26989
26990
26991
26992
26993
26994
26995
26996
.....
27092
27093
27094
27095
27096
27097
27098
27099
27100
27101
27102
27103
27104
27105
27106
.....
27136
27137
27138
27139
27140
27141
27142
27143
27144
27145
27146
27147
27148
27149
27150
.....
27163
27164
27165
27166
27167
27168
27169
27170
27171
27172
27173
27174
27175
27176
27177
.....
27183
27184
27185
27186
27187
27188
27189
27190
27191
27192
27193
27194
27195
27196
27197
.....
27216
27217
27218
27219
27220
27221
27222
27223
27224
27225
27226
27227
27228
27229
27230
27231
27232
27233
27234
27235
27236
27237
27238
27239
27240
27241
27242
27243
27244
27245
27246
27247
27248
27249
27250
27251
27252
27253
.....
27278
27279
27280
27281
27282
27283
27284
27285
27286
27287
27288
27289
27290
27291
27292
27293
27294
27295
27296
27297
27298
.....
27300
27301
27302
27303
27304
27305
27306
27307
27308
27309
27310
27311
27312
27313
27314
27315
27316
27317
27318
27319
27320
27321
27322
27323
27324
27325
27326
27327
27328
27329
.....
27359
27360
27361
27362
27363
27364
27365
27366
27367
27368
27369
27370
27371
27372
27373
.....
27394
27395
27396
27397
27398
27399
27400
27401
27402
27403
27404
27405
27406
27407
27408
27409
27410
27411
27412
27413
27414
27415
27416
27417
27418
27419
27420
27421
27422
27423
27424
27425
27426
27427
27428
27429
27430
27431
27432
27433
27434
27435
27436
27437
27438
27439
27440
27441
27442
27443
27444
27445
27446
27447
27448
27449
27450
27451
27452
27453
27454
27455
27456
27457
27458
27459
.....
27463
27464
27465
27466
27467
27468
27469
27470
27471
27472
27473
27474
27475
27476
27477
27478
27479
27480
27481
27482
27483
27484
27485
27486
27487
27488
27489
27490
27491
27492
27493
27494
27495
27496
27497
27498
27499
27500
27501
27502
27503
27504
27505
27506
.....
27522
27523
27524
27525
27526
27527
27528
27529
27530
27531
27532
27533
27534

27535
27536
27537
27538
27539
27540
27541
27542
27543
27544
27545
27546
27547
27548
27549
27550
27551
27552
27553
27554
27555
27556
27557
27558
27559
27560
27561
27562
27563
27564
27565
27566
27567
27568
27569
27570
27571
27572
27573
27574
27575
27576

27577
27578
27579
27580
27581
27582
27583
27584
27585
27586
27587
27588
27589
27590
27591
27592
27593
27594
27595
27596
27597
27598
27599
27600
27601
27602
27603
27604
27605
27606
27607
27608
27609
27610
27611
27612
27613
27614
27615
27616
27617
27618
27619
27620
27621
27622
27623
27624
27625
27626
27627
27628
27629
27630
27631
27632
27633
27634
27635
27636
27637
27638
27639
27640
27641
27642

27643
27644
27645
27646
27647
27648
27649
27650
.....
27661
27662
27663
27664
27665
27666
27667
27668
27669
27670
27671
27672
27673
27674

27675
27676
27677
27678
27679
27680
27681
27682
.....
27687
27688
27689
27690
27691
27692
27693
27694
27695
27696
27697
27698
27699
27700
27701
27702
27703
27704
27705
27706
27707
27708
27709
27710
.....
27716
27717
27718
27719
27720
27721
27722
27723
27724
27725
27726
27727
27728
27729
27730
.....
27751
27752
27753
27754
27755
27756
27757
27758
27759
27760
27761
27762

27763
27764
27765
27766
27767
27768
27769
.....
27797
27798
27799
27800
27801
27802
27803
27804
27805
27806
27807
27808
27809
27810
27811

27812
27813
27814
27815
27816
27817
27818
27819
27820
27821
27822
27823
27824
27825
.....
27827
27828
27829
27830
27831
27832
27833







27834
27835
27836
27837
27838
27839
27840
27841
27842
27843
27844
27845
27846
27847
27848
27849
27850
27851
27852
27853
27854






27855
27856
27857
27858
27859
27860
27861
27862
27863
27864
27865
27866
27867
27868
27869
27870
27871
27872
27873
27874
27875
27876
27877
27878

27879






27880
27881
27882

27883
27884
27885

27886
27887
27888
27889
27890
27891
27892
27893
27894
27895
27896
27897
27898
27899
27900
27901
27902
27903
27904
27905
.....
27930
27931
27932
27933
27934
27935
27936
27937
27938
27939
27940
27941
27942
27943
27944
.....
27951
27952
27953
27954
27955
27956
27957
27958
27959
27960
27961
27962
27963
27964
27965
27966
27967
27968
27969
27970
27971
27972
27973
.....
27983
27984
27985
27986
27987
27988
27989
27990
27991
27992
27993
27994
27995
27996
27997
.....
27998
27999
28000
28001
28002
28003
28004
28005
28006
28007
28008
28009
28010
28011
28012
28013
28014
28015
28016
28017
28018
28019
28020
28021
28022
28023
28024
28025
28026
28027
28028
28029
28030
.....
28037
28038
28039
28040
28041
28042
28043
28044
28045
28046
28047
28048
28049
28050
28051
28052
28053
28054
28055
28056
28057
28058
28059
28060
28061
28062
28063
28064
28065
28066
28067
28068
28069
28070



28071
28072
28073
28074



28075
28076
28077
28078
28079
28080



28081
28082
28083
28084
28085
28086
28087




28088
28089
28090
28091
28092



28093
28094
28095
28096
28097
28098




28099
28100
28101
28102
28103
28104



28105
28106
28107
28108
28109
28110
28111
28112
.....
28119
28120
28121
28122
28123
28124
28125
28126
28127
28128
28129
28130
28131
28132
28133
28134
28135
28136
28137
28138
28139
28140
28141
28142
28143
28144
28145
28146
28147
28148
.....
28155
28156
28157
28158
28159
28160
28161
28162
28163
28164
28165
28166
28167
28168
28169
28170
28171
28172
28173
28174
28175
28176
28177
28178
28179
28180
28181
28182
28183
28184
28185
28186
28187
28188
28189
28190
28191
28192
28193
28194
28195
28196
28197
28198
28199
28200
28201
28202
28203
.....
28243
28244
28245
28246
28247
28248
28249
28250
28251
28252
28253
28254
28255


28256
28257
28258
28259
28260
28261
28262
28263
28264
28265
28266
28267
28268
28269
28270
28271
28272
28273
28274
28275
28276


28277
28278
28279

28280
28281
28282
28283
28284
28285
28286
28287
28288
28289
28290
28291
28292
28293
28294
.....
28301
28302
28303
28304
28305
28306
28307
28308
28309
28310
28311
28312
28313
28314
28315
.....
28316
28317
28318
28319
28320
28321
28322
28323
28324
28325
28326
28327
28328
28329
28330
28331
28332
28333
28334
28335
28336
28337
28338
28339
28340
28341
28342
28343
28344
28345
28346
28347
28348
28349
28350
28351
28352
28353
28354
28355
28356
28357
28358
28359
28360
28361
28362
28363
28364
28365
28366
28367
28368
28369
28370
28371
28372
28373
28374
28375
28376
28377
28378
28379
28380
28381
28382
28383
28384
28385
28386
.....
31262
31263
31264
31265
31266
31267
31268
31269
31270
31271
31272
31273
31274
31275
31276
31277
31278
31279
31280
31281
31282
31283
31284
31285
31286
31287
31288
31289
31290
31291
31292
31293
31294
.....
31498
31499
31500
31501
31502
31503
31504
31505
31506
31507
31508
31509
31510
31511
31512
.....
31513
31514
31515
31516
31517
31518
31519
31520
31521
31522
31523
31524
31525
31526
31527
.....
31531
31532
31533
31534
31535
31536
31537
31538
31539
31540
31541
31542
31543
31544
31545
.....
31548
31549
31550
31551
31552
31553
31554
31555
31556
31557
31558
31559
31560
31561
31562
.....
31573
31574
31575
31576
31577
31578
31579
31580
31581
31582
31583
31584
31585
31586
31587
.....
31589
31590
31591
31592
31593
31594
31595
31596
31597
31598
31599
31600
31601
31602
31603
31604
31605
31606
31607
31608
31609
31610
.....
31619
31620
31621
31622
31623
31624
31625
31626
31627
31628
31629
31630
31631
31632
31633
    - BREAKING CHANGE in TSQLRestServerCallBackParams which is replace by the
      TSQLRestServerURIContext class: in addition, all method-based services
      should be a procedure, and use Ctxt.Results()/Error() methods to return
      any content - new definition of Ctxt features now full access to
      incoming/outgoing context and parameters, especially via
      the new Input*[] properties, for easy URI parameter retrieval, and
      also allow define specific URI routing by a dedicated class



    - URI routing for interface-based service is now specified by the two
      TSQLRestRoutingREST and TSQLRestRoutingJSON_RPC classes (inheriting from
      the abstract TSQLRestServerURIContext class) instead of rmJSON and
      rmJSON_RPC enums - it allows any custom URI routing by inheritance
    - BREAKING CHANGE of TJSONWriter.WriteObject() method and ObjectToJSON()
      function: serialization is now defined with TTextWriterWriteObjectOptions
      set - therefore, TJSONSerializerCustomWriter callback signature changed
................................................................................
    - moved SQLFromSelectWhere() from a global function to a TSQLModel method
      (to prepare "Table per class hierarchy" mapping in mORMot)
    - SQLParamContent() / ExtractInlineParameters() functions moved to SynCommons
    - TSQLAuthUser and TSQLAuthGroup have now "index ..." attributes to their
      RawUTF8 properties, to allow direct handling in external databases
    - new protected TSQLRestServer.InternalAdaptSQL method, extracted from URI()
      process to also be called by TSQLRestServer.InternalListJSON() for proper
      TSQLRestServerStatic.AdaptSQLForEngineList(SQL) call
    - new TSQLRestServerStatic.fOutInternalStateForcedRefresh protected field to
      optionally force the refresh of the content
    - new TSQLRestServer.OnBlobUpdateEvent: TNotifyFieldSQLEvent event handler
      to implement feature request [4cafc41f67]
    - new protected TSQLRestServer.InternalUpdateEvent virtual method, to allow
      a server-wide update notification, not coupled to OnUpdateEvent callback -
      see feature request [5688e97251]
    - TSQLRestServerStaticInMemory.AdaptSQLForEngineList() will now handle
      'select count(*') from TableName' statements directly, and any RESTful
      requests from client
    - fixed issue in TSQLRestServerStaticInMemory.EngineList() when only ID
    - changed TSQLAccessRights and TSQLAuthGroup.SQLAccessRights CSV format
      to use 'first-last,' pattern to regroup set bits (reduce storage size)
    - added overloaded TSQLAccessRights.Edit() method using TSQLOccasions set
    - added reOneSessionPerUser kind of remote execution in TSQLAccessRight
    - introducing TSQLRestClientURI.InternalCheckOpen/InternalClose methods to
      properly handle remote connection and re-connection
    - added TSQLRestClientURI.LastErrorCode/LastErrorMessage/LastErrorException
................................................................................
    - introducing TSQLRecordInterfaced class, if your TSQLRecord definition
      should be able to implement interfaces
    - added optional CustomFields parameter to TSQLRestClientURI.BatchUpdate()
      and BatchAdd() methods
    - added TSQLRestClientURI.ServerTimeStampSynchronize method to force time
      synchronization with the server - can be handy to test the connection 
    - added TSQLRest.TableHasRows/TableRowCount methods, and overriden direct
      implementation for TSQLRestServer/TSQLRestServerStaticInMemory (including
      SQL pattern recognition for TSQLRestServerStaticInMemory)
    - added TSQLRest.RetrieveList method to retrieve a TObjectList of TSQLRecord
    - "rowCount": is added in TSQLRestServerStaticInMemory.GetJSONValues,
      TSQLTable.GetJSONValues and in TSQLTableJSON.ParseAndConvert, at the end
      of the non expanded JSON content, if needed - improves client performance
    - UpdateBlobFields() and RetrieveBlobFields() methods are now defined at
      TSQLRest level, with dedicated implementation for TSQLRestClient* and
      TSQLRestServer* classes - implements feature request [34664934a9]
    - fixed TSQLRestServerStaticInMemory.UpdateBlobFields() to return true
      if no BLOB field is defined (as with TSQLRestServer) - ticket [bfa13889d5]
    - fixed issue in TSQLRestServerStaticInMemory.GetJSONValues(), and handle
      optional LIMIT clause in this very same method
    - fixed unexpected issue in TSQLRestClientURI.BatchSend() when nothing is
      to be sent
    - fix potential GDI handle resource leak in TSQLRestClientURIMessage.Create
    - introducing TSQLRestClientURIMessage.DoNotProcessMessages property
    - TSQLRestClientURINamedPipe.InternalCheckOpen/InternalURI refactoring
    - added TInterfacedObjectWithCustomCreate kind of class, making easy to
................................................................................
    - TSQLRestServer.LaunchCallBack() is now inlined in TSQLRestServer.URI()
    - fixed ticket [a5e3564e48] about RecordRef typecast (and enhance comments)
    - fixed ticket [4f4dd18ad9] about TPropInfo.IsStored not handling methods
      callbacks, e.g. for TPersistent storage
    - fixed ticket [21c2d5ae96] when inserting/updating blob-only table content
    - fixed ticket [7e9f06bf1a] to let TSQLTable.FieldLengthMax() use caption
      text for enumeration columns
    - fixed ticket [28545a4ce0] about TSQLRestServerStaticInMemory.EngineDelete
      not thread-safe when run directly on server side
    - fixed ticket [027bb9678d] - now TSQLRecordRTree class works as expected
    - fixed ticket [876a097316] about TSQLRest.Add() when ForcedID<>0
    - implement ticket [e3f9742865] for enhanced JSON in woHumanReadable mode
    - fixed GPF issue in TServiceFactoryServer after instance time-out deletion
    - added TSQLPropInfo.PropertyIndex member
    - added TSQLRecordProperties.SimpleFieldsCount[] array
................................................................................
      {$ifndef NOVARIANTS}, sftVariant{$endif}];

type
  /// define how TJSONObjectDecoder.Decode() will handle JSON string values
  TJSONObjectDecoderParams = (pInlined, pQuoted, pNonQuoted);

  /// record/object helper to handle JSON object decoding
  // - used e.g. by GetJSONObjectAsSQL() function or TSQLRestServerStaticExternal
  // ExecuteFromJSON and InternalBatchStop methods
  TJSONObjectDecoder = {$ifdef UNICODE}record{$else}object{$endif}
    /// contains the decoded field names or value
    FieldNames, FieldValues: array[0..MAX_SQLFIELDS-1] of RawUTF8;
    /// Decode() will set each field type approximation
    // - will recognize also JSON_BASE64_MAGIC/JSON_SQLDATE_MAGIC prefix
    FieldTypeApproximation: array[0..MAX_SQLFIELDS-1] of
................................................................................
      Params: TJSONObjectDecoderParams; RowID: Integer=0; ReplaceRowIDWithID: Boolean=false); overload;
    /// encode as a SQL-ready INSERT or UPDATE statement
    // - after a successfull call to Decode()
    // - escape SQL strings, according to the official SQLite3 documentation
    // (i.e. ' inside a string is stored as '')
    // - if InlinedParams was TRUE, it will create prepared parameters like
    // 'COL1=:("VAL1"):, COL2=:(VAL2):'
    // - called by GetJSONObjectAsSQL() function or TSQLRestServerStaticExternal
    function EncodeAsSQL(Update: boolean): RawUTF8;
    /// encode as a SQL-ready INSERT or UPDATE statement with ? as values
    // - after a successfull call to Decode()
    // - FieldValues[] content will be ignored
    // - Occasion can be only soInsert or soUpdate
    // - for soInsert, it will create an INSERT with multiple VALUES if
    //  MultipleInsertCount>1, like 'INSERT ... VALUES (?,?), (?,?), ....'
................................................................................
{$M+} { we need the RTTI information to be compiled for the published
        properties of these classes and their children (like TPersistent),
        to enable ORM - must be defined at the forward definition level }
  TSQLRecord = class;      // published properties = ORM fields/columns
  TSQLRecordMany = class;
  TSQLAuthUser = class;
  TSQLRestServer = class;  // published methods = RESTful callbacks handlers
  TSQLRestServerStatic = class;
  TSQLRestClientURI = class;
{$M-}

  /// class-reference type (metaclass) of TSQLRecord
  TSQLRecordClass = class of TSQLRecord;

  PClass = ^TClass;
................................................................................
  // TSQLRecordMany) after registration for an external DB via a call to
  // VirtualTableExternalRegister() from mORMotDB unit
  TSQLRecordVirtualKind = (
    rSQLite3, rFTS3, rFTS4, rRTree, rCustomForcedID, rCustomAutoID);

  /// kind of (static) database server implementation available
  // - sMainEngine will identify the default main SQlite3 engine
  // - sStaticDataTable will identify a TSQLRestServerStaticInMemory - i.e.
  // TSQLRestServer.fStaticData[] which can work without SQLite3
  // - sVirtualTable will identify virtual TSQLRestServerStatic classes - i.e.
  // TSQLRestServer.fStaticVirtualTable[] which points to SQLite3 virtual tables
  // (e.g. TObjectList or external databases)
  TSQLRestServerKind = (sMainEngine, sStaticDataTable, sVirtualTable);
  /// pointer to the kind of (static) database server implementation
  PSQLRestServerKind = ^TSQLRestServerKind;

  /// some information about a given TSQLRecord class properties
................................................................................
    /// the Table as specified at the URI level (if any)
    Table: TSQLRecordClass;
    /// the index in the Model of the Table specified at the URI level (if any)
    TableIndex: integer;
    /// the RTTI properties of the Table specified at the URI level (if any)
    TableRecordProps: TSQLModelRecordProperties;
    /// the RESTful instance implementing the Table specified at the URI level (if any)
    // - equals Server most of the time, but may be an in-memory/virtual instance

    TableEngine: TSQLRestServer;
    /// the associated TSQLRecord.ID, as decoded from URI scheme
    // - this property will be set from incoming URI, even if RESTful
    // authentication is not enabled
    TableID: integer;
    /// the index of the callback published method within the internal class list
    MethodIndex: integer;
    /// the service identified by an interface-based URI
................................................................................
    /// the corresponding TAuthSession.User.ID value
    // - is undefined if Session is 0 or 1 (no authentication running)
    SessionUser: integer;
    /// the corresponding TAuthSession.User.GroupRights.ID value
    // - is undefined if Session is 0 or 1 (no authentication running)
    SessionGroup: integer;
    /// the static instance corresponding to the associated Table (if any)
    Static: TSQLRestServerStatic;
    /// the kind of static instance corresponding to the associated Table (if any)
    StaticKind: TSQLRestServerKind;
    /// optional error message which will be transmitted as JSON error (if set)
    CustomErrorMsg: RawUTF8;
    {$ifdef WITHLOG}
    /// associated logging instance
    // - you can use it to log some process on the server side
................................................................................
    end;
    /// log the corresponding text (if logging is enabled)
    procedure InternalLog(const Text: RawUTF8; Level: TSynLogInfo);
      {$ifdef HASINLINE}inline;{$endif}
    procedure SetRoutingClass(aServicesRouting: TSQLRestServerURIContextClass);
    /// retrieve a list of members as JSON encoded data - used by OneFieldValue()
    // and MultiFieldValue() public functions below
    // - overriden in TSQLRestClientURI and TSQLRestServer with proper method
    function InternalListJSON(Table: TSQLRecordClass; const SQL: RawUTF8): TSQLTableJSON;
      overload; virtual; abstract;
    /// retrieve a list of members as JSON encoded data - used by OneFieldValue()
    // and MultiFieldValue() public functions below
    // - call virtual abstract InternalListJSON() method to get the list content
    // - FieldName can be a CSV list of needed field names, if needed
    function InternalListJSON(Table: TSQLRecordClass; const FieldName, WhereClause: RawUTF8): TSQLTableJSON; overload;
    /// retrieve all fields for a list of members JSON encoded data
    // - this special method gets all fields content for a specified table:
    // the resulting TSQLTableJSON content can be used to fill whole records
    // instances by using the TSQLRecord.FillPrepare() and TSQLRecord.FillRow()
................................................................................
    // - this default implementation returns always true
    // - e.g. you can add digital signature to a record to disallow record editing
    // - the ErrorMsg can be set to a variable, which will contain an explicit
    // error message
    function RecordCanBeUpdated(Table: TSQLRecordClass; ID: integer; Action: TSQLEvent;
      ErrorMsg: PRawUTF8 = nil): boolean; virtual;
    /// internal method used by Delete(Table,SQLWhere) method
    function InternalDelete(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
      var IDs: TIntegerDynArray): boolean; 
    /// retrieve the server time stamp
    // - default implementation will use fServerTimeStampOffset to compute
    // the value from PC time (i.e. Now+fServerTimeStampOffset as TTimeLog)
    // - inherited classes may override this method, or set the appropriate
    // value in fServerTimeStampOffset protected field
    function GetServerTimeStamp: TTimeLog; virtual;
................................................................................
    // and direct JSON export, avoiding a TSQLTable which allocates memory for every
    // field values before the JSON export
    // - can be called for a single Table (ModelRoot/Table), or with low level SQL
    // query (ModelRoot + SQL sent as request body)
    // - if ReturnedRowCount points to an integer variable, it must be filled with
    // the number of row data returned (excluding field names)
    // - this method must be implemented in a thread-safe manner
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; virtual; abstract;







    /// create a new member (implements REST POST Collection)
    // - SentData can contain the JSON object with field values to be added
    // - class is taken from Model.Tables[TableModelIndex]
    // - returns the TSQLRecord ID/ROWID value, 0 on error
    // - is a "RowID":.. or "ID":.. member is set in SentData, it shall force its
    // value as insertion ID
    // - override this method for proper calling the database engine
    // - this method must be implemented in a thread-safe manner
    function EngineAdd(Table: TSQLRecordClass; const SentData: RawUTF8): integer; virtual; abstract;
    /// update a member (implements REST PUT Member)
    // - SentData can contain the JSON object with field values to be added
    // - returns true on success
    // - override this method for proper calling the database engine
    // - this method must be implemented in a thread-safe manner
    function EngineUpdate(Table: TSQLRecordClass; ID: integer; const SentData: RawUTF8): boolean; virtual; abstract;
    /// delete a member (implements REST DELETE Member)
    // - returns true on success
    // - override this method for proper calling the database engine
    // - this method must be implemented in a thread-safe manner
    function EngineDelete(Table: TSQLRecordClass; ID: integer): boolean; virtual; abstract;
    /// delete several members, from a WHERE clause
    // - IDs[] contains the already-computed matching IDs for SQLWhere
    // - returns true on success
    // - override this method for proper calling the database engine, i.e.
    // using either IDs[] or a faster SQL statement
    // - this method must be implemented in a thread-safe manner
    function EngineDeleteWhere(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; virtual; abstract;
    /// get a blob field content from its member ID and field name
    // - implements REST GET member with a supplied blob field name
    // - returns TRUE on success
    // - returns the data of this blob as raw binary (not JSON) in BlobData
    // - override this method for proper data retrieval from the database engine
    // - this method must be implemented in a thread-safe manner
    function EngineRetrieveBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; virtual; abstract;
    /// update a blob field content from its member ID and field name
    // - implements REST PUT member with a supplied blob field name
    // - returns TRUE on success
    // - the data of this blob must be specified as raw binary (not JSON) in BlobData
    // - override this method for proper data retrieval from the database engine
    // - this method must be implemented in a thread-safe manner
    function EngineUpdateBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; virtual; abstract;
    /// update an individual record field value from a specified ID or Value
    // - return true on success
    // - will allow execution of requests like
    // $ UPDATE tablename SET setfieldname=setvalue WHERE wherefieldname=wherevalue
    // - SetValue and WhereValue parameters must match our inline format, i.e.
    // by double quoted with " for strings, or be plain text for numbers - e.g.
    // $ Client.EngineUpdateField(TSQLMyRecord,'FirstName','"Smith"','RowID','10')
    // - this method must be implemented in a thread-safe manner
    function EngineUpdateField(Table: TSQLRecordClass;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; virtual; abstract;
  public
    /// initialize the class, and associate it to a specified database Model
    constructor Create(aModel: TSQLModel); virtual;
    /// release internal used instances
    // - e.g. release associated TSQLModel or TServiceContainer
    destructor Destroy; override;
................................................................................
      Value: TSQLRecord): boolean; overload;
    /// get a member from its ID
    // - return true on success
    // - Execute 'SELECT * FROM TableName WHERE ID=:(aID): LIMIT 1' SQL Statememt
    // - if ForUpdate is true, the REST method is LOCK and not GET: it tries to lock
    // the corresponding record, then retrieve its content; caller has to call
    // UnLock() method after Value usage, to release the record
    // - this method is defined as abstract, i.e. there is no default implementation -
    // it must be implemented 100% RestFul with a GET ModelRoot/TableName/TableID
    // and handle the LOCK command if necessary: real RESTful class
    // should implement a GET member from URI in an overriden method
    // - the TSQLRawBlob (BLOB) fields are not retrieved by this method, to
    // preserve bandwidth: use the RetrieveBlob() methods for handling
    // BLOB fields, or set either the TSQLRestClientURI.ForceBlobTransfert
    // or TSQLRestClientURI.ForceBlobTransfertTable[] properties
    // - the TSQLRecordMany fields are not retrieved either: they are separate
    // instances created by TSQLRecordMany.Create, with dedicated methods to
    // access to the separated pivot table
   function Retrieve(aID: integer; Value: TSQLRecord;
      ForUpdate: boolean=false): boolean; overload; virtual; abstract;
    /// get a member from its TRecordReference property content
    // - instead of the other Retrieve() methods, this implementation Create an
    // instance, with the appropriated class stored in Reference
    // - returns nil on any error (invalid Reference e.g.)
    // - if ForUpdate is true, the REST method is LOCK and not GET: it tries to lock
    // the corresponding record, then retrieve its content; caller has to call
    // UnLock() method after Value usage, to release the record
................................................................................
    // responsible of freeing the instance
    // - this TObjectList will contain a list of all matching records
    // - return nil on error
    function RetrieveList(Table: TSQLRecordClass; FormatSQLWhere: PUTF8Char;
      const BoundsSQLWhere: array of const; const aCustomFieldsCSV: RawUTF8=''): TObjectList;
    /// Execute directly a SQL statement, expecting a list of results
    // - return a result table on success, nil on failure

    function ExecuteList(const Tables: array of TSQLRecordClass; const SQL: RawUTF8): TSQLTableJSON; virtual; abstract;
    /// unlock the corresponding record


    // - use our custom UNLOCK REST-like method
    // - returns true on success
    function UnLock(Table: TSQLRecordClass; aID: integer): boolean; overload; virtual; abstract;
    /// unlock the corresponding record


    // - use our custom UNLOCK REST-like method
    // - calls internally UnLock() above
    // - returns true on success
    function UnLock(Rec: TSQLRecord): boolean; overload;
    /// create a new member (implements REST POST Collection)
    // - if SendData is true, client sends the current content of Value with the
    // request, otherwize record is created with default values
................................................................................
    // - on success, returns the new ROWID value; on error, returns 0
    // - on success, Value.ID is updated with the new ROWID
    // - the TSQLRawBlob(BLOB) fields values are not set by this method, to
    // preserve bandwidth
    // - the TSQLRecordMany fields are not set either: they are separate
    // instances created by TSQLRecordMany.Create, with dedicated methods to
    // access to the separated pivot table

    function Add(Value: TSQLRecord; SendData: boolean; ForceID: boolean=false): integer; overload; virtual; abstract;
    /// create a new member, from a supplied list of field values
    // - the aSimpleFields parameters must follow explicitely the order of published
    // properties of the supplied aTable class, excepting the TSQLRawBlob and
    // TSQLRecordMany kind (i.e. only so called "simple fields")
    // - the aSimpleFields must have exactly the same count of parameters as
    // there are "simple fields" in the published properties
    // - if ForcedID is set to non null, client sends this ID to be used
................................................................................

    /// access the internal caching parameters for a given TSQLRecord
    // - purpose of this caching mechanism is to speed up retrieval of some
    // common values at either Client or Server level (like configuration settings)
    // - only caching synchronization is about the direct RESTful/CRUD commands:
    // RETRIEVE, ADD, UPDATE and DELETE (that is, a complex direct SQL UPDATE or
    // via TSQLRecordMany pattern won't be taken in account - only exception is
    // TSQLRestServerStatic tables accessed as SQLite3 virtual table)
    // - this caching will be located at the TSQLRest level, that is no automated
    // synchronization is implemented between TSQLRestClient and TSQLRestServer:
    // you shall ensure that your code won't fail due to this restriction
    // - use Cache.SetCache() and Cache.SetTimeOut() methods to set the appropriate
    // configuration for this particular TSQLRest instance 
    property Cache: TSQLRestCache read GetCache;

    /// get a blob field content from its record ID and supplied blob field name
    // - implements REST GET member with a supplied member ID and a blob field name
    // - return true on success
    // - this method is defined as abstract, i.e. there is no default implementation:
    // it must be implemented 100% RestFul with a
    // GET ModelRoot/TableName/TableID/BlobFieldName request for example
    // - this method retrieve the blob data as a TSQLRawBlob string

    function RetrieveBlob(Table: TSQLRecordClass; aID: integer;
      const BlobFieldName: RawUTF8; out BlobData: TSQLRawBlob): boolean; overload; virtual; abstract;
    /// get a blob field content from its record ID and supplied blob field name
    // - implements REST GET member with a supplied member ID and a blob field name
    // - return true on success
    // - this method will create a TStream instance (which must be freed by the
    // caller after use) and fill it with the blob data
    function RetrieveBlob(Table: TSQLRecordClass; aID: integer;
      const BlobFieldName: RawUTF8; out BlobStream: THeapMemoryStream): boolean; overload;
    /// update a blob field from its record ID and supplied blob field name
    // - implements REST PUT member with a supplied member ID and field name
    // - return true on success
    // - this default method call RecordCanBeUpdated() to check if the action is
    // allowed
    // - this method expect the Blob data to be supplied as TSQLRawBlob

    function UpdateBlob(Table: TSQLRecordClass; aID: integer;
      const BlobFieldName: RawUTF8; const BlobData: TSQLRawBlob): boolean; overload; virtual; abstract;
    /// update a blob field from its record ID and blob field name
    // - implements REST PUT member with a supplied member ID and field name
    // - return true on success
    // - this default method call RecordCanBeUpdated() to check if the action is
    // allowed
    // - this method expect the Blob data to be supplied as a TStream: it will
    // send the whole stream content (from its beginning position upto its
................................................................................
    // - use the TSQLAuthGroup.AccessRights CSV format
    function ToString: RawUTF8;
    /// unserialize the content from TEXT
    // - use the TSQLAuthGroup.AccessRights CSV format
    procedure FromString(P: PUTF8Char);
  end;

  TSQLRestServerStaticClass = class of TSQLRestServerStatic;
  TSQLRestServerStaticInMemory = class;
  TSQLVirtualTableModule = class;


  {/ table containing the available user access rights for authentication
    - this class should be added to the TSQLModel, together with TSQLAuthUser,
      to allow authentication support
    - you can inherit from it to add your custom properties to each user info:
................................................................................
    /// able to set the PasswordHashHexa field from a plain password content
    // - in fact, PasswordHashHexa := SHA256('salt'+PasswordPlain) in UTF-8
    property PasswordPlain: RawUTF8 write SetPasswordPlain;
  published
    /// the User identification Name, as entered at log-in
    // - the same identifier can be used only once (this column is marked as
    // unique via a "stored AS_UNIQUE" - i.e. "stored false" - attribute), and
    // therefore indexed in the database (e.g. hashed in TSQLRestServerStaticInMemory)
    property LogonName: RawUTF8 index 20 read fLogonName write fLogonName stored AS_UNIQUE;
    /// the User Name, as may be displayed or printed
    property DisplayName: RawUTF8 index 50 read fDisplayName write fDisplayName;
    /// the hexa encoded associated SHA-256 hash of the password
    property PasswordHashHexa: RawUTF8 index 64 read fPasswordHashHexa write fPasswordHashHexa;
    /// the associated access rights of this user
    // - access rights are managed by group
................................................................................
    fSQLAuthGroupClass: TSQLAuthGroupClass;
    /// how in-memory sessions are handled
    fSessionClass: TAuthSessionClass;
    /// will contain the in-memory representation of some static tables
    // - this array has the same length as the associated Model.Tables[]
    // - fStaticData[] will contain pure in-memory tables, not declared as
    // SQLite3 virtual tables, therefore not available from joined SQL statements
    fStaticData: array of TSQLRestServerStatic;
    /// map TSQLRestServerStaticInMemory or TSQLRestServerStaticExternal engines
    // - this array has the same length as the associated Model.Tables[]
    // - fStaticVirtualTable[] will contain in-memory or external tables declared
    // as SQLite3 virtual tables, therefore available from joined SQL statements
    fStaticVirtualTable: array of TSQLRestServerStatic;
    /// in-memory storage of TAuthSession instances
    fSessions: TObjectList;
    /// used to compute genuine TAuthSession.ID cardinal value
    fSessionCounter: cardinal;
    /// mutex used to make fSessions[] use thread-safe
    fSessionCriticalSection: TRTLCriticalSection;
    fSessionAuthentications: IObjectDynArray; // must be defined before the array
................................................................................
    // - use "string" type, i.e. UnicodeString for Delphi 2009+, in order
    // to call directly the correct FindWindow?()=FindWindow Win32 API
    fServerWindowName: string;
{$endif}
    fPublishedMethod: TSQLRestServerMethods;
    fPublishedMethods: TDynArrayHashed;
    /// fast get the associated static server, if any
    function GetStaticDataServer(aClass: TSQLRecordClass): TSQLRestServerStatic;
    /// retrieve a TSQLRestServerStatic instance associated to a Virtual Table
    // - is e.g. TSQLRestServerStaticInMemory instance associated to a
    // TSQLVirtualTableBinary or TSQLVirtualTableJSON class
    // - may be a TSQLRestServerStaticExternal (as defined in mORMotDB unit)
    // for a virtual table giving access to an external database
    function GetVirtualTable(aClass: TSQLRecordClass): TSQLRestServerStatic;
    /// fast get the associated static server or Virtual table, if any
    // - this can be used to call directly the TSQLRestServerStatic instance
    // on the server side
    // - same as a dual call to StaticDataServer[aClass] + StaticVirtualTable[aClass]
    // - TSQLRestServer.URI will make a difference between the a static server
    // or a TSQLVirtualTable, but this method won't - you can set a reference
    // to a TSQLRestServerKind variable to retrieve the database server type
    function GetStaticDataServerOrVirtualTable(aClass: TSQLRecordClass;
      Kind: PSQLRestServerKind=nil): TSQLRestServerStatic; overload;
    /// overloaded method using table index in associated Model
    function GetStaticDataServerOrVirtualTable(aTableIndex: integer;
      Kind: PSQLRestServerKind=nil): TSQLRestServerStatic; overload;
       {$ifdef HASINLINE}inline;{$endif}
    /// retrieve a list of members as JSON encoded data - used by OneFieldValue()
    // and MultiFieldValue() public functions
    function InternalListJSON(Table: TSQLRecordClass; const SQL: RawUTF8): TSQLTableJSON; override;
    function InternalAdaptSQL(TableIndex: integer; var SQL: RawUTF8): TSQLRestServerStatic;
    function InternalListRawUTF8(TableIndex: integer; const SQL: RawUTF8): RawUTF8;
    /// this method is overriden for setting the NoAJAXJSON field
    // of all associated TSQLRestServerStatic servers
    procedure SetNoAJAXJSON(const Value: boolean); virtual;
    /// add a new session to the internal session list
    // - do not use this method directly: this callback is to be used by
    // TSQLRestServerAuthentication* classes
    // - will check that the logon name is valid
    procedure SessionCreate(var User: TSQLAuthUser; Ctxt: TSQLRestServerURIContext;
      out Session: TAuthSession); virtual;
................................................................................
    // - this method is not thread-safe: caller should use fSessionCriticalSection
    function SessionAccess(Ctxt: TSQLRestServerURIContext): TAuthSession;
    /// delete a session from its index in fSessions[]
    // - will perform any needed clean-up, and log the event
    // - this method is not thread-safe: caller should use fSessionCriticalSection
    procedure SessionDelete(aSessionIndex: integer; Ctxt: TSQLRestServerURIContext);
    /// returns TRUE if this table is worth caching (e.g. already in memory)
    // - this overriden implementation returns FALSE for TSQLRestServerStaticInMemory
    function CacheWorthItForTable(aTableIndex: cardinal): boolean; override;
    /// get a member from its ID (implements REST GET member)
    // - returns the data of this object as JSON
    // - override this method for proper data retrieval from the database engine
    // - this method must be implemented in a thread-safe manner
    // - ForUpdate parameter is used only on Client side
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; virtual; abstract;


    /// virtual method called when a record is updated
    // - default implementation will call the OnUpdateEvent/OnBlobUpdateEvent
    // methods, if defined
    // - returns true on success, false if an error occured (but action must continue)
    // - you can override this method to implement a server-wide notification,
    // but be aware it may be the first step to break the stateless architecture
    // of the framework
    function InternalUpdateEvent(aEvent: TSQLEvent; aTable: TSQLRecordClass; aID: integer;








      aIsBlobFields: PSQLFieldBits): boolean; virtual;






  public
    /// this integer property is incremented by the database engine when any SQL
    // statement changes the database contents (i.e. on any not SELECT statement)
    // - its value can be published to the client on every remote request
    // - it may be used by client to avoid retrieve data only if necessary
    // - if its value is 0, this feature is not activated on the server, and the
    // client must ignore it and always retrieve the content
................................................................................
    OnSessionClosed: TNotifySQLSession;
    /// this property can be used to specify the URI parmeters to be used
    // for query paging
    // - is set by default to PAGINGPARAMETERS_YAHOO constant by
    // TSQLRestServer.Create() constructor
    URIPagingParameters: TSQLRestServerURIPagingParameters;

    /// implement Server-Side TSQLRest Retrieval (GET or LOCK methods)
    // - uses internally EngineRetrieve() function for calling the database engine
    // (via fStaticData[] if the table is stored as Static)
    // - handles locking if necessary
    // - if ForUpdate is true, the REST method is LOCK and not GET: it tries to lock
    // the corresponding record, then retrieve its content; caller has to call
    // UnLock() method after Value usage, to release the record
    function Retrieve(aID: integer; Value: TSQLRecord; ForUpdate: boolean=false): boolean; override;
    /// Execute directly a SQL statement, expecting a list of results
    // - return a result table on success, nil on failure
    // - will call EngineList() abstract method to retrieve its JSON content
    function ExecuteList(const Tables: array of TSQLRecordClass; const SQL: RawUTF8): TSQLTableJSON; override;
    /// implement Server-Side TSQLRest adding
    // - uses internally EngineAdd() function for calling the database engine
    // - call corresponding fStaticData[] if necessary
    // - on success, returns the new ROWID value; on error, returns 0
    // - on success, Value.ID is updated with the new ROWID
    // - if aValue is TSQLRecordFTS3, Value.ID is stored to the virtual table
    function Add(Value: TSQLRecord; SendData: boolean; ForceID: boolean=false): integer; override;
    /// implement Server-Side TSQLRest update
    // - uses internally EngineUpdate() function for calling the database engine
    // - call corresponding fStaticData[] if necessary
    function Update(Value: TSQLRecord): boolean; override;
    /// implement Server-Side TSQLRest deletion
    // - uses internally EngineDelete() function for calling the database engine
    // - call corresponding fStaticData[] if necessary
    // - this record is also erased in all available TRecordReference properties
    // in the database Model, for relational database coherency
    function Delete(Table: TSQLRecordClass; ID: integer): boolean; override;
    /// implement Server-Side TSQLRest deletion with a WHERE clause
................................................................................
    // - will process all ORM-level validation, coherency checking and
    // notifications together with a low-level SQL deletion work (if possible)
    function Delete(Table: TSQLRecordClass; const SQLWhere: RawUTF8): boolean; override;
    /// overriden method for direct static class call (if any)
    function TableRowCount(Table: TSQLRecordClass): integer; override;
    /// overriden method for direct static class call (if any)
    function TableHasRows(Table: TSQLRecordClass): boolean; override;









    /// this method is called internally after any successfull deletion to
    // ensure relational database coherency
    // - delete all available TRecordReference properties pointing to this record
    // in the database Model, for database coherency
    // - delete all available TSQLRecord properties pointing to this record
    // in the database Model, for database coherency
    // - important notice: we don't use FOREIGN KEY constraints in this framework,
    // and handle all integrity check within this method (it's therefore less
    // error-prone, and more cross-database engine compatible)S
    function AfterDeleteForceCoherency(Table: TSQLRecordClass; aID: integer): boolean; virtual;
    /// implement Server-Side TSQLRest blob field content retrieval
    // - implements REST GET member with a supplied member ID and a blob field name
    // - uses internally EngineRetrieveBlob() function for calling the database engine
    // - call corresponding fStaticData[] if necessary
    // - this method retrieve the blob data as a TSQLRawBlob string
    function RetrieveBlob(Table: TSQLRecordClass; aID: integer;
      const BlobFieldName: RawUTF8; out BlobData: TSQLRawBlob): boolean; override;
    /// implement Server-Side TSQLRest blob field content update
    // - implements REST PUT member with a supplied member ID and field name
    // - uses internally EngineUpdateBlob() function for calling the database engine
    // - this method expect the blob data to be supplied as a TSQLRawBlob string
    function UpdateBlob(Table: TSQLRecordClass; aID: integer;
      const BlobFieldName: RawUTF8; const BlobData: TSQLRawBlob): boolean; override;
    /// update all BLOB fields of the supplied Value
    // - this overriden method will execute the direct static class, if any
    function UpdateBlobFields(Value: TSQLRecord): boolean; override;
    /// get all BLOB fields of the supplied value from the remote server
    // - this overriden method will execute the direct static class, if any
    function RetrieveBlobFields(Value: TSQLRecord): boolean; override;
    /// implement Server-Side TSQLRest unlocking

    // - implements our custom UNLOCK REST-like method
    // - locking is handled by TSQLServer.Model
    // - returns true on success
    function UnLock(Table: TSQLRecordClass; aID: integer): boolean; override;
    {/ end a transaction (implements REST END Member)
     - write all pending TSQLVirtualTableJSON data to the disk }
    procedure Commit(SessionID: cardinal); override;
    /// Execute directly all SQL statement (POST SQL on ModelRoot URI)
................................................................................
    // - call it just after Create, before TSQLRestServerDB.CreateMissingTables;
    // warning: if you don't call this method before CreateMissingTable method
    // is called, the table will be created as a regular table by the main
    // database engine, and won't be static
    // - can load the table content from a file if a file name is specified
    // (could be either JSON or compressed Binary format on disk)
    // - you can define a particular external engine by setting a custom class -
    // by default, it will create a TSQLRestServerStaticInMemory instance
    // - this data handles basic REST commands, since no complete SQL interpreter
    // can be implemented by TSQLRestServerStatic; to provide full SQL process,
    // you should better use a Virtual Table class, inheriting e.g. from
    // TSQLRecordVirtualTableAutoID associated with TSQLVirtualTableJSON/Binary
    // via a Model.VirtualTableRegister() call before TSQLRestServer.Create 
    // - return nil on any error, or an EModelException if the class is not in
    // the database model
    function StaticDataCreate(aClass: TSQLRecordClass;
      const aFileName: TFileName = ''; aBinaryFile: boolean=false;
      aServerClass: TSQLRestServerStaticClass=nil): TSQLRestServerStatic;
    /// call this method when the internal DB content is known to be invalid
    // - by default, all REST/CRUD requests and direct SQL statements are
    // scanned and identified as potentially able to change the internal SQL/JSON
    // cache used at SQLite3 database level; but some virtual tables (e.g.
    // TSQLRestServerStaticExternal classes defined in mORMotDB) could flush
    // the database content without proper notification
    // - this default implementation just do nothing, but SQlite3 unit
    // will call TSQLDataBase.CacheFlush method
    procedure FlushInternalDBCache; virtual;
    /// you can call this method in TThread.Execute to ensure that
    // the thread will be taken in account during process
    // - caller must specify the TThread instance running
    // - used e.g. for optExecInMainThread option in TServiceMethod.InternalExecute
    // - this default implementation will call the methods of all its internal
    // TSQLRestServerStatic instances
    // - this method shall be called from the thread just initiated: e.g.
    // if you call it from the main thread, it may fail to prepare resources
    procedure BeginCurrentThread(Sender: TThread); virtual;
    /// you can call this method just before a thread is finished to ensure
    // e.g. that the associated external DB connection will be released
    // - this default implementation will call the methods of all its internal
    // TSQLRestServerStatic instances, allowing e.g. TSQLRestServerStaticExternal
    // instances to clean their thread-specific connections
    // - this method shall be called from the thread about to be terminated: e.g.
    // if you call it from the main thread, it may fail to release resources
    // - it is set e.g. by TSQLite3HttpServer to be called from HTTP threads,
    // or by TSQLRestServerNamedPipeResponse for named-pipe server cleaning
    procedure EndCurrentThread(Sender: TThread); virtual;

................................................................................
    procedure AuthenticationRegister(const aMethods: array of TSQLRestServerAuthenticationClass); overload;
    /// call this method to remove an authentication method to the server
    procedure AuthenticationUnregister(aMethod: TSQLRestServerAuthenticationClass); overload;
    /// call this method to remove several authentication methods to the server
    procedure AuthenticationUnregister(const aMethods: array of TSQLRestServerAuthenticationClass); overload;
    /// add all published methods of a given object instance to the method-based
    // list of services
    // - all those published method signature should match TSQLRestServerCallBack  
    procedure ServiceMethodRegisterPublishedMethods(const aPrefix: RawUTF8; aInstance: TObject);
    /// call this method to disable Authentication method check for a given
    // published method name
    // - by default, only Auth and TimeStamp methods do not require the RESTful
    // authentication of the URI; you may call this method to add another method
    // to the list (e.g. for returning some HTML content from a public URI)
    procedure ServiceMethodByPassAuthentication(const aMethodName: RawUTF8);
................................................................................
    property HandleAuthentication: boolean read fHandleAuthentication;
    /// read-only access to the list of registered server-side authentication
    // methods, used for session creation
    property AuthenticationSchemes: TSQLRestServerAuthenticationDynArray
      read fSessionAuthentication;
    /// access to the Server statistics
    property Stats: TSQLRestServerStats read fStats;
    /// retrieve the TSQLRestServerStatic instance used to store and manage
    // a specified TSQLRecordClass in memory
    // - has been associated by the StaticDataCreate method
    property StaticDataServer[aClass: TSQLRecordClass]: TSQLRestServerStatic
      read GetStaticDataServer;
    /// retrieve a running TSQLRestServerStatic virtual table
    // - associated e.g. to a 'JSON' or 'Binary' virtual table module, or may
    // return a TSQLRestServerStaticExternal instance (as defined in mORMotDB)
    // - this property will return nil if there is no Virtual Table associated
    // or if the corresponding module is not a TSQLVirtualTable
    // (e.g. "pure" static tables registered by StaticDataCreate would be
    // accessible only via StaticDataServer[], not via StaticVirtualTable[])
    // - has been associated by the TSQLModel.VirtualTableRegister method or
    // the VirtualTableExternalRegister() global function
    property StaticVirtualTable[aClass: TSQLRecordClass]: TSQLRestServerStatic
      read GetVirtualTable;
    /// this property can be left to its TRUE default value, to handle any
    // TSQLVirtualTableJSON static tables (module JSON or BINARY) with direct
    // calls to the storage instance
    // - is set to TRUE by default to enable faster Direct mode
    // - in Direct mode, GET/POST/PUT/DELETE of individual records (or BLOB fields)
    // from URI() will call directly the corresponding TSQLRestServerStatic
    // instance, for better speed for most used RESTful operations; but complex
    // SQL requests (e.g. joined SELECT) will rely on the main SQL engine
    // - if set to false, will use the main SQLite3 engine for all statements
    // (should not to be used normaly, because it will add unnecessary overhead)
    property StaticVirtualTableDirect: boolean read fVirtualTableDirect
      write fVirtualTableDirect;
    /// the class inheriting from TSQLAuthUser, as defined in the model
................................................................................
    // with cmd in POST/PUT with {object} as value or DELETE with ID
    // - returns an array of integers: '[200,200,...]' or '["OK"]' if all
    // returned status codes are 200 (HTML_SUCCESS)
    // - URI are either 'ModelRoot/TableName/Batch' or 'ModelRoot/Batch'
    procedure Batch(Ctxt: TSQLRestServerURIContext);
  end;

  /// REST server with direct access to an external database engine
  // - you can set an alternate per-table database engine by using this class
  // - this abstract class is to be overriden with a proper implementation (like
  // our TSQLRestServerStaticInMemory class or TSQLRestServerStaticExternal
  // as defined in mORMotDB unit)
  TSQLRestServerStatic = class(TSQLRestServer)
  protected
    fStoredClass: TSQLRecordClass;
    fStoredClassProps: TSQLModelRecordProperties;
    fStoredClassRecordProps: TSQLRecordProperties;
    fFileName: TFileName;
    fModified: boolean;
    fOwner: TSQLRestServer;


    fBasicSQLCount: RawUTF8;
    fBasicSQLHasRows: array[boolean] of RawUTF8;
    /// any set bit in this field indicates UNIQUE field value
    fIsUnique: TSQLFieldBits;
    /// allow to force refresh for a given Static table
    // - default FALSE means to return the main TSQLRestServer.InternalState
    // - TRUE indicates that OutInternalState := cardinal(-1) will be returned
    fOutInternalStateForcedRefresh: boolean; 
    procedure Lock(WillModifyContent: boolean);
    procedure UnLock;
    /// override this method if you want to update the refresh state
    // - returns FALSE if the static table content was not modified (default
    // method implementation is to always return FALSE)
    // - returns TRUE if the table has been refreshed and its content was modified:
    // therefore the client will know he'll need to refresh some content
    function RefreshedAndModified: boolean; virtual;
    /// overriden method calling the owner (if any) to guess if this record
................................................................................
    // - InternalBatchStart/Stop may safely use a lock for multithreading:
    // implementation in TSQLRestServer.Batch use a try..finally block
    procedure InternalBatchStop; virtual;
    /// TSQLRestServer.URI use it for Static.EngineList to by-pass virtual table
    // - this default implementation will return TRUE and replace SQL with
    // SQLSelectAll[true] if it SQL equals SQLSelectAll[false] (i.e. 'SELECT *')
    // - this method is called only if the WHERE clause of SQL refers to the
    // static table name only (not needed to check it twice)  
    function AdaptSQLForEngineList(var SQL: RawUTF8): boolean; virtual;
  public
    /// initialize the server data, reading it from a file if necessary
    // - data encoding on file is UTF-8 JSON format by default, or
    // should be some binary format if aBinaryFile is set to true (this virtual
    // method will just ignore this parameter, which will be used for overriden
    // constructor only)
    constructor Create(aClass: TSQLRecordClass; aServer: TSQLRestServer;
      const aFileName: TFileName = ''; aBinaryFile: boolean=false); virtual;


    /// you can call this method in TThread.Execute to ensure that
    // the thread will be taken in account during process
    // - this overriden method will do nothing (should have been already made
    // at TSQLRestServer caller level)
    // - children classes may inherit from this method to notify e.g.
    // a third party process (like proper OLE initialization)
    procedure BeginCurrentThread(Sender: TThread); override;
    /// you can call this method just before a thread is finished to ensure
    // e.g. that the associated external DB connection will be released
    // - this overriden method will do nothing (should have been already made
    // at TSQLRestServer caller level)
    // - children classes may inherit from this method to notify e.g.
    // a third party process (like proper OLE initialization)
    procedure EndCurrentThread(Sender: TThread); override;

    /// overriden method for direct in-memory database engine call
    // - not implemented: always return false
    // - this method must be implemented to be thread-safe
    function EngineExecuteAll(const aSQL: RawUTF8): boolean; override;

    /// overriden method for direct in-memory database engine call
    // - made public since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    // - do nothing method: will return FALSE (aka error)
    function EngineUpdateField(Table: TSQLRecordClass;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;




    /// search for a numerical field value
    // - return true on success (i.e. if some values have been added to ResultID)
    // - store the results into the ResultID dynamic array
    // - faster than OneFieldValues method, which creates a temporary JSON content
    // - this default implementation will call the overloaded SearchField()
    // value after conversion of the FieldValue into RawUTF8
    function SearchField(const FieldName: RawUTF8; FieldValue: Integer;
................................................................................

    /// read only access to the file name specified by constructor
    // - you can call the TSQLRestServer.StaticDataCreate method to
    // update the file name of an already instancied static table
    property FileName: TFileName read fFileName write fFileName;
    /// read only access to a boolean value set to true if table data was modified
    property Modified: boolean read fModified write fModified;
    /// read only access to the class defining the record type stored in this REST server

    property StoredClass: TSQLRecordClass read fStoredClass;
    /// read only access to the ORM properties of the associated record type
    // - may be nil if this instance is not associated with a TSQLModel
    property StoredClassProps: TSQLModelRecordProperties read fStoredClassProps;
    /// read only access to the RTTI properties of the associated record type
    property StoredClassRecordProps: TSQLRecordProperties read fStoredClassRecordProps;
    /// read only access to the Server using this in-memory database
    property Owner: TSQLRestServer read fOwner;
  end;

  /// event prototype called by FindWhereEqual() method
  TFindWhereEqualEvent = procedure(aDest: pointer; aRec: TSQLRecord; aIndex: integer) of object;

  /// abstract REST server exposing some internal TSQLRecord-based methods
  TSQLRestServerStaticRecordBased = class(TSQLRestServerStatic)
  protected
    function EngineAdd(Table: TSQLRecordClass; const SentData: RawUTF8): integer; override;
    function EngineUpdate(Table: TSQLRecordClass; ID: integer; const SentData: RawUTF8): boolean; override;
  public
    /// manual Add of a TSQLRecord
    // - returns the ID created on success
    // - returns -1 on failure (not UNIQUE field value e.g.)
    // - on success, the Rec instance is added to the Values[] list: caller
    // doesn't need to Free it
    function AddOne(Rec: TSQLRecord; ForceID: boolean): integer; virtual; abstract;
................................................................................
    /// manual Retrieval of a TSQLRecord field values
    // - an instance of the associated static class is created
    // - and all its properties are filled from the Items[] values
    // - caller can modify these properties, then use UpdateOne() if the changes
    // have to be stored inside the Items[] list
    // - calller must always free the returned instance
    // - returns NIL if any error occured, e.g. if the supplied aID was incorrect
    // - method available since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function GetOne(aID: integer): TSQLRecord; virtual; abstract;
    /// manual Update of a TSQLRecord field values
    // - Rec.ID specifies which record is to be updated
    // - will update all properties, including BLOB fields and such
    // - returns TRUE on success, FALSE on any error (e.g. invalid Rec.ID)
    // - method available since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function UpdateOne(Rec: TSQLRecord): boolean; overload; virtual; abstract;
    /// manual Update of a TSQLRecord field values from an array of TSQLVar
    // - will update all properties, including BLOB fields and such
    // - returns TRUE on success, FALSE on any error (e.g. invalid Rec.ID)
    // - method available since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    // - this default implementation will create a temporary TSQLRecord instance
    // with the supplied Values[], and will call overloaded UpdateOne() method
    function UpdateOne(ID: integer; const Values: TSQLVarDynArray): boolean; overload; virtual;
  end;

  /// class able to handle a O(1) hashed-based search of a property in a TList 
  // - used e.g. to hash TSQLRestServerStaticInMemory field values
  TListFieldHash = class(TObjectHash)
  protected
    fValues: TList;
    fField: integer;
    fProp: TSQLPropInfo;
    fCaseInsensitive: boolean;
    /// overriden method to hash an item
................................................................................
  // common client, by using the TSQLRestServer.StaticDataCreate method:
  // it allows an unique access for both SQLite3 and Static databases
  // - handle basic REST commands, no SQL interpreter is implemented: only
  // valid SQL command is "SELECT Field1,Field2 FROM Table WHERE ID=120;", i.e
  // a one Table SELECT with one optional "WHERE fieldname = value" statement;
  // if used within a TSQLVirtualTableJSON, you'll be able to handle any kind of
  // SQL statement (even joined SELECT or such) with this memory-stored database
  // - our TSQLRestServerStatic database engine is very optimized and is a lot
  // faster than SQLite3 for such queries - but its values remain in RAM,
  // therefore it is not meant to deal with more than 100,000 rows
  // - data can be stored and retrieved from a file (JSON format is used by
  // default, if BinaryFile parameter is left to false; a proprietary compressed
  // binary format can be used instead) if a file name is supplied at creating
  // the TSQLRestServerStaticInMemory instance
  TSQLRestServerStaticInMemory = class(TSQLRestServerStaticRecordBased)
  protected
    fValue: TObjectList;
    /// true if IDs are sorted (which is the default behavior of this class),
    // for fastest ID2Index() by using a binary search algorithm
    fIDSorted: boolean;
    fCommitShouldNotUpdateFile: boolean;
    fBinaryFile: boolean;
................................................................................
    function GetJSONValues(Stream: TStream; Expand, withID: boolean;
      const Fields: TSQLFieldBits; WhereField: integer; const WhereValue: RawUTF8;
      FoundLimit,FoundOffset: integer): PtrInt;
    /// TSQLRestServer.URI use it for Static.EngineList to by-pass virtual table
    // - overriden method to handle basic queries as handled by EngineList()
    function AdaptSQLForEngineList(var SQL: RawUTF8): boolean; override;
    /// overriden methods for direct in-memory database engine thread-safe process
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    function EngineUpdate(Table: TSQLRecordClass; ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineRetrieveBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineDeleteWhere(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
  public
    /// initialize the server data, reading it from a file if necessary
    // - data encoding on file is UTF-8 JSON format by default, or
    // should be some binary format if aBinaryFile is set to true
    constructor Create(aClass: TSQLRecordClass; aServer: TSQLRestServer;
      const aFileName: TFileName = ''; aBinaryFile: boolean=false); override;
................................................................................
    // compression algorithm), with variable-length record storage: e.g. a 27 KB
    // Dali1.json content is stored into a 6 KB Dali2.data file
    // (this data has a text redundant field content in its FirstName field);
    // 502 KB People.json content is stored into a 92 KB People.data file
    // - returns the number of bytes written into Stream
    function SaveToBinary(Stream: TStream): integer;
    /// if file was modified, the file is updated on disk
    // - this method is called automaticaly when the TSQLRestServerStatic
    // instance is destroyed: should should want to call in in some cases,
    // in order to force the data to be saved regularly
    // - do nothing if the table content was not modified
    // - will write JSON content by default, or binary content if BinaryFile
    // property was set to true
    procedure UpdateFile;
    /// retrieve the index in Items[] of a particular ID
................................................................................
    /// manual Retrieval of a TSQLRecord field values
    // - an instance of the associated static class is created
    // - and all its properties are filled from the Items[] values
    // - caller can modify these properties, then use UpdateOne() if the changes
    // have to be stored inside the Items[] list
    // - calller must always free the returned instance
    // - returns NIL if any error occured, e.g. if the supplied aID was incorrect
    // - method available since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function GetOne(aID: integer): TSQLRecord; override;
    /// manual Update of a TSQLRecord field values
    // - Rec.ID specifies which record is to be updated
    // - will update all properties, including BLOB fields and such
    // - returns TRUE on success, FALSE on any error (e.g. invalid Rec.ID)
    // - method available since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function UpdateOne(Rec: TSQLRecord): boolean; override;
    /// manual Update of a TSQLRecord field values from a TSQLVar array
    // - will update all properties, including BLOB fields and such
    // - returns TRUE on success, FALSE on any error (e.g. invalid Rec.ID)
    // - method available since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function UpdateOne(ID: integer; const Values: TSQLVarDynArray): boolean; override;
    /// overriden method for direct in-memory database engine call
    // - made public since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function EngineDelete(Table: TSQLRecordClass; ID: integer): boolean; override;
    /// overriden method for direct in-memory database engine call
    // - made public since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function EngineUpdateField(Table: TSQLRecordClass;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
    /// overriden method for direct in-memory database engine call
    function UpdateBlobFields(Value: TSQLRecord): boolean; override;
    /// overriden method for direct in-memory database engine call
    function RetrieveBlobFields(Value: TSQLRecord): boolean; override;
    /// overriden method for direct in-memory database engine call
    function TableRowCount(Table: TSQLRecordClass): integer; override;
................................................................................
    /// set this property to TRUE if you want the COMMIT statement not to
    // update the associated TSQLVirtualTableJSON
    property CommitShouldNotUpdateFile: boolean read fCommitShouldNotUpdateFile
      write fCommitShouldNotUpdateFile;
  end;

  /// REST server with direct access to a memory database, to be used as
  // external table
  // - this is the kind of in-memory table expected by TSQLVirtualTableJSON,
  // in order to be consistent with the internal DB cache
  TSQLRestServerStaticInMemoryExternal = class(TSQLRestServerStaticInMemory)
  public

    /// call this method when the internal DB content is known to be invalid
    // - by default, all REST/CRUD requests and direct SQL statements are
    // scanned and identified as potentially able to change the internal SQL/JSON
    // cache used at SQLite3 database level; but TSQLVirtualTableJSON virtual
    // tables could flush the database content without proper notification
    // - this overriden implementation will call Owner.FlushInternalDBCache
    procedure FlushInternalDBCache; override;
  end;

  /// a REST server using only in-memory tables
  // - this server will use TSQLRestServerStaticInMemory instances to handle
  // the data in memory, and optionally persist the data on disk as JSON or
  // binary files
  // - so it will not handle all SQL requests, just basic CRUD commands on
  // separated tables
  // - at least, it will compile as a TSQLRestServer without complaining for
  // pure abstract methods; it can be used to host some services if database
  // and ORM needs are basic (e.g. if only authentication and CRUD are needed)
  TSQLRestServerFullMemory = class(TSQLRestServer)
  protected
    fFileName: TFileName;
    fBinaryFile: Boolean;
    fStaticDataCount: cardinal;
    function GetStatic(Table: TSQLRecordClass): TSQLRestServerStaticInMemory;
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    function EngineAdd(Table: TSQLRecordClass; const SentData: RawUTF8): integer; override;
    function EngineUpdate(Table: TSQLRecordClass; ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDelete(Table: TSQLRecordClass; ID: integer): boolean; override;
    function EngineDeleteWhere(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    function EngineRetrieveBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateField(Table: TSQLRecordClass;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
  public
    /// initialize a REST server with a database file
    // - all classes of the model will be created as TSQLRestServerStaticInMemory
    // - then data persistence will be created using aFileName
    // - if aFileName is left void (''), data will not be persistent
    constructor Create(aModel: TSQLModel; const aFileName: TFileName='';
      aBinaryFile: boolean=false; aHandleUserAuthentication: boolean=false); reintroduce; virtual;
    /// write any modification on file (if needed), and release all used memory
    destructor Destroy; override;
    /// Missing tables are created if they don't exist yet for every TSQLRecord
................................................................................
    procedure UpdateToFile; virtual;
    /// overriden method for direct in-memory database engine call
    // - not implemented: always return false
    function EngineExecuteAll(const aSQL: RawUTF8): boolean; override;
    /// the file name used for data persistence
    property FileName: TFileName read fFileName write fFileName;
    /// set if the file content is to be compressed binary, or standard JSON
    // - it will use TSQLRestServerStaticInMemory LoadFromJSON/LoadFromBinary
    // SaveToJSON/SaveToBinary methods for optimized storage
    property BinaryFile: Boolean read fBinaryFile write fBinaryFile;
  end;

  /// a REST server using a TSQLRestClient for all its ORM process
  // - this server will use an internal TSQLRestClient instance to handle
  // all ORM operations (i.e. access to objects)
................................................................................
  // easily implement a proxy architecture (for instance, as a DMZ for
  // publishing services, but letting ORM process stay out of scope)
  TSQLRestServerRemoteDB = class(TSQLRestServer)
  protected
    fClient: TSQLRestClient;
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    function EngineAdd(Table: TSQLRecordClass; const SentData: RawUTF8): integer; override;
    function EngineUpdate(Table: TSQLRecordClass; ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDelete(Table: TSQLRecordClass; ID: integer): boolean; override;
    function EngineDeleteWhere(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    function EngineRetrieveBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateField(Table: TSQLRecordClass;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
  public
    /// initialize a REST server associated to a given TSQLRestClient instance
    // - the specified TSQLRestClient will be used for all ORM and data process
    // - the supplied TSQLRestClient.Model will be used for TSQLRestServerRemoteDB
    // - note that the TSQLRestClient instance won't be freed - caller shall
    // manage its life time
................................................................................
    procedure SetForceBlobTransfert(Value: boolean);
    function GetForceBlobTransfertTable(aTable: TSQLRecordClass): Boolean;
    procedure SetForceBlobTransfertTable(aTable: TSQLRecordClass; aValue: Boolean);
    /// get a member from its ID (implements REST GET member)
    // - returns the data of this object as JSON
    // - override this method for proper data retrieval from the database engine
    // - this method must be implemented in a thread-safe manner
    function EngineRetrieve(TableModelIndex: integer; ID: integer;
      ForUpdate: boolean; var InternalState: cardinal; var Resp: RawUTF8): boolean; virtual; abstract;
    /// this method is called before updating any record
    // - should return FALSE to force no update
    // - can be use to update some field values just before saving to the database
    // (e.g. for digital signing purpose)
    // - this default method just return TRUE (i.e. OK to update)
    function BeforeUpdateEvent(Value: TSQLRecord): Boolean; virtual;


  public
    /// create a new member (implements REST POST Collection)
    // - URI is 'ModelRoot/TableName' with POST method
    // - if SendData is true, content of Value is sent to the server as JSON
    // - if ForceID is true, client sends the Value.ID field to use this ID
    // - server must return Status 201/HTML_CREATED on success
    // - server must send on success an header entry with
................................................................................
    // - on success, Value.ID is updated with the new ROWID
    // - if aValue is TSQLRecordFTS3, Value.ID is stored to the virtual table
    function Add(Value: TSQLRecord; SendData: boolean; ForceID: boolean=false): integer; override;
    /// update a member (implements REST PUT Collection/Member)
    // - URI is 'ModelRoot/TableName/TableID' with PUT method
    // - server must return Status 200/HTML_SUCCESS OK on success
    function Update(Value: TSQLRecord): boolean; override;
    /// delete a member (implements REST DELETE Collection/Member)
    // - URI is 'ModelRoot/TableName/TableID' with DELETE method
    // - server must return Status 200/HTML_SUCCESS OK on success
    function Delete(Table: TSQLRecordClass; ID: integer): boolean; override;
    /// get a member from its ID (implements REST GET Collection/Member)
    // - URI is 'ModelRoot/TableName/TableID' with GET method
    // - server must return Status 200/HTML_SUCCESS OK on success
    // - if ForUpdate is true, the REST method is LOCK and not GET: it tries to lock
    // the corresponding record, then retrieve its content; caller has to call
    // UnLock() method after Value usage, to release the record
    function Retrieve(aID: integer; Value: TSQLRecord; ForUpdate: boolean=false): boolean; override;
    /// get a blob field content from its record ID and supplied blob field name
    // - implements REST GET member with a supplied member ID and a blob field name
    // - URI is 'ModelRoot/TableName/TableID/BlobFieldName' with GET method
    // - server must return Status 200/HTML_SUCCESS OK on success
    // - this method retrieve the blob data as a TSQLRawBlob string
    function RetrieveBlob(Table: TSQLRecordClass; aID: integer;
      const BlobFieldName: RawUTF8; out BlobData: TSQLRawBlob): boolean; override;
    /// update a blob field from its record ID and supplied blob field name
    // - implements REST PUT member with a supplied member ID and field name
    // - URI is 'ModelRoot/TableName/TableID/BlobFieldName' with PUT method
    // - server must return Status 200/HTML_SUCCESS OK on success
    // - this method expect the blob data to be supplied as a TSQLRawBlob string
    function UpdateBlob(Table: TSQLRecordClass; aID: integer;
      const BlobFieldName: RawUTF8; const BlobData: TSQLRawBlob): boolean; override;
    /// get a member from its ID (implements REST GET Collection/Member)
    // - URI is 'ModelRoot/TableName/TableID' with GET method
    // - returns true on server returned 200/HTML_SUCCESS OK success, false on error
    // - set Refreshed to true if the content changed
    function Refresh(aID: integer; Value: TSQLRecord; var Refreshed: boolean): boolean;

    /// retrieve a list of members as a TSQLTable (implements REST GET Collection)
................................................................................
    // - a next call to InternalCheckOpen method shall re-open the connection 
    procedure InternalClose; virtual; abstract;
    /// calls 'ModelRoot/TableName/TableID' with appropriate REST method
    // - uses GET method if ForUpdate is false
    // - uses LOCK method if ForUpdate is true
    function URIGet(Table: TSQLRecordClass; ID: integer; var Resp: RawUTF8;
      ForUpdate: boolean=false): Int64Rec;
    /// retrieve a list of members as JSON encoded data - used by OneFieldValue()
    // and MultiFieldValue() public functions
    function InternalListJSON(Table: TSQLRecordClass; const SQL: RawUTF8): TSQLTableJSON; override;
    // overriden methods
    function EngineRetrieve(TableModelIndex: integer; ID: integer;

      ForUpdate: boolean; var InternalState: cardinal; var Resp: RawUTF8): boolean; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    function EngineAdd(Table: TSQLRecordClass; const SentData: RawUTF8): integer; override;
    function EngineUpdate(Table: TSQLRecordClass; ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDelete(Table: TSQLRecordClass; ID: integer): boolean; override;
    function EngineDeleteWhere(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    function EngineRetrieveBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateField(Table: TSQLRecordClass;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
    procedure SetLastException(E: Exception=nil; ErrorCode: integer=HTML_BADREQUEST);
  public
    /// initialize REST client instance
    constructor Create(aModel: TSQLModel); override;
    /// release memory and close client connection
    // - also unlock all still locked records by this client
................................................................................
    /// a set of features of a Virtual Table
    Features: TSQLVirtualTableFeatures;
    /// the associated cursor class
    CursorClass: TSQLVirtualTableCursorClass;
    /// the associated TSQLRecord class
    // - used to retrieve the field structure with all collations
    RecordClass: TSQLRecordClass;
    /// the associated TSQLRestServerStatic class used for storage
    // - is e.g. TSQLRestServerStaticInMemory for TSQLVirtualTableJSON,
    // TSQLRestServerStaticExternal for TSQLVirtualTableExternal, or nil for
    // TSQLVirtualTableLog
    StaticClass: TSQLRestServerStaticClass;
    /// can be used to customize the extension of the filename
    // - the '.' is not to be included
    FileExtension: TFileName;
  end;

  {/ parent class able to define a Virtual Table module
   - in order to implement a new Virtual Table type, you'll have to define a so
................................................................................
    function FileName(const aTableName: RawUTF8): TFileName; virtual;
    /// the Virtual Table module features
    property Features: TSQLVirtualTableFeatures read fFeatures.Features;
    /// the associated virtual table class
    property TableClass: TSQLVirtualTableClass read fTableClass;
    /// the associated virtual table cursor class
    property CursorClass: TSQLVirtualTableCursorClass read fFeatures.CursorClass;
    /// the associated TSQLRestServerStatic class used for storage
    // - e.g. returns TSQLRestServerStaticInMemory for TSQLVirtualTableJSON,
    // or TSQLRestServerStaticExternal for TSQLVirtualTableExternal, or
    // either nil for TSQLVirtualTableLog
    property StaticClass: TSQLRestServerStaticClass read fFeatures.StaticClass;
    /// the associated TSQLRecord class
    // - is mostly nil, e.g. for TSQLVirtualTableJSON
    // - used to retrieve the field structure for TSQLVirtualTableLog e.g.
    property RecordClass: TSQLRecordClass read fFeatures.RecordClass;
    /// the extension of the filename (without any left '.')
    property FileExtension: TFileName read fFeatures.FileExtension;
    /// the full path to be used for the filename
................................................................................
     virtual methods to allow content writing to the virtual table
   - the same virtual table mechanism can be used with several database module,
     with diverse database engines }
  TSQLVirtualTable = class
  protected
    fModule: TSQLVirtualTableModule;
    fTableName: RawUTF8;
    fStatic: TSQLRestServerStatic;

  public
    /// create the virtual table access instance
    // - the created instance will be released when the virtual table will be
    // disconnected from the DB connection (e.g. xDisconnect method for SQLite3)
    // - shall raise an exception in case of invalid parameters (e.g. if the
    // supplied module is not associated to a TSQLRestServer instance)
    // - aTableName will be checked against the current aModule.Server.Model
    // to retrieve the corresponding TSQLRecordVirtualTableAutoID class and
    // create any associated Static: TSQLRestServerStatic instance
    constructor Create(aModule: TSQLVirtualTableModule; const aTableName: RawUTF8;
      FieldCount: integer; Fields: PPUTF8CharArray); virtual;
    /// release the associated memory, especially the Static instance
    destructor Destroy; override;
    /// retrieve the corresponding module name
    // - will use the class name, triming any T/TSQL/TSQLVirtual/TSQLVirtualTable* 
    // - when the class is instanciated, it will be faster to retrieve the same
................................................................................
    // SQLite database - it could make bad written code slow even with Virtual
    // Tables
    function Transaction(aState: TSQLVirtualTableTransaction; aSavePoint: integer): boolean; virtual;
    /// called to rename the virtual table
    // - by default, returns false, i.e. always fails
    function Rename(const NewName: RawUTF8): boolean; virtual;
    /// the associated virtual table storage instance
    // - can be e.g. a TSQLRestServerStaticInMemory for TSQLVirtualTableJSON,
    // or a TSQLRestServerStaticExternal for TSQLVirtualTableExternal, or nil
    // for TSQLVirtualTableLog
    property Static: TSQLRestServerStatic read fStatic;


  end;

  {/ abstract class able to define a Virtual Table cursor
    - override the Search/HasData/Column/Next abstract virtual methods to
    implement the search process }
  TSQLVirtualTableCursor = class
  protected
................................................................................
    // - will check the fCurrent/fMax protected properties values
    function Next: boolean; override;
    /// called to begin a search in the virtual table
    // - this no-op version will mark EOF, i.e. fCurrent=0 and fMax=-1
    function Search(const Prepared: TSQLVirtualTablePrepared): boolean; override;
  end;
  
  {/ A Virtual Table cursor for reading a TSQLRestServerStaticInMemory content
    - this is the cursor class associated to TSQLVirtualTableJSON }
  TSQLVirtualTableCursorJSON = class(TSQLVirtualTableCursorIndex)
  public
    /// called to begin a search in the virtual table
    // - the TSQLVirtualTablePrepared parameters were set by
    // TSQLVirtualTable.Prepare and will contain both WHERE and ORDER BY statements
    // (retrieved by x_BestIndex from a TSQLite3IndexInfo structure)
................................................................................
    function Search(const Prepared: TSQLVirtualTablePrepared): boolean; override;
    /// called to retrieve a column value of the current data row into a TSQLVar
    // - if aColumn=-1, will return the row ID as varInt64 into aResult
    // - will return false in case of an error, true on success
    function Column(aColumn: integer; var aResult: TSQLVar): boolean; override;
  end;

  {/ A TSQLRestServerStaticInMemory-based virtual table using JSON storage
   - for ORM access, you should use TSQLModel.VirtualTableRegister method to
     associated this virtual table module to a TSQLRecordVirtualTableAutoID class
   - transactions are not handled by this module
   - by default, no data is written on disk: you will need to call explicitly
     aServer.StaticVirtualTable[aClass].UpdateToFile for file creation or refresh
   - file extension is set to '.json' }
  TSQLVirtualTableJSON = class(TSQLVirtualTable)
................................................................................
    // - column order follows the Structure method, i.e.
    // StoredClassRecordProps.Fields[] order
    // - returns true on success, false otherwise
    // - does nothing by default, and returns false, i.e. always fails
    function Update(oldRowID, newRowID: Int64; var Values: TSQLVarDynArray): boolean; override;
  end;

  {/ A TSQLRestServerStaticInMemory-based virtual table using Binary storage
   - for ORM access, you should use TSQLModel.VirtualTableRegister method to
     associated this virtual table module to a TSQLRecordVirtualTableAutoID class
   - transactions are not handled by this module
   - by default, no data is written on disk: you will need to call explicitly
     aServer.StaticVirtualTable[aClass].UpdateToFile for file creation or refresh
   - binary format is more efficient in term of speed and disk usage than
     the JSON format implemented by TSQLVirtualTableJSON
................................................................................
    InitializeCriticalSection(fAcquireExecution[cmd].Lock);
end;

destructor TSQLRest.Destroy;
var cmd: TSQLRestServerURIContextCommand;
begin
  if (fModel<>nil) and (fModel.fRestOwner=self) then
    // make sure we are the Owner (TSQLRestServerStatic has fModel<>nil e.g.)
    FreeAndNil(fModel);
  fServices.Free;
  fCache.Free;
  for cmd := Low(cmd) to high(cmd) do begin
    DeleteCriticalSection(fAcquireExecution[cmd].Lock);
    fAcquireExecution[cmd].Thread.Free;
  end;
................................................................................
  result := false;
  if (Strings<>nil) and (self<>nil) and (Table<>nil) then
  try
    {$ifndef LVCL}
    Strings.BeginUpdate;
    {$endif}
    Strings.Clear;
    T := InternalListJSON(Table,
      SQLFromSelect(Table.SQLTableName,'ID,'+FieldName,WhereClause,''));
    if T<>nil then
    try
      if (T.FieldCount=2) and (T.RowCount>0) then begin
        for Row := 1 to T.RowCount do begin // ignore Row 0 i.e. field names
          aID := GetInteger(T.Get(Row,0));
          Strings.AddObject(UTF8ToString(T.GetU(Row,1)),pointer(aID));
................................................................................
  WhereClause: RawUTF8): TSQLTableJSON;
begin
  if (self=nil) or (Table=nil) then
    result := nil else
    with Table.RecordProps do
    if (PosEx(RawUTF8(','),FieldName,1)=0) and not IsFieldName(FieldName) then
      result := nil else // prevent SQL error
      result := InternalListJSON(Table,
        SQLFromSelect(SQLTableName,FieldName,WhereClause,''));
end;     

function TSQLRest.InternalListRecordsJSON(Table: TSQLRecordClass;
  const WhereClause: RawUTF8): TSQLTableJSON;
begin
  if (self=nil) or (Table=nil) then
    result := nil else
    result := InternalListJSON(Table,Model.Props[Table].SQLFromSelectWhere('*',WhereClause));
end;

function TSQLRest.MultiFieldValues(Table: TSQLRecordClass; FieldNames: RawUTF8;
  const WhereClause: RawUTF8): TSQLTableJSON;
var P: PUTF8Char;
    aFieldName: RawUTF8;
begin
................................................................................
      P := pointer(FieldNames);
      repeat
        aFieldName := Trim(GetNextItem(P));
        if not Props.IsFieldName(aFieldName) then
          exit; // invalid field name
      until P=nil;
    end;
    result := InternalListJSON(Table,SQLFromSelectWhere(FieldNames,WhereClause));
  end;
end;

function TSQLRest.MultiFieldValues(Table: TSQLRecordClass; const FieldNames: RawUTF8;
  WhereClauseFormat: PUTF8Char; const BoundsSQLWhere: array of const): TSQLTableJSON;
begin
  result := MultiFieldValues(Table,FieldNames,FormatUTF8(WhereClauseFormat,[],BoundsSQLWhere));
................................................................................
    for i := 0 to high(FieldName) do
      if not IsFieldName(FieldName[i]) then
        exit else // prevent SQL error
        if SQL='' then
          SQL := 'SELECT '+FieldName[i] else
          SQL := SQL+','+FieldName[i];
    SQL := SQL+' FROM '+SQLTableName+' WHERE '+WhereClause+' LIMIT 1;';
    T := InternalListJSON(Table,SQL);
    if T<>nil then
    try
      if (T.FieldCount<>length(FieldName)) or (T.RowCount<=0) then
        exit;
      // get field values from the first (and unique) row
      for i := 0 to T.FieldCount-1 do
        FieldValue[i] := T.fResults[T.FieldCount+i];
................................................................................
end;

function TSQLRest.Retrieve(const SQLWhere: RawUTF8; Value: TSQLRecord): boolean;
var T: TSQLTable;
begin
  if (self=nil) or (Value=nil) then
    T := nil else
    T := InternalListJSON(PSQLRecordClass(Value)^,Model.Props[PSQLRecordClass(Value)^].
      SQLFromSelectWhere('*',SQLWhere+' LIMIT 1'));
  if T=nil then
    result := false else
    try
      if T.RowCount>=1 then begin
        Value.FillFrom(T,1); // fetch data from first result row
        result := true;
      end else
................................................................................
  try
    result := TObjectList.Create;
    T.ToObjectList(result,Table);
  finally
    T.Free;
  end;
end;






























function TSQLRest.Retrieve(WhereClauseFmt: PUTF8Char; const Args,Bounds: array of const;
  Value: TSQLRecord): boolean;
begin
  result := Retrieve(FormatUTF8(WhereClauseFmt,Args,Bounds),Value);
end;

................................................................................
function TSQLRest.RecordCanBeUpdated(Table: TSQLRecordClass; ID: integer; Action: TSQLEvent;
  ErrorMsg: PRawUTF8 = nil): boolean;
begin
  result := true; // accept by default -> override this method to customize this
end;

function TSQLRest.Delete(Table: TSQLRecordClass; ID: integer): boolean;

begin

  result := RecordCanBeUpdated(Table,ID,seDelete);



end;


function TSQLRest.InternalDelete(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
  var IDs: TIntegerDynArray): boolean;
var i: integer;
begin
  result := false;
  if (not OneFieldValues(Table,'RowID',SQLWhere,IDs)) or
     (IDs=nil) then
    exit;
................................................................................
    fCache.NotifyDeletion(Table,IDs[i]);
  result := true;
end;

function TSQLRest.Delete(Table: TSQLRecordClass; const SQLWhere: RawUTF8): boolean;
var IDs: TIntegerDynArray;
begin
  if InternalDelete(Table,SQLWhere,IDs) then
    result := EngineDeleteWhere(Table,SQLWhere,IDs) else
    result := false;
end;

function TSQLRest.Delete(Table: TSQLRecordClass; FormatSQLWhere: PUTF8Char;
  const BoundsSQLWhere: array of const): boolean;
begin
  result := Delete(Table,FormatUTF8(FormatSQLWhere,[],BoundsSQLWhere));
end;

function TSQLRest.Update(Value: TSQLRecord): boolean;


begin
  result := (Value<>nil) and (Value.fID<>0) and

    RecordCanBeUpdated(PSQLRecordClass(Value)^,Value.fID,seUpdate);








end;

function TSQLRest.Update(aTable: TSQLRecordClass; aID: integer;
  const aSimpleFields: array of const): boolean;
var Value: TSQLRecord;
begin
  result := false; // means error
................................................................................
      exit;
    Value.fID := aID;
    result := Update(Value);
  finally
    Value.Free;
  end;
end;



























function TSQLRest.Add(aTable: TSQLRecordClass; const aSimpleFields: array of const;
  ForcedID: integer=0): integer;
var Value: TSQLRecord;
begin
  result := 0; // means error
  if (self=nil) or (aTable=nil) then
................................................................................
    qoSoundsLikeFrench,
    qoSoundsLikeSpanish:
      result := PSynSoundEx(Reference)^.UTF8(Value);
  end;
end;

function TSQLRest.RetrieveBlob(Table: TSQLRecordClass; aID: integer;
  const BlobFieldName: RawUTF8;
  out BlobStream: THeapMemoryStream): boolean;
var BlobData: TSQLRawBlob;
begin
  BlobStream := THeapMemoryStream.Create;
  result := RetrieveBlob(Table,aID,BlobFieldName,BlobData);
  if not result or (BlobData='') then
    exit;
  BlobStream.Write(pointer(BlobData)^,length(BlobData));
................................................................................
begin
  if (self=nil) or (BlobData=nil) or (BlobSize<0) then
    result := false else begin
    SetString(Blob,PAnsiChar(BlobData),BlobSize);
    result := UpdateBlob(Table,aID,BlobFieldName,Blob);
  end;
end;





























function TSQLRest.UpdateBlobFields(Value: TSQLRecord): boolean;
var BlobData: RawByteString;
    i: integer;
begin
  result := false;
  if (Value=nil) or (Value.fID<=0) then
    exit;
  with Value.RecordProps do
  if BlobFields<>nil then

    for i := 0 to high(BlobFields) do begin
      GetLongStrProp(Value,BlobFields[i].PropInfo,BlobData);
      if not EngineUpdateBlob(PSQLRecordClass(Value)^,Value.fID,BlobFields[i].PropInfo,BlobData) then
        exit;
    end;

  result := true;
end;

function TSQLRest.RetrieveBlobFields(Value: TSQLRecord): boolean;
var BlobData: TSQLRawBlob;
    i: integer;
begin
  result := false;
  if (Self=nil) or (Value=nil) or (Value.fID<=0) then
    exit;
  with Value.RecordProps do
  if BlobFields<>nil then

    for i := 0 to high(BlobFields) do
      if EngineRetrieveBlob(PSQLRecordClass(Value)^,Value.fID,BlobFields[i].PropInfo,BlobData) then
        SetLongStrProp(Value,BlobFields[i].PropInfo,BlobData) else
        exit;

  result := true;
end;

function TSQLRest.TableRowCount(Table: TSQLRecordClass): integer;
var T: TSQLTableJSON;
begin
  if (self=nil) or (Table=nil) then
    T := nil else

    T := InternalListJSON(Table,'SELECT Count(*) FROM '+Table.RecordProps.SQLTableName);
  if T<>nil then
  try
    Result := T.GetAsInteger(1,0);
  finally
    T.Free;
  end else
    Result := -1;
................................................................................
end;

function TSQLRest.TableHasRows(Table: TSQLRecordClass): boolean;
var T: TSQLTableJSON;
begin
  if (self=nil) or (Table=nil) then
    T := nil else
    T := InternalListJSON(Table,
      'SELECT RowID FROM '+Table.RecordProps.SQLTableName+' LIMIT 1');
  if T<>nil then
  try
    Result := T.RowCount>0;
  finally
    T.Free;
  end else
    Result := false;
end;










function TSQLRest.MainFieldValue(Table: TSQLRecordClass; ID: Integer;
   ReturnFirstIfNoUnique: boolean=false): RawUTF8;
begin
  if (self=nil) or (Table=nil) or (ID<=0) then
    result := '' else begin
    result := Table.RecordProps.MainFieldName(Table,ReturnFirstIfNoUnique);
................................................................................
  end;
end;

function TSQLRestClientURI.UpdateFromServer(const Data: array of TObject; out Refreshed: boolean;
  PCurrentRow: PInteger): boolean;
// notes about refresh mechanism:
// - if server doesn't implement InternalState, its value is 0 -> always refresh
// - if any TSQLTableJSON or TSQLRecord belongs to a TSQLRestServerStatic,
// the Server stated fInternalState=cardinal(-1) for them -> always refresh
var i: integer;
    State: cardinal;
    Resp: RawUTF8;
    T: TSQLTableJSON;
    TRefreshed: boolean; // to check for each Table refresh
const TState: array[boolean] of TOnTableUpdateState = (tusNoChange,tusChanged);
................................................................................
        exit else
        InternalState := Hi;
    result := TSQLTableJSON.CreateFromTables(Tables,SQL,Resp); // get data
  end;
  result.fInternalState := InternalState;
end;

function TSQLRestClientURI.InternalListJSON(Table: TSQLRecordClass; const SQL: RawUTF8): TSQLTableJSON;
begin
  result := ExecuteList([Table],SQL);
end;

procedure TSQLRestClientURI.SessionClose;
var tmp: RawUTF8;
begin
  if (self<>nil) and (fSessionUser<>nil) and
     (fSessionID<>CONST_AUTHENTICATION_SESSION_NOT_STARTED) then
  try
    // notify session closed to server
................................................................................
    // may not contain all cached fields -> delete from cache
    fCache.NotifyDeletion(Value.RecordClass,Value.fID) else
    fCache.Notify(Value,soUpdate);
  result := fBatchCount;
  inc(fBatchCount);
end;

function TSQLRestClientURI.EngineAdd(Table: TSQLRecordClass;
  const SentData: RawUTF8): integer;
var P: PUTF8Char;
    Head: RawUTF8;
begin
  result := 0;

  if URI(Model.URI[Table],'POST',nil,@Head,@SentData).Lo<>HTML_CREATED then
    exit; // response must be '201 Created'
  P := pointer(Head); // we need to check the headers
  if P<>nil then
  repeat
    // find ID from 'Location: Member Entry URI' header entry
    if IdemPChar(P,'LOCATION:') then begin // 'Location: root/People/11012' e.g.
      inc(P,9);
................................................................................
    end;
    while not (P^ in [#0,#13]) do inc(P);
    if P^=#0 then break else inc(P);
    if P^=#10 then inc(P);
  until false;
end;

function TSQLRestClientURI.EngineDelete(Table: TSQLRecordClass; ID: integer): boolean;

begin

  result := URI(Model.getURIID(Table,ID),'DELETE').Lo=HTML_SUCCESS;
end;

function TSQLRestClientURI.EngineDeleteWhere(Table: TSQLRecordClass;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;

begin  // ModelRoot/TableName?where=WhereClause to delete members

  result := URI(Model.getURI(Table)+'?where='+UrlEncode(SQLWhere),'DELETE').Lo=HTML_SUCCESS;
end;

function TSQLRestClientURI.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean; ReturnedRowCount: PPtrInt): RawUTF8;
begin
  if (self=nil) or (SQL='') or (ReturnedRowCount<>nil) or
     (URI(Model.Root,'GET',@result,nil,@SQL).Lo<>HTML_SUCCESS) then
    result := ''
end;

function TSQLRestClientURI.EngineRetrieve(TableModelIndex, ID: integer;
  ForUpdate: boolean; var InternalState: cardinal; var Resp: RawUTF8): boolean;
begin
  if cardinal(TableModelIndex)<=cardinal(Model.fTablesMax) then
  with URIGet(Model.Tables[TableModelIndex],ID,Resp,ForUpdate) do
    if Lo=HTML_SUCCESS then begin
      InternalState := Hi;
      result := true;
    end else
      result := false else
      result := false;
end;

function TSQLRestClientURI.EngineRetrieveBlob(Table: TSQLRecordClass;
  aID: integer; BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;

begin
  if (self=nil) or (aID<=0) or (BlobField=nil) then
    result := false else
    // URI is 'ModelRoot/TableName/TableID/BlobFieldName' with GET method
    result := URI(Model.getURICallBack(BlobField^.Name,Table,aID),
      'GET',@BlobData).Lo=HTML_SUCCESS;
end;


function TSQLRestClientURI.EngineUpdate(Table: TSQLRecordClass;
  ID: integer; const SentData: RawUTF8): boolean;
begin

  result := URI(Model.getURIID(Table,ID),'PUT',nil,nil,@SentData).Lo=HTML_SUCCESS;
end;

function TSQLRestClientURI.EngineUpdateBlob(Table: TSQLRecordClass;
  aID: integer; BlobField: PPropInfo;
  const BlobData: TSQLRawBlob): boolean;
var Head: RawUTF8;
begin
  Head := 'Content-Type: application/octet-stream';
  if (self=nil) or (aID<=0) or (BlobField=nil) then
    result := false else
    // PUT ModelRoot/TableName/TableID/BlobFieldName 
    result := URI(FormatUTF8('%/%/%',[Model.URI[Table],aID,BlobField^.Name]),
       'PUT',nil,@Head,@BlobData).Lo=HTML_SUCCESS;
end;


function TSQLRestClientURI.EngineUpdateField(Table: TSQLRecordClass;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
begin
  if (self=nil) or (Table=nil) then
    result := false else
    // PUT ModelRoot/TableName?setname=..&set=..&wherename=..&where=..
    result := URI(FormatUTF8('%?setname=%&set=%&wherename=%&where=%',

      [Model.URI[Table],SetFieldName,UrlEncode(SetValue),WhereFieldName,
       UrlEncode(WhereValue)]),'PUT').Lo=HTML_SUCCESS;
end;


{ TSQLRestServer }

{$ifdef MSWINDOWS}
const
................................................................................
    sleep(200); // way some time any request is finished in another thread
  end;
  // close any running named-pipe or GDI-messages server instance
  CloseServerNamedPipe;
  CloseServerMessage;
{$endif}
  for i := 0 to high(fStaticData) do
    // free all TSQLRestServerStatic objects and update file if necessary
    fStaticData[i].Free;
  fSessions.Free;
  DeleteCriticalSection(fSessionCriticalSection);
  inherited Destroy; // calls fServices.Free which will update fStats
  if not InheritsFrom(TSQLRestServerStatic) then
    InternalLog(Stats.DebugMessage,sllInfo);
  fStats.Free; 
end;

function TSQLRestServer.GetStaticDataServer(aClass: TSQLRecordClass): TSQLRestServerStatic;
begin
  if (self<>nil) and (fStaticData<>nil) then
   result := fStaticData[Model.GetTableIndexExisting(aClass)] else
   result := nil;
end;

function TSQLRestServer.GetStaticDataServerOrVirtualTable(aClass: TSQLRecordClass;
  Kind: PSQLRestServerKind=nil): TSQLRestServerStatic;
begin
  if (fStaticData=nil) and (fStaticVirtualTable=nil) then
    result := nil else
    result := GetStaticDataServerOrVirtualTable(Model.GetTableIndexExisting(aClass),Kind);
end;

function TSQLRestServer.GetStaticDataServerOrVirtualTable(aTableIndex: integer;
  Kind: PSQLRestServerKind=nil): TSQLRestServerStatic;
begin
  result := nil;
  if Kind<>nil then
    Kind^ := sMainEngine;
  if aTableIndex>=0 then begin
    if fStaticData<>nil then
      result := fStaticData[aTableIndex];
................................................................................
      result := fStaticVirtualTable[aTableIndex];
      if Kind<>nil then
        Kind^ := sVirtualTable;
    end;
  end;
end;

function TSQLRestServer.GetVirtualTable(aClass: TSQLRecordClass): TSQLRestServerStatic;
var i: integer;
begin
  result := nil;
  if fStaticVirtualTable<>nil then begin
    i := Model.GetTableIndexExisting(aClass);
    if (i>=0) and (Model.TableProps[i].Kind in IS_CUSTOM_VIRTUAL) then
      result := fStaticVirtualTable[i];
  end;
end;

function TSQLRestServer.StaticDataCreate(aClass: TSQLRecordClass;
  const aFileName: TFileName; aBinaryFile: boolean;
  aServerClass: TSQLRestServerStaticClass): TSQLRestServerStatic;
var i: integer;
begin
  result := nil;
  i := Model.GetTableIndexExisting(aClass);
  if fStaticData<>nil then
    result := fStaticData[i];
  if result<>nil then
    // class already registered -> update file name
    result.fFileName := aFileName else begin
    // class not already registered -> register now
    if aServerClass=nil then
      aServerClass := TSQLRestServerStaticInMemory; // default in-memory engine
    result := aServerClass.Create(aClass,self,aFileName,aBinaryFile);
    if length(fStaticData)<length(Model.Tables) then
      SetLength(fStaticData,length(Model.Tables));
    fStaticData[i] := result;
  end;
end;

................................................................................
      SetLength(result,P-pointer(Result)); // trim right
  end;
  if result='' then // by default, a SQLite3 query is ordered by ID
    result := 'RowID';
end;

procedure TSQLRestServer.SetNoAJAXJSON(const Value: boolean);
var i: integer;
begin
  fNoAJAXJSON := Value;
  for i := 0 to high(fStaticData) do
    if fStaticData[i]<>nil then
      fStaticData[i].NoAJAXJSON := Value; // set JSON format for static also
end;

function TSQLRestServer.InternalAdaptSQL(TableIndex: integer; var SQL: RawUTF8): TSQLRestServerStatic;
begin
  result := nil;
  if (self<>nil) and (TableIndex>=0) then begin // SQL refers to this unique table
    if fStaticData<>nil then
      // no SQLite3 module available for fStaticData[] -> we need to
      // retrieve manualy any static table from the SQL SELECT statement
      result := fStaticData[TableIndex];
................................................................................
        result := nil;
    end;
  end;
end;

function TSQLRestServer.InternalListRawUTF8(TableIndex: integer; const SQL: RawUTF8): RawUTF8;
var aSQL: RawUTF8;
    Static: TSQLRestServerStatic;
begin
  aSQL := SQL;
  Static := InternalAdaptSQL(TableIndex,aSQL);
  if Static<>nil then
     // this SQL statement is handled by direct connection, faster adaptation
    result := Static.EngineList(aSQL) else
    // complex TSQLVirtualTableJSON/External queries will rely on virtual table
    result := EngineList(SQL);
  if result='[]'#$A then
    result := '';
end;

function TSQLRestServer.InternalListJSON(Table: TSQLRecordClass; const SQL: RawUTF8): TSQLTableJSON;
var JSON: RawUTF8;
begin
  JSON := InternalListRawUTF8(Model.GetTableIndexExisting(Table),SQL);
  if JSON<>'' then
    result := TSQLTableJSON.CreateFromTables([Table],SQL,JSON) else
    result := nil;
end;

function TSQLRestServer.Retrieve(aID: integer; Value: TSQLRecord;
  ForUpdate: boolean): boolean;
var TableIndex: integer; // used by EngineRetrieve() for SQL statement caching
    Resp: RawUTF8;
    Static: TSQLRestServerStatic;
begin // this version handles locking and use fast EngineRetrieve() method
  // check parameters
  result := false;
  if Value=nil  then
    exit; // avoid GPF
  Value.fID := 0;
  if (self=nil) or (aID=0) then
    exit;
  TableIndex := Model.GetTableIndexExisting(PSQLRecordClass(Value)^);
  // try to lock before retrieval (if ForUpdate)
  if ForUpdate and not Model.Lock(TableIndex,aID) then
    exit;
  // try to retrieve existing JSON from internal cache
  Resp := fCache.Retrieve(TableIndex,aID);
  if Resp='' then begin
    // get JSON object '{...}' in Resp from corresponding EngineRetrieve() method
    Static := GetStaticDataServerOrVirtualTable(TableIndex);
    if Static<>nil then
      Resp := Static.EngineRetrieve(TableIndex,aID) else
      Resp := EngineRetrieve(TableIndex,aID);
    if Resp='' then
      exit;
  end;
  // fill Value from JSON if was correctly retrieved
  Value.FillFrom(Resp);
  result := true;
end;

function TSQLRestServer.ExecuteList(const Tables: array of TSQLRecordClass; const SQL: RawUTF8): TSQLTableJSON;
var JSON: RawUTF8;
begin
  JSON := EngineList(SQL,false);
  if JSON<>'' then
    result := TSQLTableJSON.CreateFromTables(Tables,SQL,JSON) else
    result := nil;
end;

function TSQLRestServer.UnLock(Table: TSQLRecordClass; aID: integer): boolean;
begin
  result := Model.UnLock(Table,aID);
end;

procedure TSQLRestServer.Commit(SessionID: cardinal);
var i: integer;
begin
  inherited Commit(SessionID);
  if self<>nil then
    for i := 0 to high(fStaticVirtualTable) do
    if fStaticVirtualTable[i]<>nil then
    with TSQLRestServerStaticInMemory(fStaticVirtualTable[i]) do 
      if InheritsFrom(TSQLRestServerStaticInMemory) and not CommitShouldNotUpdateFile then
        UpdateFile; // will do nothing if not Modified
end;

function TSQLRestServer.Add(Value: TSQLRecord; SendData: boolean;
  ForceID: boolean=false): integer;
var JSONValues: RawUTF8;
    Static: TSQLRestServerStatic;
begin
  if (self=nil) or (Value=nil) then begin
    result := 0;
    exit;
  end;
  if SendData then begin
    Value.ComputeFieldsBeforeWrite(self,seAdd); // update TModTime/TCreateTime fields
    if Model.Props[PSQLRecordClass(Value)^].Kind in INSERT_WITH_ID then
      ForceID := true;
    JSONValues := Value.GetJSONValues(true, // true=expanded
      (Value.fID<>0) and ForceID,soInsert);
  end else
    JSONValues := '';
  // on success, returns the new ROWID value; on error, returns 0
  Static := GetStaticDataServerOrVirtualTable(PSQLRecordClass(Value)^);
  if Static<>nil then // faster direct call
    result := Static.EngineAdd(PSQLRecordClass(Value)^,JSONValues) else
    result := EngineAdd(PSQLRecordClass(Value)^,JSONValues);
  // on success, Value.ID is updated with the new ROWID
  Value.fID := result;
  if SendData then
    fCache.Notify(PSQLRecordClass(Value)^,result,JSONValues,soInsert);
end;

function TSQLRestServer.Update(Value: TSQLRecord): boolean;
var JSONValues: RawUTF8;
    Static: TSQLRestServerStatic;
begin
  if (self=nil) or (Value=nil) or not inherited Update(Value) then begin
    result := false; // current user don't have enough right to update this record 
    exit;
  end;
  Value.ComputeFieldsBeforeWrite(self,seUpdate); // update sftModTime fields
  JSONValues := Value.GetJSONValues(true,false,soUpdate); // expanded + without ID
  fCache.Notify(Value,soUpdate); // JSONValues on update may not be enough for cache
  Static := GetStaticDataServerOrVirtualTable(PSQLRecordClass(Value)^);
  if Static<>nil then // faster direct call
    result := Static.EngineUpdate(PSQLRecordClass(Value)^,Value.fID,JSONValues) else
    result := EngineUpdate(PSQLRecordClass(Value)^,Value.fID,JSONValues);
end;

function TSQLRestServer.Delete(Table: TSQLRecordClass; ID: integer): boolean;
var Static: TSQLRestServerStatic;
begin
  if not inherited Delete(Table,ID) then begin  // call RecordCanBeUpdated()
    result := false;
    exit;
  end;
  fCache.NotifyDeletion(Table,ID);
  Static := GetStaticDataServerOrVirtualTable(Table);
  if Static<>nil then // faster direct call
    result := Static.EngineDelete(Table,ID) else
    result := EngineDelete(Table,ID);
  if result then
    // force relational database coherency (i.e. our FOREIGN KEY implementation)
    AfterDeleteForceCoherency(Table,ID);
end;

function TSQLRestServer.Delete(Table: TSQLRecordClass; const SQLWhere: RawUTF8): boolean;
var IDs: TIntegerDynArray;
    i: integer;
    Static: TSQLRestServerStatic;
begin
  result := false;
  if not InternalDelete(Table,SQLWhere,IDs) then
    exit;
  Static := GetStaticDataServerOrVirtualTable(Table);
  if Static<>nil then // faster direct call
    result := Static.EngineDeleteWhere(Table,SQLWhere,IDs) else
    result := EngineDeleteWhere(Table,SQLWhere,IDs);
  if result then
    // force relational database coherency (i.e. our FOREIGN KEY implementation)
    for i := 0 to high(IDs) do
      AfterDeleteForceCoherency(Table,IDs[i]);
end;

function TSQLRestServer.TableRowCount(Table: TSQLRecordClass): integer;
var Static: TSQLRestServerStatic;
begin
  Static := GetStaticDataServerOrVirtualTable(Table);
  if Static<>nil then // faster direct call
    result := Static.TableRowCount(Table) else
    result := inherited TableRowCount(Table);
end;

function TSQLRestServer.TableHasRows(Table: TSQLRecordClass): boolean;
var Static: TSQLRestServerStatic;
begin
  Static := GetStaticDataServerOrVirtualTable(Table);
  if Static<>nil then // faster direct call
    result := Static.TableHasRows(Table) else
    result := inherited TableHasRows(Table);
end;

function TSQLRestServer.RetrieveBlob(Table: TSQLRecordClass; aID: integer;
  const BlobFieldName: RawUTF8; out BlobData: TSQLRawBlob): boolean;
var Static: TSQLRestServerStatic;
    BlobField: PPropInfo;
begin
  result := false;
  if (self=nil) or (aID<=0) then
    exit;
  BlobField := Table.RecordProps.BlobFieldPropFromRawUTF8(BlobFieldName);
  if BlobField=nil then
    exit;
  Static := GetStaticDataServerOrVirtualTable(Table);
  if Static<>nil then // faster direct call
     result := Static.EngineRetrieveBlob(Table,aID,BlobField,BlobData) else
     result := EngineRetrieveBlob(Table,aID,BlobField,BlobData);
end;

function TSQLRestServer.UpdateBlob(Table: TSQLRecordClass; aID: integer;
  const BlobFieldName: RawUTF8; const BlobData: TSQLRawBlob): boolean;
var Static: TSQLRestServerStatic;
    BlobField: PPropInfo;
begin
  result := false;
  if (self=nil) or (aID<=0) or not RecordCanBeUpdated(Table,aID,seUpdate) then
    exit;
  BlobField := Table.RecordProps.BlobFieldPropFromRawUTF8(BlobFieldName);
  if BlobField=nil then
    exit;
  Static := GetStaticDataServerOrVirtualTable(Table);
  if Static<>nil then // faster direct call
     result := Static.EngineUpdateBlob(Table,aID,BlobField,BlobData) else
     result := EngineUpdateBlob(Table,aID,BlobField,BlobData);
end;

function TSQLRestServer.UpdateBlobFields(Value: TSQLRecord): boolean;
var Static: TSQLRestServerStatic;
begin

  if (Value=nil) or (Value.fID<=0) then
    result := false else begin
    Static := GetStaticDataServerOrVirtualTable(PSQLRecordClass(Value)^);
    if Static<>nil then // faster direct call
      result := Static.UpdateBlobFields(Value) else
      result := inherited UpdateBlobFields(Value);
  end;
end;

function TSQLRestServer.RetrieveBlobFields(Value: TSQLRecord): boolean;
var Static: TSQLRestServerStatic;
begin
  if Value=nil then
    result := false else begin
    Static := GetStaticDataServerOrVirtualTable(PSQLRecordClass(Value)^);
    if Static<>nil then // faster direct call
      result := Static.RetrieveBlobFields(Value) else
      result := inherited RetrieveBlobFields(Value);
  end;
................................................................................

function TSQLRestServer.AfterDeleteForceCoherency(Table: TSQLRecordClass;
  aID: integer): boolean;
var T: integer;
    Tab: TSQLRecordClass;
    Where: PtrUInt;
    RecRef: TRecordReference;
    Static: TSQLRestServerStatic;
    W: RawUTF8;
begin
  result := true; // success if no property found
  {$ifndef CPU64}
  Where := 0; // make compiler happy
  {$endif}
  RecRef := RecordReference(Model,Table,aID);
................................................................................
    else continue;
    end;
    // set Field=0 where Field references aID
    UInt32ToUTF8(Where,W);
    Tab := Model.Tables[TableIndex];
    Static := GetStaticDataServerOrVirtualTable(Tab);
    if Static<>nil then // fast direct call

       result := Static.EngineUpdateField(Tab,FieldType.Name,'0',FieldType.Name,W) else

       result := EngineUpdateField(Tab,FieldType.Name,'0',FieldType.Name,W);
  end;
end;

function TSQLRestServer.CreateSQLMultiIndex(Table: TSQLRecordClass;
  const FieldNames: array of RawUTF8; Unique: boolean; IndexName: RawUTF8=''): boolean;
var SQL: RawUTF8;
    i, TableIndex: integer;
    Props: TSQLRecordProperties;
    Static: TSQLRestServerStatic;
begin
  result := false;
  if (Self=nil) or InheritsFrom(TSQLRestServerStatic) or (high(FieldNames)<0) then
    exit; // avoid endless loop for TSQLRestServerStatic with no overriden method
  TableIndex := Model.GetTableIndexExisting(Table);
  if fStaticVirtualTable<>nil then begin
    Static := fStaticVirtualTable[TableIndex];
    if Static<>nil then begin
      if not Static.InheritsFrom(TSQLRestServerStaticInMemory) then
         // will try to create an index on the static table (e.g. for external DB)
         result := Static.CreateSQLMultiIndex(Table,FieldNames,Unique,IndexName);
      exit;
    end;
  end;
  if (high(FieldNames)=0) and IsRowID(pointer(FieldNames[0])) then begin
    result := true; // SQLite3 has always its ID/RowID primary key indexed
................................................................................
          SQL := Call.InBody;
        SQLisSelect := isSelect(pointer(SQL));
        if (SQL<>'') and
          (SQLisSelect or (reSQL in Call.RestAccessRights^.AllowRemoteExecute)) then begin
          // no user check for SELECT: see TSQLAccessRights.GET comment
          Static := Server.InternalAdaptSQL(
            Server.Model.GetTableIndexFromSQLSelect(SQL,false),SQL);
          if Static<>nil then
            TableEngine := Static;
          Call.OutBody := TableEngine.EngineList(SQL);


          // security note: only first statement is run by EngineList()
          if Call.OutBody<>'' then begin // got JSON list '[{...}]' ?
            Call.OutStatus := HTML_SUCCESS;  // 200 OK
            if not SQLisSelect then
              inc(Server.fStats.fModified);
          end;
        end;
................................................................................
            if Server.Model.Lock(TableIndex,TableID) then
              Method := mGET; // mark successfully locked
        if Method<>mLOCK then
          if URIBlobFieldName<>'' then begin
            // GET ModelRoot/TableName/TableID/BlobFieldName: retrieve BLOB field content
            Blob := Table.RecordProps.BlobFieldPropFromRawUTF8(URIBlobFieldName);
            if Blob<>nil then begin

              if TableEngine.EngineRetrieveBlob(Table,TableID,Blob,TSQLRawBlob(Call.OutBody)) then begin
                Call.OutHead := HEADER_CONTENT_TYPE+
                  GetMimeContentType(pointer(Call.OutBody),Length(Call.OutBody));
                Call.OutStatus := HTML_SUCCESS; // 200 OK
              end;
            end;
          end else begin
            // GET ModelRoot/TableName/TableID: retrieve a member content, JSON encoded
            Call.OutBody := Server.fCache.Retrieve(TableIndex,TableID);
            if Call.OutBody='' then begin
              // get JSON object '{...}'


              Call.OutBody := TableEngine.EngineRetrieve(TableIndex,TableID);
              // cache if expected
              Server.fCache.Notify(TableIndex,TableID,Call.OutBody,soSelect);
            end;
            if Call.OutBody<>'' then // if something was found
              Call.OutStatus := HTML_SUCCESS; // 200 OK
          end;
      end else
................................................................................
            if SameTextU(SQLDir,'DESC') then
              SQLSort := SQLSort+' DESC'; // allow DESC, default is ASC
            SQLWhere := SQLWhere+' ORDER BY '+SQLSort;
          end;
          SQLWhere := trim(SQLWhere);
          if (SQLResults<>0) and not ContainsUTF8(pointer(SQLWhere),'LIMIT ') then begin
            if (Server.URIPagingParameters.SendTotalRowsCountFmt<>nil) then begin
              ResultList := Server.InternalListJSON(Table,
                Server.Model.TableProps[TableIndex].SQLFromSelectWhere('Count(*)',SQLWhere));
              if ResultList<>nil then begin
                SQLTotalRowsCount := ResultList.GetAsInteger(1,0);
                ResultList.Free;
              end;
            end;
            SQLWhere := FormatUTF8('% LIMIT % OFFSET %',[SQLWhere,SQLResults,SQLStartIndex]);
................................................................................
      end;
    end else
    // here, Table<>nil and TableIndex in [0..MAX_SQLTABLES-1]
    if not (TableIndex in Call.RestAccessRights^.POST) then // check User
      Call.OutStatus := HTML_NOTALLOWED else
    if TableID<0 then begin
      // ModelRoot/TableName with possible JSON SentData: create a new member
      TableID := TableEngine.EngineAdd(Table,Call.InBody);
      if TableID<>0 then begin
        Call.OutStatus := HTML_CREATED; // 201 Created
        Call.OutHead := 'Location: '+URI+'/'+
          {$ifndef ENHANCEDRTL}Int32ToUtf8{$else}IntToStr{$endif}(TableID);
        Server.fCache.Notify(TableIndex,TableID,Call.InBody,soInsert);
        inc(Server.fStats.fModified);
      end;
................................................................................
      if not Server.RecordCanBeUpdated(Table,TableID,seUpdate,@CustomErrorMsg) then
        Call.OutStatus := HTML_NOTMODIFIED else begin
        OK := false;
        if URIBlobFieldName<>'' then begin
          // PUT ModelRoot/TableName/TableID/BlobFieldName: update BLOB field content
          Blob := Table.RecordProps.BlobFieldPropFromRawUTF8(URIBlobFieldName);
          if Blob<>nil then
            OK := TableEngine.EngineUpdateBlob(Table,TableID,Blob,Call.InBody);
        end else begin
          // ModelRoot/TableName/TableID with JSON SentData: update a member
          OK := TableEngine.EngineUpdate(Table,TableID,Call.InBody);
          if OK then
            Server.fCache.NotifyDeletion(TableIndex,TableID); // flush (no CreateTime in JSON)
        end;
        if OK then begin
          Call.OutStatus := HTML_SUCCESS; // 200 OK
          inc(Server.fStats.fModified);
        end;
................................................................................
        repeat
          UrlDecodeValue(Parameters,'SETNAME=',SQLSelect);
          UrlDecodeValue(Parameters,'SET=',SQLDir);
          UrlDecodeValue(Parameters,'WHERENAME=',SQLSort);
          UrlDecodeValue(Parameters,'WHERE=',SQLWhere,@Parameters);
        until Parameters=nil;
        if (SQLSelect<>'') and (SQLDir<>'') and (SQLSort<>'') and (SQLWhere<>'') then

          if TableEngine.EngineUpdateField(Table,SQLSelect,SQLDir,SQLSort,SQLWhere) then begin
            Call.OutStatus := HTML_SUCCESS; // 200 OK
            inc(Server.fStats.fModified);
          end;
      end;
  end;
  mDELETE:
    if Table<>nil then
      if TableID>0 then
        // ModelRoot/TableName/TableID to delete a member
        if not (TableIndex in Call.RestAccessRights^.DELETE) then // check User
          Call.OutStatus := HTML_NOTALLOWED else
        if not Server.RecordCanBeUpdated(Table,TableID,seDelete,@CustomErrorMsg) then
          Call.OutStatus := HTML_NOTMODIFIED else begin
          if TableEngine.EngineDelete(Table,TableID) and
             Server.AfterDeleteForceCoherency(Table,TableID) then begin
            Call.OutStatus := HTML_SUCCESS; // 200 OK
            Server.fCache.NotifyDeletion(TableIndex,TableID);
            inc(Server.fStats.fModified);
          end;
        end else
      if Parameters<>nil then
................................................................................
              end;
              break;
            end;
          until Parameters=nil;
        end;
  mBEGIN: begin      // BEGIN TRANSACTION
    // TSQLVirtualTableJSON/External will rely on SQLite3 module
    // and also TSQLRestServerStaticInMemory, since COMMIT/ROLLBACK have Static=nil
    if Server.TransactionBegin(Table,Session) then begin
      if (Static<>nil) and (StaticKind=sVirtualTable) then
        Static.TransactionBegin(Table,Session) else
      if (Static=nil) and (Server.fTransactionTable<>nil) then begin
        Static := Server.StaticVirtualTable[Server.fTransactionTable];
        if Static<>nil then
          Static.TransactionBegin(Table,Session);
................................................................................
end;

procedure TSQLRestServer.Batch(Ctxt: TSQLRestServerURIContext);
var EndOfObject: AnsiChar;
    wasString, OK: boolean;
    TableName, Value, ErrMsg: RawUTF8;
    URIMethod, RunningBatchURIMethod: TSQLURIMethod;
    RunningBatchStatic: TSQLRestServerStatic; { TODO: allow nested batch between tables? }
    Sent, Method, MethodTable: PUTF8Char;
    AutomaticTransactionPerRow, RowCountForCurrentTransaction: cardinal;
    i, ID, Count: integer;
    Results: TIntegerDynArray;
    RunTable: TSQLRecordClass;

    RunStatic: TSQLRestServerStatic;
    RunStaticKind: TSQLRestServerKind;
begin
  if Ctxt.Method<>mPUT then begin
    Ctxt.Error('PUT only');
    exit;
  end;
  Sent := pointer(Ctxt.Call.InBody);
................................................................................
        Ctxt.Error('Missing CMD');
        exit;
      end;
      MethodTable := PosChar(Method,'@');
      if MethodTable=nil then begin // e.g. '{"Table":[...,"POST":{object},...]}'
        RunTable := Ctxt.Table;
        RunStatic := Ctxt.Static;

        RunStaticKind := Ctxt.StaticKind;
      end else begin                // e.g. '[...,"POST@Table":{object},...]'
        i := Model.GetTableIndex(MethodTable+1);
        if i<0 then begin
          Ctxt.Error('Unknown @Table');
          exit;
        end;
        RunTable := Model.Tables[i];
        RunStatic := GetStaticDataServerOrVirtualTable(i,@RunStaticKind);
      end;
      if Count>=length(Results) then
        SetLength(Results,Count+256+Count shr 3);
      // get CRUD method (ignoring @ char if appended after method name)
      if IdemPChar(Method,'DELETE') then
        URIMethod := mDELETE else
      if IdemPChar(Method,'POST') then
................................................................................
      mDELETE: begin // '{"Table":[...,"DELETE":ID,...]}' or '[...,"DELETE@Table":ID,...]'
        ID := GetInteger(GetJSONField(Sent,Sent,@wasString,@EndOfObject));
        if (ID<=0) or wasString or
           not RecordCanBeUpdated(RunTable,ID,seDelete,@ErrMsg) then begin
          Ctxt.Error(ErrMsg,HTML_NOTMODIFIED);
          exit;
        end;
        if RunStatic<>nil then
          OK := RunStatic.EngineDelete(RunTable,ID) else
          OK := EngineDelete(RunTable,ID);
        if OK then begin
          fCache.NotifyDeletion(RunTable,ID);
          if (RunningBatchStatic<>nil) or
             AfterDeleteForceCoherency(RunTable,ID) then
            Results[Count] := HTML_SUCCESS; // 200 OK
        end;
      end;
................................................................................
      mPOST: begin // '{"Table":[...,"POST":{object},...]}' or '[...,"POST@Table":{object},...]'
        Value := JSONGetObject(Sent,nil,EndOfObject);
        if (Sent=nil) or
           not RecordCanBeUpdated(RunTable,0,seAdd,@ErrMsg)  then begin
          Ctxt.Error(ErrMsg,HTML_NOTMODIFIED);
          exit;
        end;
        if RunStatic<>nil then
          ID := RunStatic.EngineAdd(RunTable,Value) else
          ID := EngineAdd(RunTable,Value);
        Results[Count] := ID;
        fCache.Notify(RunTable,ID,Value,soInsert);
      end;
      mPUT: begin // '{"Table":[...,"PUT":{object},...]}' or '[...,"PUT@Table":{object},...]'
        Value := JSONGetObject(Sent,@ID,EndOfObject);
        if (Sent=nil) or (Value='') then
          exit;
        if RunStatic<>nil then
          OK := RunStatic.EngineUpdate(RunTable,ID,Value) else
          OK := EngineUpdate(RunTable,ID,Value);
        if OK then begin
          Results[Count] := HTML_SUCCESS; // 200 OK
          fCache.NotifyDeletion(RunTable,ID); // Value does not have CreateTime e.g.
          // or may be complete -> update won't work as expected -> delete from cache
        end;
      end;
      else exit; // unknown method
................................................................................
end;

function TSQLRestServer.CacheWorthItForTable(aTableIndex: cardinal): boolean;
begin
  if self=nil then
    result := false else
    result := (aTableIndex>=cardinal(length(fStaticData))) or
      (not fStaticData[aTableIndex].InheritsFrom(TSQLRestServerStaticInMemory));
end;

procedure TSQLRestServer.BeginCurrentThread(Sender: TThread);
var i: integer;
    CurrentThreadId: cardinal;
begin
  InterlockedIncrement(fStats.fCurrentThreadCount);
................................................................................
    if (aEvent=seUpdateBlob) and Assigned(OnBlobUpdateEvent) then
      result := OnBlobUpdateEvent(self,seUpdate,aTable,aID,aIsBlobFields^) else
      result := true else
  if Assigned(OnUpdateEvent) then
    result := OnUpdateEvent(self,aEvent,aTable,aID) else
    result := true; // true on success, false if error (but action continues)
end;




























































































function CurrentServiceContext: TServiceRunningContext;
begin
  result := ServiceContext;
end;


................................................................................
  if (self=nil) or (Freq=0) then
    result := '0' else
    result := MicroSecToString((ProcessTimeCounter*(1000*1000))div Freq);
end;
{$endif}


{ TSQLRestServerStaticRecordBased }

function TSQLRestServerStaticRecordBased.EngineAdd(Table: TSQLRecordClass;
  const SentData: RawUTF8): integer;
var Rec: TSQLRecord;
begin
  result := 0; // mark error
  if (self=nil) or (Table<>fStoredClass) then
    exit;
  Rec := fStoredClass.Create;
  try
    Rec.FillFrom(SentData);
    Lock(true);
    try
      result := AddOne(Rec,Rec.fID>0);
    finally
      UnLock;
    end;
  finally
    if result<=0 then
      Rec.Free; // on success, will be freed by fValue TObjectList
  end;
end;

function TSQLRestServerStaticRecordBased.EngineUpdate(
  Table: TSQLRecordClass; ID: integer; const SentData: RawUTF8): boolean;
var Rec: TSQLRecord;
begin
  // this implementation won't handle partial fields update (e.g. BatchUpdate
  // after FillPrepare) - but TSQLRestServerStaticInMemory.EngineUpdate will

  if (self=nil) or (Table<>fStoredClass) then begin
    result := false; // mark error
    exit;
  end;
  Lock(true);
  try
    Rec := fStoredClass.Create;
    try
      Rec.FillFrom(SentData);

      result := UpdateOne(Rec);
    finally
      Rec.Free;
    end;
  finally
    UnLock;
  end;
end;

function TSQLRestServerStaticRecordBased.UpdateOne(ID: integer;
  const Values: TSQLVarDynArray): boolean;
var Rec: TSQLRecord;
begin
  if self=nil then begin
    result := false; // mark error
    exit;
  end;
  Lock(true);
  try
    Rec := fStoredClass.Create;
    try
      Rec.SetFieldSQLVars(Values);
      Rec.fID := ID;
      result := UpdateOne(Rec);
    finally
      Rec.Free;
    end;
  finally
    UnLock;
  end;
end;


{ TSQLRestServerStatic }

function TSQLRestServerStaticInMemory.AddOne(Rec: TSQLRecord; ForceID: boolean): integer;
var ndx,i: integer;
begin
  if (self=nil) or (Rec=nil) then begin
    result := -1; // mark error
    exit;
  end;
  if ForceID then
................................................................................
      exit;
    end;
  fModified := true;
  if Owner<>nil then
     Owner.InternalUpdateEvent(seAdd,PSQLRecordClass(Rec)^,result,nil);
end;

function TSQLRestServerStaticInMemory.UniqueFieldsUpdateOK(aRec: TSQLRecord; aUpdateIndex: integer): boolean;
var i,ndx: integer;
begin
  if fUniqueFields<>nil then begin
    result := false;
    with fUniqueFields do
      for i := 0 to Count-1 do begin
        ndx := TListFieldHash(List[i]).Find(aRec);
................................................................................
        if (ndx>=0) and (ndx<>aUpdateIndex) then
          exit; // duplicate found at another entry
      end;
  end;
  result := true;
end;

function TSQLRestServerStaticInMemory.UniqueFieldHash(aFieldIndex: integer): TListFieldHash;
var i: integer;
begin
  if (fUniqueFields<>nil) and
     (cardinal(aFieldIndex)<cardinal(fStoredClassRecordProps.Fields.Count)) then
    with fUniqueFields do
      for i := 0 to Count-1 do begin
        result := List[i];
        if result.FieldIndex=aFieldIndex then
          exit;
      end;
  result := nil;
end;

constructor TSQLRestServerStaticInMemory.Create(aClass: TSQLRecordClass; aServer: TSQLRestServer;
  const aFileName: TFileName = ''; aBinaryFile: boolean=false);
var JSON: RawUTF8;
    Stream: TStream;
    F: integer;
begin
  inherited Create(aClass,aServer,aFileName,aBinaryFile);
  if (fStoredClassProps<>nil) and (fStoredClassProps.Kind in INSERT_WITH_ID) then
    raise EModelException.CreateFmt('%s: %s virtual table can''t be static',
      [fStoredClassRecordProps.SQLTableName,aClass.ClassName]);
  fBinaryFile := aBinaryFile;
  fValue := TObjectList.Create;
  fSearchRec := fStoredClass.Create;
  fIDSorted := true; // sorted by design of this class (may change in children)
  if (ClassType<>TSQLRestServerStaticInMemory) and (fStoredClassProps<>nil) then
    with fStoredClassProps do begin // used by AdaptSQLForEngineList() method
      fBasicUpperSQLSelect[false] := UpperCase(SQL.SelectAllWithRowID);
      SetLength(fBasicUpperSQLSelect[false],length(fBasicUpperSQLSelect[false])-1); // trim right ';'
      fBasicUpperSQLSelect[true] := StringReplaceAll(fBasicUpperSQLSelect[false],' ROWID,',' ID,');
    end;
  if not IsZero(fIsUnique) then begin
    fUniqueFields := TObjectList.Create;
................................................................................
      finally
        Stream.Free;
      end;
    end;
  end;
end;

function TSQLRestServerStaticInMemory.EngineDelete(Table: TSQLRecordClass; ID: integer): boolean;
var i,F: integer;
begin
  if (self=nil) or (ID<=0) or (Table<>fStoredClass) or
     not RecordCanBeUpdated(Table,ID,seDelete) then begin
    result := false;
    exit;
  end;
  Lock(True);
  try
    i := IDToIndex(ID);
    if i<0 then
      result := false else begin
      if fUniqueFields<>nil then
        for F := 0 to fUniqueFields.Count-1 do
          TListFieldHash(fUniqueFields.List[F]).Invalidate;
      if Owner<>nil then
         Owner.InternalUpdateEvent(seDelete,Table,ID,nil); // notify BEFORE deletion
      fValue.Delete(i); // TObjectList.Delete() will Free record
      fModified := true;
      result := true;
    end;
  finally
    UnLock;
  end;
end;

function TSQLRestServerStaticInMemory.EngineDeleteWhere(Table: TSQLRecordClass;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
var ndx: TIntegerDynArray;
    n,i: integer;
begin // RecordCanBeUpdated() has already been called
  result := false;
  n := length(IDs);
  SetLength(ndx,n);
  dec(n);
  Lock(True);
  try
    for i := 0 to n do begin
      ndx[i] := IDToIndex(IDs[i]);
      if ndx[i]<0 then
        exit;
    end;
    if fUniqueFields<>nil then
      for i := 0 to fUniqueFields.Count-1 do
        TListFieldHash(fUniqueFields.List[i]).Invalidate;
    if Owner<>nil then
      for i := 0 to n do
        Owner.InternalUpdateEvent(seDelete,Table,IDs[i],nil); // notify BEFORE deletion
    QuickSortInteger(pointer(ndx),0,n); // deletion a bit faster in reverse order
    for i := n downto 0 do
      fValue.Delete(ndx[i]);
    fModified := true;
    result := true;
  finally
    UnLock;
  end;
end;

destructor TSQLRestServerStaticInMemory.Destroy;
begin
  UpdateFile;
  fValue.Free; // TObjectList.Destroy will free all stored TSQLRecord instances
  fUniqueFields.Free;
  fSearchRec.Free;
  inherited Destroy;
end;

function TSQLRestServerStaticInMemory.GetCount: integer;
begin
  if Self<>nil then
    result := fValue.Count else
    result := 0;
end;

function TSQLRestServerStaticInMemory.GetID(Index: integer): integer;
begin
  with fValue do
    if (self=nil) or (cardinal(Index)>=cardinal(Count)) then
      result := 0 else
      result := TSQLRecord(List[Index]).fID;
end;

function TSQLRestServerStaticInMemory.GetItem(Index: integer): TSQLRecord;
begin
  if self<>nil then
    with fValue do
      if cardinal(Index)>=cardinal(Count) then
        raise EORMException.Create('GetItem out of range') else
        result := List[Index] else
    result := nil;
end;

procedure TSQLRestServerStaticInMemory.GetJSONValuesEvent(aDest: pointer;
  aRec: TSQLRecord; aIndex: integer);
var W: TJSONSerializer absolute aDest;
begin
  aRec.GetJSONValues(W);
  W.Add(',');
end;

procedure TSQLRestServerStaticInMemory.AddIntegerDynArrayEvent(
  aDest: pointer; aRec: TSQLRecord; aIndex: integer);
var Ints: TList absolute aDest;
begin
  Ints.Add(pointer(aIndex));
end;

procedure TSQLRestServerStaticInMemory.DoNothingEvent(aDest: pointer; aRec: TSQLRecord; aIndex: integer);
begin
end;

function TSQLRestServerStaticInMemory.AdaptSQLForEngineList(var SQL: RawUTF8): boolean;
var P: PUTF8Char;
    Prop: RawUTF8;
    WithoutRowID: boolean;
begin
  result := inherited AdaptSQLForEngineList(SQL);
  if result then
    exit; // 'select * from table'
................................................................................
  if PWord(P)^=ord(')')+ord(':')shl 8 then
    inc(P,2); // ignore :(...): parameter
  P := GotoNextNotSpace(P);
  if (P^ in [#0,';']) or IdemPChar(P,'LIMIT 1;') then
    result := true; // properly ended the WHERE clause as 'FIELDNAME=value'
end;

function TSQLRestServerStaticInMemory.FindWhereEqual(WhereField: integer;
  const WhereValue: RawUTF8; OnFind: TFindWhereEqualEvent; Dest: pointer;
  FoundLimit,FoundOffset: integer): PtrInt;
var i, ndx: integer;
    aValue: PtrInt;
    err, currentRow: integer;
    P: TSQLPropInfo;
    Hash: TListFieldHash;
................................................................................
        inc(result);
        if result>=FoundLimit then
          exit;
      end;
  end;
end;

function TSQLRestServerStaticInMemory.GetJSONValues(Stream: TStream;
  Expand, withID: boolean; const Fields: TSQLFieldBits;
  WhereField: integer; const WhereValue: RawUTF8;
  FoundLimit,FoundOffset: integer): PtrInt;
var i,KnownRowsCount: integer;
    W: TJSONSerializer;
begin // exact same format as TSQLTable.GetJSONValues()
  result := 0;
................................................................................
    end;
    W.EndJSONObject(KnownRowsCount,result);
  finally
    W.Free;
  end;
end;

function TSQLRestServerStaticInMemory.IDToIndex(ID: integer): integer;
var L, R, cmp: integer;
begin
  if self<>nil then
  with fValue do begin
    R := Count-1;
    if fIDSorted and (R>=8) then begin
      // IDs are sorted -> use fast binary search algorithm
................................................................................
      for result := 0 to R do
        if TSQLRecord(List[result]).fID=ID then
          exit;
  end;
  result := -1;
end;

function TSQLRestServerStaticInMemory.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8;
// - GetJSONValues/FindWhereEqual will handle basic REST commands (not all SQL)
// only valid SQL command is "SELECT Field1,Field2 FROM Table WHERE ID=120;",
// i.e one Table SELECT with one optional "WHERE fieldname = value" statement
// - handle also basic "SELECT Count(*) FROM TableName;" SQL statement
// Note: this is sufficient for OneFieldValue() and MultiFieldValue() to work
var MS: TRawByteStringStream;
................................................................................
end;
begin
  ResCount := 0;
  if self=nil then begin
    result := '';
    exit;
  end;
  Lock(false);
  try
    if IdemPropNameU(fBasicSQLCount,SQL) then
      SetCount(TableRowCount(fStoredClass)) else
    if IdemPropNameU(fBasicSQLHasRows[false],SQL) or
       IdemPropNameU(fBasicSQLHasRows[true],SQL) then
      if TableRowCount(fStoredClass)=0 then begin
        result := '{"fieldCount":1,"values":["RowID"]}'#$A;
................................................................................
            // was "SELECT Count(*) FROM TableName WHERE ..."
            SetCount(FindWhereEqual(WhereField,WhereValue,DoNothingEvent,nil,0,0)) else
            // invalid "SELECT FROM Table" ?
            exit else begin
          // save rows as JSON, with appropriate search according to Where* arguments
          MS := TRawByteStringStream.Create;
          try
            ResCount := GetJSONValues(MS,ForceAJAX or (not NoAJAXJSON),
              withID,Fields,WhereField,WhereValue,FoundLimit,FoundOffset);
            result := MS.DataString;
          finally
            MS.Free;
          end;
        end;
      finally
        Free;
      end;
    end;
  finally
    UnLock;
  end;
  if ReturnedRowCount<>nil then
    ReturnedRowCount^ := ResCount;
end;

procedure TSQLRestServerStaticInMemory.LoadFromJSON(const aJSON: RawUTF8);
begin
  LoadFromJSON(Pointer(aJSON),length(aJSON));
end;

procedure TSQLRestServerStaticInMemory.LoadFromJSON(JSONBuffer: PUTF8Char; JSONBufferLen: integer);
  function IsSorted(U: PPUTF8Char; RowCount, FieldCount: integer): boolean;
  var i, aID, lastID: integer;
  begin
    result := false;
    lastID := 0;
    for i := 1 to RowCount do begin
      aID := GetInteger(U^);
................................................................................
    // create TSQLRecord instances with data from T
    T.ToObjectList(fValue,fStoredClass);
  finally
    T.Free;
  end;
end;

procedure TSQLRestServerStaticInMemory.SaveToJSON(Stream: TStream; Expand: Boolean);
begin
  if self<>nil then
    GetJSONValues(Stream,Expand,true,ALL_FIELDS,-1,'',0,0);
end;

function TSQLRestServerStaticInMemory.SaveToJSON(Expand: Boolean): RawUTF8;
var MS: TRawByteStringStream;
begin
  if self=nil then
    result := '' else begin
    MS := TRawByteStringStream.Create;
    try
      SaveToJSON(MS,Expand);
................................................................................
    finally
      MS.Free;
    end;
  end;
end;

const
  TSQLRESTSERVERSTATICINMEMORY_MAGIC = $A5ABA5A5;

function TSQLRestServerStaticInMemory.LoadFromBinary(Stream: TStream): boolean;
var R: TFileBufferReader;
    MS: TMemoryStream;
    n, i, f: integer;
    FieldNames: TRawUTF8DynArray;
    IDs: TIntegerDynArray;
    P: PAnsiChar;
    aRecord: TSQLRecord;
    FieldTypes: array[0..MAX_SQLFIELDS-1] of TSQLFieldType;
begin
  result := false;
  if self=nil then
    exit;
  MS := StreamUnSynLZ(Stream,TSQLRESTSERVERSTATICINMEMORY_MAGIC);
  if MS<>nil then
  with fStoredClassRecordProps do
  try
    // check header: expect same exact RTTI
    R.OpenFrom(MS.Memory,MS.Size);
    if (R.ReadRawUTF8<>RawUTF8(ClassName)) or
       (R.ReadRawUTF8<>SQLTableName) or
................................................................................
    Result := true;
  finally
    R.Close;
    MS.Free;
  end;
end;

function TSQLRestServerStaticInMemory.SaveToBinary(Stream: TStream): integer;
var W: TFileBufferWriter;
    MS: THeapMemoryStream;
    IDs: TIntegerDynArray;
    FieldNames: TRawUTF8DynArray;
    i, f: integer;
begin
  result := 0;
................................................................................
    W.WriteVarUInt32Array(IDs,Count,wkSorted); // efficient ID storage
    // write content, grouped by field (for better compression)
    for f := 0 to Fields.Count-1 do
      with Fields.List[f], fValue do
        for i := 0 to Count-1 do
          GetBinary(TSQLRecord(List[i]),W);
    W.Flush;
    result := StreamSynLZ(MS,Stream,TSQLRESTSERVERSTATICINMEMORY_MAGIC);
  finally
    W.Free;
    MS.Free;
  end;
end;

function TSQLRestServerStaticInMemory.EngineRetrieve(TableModelIndex, ID: integer): RawUTF8;
var i: integer;
begin // TableModelIndex is not usefull here
  Lock(false);
  try
    i := IDToIndex(ID);
    if i<0 then
      result := '' else
      result := TSQLRecord(fValue.List[i]).GetJSONValues(true,true,soSelect);
  finally
    UnLock;
  end;
end;

function TSQLRestServerStaticInMemory.GetOne(aID: integer): TSQLRecord;
var i: integer;
begin
  i := IDToIndex(aID);
  if i<0 then
    result := nil else begin
    result := fStoredClass.Create;
    CopyObject(fValue.List[i],Result);
    result.fID := aID;
  end;
end;

function TSQLRestServerStaticInMemory.EngineUpdate(Table: TSQLRecordClass; ID: integer;
  const SentData: RawUTF8): boolean;
var i: integer;
    Orig,Rec: TSQLRecord;
begin
  // this implementation will handle partial fields update (e.g.
  // FillPrepare+BatchUpdate or TSQLRestServerRemoteDB.UpdateField)
  // but TSQLRestServerStaticRecordBased.EngineUpdate won't
  result := false;
  if (self=nil) or (Table<>fStoredClass) then
    exit;
  if SentData='' then begin
    result := True;
    exit;
  end;
  Lock(true);
  try
    i := IDToIndex(ID);
    if (i<0) or not RecordCanBeUpdated(Table,ID,seUpdate) then
      exit;
    if fUniqueFields<>nil then begin
      Orig := TSQLRecord(fValue.List[i]);
      Rec := Orig.CreateCopy; // copy since can be a partial update
      Rec.FillFrom(SentData); // overwrite updated properties
      if not UniqueFieldsUpdateOK(Rec,i) then begin
        Rec.Free; // stored false property duplicated value -> error
................................................................................
      TSQLRecord(fValue.List[i]) := Rec; // update item in list
    end else
      // direct in-place partial update
      TSQLRecord(fValue.List[i]).FillFrom(SentData);
    fModified := true;
    result := true;
    if Owner<>nil then
      Owner.InternalUpdateEvent(seUpdate,Table,ID,nil);
  finally
    UnLock;
  end;
end;

function TSQLRestServerStaticInMemory.UpdateOne(Rec: TSQLRecord): boolean;
var i: integer;
begin
  result := false;
  if (Rec=nil) or (PSQLRecordClass(Rec)^<>fStoredClass) or (Rec.fID<=0) then
    exit;
  Lock(true);
  try
    i := IDToIndex(Rec.fID);
    if (i<0) or not RecordCanBeUpdated(fStoredClass,Rec.fID,seUpdate) then
      exit;
    if (fUniqueFields<>nil) and not UniqueFieldsUpdateOK(Rec,i) then
      exit; // stored false property duplicated value -> error
    CopyObject(Rec,fValue.List[i]);
    fModified := true;
    result := true;
    if Owner<>nil then
      Owner.InternalUpdateEvent(seUpdate,fStoredClass,Rec.fID,nil);
  finally
    UnLock;
  end;
end;

function TSQLRestServerStaticInMemory.UpdateOne(ID: integer;
  const Values: TSQLVarDynArray): boolean;
var i: integer;
    Orig,Rec: TSQLRecord;
begin
  result := false;
  if ID<=0 then
    exit;
................................................................................
    exit;
  fModified := true;
  result := true;
  if Owner<>nil then
    Owner.InternalUpdateEvent(seUpdate,fStoredClass,ID,nil);
end;

function TSQLRestServerStaticInMemory.EngineRetrieveBlob(Table: TSQLRecordClass;
  aID: integer; BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
var i: integer;
begin
  result := false;
  if (self=nil) or (Table<>fStoredClass) or not BlobField^.IsBlob then

    exit;
  Lock(false);
  try
    i := IDToIndex(aID);
    if i<0 then
      exit;
    // get result blob directly from RTTI property description
    GetLongStrProp(fValue.List[i],BlobField,RawByteString(BlobData));
    result := true;
  finally
    UnLock;
  end;
end;

function TSQLRestServerStaticInMemory.RetrieveBlobFields(Value: TSQLRecord): boolean;
var i,f: integer;
begin
  result := false;
  if (Value<>nil) and (Value.fID>0) and (PSQLRecordClass(Value)^=fStoredClass) then
  with Value.RecordProps do
  if BlobFields<>nil then begin
    Lock(false);
    try
      i := IDToIndex(Value.fID);
      if i<0 then
        exit;
      for f := 0 to high(BlobFields) do
        BlobFields[f].CopyValue(fValue.List[i],Value);
      result := true;
    finally
      UnLock;
    end;
  end;
end;

function TSQLRestServerStaticInMemory.EngineUpdateBlob(Table: TSQLRecordClass;
  aID: integer; BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean;
var i: integer;
    AffectedField: TSQLFieldBits;
begin
  result := false;
  if (self=nil) or  (Table<>fStoredClass) or not BlobField^.IsBlob then

    exit;
  Lock(true);
  try
    i := IDToIndex(aID);
    if (i<0) or not RecordCanBeUpdated(Table,aID,seUpdate) then
      exit;
    // set blob value directly from RTTI property description
    SetLongStrProp(fValue.List[i],BlobField,BlobData);
    if Owner<>nil then begin
      fStoredClassRecordProps.FieldIndexsFromBlobField(BlobField,AffectedField);
      Owner.InternalUpdateEvent(seUpdateBlob,fStoredClass,aID,@AffectedField);
    end;
    result := true;
  finally
    UnLock;
  end;
end;

function TSQLRestServerStaticInMemory.UpdateBlobFields(Value: TSQLRecord): boolean;
var i,f: integer;
begin
  result := false;
  if (Value<>nil) and (Value.fID>0) and (PSQLRecordClass(Value)^=fStoredClass) then
  with Value.RecordProps do
  if BlobFields<>nil then begin
    Lock(true);
    try
      i := IDToIndex(Value.fID);
      if (i<0) or not RecordCanBeUpdated(Table,Value.fID,seUpdate) then
        exit;
      for f := 0 to high(BlobFields) do
        BlobFields[f].CopyValue(Value,fValue.List[i]);
      if Owner<>nil then
        Owner.InternalUpdateEvent(seUpdateBlob,fStoredClass,Value.fID,
          @fStoredClassRecordProps.BlobFieldsBits);
      result := true;
    finally
      UnLock;
    end;
  end else
    result := true; // as TSQLRest.UpdateblobFields()
end;

function TSQLRestServerStaticInMemory.TableRowCount(Table: TSQLRecordClass): integer;
begin
  if Table<>fStoredClass then
    result := 0 else
    result := fValue.Count;
end;

function TSQLRestServerStaticInMemory.TableHasRows(Table: TSQLRecordClass): boolean;
begin
  if Table<>fStoredClass then
    result := false else
    result := fValue.Count>0;
end;

function TSQLRestServerStaticInMemory.EngineUpdateField(Table: TSQLRecordClass;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
var SetField: TSQLPropInfo;
    WhereValueString, SetValueString: RawUTF8;
    Where: TList;
    i, ndx, WhereFieldIndex: integer;
    Rec: TSQLRecord;
begin
  result := false;

  if (self=nil) or (Table<>fStoredClass) or (SetFieldName='') or (SetValue='') or
     (WhereFieldName='') or (WhereValue='') then
    exit;
  // handle destination field RTTI
  SetField := fStoredClassRecordProps.Fields.ByRawUTF8Name(SetFieldName);
  if SetField=nil then
    exit; // don't allow setting ID field, which is Read Only
  if SetValue[1]='"' then
................................................................................
    inc(WhereFieldIndex); // FindWhereEqual() expects index = RTTI+1
  end;
  if WhereValue[1]='"' then
    UnQuoteSQLString(pointer(WhereValue),WhereValueString) else
    WhereValueString := WhereValue;
  // search indexes, then apply updates
  Where := TList.Create;
  Lock(true);
  try 
    // find matching Where[]
    if FindWhereEqual(WhereFieldIndex,WhereValueString,AddIntegerDynArrayEvent,Where,0,0)=0 then
      exit; // Where.Count=0 -> nothing to update
    // check that all records can be updated
    for i := 0 to Where.Count-1 do

      if not RecordCanBeUpdated(Table,TSQLRecord(fValue.List[PtrInt(Where.List[i])]).fID,seUpdate) then
        exit; // one record update fails -> abort all
    if fUniqueFields<>nil then
      for i := 0 to fUniqueFields.Count-1 do
      with TListFieldHash(fUniqueFields.List[i]) do
        if Field=SetField then
          if Where.Count>1 then // unique field can't allow multiple sets
            exit else begin
................................................................................
          end;
    // update field value
    for i := 0 to Where.Count-1 do begin
      Rec := fValue.List[PtrInt(Where.List[i])];
      SetField.SetValue(Rec,pointer(SetValueString),false);
      fModified := true;
      if Owner<>nil then
        Owner.InternalUpdateEvent(seUpdate,Table,Rec.fID,nil);
      result := true;
    end;
  finally
    UnLock;
    Where.Free;
  end;
end;

procedure TSQLRestServerStaticInMemory.UpdateFile;
var F: TFileStream;
begin
  if (self=nil) or not Modified or (FileName='') then
    exit;
  if fValue.Count=0 then
    DeleteFile(FileName) else begin
    F := TFileStream.Create(FileName,fmCreate);
................................................................................
    finally
      F.Free;
    end;
  end;
  fModified := false;
end;

function TSQLRestServerStaticInMemory.SearchField(const FieldName, FieldValue: RawUTF8; var ResultID: TIntegerDynArray): boolean;
var n, WhereField: integer;
    {$ifdef CPU64}i: integer;{$endif}
    Where: TList;
begin
  result := false;
  SetLength(ResultID,0);
  if (self=nil) or (fValue.Count=0) then
................................................................................
    {$endif}
  finally
    Where.Free;
  end;
end;


{ TSQLRestServerStaticInMemoryExternal }

procedure TSQLRestServerStaticInMemoryExternal.FlushInternalDBCache;
begin
  if Owner<>nil then

    Owner.FlushInternalDBCache;
end;


{ TListFieldHash }

function TListFieldHash.Compare(Item1,Item2: TObject): boolean;
................................................................................
  with fValues do
    if cardinal(Index)<cardinal(Count) then
      result := List[Index] else
      result := nil;
end;


{ TSQLRestServerStatic }

constructor TSQLRestServerStatic.Create(aClass: TSQLRecordClass;
  aServer: TSQLRestServer; const aFileName: TFileName; aBinaryFile: boolean);
begin
  inherited Create(nil,false);
  if aClass=nil then
    raise EBusinessLayerException.CreateFmt('%s.Create expect a class',[ClassName]);

  fStoredClass := aClass;
  fStoredClassRecordProps := aClass.RecordProps;
  if aServer<>nil then begin
    fOwner := aServer;
    fModel := aServer.Model;
    fStoredClassProps := fModel.Props[aClass];
    fNoAJAXJSON := aServer.fNoAJAXJSON; // expanded as main Server
  end else
    // if no server is defined, simply use the first model using this class
    if fStoredClassRecordProps.fModel<>nil then
    with fStoredClassRecordProps.fModel[0] do begin
      fModel := Model;
      fStoredClassProps := Properties;
    end;
................................................................................
  fFileName := aFileName;
  fBasicSQLCount := 'SELECT COUNT(*) FROM '+fStoredClassRecordProps.SQLTableName;
  fBasicSQLHasRows[false] := 'SELECT RowID FROM '+fStoredClassRecordProps.SQLTableName+' LIMIT 1';
  fBasicSQLHasRows[true] := fBasicSQLHasRows[false];
  system.delete(fBasicSQLHasRows[true],8,3);
end;








procedure TSQLRestServerStatic.BeginCurrentThread(Sender: TThread);
begin
  // nothing to do in this basic REST static class
end;

procedure TSQLRestServerStatic.EndCurrentThread(Sender: TThread);
begin
  // nothing to do in this basic REST static class
end;

function TSQLRestServerStatic.EngineExecuteAll(const aSQL: RawUTF8): boolean;
begin
  result := false; // not implemented in this basic REST static class
end;

function TSQLRestServerStatic.EngineUpdateField(Table: TSQLRecordClass;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
begin
  result := false; // not implemented in this basic REST static class
end;







function TSQLRestServerStatic.SearchField(const FieldName: RawUTF8;
  FieldValue: Integer; var ResultID: TIntegerDynArray): boolean;
begin
  result := SearchField(FieldName,Int32ToUTF8(FieldValue),ResultID);
end;

function TSQLRestServerStatic.RecordCanBeUpdated(Table: TSQLRecordClass;
  ID: integer; Action: TSQLEvent; ErrorMsg: PRawUTF8 = nil): boolean;
begin
  result := (self<>nil) and
    ((Owner=nil) or Owner.RecordCanBeUpdated(Table,ID,Action,ErrorMsg));
end;

function TSQLRestServerStatic.RefreshedAndModified: boolean;
begin
  result := false; // no refresh necessary with "normal" static tables
end;

procedure TSQLRestServerStatic.Lock(WillModifyContent: boolean);
begin
  if self<>nil then begin
    EnterCriticalSection(fSessionCriticalSection);
    if WillModifyContent then
      FlushInternalDBCache;

  end;






end;

procedure TSQLRestServerStatic.UnLock;

begin
  if self<>nil then
    LeaveCriticalSection(fSessionCriticalSection);

end;

function TSQLRestServerStatic.InternalBatchStart(Method: TSQLURIMethod): boolean;
begin
  result := false;
end;

procedure TSQLRestServerStatic.InternalBatchStop;
begin
  // do nothing method
end;

function TSQLRestServerStatic.AdaptSQLForEngineList(var SQL: RawUTF8): boolean; 
begin
  if fStoredClassProps=nil then
    result := false else begin
    result := IdemPropNameU(fStoredClassProps.SQL.SelectAllWithRowID,SQL);
    if result then
      SQL := fStoredClassProps.SQL.SelectAllWithID else
      result := IdemPropNameU(fStoredClassProps.SQL.SelectAllWithID,SQL);
................................................................................
  if integer(fStaticDataCount)<>length(fModel.Tables) then begin
    for t := fStaticDataCount to high(fModel.Tables) do
      StaticDataCreate(fModel.Tables[t]);
    fStaticDataCount := length(fModel.Tables);
  end;
  // initialize new tables
  for t := 0 to fStaticDataCount-1 do
    with TSQLRestServerStaticInMemory(fStaticData[t]) do
    if Count=0 then // emulates TSQLRestServerDB.CreateMissingTables
      StoredClass.InitializeTable(Self,'');
end;

destructor TSQLRestServerFullMemory.Destroy;
begin
  UpdateToFile;
................................................................................
    t: integer;
    S: TFileStream;
    wasString: boolean;
begin
  if (self=nil) or (fFileName='') or not FileExists(fFileName) then
    exit;
  for t := 0 to fStaticDataCount-1 do
    TSQLRestServerStaticInMemory(fStaticData[t]).fValue.Clear;
  if fBinaryFile then begin
    S := TFileStream.Create(FileName,fmOpenRead or fmShareDenyNone);
    try
      if ReadStringFromStream(S)=RawUTF8(ClassName)+'00' then
      repeat
        t := Model.GetTableIndex(ReadStringFromStream(S));
      until (t<0) or
        not TSQLRestServerStaticInMemory(fStaticData[t]).LoadFromBinary(S);
    finally
      S.Free;
    end;
  end else begin // [{"AuthUser":[{....},{...}]},{"AuthGroup":[{...},{...}]}]
    JSON := StringFromFile(fFileName);
    if JSON='' then
      exit;
................................................................................
      t := Model.GetTableIndex(TableName);
      if t<0 then
        exit;
      Data := P;
      P := GotoNextJSONObjectOrArray(P);
      if P=nil then
        break else
        TSQLRestServerStaticInMemory(fStaticData[t]).LoadFromJSON(Data,P-Data);
    until false;
  end;
end;

procedure TSQLRestServerFullMemory.UpdateToFile;
const CHARS: array[0..6] of AnsiChar = '[{":,}]';
var S: TFileStream;                //   0123456
................................................................................
    t: integer;
    Modified: boolean;
begin
  if (self=nil) or (FileName='') then
    exit;
  Modified := false;
  for t := 0 to fStaticDataCount-1 do
    if TSQLRestServerStaticInMemory(fStaticData[t]).Modified then begin
      Modified := true;
      break;
    end;
  if not Modified then
    exit;
  S := TFileStream.Create(FileName,fmCreate);
  try
    if fBinaryFile then begin
      WriteStringToStream(S,RawUTF8(ClassName)+'00');
      for t := 0 to fStaticDataCount-1 do
      with TSQLRestServerStaticInMemory(fStaticData[t]) do begin
        WriteStringToStream(S,fStoredClassRecordProps.SQLTableName);
        SaveToBinary(S);
      end;
    end else begin
      S.Write(CHARS[0],1);
      for t := 0 to fStaticDataCount-1 do
      with TSQLRestServerStaticInMemory(fStaticData[t]) do begin
        S.Write(CHARS[1],2);
        with fStoredClassRecordProps do
          S.Write(pointer(SQLTableName)^,length(SQLTableName));
        S.Write(CHARS[2],2);
        SaveToJSON(S,true);
        S.Write(CHARS[5],1);
        if t<integer(fStaticDataCount-1) then
................................................................................
  end;
end;

function TSQLRestServerFullMemory.EngineRetrieve(TableModelIndex, ID: integer): RawUTF8;
begin
  if cardinal(TableModelIndex)>=fStaticDataCount then
    result := '' else
    result := TSQLRestServerStaticInMemory(fStaticData[TableModelIndex]).EngineRetrieve(0,ID);
end;

function TSQLRestServerFullMemory.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8;
var TableIndex: integer;
begin
  TableIndex := Model.GetTableIndexFromSQLSelect(SQL,true);
  if TableIndex<0 then
    result := '' else
    result := TSQLRestServerStaticInMemory(fStaticData[TableIndex]).
      EngineList(SQL,ForceAJAX,ReturnedRowCount);
end;

function TSQLRestServerFullMemory.GetStatic(Table: TSQLRecordClass): TSQLRestServerStaticInMemory;
var t: cardinal;
begin
  t := fModel.GetTableIndexExisting(Table);
  if t<fStaticDataCount then
    result := TSQLRestServerStaticInMemory(fStaticData[t]) else
    result := nil;
end;

function TSQLRestServerFullMemory.EngineUpdate(Table: TSQLRecordClass; ID: integer;
  const SentData: RawUTF8): boolean;
begin
  result := GetStatic(Table).EngineUpdate(Table,ID,SentData);



end;

function TSQLRestServerFullMemory.EngineDelete(Table: TSQLRecordClass; ID: integer): boolean;
begin



  result := GetStatic(Table).EngineDelete(Table,ID);
end;

function TSQLRestServerFullMemory.EngineDeleteWhere(Table: TSQLRecordClass;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
begin



  result := GetStatic(Table).EngineDeleteWhere(Table,SQLWhere,IDs);
end;

function TSQLRestServerFullMemory.EngineRetrieveBlob(Table: TSQLRecordClass; aID: integer;
  BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
begin
  result := GetStatic(Table).EngineRetrieveBlob(Table,aID,BlobField,BlobData);




end;

function TSQLRestServerFullMemory.EngineUpdateBlob(Table: TSQLRecordClass; aID: integer;
  BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean;
begin



  result := GetStatic(Table).EngineUpdateBlob(Table,aID,BlobField,BlobData);
end;

function TSQLRestServerFullMemory.EngineAdd(Table: TSQLRecordClass; const SentData: RawUTF8): integer;
begin
  result := GetStatic(Table).EngineAdd(Table,SentData);




end;

function TSQLRestServerFullMemory.EngineUpdateField(Table: TSQLRecordClass;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
begin
  result := GetStatic(Table).EngineUpdateField(Table,SetFieldName,SetValue,



    WhereFieldName,WhereValue);
end;

function TSQLRestServerFullMemory.EngineExecuteAll(const aSQL: RawUTF8): boolean;
begin
  result := false; // not implemented in this basic REST server class
end;

................................................................................
begin
  if aRemoteClient=nil then
    raise EORMException.CreateFmt('%s creation with no remote client',[ClassName]);
  inherited Create(aRemoteClient.Model,aHandleUserAuthentication);
  fClient := aRemoteClient;
end;

function TSQLRestServerRemoteDB.EngineAdd(Table: TSQLRecordClass;
  const SentData: RawUTF8): integer;
begin
  result := fClient.EngineAdd(Table,SentData);
end;

function TSQLRestServerRemoteDB.EngineDelete(Table: TSQLRecordClass;
  ID: integer): boolean;
begin
  result := fClient.EngineDelete(Table,ID);
end;

function TSQLRestServerRemoteDB.EngineDeleteWhere(Table: TSQLRecordClass;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
begin
  result := fClient.EngineDeleteWhere(Table,SQLWhere,IDs);
end;

function TSQLRestServerRemoteDB.EngineExecuteAll(const aSQL: RawUTF8): boolean;
begin
  result := fClient.EngineExecute(aSQL);
end;

................................................................................
function TSQLRestServerRemoteDB.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean; ReturnedRowCount: PPtrInt): RawUTF8;
begin
  result := fClient.EngineList(SQL,ForceAJAX,ReturnedRowCount);
end;

function TSQLRestServerRemoteDB.EngineRetrieve(TableModelIndex, ID: integer): RawUTF8;
var Dummy: cardinal;
begin
  if not fClient.EngineRetrieve(TableModelIndex,ID,False,Dummy,result) then
    result := '';
end;

function TSQLRestServerRemoteDB.EngineRetrieveBlob(Table: TSQLRecordClass;
  aID: integer; BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
begin
  if (self=nil) or (BlobField=nil) then
    result := false else
    result := fClient.EngineRetrieveBlob(Table,aID,BlobField,BlobData);
end;

function TSQLRestServerRemoteDB.EngineUpdate(Table: TSQLRecordClass;
  ID: integer; const SentData: RawUTF8): boolean;
begin
  result := fClient.EngineUpdate(Table,ID,SentData);
end;

function TSQLRestServerRemoteDB.EngineUpdateBlob(Table: TSQLRecordClass;
  aID: integer; BlobField: PPropInfo;
  const BlobData: TSQLRawBlob): boolean;
begin
  if (self=nil) or (BlobField=nil) then
    result := false else
    result := fClient.EngineUpdateBlob(Table,aID,BlobField,BlobData);
end;

function TSQLRestServerRemoteDB.EngineUpdateField(Table: TSQLRecordClass;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
begin
  if (self=nil) or (Table.RecordProps.Fields.IndexByName(SetFieldName)<0) then
    result := false else
    result := fClient.EngineUpdateField(Table,SetFieldName,SetValue,WhereFieldName,WhereValue);
end;

function TSQLRestServerRemoteDB.AfterDeleteForceCoherency(Table: TSQLRecordClass;
  aID: integer): boolean;
begin
  result := true; // coherency will be performed on the server side
end;
................................................................................
    if aValue then
      SetLength(fForceBlobTransfert,fModel.fTablesMax+1) else
      exit; // nothing to set
  fForceBlobTransfert[i] := aValue;
end;

function TSQLRestClient.Add(Value: TSQLRecord; SendData: boolean; ForceID: boolean=false): integer;
var Data: RawUTF8;
    TableIndex: integer;
begin
  result := 0;
  if (Value=nil) or (self=nil) then
    exit;


  TableIndex := Model.GetTableIndexExisting(PSQLRecordClass(Value)^);
  if SendData then begin // send content of Value to the server as JSON
    Value.ComputeFieldsBeforeWrite(self,seAdd); // update TModTime/TCreateTime fields
    if Model.fTableProps[TableIndex].Kind in INSERT_WITH_ID then
      ForceID := true;
    Data := Value.GetJSONValues(True, // Expanded=true
      (Value.fID<>0) and ForceID,soInsert);
  end;
  // POST/Insert Collection
  result := EngineAdd(PSQLRecordClass(Value)^,Data);
  if result<=0 then
    exit;
  if SendData then
    fCache.Notify(TableIndex,result,Data,soInsert);
  Value.fID := result;
  if (fForceBlobTransfert<>nil) and fForceBlobTransfert[TableIndex] then
    UpdateBlobFields(Value);
end;

function TSQLRestClient.Delete(Table: TSQLRecordClass; ID: integer): boolean;
var TableIndex: integer;


begin
  TableIndex := Model.GetTableIndexExisting(Table);
  if not inherited Delete(Table,ID) then

    result := false else begin
    fCache.NotifyDeletion(TableIndex,ID);
    result := EngineDelete(Table,ID);
  end;
end;

function TSQLRestClient.Retrieve(aID: integer; Value: TSQLRecord;
      ForUpdate: boolean=false): boolean;
var Resp: RawUTF8;
    TableIndex: integer;
begin
  result := false;
  if (self=nil) or (aID<=0) or (Value=nil) then
    exit;
  TableIndex := Model.GetTableIndexExisting(PSQLRecordClass(Value)^);
................................................................................
      Value.FillFrom(Resp);
      Value.fID := aID; // JSON object may not contain the ID
      result := true;
      exit; // fast retrieved from internal Client cache (BLOBs ignored)
    end;
  end;
  try
    if EngineRetrieve(TableIndex,aID,ForUpdate,Value.fInternalState,Resp) then begin
      Value.FillFrom(Resp);
      Value.fID := aID; // JSON object may not contain the ID
      if (fForceBlobTransfert<>nil) and fForceBlobTransfert[TableIndex] then
        result := RetrieveBlobFields(Value) else
        result := true;
      ForUpdate := false; // any exception shall unlock the record
    end;
................................................................................
  finally
    if ForUpdate then
      Model.UnLock(TableIndex,aID);
  end;
end;

function TSQLRestClient.Update(Value: TSQLRecord): boolean;
var JSON: RawUTF8;
begin
  if (self=nil) or not inherited Update(Value) or
     not BeforeUpdateEvent(Value) then begin
    result := false;
    exit;
  end;
  Value.ComputeFieldsBeforeWrite(self,seUpdate); // update sftModTime fields
  JSON := Value.GetJSONValues(true,false,soUpdate); // expanded + without ID
  result := EngineUpdate(PSQLRecordClass(Value)^,Value.fID,JSON);
  if result then begin
    if (fForceBlobTransfert<>nil) and
       fForceBlobTransfert[Model.GetTableIndexExisting(PSQLRecordClass(Value)^)] then
      result := UpdateBlobFields(Value);
    fCache.Notify(Value,soUpdate); // JSON may not include all fields on update
    if result and assigned(OnRecordUpdate) then
      OnRecordUpdate(Value);
  end;
end;

function TSQLRestClient.RetrieveBlob(Table: TSQLRecordClass;
  aID: integer; const BlobFieldName: RawUTF8;
  out BlobData: TSQLRawBlob): boolean;
var P: PPropInfo;
begin
  result := false;
  if (self=nil) or (aID<=0) then
    exit;
  P := Table.RecordProps.BlobFieldPropFromRawUTF8(BlobFieldName);
  if P<>nil then
    result := EngineRetrieveBlob(Table,aID,P,BlobData);
end;

function TSQLRestClient.UpdateBlob(Table: TSQLRecordClass; aID: integer;
  const BlobFieldName: RawUTF8; const BlobData: TSQLRawBlob): boolean;
var BlobField: PPropInfo;
begin
  result := false;
  if (self=nil) or (aID<=0) or not RecordCanBeUpdated(Table,aID,seUpdate) then
    exit;
  BlobField := Table.RecordProps.BlobFieldPropFromRawUTF8(BlobFieldName);
  if BlobField<>nil then
    result := EngineUpdateBlob(Table,aID,BlobField,BlobData);
end;

function TSQLRestClient.BeforeUpdateEvent(Value: TSQLRecord): Boolean;
begin
  Result := true; // by default, just allow the update to proceed
end;

function TSQLRestClient.Refresh(aID: integer; Value: TSQLRecord;
  var Refreshed: boolean): boolean;
var Resp, Original: RawUTF8;
begin
  result := false;
  if (aID>0) and (self<>nil) and (Value<>nil) then
    if EngineRetrieve(Model.GetTableIndexExisting(PSQLRecordClass(Value)^),aID,False,
       Value.fInternalState,Resp) then begin
      Original := Value.GetJSONValues(IsNotAjaxJSON(pointer(Resp)),true,soSelect);
      Resp := trim(Resp);
      if (Resp<>'') and (Resp[1]='[') then // '[{....}]' -> '{...}'
        Resp := copy(Resp,2,length(Resp)-2);
      if Original<>Resp then begin // did the content really change?
        Refreshed := true;
................................................................................


{ TSQLVirtualTable }

constructor TSQLVirtualTable.Create(aModule: TSQLVirtualTableModule;
  const aTableName: RawUTF8; FieldCount: integer; Fields: PPUTF8CharArray);
var aTable: TSQLRecordClass;
    aTableIndex: integer;
begin
  if (aModule=nil) or (aTableName='') then
    raise EModelException.CreateFmt('Invalid parameters to %s.Create',[ClassName]);
  fModule := aModule;
  fTableName := aTableName;
  if fModule.fFeatures.StaticClass<>nil then
    // create new fStatic instance e.g. for TSQLVirtualTableLog
    if fModule.Server=nil then
      raise EModelException.CreateFmt('Missing aModule.Server for %s.Create',[ClassName]) else
    with fModule.Server do begin
      aTableIndex := Model.GetTableIndex(aTableName);
      if aTableIndex>=0 then begin
        aTable := Model.Tables[aTableIndex];
        fStatic := fModule.fFeatures.StaticClass.Create(aTable,fModule.Server,
          fModule.FileName(aTableName),self.InheritsFrom(TSQLVirtualTableBinary));
        if length(fStaticVirtualTable)<>length(Model.Tables) then
          SetLength(fStaticVirtualTable,length(Model.Tables));
        fStaticVirtualTable[aTableIndex] := fStatic;
      end;
    end;
end;

destructor TSQLVirtualTable.Destroy;
var aTableIndex: cardinal;
begin
................................................................................
  if result and (Static.Owner<>nil) then
    Static.Owner.fCache.NotifyDeletion(Static.StoredClass,aRowID);
end;

function TSQLVirtualTableJSON.Drop: boolean;
begin
  if (self<>nil) and (Static<>nil) then
  with Static as TSQLRestServerStaticInMemory do begin
    RollBack(0); // close any pending transaction
    fValue.Clear;
    Modified := true; // force update file after clear
    UpdateFile;
    result := true;
  end else
    result := false;
................................................................................
end;

class procedure TSQLVirtualTableJSON.GetTableModuleProperties(
  var aProperties: TVirtualTableModuleProperties);
begin
  aProperties.Features := [vtWrite,vtWhereIDPrepared];
  aProperties.CursorClass := TSQLVirtualTableCursorJSON;
  aProperties.StaticClass := TSQLRestServerStaticInMemoryExternal; // will flush Cache
  if InheritsFrom(TSQLVirtualTableBinary) then
    aProperties.FileExtension := 'data';
  // default will follow the class name, e.g. '.json' for TSQLVirtualTableJSON
end;

function TSQLVirtualTableJSON.Insert(aRowID: Int64;
  var Values: TSQLVarDynArray; out insertedRowID: Int64): boolean;
................................................................................
  if (self=nil) or (Static=nil) then
    exit;
  aRecord := Static.StoredClass.Create;
  try
    if aRecord.SetFieldSQLVars(Values) then begin
      if aRowID>0 then
        aRecord.fID := aRowID;
      insertedRowID := (Static as TSQLRestServerStaticInMemory).AddOne(aRecord,aRowID>0);
      if insertedRowID>0 then begin
        if Static.Owner<>nil then
          Static.Owner.fCache.Notify(aRecord,soInsert);
        result := true;
      end;
    end;
  finally
................................................................................
  end;
end;

function TSQLVirtualTableJSON.Prepare(var Prepared: TSQLVirtualTablePrepared): boolean;
begin
  result := inherited Prepare(Prepared); // optimize ID=? WHERE clause
  if result and (Static<>nil) then
  with Static as TSQLRestServerStaticInMemory do begin
    if Prepared.IsWhereOneFieldEquals then
    with Prepared.Where[0] do
    if UniqueFieldHash(Column)<>nil then begin
      Value.VType := ftNull; // mark TSQLVirtualTableCursorJSON expects it
      OmitCheck := true;
      Prepared.EstimatedCost := 1; 
    end;
................................................................................
  var Values: TSQLVarDynArray): boolean;
var i: integer;
begin
  result := false;
  if (self=nil) or (Static=nil) or
     (oldRowID<>newRowID) or (newRowID<=0) then // don't allow ID change
    exit;
  with Static as TSQLRestServerStaticInMemory do
    if UpdateOne(newRowID,Values) then begin
      if Static.Owner<>nil then begin
        i := IDToIndex(newRowID);
        if i>=0 then
          Static.Owner.fCache.Notify(TSQLRecord(fValue.List[i]),soUpdate);
      end;
      result := true;
................................................................................
end;


{ TSQLVirtualTableCursorJSON }

function TSQLVirtualTableCursorJSON.Column(aColumn: integer;
  var aResult: TSQLVar): boolean;
var Static: TSQLRestServerStaticInMemory;
begin
  if (self=nil) or (fCurrent>fMax) or
     (TSQLVirtualTableJSON(Table).Static=nil) then begin
    result := false;
    exit;
  end;
  Static := TSQLRestServerStaticInMemory(TSQLVirtualTableJSON(Table).Static);
  if Cardinal(fCurrent)>=Cardinal(Static.fValue.Count) then
    result := False else begin
    if aColumn=VIRTUAL_TABLE_ROWID_COLUMN then begin
      aResult.VType := ftInt64;
      aResult.VInt64 := TSQLRecord(Static.fValue.List[fCurrent]).fID;
    end else
    with Static.fStoredClassRecordProps.Fields do
................................................................................
  const Prepared: TSQLVirtualTablePrepared): boolean;
var Hash: TListFieldHash;
begin
  result := inherited Search(Prepared); // mark EOF by default
  if (not result) or (not Table.InheritsFrom(TSQLVirtualTableJSON)) or
     (TSQLVirtualTableJSON(Table).Static=nil) then
    result := false else
    with TSQLRestServerStaticInMemory(TSQLVirtualTableJSON(Table).Static) do begin
      if Count>0 then // if something to search in
        if Prepared.IsWhereIDEquals(false) then begin // ID=?
          fMax := IDToIndex(Prepared.Where[0].Value.VInt64); // binary search
          if fMax>=0 then
            fCurrent := fMax; // ID found
        end else
        if Prepared.IsWhereOneFieldEquals then







>
>
>







 







|
|






|


|







 







|
|

|





|

|







 







|







 







|







 







|







 







|







 







|

|







 







|
>
|







 







|







 







<
<
<
<
<







 







|







 







|
>
>
>
>
>
>
>








|





|




|






|







|
|






|









|







 







|
<
<
<








|







 







>
|

>
>
|



>
>







 







>
|







 







|













|
>

|












|
>

|







 







|
|







 







|







 







|
|



|







 







|
|
|

|

|

|






|


|



<
|


|







 







|

|
|
|
<
<
|
>
>
|
|
|
|
|
|
|
|
>
>
>
>
>
>
>
>
|
>
>
>
>
>
>







 







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







 







>
>
>
>
>
>
>
>
>










<
<
<
<
<
<
<
<
<
<
<
<
<







>
|







 







|

|







|




|









|






|







 







|







 







|


|

|

|






|






|







 







|


|

|







>
>








|
|







 







|


|





|
>
>






|






|

|
|
|
|
>

|


|

>
>
>
>







 







|
>






|







|

|
|







 







|






|





|






|
|







 







|





|
|







 







|

|
|

|

|







 







|







 







|






|





|



|

|

|

|







 







|


|

>
|





|



|












|


|
|
|
|

|

|

|



|







 







|







 







|
|
|
|

|

|

|







 







|







>
>







 







<
<
<
<







<
<
<
<
<
<
<
<
<
<
<
<
<
<







 







<
<
<

<
>
|

|
|
|
|

|

|

|







 







|
|
|

|







 







|
|
|

|







 







|
>








|







 







|
|

|
>
>







 







|







 







|







 







|







 







|







 







|







 







|








|







 







|







 







|







 







|
|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>

>
|
>
>
>
|
>

|







 







|
|










>
>

<
>
|
>
>
>
>
>
>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







<
|







 








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>


|





|
>


|


>





|





|
>

|


>








>
|







 







|









>
>
>
>
>
>
>
>
>







 







|







 







<
<
<
<
<







 







|


|


>
|







 







|
>

>
|


|

>

>
|





|




|












|
|
>


|

|
|
|
>

|
|

>
|


|
<
|
|



|
|
|
|
|
>

|


|



>
|
|







 







|




|




|







|

|





|







 







|












|











|







 







<


<
<
<


|







 







|







|




<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<












|
|



<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<

<

|
<
<
<
<
<
<
<
<








<


|

<
<
<
|







|








|







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<

|
<
>










|
|







 







|







 







>
|
>
|








|


|
|




|







 







|

|
>
>







 







>
|










>
>
|







 







|







 







|







 







|


|







 







>
|













|







 







|







 







|





>
|







 







>


|
|



|
|







 







<
<
|







 







<
<
|







<
<
|







 







|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|

|




|




|



|







|
|



|
>
|



|




>





|



|



|



|










|




|

|







 







|







 







|













|













|







 







|


|
|



|







|
|
|




|



|








|











|






|



|








|






|







|









|







|






|



|







 







|







 







|







 







|







 







|







 







|







 







|











|





|




|







 







|





|







 







|

|












|







 







|







 







|






|


|






|



|











|






|

|





|


|







 







|

|



|





|












|



|







 







|
|



|
>

|








|



|






|








|




|
|




|
>

|


|









|



|






|











|





|






|






|








>
|







 







|






>
|







 







|



|




|







 







|







 







|

|

|
>







 







|

|


|


>






<







 







>
>
>
>
>
>
>
|




|




|
<
<
<
<
<





>
>
>
>
>
>
|





|


<
|


|




|

<
|
<
<
>
|
>
>
>
>
>
>


<
>

<
<
>


|




|




|







 







|







 







|







|







 







|







 







|










|






|







 







|









|



|




|



|


|
>
>
>


|

>
>
>
|


|


>
>
>
|


|


<
>
>
>
>


|


>
>
>
|


|

<
>
>
>
>


|
|

<
>
>
>
|







 







|


|


|
<

|


|


|







 







<

|
<


|
|



|


|


|


|
<
|



|


|


<
<
|







 







<
<

<
<
<
>
>
|
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
|


<
<
>
>

<
<
>
|
<
<
|
<


|







 







|







 







<

<
|
<
<
<
<
<
<




<





<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<











|







 







<










|
|
|




|







 







|







 







|







 







|







 







|







 







|







 







|






|







 







|







679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
...
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
...
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
...
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
....
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
....
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
....
3092
3093
3094
3095
3096
3097
3098
3099
3100
3101
3102
3103
3104
3105
3106
....
3188
3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199
3200
3201
3202
3203
3204
....
3627
3628
3629
3630
3631
3632
3633
3634
3635
3636
3637
3638
3639
3640
3641
3642
3643
....
3677
3678
3679
3680
3681
3682
3683
3684
3685
3686
3687
3688
3689
3690
3691
....
8304
8305
8306
8307
8308
8309
8310





8311
8312
8313
8314
8315
8316
8317
....
8322
8323
8324
8325
8326
8327
8328
8329
8330
8331
8332
8333
8334
8335
8336
....
8354
8355
8356
8357
8358
8359
8360
8361
8362
8363
8364
8365
8366
8367
8368
8369
8370
8371
8372
8373
8374
8375
8376
8377
8378
8379
8380
8381
8382
8383
8384
8385
8386
8387
8388
8389
8390
8391
8392
8393
8394
8395
8396
8397
8398
8399
8400
8401
8402
8403
8404
8405
8406
8407
8408
8409
8410
8411
8412
8413
8414
8415
8416
8417
8418
8419
8420
8421
8422
8423
8424
8425
8426
8427
8428
....
8627
8628
8629
8630
8631
8632
8633
8634



8635
8636
8637
8638
8639
8640
8641
8642
8643
8644
8645
8646
8647
8648
8649
8650
....
8667
8668
8669
8670
8671
8672
8673
8674
8675
8676
8677
8678
8679
8680
8681
8682
8683
8684
8685
8686
8687
8688
8689
8690
8691
....
8694
8695
8696
8697
8698
8699
8700
8701
8702
8703
8704
8705
8706
8707
8708
8709
....
8755
8756
8757
8758
8759
8760
8761
8762
8763
8764
8765
8766
8767
8768
8769
8770
8771
8772
8773
8774
8775
8776
8777
8778
8779
8780
8781
8782
8783
8784
8785
8786
8787
8788
8789
8790
8791
8792
8793
8794
8795
8796
8797
8798
8799
8800
8801
8802
....
9245
9246
9247
9248
9249
9250
9251
9252
9253
9254
9255
9256
9257
9258
9259
9260
....
9330
9331
9332
9333
9334
9335
9336
9337
9338
9339
9340
9341
9342
9343
9344
....
9675
9676
9677
9678
9679
9680
9681
9682
9683
9684
9685
9686
9687
9688
9689
9690
9691
9692
9693
9694
....
9702
9703
9704
9705
9706
9707
9708
9709
9710
9711
9712
9713
9714
9715
9716
9717
9718
9719
9720
9721
9722
9723
9724
9725
9726
9727
9728
9729
9730

9731
9732
9733
9734
9735
9736
9737
9738
9739
9740
9741
....
9745
9746
9747
9748
9749
9750
9751
9752
9753
9754
9755
9756


9757
9758
9759
9760
9761
9762
9763
9764
9765
9766
9767
9768
9769
9770
9771
9772
9773
9774
9775
9776
9777
9778
9779
9780
9781
9782
9783
9784
9785
9786
9787
9788
9789
....
9811
9812
9813
9814
9815
9816
9817























9818
9819
9820
9821
9822
9823
9824
....
9825
9826
9827
9828
9829
9830
9831
9832
9833
9834
9835
9836
9837
9838
9839
9840
9841
9842
9843
9844
9845
9846
9847
9848
9849
9850













9851
9852
9853
9854
9855
9856
9857
9858
9859
9860
9861
9862
9863
9864
9865
9866
....
9944
9945
9946
9947
9948
9949
9950
9951
9952
9953
9954
9955
9956
9957
9958
9959
9960
9961
9962
9963
9964
9965
9966
9967
9968
9969
9970
9971
9972
9973
9974
9975
9976
9977
9978
9979
9980
9981
9982
9983
9984
9985
9986
9987
9988
9989
9990
.....
10020
10021
10022
10023
10024
10025
10026
10027
10028
10029
10030
10031
10032
10033
10034
.....
10121
10122
10123
10124
10125
10126
10127
10128
10129
10130
10131
10132
10133
10134
10135
10136
10137
10138
10139
10140
10141
10142
10143
10144
10145
10146
10147
10148
10149
10150
10151
10152
10153
10154
10155
10156
.....
10198
10199
10200
10201
10202
10203
10204
10205
10206
10207
10208
10209
10210
10211
10212
10213
10214
10215
10216
10217
10218
10219
10220
10221
10222
10223
10224
10225
10226
10227
10228
10229
10230
10231
10232
10233
10234
10235
10236
.....
10253
10254
10255
10256
10257
10258
10259
10260
10261
10262
10263
10264
10265
10266
10267
10268
10269
10270
10271
10272
10273
10274
10275
10276
10277
10278
10279
10280
10281
10282
10283
10284
10285
10286
10287
10288
10289
10290
10291
10292
10293
10294
10295
10296
10297
10298
10299
10300
10301
10302
10303
10304
10305
10306
10307
10308
.....
10316
10317
10318
10319
10320
10321
10322
10323
10324
10325
10326
10327
10328
10329
10330
10331
10332
10333
10334
10335
10336
10337
10338
10339
10340
10341
10342
10343
10344
10345
10346
10347
10348
10349
.....
10350
10351
10352
10353
10354
10355
10356
10357
10358
10359
10360
10361
10362
10363
10364
10365
10366
10367
10368
10369
10370
10371
10372
10373
10374
10375
10376
10377
10378
10379
10380
10381
10382
10383
10384
10385
.....
10415
10416
10417
10418
10419
10420
10421
10422
10423
10424
10425
10426
10427
10428
10429
10430
10431
10432
10433
10434
10435
10436
.....
10456
10457
10458
10459
10460
10461
10462
10463
10464
10465
10466
10467
10468
10469
10470
10471
10472
10473
10474
10475
10476
10477
.....
10501
10502
10503
10504
10505
10506
10507
10508
10509
10510
10511
10512
10513
10514
10515
.....
10525
10526
10527
10528
10529
10530
10531
10532
10533
10534
10535
10536
10537
10538
10539
10540
10541
10542
10543
10544
10545
10546
10547
10548
10549
10550
10551
10552
10553
10554
10555
10556
10557
10558
10559
10560
10561
10562
.....
10591
10592
10593
10594
10595
10596
10597
10598
10599
10600
10601
10602
10603
10604
10605
10606
10607
10608
10609
10610
10611
10612
10613
10614
10615
10616
10617
10618
10619
10620
10621
10622
10623
10624
10625
10626
10627
10628
10629
10630
10631
10632
10633
10634
10635
10636
10637
10638
10639
10640
10641
10642
10643
10644
10645
10646
10647
10648
10649
10650
.....
10664
10665
10666
10667
10668
10669
10670
10671
10672
10673
10674
10675
10676
10677
10678
.....
10681
10682
10683
10684
10685
10686
10687
10688
10689
10690
10691
10692
10693
10694
10695
10696
10697
10698
10699
10700
10701
10702
10703
10704
.....
10746
10747
10748
10749
10750
10751
10752
10753
10754
10755
10756
10757
10758
10759
10760
10761
10762
10763
10764
10765
10766
10767
10768
10769
.....
10772
10773
10774
10775
10776
10777
10778




10779
10780
10781
10782
10783
10784
10785














10786
10787
10788
10789
10790
10791
10792
.....
10947
10948
10949
10950
10951
10952
10953



10954

10955
10956
10957
10958
10959
10960
10961
10962
10963
10964
10965
10966
10967
10968
10969
10970
10971
10972
10973
10974
.....
11569
11570
11571
11572
11573
11574
11575
11576
11577
11578
11579
11580
11581
11582
11583
11584
11585
11586
11587
.....
11611
11612
11613
11614
11615
11616
11617
11618
11619
11620
11621
11622
11623
11624
11625
11626
11627
11628
11629
.....
11650
11651
11652
11653
11654
11655
11656
11657
11658
11659
11660
11661
11662
11663
11664
11665
11666
11667
11668
11669
11670
11671
11672
11673
11674
.....
11730
11731
11732
11733
11734
11735
11736
11737
11738
11739
11740
11741
11742
11743
11744
11745
11746
11747
11748
11749
.....
11807
11808
11809
11810
11811
11812
11813
11814
11815
11816
11817
11818
11819
11820
11821
.....
11829
11830
11831
11832
11833
11834
11835
11836
11837
11838
11839
11840
11841
11842
11843
.....
11872
11873
11874
11875
11876
11877
11878
11879
11880
11881
11882
11883
11884
11885
11886
.....
21647
21648
21649
21650
21651
21652
21653
21654
21655
21656
21657
21658
21659
21660
21661
.....
21748
21749
21750
21751
21752
21753
21754
21755
21756
21757
21758
21759
21760
21761
21762
.....
21870
21871
21872
21873
21874
21875
21876
21877
21878
21879
21880
21881
21882
21883
21884
21885
21886
21887
21888
21889
21890
21891
21892
21893
.....
21900
21901
21902
21903
21904
21905
21906
21907
21908
21909
21910
21911
21912
21913
21914
.....
21934
21935
21936
21937
21938
21939
21940
21941
21942
21943
21944
21945
21946
21947
21948
.....
21954
21955
21956
21957
21958
21959
21960
21961
21962
21963
21964
21965
21966
21967
21968
21969
.....
21989
21990
21991
21992
21993
21994
21995
21996
21997
21998
21999
22000
22001
22002
22003
22004
22005
22006
22007
22008
22009
22010
22011
22012
22013
22014
22015
22016
22017
22018
22019
22020
22021
22022
22023
22024
22025
22026
22027
22028
22029
22030
22031
.....
22116
22117
22118
22119
22120
22121
22122
22123
22124
22125
22126
22127
22128
22129
22130
22131
22132
22133
22134
22135
22136
22137
22138
22139
22140
.....
22145
22146
22147
22148
22149
22150
22151
22152
22153
22154
22155
22156
22157
22158
22159
22160
22161
22162
22163
22164
22165
22166

22167
22168
22169
22170
22171
22172
22173
22174
22175
22176
22177
22178
22179
22180
22181
22182
22183
.....
22189
22190
22191
22192
22193
22194
22195
22196
22197
22198
22199
22200
22201
22202
22203
22204
22205
22206
22207
22208
22209
22210
22211
22212
22213
22214
22215
22216
22217
22218
22219
22220
22221
22222
22223
22224
22225
22226
22227
22228
.....
22302
22303
22304
22305
22306
22307
22308

22309
22310
22311
22312
22313
22314
22315
22316
.....
22339
22340
22341
22342
22343
22344
22345
22346
22347
22348
22349
22350
22351
22352
22353
22354
22355
22356
22357
22358
22359
22360
22361
22362
22363
22364
22365
22366
22367
22368
22369
22370
22371
22372
22373
22374
22375
22376
22377
22378
22379
22380
22381
22382
22383
22384
22385
22386
22387
22388
22389
22390
22391
22392
22393
22394
22395
22396
22397
22398
22399
22400
22401
22402
22403
22404
22405
22406
22407
22408
22409
22410
22411
22412
22413
22414
22415
22416
22417
22418
22419
22420
22421
22422
22423
22424
22425
.....
22426
22427
22428
22429
22430
22431
22432
22433
22434
22435
22436
22437
22438
22439
22440
22441
22442
22443
22444
22445
22446
22447
22448
22449
22450
22451
22452
22453
22454
22455
22456
22457
22458
.....
22990
22991
22992
22993
22994
22995
22996
22997
22998
22999
23000
23001
23002
23003
23004
.....
23071
23072
23073
23074
23075
23076
23077





23078
23079
23080
23081
23082
23083
23084
.....
23643
23644
23645
23646
23647
23648
23649
23650
23651
23652
23653
23654
23655
23656
23657
23658
23659
23660
23661
23662
23663
23664
.....
23671
23672
23673
23674
23675
23676
23677
23678
23679
23680
23681
23682
23683
23684
23685
23686
23687
23688
23689
23690
23691
23692
23693
23694
23695
23696
23697
23698
23699
23700
23701
23702
23703
23704
23705
23706
23707
23708
23709
23710
23711
23712
23713
23714
23715
23716
23717
23718
23719
23720
23721
23722
23723
23724
23725
23726
23727
23728
23729
23730
23731
23732
23733

23734
23735
23736
23737
23738
23739
23740
23741
23742
23743
23744
23745
23746
23747
23748
23749
23750
23751
23752
23753
23754
23755
23756
23757
23758
23759
23760
23761
23762
.....
24043
24044
24045
24046
24047
24048
24049
24050
24051
24052
24053
24054
24055
24056
24057
24058
24059
24060
24061
24062
24063
24064
24065
24066
24067
24068
24069
24070
24071
24072
24073
24074
24075
24076
24077
24078
24079
24080
24081
24082
24083
.....
24089
24090
24091
24092
24093
24094
24095
24096
24097
24098
24099
24100
24101
24102
24103
24104
24105
24106
24107
24108
24109
24110
24111
24112
24113
24114
24115
24116
24117
24118
24119
24120
24121
24122
24123
24124
24125
24126
24127
24128
.....
24147
24148
24149
24150
24151
24152
24153

24154
24155



24156
24157
24158
24159
24160
24161
24162
24163
24164
24165
.....
24171
24172
24173
24174
24175
24176
24177
24178
24179
24180
24181
24182
24183
24184
24185
24186
24187
24188
24189
24190



















































24191
24192
24193
24194
24195
24196
24197
24198
24199
24200
24201
24202
24203
24204
24205
24206
24207













































24208

24209
24210








24211
24212
24213
24214
24215
24216
24217
24218

24219
24220
24221
24222



24223
24224
24225
24226
24227
24228
24229
24230
24231
24232
24233
24234
24235
24236
24237
24238
24239
24240
24241
24242
24243
24244
24245
24246
24247


































24248
24249

24250
24251
24252
24253
24254
24255
24256
24257
24258
24259
24260
24261
24262
24263
24264
24265
24266
24267
24268
24269
.....
24271
24272
24273
24274
24275
24276
24277
24278
24279
24280
24281
24282
24283
24284
24285
.....
24296
24297
24298
24299
24300
24301
24302
24303
24304
24305
24306
24307
24308
24309
24310
24311
24312
24313
24314
24315
24316
24317
24318
24319
24320
24321
24322
24323
24324
24325
24326
24327
24328
24329
24330
24331
.....
24836
24837
24838
24839
24840
24841
24842
24843
24844
24845
24846
24847
24848
24849
24850
24851
24852
24853
24854
.....
24866
24867
24868
24869
24870
24871
24872
24873
24874
24875
24876
24877
24878
24879
24880
24881
24882
24883
24884
24885
24886
24887
24888
24889
24890
24891
24892
24893
24894
.....
24923
24924
24925
24926
24927
24928
24929
24930
24931
24932
24933
24934
24935
24936
24937
.....
25011
25012
25013
25014
25015
25016
25017
25018
25019
25020
25021
25022
25023
25024
25025
.....
25034
25035
25036
25037
25038
25039
25040
25041
25042
25043
25044
25045
25046
25047
25048
25049
25050
25051
.....
25058
25059
25060
25061
25062
25063
25064
25065
25066
25067
25068
25069
25070
25071
25072
25073
25074
25075
25076
25077
25078
25079
25080
25081
25082
25083
25084
25085
25086
25087
.....
25100
25101
25102
25103
25104
25105
25106
25107
25108
25109
25110
25111
25112
25113
25114
.....
25599
25600
25601
25602
25603
25604
25605
25606
25607
25608
25609
25610
25611
25612
25613
25614
25615
25616
25617
25618
25619
25620
.....
25664
25665
25666
25667
25668
25669
25670
25671
25672
25673
25674
25675
25676
25677
25678
25679
25680
25681
25682
25683
25684
25685
25686
25687
.....
25712
25713
25714
25715
25716
25717
25718


25719
25720
25721
25722
25723
25724
25725
25726
.....
25727
25728
25729
25730
25731
25732
25733


25734
25735
25736
25737
25738
25739
25740
25741


25742
25743
25744
25745
25746
25747
25748
25749
.....
25904
25905
25906
25907
25908
25909
25910
25911
25912
25913
25914
25915
25916
25917
25918
.....
25981
25982
25983
25984
25985
25986
25987
25988
25989
25990
25991
25992
25993
25994
25995
25996
25997
25998
25999
26000
26001
26002
26003
26004
26005
26006
26007
26008
26009
26010
26011
26012
26013
26014
26015
26016
26017
26018
26019
26020
26021
26022
26023
26024
26025
26026
26027
26028
26029
26030
26031
26032
26033
26034
26035
26036
26037
26038
26039
26040
26041
26042
26043
26044
26045
26046
26047
26048
26049
26050
26051
26052
26053
26054
26055
26056
26057
26058
26059
26060
26061
26062
26063
26064
26065
26066
26067
26068
26069
26070
26071
26072
26073
26074
26075
26076
26077
26078
26079
26080
26081
26082
26083
26084
26085
.....
26696
26697
26698
26699
26700
26701
26702
26703
26704
26705
26706
26707
26708
26709
26710
26711
26712
26713
26714
26715
26716
26717
26718
26719
26720
26721
26722
26723
26724
26725
26726
26727
26728
26729
26730
26731
26732
26733
26734
26735
26736
26737
26738
26739
26740
26741
26742
26743
26744
26745
26746
26747
26748
26749
26750
26751
26752
26753
26754
26755
26756
26757
26758
26759
26760
26761
26762
26763
26764
26765
26766
26767
26768
26769
26770
26771
26772
26773
26774
26775
26776
26777
26778
26779
26780
26781
26782
26783
26784
26785
26786
.....
26802
26803
26804
26805
26806
26807
26808
26809
26810
26811
26812
26813
26814
26815
26816
.....
26817
26818
26819
26820
26821
26822
26823
26824
26825
26826
26827
26828
26829
26830
26831
26832
26833
26834
26835
26836
26837
26838
26839
26840
26841
26842
26843
26844
26845
26846
26847
26848
26849
26850
26851
26852
26853
26854
26855
26856
26857
26858
26859
.....
26880
26881
26882
26883
26884
26885
26886
26887
26888
26889
26890
26891
26892
26893
26894
26895
26896
26897
26898
26899
26900
26901
26902
26903
26904
26905
26906
26907
26908
26909
26910
26911
26912
26913
26914
26915
26916
26917
26918
26919
26920
26921
26922
26923
26924
26925
26926
26927
26928
26929
26930
26931
26932
26933
26934
26935
26936
26937
26938
26939
26940
26941
26942
26943
26944
26945
26946
26947
26948
26949
26950
26951
26952
26953
26954
26955
26956
26957
26958
26959
26960
26961
26962
26963
26964
26965
26966
26967
26968
26969
26970
26971
26972
26973
26974
26975
26976
26977
26978
26979
26980
26981
26982
26983
26984
26985
26986
26987
26988
26989
26990
26991
26992
26993
26994
26995
26996
26997
26998
26999
27000
27001
27002
27003
27004
27005
27006
.....
27041
27042
27043
27044
27045
27046
27047
27048
27049
27050
27051
27052
27053
27054
27055
.....
27151
27152
27153
27154
27155
27156
27157
27158
27159
27160
27161
27162
27163
27164
27165
.....
27195
27196
27197
27198
27199
27200
27201
27202
27203
27204
27205
27206
27207
27208
27209
.....
27222
27223
27224
27225
27226
27227
27228
27229
27230
27231
27232
27233
27234
27235
27236
.....
27242
27243
27244
27245
27246
27247
27248
27249
27250
27251
27252
27253
27254
27255
27256
.....
27275
27276
27277
27278
27279
27280
27281
27282
27283
27284
27285
27286
27287
27288
27289
27290
27291
27292
27293
27294
27295
27296
27297
27298
27299
27300
27301
27302
27303
27304
27305
27306
27307
27308
27309
27310
27311
27312
.....
27337
27338
27339
27340
27341
27342
27343
27344
27345
27346
27347
27348
27349
27350
27351
27352
27353
27354
27355
27356
27357
.....
27359
27360
27361
27362
27363
27364
27365
27366
27367
27368
27369
27370
27371
27372
27373
27374
27375
27376
27377
27378
27379
27380
27381
27382
27383
27384
27385
27386
27387
27388
.....
27418
27419
27420
27421
27422
27423
27424
27425
27426
27427
27428
27429
27430
27431
27432
.....
27453
27454
27455
27456
27457
27458
27459
27460
27461
27462
27463
27464
27465
27466
27467
27468
27469
27470
27471
27472
27473
27474
27475
27476
27477
27478
27479
27480
27481
27482
27483
27484
27485
27486
27487
27488
27489
27490
27491
27492
27493
27494
27495
27496
27497
27498
27499
27500
27501
27502
27503
27504
27505
27506
27507
27508
27509
27510
27511
27512
27513
27514
27515
27516
27517
27518
.....
27522
27523
27524
27525
27526
27527
27528
27529
27530
27531
27532
27533
27534
27535
27536
27537
27538
27539
27540
27541
27542
27543
27544
27545
27546
27547
27548
27549
27550
27551
27552
27553
27554
27555
27556
27557
27558
27559
27560
27561
27562
27563
27564
27565
.....
27581
27582
27583
27584
27585
27586
27587
27588
27589
27590
27591
27592
27593
27594
27595
27596
27597
27598
27599
27600
27601
27602
27603
27604
27605
27606
27607
27608
27609
27610
27611
27612
27613
27614
27615
27616
27617
27618
27619
27620
27621
27622
27623
27624
27625
27626
27627
27628
27629
27630
27631
27632
27633
27634
27635
27636
27637
27638
27639
27640
27641
27642
27643
27644
27645
27646
27647
27648
27649
27650
27651
27652
27653
27654
27655
27656
27657
27658
27659
27660
27661
27662
27663
27664
27665
27666
27667
27668
27669
27670
27671
27672
27673
27674
27675
27676
27677
27678
27679
27680
27681
27682
27683
27684
27685
27686
27687
27688
27689
27690
27691
27692
27693
27694
27695
27696
27697
27698
27699
27700
27701
27702
27703
27704
27705
27706
27707
27708
27709
27710
27711
27712
.....
27723
27724
27725
27726
27727
27728
27729
27730
27731
27732
27733
27734
27735
27736
27737
27738
27739
27740
27741
27742
27743
27744
27745
.....
27750
27751
27752
27753
27754
27755
27756
27757
27758
27759
27760
27761
27762
27763
27764
27765
27766
27767
27768
27769
27770
27771
27772
27773
.....
27779
27780
27781
27782
27783
27784
27785
27786
27787
27788
27789
27790
27791
27792
27793
.....
27814
27815
27816
27817
27818
27819
27820
27821
27822
27823
27824
27825
27826
27827
27828
27829
27830
27831
27832
27833
.....
27861
27862
27863
27864
27865
27866
27867
27868
27869
27870
27871
27872
27873
27874
27875
27876
27877
27878
27879
27880
27881
27882

27883
27884
27885
27886
27887
27888
27889
.....
27891
27892
27893
27894
27895
27896
27897
27898
27899
27900
27901
27902
27903
27904
27905
27906
27907
27908
27909
27910
27911
27912
27913
27914
27915





27916
27917
27918
27919
27920
27921
27922
27923
27924
27925
27926
27927
27928
27929
27930
27931
27932
27933
27934
27935

27936
27937
27938
27939
27940
27941
27942
27943
27944
27945

27946


27947
27948
27949
27950
27951
27952
27953
27954
27955
27956

27957
27958


27959
27960
27961
27962
27963
27964
27965
27966
27967
27968
27969
27970
27971
27972
27973
27974
27975
27976
27977
27978
27979
.....
28004
28005
28006
28007
28008
28009
28010
28011
28012
28013
28014
28015
28016
28017
28018
.....
28025
28026
28027
28028
28029
28030
28031
28032
28033
28034
28035
28036
28037
28038
28039
28040
28041
28042
28043
28044
28045
28046
28047
.....
28057
28058
28059
28060
28061
28062
28063
28064
28065
28066
28067
28068
28069
28070
28071
.....
28072
28073
28074
28075
28076
28077
28078
28079
28080
28081
28082
28083
28084
28085
28086
28087
28088
28089
28090
28091
28092
28093
28094
28095
28096
28097
28098
28099
28100
28101
28102
28103
28104
.....
28111
28112
28113
28114
28115
28116
28117
28118
28119
28120
28121
28122
28123
28124
28125
28126
28127
28128
28129
28130
28131
28132
28133
28134
28135
28136
28137
28138
28139
28140
28141
28142
28143
28144
28145
28146
28147
28148
28149
28150
28151
28152
28153
28154
28155
28156
28157
28158
28159
28160
28161
28162
28163
28164
28165
28166
28167
28168
28169

28170
28171
28172
28173
28174
28175
28176
28177
28178
28179
28180
28181
28182
28183
28184
28185
28186

28187
28188
28189
28190
28191
28192
28193
28194
28195

28196
28197
28198
28199
28200
28201
28202
28203
28204
28205
28206
.....
28213
28214
28215
28216
28217
28218
28219
28220
28221
28222
28223
28224
28225
28226

28227
28228
28229
28230
28231
28232
28233
28234
28235
28236
28237
28238
28239
28240
28241
.....
28248
28249
28250
28251
28252
28253
28254

28255
28256

28257
28258
28259
28260
28261
28262
28263
28264
28265
28266
28267
28268
28269
28270
28271
28272
28273

28274
28275
28276
28277
28278
28279
28280
28281
28282
28283


28284
28285
28286
28287
28288
28289
28290
28291
.....
28331
28332
28333
28334
28335
28336
28337


28338



28339
28340
28341















28342
28343
28344


28345
28346
28347


28348
28349


28350

28351
28352
28353
28354
28355
28356
28357
28358
28359
28360
.....
28367
28368
28369
28370
28371
28372
28373
28374
28375
28376
28377
28378
28379
28380
28381
.....
28382
28383
28384
28385
28386
28387
28388

28389

28390






28391
28392
28393
28394

28395
28396
28397
28398
28399

























28400
28401
28402
28403
28404
28405
28406
28407
28408
28409
28410
28411
28412
28413
28414
28415
28416
28417
28418
.....
31294
31295
31296
31297
31298
31299
31300

31301
31302
31303
31304
31305
31306
31307
31308
31309
31310
31311
31312
31313
31314
31315
31316
31317
31318
31319
31320
31321
31322
31323
31324
31325
.....
31529
31530
31531
31532
31533
31534
31535
31536
31537
31538
31539
31540
31541
31542
31543
.....
31544
31545
31546
31547
31548
31549
31550
31551
31552
31553
31554
31555
31556
31557
31558
.....
31562
31563
31564
31565
31566
31567
31568
31569
31570
31571
31572
31573
31574
31575
31576
.....
31579
31580
31581
31582
31583
31584
31585
31586
31587
31588
31589
31590
31591
31592
31593
.....
31604
31605
31606
31607
31608
31609
31610
31611
31612
31613
31614
31615
31616
31617
31618
.....
31620
31621
31622
31623
31624
31625
31626
31627
31628
31629
31630
31631
31632
31633
31634
31635
31636
31637
31638
31639
31640
31641
.....
31650
31651
31652
31653
31654
31655
31656
31657
31658
31659
31660
31661
31662
31663
31664
    - BREAKING CHANGE in TSQLRestServerCallBackParams which is replace by the
      TSQLRestServerURIContext class: in addition, all method-based services
      should be a procedure, and use Ctxt.Results()/Error() methods to return
      any content - new definition of Ctxt features now full access to
      incoming/outgoing context and parameters, especially via
      the new Input*[] properties, for easy URI parameter retrieval, and
      also allow define specific URI routing by a dedicated class
    - BREAKING CHANGE; TSQLRestServerStatic* classes are now renamed as
      TSQLRestStorage* and do not inherit from TSQLRestServer but plain TSQLRest
      for a much cleaner design
    - URI routing for interface-based service is now specified by the two
      TSQLRestRoutingREST and TSQLRestRoutingJSON_RPC classes (inheriting from
      the abstract TSQLRestServerURIContext class) instead of rmJSON and
      rmJSON_RPC enums - it allows any custom URI routing by inheritance
    - BREAKING CHANGE of TJSONWriter.WriteObject() method and ObjectToJSON()
      function: serialization is now defined with TTextWriterWriteObjectOptions
      set - therefore, TJSONSerializerCustomWriter callback signature changed
................................................................................
    - moved SQLFromSelectWhere() from a global function to a TSQLModel method
      (to prepare "Table per class hierarchy" mapping in mORMot)
    - SQLParamContent() / ExtractInlineParameters() functions moved to SynCommons
    - TSQLAuthUser and TSQLAuthGroup have now "index ..." attributes to their
      RawUTF8 properties, to allow direct handling in external databases
    - new protected TSQLRestServer.InternalAdaptSQL method, extracted from URI()
      process to also be called by TSQLRestServer.InternalListJSON() for proper
      TSQLRestStorage.AdaptSQLForEngineList(SQL) call
    - new TSQLRestStorage.fOutInternalStateForcedRefresh protected field to
      optionally force the refresh of the content
    - new TSQLRestServer.OnBlobUpdateEvent: TNotifyFieldSQLEvent event handler
      to implement feature request [4cafc41f67]
    - new protected TSQLRestServer.InternalUpdateEvent virtual method, to allow
      a server-wide update notification, not coupled to OnUpdateEvent callback -
      see feature request [5688e97251]
    - TSQLRestStorageInMemory.AdaptSQLForEngineList() will now handle
      'select count(*') from TableName' statements directly, and any RESTful
      requests from client
    - fixed issue in TSQLRestStorageInMemory.EngineList() when only ID
    - changed TSQLAccessRights and TSQLAuthGroup.SQLAccessRights CSV format
      to use 'first-last,' pattern to regroup set bits (reduce storage size)
    - added overloaded TSQLAccessRights.Edit() method using TSQLOccasions set
    - added reOneSessionPerUser kind of remote execution in TSQLAccessRight
    - introducing TSQLRestClientURI.InternalCheckOpen/InternalClose methods to
      properly handle remote connection and re-connection
    - added TSQLRestClientURI.LastErrorCode/LastErrorMessage/LastErrorException
................................................................................
    - introducing TSQLRecordInterfaced class, if your TSQLRecord definition
      should be able to implement interfaces
    - added optional CustomFields parameter to TSQLRestClientURI.BatchUpdate()
      and BatchAdd() methods
    - added TSQLRestClientURI.ServerTimeStampSynchronize method to force time
      synchronization with the server - can be handy to test the connection 
    - added TSQLRest.TableHasRows/TableRowCount methods, and overriden direct
      implementation for TSQLRestServer/TSQLRestStorageInMemory (including
      SQL pattern recognition for TSQLRestStorageInMemory)
    - added TSQLRest.RetrieveList method to retrieve a TObjectList of TSQLRecord
    - "rowCount": is added in TSQLRestStorageInMemory.GetJSONValues,
      TSQLTable.GetJSONValues and in TSQLTableJSON.ParseAndConvert, at the end
      of the non expanded JSON content, if needed - improves client performance
    - UpdateBlobFields() and RetrieveBlobFields() methods are now defined at
      TSQLRest level, with dedicated implementation for TSQLRestClient* and
      TSQLRestServer* classes - implements feature request [34664934a9]
    - fixed TSQLRestStorageInMemory.UpdateBlobFields() to return true
      if no BLOB field is defined (as with TSQLRestServer) - ticket [bfa13889d5]
    - fixed issue in TSQLRestStorageInMemory.GetJSONValues(), and handle
      optional LIMIT clause in this very same method
    - fixed unexpected issue in TSQLRestClientURI.BatchSend() when nothing is
      to be sent
    - fix potential GDI handle resource leak in TSQLRestClientURIMessage.Create
    - introducing TSQLRestClientURIMessage.DoNotProcessMessages property
    - TSQLRestClientURINamedPipe.InternalCheckOpen/InternalURI refactoring
    - added TInterfacedObjectWithCustomCreate kind of class, making easy to
................................................................................
    - TSQLRestServer.LaunchCallBack() is now inlined in TSQLRestServer.URI()
    - fixed ticket [a5e3564e48] about RecordRef typecast (and enhance comments)
    - fixed ticket [4f4dd18ad9] about TPropInfo.IsStored not handling methods
      callbacks, e.g. for TPersistent storage
    - fixed ticket [21c2d5ae96] when inserting/updating blob-only table content
    - fixed ticket [7e9f06bf1a] to let TSQLTable.FieldLengthMax() use caption
      text for enumeration columns
    - fixed ticket [28545a4ce0] about TSQLRestStorageInMemory.EngineDelete
      not thread-safe when run directly on server side
    - fixed ticket [027bb9678d] - now TSQLRecordRTree class works as expected
    - fixed ticket [876a097316] about TSQLRest.Add() when ForcedID<>0
    - implement ticket [e3f9742865] for enhanced JSON in woHumanReadable mode
    - fixed GPF issue in TServiceFactoryServer after instance time-out deletion
    - added TSQLPropInfo.PropertyIndex member
    - added TSQLRecordProperties.SimpleFieldsCount[] array
................................................................................
      {$ifndef NOVARIANTS}, sftVariant{$endif}];

type
  /// define how TJSONObjectDecoder.Decode() will handle JSON string values
  TJSONObjectDecoderParams = (pInlined, pQuoted, pNonQuoted);

  /// record/object helper to handle JSON object decoding
  // - used e.g. by GetJSONObjectAsSQL() function or TSQLRestStorageExternal
  // ExecuteFromJSON and InternalBatchStop methods
  TJSONObjectDecoder = {$ifdef UNICODE}record{$else}object{$endif}
    /// contains the decoded field names or value
    FieldNames, FieldValues: array[0..MAX_SQLFIELDS-1] of RawUTF8;
    /// Decode() will set each field type approximation
    // - will recognize also JSON_BASE64_MAGIC/JSON_SQLDATE_MAGIC prefix
    FieldTypeApproximation: array[0..MAX_SQLFIELDS-1] of
................................................................................
      Params: TJSONObjectDecoderParams; RowID: Integer=0; ReplaceRowIDWithID: Boolean=false); overload;
    /// encode as a SQL-ready INSERT or UPDATE statement
    // - after a successfull call to Decode()
    // - escape SQL strings, according to the official SQLite3 documentation
    // (i.e. ' inside a string is stored as '')
    // - if InlinedParams was TRUE, it will create prepared parameters like
    // 'COL1=:("VAL1"):, COL2=:(VAL2):'
    // - called by GetJSONObjectAsSQL() function or TSQLRestStorageExternal
    function EncodeAsSQL(Update: boolean): RawUTF8;
    /// encode as a SQL-ready INSERT or UPDATE statement with ? as values
    // - after a successfull call to Decode()
    // - FieldValues[] content will be ignored
    // - Occasion can be only soInsert or soUpdate
    // - for soInsert, it will create an INSERT with multiple VALUES if
    //  MultipleInsertCount>1, like 'INSERT ... VALUES (?,?), (?,?), ....'
................................................................................
{$M+} { we need the RTTI information to be compiled for the published
        properties of these classes and their children (like TPersistent),
        to enable ORM - must be defined at the forward definition level }
  TSQLRecord = class;      // published properties = ORM fields/columns
  TSQLRecordMany = class;
  TSQLAuthUser = class;
  TSQLRestServer = class;  // published methods = RESTful callbacks handlers
  TSQLRestStorage = class;
  TSQLRestClientURI = class;
{$M-}

  /// class-reference type (metaclass) of TSQLRecord
  TSQLRecordClass = class of TSQLRecord;

  PClass = ^TClass;
................................................................................
  // TSQLRecordMany) after registration for an external DB via a call to
  // VirtualTableExternalRegister() from mORMotDB unit
  TSQLRecordVirtualKind = (
    rSQLite3, rFTS3, rFTS4, rRTree, rCustomForcedID, rCustomAutoID);

  /// kind of (static) database server implementation available
  // - sMainEngine will identify the default main SQlite3 engine
  // - sStaticDataTable will identify a TSQLRestStorageInMemory - i.e.
  // TSQLRestServer.fStaticData[] which can work without SQLite3
  // - sVirtualTable will identify virtual TSQLRestStorage classes - i.e.
  // TSQLRestServer.fStaticVirtualTable[] which points to SQLite3 virtual tables
  // (e.g. TObjectList or external databases)
  TSQLRestServerKind = (sMainEngine, sStaticDataTable, sVirtualTable);
  /// pointer to the kind of (static) database server implementation
  PSQLRestServerKind = ^TSQLRestServerKind;

  /// some information about a given TSQLRecord class properties
................................................................................
    /// the Table as specified at the URI level (if any)
    Table: TSQLRecordClass;
    /// the index in the Model of the Table specified at the URI level (if any)
    TableIndex: integer;
    /// the RTTI properties of the Table specified at the URI level (if any)
    TableRecordProps: TSQLModelRecordProperties;
    /// the RESTful instance implementing the Table specified at the URI level (if any)
    // - equals TSQLRestServer most of the time, but may be an TSQLRestStorage
    // for any in-memory/MongoDB/virtual instance
    TableEngine: TSQLRest;
    /// the associated TSQLRecord.ID, as decoded from URI scheme
    // - this property will be set from incoming URI, even if RESTful
    // authentication is not enabled
    TableID: integer;
    /// the index of the callback published method within the internal class list
    MethodIndex: integer;
    /// the service identified by an interface-based URI
................................................................................
    /// the corresponding TAuthSession.User.ID value
    // - is undefined if Session is 0 or 1 (no authentication running)
    SessionUser: integer;
    /// the corresponding TAuthSession.User.GroupRights.ID value
    // - is undefined if Session is 0 or 1 (no authentication running)
    SessionGroup: integer;
    /// the static instance corresponding to the associated Table (if any)
    Static: TSQLRestStorage;
    /// the kind of static instance corresponding to the associated Table (if any)
    StaticKind: TSQLRestServerKind;
    /// optional error message which will be transmitted as JSON error (if set)
    CustomErrorMsg: RawUTF8;
    {$ifdef WITHLOG}
    /// associated logging instance
    // - you can use it to log some process on the server side
................................................................................
    end;
    /// log the corresponding text (if logging is enabled)
    procedure InternalLog(const Text: RawUTF8; Level: TSynLogInfo);
      {$ifdef HASINLINE}inline;{$endif}
    procedure SetRoutingClass(aServicesRouting: TSQLRestServerURIContextClass);
    /// retrieve a list of members as JSON encoded data - used by OneFieldValue()
    // and MultiFieldValue() public functions below





    // - call virtual abstract InternalListJSON() method to get the list content
    // - FieldName can be a CSV list of needed field names, if needed
    function InternalListJSON(Table: TSQLRecordClass; const FieldName, WhereClause: RawUTF8): TSQLTableJSON; overload;
    /// retrieve all fields for a list of members JSON encoded data
    // - this special method gets all fields content for a specified table:
    // the resulting TSQLTableJSON content can be used to fill whole records
    // instances by using the TSQLRecord.FillPrepare() and TSQLRecord.FillRow()
................................................................................
    // - this default implementation returns always true
    // - e.g. you can add digital signature to a record to disallow record editing
    // - the ErrorMsg can be set to a variable, which will contain an explicit
    // error message
    function RecordCanBeUpdated(Table: TSQLRecordClass; ID: integer; Action: TSQLEvent;
      ErrorMsg: PRawUTF8 = nil): boolean; virtual;
    /// internal method used by Delete(Table,SQLWhere) method
    function InternalDeleteNotifyAndGetIDs(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
      var IDs: TIntegerDynArray): boolean; 
    /// retrieve the server time stamp
    // - default implementation will use fServerTimeStampOffset to compute
    // the value from PC time (i.e. Now+fServerTimeStampOffset as TTimeLog)
    // - inherited classes may override this method, or set the appropriate
    // value in fServerTimeStampOffset protected field
    function GetServerTimeStamp: TTimeLog; virtual;
................................................................................
    // and direct JSON export, avoiding a TSQLTable which allocates memory for every
    // field values before the JSON export
    // - can be called for a single Table (ModelRoot/Table), or with low level SQL
    // query (ModelRoot + SQL sent as request body)
    // - if ReturnedRowCount points to an integer variable, it must be filled with
    // the number of row data returned (excluding field names)
    // - this method must be implemented in a thread-safe manner
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false;
      ReturnedRowCount: PPtrInt=nil): RawUTF8; virtual; abstract;
    /// get a member from its ID (implements REST GET member)
    // - returns the data of this object as JSON
    // - override this method for proper data retrieval from the database engine
    // - this method must be implemented in a thread-safe manner
    // - ForUpdate parameter is used only on Client side
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; virtual; abstract;
    /// create a new member (implements REST POST Collection)
    // - SentData can contain the JSON object with field values to be added
    // - class is taken from Model.Tables[TableModelIndex]
    // - returns the TSQLRecord ID/ROWID value, 0 on error
    // - is a "RowID":.. or "ID":.. member is set in SentData, it shall force its
    // value as insertion ID
    // - override this method for proper calling the database engine
    // - this method must be implemented in a thread-safe manner
    function EngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer; virtual; abstract;
    /// update a member (implements REST PUT Member)
    // - SentData can contain the JSON object with field values to be added
    // - returns true on success
    // - override this method for proper calling the database engine
    // - this method must be implemented in a thread-safe manner
    function EngineUpdate(TableModelIndex, ID: integer; const SentData: RawUTF8): boolean; virtual; abstract;
    /// delete a member (implements REST DELETE Member)
    // - returns true on success
    // - override this method for proper calling the database engine
    // - this method must be implemented in a thread-safe manner
    function EngineDelete(TableModelIndex, ID: integer): boolean; virtual; abstract;
    /// delete several members, from a WHERE clause
    // - IDs[] contains the already-computed matching IDs for SQLWhere
    // - returns true on success
    // - override this method for proper calling the database engine, i.e.
    // using either IDs[] or a faster SQL statement
    // - this method must be implemented in a thread-safe manner
    function EngineDeleteWhere(TableModelIndex: integer; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; virtual; abstract;
    /// get a blob field content from its member ID and field name
    // - implements REST GET member with a supplied blob field name
    // - returns TRUE on success
    // - returns the data of this blob as raw binary (not JSON) in BlobData
    // - override this method for proper data retrieval from the database engine
    // - this method must be implemented in a thread-safe manner
    function EngineRetrieveBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; virtual; abstract; 
    /// update a blob field content from its member ID and field name
    // - implements REST PUT member with a supplied blob field name
    // - returns TRUE on success
    // - the data of this blob must be specified as raw binary (not JSON) in BlobData
    // - override this method for proper data retrieval from the database engine
    // - this method must be implemented in a thread-safe manner
    function EngineUpdateBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; virtual; abstract;
    /// update an individual record field value from a specified ID or Value
    // - return true on success
    // - will allow execution of requests like
    // $ UPDATE tablename SET setfieldname=setvalue WHERE wherefieldname=wherevalue
    // - SetValue and WhereValue parameters must match our inline format, i.e.
    // by double quoted with " for strings, or be plain text for numbers - e.g.
    // $ Client.EngineUpdateField(TSQLMyRecord,'FirstName','"Smith"','RowID','10')
    // - this method must be implemented in a thread-safe manner
    function EngineUpdateField(TableModelIndex: integer; 
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; virtual; abstract;
  public
    /// initialize the class, and associate it to a specified database Model
    constructor Create(aModel: TSQLModel); virtual;
    /// release internal used instances
    // - e.g. release associated TSQLModel or TServiceContainer
    destructor Destroy; override;
................................................................................
      Value: TSQLRecord): boolean; overload;
    /// get a member from its ID
    // - return true on success
    // - Execute 'SELECT * FROM TableName WHERE ID=:(aID): LIMIT 1' SQL Statememt
    // - if ForUpdate is true, the REST method is LOCK and not GET: it tries to lock
    // the corresponding record, then retrieve its content; caller has to call
    // UnLock() method after Value usage, to release the record
    // - this method will call EngineRetrieve() abstract method



    // - the TSQLRawBlob (BLOB) fields are not retrieved by this method, to
    // preserve bandwidth: use the RetrieveBlob() methods for handling
    // BLOB fields, or set either the TSQLRestClientURI.ForceBlobTransfert
    // or TSQLRestClientURI.ForceBlobTransfertTable[] properties
    // - the TSQLRecordMany fields are not retrieved either: they are separate
    // instances created by TSQLRecordMany.Create, with dedicated methods to
    // access to the separated pivot table
   function Retrieve(aID: integer; Value: TSQLRecord;
      ForUpdate: boolean=false): boolean; overload; virtual; 
    /// get a member from its TRecordReference property content
    // - instead of the other Retrieve() methods, this implementation Create an
    // instance, with the appropriated class stored in Reference
    // - returns nil on any error (invalid Reference e.g.)
    // - if ForUpdate is true, the REST method is LOCK and not GET: it tries to lock
    // the corresponding record, then retrieve its content; caller has to call
    // UnLock() method after Value usage, to release the record
................................................................................
    // responsible of freeing the instance
    // - this TObjectList will contain a list of all matching records
    // - return nil on error
    function RetrieveList(Table: TSQLRecordClass; FormatSQLWhere: PUTF8Char;
      const BoundsSQLWhere: array of const; const aCustomFieldsCSV: RawUTF8=''): TObjectList;
    /// Execute directly a SQL statement, expecting a list of results
    // - return a result table on success, nil on failure
    // - will call EngineList() abstract method to retrieve its JSON content
    function ExecuteList(const Tables: array of TSQLRecordClass; const SQL: RawUTF8): TSQLTableJSON; virtual;
    /// unlock the corresponding record
    // - record should have been locked previously e.g. with Retrieve() and
    // forupdate=true, i.e. retrieved not via GET with LOCK REST-like verb
    // - use our custom UNLOCK REST-like verb
    // - returns true on success
    function UnLock(Table: TSQLRecordClass; aID: integer): boolean; overload; virtual; abstract;
    /// unlock the corresponding record
    // - record should have been locked previously e.g. with Retrieve() and
    // forupdate=true, i.e. retrieved not via GET with LOCK REST-like verb
    // - use our custom UNLOCK REST-like method
    // - calls internally UnLock() above
    // - returns true on success
    function UnLock(Rec: TSQLRecord): boolean; overload;
    /// create a new member (implements REST POST Collection)
    // - if SendData is true, client sends the current content of Value with the
    // request, otherwize record is created with default values
................................................................................
    // - on success, returns the new ROWID value; on error, returns 0
    // - on success, Value.ID is updated with the new ROWID
    // - the TSQLRawBlob(BLOB) fields values are not set by this method, to
    // preserve bandwidth
    // - the TSQLRecordMany fields are not set either: they are separate
    // instances created by TSQLRecordMany.Create, with dedicated methods to
    // access to the separated pivot table
    // - this method will call EngineAdd() to perform the request
    function Add(Value: TSQLRecord; SendData: boolean; ForceID: boolean=false): integer; overload; virtual;
    /// create a new member, from a supplied list of field values
    // - the aSimpleFields parameters must follow explicitely the order of published
    // properties of the supplied aTable class, excepting the TSQLRawBlob and
    // TSQLRecordMany kind (i.e. only so called "simple fields")
    // - the aSimpleFields must have exactly the same count of parameters as
    // there are "simple fields" in the published properties
    // - if ForcedID is set to non null, client sends this ID to be used
................................................................................

    /// access the internal caching parameters for a given TSQLRecord
    // - purpose of this caching mechanism is to speed up retrieval of some
    // common values at either Client or Server level (like configuration settings)
    // - only caching synchronization is about the direct RESTful/CRUD commands:
    // RETRIEVE, ADD, UPDATE and DELETE (that is, a complex direct SQL UPDATE or
    // via TSQLRecordMany pattern won't be taken in account - only exception is
    // TSQLRestStorage tables accessed as SQLite3 virtual table)
    // - this caching will be located at the TSQLRest level, that is no automated
    // synchronization is implemented between TSQLRestClient and TSQLRestServer:
    // you shall ensure that your code won't fail due to this restriction
    // - use Cache.SetCache() and Cache.SetTimeOut() methods to set the appropriate
    // configuration for this particular TSQLRest instance 
    property Cache: TSQLRestCache read GetCache;

    /// get a blob field content from its record ID and supplied blob field name
    // - implements REST GET member with a supplied member ID and a blob field name
    // - return true on success
    // - this method is defined as abstract, i.e. there is no default implementation:
    // it must be implemented 100% RestFul with a
    // GET ModelRoot/TableName/TableID/BlobFieldName request for example
    // - this method retrieve the blob data as a TSQLRawBlob string using
    // EngineRetrieveBlob()
    function RetrieveBlob(Table: TSQLRecordClass; aID: integer;
      const BlobFieldName: RawUTF8; out BlobData: TSQLRawBlob): boolean; overload; virtual;
    /// get a blob field content from its record ID and supplied blob field name
    // - implements REST GET member with a supplied member ID and a blob field name
    // - return true on success
    // - this method will create a TStream instance (which must be freed by the
    // caller after use) and fill it with the blob data
    function RetrieveBlob(Table: TSQLRecordClass; aID: integer;
      const BlobFieldName: RawUTF8; out BlobStream: THeapMemoryStream): boolean; overload;
    /// update a blob field from its record ID and supplied blob field name
    // - implements REST PUT member with a supplied member ID and field name
    // - return true on success
    // - this default method call RecordCanBeUpdated() to check if the action is
    // allowed
    // - this method expect the Blob data to be supplied as TSQLRawBlob, using
    // EngineUpdateBlob()
    function UpdateBlob(Table: TSQLRecordClass; aID: integer;
      const BlobFieldName: RawUTF8; const BlobData: TSQLRawBlob): boolean; overload; virtual;
    /// update a blob field from its record ID and blob field name
    // - implements REST PUT member with a supplied member ID and field name
    // - return true on success
    // - this default method call RecordCanBeUpdated() to check if the action is
    // allowed
    // - this method expect the Blob data to be supplied as a TStream: it will
    // send the whole stream content (from its beginning position upto its
................................................................................
    // - use the TSQLAuthGroup.AccessRights CSV format
    function ToString: RawUTF8;
    /// unserialize the content from TEXT
    // - use the TSQLAuthGroup.AccessRights CSV format
    procedure FromString(P: PUTF8Char);
  end;

  TSQLRestStorageClass = class of TSQLRestStorage;
  TSQLRestStorageInMemory = class;
  TSQLVirtualTableModule = class;


  {/ table containing the available user access rights for authentication
    - this class should be added to the TSQLModel, together with TSQLAuthUser,
      to allow authentication support
    - you can inherit from it to add your custom properties to each user info:
................................................................................
    /// able to set the PasswordHashHexa field from a plain password content
    // - in fact, PasswordHashHexa := SHA256('salt'+PasswordPlain) in UTF-8
    property PasswordPlain: RawUTF8 write SetPasswordPlain;
  published
    /// the User identification Name, as entered at log-in
    // - the same identifier can be used only once (this column is marked as
    // unique via a "stored AS_UNIQUE" - i.e. "stored false" - attribute), and
    // therefore indexed in the database (e.g. hashed in TSQLRestStorageInMemory)
    property LogonName: RawUTF8 index 20 read fLogonName write fLogonName stored AS_UNIQUE;
    /// the User Name, as may be displayed or printed
    property DisplayName: RawUTF8 index 50 read fDisplayName write fDisplayName;
    /// the hexa encoded associated SHA-256 hash of the password
    property PasswordHashHexa: RawUTF8 index 64 read fPasswordHashHexa write fPasswordHashHexa;
    /// the associated access rights of this user
    // - access rights are managed by group
................................................................................
    fSQLAuthGroupClass: TSQLAuthGroupClass;
    /// how in-memory sessions are handled
    fSessionClass: TAuthSessionClass;
    /// will contain the in-memory representation of some static tables
    // - this array has the same length as the associated Model.Tables[]
    // - fStaticData[] will contain pure in-memory tables, not declared as
    // SQLite3 virtual tables, therefore not available from joined SQL statements
    fStaticData: array of TSQLRestStorage;
    /// map TSQLRestStorageInMemory or TSQLRestStorageExternal engines
    // - this array has the same length as the associated Model.Tables[]
    // - fStaticVirtualTable[] will contain in-memory or external tables declared
    // as SQLite3 virtual tables, therefore available from joined SQL statements
    fStaticVirtualTable: array of TSQLRestStorage;
    /// in-memory storage of TAuthSession instances
    fSessions: TObjectList;
    /// used to compute genuine TAuthSession.ID cardinal value
    fSessionCounter: cardinal;
    /// mutex used to make fSessions[] use thread-safe
    fSessionCriticalSection: TRTLCriticalSection;
    fSessionAuthentications: IObjectDynArray; // must be defined before the array
................................................................................
    // - use "string" type, i.e. UnicodeString for Delphi 2009+, in order
    // to call directly the correct FindWindow?()=FindWindow Win32 API
    fServerWindowName: string;
{$endif}
    fPublishedMethod: TSQLRestServerMethods;
    fPublishedMethods: TDynArrayHashed;
    /// fast get the associated static server, if any
    function GetStaticDataServer(aClass: TSQLRecordClass): TSQLRestStorage;
    /// retrieve a TSQLRestStorage instance associated to a Virtual Table
    // - is e.g. TSQLRestStorageInMemory instance associated to a
    // TSQLVirtualTableBinary or TSQLVirtualTableJSON class
    // - may be a TSQLRestStorageExternal (as defined in mORMotDB unit)
    // for a virtual table giving access to an external database
    function GetVirtualTable(aClass: TSQLRecordClass): TSQLRestStorage;
    /// fast get the associated static server or Virtual table, if any
    // - this can be used to call directly the TSQLRestStorage instance
    // on the server side
    // - same as a dual call to StaticDataServer[aClass] + StaticVirtualTable[aClass]
    // - TSQLRestServer.URI will make a difference between the a static server
    // or a TSQLVirtualTable, but this method won't - you can set a reference
    // to a TSQLRestServerKind variable to retrieve the database server type
    function GetStaticDataServerOrVirtualTable(aClass: TSQLRecordClass;
      Kind: PSQLRestServerKind=nil): TSQLRestStorage; overload;
    /// overloaded method using table index in associated Model
    function GetStaticDataServerOrVirtualTable(aTableIndex: integer;
      Kind: PSQLRestServerKind=nil): TSQLRestStorage; overload;
       {$ifdef HASINLINE}inline;{$endif}
    /// retrieve a list of members as JSON encoded data - used by OneFieldValue()
    // and MultiFieldValue() public functions

    function InternalAdaptSQL(TableIndex: integer; var SQL: RawUTF8): TSQLRestStorage;
    function InternalListRawUTF8(TableIndex: integer; const SQL: RawUTF8): RawUTF8;
    /// this method is overriden for setting the NoAJAXJSON field
    // of all associated TSQLRestStorage servers
    procedure SetNoAJAXJSON(const Value: boolean); virtual;
    /// add a new session to the internal session list
    // - do not use this method directly: this callback is to be used by
    // TSQLRestServerAuthentication* classes
    // - will check that the logon name is valid
    procedure SessionCreate(var User: TSQLAuthUser; Ctxt: TSQLRestServerURIContext;
      out Session: TAuthSession); virtual;
................................................................................
    // - this method is not thread-safe: caller should use fSessionCriticalSection
    function SessionAccess(Ctxt: TSQLRestServerURIContext): TAuthSession;
    /// delete a session from its index in fSessions[]
    // - will perform any needed clean-up, and log the event
    // - this method is not thread-safe: caller should use fSessionCriticalSection
    procedure SessionDelete(aSessionIndex: integer; Ctxt: TSQLRestServerURIContext);
    /// returns TRUE if this table is worth caching (e.g. already in memory)
    // - this overriden implementation returns FALSE for TSQLRestStorageInMemory
    function CacheWorthItForTable(aTableIndex: cardinal): boolean; override;
    /// overriden methods which will perform CRUD operations
    // - will call any static TSQLRestStorage, or call MainEngine*() virtual methods
    function EngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer; override;


    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override; 
    function EngineUpdate(TableModelIndex, ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDelete(TableModelIndex, ID: integer): boolean; override;
    function EngineDeleteWhere(TableModelIndex: integer; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    function EngineRetrieveBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateField(TableModelIndex: integer;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
    /// virtual methods which will perform CRUD operations on the main DB
    function MainEngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer; virtual; abstract;
    function MainEngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; virtual; abstract;
    function MainEngineList(const SQL: RawUTF8; ForceAJAX: Boolean; ReturnedRowCount: PPtrInt): RawUTF8; virtual; abstract;
    function MainEngineUpdate(TableModelIndex, ID: integer; const SentData: RawUTF8): boolean; virtual; abstract;
    function MainEngineDelete(TableModelIndex, ID: integer): boolean; virtual; abstract;
    function MainEngineDeleteWhere(TableModelIndex: integer; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; virtual; abstract;
    function MainEngineRetrieveBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; virtual; abstract;
    function MainEngineUpdateBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; virtual; abstract;
    function MainEngineUpdateField(TableModelIndex: integer;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; virtual; abstract;
  public
    /// this integer property is incremented by the database engine when any SQL
    // statement changes the database contents (i.e. on any not SELECT statement)
    // - its value can be published to the client on every remote request
    // - it may be used by client to avoid retrieve data only if necessary
    // - if its value is 0, this feature is not activated on the server, and the
    // client must ignore it and always retrieve the content
................................................................................
    OnSessionClosed: TNotifySQLSession;
    /// this property can be used to specify the URI parmeters to be used
    // for query paging
    // - is set by default to PAGINGPARAMETERS_YAHOO constant by
    // TSQLRestServer.Create() constructor
    URIPagingParameters: TSQLRestServerURIPagingParameters;
























    /// implement Server-Side TSQLRest deletion
    // - uses internally EngineDelete() function for calling the database engine
    // - call corresponding fStaticData[] if necessary
    // - this record is also erased in all available TRecordReference properties
    // in the database Model, for relational database coherency
    function Delete(Table: TSQLRecordClass; ID: integer): boolean; override;
    /// implement Server-Side TSQLRest deletion with a WHERE clause
................................................................................
    // - will process all ORM-level validation, coherency checking and
    // notifications together with a low-level SQL deletion work (if possible)
    function Delete(Table: TSQLRecordClass; const SQLWhere: RawUTF8): boolean; override;
    /// overriden method for direct static class call (if any)
    function TableRowCount(Table: TSQLRecordClass): integer; override;
    /// overriden method for direct static class call (if any)
    function TableHasRows(Table: TSQLRecordClass): boolean; override;
    /// virtual method called when a record is updated
    // - default implementation will call the OnUpdateEvent/OnBlobUpdateEvent
    // methods, if defined
    // - returns true on success, false if an error occured (but action must continue)
    // - you can override this method to implement a server-wide notification,
    // but be aware it may be the first step to break the stateless architecture
    // of the framework
    function InternalUpdateEvent(aEvent: TSQLEvent; aTable: TSQLRecordClass; aID: integer;
      aIsBlobFields: PSQLFieldBits): boolean; virtual;
    /// this method is called internally after any successfull deletion to
    // ensure relational database coherency
    // - delete all available TRecordReference properties pointing to this record
    // in the database Model, for database coherency
    // - delete all available TSQLRecord properties pointing to this record
    // in the database Model, for database coherency
    // - important notice: we don't use FOREIGN KEY constraints in this framework,
    // and handle all integrity check within this method (it's therefore less
    // error-prone, and more cross-database engine compatible)S
    function AfterDeleteForceCoherency(Table: TSQLRecordClass; aID: integer): boolean; virtual;













    /// update all BLOB fields of the supplied Value
    // - this overriden method will execute the direct static class, if any
    function UpdateBlobFields(Value: TSQLRecord): boolean; override;
    /// get all BLOB fields of the supplied value from the remote server
    // - this overriden method will execute the direct static class, if any
    function RetrieveBlobFields(Value: TSQLRecord): boolean; override;
    /// implement Server-Side TSQLRest unlocking
    // - to be called e.g. after a Retrieve() with forupdate=TRUE
    // - implements our custom UNLOCK REST-like verb
    // - locking is handled by TSQLServer.Model
    // - returns true on success
    function UnLock(Table: TSQLRecordClass; aID: integer): boolean; override;
    {/ end a transaction (implements REST END Member)
     - write all pending TSQLVirtualTableJSON data to the disk }
    procedure Commit(SessionID: cardinal); override;
    /// Execute directly all SQL statement (POST SQL on ModelRoot URI)
................................................................................
    // - call it just after Create, before TSQLRestServerDB.CreateMissingTables;
    // warning: if you don't call this method before CreateMissingTable method
    // is called, the table will be created as a regular table by the main
    // database engine, and won't be static
    // - can load the table content from a file if a file name is specified
    // (could be either JSON or compressed Binary format on disk)
    // - you can define a particular external engine by setting a custom class -
    // by default, it will create a TSQLRestStorageInMemory instance
    // - this data handles basic REST commands, since no complete SQL interpreter
    // can be implemented by TSQLRestStorage; to provide full SQL process,
    // you should better use a Virtual Table class, inheriting e.g. from
    // TSQLRecordVirtualTableAutoID associated with TSQLVirtualTableJSON/Binary
    // via a Model.VirtualTableRegister() call before TSQLRestServer.Create 
    // - return nil on any error, or an EModelException if the class is not in
    // the database model
    function StaticDataCreate(aClass: TSQLRecordClass;
      const aFileName: TFileName = ''; aBinaryFile: boolean=false;
      aServerClass: TSQLRestStorageClass=nil): TSQLRestStorage;
    /// call this method when the internal DB content is known to be invalid
    // - by default, all REST/CRUD requests and direct SQL statements are
    // scanned and identified as potentially able to change the internal SQL/JSON
    // cache used at SQLite3 database level; but some virtual tables (e.g.
    // TSQLRestStorageExternal classes defined in mORMotDB) could flush
    // the database content without proper notification
    // - this default implementation just do nothing, but SQlite3 unit
    // will call TSQLDataBase.CacheFlush method
    procedure FlushInternalDBCache; virtual;
    /// you can call this method in TThread.Execute to ensure that
    // the thread will be taken in account during process
    // - caller must specify the TThread instance running
    // - used e.g. for optExecInMainThread option in TServiceMethod.InternalExecute
    // - this default implementation will call the methods of all its internal
    // TSQLRestStorage instances
    // - this method shall be called from the thread just initiated: e.g.
    // if you call it from the main thread, it may fail to prepare resources
    procedure BeginCurrentThread(Sender: TThread); virtual;
    /// you can call this method just before a thread is finished to ensure
    // e.g. that the associated external DB connection will be released
    // - this default implementation will call the methods of all its internal
    // TSQLRestStorage instances, allowing e.g. TSQLRestStorageExternal
    // instances to clean their thread-specific connections
    // - this method shall be called from the thread about to be terminated: e.g.
    // if you call it from the main thread, it may fail to release resources
    // - it is set e.g. by TSQLite3HttpServer to be called from HTTP threads,
    // or by TSQLRestServerNamedPipeResponse for named-pipe server cleaning
    procedure EndCurrentThread(Sender: TThread); virtual;

................................................................................
    procedure AuthenticationRegister(const aMethods: array of TSQLRestServerAuthenticationClass); overload;
    /// call this method to remove an authentication method to the server
    procedure AuthenticationUnregister(aMethod: TSQLRestServerAuthenticationClass); overload;
    /// call this method to remove several authentication methods to the server
    procedure AuthenticationUnregister(const aMethods: array of TSQLRestServerAuthenticationClass); overload;
    /// add all published methods of a given object instance to the method-based
    // list of services
    // - all those published method signature should match TSQLRestServerCallBack
    procedure ServiceMethodRegisterPublishedMethods(const aPrefix: RawUTF8; aInstance: TObject);
    /// call this method to disable Authentication method check for a given
    // published method name
    // - by default, only Auth and TimeStamp methods do not require the RESTful
    // authentication of the URI; you may call this method to add another method
    // to the list (e.g. for returning some HTML content from a public URI)
    procedure ServiceMethodByPassAuthentication(const aMethodName: RawUTF8);
................................................................................
    property HandleAuthentication: boolean read fHandleAuthentication;
    /// read-only access to the list of registered server-side authentication
    // methods, used for session creation
    property AuthenticationSchemes: TSQLRestServerAuthenticationDynArray
      read fSessionAuthentication;
    /// access to the Server statistics
    property Stats: TSQLRestServerStats read fStats;
    /// retrieve the TSQLRestStorage instance used to store and manage
    // a specified TSQLRecordClass in memory
    // - has been associated by the StaticDataCreate method
    property StaticDataServer[aClass: TSQLRecordClass]: TSQLRestStorage
      read GetStaticDataServer;
    /// retrieve a running TSQLRestStorage virtual table
    // - associated e.g. to a 'JSON' or 'Binary' virtual table module, or may
    // return a TSQLRestStorageExternal instance (as defined in mORMotDB)
    // - this property will return nil if there is no Virtual Table associated
    // or if the corresponding module is not a TSQLVirtualTable
    // (e.g. "pure" static tables registered by StaticDataCreate would be
    // accessible only via StaticDataServer[], not via StaticVirtualTable[])
    // - has been associated by the TSQLModel.VirtualTableRegister method or
    // the VirtualTableExternalRegister() global function
    property StaticVirtualTable[aClass: TSQLRecordClass]: TSQLRestStorage
      read GetVirtualTable;
    /// this property can be left to its TRUE default value, to handle any
    // TSQLVirtualTableJSON static tables (module JSON or BINARY) with direct
    // calls to the storage instance
    // - is set to TRUE by default to enable faster Direct mode
    // - in Direct mode, GET/POST/PUT/DELETE of individual records (or BLOB fields)
    // from URI() will call directly the corresponding TSQLRestStorage
    // instance, for better speed for most used RESTful operations; but complex
    // SQL requests (e.g. joined SELECT) will rely on the main SQL engine
    // - if set to false, will use the main SQLite3 engine for all statements
    // (should not to be used normaly, because it will add unnecessary overhead)
    property StaticVirtualTableDirect: boolean read fVirtualTableDirect
      write fVirtualTableDirect;
    /// the class inheriting from TSQLAuthUser, as defined in the model
................................................................................
    // with cmd in POST/PUT with {object} as value or DELETE with ID
    // - returns an array of integers: '[200,200,...]' or '["OK"]' if all
    // returned status codes are 200 (HTML_SUCCESS)
    // - URI are either 'ModelRoot/TableName/Batch' or 'ModelRoot/Batch'
    procedure Batch(Ctxt: TSQLRestServerURIContext);
  end;

  /// REST class with direct access to an external database engine
  // - you can set an alternate per-table database engine by using this class
  // - this abstract class is to be overriden with a proper implementation (like
  // our TSQLRestStorageInMemory class or TSQLRestStorageExternal
  // as defined in mORMotDB unit)
  TSQLRestStorage = class(TSQLRest)
  protected
    fStoredClass: TSQLRecordClass;
    fStoredClassProps: TSQLModelRecordProperties;
    fStoredClassRecordProps: TSQLRecordProperties;
    fFileName: TFileName;
    fModified: boolean;
    fOwner: TSQLRestServer;
    fStorageCriticalSection: TRTLCriticalSection;
    fStorageCriticalSectionCount: integer;
    fBasicSQLCount: RawUTF8;
    fBasicSQLHasRows: array[boolean] of RawUTF8;
    /// any set bit in this field indicates UNIQUE field value
    fIsUnique: TSQLFieldBits;
    /// allow to force refresh for a given Static table
    // - default FALSE means to return the main TSQLRestServer.InternalState
    // - TRUE indicates that OutInternalState := cardinal(-1) will be returned
    fOutInternalStateForcedRefresh: boolean; 
    procedure StorageLock(WillModifyContent: boolean); virtual;
    procedure StorageUnLock; virtual;
    /// override this method if you want to update the refresh state
    // - returns FALSE if the static table content was not modified (default
    // method implementation is to always return FALSE)
    // - returns TRUE if the table has been refreshed and its content was modified:
    // therefore the client will know he'll need to refresh some content
    function RefreshedAndModified: boolean; virtual;
    /// overriden method calling the owner (if any) to guess if this record
................................................................................
    // - InternalBatchStart/Stop may safely use a lock for multithreading:
    // implementation in TSQLRestServer.Batch use a try..finally block
    procedure InternalBatchStop; virtual;
    /// TSQLRestServer.URI use it for Static.EngineList to by-pass virtual table
    // - this default implementation will return TRUE and replace SQL with
    // SQLSelectAll[true] if it SQL equals SQLSelectAll[false] (i.e. 'SELECT *')
    // - this method is called only if the WHERE clause of SQL refers to the
    // static table name only (not needed to check it twice)
    function AdaptSQLForEngineList(var SQL: RawUTF8): boolean; virtual;
  public
    /// initialize the storage data, reading it from a file if necessary
    // - data encoding on file is UTF-8 JSON format by default, or
    // should be some binary format if aBinaryFile is set to true (this virtual
    // method will just ignore this parameter, which will be used for overriden
    // constructor only)
    constructor Create(aClass: TSQLRecordClass; aServer: TSQLRestServer;
      const aFileName: TFileName = ''; aBinaryFile: boolean=false); reintroduce; virtual;
    /// finalize the storage instance
    destructor Destroy; override;
    /// you can call this method in TThread.Execute to ensure that
    // the thread will be taken in account during process
    // - this overriden method will do nothing (should have been already made
    // at TSQLRestServer caller level)
    // - children classes may inherit from this method to notify e.g.
    // a third party process (like proper OLE initialization)
    procedure BeginCurrentThread(Sender: TThread); virtual;
    /// you can call this method just before a thread is finished to ensure
    // e.g. that the associated external DB connection will be released
    // - this overriden method will do nothing (should have been already made
    // at TSQLRestServer caller level)
    // - children classes may inherit from this method to notify e.g.
    // a third party process (like proper OLE initialization)
    procedure EndCurrentThread(Sender: TThread); virtual;

    /// implement TSQLRest unlocking (UNLOCK verb)
    // - to be called e.g. after a Retrieve() with forupdate=TRUE
    // - locking is handled at (Owner.)Model level
    // - returns true on success
    function UnLock(Table: TSQLRecordClass; aID: integer): boolean; override;
    /// overriden method for direct in-memory database engine call
    // - made public since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    // - do nothing method: will return FALSE (aka error)
    function EngineUpdateField(TableModelIndex: integer;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
    /// create one index for all specific FieldNames at once
    // - do nothing method: will return FALSE (aka error)
    function CreateSQLMultiIndex(Table: TSQLRecordClass; const FieldNames: array of RawUTF8;
      Unique: boolean; IndexName: RawUTF8=''): boolean; virtual;
    /// search for a numerical field value
    // - return true on success (i.e. if some values have been added to ResultID)
    // - store the results into the ResultID dynamic array
    // - faster than OneFieldValues method, which creates a temporary JSON content
    // - this default implementation will call the overloaded SearchField()
    // value after conversion of the FieldValue into RawUTF8
    function SearchField(const FieldName: RawUTF8; FieldValue: Integer;
................................................................................

    /// read only access to the file name specified by constructor
    // - you can call the TSQLRestServer.StaticDataCreate method to
    // update the file name of an already instancied static table
    property FileName: TFileName read fFileName write fFileName;
    /// read only access to a boolean value set to true if table data was modified
    property Modified: boolean read fModified write fModified;
    /// read only access to the class defining the record type stored in this
    // REST storage
    property StoredClass: TSQLRecordClass read fStoredClass;
    /// read only access to the ORM properties of the associated record type
    // - may be nil if this instance is not associated with a TSQLModel
    property StoredClassProps: TSQLModelRecordProperties read fStoredClassProps;
    /// read only access to the RTTI properties of the associated record type
    property StoredClassRecordProps: TSQLRecordProperties read fStoredClassRecordProps;
    /// read only access to the TSQLRestServer using this in-memory database
    property Owner: TSQLRestServer read fOwner;
  end;

  /// event prototype called by FindWhereEqual() method
  TFindWhereEqualEvent = procedure(aDest: pointer; aRec: TSQLRecord; aIndex: integer) of object;

  /// abstract REST server exposing some internal TSQLRecord-based methods
  TSQLRestStorageRecordBased = class(TSQLRestStorage)
  protected
    function EngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer; override;
    function EngineUpdate(TableModelIndex, ID: integer; const SentData: RawUTF8): boolean; override;
  public
    /// manual Add of a TSQLRecord
    // - returns the ID created on success
    // - returns -1 on failure (not UNIQUE field value e.g.)
    // - on success, the Rec instance is added to the Values[] list: caller
    // doesn't need to Free it
    function AddOne(Rec: TSQLRecord; ForceID: boolean): integer; virtual; abstract;
................................................................................
    /// manual Retrieval of a TSQLRecord field values
    // - an instance of the associated static class is created
    // - and all its properties are filled from the Items[] values
    // - caller can modify these properties, then use UpdateOne() if the changes
    // have to be stored inside the Items[] list
    // - calller must always free the returned instance
    // - returns NIL if any error occured, e.g. if the supplied aID was incorrect
    // - method available since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function GetOne(aID: integer): TSQLRecord; virtual; abstract;
    /// manual Update of a TSQLRecord field values
    // - Rec.ID specifies which record is to be updated
    // - will update all properties, including BLOB fields and such
    // - returns TRUE on success, FALSE on any error (e.g. invalid Rec.ID)
    // - method available since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function UpdateOne(Rec: TSQLRecord): boolean; overload; virtual; abstract;
    /// manual Update of a TSQLRecord field values from an array of TSQLVar
    // - will update all properties, including BLOB fields and such
    // - returns TRUE on success, FALSE on any error (e.g. invalid Rec.ID)
    // - method available since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    // - this default implementation will create a temporary TSQLRecord instance
    // with the supplied Values[], and will call overloaded UpdateOne() method
    function UpdateOne(ID: integer; const Values: TSQLVarDynArray): boolean; overload; virtual;
  end;

  /// class able to handle a O(1) hashed-based search of a property in a TList
  // - used e.g. to hash TSQLRestStorageInMemory field values
  TListFieldHash = class(TObjectHash)
  protected
    fValues: TList;
    fField: integer;
    fProp: TSQLPropInfo;
    fCaseInsensitive: boolean;
    /// overriden method to hash an item
................................................................................
  // common client, by using the TSQLRestServer.StaticDataCreate method:
  // it allows an unique access for both SQLite3 and Static databases
  // - handle basic REST commands, no SQL interpreter is implemented: only
  // valid SQL command is "SELECT Field1,Field2 FROM Table WHERE ID=120;", i.e
  // a one Table SELECT with one optional "WHERE fieldname = value" statement;
  // if used within a TSQLVirtualTableJSON, you'll be able to handle any kind of
  // SQL statement (even joined SELECT or such) with this memory-stored database
  // - our TSQLRestStorage database engine is very optimized and is a lot
  // faster than SQLite3 for such queries - but its values remain in RAM,
  // therefore it is not meant to deal with more than 100,000 rows
  // - data can be stored and retrieved from a file (JSON format is used by
  // default, if BinaryFile parameter is left to false; a proprietary compressed
  // binary format can be used instead) if a file name is supplied at creating
  // the TSQLRestStorageInMemory instance
  TSQLRestStorageInMemory = class(TSQLRestStorageRecordBased)
  protected
    fValue: TObjectList;
    /// true if IDs are sorted (which is the default behavior of this class),
    // for fastest ID2Index() by using a binary search algorithm
    fIDSorted: boolean;
    fCommitShouldNotUpdateFile: boolean;
    fBinaryFile: boolean;
................................................................................
    function GetJSONValues(Stream: TStream; Expand, withID: boolean;
      const Fields: TSQLFieldBits; WhereField: integer; const WhereValue: RawUTF8;
      FoundLimit,FoundOffset: integer): PtrInt;
    /// TSQLRestServer.URI use it for Static.EngineList to by-pass virtual table
    // - overriden method to handle basic queries as handled by EngineList()
    function AdaptSQLForEngineList(var SQL: RawUTF8): boolean; override;
    /// overriden methods for direct in-memory database engine thread-safe process
    function EngineRetrieve(TableModelIndex, ID: integer): RawUTF8; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    function EngineUpdate(TableModelIndex, ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineRetrieveBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineDeleteWhere(TableModelIndex: integer; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
  public
    /// initialize the server data, reading it from a file if necessary
    // - data encoding on file is UTF-8 JSON format by default, or
    // should be some binary format if aBinaryFile is set to true
    constructor Create(aClass: TSQLRecordClass; aServer: TSQLRestServer;
      const aFileName: TFileName = ''; aBinaryFile: boolean=false); override;
................................................................................
    // compression algorithm), with variable-length record storage: e.g. a 27 KB
    // Dali1.json content is stored into a 6 KB Dali2.data file
    // (this data has a text redundant field content in its FirstName field);
    // 502 KB People.json content is stored into a 92 KB People.data file
    // - returns the number of bytes written into Stream
    function SaveToBinary(Stream: TStream): integer;
    /// if file was modified, the file is updated on disk
    // - this method is called automaticaly when the TSQLRestStorage
    // instance is destroyed: should should want to call in in some cases,
    // in order to force the data to be saved regularly
    // - do nothing if the table content was not modified
    // - will write JSON content by default, or binary content if BinaryFile
    // property was set to true
    procedure UpdateFile;
    /// retrieve the index in Items[] of a particular ID
................................................................................
    /// manual Retrieval of a TSQLRecord field values
    // - an instance of the associated static class is created
    // - and all its properties are filled from the Items[] values
    // - caller can modify these properties, then use UpdateOne() if the changes
    // have to be stored inside the Items[] list
    // - calller must always free the returned instance
    // - returns NIL if any error occured, e.g. if the supplied aID was incorrect
    // - method available since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function GetOne(aID: integer): TSQLRecord; override;
    /// manual Update of a TSQLRecord field values
    // - Rec.ID specifies which record is to be updated
    // - will update all properties, including BLOB fields and such
    // - returns TRUE on success, FALSE on any error (e.g. invalid Rec.ID)
    // - method available since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function UpdateOne(Rec: TSQLRecord): boolean; override;
    /// manual Update of a TSQLRecord field values from a TSQLVar array
    // - will update all properties, including BLOB fields and such
    // - returns TRUE on success, FALSE on any error (e.g. invalid Rec.ID)
    // - method available since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function UpdateOne(ID: integer; const Values: TSQLVarDynArray): boolean; override;
    /// overriden method for direct in-memory database engine call
    // - made public since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function EngineDelete(TableModelIndex, ID: integer): boolean; override;
    /// overriden method for direct in-memory database engine call
    // - made public since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function EngineUpdateField(TableModelIndex: integer;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
    /// overriden method for direct in-memory database engine call
    function UpdateBlobFields(Value: TSQLRecord): boolean; override;
    /// overriden method for direct in-memory database engine call
    function RetrieveBlobFields(Value: TSQLRecord): boolean; override;
    /// overriden method for direct in-memory database engine call
    function TableRowCount(Table: TSQLRecordClass): integer; override;
................................................................................
    /// set this property to TRUE if you want the COMMIT statement not to
    // update the associated TSQLVirtualTableJSON
    property CommitShouldNotUpdateFile: boolean read fCommitShouldNotUpdateFile
      write fCommitShouldNotUpdateFile;
  end;

  /// REST server with direct access to a memory database, to be used as
  // an external SQLite3 Virtual table
  // - this is the kind of in-memory table expected by TSQLVirtualTableJSON,
  // in order to be consistent with the internal DB cache
  TSQLRestStorageInMemoryExternal = class(TSQLRestStorageInMemory)
  public
    /// this overriden method will notify the Owner when the internal DB content
    // is known to be invalid
    // - by default, all REST/CRUD requests and direct SQL statements are
    // scanned and identified as potentially able to change the internal SQL/JSON
    // cache used at SQLite3 database level; but TSQLVirtualTableJSON virtual
    // tables could flush the database content without proper notification
    // - this overriden implementation will call Owner.FlushInternalDBCache
    procedure StorageLock(WillModifyContent: boolean); override;
  end;

  /// a REST server using only in-memory tables
  // - this server will use TSQLRestStorageInMemory instances to handle
  // the data in memory, and optionally persist the data on disk as JSON or
  // binary files
  // - so it will not handle all SQL requests, just basic CRUD commands on
  // separated tables
  // - at least, it will compile as a TSQLRestServer without complaining for
  // pure abstract methods; it can be used to host some services if database
  // and ORM needs are basic (e.g. if only authentication and CRUD are needed)
  TSQLRestServerFullMemory = class(TSQLRestServer)
  protected
    fFileName: TFileName;
    fBinaryFile: Boolean;
    fStaticDataCount: cardinal;
    function GetStatic(Table: TSQLRecordClass): TSQLRestStorageInMemory;
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    function EngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer; override;
    function EngineUpdate(TableModelIndex, ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDelete(TableModelIndex, ID: integer): boolean; override;
    function EngineDeleteWhere(TableModelIndex: integer; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    function EngineRetrieveBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateField(TableModelIndex: integer;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
  public
    /// initialize a REST server with a database file
    // - all classes of the model will be created as TSQLRestStorageInMemory
    // - then data persistence will be created using aFileName
    // - if aFileName is left void (''), data will not be persistent
    constructor Create(aModel: TSQLModel; const aFileName: TFileName='';
      aBinaryFile: boolean=false; aHandleUserAuthentication: boolean=false); reintroduce; virtual;
    /// write any modification on file (if needed), and release all used memory
    destructor Destroy; override;
    /// Missing tables are created if they don't exist yet for every TSQLRecord
................................................................................
    procedure UpdateToFile; virtual;
    /// overriden method for direct in-memory database engine call
    // - not implemented: always return false
    function EngineExecuteAll(const aSQL: RawUTF8): boolean; override;
    /// the file name used for data persistence
    property FileName: TFileName read fFileName write fFileName;
    /// set if the file content is to be compressed binary, or standard JSON
    // - it will use TSQLRestStorageInMemory LoadFromJSON/LoadFromBinary
    // SaveToJSON/SaveToBinary methods for optimized storage
    property BinaryFile: Boolean read fBinaryFile write fBinaryFile;
  end;

  /// a REST server using a TSQLRestClient for all its ORM process
  // - this server will use an internal TSQLRestClient instance to handle
  // all ORM operations (i.e. access to objects)
................................................................................
  // easily implement a proxy architecture (for instance, as a DMZ for
  // publishing services, but letting ORM process stay out of scope)
  TSQLRestServerRemoteDB = class(TSQLRestServer)
  protected
    fClient: TSQLRestClient;
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    function EngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer; override;
    function EngineUpdate(TableModelIndex, ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDelete(TableModelIndex, ID: integer): boolean; override;
    function EngineDeleteWhere(TableModelIndex: integer; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    function EngineRetrieveBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateField(TableModelIndex: integer;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
  public
    /// initialize a REST server associated to a given TSQLRestClient instance
    // - the specified TSQLRestClient will be used for all ORM and data process
    // - the supplied TSQLRestClient.Model will be used for TSQLRestServerRemoteDB
    // - note that the TSQLRestClient instance won't be freed - caller shall
    // manage its life time
................................................................................
    procedure SetForceBlobTransfert(Value: boolean);
    function GetForceBlobTransfertTable(aTable: TSQLRecordClass): Boolean;
    procedure SetForceBlobTransfertTable(aTable: TSQLRecordClass; aValue: Boolean);
    /// get a member from its ID (implements REST GET member)
    // - returns the data of this object as JSON
    // - override this method for proper data retrieval from the database engine
    // - this method must be implemented in a thread-safe manner
    function ClientRetrieve(TableModelIndex: integer; ID: integer;
      ForUpdate: boolean; var InternalState: cardinal; var Resp: RawUTF8): boolean; virtual; abstract;
    /// this method is called before updating any record
    // - should return FALSE to force no update
    // - can be use to update some field values just before saving to the database
    // (e.g. for digital signing purpose)
    // - this default method just return TRUE (i.e. OK to update)
    function BeforeUpdateEvent(Value: TSQLRecord): Boolean; virtual;
    /// overriden method which will call ClientRetrieve()
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; override;
  public
    /// create a new member (implements REST POST Collection)
    // - URI is 'ModelRoot/TableName' with POST method
    // - if SendData is true, content of Value is sent to the server as JSON
    // - if ForceID is true, client sends the Value.ID field to use this ID
    // - server must return Status 201/HTML_CREATED on success
    // - server must send on success an header entry with
................................................................................
    // - on success, Value.ID is updated with the new ROWID
    // - if aValue is TSQLRecordFTS3, Value.ID is stored to the virtual table
    function Add(Value: TSQLRecord; SendData: boolean; ForceID: boolean=false): integer; override;
    /// update a member (implements REST PUT Collection/Member)
    // - URI is 'ModelRoot/TableName/TableID' with PUT method
    // - server must return Status 200/HTML_SUCCESS OK on success
    function Update(Value: TSQLRecord): boolean; override;




    /// get a member from its ID (implements REST GET Collection/Member)
    // - URI is 'ModelRoot/TableName/TableID' with GET method
    // - server must return Status 200/HTML_SUCCESS OK on success
    // - if ForUpdate is true, the REST method is LOCK and not GET: it tries to lock
    // the corresponding record, then retrieve its content; caller has to call
    // UnLock() method after Value usage, to release the record
    function Retrieve(aID: integer; Value: TSQLRecord; ForUpdate: boolean=false): boolean; override;














    /// get a member from its ID (implements REST GET Collection/Member)
    // - URI is 'ModelRoot/TableName/TableID' with GET method
    // - returns true on server returned 200/HTML_SUCCESS OK success, false on error
    // - set Refreshed to true if the content changed
    function Refresh(aID: integer; Value: TSQLRecord; var Refreshed: boolean): boolean;

    /// retrieve a list of members as a TSQLTable (implements REST GET Collection)
................................................................................
    // - a next call to InternalCheckOpen method shall re-open the connection 
    procedure InternalClose; virtual; abstract;
    /// calls 'ModelRoot/TableName/TableID' with appropriate REST method
    // - uses GET method if ForUpdate is false
    // - uses LOCK method if ForUpdate is true
    function URIGet(Table: TSQLRecordClass; ID: integer; var Resp: RawUTF8;
      ForUpdate: boolean=false): Int64Rec;



    // overriden methods

    function ClientRetrieve(TableModelIndex, ID: integer; ForUpdate: boolean;
      var InternalState: cardinal; var Resp: RawUTF8): boolean; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    function EngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer; override;
    function EngineUpdate(TableModelIndex, ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDelete(TableModelIndex, ID: integer): boolean; override;
    function EngineDeleteWhere(TableModelIndex: integer; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    function EngineRetrieveBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateField(TableModelIndex: integer;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
    procedure SetLastException(E: Exception=nil; ErrorCode: integer=HTML_BADREQUEST);
  public
    /// initialize REST client instance
    constructor Create(aModel: TSQLModel); override;
    /// release memory and close client connection
    // - also unlock all still locked records by this client
................................................................................
    /// a set of features of a Virtual Table
    Features: TSQLVirtualTableFeatures;
    /// the associated cursor class
    CursorClass: TSQLVirtualTableCursorClass;
    /// the associated TSQLRecord class
    // - used to retrieve the field structure with all collations
    RecordClass: TSQLRecordClass;
    /// the associated TSQLRestStorage class used for storage
    // - is e.g. TSQLRestStorageInMemory for TSQLVirtualTableJSON,
    // TSQLRestStorageExternal for TSQLVirtualTableExternal, or nil for
    // TSQLVirtualTableLog
    StaticClass: TSQLRestStorageClass;
    /// can be used to customize the extension of the filename
    // - the '.' is not to be included
    FileExtension: TFileName;
  end;

  {/ parent class able to define a Virtual Table module
   - in order to implement a new Virtual Table type, you'll have to define a so
................................................................................
    function FileName(const aTableName: RawUTF8): TFileName; virtual;
    /// the Virtual Table module features
    property Features: TSQLVirtualTableFeatures read fFeatures.Features;
    /// the associated virtual table class
    property TableClass: TSQLVirtualTableClass read fTableClass;
    /// the associated virtual table cursor class
    property CursorClass: TSQLVirtualTableCursorClass read fFeatures.CursorClass;
    /// the associated TSQLRestStorage class used for storage
    // - e.g. returns TSQLRestStorageInMemory for TSQLVirtualTableJSON,
    // or TSQLRestStorageExternal for TSQLVirtualTableExternal, or
    // either nil for TSQLVirtualTableLog
    property StaticClass: TSQLRestStorageClass read fFeatures.StaticClass;
    /// the associated TSQLRecord class
    // - is mostly nil, e.g. for TSQLVirtualTableJSON
    // - used to retrieve the field structure for TSQLVirtualTableLog e.g.
    property RecordClass: TSQLRecordClass read fFeatures.RecordClass;
    /// the extension of the filename (without any left '.')
    property FileExtension: TFileName read fFeatures.FileExtension;
    /// the full path to be used for the filename
................................................................................
     virtual methods to allow content writing to the virtual table
   - the same virtual table mechanism can be used with several database module,
     with diverse database engines }
  TSQLVirtualTable = class
  protected
    fModule: TSQLVirtualTableModule;
    fTableName: RawUTF8;
    fStatic: TSQLRestStorage;
    fStaticTableIndex: integer;
  public
    /// create the virtual table access instance
    // - the created instance will be released when the virtual table will be
    // disconnected from the DB connection (e.g. xDisconnect method for SQLite3)
    // - shall raise an exception in case of invalid parameters (e.g. if the
    // supplied module is not associated to a TSQLRestServer instance)
    // - aTableName will be checked against the current aModule.Server.Model
    // to retrieve the corresponding TSQLRecordVirtualTableAutoID class and
    // create any associated Static: TSQLRestStorage instance
    constructor Create(aModule: TSQLVirtualTableModule; const aTableName: RawUTF8;
      FieldCount: integer; Fields: PPUTF8CharArray); virtual;
    /// release the associated memory, especially the Static instance
    destructor Destroy; override;
    /// retrieve the corresponding module name
    // - will use the class name, triming any T/TSQL/TSQLVirtual/TSQLVirtualTable* 
    // - when the class is instanciated, it will be faster to retrieve the same
................................................................................
    // SQLite database - it could make bad written code slow even with Virtual
    // Tables
    function Transaction(aState: TSQLVirtualTableTransaction; aSavePoint: integer): boolean; virtual;
    /// called to rename the virtual table
    // - by default, returns false, i.e. always fails
    function Rename(const NewName: RawUTF8): boolean; virtual;
    /// the associated virtual table storage instance
    // - can be e.g. a TSQLRestStorageInMemory for TSQLVirtualTableJSON,
    // or a TSQLRestStorageExternal for TSQLVirtualTableExternal, or nil
    // for TSQLVirtualTableLog
    property Static: TSQLRestStorage read fStatic;
    /// the associated virtual table storage index in its Model.Tables[] array
    property StaticTableIndex: integer read fStaticTableIndex;
  end;

  {/ abstract class able to define a Virtual Table cursor
    - override the Search/HasData/Column/Next abstract virtual methods to
    implement the search process }
  TSQLVirtualTableCursor = class
  protected
................................................................................
    // - will check the fCurrent/fMax protected properties values
    function Next: boolean; override;
    /// called to begin a search in the virtual table
    // - this no-op version will mark EOF, i.e. fCurrent=0 and fMax=-1
    function Search(const Prepared: TSQLVirtualTablePrepared): boolean; override;
  end;
  
  {/ A Virtual Table cursor for reading a TSQLRestStorageInMemory content
    - this is the cursor class associated to TSQLVirtualTableJSON }
  TSQLVirtualTableCursorJSON = class(TSQLVirtualTableCursorIndex)
  public
    /// called to begin a search in the virtual table
    // - the TSQLVirtualTablePrepared parameters were set by
    // TSQLVirtualTable.Prepare and will contain both WHERE and ORDER BY statements
    // (retrieved by x_BestIndex from a TSQLite3IndexInfo structure)
................................................................................
    function Search(const Prepared: TSQLVirtualTablePrepared): boolean; override;
    /// called to retrieve a column value of the current data row into a TSQLVar
    // - if aColumn=-1, will return the row ID as varInt64 into aResult
    // - will return false in case of an error, true on success
    function Column(aColumn: integer; var aResult: TSQLVar): boolean; override;
  end;

  {/ A TSQLRestStorageInMemory-based virtual table using JSON storage
   - for ORM access, you should use TSQLModel.VirtualTableRegister method to
     associated this virtual table module to a TSQLRecordVirtualTableAutoID class
   - transactions are not handled by this module
   - by default, no data is written on disk: you will need to call explicitly
     aServer.StaticVirtualTable[aClass].UpdateToFile for file creation or refresh
   - file extension is set to '.json' }
  TSQLVirtualTableJSON = class(TSQLVirtualTable)
................................................................................
    // - column order follows the Structure method, i.e.
    // StoredClassRecordProps.Fields[] order
    // - returns true on success, false otherwise
    // - does nothing by default, and returns false, i.e. always fails
    function Update(oldRowID, newRowID: Int64; var Values: TSQLVarDynArray): boolean; override;
  end;

  {/ A TSQLRestStorageInMemory-based virtual table using Binary storage
   - for ORM access, you should use TSQLModel.VirtualTableRegister method to
     associated this virtual table module to a TSQLRecordVirtualTableAutoID class
   - transactions are not handled by this module
   - by default, no data is written on disk: you will need to call explicitly
     aServer.StaticVirtualTable[aClass].UpdateToFile for file creation or refresh
   - binary format is more efficient in term of speed and disk usage than
     the JSON format implemented by TSQLVirtualTableJSON
................................................................................
    InitializeCriticalSection(fAcquireExecution[cmd].Lock);
end;

destructor TSQLRest.Destroy;
var cmd: TSQLRestServerURIContextCommand;
begin
  if (fModel<>nil) and (fModel.fRestOwner=self) then
    // make sure we are the Owner (TSQLRestStorage has fModel<>nil e.g.)
    FreeAndNil(fModel);
  fServices.Free;
  fCache.Free;
  for cmd := Low(cmd) to high(cmd) do begin
    DeleteCriticalSection(fAcquireExecution[cmd].Lock);
    fAcquireExecution[cmd].Thread.Free;
  end;
................................................................................
  result := false;
  if (Strings<>nil) and (self<>nil) and (Table<>nil) then
  try
    {$ifndef LVCL}
    Strings.BeginUpdate;
    {$endif}
    Strings.Clear;
    T := ExecuteList([Table],
      SQLFromSelect(Table.SQLTableName,'ID,'+FieldName,WhereClause,''));
    if T<>nil then
    try
      if (T.FieldCount=2) and (T.RowCount>0) then begin
        for Row := 1 to T.RowCount do begin // ignore Row 0 i.e. field names
          aID := GetInteger(T.Get(Row,0));
          Strings.AddObject(UTF8ToString(T.GetU(Row,1)),pointer(aID));
................................................................................
  WhereClause: RawUTF8): TSQLTableJSON;
begin
  if (self=nil) or (Table=nil) then
    result := nil else
    with Table.RecordProps do
    if (PosEx(RawUTF8(','),FieldName,1)=0) and not IsFieldName(FieldName) then
      result := nil else // prevent SQL error
      result := ExecuteList([Table],
        SQLFromSelect(SQLTableName,FieldName,WhereClause,''));
end;     

function TSQLRest.InternalListRecordsJSON(Table: TSQLRecordClass;
  const WhereClause: RawUTF8): TSQLTableJSON;
begin
  if (self=nil) or (Table=nil) then
    result := nil else
    result := ExecuteList([Table],Model.Props[Table].SQLFromSelectWhere('*',WhereClause));
end;

function TSQLRest.MultiFieldValues(Table: TSQLRecordClass; FieldNames: RawUTF8;
  const WhereClause: RawUTF8): TSQLTableJSON;
var P: PUTF8Char;
    aFieldName: RawUTF8;
begin
................................................................................
      P := pointer(FieldNames);
      repeat
        aFieldName := Trim(GetNextItem(P));
        if not Props.IsFieldName(aFieldName) then
          exit; // invalid field name
      until P=nil;
    end;
    result := ExecuteList([Table],SQLFromSelectWhere(FieldNames,WhereClause));
  end;
end;

function TSQLRest.MultiFieldValues(Table: TSQLRecordClass; const FieldNames: RawUTF8;
  WhereClauseFormat: PUTF8Char; const BoundsSQLWhere: array of const): TSQLTableJSON;
begin
  result := MultiFieldValues(Table,FieldNames,FormatUTF8(WhereClauseFormat,[],BoundsSQLWhere));
................................................................................
    for i := 0 to high(FieldName) do
      if not IsFieldName(FieldName[i]) then
        exit else // prevent SQL error
        if SQL='' then
          SQL := 'SELECT '+FieldName[i] else
          SQL := SQL+','+FieldName[i];
    SQL := SQL+' FROM '+SQLTableName+' WHERE '+WhereClause+' LIMIT 1;';
    T := ExecuteList([Table],SQL);
    if T<>nil then
    try
      if (T.FieldCount<>length(FieldName)) or (T.RowCount<=0) then
        exit;
      // get field values from the first (and unique) row
      for i := 0 to T.FieldCount-1 do
        FieldValue[i] := T.fResults[T.FieldCount+i];
................................................................................
end;

function TSQLRest.Retrieve(const SQLWhere: RawUTF8; Value: TSQLRecord): boolean;
var T: TSQLTable;
begin
  if (self=nil) or (Value=nil) then
    T := nil else
    T := ExecuteList([PSQLRecordClass(Value)^],
      Model.Props[PSQLRecordClass(Value)^].SQLFromSelectWhere('*',SQLWhere+' LIMIT 1'));
  if T=nil then
    result := false else
    try
      if T.RowCount>=1 then begin
        Value.FillFrom(T,1); // fetch data from first result row
        result := true;
      end else
................................................................................
  try
    result := TObjectList.Create;
    T.ToObjectList(result,Table);
  finally
    T.Free;
  end;
end;

function TSQLRest.Retrieve(aID: integer; Value: TSQLRecord;
  ForUpdate: boolean): boolean;
var TableIndex: integer; // used by EngineRetrieve() for SQL statement caching
    Resp: RawUTF8;
begin // this version handles locking and use fast EngineRetrieve() method
  // check parameters
  result := false;
  if Value=nil  then
    exit; // avoid GPF
  Value.fID := 0;
  if (self=nil) or (aID=0) then
    exit;
  TableIndex := Model.GetTableIndexExisting(PSQLRecordClass(Value)^);
  // try to lock before retrieval (if ForUpdate)
  if ForUpdate and not Model.Lock(TableIndex,aID) then
    exit;
  // try to retrieve existing JSON from internal cache
  Resp := fCache.Retrieve(TableIndex,aID);
  if Resp='' then begin
    // get JSON object '{...}' in Resp from corresponding EngineRetrieve() method
    Resp := EngineRetrieve(TableIndex,aID);
    if Resp='' then
      exit;
  end;
  // fill Value from JSON if was correctly retrieved
  Value.FillFrom(Resp);
  result := true;
end;

function TSQLRest.Retrieve(WhereClauseFmt: PUTF8Char; const Args,Bounds: array of const;
  Value: TSQLRecord): boolean;
begin
  result := Retrieve(FormatUTF8(WhereClauseFmt,Args,Bounds),Value);
end;

................................................................................
function TSQLRest.RecordCanBeUpdated(Table: TSQLRecordClass; ID: integer; Action: TSQLEvent;
  ErrorMsg: PRawUTF8 = nil): boolean;
begin
  result := true; // accept by default -> override this method to customize this
end;

function TSQLRest.Delete(Table: TSQLRecordClass; ID: integer): boolean;
var TableIndex: integer;
begin
  TableIndex := Model.GetTableIndexExisting(Table);
  if not RecordCanBeUpdated(Table,ID,seDelete) then
    result := false else begin
    fCache.NotifyDeletion(TableIndex,ID);
    result := EngineDelete(TableIndex,ID);
  end;
end;

function TSQLRest.InternalDeleteNotifyAndGetIDs(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
  var IDs: TIntegerDynArray): boolean;
var i: integer;
begin
  result := false;
  if (not OneFieldValues(Table,'RowID',SQLWhere,IDs)) or
     (IDs=nil) then
    exit;
................................................................................
    fCache.NotifyDeletion(Table,IDs[i]);
  result := true;
end;

function TSQLRest.Delete(Table: TSQLRecordClass; const SQLWhere: RawUTF8): boolean;
var IDs: TIntegerDynArray;
begin
  if InternalDeleteNotifyAndGetIDs(Table,SQLWhere,IDs) then
    result := EngineDeleteWhere(Model.GetTableIndexExisting(Table),SQLWhere,IDs) else
    result := false;
end;

function TSQLRest.Delete(Table: TSQLRecordClass; FormatSQLWhere: PUTF8Char;
  const BoundsSQLWhere: array of const): boolean;
begin
  result := Delete(Table,FormatUTF8(FormatSQLWhere,[],BoundsSQLWhere));
end;

function TSQLRest.Update(Value: TSQLRecord): boolean;
var JSONValues: RawUTF8;
    TableIndex: integer;
begin

  if (self=nil) or (Value=nil) or (Value.fID=0) or
    not RecordCanBeUpdated(PSQLRecordClass(Value)^,Value.fID,seUpdate) then begin
    result := false; // current user don't have enough right to update this record
    exit;
  end;
  TableIndex := Model.GetTableIndexExisting(PSQLRecordClass(Value)^);
  Value.ComputeFieldsBeforeWrite(self,seUpdate); // update sftModTime fields
  JSONValues := Value.GetJSONValues(true,false,soUpdate); // expanded + without ID
  fCache.Notify(Value,soUpdate); // JSONValues on update may not be enough for cache
  result := EngineUpdate(TableIndex,Value.fID,JSONValues);
end;

function TSQLRest.Update(aTable: TSQLRecordClass; aID: integer;
  const aSimpleFields: array of const): boolean;
var Value: TSQLRecord;
begin
  result := false; // means error
................................................................................
      exit;
    Value.fID := aID;
    result := Update(Value);
  finally
    Value.Free;
  end;
end;

function TSQLRest.Add(Value: TSQLRecord; SendData: boolean;
  ForceID: boolean=false): integer;
var JSONValues: RawUTF8;
    TableIndex: integer;
begin
  if (self=nil) or (Value=nil) then begin
    result := 0;
    exit;
  end;
  TableIndex := Model.GetTableIndexExisting(PSQLRecordClass(Value)^);
  if SendData then begin
    Value.ComputeFieldsBeforeWrite(self,seAdd); // update TModTime/TCreateTime fields
    if Model.TableProps[TableIndex].Kind in INSERT_WITH_ID then
      ForceID := true;
    JSONValues := Value.GetJSONValues(true, // true=expanded
      (Value.fID<>0) and ForceID,soInsert);
  end else
    JSONValues := '';
  // on success, returns the new ROWID value; on error, returns 0
  result := EngineAdd(TableIndex,JSONValues); // will call static
  // on success, Value.ID is updated with the new ROWID
  Value.fID := result;
  if SendData then
    fCache.Notify(PSQLRecordClass(Value)^,result,JSONValues,soInsert);
end;

function TSQLRest.Add(aTable: TSQLRecordClass; const aSimpleFields: array of const;
  ForcedID: integer=0): integer;
var Value: TSQLRecord;
begin
  result := 0; // means error
  if (self=nil) or (aTable=nil) then
................................................................................
    qoSoundsLikeFrench,
    qoSoundsLikeSpanish:
      result := PSynSoundEx(Reference)^.UTF8(Value);
  end;
end;

function TSQLRest.RetrieveBlob(Table: TSQLRecordClass; aID: integer;

  const BlobFieldName: RawUTF8; out BlobStream: THeapMemoryStream): boolean;
var BlobData: TSQLRawBlob;
begin
  BlobStream := THeapMemoryStream.Create;
  result := RetrieveBlob(Table,aID,BlobFieldName,BlobData);
  if not result or (BlobData='') then
    exit;
  BlobStream.Write(pointer(BlobData)^,length(BlobData));
................................................................................
begin
  if (self=nil) or (BlobData=nil) or (BlobSize<0) then
    result := false else begin
    SetString(Blob,PAnsiChar(BlobData),BlobSize);
    result := UpdateBlob(Table,aID,BlobFieldName,Blob);
  end;
end;

function TSQLRest.RetrieveBlob(Table: TSQLRecordClass; aID: integer;
  const BlobFieldName: RawUTF8; out BlobData: TSQLRawBlob): boolean;
var BlobField: PPropInfo;
begin
  result := false;
  if (self=nil) or (aID<=0) then
    exit;
  BlobField := Table.RecordProps.BlobFieldPropFromRawUTF8(BlobFieldName);
  if BlobField=nil then
    exit;
  result := EngineRetrieveBlob(
    Model.GetTableIndexExisting(Table),aID,BlobField,BlobData);
end;

function TSQLRest.UpdateBlob(Table: TSQLRecordClass; aID: integer;
  const BlobFieldName: RawUTF8; const BlobData: TSQLRawBlob): boolean;
var BlobField: PPropInfo;
begin
  result := false;
  if (self=nil) or (aID<=0) or not RecordCanBeUpdated(Table,aID,seUpdate) then
    exit;
  BlobField := Table.RecordProps.BlobFieldPropFromRawUTF8(BlobFieldName);
  if BlobField=nil then
    exit;
  result := EngineUpdateBlob(
    Model.GetTableIndexExisting(Table),aID,BlobField,BlobData);
end;

function TSQLRest.UpdateBlobFields(Value: TSQLRecord): boolean;
var BlobData: RawByteString;
    TableIndex, i: integer;
begin
  result := false;
  if (Value=nil) or (Value.fID<=0) then
    exit;
  with Value.RecordProps do
  if BlobFields<>nil then begin
    TableIndex := self.fModel.GetTableIndexExisting(PSQLRecordClass(Value)^);
    for i := 0 to high(BlobFields) do begin
      GetLongStrProp(Value,BlobFields[i].PropInfo,BlobData);
      if not EngineUpdateBlob(TableIndex,Value.fID,BlobFields[i].PropInfo,BlobData) then
        exit;
    end;
  end;
  result := true;
end;

function TSQLRest.RetrieveBlobFields(Value: TSQLRecord): boolean;
var BlobData: TSQLRawBlob;
    TableIndex, i: integer;
begin
  result := false;
  if (Self=nil) or (Value=nil) or (Value.fID<=0) then
    exit;
  with Value.RecordProps do
  if BlobFields<>nil then begin
    TableIndex := self.fModel.GetTableIndexExisting(PSQLRecordClass(Value)^);
    for i := 0 to high(BlobFields) do
      if EngineRetrieveBlob(TableIndex,Value.fID,BlobFields[i].PropInfo,BlobData) then
        SetLongStrProp(Value,BlobFields[i].PropInfo,BlobData) else
        exit;
  end;
  result := true;
end;

function TSQLRest.TableRowCount(Table: TSQLRecordClass): integer;
var T: TSQLTableJSON;
begin
  if (self=nil) or (Table=nil) then
    T := nil else
    T := ExecuteList([Table],
      'SELECT Count(*) FROM '+Table.RecordProps.SQLTableName);
  if T<>nil then
  try
    Result := T.GetAsInteger(1,0);
  finally
    T.Free;
  end else
    Result := -1;
................................................................................
end;

function TSQLRest.TableHasRows(Table: TSQLRecordClass): boolean;
var T: TSQLTableJSON;
begin
  if (self=nil) or (Table=nil) then
    T := nil else
    T := ExecuteList([Table],
      'SELECT RowID FROM '+Table.RecordProps.SQLTableName+' LIMIT 1');
  if T<>nil then
  try
    Result := T.RowCount>0;
  finally
    T.Free;
  end else
    Result := false;
end;

function TSQLRest.ExecuteList(const Tables: array of TSQLRecordClass; const SQL: RawUTF8): TSQLTableJSON;
var JSON: RawUTF8;
begin
  JSON := EngineList(SQL,false);
  if JSON<>'' then
    result := TSQLTableJSON.CreateFromTables(Tables,SQL,JSON) else
    result := nil;
end;

function TSQLRest.MainFieldValue(Table: TSQLRecordClass; ID: Integer;
   ReturnFirstIfNoUnique: boolean=false): RawUTF8;
begin
  if (self=nil) or (Table=nil) or (ID<=0) then
    result := '' else begin
    result := Table.RecordProps.MainFieldName(Table,ReturnFirstIfNoUnique);
................................................................................
  end;
end;

function TSQLRestClientURI.UpdateFromServer(const Data: array of TObject; out Refreshed: boolean;
  PCurrentRow: PInteger): boolean;
// notes about refresh mechanism:
// - if server doesn't implement InternalState, its value is 0 -> always refresh
// - if any TSQLTableJSON or TSQLRecord belongs to a TSQLRestStorage,
// the Server stated fInternalState=cardinal(-1) for them -> always refresh
var i: integer;
    State: cardinal;
    Resp: RawUTF8;
    T: TSQLTableJSON;
    TRefreshed: boolean; // to check for each Table refresh
const TState: array[boolean] of TOnTableUpdateState = (tusNoChange,tusChanged);
................................................................................
        exit else
        InternalState := Hi;
    result := TSQLTableJSON.CreateFromTables(Tables,SQL,Resp); // get data
  end;
  result.fInternalState := InternalState;
end;






procedure TSQLRestClientURI.SessionClose;
var tmp: RawUTF8;
begin
  if (self<>nil) and (fSessionUser<>nil) and
     (fSessionID<>CONST_AUTHENTICATION_SESSION_NOT_STARTED) then
  try
    // notify session closed to server
................................................................................
    // may not contain all cached fields -> delete from cache
    fCache.NotifyDeletion(Value.RecordClass,Value.fID) else
    fCache.Notify(Value,soUpdate);
  result := fBatchCount;
  inc(fBatchCount);
end;

function TSQLRestClientURI.EngineAdd(TableModelIndex: integer; 
  const SentData: RawUTF8): integer;
var P: PUTF8Char;
    url, Head: RawUTF8;
begin
  result := 0;
  url := Model.URI[Model.Tables[TableModelIndex]];
  if URI(url,'POST',nil,@Head,@SentData).Lo<>HTML_CREATED then
    exit; // response must be '201 Created'
  P := pointer(Head); // we need to check the headers
  if P<>nil then
  repeat
    // find ID from 'Location: Member Entry URI' header entry
    if IdemPChar(P,'LOCATION:') then begin // 'Location: root/People/11012' e.g.
      inc(P,9);
................................................................................
    end;
    while not (P^ in [#0,#13]) do inc(P);
    if P^=#0 then break else inc(P);
    if P^=#10 then inc(P);
  until false;
end;

function TSQLRestClientURI.EngineDelete(TableModelIndex, ID: integer): boolean;
var url: RawUTF8;
begin
  url := Model.getURIID(Model.Tables[TableModelIndex],ID);
  result := URI(url,'DELETE').Lo=HTML_SUCCESS;
end;

function TSQLRestClientURI.EngineDeleteWhere(TableModelIndex: Integer;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
var url: RawUTF8;
begin  // ModelRoot/TableName?where=WhereClause to delete members
  url := Model.getURI(Model.Tables[TableModelIndex])+'?where='+UrlEncode(SQLWhere);
  result := URI(url,'DELETE').Lo=HTML_SUCCESS;
end;

function TSQLRestClientURI.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean; ReturnedRowCount: PPtrInt): RawUTF8;
begin
  if (self=nil) or (SQL='') or (ReturnedRowCount<>nil) or 
     (URI(Model.Root,'GET',@result,nil,@SQL).Lo<>HTML_SUCCESS) then
    result := ''
end;

function TSQLRestClientURI.ClientRetrieve(TableModelIndex, ID: integer;
  ForUpdate: boolean; var InternalState: cardinal; var Resp: RawUTF8): boolean;
begin
  if cardinal(TableModelIndex)<=cardinal(Model.fTablesMax) then
  with URIGet(Model.Tables[TableModelIndex],ID,Resp,ForUpdate) do
    if Lo=HTML_SUCCESS then begin
      InternalState := Hi;
      result := true;
    end else
      result := false else
      result := false;
end;

function TSQLRestClientURI.EngineRetrieveBlob(TableModelIndex, aID: integer;
  BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
var url: RawUTF8;
begin
  if (self=nil) or (aID<=0) or (BlobField=nil) then
    result := false else begin
    // URI is 'ModelRoot/TableName/TableID/BlobFieldName' with GET method
    url := Model.getURICallBack(BlobField^.Name,Model.Tables[TableModelIndex],aID);
    result := URI(url,'GET',@BlobData).Lo=HTML_SUCCESS;
  end;
end;

function TSQLRestClientURI.EngineUpdate(TableModelIndex, ID: integer;
  const SentData: RawUTF8): boolean;
begin
  result := URI(Model.getURIID(Model.Tables[TableModelIndex],ID),'PUT',
    nil,nil,@SentData).Lo=HTML_SUCCESS;
end;

function TSQLRestClientURI.EngineUpdateBlob(TableModelIndex, aID: integer;

  BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean;
var url, Head: RawUTF8;
begin
  Head := 'Content-Type: application/octet-stream';
  if (self=nil) or (aID<=0) or (BlobField=nil) then
    result := false else begin
    // PUT ModelRoot/TableName/TableID/BlobFieldName
    url := FormatUTF8('%/%/%',[Model.URI[Model.Tables[TableModelIndex]],aID,BlobField^.Name]);
    result := URI(url,'PUT',nil,@Head,@BlobData).Lo=HTML_SUCCESS;
  end;
end;

function TSQLRestClientURI.EngineUpdateField(TableModelIndex: integer;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
begin
  if (self=nil) or (TableModelIndex<0) then
    result := false else
    // PUT ModelRoot/TableName?setname=..&set=..&wherename=..&where=..
    result := URI(FormatUTF8('%?setname=%&set=%&wherename=%&where=%',
      [Model.URI[Model.Tables[TableModelIndex]],
       SetFieldName,UrlEncode(SetValue),
       WhereFieldName,UrlEncode(WhereValue)]),'PUT').Lo=HTML_SUCCESS;
end;


{ TSQLRestServer }

{$ifdef MSWINDOWS}
const
................................................................................
    sleep(200); // way some time any request is finished in another thread
  end;
  // close any running named-pipe or GDI-messages server instance
  CloseServerNamedPipe;
  CloseServerMessage;
{$endif}
  for i := 0 to high(fStaticData) do
    // free all TSQLRestStorage objects and update file if necessary
    fStaticData[i].Free;
  fSessions.Free;
  DeleteCriticalSection(fSessionCriticalSection);
  inherited Destroy; // calls fServices.Free which will update fStats
  if not InheritsFrom(TSQLRestStorage) then
    InternalLog(Stats.DebugMessage,sllInfo);
  fStats.Free; 
end;

function TSQLRestServer.GetStaticDataServer(aClass: TSQLRecordClass): TSQLRestStorage;
begin
  if (self<>nil) and (fStaticData<>nil) then
   result := fStaticData[Model.GetTableIndexExisting(aClass)] else
   result := nil;
end;

function TSQLRestServer.GetStaticDataServerOrVirtualTable(aClass: TSQLRecordClass;
  Kind: PSQLRestServerKind=nil): TSQLRestStorage;
begin
  if (aClass=nil) or (fStaticData=nil) and (fStaticVirtualTable=nil) then
    result := nil else
    result := GetStaticDataServerOrVirtualTable(Model.GetTableIndexExisting(aClass),Kind);
end;

function TSQLRestServer.GetStaticDataServerOrVirtualTable(aTableIndex: integer;
  Kind: PSQLRestServerKind=nil): TSQLRestStorage;
begin
  result := nil;
  if Kind<>nil then
    Kind^ := sMainEngine;
  if aTableIndex>=0 then begin
    if fStaticData<>nil then
      result := fStaticData[aTableIndex];
................................................................................
      result := fStaticVirtualTable[aTableIndex];
      if Kind<>nil then
        Kind^ := sVirtualTable;
    end;
  end;
end;

function TSQLRestServer.GetVirtualTable(aClass: TSQLRecordClass): TSQLRestStorage;
var i: integer;
begin
  result := nil;
  if fStaticVirtualTable<>nil then begin
    i := Model.GetTableIndexExisting(aClass);
    if (i>=0) and (Model.TableProps[i].Kind in IS_CUSTOM_VIRTUAL) then
      result := fStaticVirtualTable[i];
  end;
end;

function TSQLRestServer.StaticDataCreate(aClass: TSQLRecordClass;
  const aFileName: TFileName; aBinaryFile: boolean;
  aServerClass: TSQLRestStorageClass): TSQLRestStorage;
var i: integer;
begin
  result := nil;
  i := Model.GetTableIndexExisting(aClass);
  if fStaticData<>nil then
    result := fStaticData[i];
  if result<>nil then
    // class already registered -> update file name
    result.fFileName := aFileName else begin
    // class not already registered -> register now
    if aServerClass=nil then
      aServerClass := TSQLRestStorageInMemory; // default in-memory engine
    result := aServerClass.Create(aClass,self,aFileName,aBinaryFile);
    if length(fStaticData)<length(Model.Tables) then
      SetLength(fStaticData,length(Model.Tables));
    fStaticData[i] := result;
  end;
end;

................................................................................
      SetLength(result,P-pointer(Result)); // trim right
  end;
  if result='' then // by default, a SQLite3 query is ordered by ID
    result := 'RowID';
end;

procedure TSQLRestServer.SetNoAJAXJSON(const Value: boolean);

begin
  fNoAJAXJSON := Value;



end;

function TSQLRestServer.InternalAdaptSQL(TableIndex: integer; var SQL: RawUTF8): TSQLRestStorage;
begin
  result := nil;
  if (self<>nil) and (TableIndex>=0) then begin // SQL refers to this unique table
    if fStaticData<>nil then
      // no SQLite3 module available for fStaticData[] -> we need to
      // retrieve manualy any static table from the SQL SELECT statement
      result := fStaticData[TableIndex];
................................................................................
        result := nil;
    end;
  end;
end;

function TSQLRestServer.InternalListRawUTF8(TableIndex: integer; const SQL: RawUTF8): RawUTF8;
var aSQL: RawUTF8;
    Static: TSQLRestStorage;
begin
  aSQL := SQL;
  Static := InternalAdaptSQL(TableIndex,aSQL);
  if Static<>nil then
     // this SQL statement is handled by direct connection, faster adaptation
    result := Static.EngineList(aSQL) else
    // complex TSQLVirtualTableJSON/External queries will rely on virtual table
    result := MainEngineList(SQL,false,nil);
  if result='[]'#$A then
    result := '';
end;




















































function TSQLRestServer.UnLock(Table: TSQLRecordClass; aID: integer): boolean;
begin
  result := Model.UnLock(Table,aID);
end;

procedure TSQLRestServer.Commit(SessionID: cardinal);
var i: integer;
begin
  inherited Commit(SessionID);
  if self<>nil then
    for i := 0 to high(fStaticVirtualTable) do
    if fStaticVirtualTable[i]<>nil then
    with TSQLRestStorageInMemory(fStaticVirtualTable[i]) do 
      if InheritsFrom(TSQLRestStorageInMemory) and not CommitShouldNotUpdateFile then
        UpdateFile; // will do nothing if not Modified
end;














































function TSQLRestServer.Delete(Table: TSQLRecordClass; ID: integer): boolean;

begin
  result := inherited Delete(Table,ID); // call EngineDelete








  if result then
    // force relational database coherency (i.e. our FOREIGN KEY implementation)
    AfterDeleteForceCoherency(Table,ID);
end;

function TSQLRestServer.Delete(Table: TSQLRecordClass; const SQLWhere: RawUTF8): boolean;
var IDs: TIntegerDynArray;
    i: integer;

begin
  result := false;
  if not InternalDeleteNotifyAndGetIDs(Table,SQLWhere,IDs) then
    exit;



  result := EngineDeleteWhere(Model.GetTableIndexExisting(Table),SQLWhere,IDs);
  if result then
    // force relational database coherency (i.e. our FOREIGN KEY implementation)
    for i := 0 to high(IDs) do
      AfterDeleteForceCoherency(Table,IDs[i]);
end;

function TSQLRestServer.TableRowCount(Table: TSQLRecordClass): integer;
var Static: TSQLRestStorage;
begin
  Static := GetStaticDataServerOrVirtualTable(Table);
  if Static<>nil then // faster direct call
    result := Static.TableRowCount(Table) else
    result := inherited TableRowCount(Table);
end;

function TSQLRestServer.TableHasRows(Table: TSQLRecordClass): boolean;
var Static: TSQLRestStorage;
begin
  Static := GetStaticDataServerOrVirtualTable(Table);
  if Static<>nil then // faster direct call
    result := Static.TableHasRows(Table) else
    result := inherited TableHasRows(Table);
end;



































function TSQLRestServer.UpdateBlobFields(Value: TSQLRecord): boolean;
var Static: TSQLRestStorage;

begin // overriden method to update all BLOB fields at once
  if (Value=nil) or (Value.fID<=0) then
    result := false else begin
    Static := GetStaticDataServerOrVirtualTable(PSQLRecordClass(Value)^);
    if Static<>nil then // faster direct call
      result := Static.UpdateBlobFields(Value) else
      result := inherited UpdateBlobFields(Value);
  end;
end;

function TSQLRestServer.RetrieveBlobFields(Value: TSQLRecord): boolean;
var Static: TSQLRestStorage;
begin // overriden method to update all BLOB fields at once
  if Value=nil then
    result := false else begin
    Static := GetStaticDataServerOrVirtualTable(PSQLRecordClass(Value)^);
    if Static<>nil then // faster direct call
      result := Static.RetrieveBlobFields(Value) else
      result := inherited RetrieveBlobFields(Value);
  end;
................................................................................

function TSQLRestServer.AfterDeleteForceCoherency(Table: TSQLRecordClass;
  aID: integer): boolean;
var T: integer;
    Tab: TSQLRecordClass;
    Where: PtrUInt;
    RecRef: TRecordReference;
    Static: TSQLRestStorage;
    W: RawUTF8;
begin
  result := true; // success if no property found
  {$ifndef CPU64}
  Where := 0; // make compiler happy
  {$endif}
  RecRef := RecordReference(Model,Table,aID);
................................................................................
    else continue;
    end;
    // set Field=0 where Field references aID
    UInt32ToUTF8(Where,W);
    Tab := Model.Tables[TableIndex];
    Static := GetStaticDataServerOrVirtualTable(Tab);
    if Static<>nil then // fast direct call
       result := Static.EngineUpdateField(TableIndex,
         FieldType.Name,'0',FieldType.Name,W) else
       result := MainEngineUpdateField(TableIndex,
         FieldType.Name,'0',FieldType.Name,W);
  end;
end;

function TSQLRestServer.CreateSQLMultiIndex(Table: TSQLRecordClass;
  const FieldNames: array of RawUTF8; Unique: boolean; IndexName: RawUTF8=''): boolean;
var SQL: RawUTF8;
    i, TableIndex: integer;
    Props: TSQLRecordProperties;
    Static: TSQLRestStorage;
begin
  result := false;
  if (Self=nil) or InheritsFrom(TSQLRestStorage) or (high(FieldNames)<0) then
    exit; // avoid endless loop for TSQLRestStorage with no overriden method
  TableIndex := Model.GetTableIndexExisting(Table);
  if fStaticVirtualTable<>nil then begin
    Static := fStaticVirtualTable[TableIndex];
    if Static<>nil then begin
      if not Static.InheritsFrom(TSQLRestStorageInMemory) then
         // will try to create an index on the static table (e.g. for external DB)
         result := Static.CreateSQLMultiIndex(Table,FieldNames,Unique,IndexName);
      exit;
    end;
  end;
  if (high(FieldNames)=0) and IsRowID(pointer(FieldNames[0])) then begin
    result := true; // SQLite3 has always its ID/RowID primary key indexed
................................................................................
          SQL := Call.InBody;
        SQLisSelect := isSelect(pointer(SQL));
        if (SQL<>'') and
          (SQLisSelect or (reSQL in Call.RestAccessRights^.AllowRemoteExecute)) then begin
          // no user check for SELECT: see TSQLAccessRights.GET comment
          Static := Server.InternalAdaptSQL(
            Server.Model.GetTableIndexFromSQLSelect(SQL,false),SQL);
          if Static<>nil then  begin
            TableEngine := Static;
            Call.OutBody := TableEngine.EngineList(SQL);
          end else
            Call.OutBody := Server.MainEngineList(SQL,false,nil);
          // security note: only first statement is run by EngineList()
          if Call.OutBody<>'' then begin // got JSON list '[{...}]' ?
            Call.OutStatus := HTML_SUCCESS;  // 200 OK
            if not SQLisSelect then
              inc(Server.fStats.fModified);
          end;
        end;
................................................................................
            if Server.Model.Lock(TableIndex,TableID) then
              Method := mGET; // mark successfully locked
        if Method<>mLOCK then
          if URIBlobFieldName<>'' then begin
            // GET ModelRoot/TableName/TableID/BlobFieldName: retrieve BLOB field content
            Blob := Table.RecordProps.BlobFieldPropFromRawUTF8(URIBlobFieldName);
            if Blob<>nil then begin
              if TableEngine.EngineRetrieveBlob(TableIndex,
                   TableID,Blob,TSQLRawBlob(Call.OutBody)) then begin
                Call.OutHead := HEADER_CONTENT_TYPE+
                  GetMimeContentType(pointer(Call.OutBody),Length(Call.OutBody));
                Call.OutStatus := HTML_SUCCESS; // 200 OK
              end;
            end;
          end else begin
            // GET ModelRoot/TableName/TableID: retrieve a member content, JSON encoded
            Call.OutBody := Server.fCache.Retrieve(TableIndex,TableID);
            if Call.OutBody='' then begin
              // get JSON object '{...}'
              if Static<>nil then
                Call.OutBody := Static.EngineRetrieve(TableIndex,TableID) else
                Call.OutBody := Server.MainEngineRetrieve(TableIndex,TableID);
              // cache if expected
              Server.fCache.Notify(TableIndex,TableID,Call.OutBody,soSelect);
            end;
            if Call.OutBody<>'' then // if something was found
              Call.OutStatus := HTML_SUCCESS; // 200 OK
          end;
      end else
................................................................................
            if SameTextU(SQLDir,'DESC') then
              SQLSort := SQLSort+' DESC'; // allow DESC, default is ASC
            SQLWhere := SQLWhere+' ORDER BY '+SQLSort;
          end;
          SQLWhere := trim(SQLWhere);
          if (SQLResults<>0) and not ContainsUTF8(pointer(SQLWhere),'LIMIT ') then begin
            if (Server.URIPagingParameters.SendTotalRowsCountFmt<>nil) then begin
              ResultList := Server.ExecuteList([Table],
                Server.Model.TableProps[TableIndex].SQLFromSelectWhere('Count(*)',SQLWhere));
              if ResultList<>nil then begin
                SQLTotalRowsCount := ResultList.GetAsInteger(1,0);
                ResultList.Free;
              end;
            end;
            SQLWhere := FormatUTF8('% LIMIT % OFFSET %',[SQLWhere,SQLResults,SQLStartIndex]);
................................................................................
      end;
    end else
    // here, Table<>nil and TableIndex in [0..MAX_SQLTABLES-1]
    if not (TableIndex in Call.RestAccessRights^.POST) then // check User
      Call.OutStatus := HTML_NOTALLOWED else
    if TableID<0 then begin
      // ModelRoot/TableName with possible JSON SentData: create a new member
      TableID := TableEngine.EngineAdd(TableIndex,Call.InBody);
      if TableID<>0 then begin
        Call.OutStatus := HTML_CREATED; // 201 Created
        Call.OutHead := 'Location: '+URI+'/'+
          {$ifndef ENHANCEDRTL}Int32ToUtf8{$else}IntToStr{$endif}(TableID);
        Server.fCache.Notify(TableIndex,TableID,Call.InBody,soInsert);
        inc(Server.fStats.fModified);
      end;
................................................................................
      if not Server.RecordCanBeUpdated(Table,TableID,seUpdate,@CustomErrorMsg) then
        Call.OutStatus := HTML_NOTMODIFIED else begin
        OK := false;
        if URIBlobFieldName<>'' then begin
          // PUT ModelRoot/TableName/TableID/BlobFieldName: update BLOB field content
          Blob := Table.RecordProps.BlobFieldPropFromRawUTF8(URIBlobFieldName);
          if Blob<>nil then
            OK := TableEngine.EngineUpdateBlob(TableIndex,TableID,Blob,Call.InBody);
        end else begin
          // ModelRoot/TableName/TableID with JSON SentData: update a member
          OK := TableEngine.EngineUpdate(TableIndex,TableID,Call.InBody);
          if OK then
            Server.fCache.NotifyDeletion(TableIndex,TableID); // flush (no CreateTime in JSON)
        end;
        if OK then begin
          Call.OutStatus := HTML_SUCCESS; // 200 OK
          inc(Server.fStats.fModified);
        end;
................................................................................
        repeat
          UrlDecodeValue(Parameters,'SETNAME=',SQLSelect);
          UrlDecodeValue(Parameters,'SET=',SQLDir);
          UrlDecodeValue(Parameters,'WHERENAME=',SQLSort);
          UrlDecodeValue(Parameters,'WHERE=',SQLWhere,@Parameters);
        until Parameters=nil;
        if (SQLSelect<>'') and (SQLDir<>'') and (SQLSort<>'') and (SQLWhere<>'') then
          if TableEngine.EngineUpdateField(TableIndex,
               SQLSelect,SQLDir,SQLSort,SQLWhere) then begin
            Call.OutStatus := HTML_SUCCESS; // 200 OK
            inc(Server.fStats.fModified);
          end;
      end;
  end;
  mDELETE:
    if Table<>nil then
      if TableID>0 then
        // ModelRoot/TableName/TableID to delete a member
        if not (TableIndex in Call.RestAccessRights^.DELETE) then // check User
          Call.OutStatus := HTML_NOTALLOWED else
        if not Server.RecordCanBeUpdated(Table,TableID,seDelete,@CustomErrorMsg) then
          Call.OutStatus := HTML_NOTMODIFIED else begin
          if TableEngine.EngineDelete(TableIndex,TableID) and
             Server.AfterDeleteForceCoherency(Table,TableID) then begin
            Call.OutStatus := HTML_SUCCESS; // 200 OK
            Server.fCache.NotifyDeletion(TableIndex,TableID);
            inc(Server.fStats.fModified);
          end;
        end else
      if Parameters<>nil then
................................................................................
              end;
              break;
            end;
          until Parameters=nil;
        end;
  mBEGIN: begin      // BEGIN TRANSACTION
    // TSQLVirtualTableJSON/External will rely on SQLite3 module
    // and also TSQLRestStorageInMemory, since COMMIT/ROLLBACK have Static=nil
    if Server.TransactionBegin(Table,Session) then begin
      if (Static<>nil) and (StaticKind=sVirtualTable) then
        Static.TransactionBegin(Table,Session) else
      if (Static=nil) and (Server.fTransactionTable<>nil) then begin
        Static := Server.StaticVirtualTable[Server.fTransactionTable];
        if Static<>nil then
          Static.TransactionBegin(Table,Session);
................................................................................
end;

procedure TSQLRestServer.Batch(Ctxt: TSQLRestServerURIContext);
var EndOfObject: AnsiChar;
    wasString, OK: boolean;
    TableName, Value, ErrMsg: RawUTF8;
    URIMethod, RunningBatchURIMethod: TSQLURIMethod;
    RunningBatchStatic: TSQLRestStorage; { TODO: allow nested batch between tables? }
    Sent, Method, MethodTable: PUTF8Char;
    AutomaticTransactionPerRow, RowCountForCurrentTransaction: cardinal;
    i, ID, Count: integer;
    Results: TIntegerDynArray;
    RunTable: TSQLRecordClass;
    RunTableIndex: integer;
    RunStatic: TSQLRestStorage;
    RunStaticKind: TSQLRestServerKind;
begin
  if Ctxt.Method<>mPUT then begin
    Ctxt.Error('PUT only');
    exit;
  end;
  Sent := pointer(Ctxt.Call.InBody);
................................................................................
        Ctxt.Error('Missing CMD');
        exit;
      end;
      MethodTable := PosChar(Method,'@');
      if MethodTable=nil then begin // e.g. '{"Table":[...,"POST":{object},...]}'
        RunTable := Ctxt.Table;
        RunStatic := Ctxt.Static;
        RunTableIndex := Ctxt.TableIndex;
        RunStaticKind := Ctxt.StaticKind;
      end else begin                // e.g. '[...,"POST@Table":{object},...]'
        RunTableIndex := Model.GetTableIndex(MethodTable+1);
        if RunTableIndex<0 then begin
          Ctxt.Error('Unknown @Table');
          exit;
        end;
        RunTable := Model.Tables[RunTableIndex];
        RunStatic := GetStaticDataServerOrVirtualTable(RunTableIndex,@RunStaticKind);
      end;
      if Count>=length(Results) then
        SetLength(Results,Count+256+Count shr 3);
      // get CRUD method (ignoring @ char if appended after method name)
      if IdemPChar(Method,'DELETE') then
        URIMethod := mDELETE else
      if IdemPChar(Method,'POST') then
................................................................................
      mDELETE: begin // '{"Table":[...,"DELETE":ID,...]}' or '[...,"DELETE@Table":ID,...]'
        ID := GetInteger(GetJSONField(Sent,Sent,@wasString,@EndOfObject));
        if (ID<=0) or wasString or
           not RecordCanBeUpdated(RunTable,ID,seDelete,@ErrMsg) then begin
          Ctxt.Error(ErrMsg,HTML_NOTMODIFIED);
          exit;
        end;


        OK := EngineDelete(RunTableIndex,ID);
        if OK then begin
          fCache.NotifyDeletion(RunTable,ID);
          if (RunningBatchStatic<>nil) or
             AfterDeleteForceCoherency(RunTable,ID) then
            Results[Count] := HTML_SUCCESS; // 200 OK
        end;
      end;
................................................................................
      mPOST: begin // '{"Table":[...,"POST":{object},...]}' or '[...,"POST@Table":{object},...]'
        Value := JSONGetObject(Sent,nil,EndOfObject);
        if (Sent=nil) or
           not RecordCanBeUpdated(RunTable,0,seAdd,@ErrMsg)  then begin
          Ctxt.Error(ErrMsg,HTML_NOTMODIFIED);
          exit;
        end;


        ID := EngineAdd(RunTableIndex,Value);
        Results[Count] := ID;
        fCache.Notify(RunTable,ID,Value,soInsert);
      end;
      mPUT: begin // '{"Table":[...,"PUT":{object},...]}' or '[...,"PUT@Table":{object},...]'
        Value := JSONGetObject(Sent,@ID,EndOfObject);
        if (Sent=nil) or (Value='') then
          exit;


        OK := EngineUpdate(RunTableIndex,ID,Value);
        if OK then begin
          Results[Count] := HTML_SUCCESS; // 200 OK
          fCache.NotifyDeletion(RunTable,ID); // Value does not have CreateTime e.g.
          // or may be complete -> update won't work as expected -> delete from cache
        end;
      end;
      else exit; // unknown method
................................................................................
end;

function TSQLRestServer.CacheWorthItForTable(aTableIndex: cardinal): boolean;
begin
  if self=nil then
    result := false else
    result := (aTableIndex>=cardinal(length(fStaticData))) or
      (not fStaticData[aTableIndex].InheritsFrom(TSQLRestStorageInMemory));
end;

procedure TSQLRestServer.BeginCurrentThread(Sender: TThread);
var i: integer;
    CurrentThreadId: cardinal;
begin
  InterlockedIncrement(fStats.fCurrentThreadCount);
................................................................................
    if (aEvent=seUpdateBlob) and Assigned(OnBlobUpdateEvent) then
      result := OnBlobUpdateEvent(self,seUpdate,aTable,aID,aIsBlobFields^) else
      result := true else
  if Assigned(OnUpdateEvent) then
    result := OnUpdateEvent(self,aEvent,aTable,aID) else
    result := true; // true on success, false if error (but action continues)
end;

function TSQLRestServer.EngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer;
var Static: TSQLRestStorage;
begin
  Static := GetStaticDataServerOrVirtualTable(TableModelIndex);
  if Static=nil then
    result := MainEngineAdd(TableModelIndex,SentData) else
    result := Static.EngineAdd(TableModelIndex,SentData);
end;

function TSQLRestServer.EngineRetrieve(TableModelIndex, ID: integer): RawUTF8;
var Static: TSQLRestStorage;
begin
  Static := GetStaticDataServerOrVirtualTable(TableModelIndex);
  if Static=nil then
    result := MainEngineRetrieve(TableModelIndex,ID) else
    result := Static.EngineRetrieve(TableModelIndex,ID);
end;

function TSQLRestServer.EngineList(const SQL: RawUTF8; ForceAJAX: Boolean;
  ReturnedRowCount: PPtrInt): RawUTF8;
var Static: TSQLRestStorage;
    StaticSQL: RawUTF8;
begin
  StaticSQL := SQL;
  Static := InternalAdaptSQL(Model.GetTableIndexFromSQLSelect(SQL,false),StaticSQL);
  if Static=nil then
    result := MainEngineList(SQL,ForceAJAX,ReturnedRowCount) else
    result := Static.EngineList(StaticSQL,ForceAJAX,ReturnedRowCount);
end;

function TSQLRestServer.EngineUpdate(TableModelIndex, ID: integer;
  const SentData: RawUTF8): boolean;
var Static: TSQLRestStorage;
begin
  Static := GetStaticDataServerOrVirtualTable(TableModelIndex);
  if Static=nil then
    result := MainEngineUpdate(TableModelIndex,ID,SentData) else
    result := Static.EngineUpdate(TableModelIndex,ID,SentData);
end;

function TSQLRestServer.EngineDelete(TableModelIndex, ID: integer): boolean;
var Static: TSQLRestStorage;
begin
  Static := GetStaticDataServerOrVirtualTable(TableModelIndex);
  if Static=nil then
    result := MainEngineDelete(TableModelIndex,ID) else
    result := Static.EngineDelete(TableModelIndex,ID);
end;

function TSQLRestServer.EngineDeleteWhere(TableModelIndex: integer;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
var Static: TSQLRestStorage;
begin
  Static := GetStaticDataServerOrVirtualTable(TableModelIndex);
  if Static=nil then
    result := MainEngineDeleteWhere(TableModelIndex,SQLWhere,IDs) else
    result := Static.EngineDeleteWhere(TableModelIndex,SQLWhere,IDs);
end;

function TSQLRestServer.EngineRetrieveBlob(TableModelIndex, aID: integer;
  BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
var Static: TSQLRestStorage;
begin
  Static := GetStaticDataServerOrVirtualTable(TableModelIndex);
  if Static=nil then
    result := MainEngineRetrieveBlob(TableModelIndex,aID,BlobField,BlobData) else
    result := Static.EngineRetrieveBlob(TableModelIndex,aID,BlobField,BlobData);
end;

function TSQLRestServer.EngineUpdateBlob(TableModelIndex, aID: integer;
  BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean;
var Static: TSQLRestStorage;
begin
  Static := GetStaticDataServerOrVirtualTable(TableModelIndex);
  if Static=nil then
    result := MainEngineUpdateBlob(TableModelIndex,aID,BlobField,BlobData) else
    result := Static.EngineUpdateBlob(TableModelIndex,aID,BlobField,BlobData);
end;

function TSQLRestServer.EngineUpdateField(TableModelIndex: integer;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; 
var Static: TSQLRestStorage;
begin
  Static := GetStaticDataServerOrVirtualTable(TableModelIndex);
  if Static=nil then
    result := MainEngineUpdateField(TableModelIndex,SetFieldName,SetValue,
      WhereFieldName,WhereValue) else
    result := Static.EngineUpdateField(TableModelIndex,SetFieldName,SetValue,
      WhereFieldName,WhereValue);
end;

function CurrentServiceContext: TServiceRunningContext;
begin
  result := ServiceContext;
end;


................................................................................
  if (self=nil) or (Freq=0) then
    result := '0' else
    result := MicroSecToString((ProcessTimeCounter*(1000*1000))div Freq);
end;
{$endif}


{ TSQLRestStorageRecordBased }

function TSQLRestStorageRecordBased.EngineAdd(TableModelIndex: integer;
  const SentData: RawUTF8): integer;
var Rec: TSQLRecord;
begin
  result := 0; // mark error
  if (self=nil) or (Model.Tables[TableModelIndex]<>fStoredClass) then
    exit;
  Rec := fStoredClass.Create;
  try
    Rec.FillFrom(SentData);
    StorageLock(true);
    try
      result := AddOne(Rec,Rec.fID>0);
    finally
      StorageUnLock;
    end;
  finally
    if result<=0 then
      Rec.Free; // on success, will be freed by fValue TObjectList
  end;
end;

function TSQLRestStorageRecordBased.EngineUpdate(TableModelIndex, ID: integer;
  const SentData: RawUTF8): boolean;
var Rec: TSQLRecord;
begin
  // this implementation won't handle partial fields update (e.g. BatchUpdate
  // after FillPrepare) - but TSQLRestStorageInMemory.EngineUpdate will
  if (ID<=0) or (TableModelIndex<0) or
     (Model.Tables[TableModelIndex]<>fStoredClass) then begin
    result := false; // mark error
    exit;
  end;
  StorageLock(true);
  try
    Rec := fStoredClass.Create;
    try
      Rec.FillFrom(SentData);
      Rec.fID := ID;
      result := UpdateOne(Rec);
    finally
      Rec.Free;
    end;
  finally
    StorageUnLock;
  end;
end;

function TSQLRestStorageRecordBased.UpdateOne(ID: integer;
  const Values: TSQLVarDynArray): boolean;
var Rec: TSQLRecord;
begin
  if (ID<=0) then begin
    result := false; // mark error
    exit;
  end;
  StorageLock(true);
  try
    Rec := fStoredClass.Create;
    try
      Rec.SetFieldSQLVars(Values);
      Rec.fID := ID;
      result := UpdateOne(Rec);
    finally
      Rec.Free;
    end;
  finally
    StorageUnLock;
  end;
end;


{ TSQLRestStorage }

function TSQLRestStorageInMemory.AddOne(Rec: TSQLRecord; ForceID: boolean): integer;
var ndx,i: integer;
begin
  if (self=nil) or (Rec=nil) then begin
    result := -1; // mark error
    exit;
  end;
  if ForceID then
................................................................................
      exit;
    end;
  fModified := true;
  if Owner<>nil then
     Owner.InternalUpdateEvent(seAdd,PSQLRecordClass(Rec)^,result,nil);
end;

function TSQLRestStorageInMemory.UniqueFieldsUpdateOK(aRec: TSQLRecord; aUpdateIndex: integer): boolean;
var i,ndx: integer;
begin
  if fUniqueFields<>nil then begin
    result := false;
    with fUniqueFields do
      for i := 0 to Count-1 do begin
        ndx := TListFieldHash(List[i]).Find(aRec);
................................................................................
        if (ndx>=0) and (ndx<>aUpdateIndex) then
          exit; // duplicate found at another entry
      end;
  end;
  result := true;
end;

function TSQLRestStorageInMemory.UniqueFieldHash(aFieldIndex: integer): TListFieldHash;
var i: integer;
begin
  if (fUniqueFields<>nil) and
     (cardinal(aFieldIndex)<cardinal(fStoredClassRecordProps.Fields.Count)) then
    with fUniqueFields do
      for i := 0 to Count-1 do begin
        result := List[i];
        if result.FieldIndex=aFieldIndex then
          exit;
      end;
  result := nil;
end;

constructor TSQLRestStorageInMemory.Create(aClass: TSQLRecordClass; aServer: TSQLRestServer;
  const aFileName: TFileName = ''; aBinaryFile: boolean=false);
var JSON: RawUTF8;
    Stream: TStream;
    F: integer;
begin
  inherited Create(aClass,aServer,aFileName,aBinaryFile);
  if (fStoredClassProps<>nil) and (fStoredClassProps.Kind in INSERT_WITH_ID) then
    raise EModelException.CreateFmt('%s: %s virtual table can''t be static',
      [fStoredClassRecordProps.SQLTableName,aClass.ClassName]);
  fBinaryFile := aBinaryFile;
  fValue := TObjectList.Create;
  fSearchRec := fStoredClass.Create;
  fIDSorted := true; // sorted by design of this class (may change in children)
  if (ClassType<>TSQLRestStorageInMemory) and (fStoredClassProps<>nil) then
    with fStoredClassProps do begin // used by AdaptSQLForEngineList() method
      fBasicUpperSQLSelect[false] := UpperCase(SQL.SelectAllWithRowID);
      SetLength(fBasicUpperSQLSelect[false],length(fBasicUpperSQLSelect[false])-1); // trim right ';'
      fBasicUpperSQLSelect[true] := StringReplaceAll(fBasicUpperSQLSelect[false],' ROWID,',' ID,');
    end;
  if not IsZero(fIsUnique) then begin
    fUniqueFields := TObjectList.Create;
................................................................................
      finally
        Stream.Free;
      end;
    end;
  end;
end;

function TSQLRestStorageInMemory.EngineDelete(TableModelIndex, ID: integer): boolean;
var i,F: integer;
begin
  if (self=nil) or (ID<=0) or (TableModelIndex<0) or
     (Model.Tables[TableModelIndex]<>fStoredClass) then begin
    result := false;
    exit;
  end;
  StorageLock(True);
  try
    i := IDToIndex(ID);
    if i<0 then
      result := false else begin
      if fUniqueFields<>nil then
        for F := 0 to fUniqueFields.Count-1 do
          TListFieldHash(fUniqueFields.List[F]).Invalidate;
      if Owner<>nil then // notify BEFORE deletion
         Owner.InternalUpdateEvent(seDelete,fStoredClass,ID,nil);
      fValue.Delete(i);  // TObjectList.Delete() will Free record
      fModified := true;
      result := true;
    end;
  finally
    StorageUnLock;
  end;
end;

function TSQLRestStorageInMemory.EngineDeleteWhere(TableModelIndex: Integer;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
var ndx: TIntegerDynArray;
    n,i: integer;
begin // RecordCanBeUpdated() has already been called
  result := false;
  n := length(IDs);
  SetLength(ndx,n);
  dec(n);
  StorageLock(True);
  try
    for i := 0 to n do begin
      ndx[i] := IDToIndex(IDs[i]);
      if ndx[i]<0 then
        exit;
    end;
    if fUniqueFields<>nil then
      for i := 0 to fUniqueFields.Count-1 do
        TListFieldHash(fUniqueFields.List[i]).Invalidate;
    if Owner<>nil then
      for i := 0 to n do
        Owner.InternalUpdateEvent(seDelete,fStoredClass,IDs[i],nil); // notify BEFORE deletion
    QuickSortInteger(pointer(ndx),0,n); // deletion a bit faster in reverse order
    for i := n downto 0 do
      fValue.Delete(ndx[i]);
    fModified := true;
    result := true;
  finally
    StorageUnLock;
  end;
end;

destructor TSQLRestStorageInMemory.Destroy;
begin
  UpdateFile;
  fValue.Free; // TObjectList.Destroy will free all stored TSQLRecord instances
  fUniqueFields.Free;
  fSearchRec.Free;
  inherited Destroy;
end;

function TSQLRestStorageInMemory.GetCount: integer;
begin
  if Self<>nil then
    result := fValue.Count else
    result := 0;
end;

function TSQLRestStorageInMemory.GetID(Index: integer): integer;
begin
  with fValue do
    if (self=nil) or (cardinal(Index)>=cardinal(Count)) then
      result := 0 else
      result := TSQLRecord(List[Index]).fID;
end;

function TSQLRestStorageInMemory.GetItem(Index: integer): TSQLRecord;
begin
  if self<>nil then
    with fValue do
      if cardinal(Index)>=cardinal(Count) then
        raise EORMException.Create('GetItem out of range') else
        result := List[Index] else
    result := nil;
end;

procedure TSQLRestStorageInMemory.GetJSONValuesEvent(aDest: pointer;
  aRec: TSQLRecord; aIndex: integer);
var W: TJSONSerializer absolute aDest;
begin
  aRec.GetJSONValues(W);
  W.Add(',');
end;

procedure TSQLRestStorageInMemory.AddIntegerDynArrayEvent(
  aDest: pointer; aRec: TSQLRecord; aIndex: integer);
var Ints: TList absolute aDest;
begin
  Ints.Add(pointer(aIndex));
end;

procedure TSQLRestStorageInMemory.DoNothingEvent(aDest: pointer; aRec: TSQLRecord; aIndex: integer);
begin
end;

function TSQLRestStorageInMemory.AdaptSQLForEngineList(var SQL: RawUTF8): boolean;
var P: PUTF8Char;
    Prop: RawUTF8;
    WithoutRowID: boolean;
begin
  result := inherited AdaptSQLForEngineList(SQL);
  if result then
    exit; // 'select * from table'
................................................................................
  if PWord(P)^=ord(')')+ord(':')shl 8 then
    inc(P,2); // ignore :(...): parameter
  P := GotoNextNotSpace(P);
  if (P^ in [#0,';']) or IdemPChar(P,'LIMIT 1;') then
    result := true; // properly ended the WHERE clause as 'FIELDNAME=value'
end;

function TSQLRestStorageInMemory.FindWhereEqual(WhereField: integer;
  const WhereValue: RawUTF8; OnFind: TFindWhereEqualEvent; Dest: pointer;
  FoundLimit,FoundOffset: integer): PtrInt;
var i, ndx: integer;
    aValue: PtrInt;
    err, currentRow: integer;
    P: TSQLPropInfo;
    Hash: TListFieldHash;
................................................................................
        inc(result);
        if result>=FoundLimit then
          exit;
      end;
  end;
end;

function TSQLRestStorageInMemory.GetJSONValues(Stream: TStream;
  Expand, withID: boolean; const Fields: TSQLFieldBits;
  WhereField: integer; const WhereValue: RawUTF8;
  FoundLimit,FoundOffset: integer): PtrInt;
var i,KnownRowsCount: integer;
    W: TJSONSerializer;
begin // exact same format as TSQLTable.GetJSONValues()
  result := 0;
................................................................................
    end;
    W.EndJSONObject(KnownRowsCount,result);
  finally
    W.Free;
  end;
end;

function TSQLRestStorageInMemory.IDToIndex(ID: integer): integer;
var L, R, cmp: integer;
begin
  if self<>nil then
  with fValue do begin
    R := Count-1;
    if fIDSorted and (R>=8) then begin
      // IDs are sorted -> use fast binary search algorithm
................................................................................
      for result := 0 to R do
        if TSQLRecord(List[result]).fID=ID then
          exit;
  end;
  result := -1;
end;

function TSQLRestStorageInMemory.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8;
// - GetJSONValues/FindWhereEqual will handle basic REST commands (not all SQL)
// only valid SQL command is "SELECT Field1,Field2 FROM Table WHERE ID=120;",
// i.e one Table SELECT with one optional "WHERE fieldname = value" statement
// - handle also basic "SELECT Count(*) FROM TableName;" SQL statement
// Note: this is sufficient for OneFieldValue() and MultiFieldValue() to work
var MS: TRawByteStringStream;
................................................................................
end;
begin
  ResCount := 0;
  if self=nil then begin
    result := '';
    exit;
  end;
  StorageLock(false);
  try
    if IdemPropNameU(fBasicSQLCount,SQL) then
      SetCount(TableRowCount(fStoredClass)) else
    if IdemPropNameU(fBasicSQLHasRows[false],SQL) or
       IdemPropNameU(fBasicSQLHasRows[true],SQL) then
      if TableRowCount(fStoredClass)=0 then begin
        result := '{"fieldCount":1,"values":["RowID"]}'#$A;
................................................................................
            // was "SELECT Count(*) FROM TableName WHERE ..."
            SetCount(FindWhereEqual(WhereField,WhereValue,DoNothingEvent,nil,0,0)) else
            // invalid "SELECT FROM Table" ?
            exit else begin
          // save rows as JSON, with appropriate search according to Where* arguments
          MS := TRawByteStringStream.Create;
          try
            ResCount := GetJSONValues(MS,ForceAJAX or (Owner=nil) or not Owner.NoAJAXJSON,
              withID,Fields,WhereField,WhereValue,FoundLimit,FoundOffset);
            result := MS.DataString;
          finally
            MS.Free;
          end;
        end;
      finally
        Free;
      end;
    end;
  finally
    StorageUnLock;
  end;
  if ReturnedRowCount<>nil then
    ReturnedRowCount^ := ResCount;
end;

procedure TSQLRestStorageInMemory.LoadFromJSON(const aJSON: RawUTF8);
begin
  LoadFromJSON(Pointer(aJSON),length(aJSON));
end;

procedure TSQLRestStorageInMemory.LoadFromJSON(JSONBuffer: PUTF8Char; JSONBufferLen: integer);
  function IsSorted(U: PPUTF8Char; RowCount, FieldCount: integer): boolean;
  var i, aID, lastID: integer;
  begin
    result := false;
    lastID := 0;
    for i := 1 to RowCount do begin
      aID := GetInteger(U^);
................................................................................
    // create TSQLRecord instances with data from T
    T.ToObjectList(fValue,fStoredClass);
  finally
    T.Free;
  end;
end;

procedure TSQLRestStorageInMemory.SaveToJSON(Stream: TStream; Expand: Boolean);
begin
  if self<>nil then
    GetJSONValues(Stream,Expand,true,ALL_FIELDS,-1,'',0,0);
end;

function TSQLRestStorageInMemory.SaveToJSON(Expand: Boolean): RawUTF8;
var MS: TRawByteStringStream;
begin
  if self=nil then
    result := '' else begin
    MS := TRawByteStringStream.Create;
    try
      SaveToJSON(MS,Expand);
................................................................................
    finally
      MS.Free;
    end;
  end;
end;

const
  TSQLRestStorageINMEMORY_MAGIC = $A5ABA5A5;

function TSQLRestStorageInMemory.LoadFromBinary(Stream: TStream): boolean;
var R: TFileBufferReader;
    MS: TMemoryStream;
    n, i, f: integer;
    FieldNames: TRawUTF8DynArray;
    IDs: TIntegerDynArray;
    P: PAnsiChar;
    aRecord: TSQLRecord;
    FieldTypes: array[0..MAX_SQLFIELDS-1] of TSQLFieldType;
begin
  result := false;
  if self=nil then
    exit;
  MS := StreamUnSynLZ(Stream,TSQLRestStorageINMEMORY_MAGIC);
  if MS<>nil then
  with fStoredClassRecordProps do
  try
    // check header: expect same exact RTTI
    R.OpenFrom(MS.Memory,MS.Size);
    if (R.ReadRawUTF8<>RawUTF8(ClassName)) or
       (R.ReadRawUTF8<>SQLTableName) or
................................................................................
    Result := true;
  finally
    R.Close;
    MS.Free;
  end;
end;

function TSQLRestStorageInMemory.SaveToBinary(Stream: TStream): integer;
var W: TFileBufferWriter;
    MS: THeapMemoryStream;
    IDs: TIntegerDynArray;
    FieldNames: TRawUTF8DynArray;
    i, f: integer;
begin
  result := 0;
................................................................................
    W.WriteVarUInt32Array(IDs,Count,wkSorted); // efficient ID storage
    // write content, grouped by field (for better compression)
    for f := 0 to Fields.Count-1 do
      with Fields.List[f], fValue do
        for i := 0 to Count-1 do
          GetBinary(TSQLRecord(List[i]),W);
    W.Flush;
    result := StreamSynLZ(MS,Stream,TSQLRestStorageINMEMORY_MAGIC);
  finally
    W.Free;
    MS.Free;
  end;
end;

function TSQLRestStorageInMemory.EngineRetrieve(TableModelIndex, ID: integer): RawUTF8;
var i: integer;
begin // TableModelIndex is not usefull here
  StorageLock(false);
  try
    i := IDToIndex(ID);
    if i<0 then
      result := '' else
      result := TSQLRecord(fValue.List[i]).GetJSONValues(true,true,soSelect);
  finally
    StorageUnLock;
  end;
end;

function TSQLRestStorageInMemory.GetOne(aID: integer): TSQLRecord;
var i: integer;
begin
  i := IDToIndex(aID);
  if i<0 then
    result := nil else begin
    result := fStoredClass.Create;
    CopyObject(fValue.List[i],Result);
    result.fID := aID;
  end;
end;

function TSQLRestStorageInMemory.EngineUpdate(TableModelIndex, ID: integer;
  const SentData: RawUTF8): boolean;
var i: integer;
    Orig,Rec: TSQLRecord;
begin
  // this implementation will handle partial fields update (e.g.
  // FillPrepare+BatchUpdate or TSQLRestServerRemoteDB.UpdateField)
  // but TSQLRestStorageRecordBased.EngineUpdate won't
  result := false;
  if (ID<0) or (TableModelIndex<0) or (Model.Tables[TableModelIndex]<>fStoredClass) then
    exit;
  if SentData='' then begin
    result := True;
    exit;
  end;
  StorageLock(true);
  try
    i := IDToIndex(ID);
    if (i<0) or not RecordCanBeUpdated(fStoredClass,ID,seUpdate) then
      exit;
    if fUniqueFields<>nil then begin
      Orig := TSQLRecord(fValue.List[i]);
      Rec := Orig.CreateCopy; // copy since can be a partial update
      Rec.FillFrom(SentData); // overwrite updated properties
      if not UniqueFieldsUpdateOK(Rec,i) then begin
        Rec.Free; // stored false property duplicated value -> error
................................................................................
      TSQLRecord(fValue.List[i]) := Rec; // update item in list
    end else
      // direct in-place partial update
      TSQLRecord(fValue.List[i]).FillFrom(SentData);
    fModified := true;
    result := true;
    if Owner<>nil then
      Owner.InternalUpdateEvent(seUpdate,fStoredClass,ID,nil);
  finally
    StorageUnLock;
  end;
end;

function TSQLRestStorageInMemory.UpdateOne(Rec: TSQLRecord): boolean;
var i: integer;
begin
  result := false;
  if (Rec=nil) or (PSQLRecordClass(Rec)^<>fStoredClass) or (Rec.fID<=0) then
    exit;
  StorageLock(true);
  try
    i := IDToIndex(Rec.fID);
    if (i<0) or not RecordCanBeUpdated(fStoredClass,Rec.fID,seUpdate) then
      exit;
    if (fUniqueFields<>nil) and not UniqueFieldsUpdateOK(Rec,i) then
      exit; // stored false property duplicated value -> error
    CopyObject(Rec,fValue.List[i]);
    fModified := true;
    result := true;
    if Owner<>nil then
      Owner.InternalUpdateEvent(seUpdate,fStoredClass,Rec.fID,nil);
  finally
    StorageUnLock;
  end;
end;

function TSQLRestStorageInMemory.UpdateOne(ID: integer;
  const Values: TSQLVarDynArray): boolean;
var i: integer;
    Orig,Rec: TSQLRecord;
begin
  result := false;
  if ID<=0 then
    exit;
................................................................................
    exit;
  fModified := true;
  result := true;
  if Owner<>nil then
    Owner.InternalUpdateEvent(seUpdate,fStoredClass,ID,nil);
end;

function TSQLRestStorageInMemory.EngineRetrieveBlob(TableModelIndex, aID: integer;
  BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
var i: integer;
begin
  result := false;
  if (TableModelIndex<0) or (not BlobField^.IsBlob) or
     (fModel.Tables[TableModelIndex]<>fStoredClass) then
    exit;
  StorageLock(false);
  try
    i := IDToIndex(aID);
    if i<0 then
      exit;
    // get result blob directly from RTTI property description
    GetLongStrProp(fValue.List[i],BlobField,RawByteString(BlobData));
    result := true;
  finally
    StorageUnLock;
  end;
end;

function TSQLRestStorageInMemory.RetrieveBlobFields(Value: TSQLRecord): boolean;
var i,f: integer;
begin
  result := false;
  if (Value<>nil) and (Value.fID>0) and (PSQLRecordClass(Value)^=fStoredClass) then
  with Value.RecordProps do
  if BlobFields<>nil then begin
    StorageLock(false);
    try
      i := IDToIndex(Value.fID);
      if i<0 then
        exit;
      for f := 0 to high(BlobFields) do
        BlobFields[f].CopyValue(fValue.List[i],Value);
      result := true;
    finally
      StorageUnLock;
    end;
  end;
end;

function TSQLRestStorageInMemory.EngineUpdateBlob(TableModelIndex, aID: integer;
  BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean;
var i: integer;
    AffectedField: TSQLFieldBits;
begin
  result := false;
  if (aID<0) or (TableModelIndex<0) or (not BlobField^.IsBlob) or
     (fModel.Tables[TableModelIndex]<>fStoredClass) then
    exit;
  StorageLock(true);
  try
    i := IDToIndex(aID);
    if (i<0) or not RecordCanBeUpdated(fStoredClass,aID,seUpdate) then
      exit;
    // set blob value directly from RTTI property description
    SetLongStrProp(fValue.List[i],BlobField,BlobData);
    if Owner<>nil then begin
      fStoredClassRecordProps.FieldIndexsFromBlobField(BlobField,AffectedField);
      Owner.InternalUpdateEvent(seUpdateBlob,fStoredClass,aID,@AffectedField);
    end;
    result := true;
  finally
    StorageUnLock;
  end;
end;

function TSQLRestStorageInMemory.UpdateBlobFields(Value: TSQLRecord): boolean;
var i,f: integer;
begin
  result := false;
  if (Value<>nil) and (Value.fID>0) and (PSQLRecordClass(Value)^=fStoredClass) then
  with Value.RecordProps do
  if BlobFields<>nil then begin
    StorageLock(true);
    try
      i := IDToIndex(Value.fID);
      if (i<0) or not RecordCanBeUpdated(Table,Value.fID,seUpdate) then
        exit;
      for f := 0 to high(BlobFields) do
        BlobFields[f].CopyValue(Value,fValue.List[i]);
      if Owner<>nil then
        Owner.InternalUpdateEvent(seUpdateBlob,fStoredClass,Value.fID,
          @fStoredClassRecordProps.BlobFieldsBits);
      result := true;
    finally
      StorageUnLock;
    end;
  end else
    result := true; // as TSQLRest.UpdateblobFields()
end;

function TSQLRestStorageInMemory.TableRowCount(Table: TSQLRecordClass): integer;
begin
  if Table<>fStoredClass then
    result := 0 else
    result := fValue.Count;
end;

function TSQLRestStorageInMemory.TableHasRows(Table: TSQLRecordClass): boolean;
begin
  if Table<>fStoredClass then
    result := false else
    result := fValue.Count>0;
end;

function TSQLRestStorageInMemory.EngineUpdateField(TableModelIndex: integer;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
var SetField: TSQLPropInfo;
    WhereValueString, SetValueString: RawUTF8;
    Where: TList;
    i, ndx, WhereFieldIndex: integer;
    Rec: TSQLRecord;
begin
  result := false;
  if (TableModelIndex<0) or (fModel.Tables[TableModelIndex]<>fStoredClass) or
     (SetFieldName='') or (SetValue='') or
     (WhereFieldName='') or (WhereValue='') then
    exit;
  // handle destination field RTTI
  SetField := fStoredClassRecordProps.Fields.ByRawUTF8Name(SetFieldName);
  if SetField=nil then
    exit; // don't allow setting ID field, which is Read Only
  if SetValue[1]='"' then
................................................................................
    inc(WhereFieldIndex); // FindWhereEqual() expects index = RTTI+1
  end;
  if WhereValue[1]='"' then
    UnQuoteSQLString(pointer(WhereValue),WhereValueString) else
    WhereValueString := WhereValue;
  // search indexes, then apply updates
  Where := TList.Create;
  StorageLock(true);
  try 
    // find matching Where[]
    if FindWhereEqual(WhereFieldIndex,WhereValueString,AddIntegerDynArrayEvent,Where,0,0)=0 then
      exit; // Where.Count=0 -> nothing to update
    // check that all records can be updated
    for i := 0 to Where.Count-1 do
      if not RecordCanBeUpdated(fStoredClass,
         TSQLRecord(fValue.List[PtrInt(Where.List[i])]).fID,seUpdate) then
        exit; // one record update fails -> abort all
    if fUniqueFields<>nil then
      for i := 0 to fUniqueFields.Count-1 do
      with TListFieldHash(fUniqueFields.List[i]) do
        if Field=SetField then
          if Where.Count>1 then // unique field can't allow multiple sets
            exit else begin
................................................................................
          end;
    // update field value
    for i := 0 to Where.Count-1 do begin
      Rec := fValue.List[PtrInt(Where.List[i])];
      SetField.SetValue(Rec,pointer(SetValueString),false);
      fModified := true;
      if Owner<>nil then
        Owner.InternalUpdateEvent(seUpdate,fStoredClass,Rec.fID,nil);
      result := true;
    end;
  finally
    StorageUnLock;
    Where.Free;
  end;
end;

procedure TSQLRestStorageInMemory.UpdateFile;
var F: TFileStream;
begin
  if (self=nil) or not Modified or (FileName='') then
    exit;
  if fValue.Count=0 then
    DeleteFile(FileName) else begin
    F := TFileStream.Create(FileName,fmCreate);
................................................................................
    finally
      F.Free;
    end;
  end;
  fModified := false;
end;

function TSQLRestStorageInMemory.SearchField(const FieldName, FieldValue: RawUTF8; var ResultID: TIntegerDynArray): boolean;
var n, WhereField: integer;
    {$ifdef CPU64}i: integer;{$endif}
    Where: TList;
begin
  result := false;
  SetLength(ResultID,0);
  if (self=nil) or (fValue.Count=0) then
................................................................................
    {$endif}
  finally
    Where.Free;
  end;
end;


{ TSQLRestStorageInMemoryExternal }

procedure TSQLRestStorageInMemoryExternal.StorageLock(WillModifyContent: boolean);
begin
  inherited StorageLock(WillModifyContent);
  if WillModifyContent and (Owner<>nil) then
    Owner.FlushInternalDBCache;
end;


{ TListFieldHash }

function TListFieldHash.Compare(Item1,Item2: TObject): boolean;
................................................................................
  with fValues do
    if cardinal(Index)<cardinal(Count) then
      result := List[Index] else
      result := nil;
end;


{ TSQLRestStorage }

constructor TSQLRestStorage.Create(aClass: TSQLRecordClass;
  aServer: TSQLRestServer; const aFileName: TFileName; aBinaryFile: boolean);
begin
  inherited Create(nil);
  if aClass=nil then
    raise EBusinessLayerException.CreateFmt('%s.Create expect a class',[ClassName]);
  InitializeCriticalSection(fStorageCriticalSection);
  fStoredClass := aClass;
  fStoredClassRecordProps := aClass.RecordProps;
  if aServer<>nil then begin
    fOwner := aServer;
    fModel := aServer.Model;
    fStoredClassProps := fModel.Props[aClass];

  end else
    // if no server is defined, simply use the first model using this class
    if fStoredClassRecordProps.fModel<>nil then
    with fStoredClassRecordProps.fModel[0] do begin
      fModel := Model;
      fStoredClassProps := Properties;
    end;
................................................................................
  fFileName := aFileName;
  fBasicSQLCount := 'SELECT COUNT(*) FROM '+fStoredClassRecordProps.SQLTableName;
  fBasicSQLHasRows[false] := 'SELECT RowID FROM '+fStoredClassRecordProps.SQLTableName+' LIMIT 1';
  fBasicSQLHasRows[true] := fBasicSQLHasRows[false];
  system.delete(fBasicSQLHasRows[true],8,3);
end;

destructor TSQLRestStorage.Destroy;
begin
  inherited;
  assert(fStorageCriticalSectionCount=0);
  DeleteCriticalSection(fStorageCriticalSection);
end;

procedure TSQLRestStorage.BeginCurrentThread(Sender: TThread);
begin
  // nothing to do in this basic REST static class
end;

procedure TSQLRestStorage.EndCurrentThread(Sender: TThread);
begin
  // nothing to do in this basic REST static class
end;

function TSQLRestStorage.EngineUpdateField(TableModelIndex: integer;





  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
begin
  result := false; // not implemented in this basic REST static class
end;

function TSQLRestStorage.CreateSQLMultiIndex(Table: TSQLRecordClass;
  const FieldNames: array of RawUTF8; Unique: boolean; IndexName: RawUTF8): boolean;
begin
  result := false; // not implemented in this basic REST static class
end;

function TSQLRestStorage.SearchField(const FieldName: RawUTF8;
  FieldValue: Integer; var ResultID: TIntegerDynArray): boolean;
begin
  result := SearchField(FieldName,Int32ToUTF8(FieldValue),ResultID);
end;

function TSQLRestStorage.RecordCanBeUpdated(Table: TSQLRecordClass;
  ID: integer; Action: TSQLEvent; ErrorMsg: PRawUTF8 = nil): boolean;
begin

  result := ((Owner=nil) or Owner.RecordCanBeUpdated(Table,ID,Action,ErrorMsg));
end;

function TSQLRestStorage.RefreshedAndModified: boolean;
begin
  result := false; // no refresh necessary with "normal" static tables
end;

procedure TSQLRestStorage.StorageLock(WillModifyContent: boolean);
begin

  EnterCriticalSection(fStorageCriticalSection);


  inc(fStorageCriticalSectionCount);
end;

procedure TSQLRestStorage.StorageUnLock;
begin
  dec(fStorageCriticalSectionCount);
  assert(fStorageCriticalSectionCount>=0);
  LeaveCriticalSection(fStorageCriticalSection);
end;


function TSQLRestStorage.UnLock(Table: TSQLRecordClass; aID: integer): boolean;
begin


  result := Model.UnLock(Table,aID);
end;

function TSQLRestStorage.InternalBatchStart(Method: TSQLURIMethod): boolean;
begin
  result := false;
end;

procedure TSQLRestStorage.InternalBatchStop;
begin
  // do nothing method
end;

function TSQLRestStorage.AdaptSQLForEngineList(var SQL: RawUTF8): boolean; 
begin
  if fStoredClassProps=nil then
    result := false else begin
    result := IdemPropNameU(fStoredClassProps.SQL.SelectAllWithRowID,SQL);
    if result then
      SQL := fStoredClassProps.SQL.SelectAllWithID else
      result := IdemPropNameU(fStoredClassProps.SQL.SelectAllWithID,SQL);
................................................................................
  if integer(fStaticDataCount)<>length(fModel.Tables) then begin
    for t := fStaticDataCount to high(fModel.Tables) do
      StaticDataCreate(fModel.Tables[t]);
    fStaticDataCount := length(fModel.Tables);
  end;
  // initialize new tables
  for t := 0 to fStaticDataCount-1 do
    with TSQLRestStorageInMemory(fStaticData[t]) do
    if Count=0 then // emulates TSQLRestServerDB.CreateMissingTables
      StoredClass.InitializeTable(Self,'');
end;

destructor TSQLRestServerFullMemory.Destroy;
begin
  UpdateToFile;
................................................................................
    t: integer;
    S: TFileStream;
    wasString: boolean;
begin
  if (self=nil) or (fFileName='') or not FileExists(fFileName) then
    exit;
  for t := 0 to fStaticDataCount-1 do
    TSQLRestStorageInMemory(fStaticData[t]).fValue.Clear;
  if fBinaryFile then begin
    S := TFileStream.Create(FileName,fmOpenRead or fmShareDenyNone);
    try
      if ReadStringFromStream(S)=RawUTF8(ClassName)+'00' then
      repeat
        t := Model.GetTableIndex(ReadStringFromStream(S));
      until (t<0) or
        not TSQLRestStorageInMemory(fStaticData[t]).LoadFromBinary(S);
    finally
      S.Free;
    end;
  end else begin // [{"AuthUser":[{....},{...}]},{"AuthGroup":[{...},{...}]}]
    JSON := StringFromFile(fFileName);
    if JSON='' then
      exit;
................................................................................
      t := Model.GetTableIndex(TableName);
      if t<0 then
        exit;
      Data := P;
      P := GotoNextJSONObjectOrArray(P);
      if P=nil then
        break else
        TSQLRestStorageInMemory(fStaticData[t]).LoadFromJSON(Data,P-Data);
    until false;
  end;
end;

procedure TSQLRestServerFullMemory.UpdateToFile;
const CHARS: array[0..6] of AnsiChar = '[{":,}]';
var S: TFileStream;                //   0123456
................................................................................
    t: integer;
    Modified: boolean;
begin
  if (self=nil) or (FileName='') then
    exit;
  Modified := false;
  for t := 0 to fStaticDataCount-1 do
    if TSQLRestStorageInMemory(fStaticData[t]).Modified then begin
      Modified := true;
      break;
    end;
  if not Modified then
    exit;
  S := TFileStream.Create(FileName,fmCreate);
  try
    if fBinaryFile then begin
      WriteStringToStream(S,RawUTF8(ClassName)+'00');
      for t := 0 to fStaticDataCount-1 do
      with TSQLRestStorageInMemory(fStaticData[t]) do begin
        WriteStringToStream(S,fStoredClassRecordProps.SQLTableName);
        SaveToBinary(S);
      end;
    end else begin
      S.Write(CHARS[0],1);
      for t := 0 to fStaticDataCount-1 do
      with TSQLRestStorageInMemory(fStaticData[t]) do begin
        S.Write(CHARS[1],2);
        with fStoredClassRecordProps do
          S.Write(pointer(SQLTableName)^,length(SQLTableName));
        S.Write(CHARS[2],2);
        SaveToJSON(S,true);
        S.Write(CHARS[5],1);
        if t<integer(fStaticDataCount-1) then
................................................................................
  end;
end;

function TSQLRestServerFullMemory.EngineRetrieve(TableModelIndex, ID: integer): RawUTF8;
begin
  if cardinal(TableModelIndex)>=fStaticDataCount then
    result := '' else
    result := TSQLRestStorageInMemory(fStaticData[TableModelIndex]).EngineRetrieve(0,ID);
end;

function TSQLRestServerFullMemory.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8;
var TableIndex: integer;
begin
  TableIndex := Model.GetTableIndexFromSQLSelect(SQL,true);
  if TableIndex<0 then
    result := '' else
    result := TSQLRestStorageInMemory(fStaticData[TableIndex]).
      EngineList(SQL,ForceAJAX,ReturnedRowCount);
end;

function TSQLRestServerFullMemory.GetStatic(Table: TSQLRecordClass): TSQLRestStorageInMemory;
var t: cardinal;
begin
  t := fModel.GetTableIndexExisting(Table);
  if t<fStaticDataCount then
    result := TSQLRestStorageInMemory(fStaticData[t]) else
    result := nil;
end;

function TSQLRestServerFullMemory.EngineUpdate(TableModelIndex, ID: integer;
  const SentData: RawUTF8): boolean;
begin
  if cardinal(TableModelIndex)>=fStaticDataCount then
    result := false else
    result := TSQLRestStorageInMemory(fStaticData[TableModelIndex]).
      EngineUpdate(TableModelIndex,ID,SentData);
end;

function TSQLRestServerFullMemory.EngineDelete(TableModelIndex, ID: integer): boolean;
begin
  if cardinal(TableModelIndex)>=fStaticDataCount then
    result := false else
    result := TSQLRestStorageInMemory(fStaticData[TableModelIndex]).
      EngineDelete(TableModelIndex,ID);
end;

function TSQLRestServerFullMemory.EngineDeleteWhere(TableModelIndex: Integer;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
begin
  if cardinal(TableModelIndex)>=fStaticDataCount then
    result := false else
    result := TSQLRestStorageInMemory(fStaticData[TableModelIndex]).
      EngineDeleteWhere(TableModelIndex,SQLWhere,IDs);
end;

function TSQLRestServerFullMemory.EngineRetrieveBlob(TableModelIndex, aID: integer;
  BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
begin

  if cardinal(TableModelIndex)>=fStaticDataCount then
    result := false else
    result := TSQLRestStorageInMemory(fStaticData[TableModelIndex]).
      EngineRetrieveBlob(TableModelIndex,aID,BlobField,BlobData);
end;

function TSQLRestServerFullMemory.EngineUpdateBlob(TableModelIndex, aID: integer;
  BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean;
begin
  if cardinal(TableModelIndex)>=fStaticDataCount then
    result := false else
    result := TSQLRestStorageInMemory(fStaticData[TableModelIndex]).
      EngineUpdateBlob(TableModelIndex,aID,BlobField,BlobData);
end;

function TSQLRestServerFullMemory.EngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer;
begin

  if cardinal(TableModelIndex)>=fStaticDataCount then
    result := 0 else
    result := TSQLRestStorageInMemory(fStaticData[TableModelIndex]).
      EngineAdd(TableModelIndex,SentData);
end;

function TSQLRestServerFullMemory.EngineUpdateField(TableModelIndex: integer;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
begin

  if cardinal(TableModelIndex)>=fStaticDataCount then
    result := false else
    result := TSQLRestStorageInMemory(fStaticData[TableModelIndex]).
      EngineUpdateField(TableModelIndex,SetFieldName,SetValue,WhereFieldName,WhereValue);
end;

function TSQLRestServerFullMemory.EngineExecuteAll(const aSQL: RawUTF8): boolean;
begin
  result := false; // not implemented in this basic REST server class
end;

................................................................................
begin
  if aRemoteClient=nil then
    raise EORMException.CreateFmt('%s creation with no remote client',[ClassName]);
  inherited Create(aRemoteClient.Model,aHandleUserAuthentication);
  fClient := aRemoteClient;
end;

function TSQLRestServerRemoteDB.EngineAdd(TableModelIndex: integer;
  const SentData: RawUTF8): integer;
begin
  result := fClient.EngineAdd(TableModelIndex,SentData);
end;

function TSQLRestServerRemoteDB.EngineDelete(TableModelIndex, ID: integer): boolean;

begin
  result := fClient.EngineDelete(TableModelIndex,ID);
end;

function TSQLRestServerRemoteDB.EngineDeleteWhere(TableModelIndex: Integer;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
begin
  result := fClient.EngineDeleteWhere(TableModelIndex,SQLWhere,IDs);
end;

function TSQLRestServerRemoteDB.EngineExecuteAll(const aSQL: RawUTF8): boolean;
begin
  result := fClient.EngineExecute(aSQL);
end;

................................................................................
function TSQLRestServerRemoteDB.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean; ReturnedRowCount: PPtrInt): RawUTF8;
begin
  result := fClient.EngineList(SQL,ForceAJAX,ReturnedRowCount);
end;

function TSQLRestServerRemoteDB.EngineRetrieve(TableModelIndex, ID: integer): RawUTF8;

begin
  result := fClient.EngineRetrieve(TableModelIndex,ID);

end;

function TSQLRestServerRemoteDB.EngineRetrieveBlob(TableModelIndex, aID: integer;
  BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
begin
  if (self=nil) or (BlobField=nil) then
    result := false else
    result := fClient.EngineRetrieveBlob(TableModelIndex,aID,BlobField,BlobData);
end;

function TSQLRestServerRemoteDB.EngineUpdate(TableModelIndex,
  ID: integer; const SentData: RawUTF8): boolean;
begin
  result := fClient.EngineUpdate(TableModelIndex,ID,SentData);
end;

function TSQLRestServerRemoteDB.EngineUpdateBlob(TableModelIndex, aID: integer;

  BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean;
begin
  if (self=nil) or (BlobField=nil) then
    result := false else
    result := fClient.EngineUpdateBlob(TableModelIndex,aID,BlobField,BlobData);
end;

function TSQLRestServerRemoteDB.EngineUpdateField(TableModelIndex: integer;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
begin


  result := fClient.EngineUpdateField(TableModelIndex,SetFieldName,SetValue,WhereFieldName,WhereValue);
end;

function TSQLRestServerRemoteDB.AfterDeleteForceCoherency(Table: TSQLRecordClass;
  aID: integer): boolean;
begin
  result := true; // coherency will be performed on the server side
end;
................................................................................
    if aValue then
      SetLength(fForceBlobTransfert,fModel.fTablesMax+1) else
      exit; // nothing to set
  fForceBlobTransfert[i] := aValue;
end;

function TSQLRestClient.Add(Value: TSQLRecord; SendData: boolean; ForceID: boolean=false): integer;


begin



  result := inherited Add(Value,SendData,ForceID);
  if (result>0) and (fForceBlobTransfert<>nil) and
     fForceBlobTransfert[fModel.GetTableIndexExisting(PSQLRecordClass(Value)^)] then















     UpdateBlobFields(Value);
end;



function TSQLRestClient.EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8;
var dummy: cardinal;
begin


  if not ClientRetrieve(TableModelIndex,ID,false,dummy,result) then
    result := '';


end;


function TSQLRestClient.Retrieve(aID: integer; Value: TSQLRecord;
  ForUpdate: boolean=false): boolean;
var Resp: RawUTF8;
    TableIndex: integer;
begin
  result := false;
  if (self=nil) or (aID<=0) or (Value=nil) then
    exit;
  TableIndex := Model.GetTableIndexExisting(PSQLRecordClass(Value)^);
................................................................................
      Value.FillFrom(Resp);
      Value.fID := aID; // JSON object may not contain the ID
      result := true;
      exit; // fast retrieved from internal Client cache (BLOBs ignored)
    end;
  end;
  try
    if ClientRetrieve(TableIndex,aID,ForUpdate,Value.fInternalState,Resp) then begin
      Value.FillFrom(Resp);
      Value.fID := aID; // JSON object may not contain the ID
      if (fForceBlobTransfert<>nil) and fForceBlobTransfert[TableIndex] then
        result := RetrieveBlobFields(Value) else
        result := true;
      ForUpdate := false; // any exception shall unlock the record
    end;
................................................................................
  finally
    if ForUpdate then
      Model.UnLock(TableIndex,aID);
  end;
end;

function TSQLRestClient.Update(Value: TSQLRecord): boolean;

begin

  result := BeforeUpdateEvent(Value) and inherited Update(Value);






  if result then begin
    if (fForceBlobTransfert<>nil) and
       fForceBlobTransfert[Model.GetTableIndexExisting(PSQLRecordClass(Value)^)] then
      result := UpdateBlobFields(Value);

    if result and assigned(OnRecordUpdate) then
      OnRecordUpdate(Value);
  end;
end;


























function TSQLRestClient.BeforeUpdateEvent(Value: TSQLRecord): Boolean;
begin
  Result := true; // by default, just allow the update to proceed
end;

function TSQLRestClient.Refresh(aID: integer; Value: TSQLRecord;
  var Refreshed: boolean): boolean;
var Resp, Original: RawUTF8;
begin
  result := false;
  if (aID>0) and (self<>nil) and (Value<>nil) then
    if ClientRetrieve(Model.GetTableIndexExisting(PSQLRecordClass(Value)^),aID,False,
       Value.fInternalState,Resp) then begin
      Original := Value.GetJSONValues(IsNotAjaxJSON(pointer(Resp)),true,soSelect);
      Resp := trim(Resp);
      if (Resp<>'') and (Resp[1]='[') then // '[{....}]' -> '{...}'
        Resp := copy(Resp,2,length(Resp)-2);
      if Original<>Resp then begin // did the content really change?
        Refreshed := true;
................................................................................


{ TSQLVirtualTable }

constructor TSQLVirtualTable.Create(aModule: TSQLVirtualTableModule;
  const aTableName: RawUTF8; FieldCount: integer; Fields: PPUTF8CharArray);
var aTable: TSQLRecordClass;

begin
  if (aModule=nil) or (aTableName='') then
    raise EModelException.CreateFmt('Invalid parameters to %s.Create',[ClassName]);
  fModule := aModule;
  fTableName := aTableName;
  if fModule.fFeatures.StaticClass<>nil then
    // create new fStatic instance e.g. for TSQLVirtualTableLog
    if fModule.Server=nil then
      raise EModelException.CreateFmt('Missing aModule.Server for %s.Create',[ClassName]) else
    with fModule.Server do begin
      fStaticTableIndex := Model.GetTableIndex(aTableName);
      if fStaticTableIndex>=0 then begin
        aTable := Model.Tables[fStaticTableIndex];
        fStatic := fModule.fFeatures.StaticClass.Create(aTable,fModule.Server,
          fModule.FileName(aTableName),self.InheritsFrom(TSQLVirtualTableBinary));
        if length(fStaticVirtualTable)<>length(Model.Tables) then
          SetLength(fStaticVirtualTable,length(Model.Tables));
        fStaticVirtualTable[fStaticTableIndex] := fStatic;
      end;
    end;
end;

destructor TSQLVirtualTable.Destroy;
var aTableIndex: cardinal;
begin
................................................................................
  if result and (Static.Owner<>nil) then
    Static.Owner.fCache.NotifyDeletion(Static.StoredClass,aRowID);
end;

function TSQLVirtualTableJSON.Drop: boolean;
begin
  if (self<>nil) and (Static<>nil) then
  with Static as TSQLRestStorageInMemory do begin
    RollBack(0); // close any pending transaction
    fValue.Clear;
    Modified := true; // force update file after clear
    UpdateFile;
    result := true;
  end else
    result := false;
................................................................................
end;

class procedure TSQLVirtualTableJSON.GetTableModuleProperties(
  var aProperties: TVirtualTableModuleProperties);
begin
  aProperties.Features := [vtWrite,vtWhereIDPrepared];
  aProperties.CursorClass := TSQLVirtualTableCursorJSON;
  aProperties.StaticClass := TSQLRestStorageInMemoryExternal; // will flush Cache
  if InheritsFrom(TSQLVirtualTableBinary) then
    aProperties.FileExtension := 'data';
  // default will follow the class name, e.g. '.json' for TSQLVirtualTableJSON
end;

function TSQLVirtualTableJSON.Insert(aRowID: Int64;
  var Values: TSQLVarDynArray; out insertedRowID: Int64): boolean;
................................................................................
  if (self=nil) or (Static=nil) then
    exit;
  aRecord := Static.StoredClass.Create;
  try
    if aRecord.SetFieldSQLVars(Values) then begin
      if aRowID>0 then
        aRecord.fID := aRowID;
      insertedRowID := (Static as TSQLRestStorageInMemory).AddOne(aRecord,aRowID>0);
      if insertedRowID>0 then begin
        if Static.Owner<>nil then
          Static.Owner.fCache.Notify(aRecord,soInsert);
        result := true;
      end;
    end;
  finally
................................................................................
  end;
end;

function TSQLVirtualTableJSON.Prepare(var Prepared: TSQLVirtualTablePrepared): boolean;
begin
  result := inherited Prepare(Prepared); // optimize ID=? WHERE clause
  if result and (Static<>nil) then
  with Static as TSQLRestStorageInMemory do begin
    if Prepared.IsWhereOneFieldEquals then
    with Prepared.Where[0] do
    if UniqueFieldHash(Column)<>nil then begin
      Value.VType := ftNull; // mark TSQLVirtualTableCursorJSON expects it
      OmitCheck := true;
      Prepared.EstimatedCost := 1; 
    end;
................................................................................
  var Values: TSQLVarDynArray): boolean;
var i: integer;
begin
  result := false;
  if (self=nil) or (Static=nil) or
     (oldRowID<>newRowID) or (newRowID<=0) then // don't allow ID change
    exit;
  with Static as TSQLRestStorageInMemory do
    if UpdateOne(newRowID,Values) then begin
      if Static.Owner<>nil then begin
        i := IDToIndex(newRowID);
        if i>=0 then
          Static.Owner.fCache.Notify(TSQLRecord(fValue.List[i]),soUpdate);
      end;
      result := true;
................................................................................
end;


{ TSQLVirtualTableCursorJSON }

function TSQLVirtualTableCursorJSON.Column(aColumn: integer;
  var aResult: TSQLVar): boolean;
var Static: TSQLRestStorageInMemory;
begin
  if (self=nil) or (fCurrent>fMax) or
     (TSQLVirtualTableJSON(Table).Static=nil) then begin
    result := false;
    exit;
  end;
  Static := TSQLRestStorageInMemory(TSQLVirtualTableJSON(Table).Static);
  if Cardinal(fCurrent)>=Cardinal(Static.fValue.Count) then
    result := False else begin
    if aColumn=VIRTUAL_TABLE_ROWID_COLUMN then begin
      aResult.VType := ftInt64;
      aResult.VInt64 := TSQLRecord(Static.fValue.List[fCurrent]).fID;
    end else
    with Static.fStoredClassRecordProps.Fields do
................................................................................
  const Prepared: TSQLVirtualTablePrepared): boolean;
var Hash: TListFieldHash;
begin
  result := inherited Search(Prepared); // mark EOF by default
  if (not result) or (not Table.InheritsFrom(TSQLVirtualTableJSON)) or
     (TSQLVirtualTableJSON(Table).Static=nil) then
    result := false else
    with TSQLRestStorageInMemory(TSQLVirtualTableJSON(Table).Static) do begin
      if Count>0 then // if something to search in
        if Prepared.IsWhereIDEquals(false) then begin // ID=?
          fMax := IDToIndex(Prepared.Where[0].Value.VInt64); // binary search
          if fMax>=0 then
            fCurrent := fMax; // ID found
        end else
        if Prepared.IsWhereOneFieldEquals then

Changes to SQLite3/mORMotBigTable.pas.

70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
..
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
...
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
type
  /// REST server with direct access to a Synopse Big Table external database
  // - handle all REST commands via direct TSynBigTableMetaData or
  // TSynBigTableRecord call (using a TSynBigTableTable instance)
  // - is used by TSQLRestServer.URI for faster RESTful direct access
  // - for JOINed SQL statements, the external database is also defined as
  // a SQLite3 virtual table, via the TSQLVirtualTableBigTable[Cursor] classes
  TSQLRestServerStaticBigTable = class(TSQLRestServerStaticRecordBased)
  protected
    /// the associated Big Table instance
    // - either a TSynBigTableMetaData or a TSynBigTableRecord
    fBig: TSynBigTableTable;
    // overriden methods calling the fBig instance
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; override;
    function EngineLockedNextID: Integer;
................................................................................
    // TSynBigTableRecord
    constructor Create(aClass: TSQLRecordClass; aServer: TSQLRestServer;
      const aFileName: TFileName = ''; aBinaryFile: boolean=false); override;
    /// release used memory
    // - i.e. the internal TSynBigTableMetaData / TSynBigTableRecord instance
    destructor Destroy; override;
    /// delete a row, calling the current BigTable instance
    // - made public since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function EngineDelete(Table: TSQLRecordClass; ID: integer): boolean; override;
    /// search for a numerical field value
    // - return true on success (i.e. if some values have been added to ResultID)
    // - store the results into the ResultID dynamic array
    function SearchField(const FieldName: ShortString; const FieldValue: Integer;
      var ResultID: TIntegerDynArray): boolean; overload; override;
................................................................................
    property BigTable: TSynBigTableTable read fBig;
  end;

  
implementation


{ TSQLRestServerStaticBigTable }

constructor TSQLRestServerStaticBigTable.Create(aClass: TSQLRecordClass;
  aServer: TSQLRestServer; const aFileName: TFileName;
  aBinaryFile: boolean);
begin
  inherited Create(aClass,aServer,aFileName,aBinaryFile);
  if fStoredClassProps.Kind in INSERT_WITH_ID then
    raise EModelException.CreateFmt('%s: %s virtual table can''t be static',
      [fStoredClassProps.SQLTableName,aClass.ClassName]);
  if aBinaryFile then
    fBig := TSynBigTableMetaData.Create(aFileName,fStoredClassProps.SQLTableName) else
    fBig := TSynBigTableRecord.Create(aFileName,fStoredClassProps.SQLTableName);
end;

function TSQLRestServerStaticBigTable.CreateSQLMultiIndex(
  Table: TSQLRecordClass; const FieldNames: array of RawUTF8;
  Unique: boolean; IndexName: RawUTF8): boolean;
begin
  result := false;
  if (self=nil) or (fBig=nil) then
    exit;

end;

destructor TSQLRestServerStaticBigTable.Destroy;
begin
  FreeAndNil(fBig);
  inherited Destroy;
end;

function TSQLRestServerStaticBigTable.EngineDelete(Table: TSQLRecordClass;
  ID: integer): boolean;
begin

end;

function TSQLRestServerStaticBigTable.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean): RawUTF8;
begin

end;

function TSQLRestServerStaticBigTable.EngineLockedNextID: Integer;
begin

end;

function TSQLRestServerStaticBigTable.EngineRetrieve(TableModelIndex,
  ID: integer): RawUTF8;
begin

end;

function TSQLRestServerStaticBigTable.EngineRetrieveBlob(
  Table: TSQLRecordClass; aID: integer; BlobField: PPropInfo;
  out BlobData: TSQLRawBlob): boolean;
begin

end;

function TSQLRestServerStaticBigTable.EngineSearchField(
  const FieldName: ShortString; const FieldValue: array of const;
  var ResultID: TIntegerDynArray): boolean;
begin

end;

function TSQLRestServerStaticBigTable.EngineUpdateBlob(
  Table: TSQLRecordClass; aID: integer; BlobField: PPropInfo;
  const BlobData: TSQLRawBlob): boolean;
begin

end;

function TSQLRestServerStaticBigTable.SearchField(
  const FieldName: ShortString; const FieldValue: Integer;
  var ResultID: TIntegerDynArray): boolean;
begin

end;

function TSQLRestServerStaticBigTable.SearchField(
  const FieldName: ShortString; FieldValue: RawUTF8;
  var ResultID: TIntegerDynArray): boolean;
begin

end;

function TSQLRestServerStaticBigTable.UpdateField(Table: TSQLRecordClass;
  Where: integer; const FieldName: shortstring; FieldValue: integer;
  ByID: boolean): boolean;
begin

end;

end.







|







 







|







 







|

|












|









|





|





|





|




|





|






|






|






|






|






|







70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
..
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
...
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
type
  /// REST server with direct access to a Synopse Big Table external database
  // - handle all REST commands via direct TSynBigTableMetaData or
  // TSynBigTableRecord call (using a TSynBigTableTable instance)
  // - is used by TSQLRestServer.URI for faster RESTful direct access
  // - for JOINed SQL statements, the external database is also defined as
  // a SQLite3 virtual table, via the TSQLVirtualTableBigTable[Cursor] classes
  TSQLRestStorageBigTable = class(TSQLRestStorageRecordBased)
  protected
    /// the associated Big Table instance
    // - either a TSynBigTableMetaData or a TSynBigTableRecord
    fBig: TSynBigTableTable;
    // overriden methods calling the fBig instance
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; override;
    function EngineLockedNextID: Integer;
................................................................................
    // TSynBigTableRecord
    constructor Create(aClass: TSQLRecordClass; aServer: TSQLRestServer;
      const aFileName: TFileName = ''; aBinaryFile: boolean=false); override;
    /// release used memory
    // - i.e. the internal TSynBigTableMetaData / TSynBigTableRecord instance
    destructor Destroy; override;
    /// delete a row, calling the current BigTable instance
    // - made public since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function EngineDelete(Table: TSQLRecordClass; ID: integer): boolean; override;
    /// search for a numerical field value
    // - return true on success (i.e. if some values have been added to ResultID)
    // - store the results into the ResultID dynamic array
    function SearchField(const FieldName: ShortString; const FieldValue: Integer;
      var ResultID: TIntegerDynArray): boolean; overload; override;
................................................................................
    property BigTable: TSynBigTableTable read fBig;
  end;

  
implementation


{ TSQLRestStorageBigTable }

constructor TSQLRestStorageBigTable.Create(aClass: TSQLRecordClass;
  aServer: TSQLRestServer; const aFileName: TFileName;
  aBinaryFile: boolean);
begin
  inherited Create(aClass,aServer,aFileName,aBinaryFile);
  if fStoredClassProps.Kind in INSERT_WITH_ID then
    raise EModelException.CreateFmt('%s: %s virtual table can''t be static',
      [fStoredClassProps.SQLTableName,aClass.ClassName]);
  if aBinaryFile then
    fBig := TSynBigTableMetaData.Create(aFileName,fStoredClassProps.SQLTableName) else
    fBig := TSynBigTableRecord.Create(aFileName,fStoredClassProps.SQLTableName);
end;

function TSQLRestStorageBigTable.CreateSQLMultiIndex(
  Table: TSQLRecordClass; const FieldNames: array of RawUTF8;
  Unique: boolean; IndexName: RawUTF8): boolean;
begin
  result := false;
  if (self=nil) or (fBig=nil) then
    exit;

end;

destructor TSQLRestStorageBigTable.Destroy;
begin
  FreeAndNil(fBig);
  inherited Destroy;
end;

function TSQLRestStorageBigTable.EngineDelete(Table: TSQLRecordClass;
  ID: integer): boolean;
begin

end;

function TSQLRestStorageBigTable.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean): RawUTF8;
begin

end;

function TSQLRestStorageBigTable.EngineLockedNextID: Integer;
begin

end;

function TSQLRestStorageBigTable.EngineRetrieve(TableModelIndex,
  ID: integer): RawUTF8;
begin

end;

function TSQLRestStorageBigTable.EngineRetrieveBlob(
  Table: TSQLRecordClass; aID: integer; BlobField: PPropInfo;
  out BlobData: TSQLRawBlob): boolean;
begin

end;

function TSQLRestStorageBigTable.EngineSearchField(
  const FieldName: ShortString; const FieldValue: array of const;
  var ResultID: TIntegerDynArray): boolean;
begin

end;

function TSQLRestStorageBigTable.EngineUpdateBlob(
  Table: TSQLRecordClass; aID: integer; BlobField: PPropInfo;
  const BlobData: TSQLRawBlob): boolean;
begin

end;

function TSQLRestStorageBigTable.SearchField(
  const FieldName: ShortString; const FieldValue: Integer;
  var ResultID: TIntegerDynArray): boolean;
begin

end;

function TSQLRestStorageBigTable.SearchField(
  const FieldName: ShortString; FieldValue: RawUTF8;
  var ResultID: TIntegerDynArray): boolean;
begin

end;

function TSQLRestStorageBigTable.UpdateField(Table: TSQLRecordClass;
  Where: integer; const FieldName: shortstring; FieldValue: integer;
  ByID: boolean): boolean;
begin

end;

end.

Changes to SQLite3/mORMotDB.pas.

73
74
75
76
77
78
79

80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
...
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
...
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
...
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
...
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282



283
284
285
286
287
288
289
...
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
...
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
...
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
...
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
...
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
...
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
...
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
....
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
....
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092

1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111

1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143

1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
....
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193

1194
1195
1196
1197
1198
1199
1200
....
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
....
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239


1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253


1254
1255
1256
1257
1258
1259
1260
....
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
....
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
....
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
....
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
....
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
....
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
....
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
....
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
....
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
....
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
....
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
....
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
....
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
    TSQLRecordMany) to be used with VirtualTableExternalRegister() - there was
    indeed no implementation requirement to force a specific class type
  - now create properly UNIQUE fields (i.e. "stored AS_UNIQUE") in external tables
  - handle NULL values for BLOBs as expected

  Version 1.18
  - unit SQLite3DB.pas renamed mORMotDB.pas

  - huge performance boost when inserting individual data rows, by maintaining
    the IDs in memory instead of executing "select max(id)" - added new property
    EngineAddUseSelectMaxID to unset this optimization
  - new function VirtualTableExternalRegisterAll(), to register all tables
    of a mORMot model to be handled via a specified database
  - TSQLRestServerStaticExternal.AdaptSQLForEngineList() will now accept
    'select count(*) from TableName [where...]' statements directly (virtual
    behavior for count(*) is to loop through all records, which may be slow)
  - now TSQLRestServerStaticExternal will call TSQLRestServer.OnUpdateEvent and
    OnBlobUpdateEvent callbacks, if defined (even in BATCH mode)
  - BatchDelete() will now split its batch statement executed following
    TSQLDBConnectionProperties.BatchMaxSentAtOnce property expectations 
  - now TSQLRestServerStaticExternal won't create any columns for external
    tables with unsupported published property types (sftUnknown or sftMany),
    just like TSQLRecord.GetSQLCreate() method
  - now handles TSQLDBConnectionProperties.ForcedSchemaName as expected
  - fixed issue in TSQLRestServerStaticExternal.EngineDeleteWhere() when
    calling commands like MyDB.Delete(TSQLMyClass, 'PLU < ?', [20000])
  - fixed errors when executing JOINed queries (e.g. via FillPrepareMany)
  - fixed ticket [3c41462594] in TSQLRestServerStaticExternal.ExecuteFromJSON()
  - fixed ticket [9a821d26ee] in TSQLRestServerStaticExternal.Create() not
    creating any missing field
  - ensure no INDEX is created for SQLite3 which generates an index for ID/RowID
  - ensure DESC INDEX is created for Firebird ID column, as expected for
    faster MAX(ID) execution - see http://www.firebirdfaq.org/faq205
  - fixed TSQLRestServerStaticExternal.UpdateBlobFields() to return true
    if no BLOB field is defined, and to proper handle multi-field update
  - fixed ticket [21c2d5ae96] when inserting/updating blob-only table content
  - handle null binding in TSQLRestServerStaticExternal.ExecuteInlined()
  - added TSQLRestServerStaticExternal.TableHasRows/TableRowCount overrides
  - added TSQLRestServerStaticExternal.PrepareInlinedForRows() and
    PrepareDirectForRows() methods to call new ExecutePreparedAndFetchAllAsJSON()
    method of ISQLDBStatement as expected by TSQLDBProxyStatement
  - optimized TSQLRestServerStaticExternal.UpdateBlobFields()/RetrieveBlobFields()
    methods, updating/retrieving all BLOB fields at once in the same SQL statement
  - this unit will now set SynDBLog := TSQLLog during its initialization
  - replaced confusing TVarData by a new dedicated TSQLVar memory structure,
    shared with SynDB and mORMot units (includes methods refactoring)

}

................................................................................
type
  /// REST server with direct access to a SynDB-based external database
  // - handle all REST commands, using the external SQL database connection,
  // and prepared statements
  // - is used by TSQLRestServer.URI for faster RESTful direct access
  // - for JOINed SQL statements, the external database is also defined as
  // a SQLite3 virtual table, via the TSQLVirtualTableExternal[Cursor] classes
  TSQLRestServerStaticExternal = class(TSQLRestServerStatic)
  protected
    /// values retrieved from fStoredClassProps.ExternalDB settings
    fTableName: RawUTF8;
    fProperties: TSQLDBConnectionProperties;
    fSelectOneDirectSQL, fSelectAllDirectSQL, fSelectTableHasRowsSQL: RawUTF8;
    fRetrieveBlobFieldsSQL, fUpdateBlobfieldsSQL: RawUTF8;
    fEngineUseSelectMaxID: Boolean;
................................................................................
      ExpectResults: Boolean): ISQLDBRows;
    /// overloaded method using FormatUTF8() and binding SynDB parameters
    function ExecuteDirectSQLVar(SQLFormat: PUTF8Char; const Args: array of const;
       var Params: TSQLVarDynArray; LastIntegerParam: integer; ParamsMatchCopiableFields: boolean): boolean;
    // overriden methods calling the external engine with SQL via Execute
    function EngineRetrieve(TableModelIndex, ID: integer): RawUTF8; override;
    function EngineLockedNextID: Integer; virtual;
    function EngineAdd(Table: TSQLRecordClass; const SentData: RawUTF8): integer; override;
    function EngineUpdate(Table: TSQLRecordClass; ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDeleteWhere(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    // BLOBs should be access directly, not through slower JSON Base64 encoding
    function EngineRetrieveBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineSearchField(const FieldName: ShortString;
      const FieldValue: array of const; var ResultID: TIntegerDynArray): boolean;
    // overriden method returning TRUE for next calls to EngineAdd/Update/Delete 
    // will properly handle operations until InternalBatchStop is called
    function InternalBatchStart(Method: TSQLURIMethod): boolean; override;
    // internal method called by TSQLRestServer.RunBatch() to process fast sending
................................................................................
    // in practice, just call the global VirtualTableExternalRegister() procedure
    // - RecordProps.ExternalDatabase will map the associated TSQLDBConnectionProperties
    // - RecordProps.ExternalTableName will retrieve the real full table name,
    // e.g. including any database schema prefix
    constructor Create(aClass: TSQLRecordClass; aServer: TSQLRestServer;
      const aFileName: TFileName = ''; aBinaryFile: boolean=false); override;
    /// delete a row, calling the external engine with SQL
    // - made public since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function EngineDelete(Table: TSQLRecordClass; ID: integer): boolean; override;
    /// search for a numerical field value
    // - return true on success (i.e. if some values have been added to ResultID)
    // - store the results into the ResultID dynamic array
    function SearchField(const FieldName: RawUTF8; FieldValue: Integer;
      var ResultID: TIntegerDynArray): boolean; overload; override;
    /// search for a field value, according to its SQL content representation
    // - return true on success (i.e. if some values have been added to ResultID)
................................................................................
    function TransactionBegin(aTable: TSQLRecordClass; SessionID: cardinal=1): boolean; override;
    {{ end a transaction (implements REST END Member)
     - write all pending SQL statements to the external database }
    procedure Commit(SessionID: cardinal=1); override;
    {{ abort a transaction (implements REST ABORT Member)
     - restore the previous state of the database, before the call to TransactionBegin }
    procedure RollBack(SessionID: cardinal=1); override;
    /// overriden method for direct external SQL database engine thread-safe process
    // - this method will in fact call only one (first) statement
    // - it will convert all inlined parameters (like :(1234): into bound
    // parameters)
    function EngineExecuteAll(const aSQL: RawUTF8): boolean; override;
    /// update a field value of the external database
    function EngineUpdateField(Table: TSQLRecordClass;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
     /// overriden method for direct external database engine call
    function UpdateBlobFields(Value: TSQLRecord): boolean; override;
     /// overriden method for direct external database engine call
    function RetrieveBlobFields(Value: TSQLRecord): boolean; override;



    /// create one index for all specific FieldNames at once
    // - this method will in fact call the SQLAddIndex method, if the index
    // is not already existing
    function CreateSQLMultiIndex(Table: TSQLRecordClass; const FieldNames: array of RawUTF8;
      Unique: boolean; IndexName: RawUTF8=''): boolean; override;
    /// this method is called by TSQLRestServer.EndCurrentThread method just
    // before a thread is finished to ensure that the associated external DB
................................................................................
    procedure EndCurrentThread(Sender: TThread); override;
    /// reset the internal cache of external table maximum ID
    // - next EngineAdd/BatchAdd will execute SELECT max(ID) FROM externaltable
    procedure ResetMaxIDCache;

    /// retrieve the REST server instance corresponding to an external TSQLRecord
    // - just map aServer.StaticVirtualTable[] and will return nil if not
    // a TSQLRestServerStaticExternal
    // - you can use it e.g. to call MapField() method in a fluent interface
    class function Instance(aClass: TSQLRecordClass;
      aServer: TSQLRestServer): TSQLRestServerStaticExternal;
    /// retrieve the external database connection associated to a TSQLRecord
    // - just map aServer.StaticVirtualTable[] and will return nil if not
    // a TSQLRestServerStaticExternal
    class function ConnectionProperties(aClass: TSQLRecordClass;
      aServer: TSQLRestServer): TSQLDBConnectionProperties; overload;
    /// the associated external database connection
    function ConnectionProperties: TSQLDBConnectionProperties; overload;
    /// by default, any INSERT will compute the new ID from an internal variable
    // - it is very fast and reliable, unless external IDs can be created
    // outside this engine
................................................................................
     associated this virtual table module to any TSQLRecord class
   - transactions are handled by this module, according to the external database }
  TSQLVirtualTableExternal = class(TSQLVirtualTable)
  public { overriden methods }
    /// returns the main specifications of the associated TSQLVirtualTableModule
    // - this is a read/write table, without transaction (yet), associated to the
    // TSQLVirtualTableCursorExternal cursor type, with 'External' as module name
    // and TSQLRestServerStaticExternal as the related static class
    // - no particular class is supplied here, since it will depend on the
    // associated Static TSQLRestServerStaticExternal instance
    class procedure GetTableModuleProperties(var aProperties: TVirtualTableModuleProperties);
      override;
    /// called to determine the best way to access the virtual table
    // - will prepare the request for TSQLVirtualTableCursor.Search()
    // - this overriden method will let the external DB engine perform the search,
    // using a standard SQL "SELECT * FROM .. WHERE .. ORDER BY .." statement
    // - in Where[], Expr must be set to not 0 if needed for Search method,
................................................................................
       or (aModel.Tables[i]=TSQLAuthUser)) then
      continue else
    if not VirtualTableExternalRegister(aModel,aModel.Tables[i],aExternalDB,'') then
      result := false;
end;


{ TSQLRestServerStaticExternal }

procedure TSQLRestServerStaticExternal.Commit(SessionID: cardinal);
begin
  inherited Commit(SessionID); // reset fTransactionActive + write all TSQLVirtualTableJSON
  try
    fProperties.ThreadSafeConnection.Commit;
  except
    on Exception do
      ; // just catch exception
  end;
end;

constructor TSQLRestServerStaticExternal.Create(aClass: TSQLRecordClass;
  aServer: TSQLRestServer; const aFileName: TFileName; aBinaryFile: boolean);

  procedure FieldsInternalInit;
  var i: integer;
  begin
    SetLength(fFieldsExternalToInternal,length(fFieldsExternal));
    with StoredClassProps.ExternalDB do
................................................................................
      StoredClassRecordProps.SQLTableUpdateBlobFields,'=?,','=?');
  end;
  fSelectTableHasRowsSQL := FormatUTF8('select ID from % limit 1',
    [StoredClassRecordProps.SQLTableName]);
  AdaptSQLForEngineList(fSelectTableHasRowsSQL);
end;

function TSQLRestServerStaticExternal.AdaptSQLForEngineList(var SQL: RawUTF8): boolean;
var Prop: ShortString; // to avoid any temporary memory allocation
    P: PUTF8Char;
    W: TTextWriter;

  function PropHandleField(WithAliasIfNeeded: boolean): boolean;
  var int: integer;
  begin
................................................................................
    if not fProperties.AdaptSQLLimitForEngineList(NewSQL,
       Pos.LimitRowCount,Pos.AfterSelect,Pos.WhereClause,Pos.Limit) then
      exit;
  SQL := Trim(NewSQL);
  result := true;
end;

function TSQLRestServerStaticExternal.EngineLockedNextID: Integer;
// fProperties.SQLCreate: ID Int64 PRIMARY KEY -> compute unique RowID
// (not all DB engines handle autoincrement feature - e.g. Oracle does not)
var Rows: ISQLDBRows;
begin
  if (fEngineLockedLastID=0) or EngineAddUseSelectMaxID then begin
    // first method call -> retrieve value from DB
    Rows := ExecuteDirect('select max(%) from %',
................................................................................
      fEngineLockedLastID := Rows.ColumnInt(0) else
      fEngineLockedLastID := 0;
  end;
  inc(fEngineLockedLastID);
  result := fEngineLockedLastID;
end;

function TSQLRestServerStaticExternal.InternalBatchStart(
  Method: TSQLURIMethod): boolean;
const BATCH: array[mPOST..mDELETE] of TSQLDBStatementCRUD = (
  cCreate, cUpdate, cDelete);
begin
  result := false; // means BATCH mode not supported
  if (self<>nil) and (method in [mPOST..mDELETE]) and
     (BATCH[method] in fProperties.BatchSendingAbilities) then begin
    Lock(true); // protected by try..finally in TSQLRestServer.RunBatch
    try
      if fBatchMethod<>mNone then
        raise EORMException.Create('InternalBatchStop should have been called');
      if Method=mPOST then
        fBatchAddedID := EngineLockedNextID else
        fBatchAddedID := 0;
      fBatchMethod := Method;
      fBatchCount := 0;
      result := true; // means BATCH mode is supported
    finally
      if not result then
        UnLock;
    end;
  end;
end;

procedure TSQLRestServerStaticExternal.InternalBatchStop;
var i,j,n,max,BatchBegin,BatchEnd,ValuesMax: integer;
    Query: ISQLDBStatement;
    NotifySQLEvent: TSQLEvent;
    SQL: RawUTF8;
    P: PUTF8Char;
    Fields, ExternalFields: TRawUTF8DynArray;
    Types: TSQLDBFieldTypeArray;
................................................................................
begin
  if fBatchMethod=mNone then
    raise EORMException.CreateFmt('%s.BatchMethod=mNone',[fStoredClassRecordProps.SQLTableName]);
  try
    if fBatchCount>0 then begin
      if (Owner<>nil) and (fBatchMethod=mDelete) then // notify BEFORE deletion
        for i := 0 to fBatchCount-1 do
          TSQLRestServerStaticExternal(Owner). // see protected virtual method
            InternalUpdateEvent(seDelete,fStoredClass,fBatchIDs[i],nil);
      with fProperties do
        if BatchMaxSentAtOnce>0 then
          max := BatchMaxSentAtOnce else
          max := 1000;
      BatchBegin := 0;
      BatchEnd := fBatchCount-1;
      repeat
................................................................................
        BatchBegin := BatchEnd+1;
      until BatchBegin>=fBatchCount;
      if (Owner<>nil) and (fBatchMethod in [mPost,mPut]) then begin
        if fBatchMethod=mPost then
          NotifySQLEvent := seAdd else
          NotifySQLEvent := seUpdate;
          for i := 0 to fBatchCount-1 do
            TSQLRestServerStaticExternal(Owner). // see protected virtual method
              InternalUpdateEvent(NotifySQLEvent,fStoredClass,fBatchIDs[i],nil);
      end;
    end;
  finally
    if fBatchMethod=mPost then
      fEngineLockedLastID := fBatchAddedID+fBatchCount;
    SetLength(fBatchValues,0);
    SetLength(fBatchIDs,0);
    fBatchCount := 0;
    fBatchCapacity := 0;
    fBatchMethod := mNone;
    UnLock;
  end;
end;

function TSQLRestServerStaticExternal.InternalBatchAdd(
  const aValue: RawUTF8; aID: integer): integer;
begin
  result := fBatchAddedID+fBatchCount;
  if fBatchCount>=fBatchCapacity then begin
    fBatchCapacity := fBatchCapacity+64+fBatchCount shr 3;
    SetLength(fBatchIDs,fBatchCapacity);
    if aValue<>'' then
................................................................................
    fBatchValues[fBatchCount] := aValue;
  if aID=0 then
    aID := result;
  fBatchIDs[fBatchCount] := aID;
  inc(fBatchCount);
end;

function TSQLRestServerStaticExternal.EngineAdd(Table: TSQLRecordClass;
  const SentData: RawUTF8): integer;
begin
  if (self=nil) or (Table<>fStoredClass) then
    result := 0 else // avoid GPF
  if fBatchMethod<>mNone then
    if fBatchMethod<>mPOST then
      result := 0 else
      result := InternalBatchAdd(SentData,0) else begin
    result := ExecuteFromJSON(SentData,soInsert,0); // UpdatedID=0 -> insert with EngineLockedNextID
    if (result>0) and (Owner<>nil) then
      TSQLRestServerStaticExternal(Owner). // see protected virtual method
        InternalUpdateEvent(seAdd,fStoredClass,result,nil);
  end;
end;

function TSQLRestServerStaticExternal.EngineUpdate(Table: TSQLRecordClass;
  ID: integer; const SentData: RawUTF8): boolean;
begin
  if (self=nil) or (Table<>fStoredClass) or (ID<=0) then
    result := false else
    if fBatchMethod<>mNone then
      if fBatchMethod<>mPUT then
        result := false else
        result := InternalBatchAdd(SentData,ID)>=0 else begin
      result := ExecuteFromJSON(SentData,soUpdate,ID)=ID;
      if result and (Owner<>nil) then
        TSQLRestServerStaticExternal(Owner). // see protected virtual method
          InternalUpdateEvent(seUpdate,fStoredClass,ID,nil);
    end;
end;

function TSQLRestServerStaticExternal.EngineDelete(Table: TSQLRecordClass;
  ID: integer): boolean;
begin
  if (self=nil) or (Table<>fStoredClass) or (ID<=0) then

    result := false else
    if fBatchMethod<>mNone then
      if fBatchMethod<>mDELETE then
        result := false else
        result := InternalBatchAdd('',ID)>=0 else begin
      result := ExecuteDirect('delete from % where %=?',
        [fTableName,StoredClassProps.ExternalDB.RowIDFieldName],[ID],false)<>nil;
      if result and (Owner<>nil) then
        TSQLRestServerStaticExternal(Owner). // see protected virtual method
          InternalUpdateEvent(seDelete,fStoredClass,ID,nil);
    end;
end;

function TSQLRestServerStaticExternal.EngineDeleteWhere(
  Table: TSQLRecordClass; const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
var i: integer;
begin
  result := false;
  if (self=nil) or (Table<>fStoredClass) or (SQLWhere='') or (IDs=nil) then

    exit;
  if fBatchMethod<>mNone then
    if fBatchMethod<>mDELETE then
      exit else
      for i := 0 to high(IDs) do
        InternalBatchAdd('',IDs[i]) else begin
    if Owner<>nil then // notify BEFORE deletion
      for i := 0 to high(IDs) do
        TSQLRestServerStaticExternal(Owner). // see protected virtual method
          InternalUpdateEvent(seDelete,fStoredClass,IDs[i],nil);
    if ExecuteInlined('delete from % where %',[fTableName,SQLWhere],false)=nil then
      exit;
  end;
  result := true;
end;

function TSQLRestServerStaticExternal.EngineExecuteAll(
  const aSQL: RawUTF8): boolean;
begin
  result := ExecuteInlined(aSQL,false)<>nil; // only execute the first statement
end;

function TSQLRestServerStaticExternal.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean; ReturnedRowCount: PPtrInt): RawUTF8;
var Stmt: ISQLDBStatement;
begin
  if ReturnedRowCount<>nil then
    raise ESQLDBException.CreateFmt('%s.EngineList(ReturnedRowCount<>nil)',[ClassName]);
  Stmt := PrepareInlinedForRows(SQL);
  if Stmt=nil then
    result := '' else
    Stmt.ExecutePreparedAndFetchAllAsJSON(ForceAJAX or (not NoAJAXJSON),result);

end;

function TSQLRestServerStaticExternal.EngineRetrieve(TableModelIndex, ID: integer): RawUTF8;
var Stmt: ISQLDBStatement;
begin // TableModelIndex is not usefull here
  result := '';
  if (self=nil) or (ID<=0) then
    exit;
  Stmt := PrepareDirectForRows(pointer(fSelectOneDirectSQL),[],[ID]);
  if Stmt<>nil then begin
................................................................................
      // '{"fieldCount":2,"values":["ID","FirstName"]}'#$A -> ID not found
      result := '' else
      // list '[{...}]'#10 -> object '{...}'
      result := copy(result,2,length(result)-3);
  end;
end;

function TSQLRestServerStaticExternal.TableHasRows(Table: TSQLRecordClass): boolean;
var Rows: ISQLDBRows;
begin
  if (self=nil) or (Table<>fStoredClass) then
    result := false else begin
    Rows := ExecuteDirect(pointer(fSelectTableHasRowsSQL),[],[],true);
    if Rows=nil then
      result := false else
      result := Rows.Step;
  end;
end;

function TSQLRestServerStaticExternal.TableRowCount(Table: TSQLRecordClass): integer;
var Rows: ISQLDBRows;
begin
  if (self=nil) or (Table<>fStoredClass) then
    result := 0 else begin
    Rows := ExecuteDirect('select count(*) from %',[fTableName],[],true);
    if (Rows=nil) or not Rows.Step then
      result := 0 else
      result := Rows.ColumnInt(0);
  end;
end;

function TSQLRestServerStaticExternal.EngineRetrieveBlob(
  Table: TSQLRecordClass; aID: integer; BlobField: PPropInfo;
  out BlobData: TSQLRawBlob): boolean;
var Rows: ISQLDBRows;
begin
  result := false;
  if (self=nil) or (Table<>fStoredClass) or (aID<=0) or not BlobField^.IsBlob then

    exit;
  with StoredClassProps.ExternalDB do
    Rows := ExecuteDirect('select % from % where %=?',
      [InternalToExternal(BlobField^.Name),fTableName,RowIDFieldName],[aID],true);
  if (Rows<>nil) and Rows.Step then
  try
    BlobData := Rows.ColumnBlob(0);
................................................................................
    Rows := nil;
  except
    on Exception do
      result := false;
  end;
end;

function TSQLRestServerStaticExternal.RetrieveBlobFields(Value: TSQLRecord): boolean;
var Rows: ISQLDBRows;
    f: Integer;
    data: TSQLVar;
    temp: RawByteString;
begin
  result := false;
  if (Value<>nil) and (Value.ID>0) and (PSQLRecordClass(Value)^=fStoredClass) then
................................................................................
    except
      on Exception do
        result := false;
    end;
  end;
end;

function TSQLRestServerStaticExternal.EngineUpdateField(Table: TSQLRecordClass;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
begin


  with StoredClassProps.ExternalDB do
    result := ExecuteInlined('update % set %=:(%): where %=:(%):',
      [fTableName,InternalToExternal(SetFieldName),SetValue,
       InternalToExternal(WhereFieldName),WhereValue],false)<>nil;
end;

function TSQLRestServerStaticExternal.EngineUpdateBlob(
  Table: TSQLRecordClass; aID: integer; BlobField: PPropInfo;
  const BlobData: TSQLRawBlob): boolean;
var Statement: ISQLDBStatement;
    AffectedField: TSQLFieldBits;
begin
  result := false;
  if (Table=fStoredClass) and (aID>0) and BlobField^.IsBlob then


  try
    if Owner<>nil then
      Owner.FlushInternalDBCache;
    with StoredClassProps.ExternalDB do
      Statement := fProperties.NewThreadSafeStatementPrepared(
        'update % set %=? where %=?',
        [fTableName,InternalToExternal(BlobField^.Name),RowIDFieldName],false);
................................................................................
      if BlobData='' then
        Statement.BindNull(1) else
        Statement.BindBlob(1,BlobData); // fast explicit BindBlob() call
      Statement.Bind(2,aID);
      Statement.ExecutePrepared;
      if Owner<>nil then begin
        fStoredClassRecordProps.FieldIndexsFromBlobField(BlobField,AffectedField);
        TSQLRestServerStaticExternal(Owner). // see protected virtual method
          InternalUpdateEvent(seUpdateBlob,fStoredClass,aID,@AffectedField);
      end;
      result := true; // success
    end;
  except
    on Exception do
      result := false;
  end;
end;

function TSQLRestServerStaticExternal.UpdateBlobFields(Value: TSQLRecord): boolean;
var f, aID: integer;
    temp: array of RawByteString;
    Params: TSQLVarDynArray;
begin
  result := false;
  if (Value<>nil) and (PSQLRecordClass(Value)^=fStoredClass) then
  with Value.RecordProps do
................................................................................
    SetLength(temp,length(BlobFields));
    for f := 0 to high(Params) do
      BlobFields[f].GetFieldSQLVar(Value,Params[f],temp[f]);
    result := ExecuteDirectSQLVar('update % set % where %=?',
      [fTableName,fUpdateBlobFieldsSQL,StoredClassProps.ExternalDB.RowIDFieldName],
      Params,aID,false);
    if result and (Owner<>nil) then
      TSQLRestServerStaticExternal(Owner). // see protected virtual method
        InternalUpdateEvent(seUpdateBlob,fStoredClass,aID,
          @fStoredClassRecordProps.BlobFieldsBits);
  end else
    result := true; // as TSQLRest.UpdateblobFields()
end;

function TSQLRestServerStaticExternal.PrepareInlinedForRows(const aSQL: RawUTF8): ISQLDBStatement;
begin
  result := nil; // returns nil interface on error
  if self=nil then
    exit;
  try
    result := fProperties.PrepareInlined(aSQL,true);
  except
    on Exception do
      result := nil;
  end;
end;

function TSQLRestServerStaticExternal.ExecuteInlined(const aSQL: RawUTF8;
  ExpectResults: Boolean): ISQLDBRows;
begin
  result := nil; // returns nil interface on error
  if self=nil then
    exit;
  if (not ExpectResults) and (Owner<>nil) then
    Owner.FlushInternalDBCache; // add/update/delete should flush DB cache
................................................................................
    result := fProperties.ExecuteInlined(aSQL,ExpectResults);
  except
    on Exception do
      result := nil;
  end;
end;

function TSQLRestServerStaticExternal.ExecuteInlined(SQLFormat: PUTF8Char;
  const Args: array of const; ExpectResults: Boolean): ISQLDBRows;
begin
  result := ExecuteInlined(FormatUTF8(SQLFormat,Args),ExpectResults);
end;

function TSQLRestServerStaticExternal.PrepareDirectForRows(SQLFormat: PUTF8Char;
  const Args, Params: array of const): ISQLDBStatement;
var Query: ISQLDBStatement;
begin
  result := nil;
  if self=nil then
    exit;
  Query := fProperties.NewThreadSafeStatementPrepared(SQLFormat,Args,true);
................................................................................
    result := Query;
  except
    on Exception do
      result := nil;
  end;
end;

function TSQLRestServerStaticExternal.ExecuteDirect(SQLFormat: PUTF8Char;
  const Args, Params: array of const; ExpectResults: Boolean): ISQLDBRows;
var Query: ISQLDBStatement;
begin
  result := nil;
  if self=nil then
    exit;
  if (not ExpectResults) and (Owner<>nil) then
................................................................................
    result := Query;
  except
    on Exception do
      result := nil;
  end;
end;

function TSQLRestServerStaticExternal.ExecuteDirectSQLVar(SQLFormat: PUTF8Char;
  const Args: array of const; var Params: TSQLVarDynArray; LastIntegerParam: integer;
  ParamsMatchCopiableFields: boolean): boolean;
var Query: ISQLDBStatement;
    ParamsCount, f: integer;
begin
  result := false;
  if Self<>nil then
................................................................................
    result := true; // success
  except
    on Exception do
      result := false;
  end;
end;

procedure TSQLRestServerStaticExternal.RollBack(SessionID: cardinal);
begin
  inherited RollBack(SessionID); // reset fTransactionActive
  try
    fProperties.ThreadSafeConnection.Rollback;
  except
    on Exception do
      ; // just catch exception
  end;
end;

function TSQLRestServerStaticExternal.EngineSearchField(
  const FieldName: ShortString; const FieldValue: array of const;
  var ResultID: TIntegerDynArray): boolean;
var n: Integer;
    Rows: ISQLDBRows;
begin
  n := 0;
  Rows := ExecuteDirect('select % from % where %=?',
................................................................................
  if Rows<>nil then
    while Rows.Step do
      AddInteger(ResultID,n,Rows.ColumnInt(0));
  SetLength(ResultID,n);
  result := n>0;
end;

function TSQLRestServerStaticExternal.SearchField(const FieldName: RawUTF8;
  FieldValue: Integer; var ResultID: TIntegerDynArray): boolean;
begin
  result := EngineSearchField(FieldName,[FieldValue],ResultID);
end;

function TSQLRestServerStaticExternal.SearchField(const FieldName, FieldValue: RawUTF8;
  var ResultID: TIntegerDynArray): boolean;
begin
  result := EngineSearchField(FieldName,[FieldValue],ResultID);
end;

function TSQLRestServerStaticExternal.TransactionBegin(
  aTable: TSQLRecordClass; SessionID: cardinal): boolean;
begin
  result := false;
  if (aTable=fStoredClass) and inherited TransactionBegin(aTable,SessionID) then
  try
    fProperties.ThreadSafeConnection.StartTransaction;
    result := true; // success
  except
    on Exception do
      result := false;
  end;
end;

function TSQLRestServerStaticExternal.CreateSQLMultiIndex(
  Table: TSQLRecordClass; const FieldNames: array of RawUTF8;
  Unique: boolean; IndexName: RawUTF8): boolean;
var SQL: RawUTF8;
    ExtFieldNames: TRawUTF8DynArray;
    Descending: boolean;
    i: integer;
begin
................................................................................
  end;
  StoredClassProps.ExternalDB.InternalToExternalDynArray(FieldNames,ExtFieldNames);
  SQL := fProperties.SQLAddIndex(fTableName,ExtFieldNames,Unique,Descending,IndexName);
  if (SQL<>'') and (ExecuteDirect(pointer(SQL),[],[],false)<>nil) then
    result := true;
end;

class function TSQLRestServerStaticExternal.Instance(
  aClass: TSQLRecordClass; aServer: TSQLRestServer): TSQLRestServerStaticExternal;
begin
  if (aClass=nil) or (aServer=nil) then
    result := nil else begin
    result := TSQLRestServerStaticExternal(aServer.StaticVirtualTable[aClass]);
    if result<>nil then
      if not result.InheritsFrom(TSQLRestServerStaticExternal) then
        result := nil;
  end;
end;

class function TSQLRestServerStaticExternal.ConnectionProperties(
  aClass: TSQLRecordClass; aServer: TSQLRestServer): TSQLDBConnectionProperties;
begin
  result := Instance(aClass,aServer).ConnectionProperties;
end;

function TSQLRestServerStaticExternal.ConnectionProperties: TSQLDBConnectionProperties;
begin
  if self=nil then
    result := nil else
    result := fProperties;
end;
  
function TSQLRestServerStaticExternal.ExecuteFromJSON(
  const SentData: RawUTF8; Occasion: TSQLOccasion; UpdatedID: integer): integer;
var Decoder: TJSONObjectDecoder;
    SQL: RawUTF8;
    Types: TSQLDBFieldTypeArray;
    ExternalFields: TRawUTF8DynArray;
    InsertedID, F: integer;
    Query: ISQLDBStatement;
begin
  result := 0;
  Lock(false); // avoid race condition against max(ID)
  try
    case Occasion of
    soInsert: begin
      InsertedID := JSONRetrieveIDField(pointer(SentData));
      if InsertedID=0 then // no specified "ID":... field value -> compute next
        InsertedID := EngineLockedNextID else
        if result>fEngineLockedLastID then
................................................................................
      exit; // leave result=0
    end;
    // mark success
    if UpdatedID=0 then
      result := InsertedID else
      result := UpdatedID;
  finally
    UnLock;
  end;
end;

procedure TSQLRestServerStaticExternal.EndCurrentThread(Sender: TThread);
begin
  if fProperties.InheritsFrom(TSQLDBConnectionPropertiesThreadSafe) then
    TSQLDBConnectionPropertiesThreadSafe(fProperties).EndCurrentThread;
end;

function TSQLRestServerStaticExternal.InternalFieldNameToFieldExternalIndex(
  const InternalFieldName: RawUTF8): integer;
begin
  result := StoredClassRecordProps.Fields.IndexByNameOrExcept(InternalFieldName);
  result := IntegerScanIndex(Pointer(fFieldsExternalToInternal),
    length(fFieldsExternalToInternal),result);
end;

function TSQLRestServerStaticExternal.JSONDecodedPrepareToSQL(
  var Decoder: TJSONObjectDecoder; out ExternalFields: TRawUTF8DynArray;
  out Types: TSQLDBFieldTypeArray; Occasion: TSQLOccasion): RawUTF8;
var f,k: Integer;
begin
  SetLength(ExternalFields,Decoder.FieldCount);
  for f := 0 to Decoder.FieldCount-1 do begin
    k := InternalFieldNameToFieldExternalIndex(Decoder.FieldNames[f]);
................................................................................
    StoredClassProps.ExternalDB.RowIDFieldName);
  if Occasion=soUpdate then begin
    Types[Decoder.FieldCount] := ftInt64; // add "where ID=?" parameter
    inc(Decoder.FieldCount);
  end;
end;

procedure TSQLRestServerStaticExternal.ResetMaxIDCache;
begin
  Lock(true);
  fEngineLockedLastID := 0;
  UnLock;
end;


{ TSQLVirtualTableCursorExternal }

function TSQLVirtualTableCursorExternal.Column(aColumn: integer;
  var aResult: TSQLVar): boolean;
................................................................................
function TSQLVirtualTableCursorExternal.Search(
  const Prepared: TSQLVirtualTablePrepared): boolean;
var i: integer;
begin
  result := false;
  if (Self=nil) or (Table=nil) or (Table.Static=nil) then
    exit;
  with Table.Static as TSQLRestServerStaticExternal do begin
    if fSQL='' then begin
      // compute the SQL query corresponding to this prepared request
      fSQL := fSelectAllDirectSQL;
      if Prepared.WhereCount<>0 then begin
        for i := 0 to Prepared.WhereCount-1 do
        with Prepared.Where[i] do begin
          if Operation>high(SQL_OPER_WITH_PARAM) then
................................................................................


{ TSQLVirtualTableExternal }

function TSQLVirtualTableExternal.Delete(aRowID: Int64): boolean;
begin
  result := (self<>nil) and (Static<>nil) and
    (Static as TSQLRestServerStaticExternal).EngineDelete(Static.StoredClass,aRowID);
end;

function TSQLVirtualTableExternal.Drop: boolean;
begin
  if (self=nil) or (Static=nil) then
    result := false else
    with Static as TSQLRestServerStaticExternal do
      result := ExecuteDirect('drop table %',[fTableName],[],false)<>nil;
end;

class procedure TSQLVirtualTableExternal.GetTableModuleProperties(
  var aProperties: TVirtualTableModuleProperties);
begin
  aProperties.Features := [vtWrite];
  aProperties.CursorClass := TSQLVirtualTableCursorExternal;
  aProperties.StaticClass := TSQLRestServerStaticExternal;
end;

function TSQLVirtualTableExternal.Insert(aRowID: Int64;
  var Values: TSQLVarDynArray; out insertedRowID: Int64): boolean;
begin // aRowID is just ignored here since IDs are always auto calculated
  result := false;
  if (self<>nil) and (Static<>nil) then
  with Static as TSQLRestServerStaticExternal do begin
    Lock(false); // to avoid race condition against max(RowID)
    try
      insertedRowID := EngineLockedNextID;
      with StoredClassProps.ExternalDB do
        result := ExecuteDirectSQLVar('insert into % (%,%) values (%,?)',
          [fTableName,SQL.InsertSet,RowIDFieldName,CSVOfValue('?',length(Values))],
          Values,insertedRowID,true);
    finally
      UnLock;
    end;
  end;
end;

function TSQLVirtualTableExternal.Prepare(var Prepared: TSQLVirtualTablePrepared): boolean;
var i, col: integer;
    hasIndex: boolean;
    Fields: TSQLPropInfoList;
begin
  result := inherited Prepare(Prepared); // Prepared.EstimatedCost := 1E10;
  if result and (Static<>nil) then
  with Static as TSQLRestServerStaticExternal do begin
    // mark Where[] clauses will be handled by SQL
    Fields := StoredClassRecordProps.Fields;
    result := false;
    for i := 0 to Prepared.WhereCount-1 do
      with Prepared.Where[i] do
      if (Column<>VIRTUAL_TABLE_IGNORE_COLUMN) and
         (Operation<=high(SQL_OPER_WITH_PARAM)) then begin
................................................................................
end;

function TSQLVirtualTableExternal.Update(oldRowID, newRowID: Int64;
  var Values: TSQLVarDynArray): boolean;
begin
  if (self<>nil) and (Static<>nil) and
     (oldRowID=newRowID) and (newRowID>0) then // don't allow ID change
    with Static as TSQLRestServerStaticExternal, StoredClassProps.ExternalDB do
      result := ExecuteDirectSQLVar('update % set % where %=?',
        [fTableName,SQL.UpdateSetAll,RowIDFieldName],Values,oldRowID,true) else
    result := false;
end;


initialization
  // all our SynDB related functions shall log to main TSQLLog
  SynDBLog := TSQLLog;
end.







>





|


|



|



|


|
|




|


|
|
|


|







 







|







 







|
|
|



|

|







 







|

|







 







<
<
<
<
<
<
<
<




>
>
>







 







|


|


|







 







|

|







 







|

|










|







 







|







 







|







 







|







|











|




|







 







<
|







 







<
|










|



|







 







|


|







<
|



|
|

|







<
|



<
|

<
>








<
|



|
|



|
>








<
|






<
<
<
<
<
<
|








|
>


|







 







|











|











|
<
|



|
>







 







|







 







|


>
>
|
|
|
|


|
<
|




|
>
>







 







<
|









|







 







<
|





|












|







 







|





|







 







|







 







|







 







|










|







 







|





|





|













|







 







|
|



|

|




|





|






|









|







 







|



|





|







|







 







|

|

|







 







|







 







|






|








|







|
|







|











|







 







|










73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
...
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
...
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
...
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
...
265
266
267
268
269
270
271








272
273
274
275
276
277
278
279
280
281
282
283
284
285
...
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
...
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
...
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
...
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
...
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
...
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
...
912
913
914
915
916
917
918

919
920
921
922
923
924
925
926
....
1011
1012
1013
1014
1015
1016
1017

1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
....
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061

1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076

1077
1078
1079
1080

1081
1082

1083
1084
1085
1086
1087
1088
1089
1090
1091

1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110

1111
1112
1113
1114
1115
1116
1117






1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
....
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172

1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
....
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
....
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233

1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
....
1250
1251
1252
1253
1254
1255
1256

1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
....
1282
1283
1284
1285
1286
1287
1288

1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
....
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
....
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
....
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
....
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
....
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
....
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
....
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
....
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
....
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
....
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
....
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
    TSQLRecordMany) to be used with VirtualTableExternalRegister() - there was
    indeed no implementation requirement to force a specific class type
  - now create properly UNIQUE fields (i.e. "stored AS_UNIQUE") in external tables
  - handle NULL values for BLOBs as expected

  Version 1.18
  - unit SQLite3DB.pas renamed mORMotDB.pas
  - TSQLRestServerStaticExternal renamed TSQLRestStorageExternal
  - huge performance boost when inserting individual data rows, by maintaining
    the IDs in memory instead of executing "select max(id)" - added new property
    EngineAddUseSelectMaxID to unset this optimization
  - new function VirtualTableExternalRegisterAll(), to register all tables
    of a mORMot model to be handled via a specified database
  - TSQLRestStorageExternal.AdaptSQLForEngineList() will now accept
    'select count(*) from TableName [where...]' statements directly (virtual
    behavior for count(*) is to loop through all records, which may be slow)
  - now TSQLRestStorageExternal will call TSQLRestServer.OnUpdateEvent and
    OnBlobUpdateEvent callbacks, if defined (even in BATCH mode)
  - BatchDelete() will now split its batch statement executed following
    TSQLDBConnectionProperties.BatchMaxSentAtOnce property expectations 
  - now TSQLRestStorageExternal won't create any columns for external
    tables with unsupported published property types (sftUnknown or sftMany),
    just like TSQLRecord.GetSQLCreate() method
  - now handles TSQLDBConnectionProperties.ForcedSchemaName as expected
  - fixed issue in TSQLRestStorageExternal.EngineDeleteWhere() when
    calling commands like MyDB.Delete(TSQLMyClass, 'PLU < ?', [20000])
  - fixed errors when executing JOINed queries (e.g. via FillPrepareMany)
  - fixed ticket [3c41462594] in TSQLRestStorageExternal.ExecuteFromJSON()
  - fixed ticket [9a821d26ee] in TSQLRestStorageExternal.Create() not
    creating any missing field
  - ensure no INDEX is created for SQLite3 which generates an index for ID/RowID
  - ensure DESC INDEX is created for Firebird ID column, as expected for
    faster MAX(ID) execution - see http://www.firebirdfaq.org/faq205
  - fixed TSQLRestStorageExternal.UpdateBlobFields() to return true
    if no BLOB field is defined, and to proper handle multi-field update
  - fixed ticket [21c2d5ae96] when inserting/updating blob-only table content
  - handle null binding in TSQLRestStorageExternal.ExecuteInlined()
  - added TSQLRestStorageExternal.TableHasRows/TableRowCount overrides
  - added TSQLRestStorageExternal.PrepareInlinedForRows() and
    PrepareDirectForRows() methods to call new ExecutePreparedAndFetchAllAsJSON()
    method of ISQLDBStatement as expected by TSQLDBProxyStatement
  - optimized TSQLRestStorageExternal.UpdateBlobFields()/RetrieveBlobFields()
    methods, updating/retrieving all BLOB fields at once in the same SQL statement
  - this unit will now set SynDBLog := TSQLLog during its initialization
  - replaced confusing TVarData by a new dedicated TSQLVar memory structure,
    shared with SynDB and mORMot units (includes methods refactoring)

}

................................................................................
type
  /// REST server with direct access to a SynDB-based external database
  // - handle all REST commands, using the external SQL database connection,
  // and prepared statements
  // - is used by TSQLRestServer.URI for faster RESTful direct access
  // - for JOINed SQL statements, the external database is also defined as
  // a SQLite3 virtual table, via the TSQLVirtualTableExternal[Cursor] classes
  TSQLRestStorageExternal = class(TSQLRestStorage)
  protected
    /// values retrieved from fStoredClassProps.ExternalDB settings
    fTableName: RawUTF8;
    fProperties: TSQLDBConnectionProperties;
    fSelectOneDirectSQL, fSelectAllDirectSQL, fSelectTableHasRowsSQL: RawUTF8;
    fRetrieveBlobFieldsSQL, fUpdateBlobfieldsSQL: RawUTF8;
    fEngineUseSelectMaxID: Boolean;
................................................................................
      ExpectResults: Boolean): ISQLDBRows;
    /// overloaded method using FormatUTF8() and binding SynDB parameters
    function ExecuteDirectSQLVar(SQLFormat: PUTF8Char; const Args: array of const;
       var Params: TSQLVarDynArray; LastIntegerParam: integer; ParamsMatchCopiableFields: boolean): boolean;
    // overriden methods calling the external engine with SQL via Execute
    function EngineRetrieve(TableModelIndex, ID: integer): RawUTF8; override;
    function EngineLockedNextID: Integer; virtual;
    function EngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer; override;
    function EngineUpdate(TableModelIndex, ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDeleteWhere(TableModelIndex: integer; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    // BLOBs should be access directly, not through slower JSON Base64 encoding
    function EngineRetrieveBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineSearchField(const FieldName: ShortString;
      const FieldValue: array of const; var ResultID: TIntegerDynArray): boolean;
    // overriden method returning TRUE for next calls to EngineAdd/Update/Delete 
    // will properly handle operations until InternalBatchStop is called
    function InternalBatchStart(Method: TSQLURIMethod): boolean; override;
    // internal method called by TSQLRestServer.RunBatch() to process fast sending
................................................................................
    // in practice, just call the global VirtualTableExternalRegister() procedure
    // - RecordProps.ExternalDatabase will map the associated TSQLDBConnectionProperties
    // - RecordProps.ExternalTableName will retrieve the real full table name,
    // e.g. including any database schema prefix
    constructor Create(aClass: TSQLRecordClass; aServer: TSQLRestServer;
      const aFileName: TFileName = ''; aBinaryFile: boolean=false); override;
    /// delete a row, calling the external engine with SQL
    // - made public since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function EngineDelete(TableModelIndex, ID: integer): boolean; override;
    /// search for a numerical field value
    // - return true on success (i.e. if some values have been added to ResultID)
    // - store the results into the ResultID dynamic array
    function SearchField(const FieldName: RawUTF8; FieldValue: Integer;
      var ResultID: TIntegerDynArray): boolean; overload; override;
    /// search for a field value, according to its SQL content representation
    // - return true on success (i.e. if some values have been added to ResultID)
................................................................................
    function TransactionBegin(aTable: TSQLRecordClass; SessionID: cardinal=1): boolean; override;
    {{ end a transaction (implements REST END Member)
     - write all pending SQL statements to the external database }
    procedure Commit(SessionID: cardinal=1); override;
    {{ abort a transaction (implements REST ABORT Member)
     - restore the previous state of the database, before the call to TransactionBegin }
    procedure RollBack(SessionID: cardinal=1); override;








     /// overriden method for direct external database engine call
    function UpdateBlobFields(Value: TSQLRecord): boolean; override;
     /// overriden method for direct external database engine call
    function RetrieveBlobFields(Value: TSQLRecord): boolean; override;
    /// update a field value of the external database
    function EngineUpdateField(TableModelIndex: integer;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
    /// create one index for all specific FieldNames at once
    // - this method will in fact call the SQLAddIndex method, if the index
    // is not already existing
    function CreateSQLMultiIndex(Table: TSQLRecordClass; const FieldNames: array of RawUTF8;
      Unique: boolean; IndexName: RawUTF8=''): boolean; override;
    /// this method is called by TSQLRestServer.EndCurrentThread method just
    // before a thread is finished to ensure that the associated external DB
................................................................................
    procedure EndCurrentThread(Sender: TThread); override;
    /// reset the internal cache of external table maximum ID
    // - next EngineAdd/BatchAdd will execute SELECT max(ID) FROM externaltable
    procedure ResetMaxIDCache;

    /// retrieve the REST server instance corresponding to an external TSQLRecord
    // - just map aServer.StaticVirtualTable[] and will return nil if not
    // a TSQLRestStorageExternal
    // - you can use it e.g. to call MapField() method in a fluent interface
    class function Instance(aClass: TSQLRecordClass;
      aServer: TSQLRestServer): TSQLRestStorageExternal;
    /// retrieve the external database connection associated to a TSQLRecord
    // - just map aServer.StaticVirtualTable[] and will return nil if not
    // a TSQLRestStorageExternal
    class function ConnectionProperties(aClass: TSQLRecordClass;
      aServer: TSQLRestServer): TSQLDBConnectionProperties; overload;
    /// the associated external database connection
    function ConnectionProperties: TSQLDBConnectionProperties; overload;
    /// by default, any INSERT will compute the new ID from an internal variable
    // - it is very fast and reliable, unless external IDs can be created
    // outside this engine
................................................................................
     associated this virtual table module to any TSQLRecord class
   - transactions are handled by this module, according to the external database }
  TSQLVirtualTableExternal = class(TSQLVirtualTable)
  public { overriden methods }
    /// returns the main specifications of the associated TSQLVirtualTableModule
    // - this is a read/write table, without transaction (yet), associated to the
    // TSQLVirtualTableCursorExternal cursor type, with 'External' as module name
    // and TSQLRestStorageExternal as the related static class
    // - no particular class is supplied here, since it will depend on the
    // associated Static TSQLRestStorageExternal instance
    class procedure GetTableModuleProperties(var aProperties: TVirtualTableModuleProperties);
      override;
    /// called to determine the best way to access the virtual table
    // - will prepare the request for TSQLVirtualTableCursor.Search()
    // - this overriden method will let the external DB engine perform the search,
    // using a standard SQL "SELECT * FROM .. WHERE .. ORDER BY .." statement
    // - in Where[], Expr must be set to not 0 if needed for Search method,
................................................................................
       or (aModel.Tables[i]=TSQLAuthUser)) then
      continue else
    if not VirtualTableExternalRegister(aModel,aModel.Tables[i],aExternalDB,'') then
      result := false;
end;


{ TSQLRestStorageExternal }

procedure TSQLRestStorageExternal.Commit(SessionID: cardinal);
begin
  inherited Commit(SessionID); // reset fTransactionActive + write all TSQLVirtualTableJSON
  try
    fProperties.ThreadSafeConnection.Commit;
  except
    on Exception do
      ; // just catch exception
  end;
end;

constructor TSQLRestStorageExternal.Create(aClass: TSQLRecordClass;
  aServer: TSQLRestServer; const aFileName: TFileName; aBinaryFile: boolean);

  procedure FieldsInternalInit;
  var i: integer;
  begin
    SetLength(fFieldsExternalToInternal,length(fFieldsExternal));
    with StoredClassProps.ExternalDB do
................................................................................
      StoredClassRecordProps.SQLTableUpdateBlobFields,'=?,','=?');
  end;
  fSelectTableHasRowsSQL := FormatUTF8('select ID from % limit 1',
    [StoredClassRecordProps.SQLTableName]);
  AdaptSQLForEngineList(fSelectTableHasRowsSQL);
end;

function TSQLRestStorageExternal.AdaptSQLForEngineList(var SQL: RawUTF8): boolean;
var Prop: ShortString; // to avoid any temporary memory allocation
    P: PUTF8Char;
    W: TTextWriter;

  function PropHandleField(WithAliasIfNeeded: boolean): boolean;
  var int: integer;
  begin
................................................................................
    if not fProperties.AdaptSQLLimitForEngineList(NewSQL,
       Pos.LimitRowCount,Pos.AfterSelect,Pos.WhereClause,Pos.Limit) then
      exit;
  SQL := Trim(NewSQL);
  result := true;
end;

function TSQLRestStorageExternal.EngineLockedNextID: Integer;
// fProperties.SQLCreate: ID Int64 PRIMARY KEY -> compute unique RowID
// (not all DB engines handle autoincrement feature - e.g. Oracle does not)
var Rows: ISQLDBRows;
begin
  if (fEngineLockedLastID=0) or EngineAddUseSelectMaxID then begin
    // first method call -> retrieve value from DB
    Rows := ExecuteDirect('select max(%) from %',
................................................................................
      fEngineLockedLastID := Rows.ColumnInt(0) else
      fEngineLockedLastID := 0;
  end;
  inc(fEngineLockedLastID);
  result := fEngineLockedLastID;
end;

function TSQLRestStorageExternal.InternalBatchStart(
  Method: TSQLURIMethod): boolean;
const BATCH: array[mPOST..mDELETE] of TSQLDBStatementCRUD = (
  cCreate, cUpdate, cDelete);
begin
  result := false; // means BATCH mode not supported
  if (self<>nil) and (method in [mPOST..mDELETE]) and
     (BATCH[method] in fProperties.BatchSendingAbilities) then begin
    StorageLock(true); // protected by try..finally in TSQLRestServer.RunBatch
    try
      if fBatchMethod<>mNone then
        raise EORMException.Create('InternalBatchStop should have been called');
      if Method=mPOST then
        fBatchAddedID := EngineLockedNextID else
        fBatchAddedID := 0;
      fBatchMethod := Method;
      fBatchCount := 0;
      result := true; // means BATCH mode is supported
    finally
      if not result then
        StorageUnLock;
    end;
  end;
end;

procedure TSQLRestStorageExternal.InternalBatchStop;
var i,j,n,max,BatchBegin,BatchEnd,ValuesMax: integer;
    Query: ISQLDBStatement;
    NotifySQLEvent: TSQLEvent;
    SQL: RawUTF8;
    P: PUTF8Char;
    Fields, ExternalFields: TRawUTF8DynArray;
    Types: TSQLDBFieldTypeArray;
................................................................................
begin
  if fBatchMethod=mNone then
    raise EORMException.CreateFmt('%s.BatchMethod=mNone',[fStoredClassRecordProps.SQLTableName]);
  try
    if fBatchCount>0 then begin
      if (Owner<>nil) and (fBatchMethod=mDelete) then // notify BEFORE deletion
        for i := 0 to fBatchCount-1 do

          Owner.InternalUpdateEvent(seDelete,fStoredClass,fBatchIDs[i],nil);
      with fProperties do
        if BatchMaxSentAtOnce>0 then
          max := BatchMaxSentAtOnce else
          max := 1000;
      BatchBegin := 0;
      BatchEnd := fBatchCount-1;
      repeat
................................................................................
        BatchBegin := BatchEnd+1;
      until BatchBegin>=fBatchCount;
      if (Owner<>nil) and (fBatchMethod in [mPost,mPut]) then begin
        if fBatchMethod=mPost then
          NotifySQLEvent := seAdd else
          NotifySQLEvent := seUpdate;
          for i := 0 to fBatchCount-1 do

            Owner.InternalUpdateEvent(NotifySQLEvent,fStoredClass,fBatchIDs[i],nil);
      end;
    end;
  finally
    if fBatchMethod=mPost then
      fEngineLockedLastID := fBatchAddedID+fBatchCount;
    SetLength(fBatchValues,0);
    SetLength(fBatchIDs,0);
    fBatchCount := 0;
    fBatchCapacity := 0;
    fBatchMethod := mNone;
    StorageUnLock;
  end;
end;

function TSQLRestStorageExternal.InternalBatchAdd(
  const aValue: RawUTF8; aID: integer): integer;
begin
  result := fBatchAddedID+fBatchCount;
  if fBatchCount>=fBatchCapacity then begin
    fBatchCapacity := fBatchCapacity+64+fBatchCount shr 3;
    SetLength(fBatchIDs,fBatchCapacity);
    if aValue<>'' then
................................................................................
    fBatchValues[fBatchCount] := aValue;
  if aID=0 then
    aID := result;
  fBatchIDs[fBatchCount] := aID;
  inc(fBatchCount);
end;

function TSQLRestStorageExternal.EngineAdd(TableModelIndex: integer;
  const SentData: RawUTF8): integer;
begin
  if (TableModelIndex<0) or (fModel.Tables[TableModelIndex]<>fStoredClass) then
    result := 0 else // avoid GPF
  if fBatchMethod<>mNone then
    if fBatchMethod<>mPOST then
      result := 0 else
      result := InternalBatchAdd(SentData,0) else begin
    result := ExecuteFromJSON(SentData,soInsert,0); // UpdatedID=0 -> insert with EngineLockedNextID
    if (result>0) and (Owner<>nil) then

      Owner.InternalUpdateEvent(seAdd,fStoredClass,result,nil);
  end;
end;

function TSQLRestStorageExternal.EngineUpdate(TableModelIndex, ID: integer;
  const SentData: RawUTF8): boolean;
begin
  if (ID<=0) or (TableModelIndex<0) or (Model.Tables[TableModelIndex]<>fStoredClass) then
    result := false else
    if fBatchMethod<>mNone then
      if fBatchMethod<>mPUT then
        result := false else
        result := InternalBatchAdd(SentData,ID)>=0 else begin
      result := ExecuteFromJSON(SentData,soUpdate,ID)=ID;
      if result and (Owner<>nil) then

        Owner.InternalUpdateEvent(seUpdate,fStoredClass,ID,nil);
    end;
end;


function TSQLRestStorageExternal.EngineDelete(TableModelIndex, ID: integer): boolean;
begin

  if (ID<=0) or (TableModelIndex<0) or (Model.Tables[TableModelIndex]<>fStoredClass) then
    result := false else
    if fBatchMethod<>mNone then
      if fBatchMethod<>mDELETE then
        result := false else
        result := InternalBatchAdd('',ID)>=0 else begin
      result := ExecuteDirect('delete from % where %=?',
        [fTableName,StoredClassProps.ExternalDB.RowIDFieldName],[ID],false)<>nil;
      if result and (Owner<>nil) then

        Owner.InternalUpdateEvent(seDelete,fStoredClass,ID,nil);
    end;
end;

function TSQLRestStorageExternal.EngineDeleteWhere(TableModelIndex: integer;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
var i: integer;
begin
  result := false;
  if (IDs=nil) or (SQLWhere='') or
     (TableModelIndex<0) or (Model.Tables[TableModelIndex]<>fStoredClass) then
    exit;
  if fBatchMethod<>mNone then
    if fBatchMethod<>mDELETE then
      exit else
      for i := 0 to high(IDs) do
        InternalBatchAdd('',IDs[i]) else begin
    if Owner<>nil then // notify BEFORE deletion
      for i := 0 to high(IDs) do

        Owner.InternalUpdateEvent(seDelete,fStoredClass,IDs[i],nil);
    if ExecuteInlined('delete from % where %',[fTableName,SQLWhere],false)=nil then
      exit;
  end;
  result := true;
end;







function TSQLRestStorageExternal.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean; ReturnedRowCount: PPtrInt): RawUTF8;
var Stmt: ISQLDBStatement;
begin
  if ReturnedRowCount<>nil then
    raise ESQLDBException.CreateFmt('%s.EngineList(ReturnedRowCount<>nil)',[ClassName]);
  Stmt := PrepareInlinedForRows(SQL);
  if Stmt=nil then
    result := '' else
    Stmt.ExecutePreparedAndFetchAllAsJSON(
      ForceAJAX or (Owner=nil) or (not Owner.NoAJAXJSON),result);
end;

function TSQLRestStorageExternal.EngineRetrieve(TableModelIndex, ID: integer): RawUTF8;
var Stmt: ISQLDBStatement;
begin // TableModelIndex is not usefull here
  result := '';
  if (self=nil) or (ID<=0) then
    exit;
  Stmt := PrepareDirectForRows(pointer(fSelectOneDirectSQL),[],[ID]);
  if Stmt<>nil then begin
................................................................................
      // '{"fieldCount":2,"values":["ID","FirstName"]}'#$A -> ID not found
      result := '' else
      // list '[{...}]'#10 -> object '{...}'
      result := copy(result,2,length(result)-3);
  end;
end;

function TSQLRestStorageExternal.TableHasRows(Table: TSQLRecordClass): boolean;
var Rows: ISQLDBRows;
begin
  if (self=nil) or (Table<>fStoredClass) then
    result := false else begin
    Rows := ExecuteDirect(pointer(fSelectTableHasRowsSQL),[],[],true);
    if Rows=nil then
      result := false else
      result := Rows.Step;
  end;
end;

function TSQLRestStorageExternal.TableRowCount(Table: TSQLRecordClass): integer;
var Rows: ISQLDBRows;
begin
  if (self=nil) or (Table<>fStoredClass) then
    result := 0 else begin
    Rows := ExecuteDirect('select count(*) from %',[fTableName],[],true);
    if (Rows=nil) or not Rows.Step then
      result := 0 else
      result := Rows.ColumnInt(0);
  end;
end;

function TSQLRestStorageExternal.EngineRetrieveBlob(TableModelIndex, aID: integer;

  BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
var Rows: ISQLDBRows;
begin
  result := false;
  if (aID<=0) or (not BlobField^.IsBlob) or
     (TableModelIndex<0) or (Model.Tables[TableModelIndex]<>fStoredClass) then
    exit;
  with StoredClassProps.ExternalDB do
    Rows := ExecuteDirect('select % from % where %=?',
      [InternalToExternal(BlobField^.Name),fTableName,RowIDFieldName],[aID],true);
  if (Rows<>nil) and Rows.Step then
  try
    BlobData := Rows.ColumnBlob(0);
................................................................................
    Rows := nil;
  except
    on Exception do
      result := false;
  end;
end;

function TSQLRestStorageExternal.RetrieveBlobFields(Value: TSQLRecord): boolean;
var Rows: ISQLDBRows;
    f: Integer;
    data: TSQLVar;
    temp: RawByteString;
begin
  result := false;
  if (Value<>nil) and (Value.ID>0) and (PSQLRecordClass(Value)^=fStoredClass) then
................................................................................
    except
      on Exception do
        result := false;
    end;
  end;
end;

function TSQLRestStorageExternal.EngineUpdateField(TableModelIndex: integer;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
begin
  if (TableModelIndex<0) or (Model.Tables[TableModelIndex]<>fStoredClass) then
    result := false else
    with StoredClassProps.ExternalDB do
      result := ExecuteInlined('update % set %=:(%): where %=:(%):',
        [fTableName,InternalToExternal(SetFieldName),SetValue,
         InternalToExternal(WhereFieldName),WhereValue],false)<>nil;
end;

function TSQLRestStorageExternal.EngineUpdateBlob(TableModelIndex, aID: integer;

  BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean;
var Statement: ISQLDBStatement;
    AffectedField: TSQLFieldBits;
begin
  result := false;
  if (aID<=0) or (not BlobField^.IsBlob) or
     (TableModelIndex<0) or (Model.Tables[TableModelIndex]<>fStoredClass) then
    exit;
  try
    if Owner<>nil then
      Owner.FlushInternalDBCache;
    with StoredClassProps.ExternalDB do
      Statement := fProperties.NewThreadSafeStatementPrepared(
        'update % set %=? where %=?',
        [fTableName,InternalToExternal(BlobField^.Name),RowIDFieldName],false);
................................................................................
      if BlobData='' then
        Statement.BindNull(1) else
        Statement.BindBlob(1,BlobData); // fast explicit BindBlob() call
      Statement.Bind(2,aID);
      Statement.ExecutePrepared;
      if Owner<>nil then begin
        fStoredClassRecordProps.FieldIndexsFromBlobField(BlobField,AffectedField);

        Owner.InternalUpdateEvent(seUpdateBlob,fStoredClass,aID,@AffectedField);
      end;
      result := true; // success
    end;
  except
    on Exception do
      result := false;
  end;
end;

function TSQLRestStorageExternal.UpdateBlobFields(Value: TSQLRecord): boolean;
var f, aID: integer;
    temp: array of RawByteString;
    Params: TSQLVarDynArray;
begin
  result := false;
  if (Value<>nil) and (PSQLRecordClass(Value)^=fStoredClass) then
  with Value.RecordProps do
................................................................................
    SetLength(temp,length(BlobFields));
    for f := 0 to high(Params) do
      BlobFields[f].GetFieldSQLVar(Value,Params[f],temp[f]);
    result := ExecuteDirectSQLVar('update % set % where %=?',
      [fTableName,fUpdateBlobFieldsSQL,StoredClassProps.ExternalDB.RowIDFieldName],
      Params,aID,false);
    if result and (Owner<>nil) then

      Owner.InternalUpdateEvent(seUpdateBlob,fStoredClass,aID,
          @fStoredClassRecordProps.BlobFieldsBits);
  end else
    result := true; // as TSQLRest.UpdateblobFields()
end;

function TSQLRestStorageExternal.PrepareInlinedForRows(const aSQL: RawUTF8): ISQLDBStatement;
begin
  result := nil; // returns nil interface on error
  if self=nil then
    exit;
  try
    result := fProperties.PrepareInlined(aSQL,true);
  except
    on Exception do
      result := nil;
  end;
end;

function TSQLRestStorageExternal.ExecuteInlined(const aSQL: RawUTF8;
  ExpectResults: Boolean): ISQLDBRows;
begin
  result := nil; // returns nil interface on error
  if self=nil then
    exit;
  if (not ExpectResults) and (Owner<>nil) then
    Owner.FlushInternalDBCache; // add/update/delete should flush DB cache
................................................................................
    result := fProperties.ExecuteInlined(aSQL,ExpectResults);
  except
    on Exception do
      result := nil;
  end;
end;

function TSQLRestStorageExternal.ExecuteInlined(SQLFormat: PUTF8Char;
  const Args: array of const; ExpectResults: Boolean): ISQLDBRows;
begin
  result := ExecuteInlined(FormatUTF8(SQLFormat,Args),ExpectResults);
end;

function TSQLRestStorageExternal.PrepareDirectForRows(SQLFormat: PUTF8Char;
  const Args, Params: array of const): ISQLDBStatement;
var Query: ISQLDBStatement;
begin
  result := nil;
  if self=nil then
    exit;
  Query := fProperties.NewThreadSafeStatementPrepared(SQLFormat,Args,true);
................................................................................
    result := Query;
  except
    on Exception do
      result := nil;
  end;
end;

function TSQLRestStorageExternal.ExecuteDirect(SQLFormat: PUTF8Char;
  const Args, Params: array of const; ExpectResults: Boolean): ISQLDBRows;
var Query: ISQLDBStatement;
begin
  result := nil;
  if self=nil then
    exit;
  if (not ExpectResults) and (Owner<>nil) then
................................................................................
    result := Query;
  except
    on Exception do
      result := nil;
  end;
end;

function TSQLRestStorageExternal.ExecuteDirectSQLVar(SQLFormat: PUTF8Char;
  const Args: array of const; var Params: TSQLVarDynArray; LastIntegerParam: integer;
  ParamsMatchCopiableFields: boolean): boolean;
var Query: ISQLDBStatement;
    ParamsCount, f: integer;
begin
  result := false;
  if Self<>nil then
................................................................................
    result := true; // success
  except
    on Exception do
      result := false;
  end;
end;

procedure TSQLRestStorageExternal.RollBack(SessionID: cardinal);
begin
  inherited RollBack(SessionID); // reset fTransactionActive
  try
    fProperties.ThreadSafeConnection.Rollback;
  except
    on Exception do
      ; // just catch exception
  end;
end;

function TSQLRestStorageExternal.EngineSearchField(
  const FieldName: ShortString; const FieldValue: array of const;
  var ResultID: TIntegerDynArray): boolean;
var n: Integer;
    Rows: ISQLDBRows;
begin
  n := 0;
  Rows := ExecuteDirect('select % from % where %=?',
................................................................................
  if Rows<>nil then
    while Rows.Step do
      AddInteger(ResultID,n,Rows.ColumnInt(0));
  SetLength(ResultID,n);
  result := n>0;
end;

function TSQLRestStorageExternal.SearchField(const FieldName: RawUTF8;
  FieldValue: Integer; var ResultID: TIntegerDynArray): boolean;
begin
  result := EngineSearchField(FieldName,[FieldValue],ResultID);
end;

function TSQLRestStorageExternal.SearchField(const FieldName, FieldValue: RawUTF8;
  var ResultID: TIntegerDynArray): boolean;
begin
  result := EngineSearchField(FieldName,[FieldValue],ResultID);
end;

function TSQLRestStorageExternal.TransactionBegin(
  aTable: TSQLRecordClass; SessionID: cardinal): boolean;
begin
  result := false;
  if (aTable=fStoredClass) and inherited TransactionBegin(aTable,SessionID) then
  try
    fProperties.ThreadSafeConnection.StartTransaction;
    result := true; // success
  except
    on Exception do
      result := false;
  end;
end;

function TSQLRestStorageExternal.CreateSQLMultiIndex(
  Table: TSQLRecordClass; const FieldNames: array of RawUTF8;
  Unique: boolean; IndexName: RawUTF8): boolean;
var SQL: RawUTF8;
    ExtFieldNames: TRawUTF8DynArray;
    Descending: boolean;
    i: integer;
begin
................................................................................
  end;
  StoredClassProps.ExternalDB.InternalToExternalDynArray(FieldNames,ExtFieldNames);
  SQL := fProperties.SQLAddIndex(fTableName,ExtFieldNames,Unique,Descending,IndexName);
  if (SQL<>'') and (ExecuteDirect(pointer(SQL),[],[],false)<>nil) then
    result := true;
end;

class function TSQLRestStorageExternal.Instance(
  aClass: TSQLRecordClass; aServer: TSQLRestServer): TSQLRestStorageExternal;
begin
  if (aClass=nil) or (aServer=nil) then
    result := nil else begin
    result := TSQLRestStorageExternal(aServer.StaticVirtualTable[aClass]);
    if result<>nil then
      if not result.InheritsFrom(TSQLRestStorageExternal) then
        result := nil;
  end;
end;

class function TSQLRestStorageExternal.ConnectionProperties(
  aClass: TSQLRecordClass; aServer: TSQLRestServer): TSQLDBConnectionProperties;
begin
  result := Instance(aClass,aServer).ConnectionProperties;
end;

function TSQLRestStorageExternal.ConnectionProperties: TSQLDBConnectionProperties;
begin
  if self=nil then
    result := nil else
    result := fProperties;
end;
  
function TSQLRestStorageExternal.ExecuteFromJSON(
  const SentData: RawUTF8; Occasion: TSQLOccasion; UpdatedID: integer): integer;
var Decoder: TJSONObjectDecoder;
    SQL: RawUTF8;
    Types: TSQLDBFieldTypeArray;
    ExternalFields: TRawUTF8DynArray;
    InsertedID, F: integer;
    Query: ISQLDBStatement;
begin
  result := 0;
  StorageLock(false); // avoid race condition against max(ID)
  try
    case Occasion of
    soInsert: begin
      InsertedID := JSONRetrieveIDField(pointer(SentData));
      if InsertedID=0 then // no specified "ID":... field value -> compute next
        InsertedID := EngineLockedNextID else
        if result>fEngineLockedLastID then
................................................................................
      exit; // leave result=0
    end;
    // mark success
    if UpdatedID=0 then
      result := InsertedID else
      result := UpdatedID;
  finally
    StorageUnLock;
  end;
end;

procedure TSQLRestStorageExternal.EndCurrentThread(Sender: TThread);
begin
  if fProperties.InheritsFrom(TSQLDBConnectionPropertiesThreadSafe) then
    TSQLDBConnectionPropertiesThreadSafe(fProperties).EndCurrentThread;
end;

function TSQLRestStorageExternal.InternalFieldNameToFieldExternalIndex(
  const InternalFieldName: RawUTF8): integer;
begin
  result := StoredClassRecordProps.Fields.IndexByNameOrExcept(InternalFieldName);
  result := IntegerScanIndex(Pointer(fFieldsExternalToInternal),
    length(fFieldsExternalToInternal),result);
end;

function TSQLRestStorageExternal.JSONDecodedPrepareToSQL(
  var Decoder: TJSONObjectDecoder; out ExternalFields: TRawUTF8DynArray;
  out Types: TSQLDBFieldTypeArray; Occasion: TSQLOccasion): RawUTF8;
var f,k: Integer;
begin
  SetLength(ExternalFields,Decoder.FieldCount);
  for f := 0 to Decoder.FieldCount-1 do begin
    k := InternalFieldNameToFieldExternalIndex(Decoder.FieldNames[f]);
................................................................................
    StoredClassProps.ExternalDB.RowIDFieldName);
  if Occasion=soUpdate then begin
    Types[Decoder.FieldCount] := ftInt64; // add "where ID=?" parameter
    inc(Decoder.FieldCount);
  end;
end;

procedure TSQLRestStorageExternal.ResetMaxIDCache;
begin
  StorageLock(true);
  fEngineLockedLastID := 0;
  StorageUnLock;
end;


{ TSQLVirtualTableCursorExternal }

function TSQLVirtualTableCursorExternal.Column(aColumn: integer;
  var aResult: TSQLVar): boolean;
................................................................................
function TSQLVirtualTableCursorExternal.Search(
  const Prepared: TSQLVirtualTablePrepared): boolean;
var i: integer;
begin
  result := false;
  if (Self=nil) or (Table=nil) or (Table.Static=nil) then
    exit;
  with Table.Static as TSQLRestStorageExternal do begin
    if fSQL='' then begin
      // compute the SQL query corresponding to this prepared request
      fSQL := fSelectAllDirectSQL;
      if Prepared.WhereCount<>0 then begin
        for i := 0 to Prepared.WhereCount-1 do
        with Prepared.Where[i] do begin
          if Operation>high(SQL_OPER_WITH_PARAM) then
................................................................................


{ TSQLVirtualTableExternal }

function TSQLVirtualTableExternal.Delete(aRowID: Int64): boolean;
begin
  result := (self<>nil) and (Static<>nil) and
    (Static as TSQLRestStorageExternal).EngineDelete(StaticTableIndex,aRowID);
end;

function TSQLVirtualTableExternal.Drop: boolean;
begin
  if (self=nil) or (Static=nil) then
    result := false else
    with Static as TSQLRestStorageExternal do
      result := ExecuteDirect('drop table %',[fTableName],[],false)<>nil;
end;

class procedure TSQLVirtualTableExternal.GetTableModuleProperties(
  var aProperties: TVirtualTableModuleProperties);
begin
  aProperties.Features := [vtWrite];
  aProperties.CursorClass := TSQLVirtualTableCursorExternal;
  aProperties.StaticClass := TSQLRestStorageExternal;
end;

function TSQLVirtualTableExternal.Insert(aRowID: Int64;
  var Values: TSQLVarDynArray; out insertedRowID: Int64): boolean;
begin // aRowID is just ignored here since IDs are always auto calculated
  result := false;
  if (self<>nil) and (Static<>nil) then
  with Static as TSQLRestStorageExternal do begin
    StorageLock(false); // to avoid race condition against max(RowID)
    try
      insertedRowID := EngineLockedNextID;
      with StoredClassProps.ExternalDB do
        result := ExecuteDirectSQLVar('insert into % (%,%) values (%,?)',
          [fTableName,SQL.InsertSet,RowIDFieldName,CSVOfValue('?',length(Values))],
          Values,insertedRowID,true);
    finally
      StorageUnLock;
    end;
  end;
end;

function TSQLVirtualTableExternal.Prepare(var Prepared: TSQLVirtualTablePrepared): boolean;
var i, col: integer;
    hasIndex: boolean;
    Fields: TSQLPropInfoList;
begin
  result := inherited Prepare(Prepared); // Prepared.EstimatedCost := 1E10;
  if result and (Static<>nil) then
  with Static as TSQLRestStorageExternal do begin
    // mark Where[] clauses will be handled by SQL
    Fields := StoredClassRecordProps.Fields;
    result := false;
    for i := 0 to Prepared.WhereCount-1 do
      with Prepared.Where[i] do
      if (Column<>VIRTUAL_TABLE_IGNORE_COLUMN) and
         (Operation<=high(SQL_OPER_WITH_PARAM)) then begin
................................................................................
end;

function TSQLVirtualTableExternal.Update(oldRowID, newRowID: Int64;
  var Values: TSQLVarDynArray): boolean;
begin
  if (self<>nil) and (Static<>nil) and
     (oldRowID=newRowID) and (newRowID>0) then // don't allow ID change
    with Static as TSQLRestStorageExternal, StoredClassProps.ExternalDB do
      result := ExecuteDirectSQLVar('update % set % where %=?',
        [fTableName,SQL.UpdateSetAll,RowIDFieldName],Values,oldRowID,true) else
    result := false;
end;


initialization
  // all our SynDB related functions shall log to main TSQLLog
  SynDBLog := TSQLLog;
end.

Changes to SQLite3/mORMotMongoDB.pas.

78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
...
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
...
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
...
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
...
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
...
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
...
381
382
383
384
385
386
387


388
389
390

391
392
393
394

395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419

420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442

443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
...
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502

503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
...
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587

588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
...
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
...
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
...
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
...
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
...
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
...
849
850
851
852
853
854
855
856
857
858
859
860
  /// exeception class raised by this units
  EORMMongoDBException = class(EORMException);

  /// REST server with direct access to a MongoDB external database
  // - handle all REST commands via direct SynMongoDB call
  // - is used by TSQLRestServer.URI for faster RESTful direct access
  // - JOINed SQL statements are not handled yet
  TSQLRestServerStaticMongoDB = class(TSQLRestServerStatic)
  protected
    /// the associated MongoDB collection
    fCollection: TMongoCollection;
    fEngineLastID: integer;
    fBSONProjectionSimpleFields: variant;
    fBSONProjectionBlobFields: variant;
    fBSONProjectionBlobFieldsNames: TRawUTF8DynArray;
................................................................................
    function BSONProjectionSet(var Projection: variant; WithID: boolean;
      const Fields: TSQLFieldBits; ExtFieldNames: PRawUTF8DynArray): integer;
    function GetJSONValues(const Res: TBSONDocument;
      const extFieldNames: TRawUTF8DynArray; W: TJSONSerializer): integer;
    // overridden methods calling the MongoDB external server
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    function EngineAdd(Table: TSQLRecordClass; const SentData: RawUTF8): integer; override;
    function EngineUpdate(Table: TSQLRecordClass; ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDeleteWhere(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    /// TSQLRestServer.URI use it for Static.EngineList to by-pass virtual table
    // - overridden method to handle most potential simple queries, e.g. like
    // $ SELECT Field1,RowID FROM table WHERE RowID=... AND/OR/NOT Field2=
    // - ORM field names into mapped MongoDB external field names
    // - handle statements to avoid slow virtual table loop over all rows, like
    // $ SELECT count(*) FROM table
    function AdaptSQLForEngineList(var SQL: RawUTF8): boolean; override;
    // BLOBs should be access directly, not through slower JSON Base64 encoding
    function EngineRetrieveBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    // overriden method returning TRUE for next calls to EngineAdd/Update/Delete
    // will properly handle operations until InternalBatchStop is called
    function InternalBatchStart(Method: TSQLURIMethod): boolean; override;
    // internal method called by TSQLRestServer.RunBatch() to process fast
    // BULK sending to remote MongoDB database
    procedure InternalBatchStop; override;
................................................................................
    /// get the row count of a specified table
    // - return -1 on error
    // - return the row count of the table on success
    function TableRowCount(Table: TSQLRecordClass): integer; override;
    /// check if there is some data rows in a specified table
    function TableHasRows(Table: TSQLRecordClass): boolean; override;
    /// delete a row, calling the current MongoDB server
    // - made public since a TSQLRestServerStatic instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function EngineDelete(Table: TSQLRecordClass; ID: integer): boolean; override;
    /// create one index for all specific FieldNames at once
    function CreateSQLMultiIndex(Table: TSQLRecordClass; const FieldNames: array of RawUTF8;
      Unique: boolean; IndexName: RawUTF8=''): boolean; override;

    /// drop the whole table content
    // - but you can still add items to it - whereas Collection.Drop would
    // trigger GPF issues
................................................................................
// you can customize it with the corresponding parameter
// - the TSQLRecord.ID (RowID) field is always mapped to MongoDB's _id field
// - after registration, you can tune the field-name mapping by calling
// ! aModel.Props[aClass].ExternalDB.MapField(..)
// (just a regular external DB as defined in mORMotDB.pas unit) - it may be
// a good idea to use short field names on MongoDB side, to reduce the space
// used for storage (since they will be embedded within the document data)
// - it will return the corresponding TSQLRestServerStaticMongoDB instance -
// you can access later to it and its associated collection e.g. via:
// ! (aServer.StaticDataServer[TSQLMyTable] as TSQLRestServerStaticMongoDB)
function StaticMongoDBRegister(aClass: TSQLRecordClass; aServer: TSQLRestServer;
  aMongoDatabase: TMongoDatabase; aMongoCollectionName: RawUTF8=''): TSQLRestServerStaticMongoDB;


implementation

function StaticMongoDBRegister(aClass: TSQLRecordClass; aServer: TSQLRestServer;
  aMongoDatabase: TMongoDatabase; aMongoCollectionName: RawUTF8=''): TSQLRestServerStaticMongoDB;
var Props: TSQLModelRecordProperties;
begin
  result := nil;
  if (aServer=nil) or (aClass=nil) or (aMongoDatabase=nil) then
    exit; // avoid GPF
  Props := aServer.Model.Props[aClass];
  if Props=nil then
    exit; // if aClass is not part of the model
  if aMongoCollectionName='' then
    aMongoCollectionName := Props.Props.SQLTableName;
  Props.ExternalDB.Init(Props,aMongoCollectionName,
    aMongoDatabase.CollectionOrCreate[aMongoCollectionName]);
  Props.ExternalDB.MapField('ID','_id');
  result := (aServer.StaticDataCreate(aClass,'',false,TSQLRestServerStaticMongoDB)
    as TSQLRestServerStaticMongoDB);
end;


{ TSQLRestServerStaticMongoDB }

constructor TSQLRestServerStaticMongoDB.Create(aClass: TSQLRecordClass;
  aServer: TSQLRestServer; const aFileName: TFileName;
  aBinaryFile: boolean);
var F: integer;
begin
  inherited;
  if fStoredClassProps=nil then
    raise EORMMongoDBException.CreateFmt(
................................................................................
    for F := 0 to fStoredClassRecordProps.Fields.Count do
      if F in fIsUnique then
        fCollection.EnsureIndex(
          [fStoredClassProps.ExternalDB.FieldNames[f]],true,true);

end;

function TSQLRestServerStaticMongoDB.BSONProjectionSet(var Projection: variant;
  WithID: boolean; const Fields: TSQLFieldBits; ExtFieldNames: PRawUTF8DynArray): integer;
var i,n: integer;
    Start: cardinal;
    W: TBSONWriter;
begin
  W := TBSONWriter.Create(TRawByteStringStream);
  try
................................................................................
        end;
    end;
  finally
    W.Free;
  end;
end;

function TSQLRestServerStaticMongoDB.CreateSQLMultiIndex(
  Table: TSQLRecordClass; const FieldNames: array of RawUTF8;
  Unique: boolean; IndexName: RawUTF8): boolean;
begin
  result := false;
  if (self=nil) or (fCollection=nil) or (Table<>fStoredClass) then
    exit;
  fCollection.EnsureIndex(FieldNames,true,Unique);
end;

procedure TSQLRestServerStaticMongoDB.Drop;
var DB: TMongoDatabase;
    CollName: RawUTF8;
begin
  DB := Collection.Database;
  CollName := Collection.Name;
  Collection.Drop;
  fCollection := DB.CollectionOrCreate[CollName];
  fEngineLastID := 0;
end;

destructor TSQLRestServerStaticMongoDB.Destroy;
begin
  inherited;
  FreeAndNil(fBatchWriter);
end;

function TSQLRestServerStaticMongoDB.TableHasRows(
  Table: TSQLRecordClass): boolean;
begin
  result := TableRowCount(Table)>0;
end;

function TSQLRestServerStaticMongoDB.TableRowCount(
  Table: TSQLRecordClass): integer;
begin
  if (fCollection=nil) or (Table<>fStoredClass) then
    result := 0 else
    result := fCollection.Count;
end;

function TSQLRestServerStaticMongoDB.EngineNextID: Integer;
procedure ComputeMax_ID;
var res: variant;
begin
  res := fCollection.AggregateDoc('{$group:{_id:null,max:{$max:"$_id"}}}',[]);
  if DocVariantType.IsOfType(res) then
    fEngineLastID := VariantToIntegerDef(res.max,0);
end;
begin
  if fEngineLastID=0 then
    ComputeMax_ID;
  result := InterlockedIncrement(fEngineLastID);
end;

function TSQLRestServerStaticMongoDB.DocFromJSON(const JSON: RawUTF8;
  Occasion: TSQLOccasion; var Doc: TDocVariantData): integer;
var i, ndx: integer;
    blob: RawByteString;
    info: TSQLPropInfo;
    typenfo: pointer;
    js: RawUTF8;
    MissingID: boolean;
    V: PVarData;
begin
  doc.InitJSON(JSON,[dvoValueCopiedByReference]);
  if doc.Kind<>dvObject then
    raise EORMMongoDBException.Create('Invalid JSON context');
  if not (Occasion in [soInsert,soUpdate]) then
    raise EORMMongoDBException.CreateFmt('DocFromJSON(%s)',[
      GetEnumName(TypeInfo(TSQLOccasion),ord(Occasion))^]);
  MissingID := true;
  for i := doc.Count-1 downto 0 do // downwards for doc.Delete(i) below
    if IsRowID(pointer(doc.Names[i])) then begin
................................................................................
        // sftObject,sftVariant were already converted to object from JSON
      end;
    end;
  if (Occasion=soInsert) and MissingID then begin
    result := EngineNextID;
    doc.AddValue(fStoredClassProps.ExternalDB.RowIDFieldName,result);
  end;


end;

function TSQLRestServerStaticMongoDB.EngineAdd(Table: TSQLRecordClass;

  const SentData: RawUTF8): integer;
var doc: TDocVariantData;
begin
  if (fCollection=nil) or (Table<>fStoredClass) then

    result := 0 else
    try
      result := DocFromJSON(SentData,soInsert,Doc);
      if fBatchMethod<>mNone then
        if (fBatchMethod<>mPOST) or (fBatchWriter=nil) then
          result := 0 else begin
          inc(fBatchCount);
          fBatchWriter.BSONWriteDoc(doc);
        end else begin
        fCollection.Insert([variant(doc)]);
        if Owner<>nil then
          TSQLRestServerStaticMongoDB(Owner). // to access protected method
            InternalUpdateEvent(seAdd,fStoredClass,result,nil);
      end;
    except
      result := 0;
    end;
end;

function TSQLRestServerStaticMongoDB.EngineUpdate(Table: TSQLRecordClass;
  ID: integer; const SentData: RawUTF8): boolean;
var doc: TDocVariantData;
    query,update: variant; // use explicit TBSONVariant for type safety
begin
  if (fCollection=nil) or (Table<>fStoredClass) or (ID<=0) then

    result := false else
    try
      DocFromJSON(SentData,soUpdate,Doc);
      query := BSONVariant(['_id',ID]);
      update := BSONVariant(['$set',variant(Doc)]);
      fCollection.Update(query,update);
      if Owner<>nil then
        TSQLRestServerStaticMongoDB(Owner).  // to access protected method
          InternalUpdateEvent(seUpdate,fStoredClass,ID,nil);
      result := true;
    except
      result := false;
    end;
end;

function TSQLRestServerStaticMongoDB.EngineUpdateBlob(
  Table: TSQLRecordClass; aID: integer; BlobField: PPropInfo;
  const BlobData: TSQLRawBlob): boolean;
var query,update,blob: variant; // use explicit TBSONVariant for type safety
    FieldName: RawUTF8;
    AffectedField: TSQLFieldBits;
begin
  if (fCollection=nil) or (Table<>fStoredClass) or (aID<=0) or (BlobField=nil) then

    result := false else
    try
      query := BSONVariant(['_id',aID]);
      FieldName := fStoredClassProps.ExternalDB.InternalToExternal(BlobField^.Name);
      BSONVariantType.FromBinary(BlobData,bbtGeneric,blob);
      update := BSONVariant(['$set',BSONVariant([FieldName,blob])]);
      fCollection.Update(query,update);
      if Owner<>nil then begin
        fStoredClassRecordProps.FieldIndexsFromBlobField(BlobField,AffectedField);
        TSQLRestServerStaticMongoDB(Owner).  // to access protected method
          InternalUpdateEvent(seUpdateBlob,fStoredClass,aID,@AffectedField);
      end;
      result := true;
    except
      result := false;
    end;
end;

function TSQLRestServerStaticMongoDB.UpdateBlobFields(
  Value: TSQLRecord): boolean;
var query,blob: variant;
    update: TDocVariantData;
    info: TSQLPropInfo;
    blobRaw: RawByteString;
    aID, f: integer;
begin
................................................................................
      update.AddValue(fStoredClassProps.ExternalDB.FieldNames[f],blob);
    end;
  end;
  if update.Count>0 then
    try
      fCollection.Update(query,BSONVariant(['$set',variant(update)]));
      if Owner<>nil then
        TSQLRestServerStaticMongoDB(Owner).  // to access protected method
          InternalUpdateEvent(seUpdateBlob,fStoredClass,aID,
            @fStoredClassRecordProps.BlobFieldsBits);
      result := true;
    except
      result := false;
    end;
end;

function TSQLRestServerStaticMongoDB.EngineDelete(Table: TSQLRecordClass;
  ID: integer): boolean;
begin
  result := false;
  if (fCollection<>nil) and (Table=fStoredClass) and (ID>0) then

  try
    if fBatchMethod<>mNone then
      if fBatchMethod<>mDelete then
        exit else
        AddInteger(fBatchIDs,fBatchCount,ID) else begin
      fCollection.RemoveOne(ID);
      if Owner<>nil then
        TSQLRestServerStaticMongoDB(Owner).  // to access protected method
          InternalUpdateEvent(seDelete,fStoredClass,ID,nil);
    end;
    result := true;
  except
    result := false;
  end;
end;

function TSQLRestServerStaticMongoDB.EngineDeleteWhere(
  Table: TSQLRecordClass; const SQLWhere: RawUTF8;
  const IDs: TIntegerDynArray): boolean;
var i: integer;
begin // here we use the pre-computed IDs[]
  if (fCollection=nil) or (Table<>fStoredClass) or (IDs=nil) then
    result := false else
    try
      if Owner<>nil then // notify BEFORE deletion
      for i := 0 to high(IDs) do
        TSQLRestServerStaticMongoDB(Owner).  // to access protected method
          InternalUpdateEvent(seDelete,fStoredClass,IDs[i],nil);
      fCollection.Remove(BSONVariant(
        ['_id',BSONVariant(['$in',BSONVariantFromIntegers(IDs)])]));
      result := true;
    except
      result := false;
    end;
end;

procedure TSQLRestServerStaticMongoDB.JSONFromDoc(var doc: TDocVariantData;
  var result: RawUTF8);
var i: integer;
    name: RawUTF8;
    W: TTextWriter;
begin
  if (doc.VarType<>DocVariantType.VarType) or (doc.Kind<>dvObject) or (doc.Count=0) then begin
    result := '';
................................................................................
    W.Add('}');
    W.SetText(result);
  finally
    W.Free;
  end;
end;

function TSQLRestServerStaticMongoDB.EngineRetrieve(TableModelIndex,
  ID: integer): RawUTF8;
var doc: variant;
begin
  result := '';
  if (fCollection=nil) or (ID<=0) then
    exit;
  doc := fCollection.FindDoc(BSONVariant(['_id',ID]),fBSONProjectionSimpleFields,1);
  JSONFromDoc(TDocVariantData(doc),result);
end;

function TSQLRestServerStaticMongoDB.EngineRetrieveBlob(
  Table: TSQLRecordClass; aID: integer; BlobField: PPropInfo;
  out BlobData: TSQLRawBlob): boolean;
var doc: variant;
    data: TVarData;
    FieldName: RawUTF8;
begin
  if (fCollection=nil) or (Table<>fStoredClass) or (aID<=0) or (BlobField=nil) then

    result := false else
    try
      FieldName := fStoredClassProps.ExternalDB.InternalToExternal(BlobField^.Name);
      doc := fCollection.FindDoc(BSONVariant(['_id',aID]),BSONVariant([FieldName,1]),1);
      if DocVariantType.IsOfType(doc) and
         DocVariantData(doc)^.GetVarData(FieldName,data) then
        BSONVariantType.ToBlob(variant(data),RawByteString(BlobData));
      result := true;
    except
      result := false;
    end;
end;

function TSQLRestServerStaticMongoDB.RetrieveBlobFields(
  Value: TSQLRecord): boolean;
var aID, f: Integer;
    doc: variant;
    docv: PDocVariantData;
    blob: TVarData;
    blobRaw: RawByteString;
begin
................................................................................
    end;
    result := true;
  except
    result := false;
  end;
end;

function TSQLRestServerStaticMongoDB.AdaptSQLForEngineList(
  var SQL: RawUTF8): boolean;
begin
  result := true; // we do not have any Virtual Table yet -> always accept
end;

function TSQLRestServerStaticMongoDB.GetJSONValues(const Res: TBSONDocument;
  const extFieldNames: TRawUTF8DynArray; W: TJSONSerializer): integer;
var col, colCount, colFound: integer;
    bson: PByte;
    row: TBSONElement;
    item: array of TBSONElement;
function itemFind(const aName: RawUTF8): integer;
begin
................................................................................
    W.Expand := false; //  {"fieldCount":2,"values":["col1","col2"]}
    W.CancelAll;
    fStoredClassRecordProps.SetJSONWriterColumnNames(W,0);
  end;
  W.EndJSONObject(0,result);
end;

function TSQLRestServerStaticMongoDB.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean; ReturnedRowCount: PPtrInt): RawUTF8;
var W: TJSONSerializer;
    MS: TRawByteStringStream;
    Query,Projection: variant;
    Res: TBSONDocument;
    ResCount: PtrInt;
    extFieldNames: TRawUTF8DynArray;
................................................................................
      BSONVariant([QUERY_OPS[Stmt.WhereOperator],Stmt.WhereValueVariant])]);
end;
procedure SetCount(aCount: integer);
begin
  result := FormatUTF8('[{"Count(*)":%}]'#$A,[aCount]);
  ResCount := 1;
end;
begin // same logic as in TSQLRestServerStaticInMemory.EngineList()
  ResCount := 0;
  if self=nil then begin
    result := '';
    exit;
  end;
  Lock(false);
  try
    if IdemPropNameU(fBasicSQLCount,SQL) then
      SetCount(TableRowCount(fStoredClass)) else
    if IdemPropNameU(fBasicSQLHasRows[false],SQL) or
       IdemPropNameU(fBasicSQLHasRows[true],SQL) then
      if TableRowCount(fStoredClass)=0 then begin
        result := '{"fieldCount":1,"values":["RowID"]}'#$A;
................................................................................
        BSONProjectionSet(Projection,Stmt.WithID,Stmt.Fields,@extFieldNames);
        if Stmt.FoundLimit=0 then
          Stmt.FoundLimit := maxInt;
        Res := fCollection.FindBSON(Query,Projection,Stmt.FoundLimit,Stmt.FoundOffset);
        MS := TRawByteStringStream.Create;
        try
          W := fStoredClassRecordProps.CreateJSONWriter(
            MS,ForceAJAX or (not Owner.NoAJAXJSON),Stmt.withID,Stmt.Fields,0);
          try
            ResCount := GetJSONValues(Res,extFieldNames,W);
            result := MS.DataString;
          finally
            W.Free;
          end;
        finally
................................................................................
          MS.Free;
        end;
      finally
        Stmt.Free;
      end;
    end;
  finally
    UnLock;
  end;
  if ReturnedRowCount<>nil then
    ReturnedRowCount^ := ResCount;
end;
    
function TSQLRestServerStaticMongoDB.InternalBatchStart(
  Method: TSQLURIMethod): boolean;
begin
  result := false; // means BATCH mode not supported
  if (self<>nil) and (method in [mPOST,mDELETE]) then begin
    Lock(true); // protected by try..finally in TSQLRestServer.RunBatch
    try
      if (fBatchMethod<>mNone) or (fBatchWriter<>nil) then
        raise EORMException.Create('InternalBatchStop should have been called');
      fBatchCount := 0;
      fBatchMethod := Method;
      case Method of
      mPOST: // POST=ADD=INSERT -> EngineAdd() will add to fBatchWriter
        fBatchWriter := TBSONWriter.Create(TRawByteStringStream);
      //mDELETE: // EngineDelete() will add deleted ID to fBatchIDs[]
      end;
      result := true; // means BATCH mode is supported
    finally
      if not result then
        UnLock;
    end;
  end;
end;

procedure TSQLRestServerStaticMongoDB.InternalBatchStop;
var docs: TBSONDocument;
begin
  try
    case fBatchMethod of
    mPOST: begin // Add/Insert
      if fBatchWriter.TotalWritten=0 then
        exit; // nothing to add
................................................................................
        [fStoredClassRecordProps.SQLTableName,ord(fBatchMethod)]);
    end;
  finally
    FreeAndNil(fBatchWriter);
    fBatchIDs := nil;
    fBatchCount := 0;
    fBatchMethod := mNone;
    UnLock;
  end;
end;

end.







|







 







|
|
|









|

|







 







|

|







 







|

|

|





|













|
|



|

|







 







|







 







|









|










|





|





|







|













|










|







 







>
>


<
>



|
>











<
|






|
|



|
>







<
|






|
<
|




|
>









<
|







|







 







<
|
|






<
|


|
>







<
|







|
|
|
|
|
|
|



<
|








|







 







|










|
<
|




|
>













|







 







|





|







 







|







 







|





|







 







|







 







|





|




|













|




|







 







|




78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
...
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
...
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
...
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
...
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
...
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
...
381
382
383
384
385
386
387
388
389
390
391

392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408

409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429

430
431
432
433
434
435
436
437

438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453

454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
...
483
484
485
486
487
488
489

490
491
492
493
494
495
496
497

498
499
500
501
502
503
504
505
506
507
508
509

510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527

528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
...
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578

579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
...
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
...
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
...
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
...
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
...
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
...
847
848
849
850
851
852
853
854
855
856
857
858
  /// exeception class raised by this units
  EORMMongoDBException = class(EORMException);

  /// REST server with direct access to a MongoDB external database
  // - handle all REST commands via direct SynMongoDB call
  // - is used by TSQLRestServer.URI for faster RESTful direct access
  // - JOINed SQL statements are not handled yet
  TSQLRestStorageMongoDB = class(TSQLRestStorage)
  protected
    /// the associated MongoDB collection
    fCollection: TMongoCollection;
    fEngineLastID: integer;
    fBSONProjectionSimpleFields: variant;
    fBSONProjectionBlobFields: variant;
    fBSONProjectionBlobFieldsNames: TRawUTF8DynArray;
................................................................................
    function BSONProjectionSet(var Projection: variant; WithID: boolean;
      const Fields: TSQLFieldBits; ExtFieldNames: PRawUTF8DynArray): integer;
    function GetJSONValues(const Res: TBSONDocument;
      const extFieldNames: TRawUTF8DynArray; W: TJSONSerializer): integer;
    // overridden methods calling the MongoDB external server
    function EngineRetrieve(TableModelIndex: integer; ID: integer): RawUTF8; override;
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    function EngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer; override;
    function EngineUpdate(TableModelIndex, ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDeleteWhere(TableModelIndex: Integer;const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    /// TSQLRestServer.URI use it for Static.EngineList to by-pass virtual table
    // - overridden method to handle most potential simple queries, e.g. like
    // $ SELECT Field1,RowID FROM table WHERE RowID=... AND/OR/NOT Field2=
    // - ORM field names into mapped MongoDB external field names
    // - handle statements to avoid slow virtual table loop over all rows, like
    // $ SELECT count(*) FROM table
    function AdaptSQLForEngineList(var SQL: RawUTF8): boolean; override;
    // BLOBs should be access directly, not through slower JSON Base64 encoding
    function EngineRetrieveBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    // overriden method returning TRUE for next calls to EngineAdd/Update/Delete
    // will properly handle operations until InternalBatchStop is called
    function InternalBatchStart(Method: TSQLURIMethod): boolean; override;
    // internal method called by TSQLRestServer.RunBatch() to process fast
    // BULK sending to remote MongoDB database
    procedure InternalBatchStop; override;
................................................................................
    /// get the row count of a specified table
    // - return -1 on error
    // - return the row count of the table on success
    function TableRowCount(Table: TSQLRecordClass): integer; override;
    /// check if there is some data rows in a specified table
    function TableHasRows(Table: TSQLRecordClass): boolean; override;
    /// delete a row, calling the current MongoDB server
    // - made public since a TSQLRestStorage instance may be created
    // stand-alone, i.e. without any associated Model/TSQLRestServer
    function EngineDelete(TableModelIndex, ID: integer): boolean; override;
    /// create one index for all specific FieldNames at once
    function CreateSQLMultiIndex(Table: TSQLRecordClass; const FieldNames: array of RawUTF8;
      Unique: boolean; IndexName: RawUTF8=''): boolean; override;

    /// drop the whole table content
    // - but you can still add items to it - whereas Collection.Drop would
    // trigger GPF issues
................................................................................
// you can customize it with the corresponding parameter
// - the TSQLRecord.ID (RowID) field is always mapped to MongoDB's _id field
// - after registration, you can tune the field-name mapping by calling
// ! aModel.Props[aClass].ExternalDB.MapField(..)
// (just a regular external DB as defined in mORMotDB.pas unit) - it may be
// a good idea to use short field names on MongoDB side, to reduce the space
// used for storage (since they will be embedded within the document data)
// - it will return the corresponding TSQLRestStorageMongoDB instance -
// you can access later to it and its associated collection e.g. via:
// ! (aServer.StaticDataServer[TSQLMyTable] as TSQLRestStorageMongoDB)
function StaticMongoDBRegister(aClass: TSQLRecordClass; aServer: TSQLRestServer;
  aMongoDatabase: TMongoDatabase; aMongoCollectionName: RawUTF8=''): TSQLRestStorageMongoDB;


implementation

function StaticMongoDBRegister(aClass: TSQLRecordClass; aServer: TSQLRestServer;
  aMongoDatabase: TMongoDatabase; aMongoCollectionName: RawUTF8=''): TSQLRestStorageMongoDB;
var Props: TSQLModelRecordProperties;
begin
  result := nil;
  if (aServer=nil) or (aClass=nil) or (aMongoDatabase=nil) then
    exit; // avoid GPF
  Props := aServer.Model.Props[aClass];
  if Props=nil then
    exit; // if aClass is not part of the model
  if aMongoCollectionName='' then
    aMongoCollectionName := Props.Props.SQLTableName;
  Props.ExternalDB.Init(Props,aMongoCollectionName,
    aMongoDatabase.CollectionOrCreate[aMongoCollectionName]);
  Props.ExternalDB.MapField('ID','_id');
  result := (aServer.StaticDataCreate(aClass,'',false,TSQLRestStorageMongoDB)
    as TSQLRestStorageMongoDB);
end;


{ TSQLRestStorageMongoDB }

constructor TSQLRestStorageMongoDB.Create(aClass: TSQLRecordClass;
  aServer: TSQLRestServer; const aFileName: TFileName;
  aBinaryFile: boolean);
var F: integer;
begin
  inherited;
  if fStoredClassProps=nil then
    raise EORMMongoDBException.CreateFmt(
................................................................................
    for F := 0 to fStoredClassRecordProps.Fields.Count do
      if F in fIsUnique then
        fCollection.EnsureIndex(
          [fStoredClassProps.ExternalDB.FieldNames[f]],true,true);

end;

function TSQLRestStorageMongoDB.BSONProjectionSet(var Projection: variant;
  WithID: boolean; const Fields: TSQLFieldBits; ExtFieldNames: PRawUTF8DynArray): integer;
var i,n: integer;
    Start: cardinal;
    W: TBSONWriter;
begin
  W := TBSONWriter.Create(TRawByteStringStream);
  try
................................................................................
        end;
    end;
  finally
    W.Free;
  end;
end;

function TSQLRestStorageMongoDB.CreateSQLMultiIndex(
  Table: TSQLRecordClass; const FieldNames: array of RawUTF8;
  Unique: boolean; IndexName: RawUTF8): boolean;
begin
  result := false;
  if (self=nil) or (fCollection=nil) or (Table<>fStoredClass) then
    exit;
  fCollection.EnsureIndex(FieldNames,true,Unique);
end;

procedure TSQLRestStorageMongoDB.Drop;
var DB: TMongoDatabase;
    CollName: RawUTF8;
begin
  DB := Collection.Database;
  CollName := Collection.Name;
  Collection.Drop;
  fCollection := DB.CollectionOrCreate[CollName];
  fEngineLastID := 0;
end;

destructor TSQLRestStorageMongoDB.Destroy;
begin
  inherited;
  FreeAndNil(fBatchWriter);
end;

function TSQLRestStorageMongoDB.TableHasRows(
  Table: TSQLRecordClass): boolean;
begin
  result := TableRowCount(Table)>0;
end;

function TSQLRestStorageMongoDB.TableRowCount(
  Table: TSQLRecordClass): integer;
begin
  if (fCollection=nil) or (Table<>fStoredClass) then
    result := 0 else
    result := fCollection.Count;
end;

function TSQLRestStorageMongoDB.EngineNextID: Integer;
procedure ComputeMax_ID;
var res: variant;
begin
  res := fCollection.AggregateDoc('{$group:{_id:null,max:{$max:"$_id"}}}',[]);
  if DocVariantType.IsOfType(res) then
    fEngineLastID := VariantToIntegerDef(res.max,0);
end;
begin
  if fEngineLastID=0 then
    ComputeMax_ID;
  result := InterlockedIncrement(fEngineLastID);
end;

function TSQLRestStorageMongoDB.DocFromJSON(const JSON: RawUTF8;
  Occasion: TSQLOccasion; var Doc: TDocVariantData): integer;
var i, ndx: integer;
    blob: RawByteString;
    info: TSQLPropInfo;
    typenfo: pointer;
    js: RawUTF8;
    MissingID: boolean;
    V: PVarData;
begin
  doc.InitJSON(JSON,[dvoValueCopiedByReference]);
  if (doc.Kind<>dvObject) and (Occasion<>soInsert) then
    raise EORMMongoDBException.Create('Invalid JSON context');
  if not (Occasion in [soInsert,soUpdate]) then
    raise EORMMongoDBException.CreateFmt('DocFromJSON(%s)',[
      GetEnumName(TypeInfo(TSQLOccasion),ord(Occasion))^]);
  MissingID := true;
  for i := doc.Count-1 downto 0 do // downwards for doc.Delete(i) below
    if IsRowID(pointer(doc.Names[i])) then begin
................................................................................
        // sftObject,sftVariant were already converted to object from JSON
      end;
    end;
  if (Occasion=soInsert) and MissingID then begin
    result := EngineNextID;
    doc.AddValue(fStoredClassProps.ExternalDB.RowIDFieldName,result);
  end;
  if doc.Kind<>dvObject then
    raise EORMMongoDBException.Create('Invalid JSON context');
end;


function TSQLRestStorageMongoDB.EngineAdd(TableModelIndex: integer; 
  const SentData: RawUTF8): integer;
var doc: TDocVariantData;
begin
  if (fCollection=nil) or (TableModelIndex<0) or 
    (fModel.Tables[TableModelIndex]<>fStoredClass) then
    result := 0 else
    try
      result := DocFromJSON(SentData,soInsert,Doc);
      if fBatchMethod<>mNone then
        if (fBatchMethod<>mPOST) or (fBatchWriter=nil) then
          result := 0 else begin
          inc(fBatchCount);
          fBatchWriter.BSONWriteDoc(doc);
        end else begin
        fCollection.Insert([variant(doc)]);
        if Owner<>nil then

          Owner.InternalUpdateEvent(seAdd,fStoredClass,result,nil);
      end;
    except
      result := 0;
    end;
end;

function TSQLRestStorageMongoDB.EngineUpdate(TableModelIndex, ID: integer;
  const SentData: RawUTF8): boolean;
var doc: TDocVariantData;
    query,update: variant; // use explicit TBSONVariant for type safety
begin
  if (fCollection=nil) or (ID<=0) or
     (TableModelIndex<0) or (Model.Tables[TableModelIndex]<>fStoredClass) then
    result := false else
    try
      DocFromJSON(SentData,soUpdate,Doc);
      query := BSONVariant(['_id',ID]);
      update := BSONVariant(['$set',variant(Doc)]);
      fCollection.Update(query,update);
      if Owner<>nil then

        Owner.InternalUpdateEvent(seUpdate,fStoredClass,ID,nil);
      result := true;
    except
      result := false;
    end;
end;

function TSQLRestStorageMongoDB.EngineUpdateBlob(TableModelIndex, aID: integer;

  BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean;
var query,update,blob: variant; // use explicit TBSONVariant for type safety
    FieldName: RawUTF8;
    AffectedField: TSQLFieldBits;
begin
  if (fCollection=nil) or (BlobField=nil) or (aID<=0) or
     (TableModelIndex<0) or (Model.Tables[TableModelIndex]<>fStoredClass) then
    result := false else
    try
      query := BSONVariant(['_id',aID]);
      FieldName := fStoredClassProps.ExternalDB.InternalToExternal(BlobField^.Name);
      BSONVariantType.FromBinary(BlobData,bbtGeneric,blob);
      update := BSONVariant(['$set',BSONVariant([FieldName,blob])]);
      fCollection.Update(query,update);
      if Owner<>nil then begin
        fStoredClassRecordProps.FieldIndexsFromBlobField(BlobField,AffectedField);

        Owner.InternalUpdateEvent(seUpdateBlob,fStoredClass,aID,@AffectedField);
      end;
      result := true;
    except
      result := false;
    end;
end;

function TSQLRestStorageMongoDB.UpdateBlobFields(
  Value: TSQLRecord): boolean;
var query,blob: variant;
    update: TDocVariantData;
    info: TSQLPropInfo;
    blobRaw: RawByteString;
    aID, f: integer;
begin
................................................................................
      update.AddValue(fStoredClassProps.ExternalDB.FieldNames[f],blob);
    end;
  end;
  if update.Count>0 then
    try
      fCollection.Update(query,BSONVariant(['$set',variant(update)]));
      if Owner<>nil then

        Owner.InternalUpdateEvent(seUpdateBlob,fStoredClass,aID,
          @fStoredClassRecordProps.BlobFieldsBits);
      result := true;
    except
      result := false;
    end;
end;


function TSQLRestStorageMongoDB.EngineDelete(TableModelIndex, ID: integer): boolean;
begin
  result := false;
  if (fCollection<>nil) and (TableModelIndex>=0) and
     (Model.Tables[TableModelIndex]=fStoredClass) and (ID>0) then
  try
    if fBatchMethod<>mNone then
      if fBatchMethod<>mDelete then
        exit else
        AddInteger(fBatchIDs,fBatchCount,ID) else begin
      fCollection.RemoveOne(ID);
      if Owner<>nil then

        Owner.InternalUpdateEvent(seDelete,fStoredClass,ID,nil);
    end;
    result := true;
  except
    result := false;
  end;
end;

function TSQLRestStorageMongoDB.EngineDeleteWhere(TableModelIndex: Integer;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
var i: integer;
begin // here we use the pre-computed IDs[]
  result := false;
  if (fCollection<>nil) and (TableModelIndex>=0) and
     (Model.Tables[TableModelIndex]=fStoredClass) and (IDs=nil) then
    try
      if Owner<>nil then // notify BEFORE deletion
      for i := 0 to high(IDs) do

        Owner.InternalUpdateEvent(seDelete,fStoredClass,IDs[i],nil);
      fCollection.Remove(BSONVariant(
        ['_id',BSONVariant(['$in',BSONVariantFromIntegers(IDs)])]));
      result := true;
    except
      result := false;
    end;
end;

procedure TSQLRestStorageMongoDB.JSONFromDoc(var doc: TDocVariantData;
  var result: RawUTF8);
var i: integer;
    name: RawUTF8;
    W: TTextWriter;
begin
  if (doc.VarType<>DocVariantType.VarType) or (doc.Kind<>dvObject) or (doc.Count=0) then begin
    result := '';
................................................................................
    W.Add('}');
    W.SetText(result);
  finally
    W.Free;
  end;
end;

function TSQLRestStorageMongoDB.EngineRetrieve(TableModelIndex,
  ID: integer): RawUTF8;
var doc: variant;
begin
  result := '';
  if (fCollection=nil) or (ID<=0) then
    exit;
  doc := fCollection.FindDoc(BSONVariant(['_id',ID]),fBSONProjectionSimpleFields,1);
  JSONFromDoc(TDocVariantData(doc),result);
end;

function TSQLRestStorageMongoDB.EngineRetrieveBlob(TableModelIndex, aID: integer;

  BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
var doc: variant;
    data: TVarData;
    FieldName: RawUTF8;
begin
  if (fCollection=nil) or (BlobField=nil) or (aID<=0) or
     (TableModelIndex<0) or (Model.Tables[TableModelIndex]<>fStoredClass) then
    result := false else
    try
      FieldName := fStoredClassProps.ExternalDB.InternalToExternal(BlobField^.Name);
      doc := fCollection.FindDoc(BSONVariant(['_id',aID]),BSONVariant([FieldName,1]),1);
      if DocVariantType.IsOfType(doc) and
         DocVariantData(doc)^.GetVarData(FieldName,data) then
        BSONVariantType.ToBlob(variant(data),RawByteString(BlobData));
      result := true;
    except
      result := false;
    end;
end;

function TSQLRestStorageMongoDB.RetrieveBlobFields(
  Value: TSQLRecord): boolean;
var aID, f: Integer;
    doc: variant;
    docv: PDocVariantData;
    blob: TVarData;
    blobRaw: RawByteString;
begin
................................................................................
    end;
    result := true;
  except
    result := false;
  end;
end;

function TSQLRestStorageMongoDB.AdaptSQLForEngineList(
  var SQL: RawUTF8): boolean;
begin
  result := true; // we do not have any Virtual Table yet -> always accept
end;

function TSQLRestStorageMongoDB.GetJSONValues(const Res: TBSONDocument;
  const extFieldNames: TRawUTF8DynArray; W: TJSONSerializer): integer;
var col, colCount, colFound: integer;
    bson: PByte;
    row: TBSONElement;
    item: array of TBSONElement;
function itemFind(const aName: RawUTF8): integer;
begin
................................................................................
    W.Expand := false; //  {"fieldCount":2,"values":["col1","col2"]}
    W.CancelAll;
    fStoredClassRecordProps.SetJSONWriterColumnNames(W,0);
  end;
  W.EndJSONObject(0,result);
end;

function TSQLRestStorageMongoDB.EngineList(const SQL: RawUTF8;
  ForceAJAX: Boolean; ReturnedRowCount: PPtrInt): RawUTF8;
var W: TJSONSerializer;
    MS: TRawByteStringStream;
    Query,Projection: variant;
    Res: TBSONDocument;
    ResCount: PtrInt;
    extFieldNames: TRawUTF8DynArray;
................................................................................
      BSONVariant([QUERY_OPS[Stmt.WhereOperator],Stmt.WhereValueVariant])]);
end;
procedure SetCount(aCount: integer);
begin
  result := FormatUTF8('[{"Count(*)":%}]'#$A,[aCount]);
  ResCount := 1;
end;
begin // same logic as in TSQLRestStorageInMemory.EngineList()
  ResCount := 0;
  if self=nil then begin
    result := '';
    exit;
  end;
  StorageLock(false);
  try
    if IdemPropNameU(fBasicSQLCount,SQL) then
      SetCount(TableRowCount(fStoredClass)) else
    if IdemPropNameU(fBasicSQLHasRows[false],SQL) or
       IdemPropNameU(fBasicSQLHasRows[true],SQL) then
      if TableRowCount(fStoredClass)=0 then begin
        result := '{"fieldCount":1,"values":["RowID"]}'#$A;
................................................................................
        BSONProjectionSet(Projection,Stmt.WithID,Stmt.Fields,@extFieldNames);
        if Stmt.FoundLimit=0 then
          Stmt.FoundLimit := maxInt;
        Res := fCollection.FindBSON(Query,Projection,Stmt.FoundLimit,Stmt.FoundOffset);
        MS := TRawByteStringStream.Create;
        try
          W := fStoredClassRecordProps.CreateJSONWriter(
            MS,ForceAJAX or (Owner=nil) or not Owner.NoAJAXJSON,Stmt.withID,Stmt.Fields,0);
          try
            ResCount := GetJSONValues(Res,extFieldNames,W);
            result := MS.DataString;
          finally
            W.Free;
          end;
        finally
................................................................................
          MS.Free;
        end;
      finally
        Stmt.Free;
      end;
    end;
  finally
    StorageUnLock;
  end;
  if ReturnedRowCount<>nil then
    ReturnedRowCount^ := ResCount;
end;
    
function TSQLRestStorageMongoDB.InternalBatchStart(
  Method: TSQLURIMethod): boolean;
begin
  result := false; // means BATCH mode not supported
  if (self<>nil) and (method in [mPOST,mDELETE]) then begin
    StorageLock(true); // protected by try..finally in TSQLRestServer.RunBatch
    try
      if (fBatchMethod<>mNone) or (fBatchWriter<>nil) then
        raise EORMException.Create('InternalBatchStop should have been called');
      fBatchCount := 0;
      fBatchMethod := Method;
      case Method of
      mPOST: // POST=ADD=INSERT -> EngineAdd() will add to fBatchWriter
        fBatchWriter := TBSONWriter.Create(TRawByteStringStream);
      //mDELETE: // EngineDelete() will add deleted ID to fBatchIDs[]
      end;
      result := true; // means BATCH mode is supported
    finally
      if not result then
        StorageUnLock;
    end;
  end;
end;

procedure TSQLRestStorageMongoDB.InternalBatchStop;
var docs: TBSONDocument;
begin
  try
    case fBatchMethod of
    mPOST: begin // Add/Insert
      if fBatchWriter.TotalWritten=0 then
        exit; // nothing to add
................................................................................
        [fStoredClassRecordProps.SQLTableName,ord(fBatchMethod)]);
    end;
  finally
    FreeAndNil(fBatchWriter);
    fBatchIDs := nil;
    fBatchCount := 0;
    fBatchMethod := mNone;
    StorageUnLock;
  end;
end;

end.

Changes to SQLite3/mORMotSQLite3.pas.

102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
...
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
...
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
...
639
640
641
642
643
644
645
646
647
648
649
650
651

652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
...
789
790
791
792
793
794
795
796
797
798
799
800
801
802

803
804
805

806
807
808
809
810
811
812
813
814
815
816
817
818

819
820
821
822
823
824
825
...
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
....
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036

1037
1038
1039

1040
1041
1042
1043
1044
1045
1046
....
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
....
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113

1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124

1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
























1151
1152
1153
1154
1155
1156
1157
1158
1159
....
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
....
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
      value in milliseconds
    - now handles User Defined Functions, via sqlite3_create_function_v2 and
      corresponding sqlite3_result_* functions: as sample, the MOD() function
      is defined in any database opened via TSQLDataBase (it was needed
      to have compatibility with Oracle/MySQL/MSSQL/PostGreSQL engines)
    - protect the TSQLDatabase methods called when self is nil, which could
      occur if the database is not yet initialized (could occur if only a
      TSQLRestServerStatic exists, like in TTestSQLite3Engine._TSQLRestClientDB)
    - new SOUNDEX() function available in SQL statements (calling SoundExUTF8)
      and associated SOUNDEXFR/SOUNDEXES for french or spanish Soundex variants
    - fixed an issue found out by WladiD about all collation functions:
      If a field with your custom collate ISO8601 is empty '' (not NULL),
      then SQLite calls the registered collate function with length 0 for s1len
      or s2len, but the pointers s1 or s2 map to the string of the previous call
    - added sqlite3_result_error() call to make wrong parameter count error
................................................................................
    // :(%): parameter values; in this case, TSQLRequest.Close must not be called
    // - expect sftBlob, sftBlobDynArray and sftBlobRecord properties
    // to be encoded as ':("\uFFF0base64encodedbinary"):'
    function GetAndPrepareStatement(const SQL: RawUTF8): PSQLRequest;
    /// reset the cache if necessary
    procedure SetNoAJAXJSON(const Value: boolean); override;
    /// overriden methods for direct sqlite3 database engine call:
    function EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false; ReturnedRowCount: PPtrInt=nil): RawUTF8; override;
    function EngineRetrieve(TableModelIndex, ID: integer): RawUTF8; override;
    function EngineAdd(Table: TSQLRecordClass; const SentData: RawUTF8): integer; override;
    function EngineUpdate(Table: TSQLRecordClass; ID: integer; const SentData: RawUTF8): boolean; override;
    function EngineDelete(Table: TSQLRecordClass; ID: integer): boolean; override;
    function EngineDeleteWhere(Table: TSQLRecordClass; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    function EngineRetrieveBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateBlob(Table: TSQLRecordClass; aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function EngineUpdateField(Table: TSQLRecordClass;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
    /// execute one SQL statement
    // - intercept any DB exception and return false on error, true on success
    // - optional LastInsertedID can be set (if ValueInt/ValueUTF8 are nil) to
    // retrieve the proper ID when aSQL is an INSERT statement (thread safe)
    function EngineExecute(const aSQL: RawUTF8; ValueInt: PInt64=nil; ValueUTF8: PRawUTF8=nil;
      LastInsertedID: PInt64=nil): boolean; overload;
................................................................................
    // with already registered modules via RegisterVirtualTableModule()
    // - you can override this method to call e.g. DB.RegisterSQLFunction()
    procedure InitializeEngine; virtual;
    /// call this method when the internal DB content is known to be invalid
    // - by default, all REST/CRUD requests and direct SQL statements are
    // scanned and identified as potentially able to change the internal SQL/JSON
    // cache used at SQLite3 database level; but some virtual tables (e.g.
    // TSQLRestServerStaticExternal classes defined in SQLite3DB) could flush
    // the database content without proper notification
    // - this overriden implementation will call TSQLDataBase.CacheFlush method
    procedure FlushInternalDBCache; override;
  public
    /// initialize a REST server with a database
    // - any needed TSQLVirtualTable class should have been already registered
    // via the RegisterVirtualTableModule() method
................................................................................
      sptText:    result^.Bind(i+1,Values[i]);
      sptBlob:    result^.Bind(i+1,pointer(Values[i]),length(Values[i]));
      sptInteger: result^.Bind(i+1,GetInt64(pointer(Values[i])));
      sptFloat:   result^.Bind(i+1,GetExtended(pointer(Values[i])));
    end;
end;

function TSQLRestServerDB.EngineAdd(Table: TSQLRecordClass; const SentData: RawUTF8): integer;
var SQL: RawUTF8;
    LastID: Int64;
begin
  if (self=nil) or (Table=nil) then begin
    result := 0; // avoid GPF

    exit;
  end;
  SQL := 'INSERT INTO '+Table.RecordProps.SQLTableName;
  if trim(SentData)='' then
    SQL := SQL+' DEFAULT VALUES;' else
    SQL := SQL+GetJSONObjectAsSQL(SentData,false,true,
      JSONRetrieveIDField(pointer(SentData)))+';';
  if EngineExecute(SQL,nil,nil,@LastID) then begin
    result := LastID;
    InternalUpdateEvent(seAdd,Table,result,nil);
  end else
    result := 0;
end;

procedure InternalRTreeIn(Context: TSQLite3FunctionContext;
  argc: integer; var argv: TSQLite3ValueArray); {$ifndef SQLITE3_FASTCALL}cdecl;{$endif}
var aRTree: TSQLRecordRTreeClass;
    BlobA, BlobB: pointer;
begin
................................................................................
    on E: Exception do begin
      DB.RollBack; // will close any active Transaction
      raise;     // caller must handle exception
    end;
  end;
end;

function TSQLRestServerDB.EngineDelete(Table: TSQLRecordClass; ID: integer): boolean;
begin
  if (Table=nil) or (ID<=0) then begin
    result := false; // avoid GPF
    exit;
  end;
  InternalUpdateEvent(seDelete,Table,ID,nil); // notify BEFORE deletion

  result := EngineExecuteFmt(
    'DELETE FROM % WHERE RowID=:(%):;',[Table.RecordProps.SQLTableName,ID]);
end;


function TSQLRestServerDB.EngineDeleteWhere(Table: TSQLRecordClass;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
var i: integer;
begin
  if (Table=nil) or (SQLWhere='') or (IDs=nil) then begin
    result := false;
    exit;
  end;
  for i := 0 to high(IDs) do
    InternalUpdateEvent(seDelete,Table,IDs[i],nil); // notify BEFORE deletion
  result := EngineExecuteFmt(
    'DELETE FROM % WHERE %',[Table.RecordProps.SQLTableName,SQLWhere]);

end;

destructor TSQLRestServerDB.Destroy;
var i: integer;
begin
  try
    if fRegisteredVirtualTableModules<>nil then
................................................................................
      LogToTextFile('TSQLRestServerDB.EngineExecuteAll Error: '+RawUTF8(E.Message)+#13#10+aSQL);
      {$endif}
      result := false;
    end;
  end;
end;

function TSQLRestServerDB.EngineList(const SQL: RawUTF8; ForceAJAX: Boolean=false;
  ReturnedRowCount: PPtrInt=nil): RawUTF8;
var Req: PSQLRequest;
    MS: TRawByteStringStream;
    RowCount: integer;
begin
  result := '';
  RowCount := 0;
  if (self<>nil) and (DB<>nil) and (SQL<>'') then begin
................................................................................
      DB.UnLockJSON(result,RowCount);
    end;
  end;
  if ReturnedRowCount<>nil then
    ReturnedRowCount^ := RowCount;
end;

function TSQLRestServerDB.EngineRetrieve(TableModelIndex, ID: integer): RawUTF8;
var aSQL: RawUTF8;
begin
  if (self=nil) or (cardinal(TableModelIndex)>=cardinal(length(Model.TableProps))) then
    exit;
  with Model.TableProps[TableModelIndex] do
    aSQL := FormatUTF8('SELECT % FROM % WHERE RowID=:(%):;',
      [SQL.TableSimpleFields[true,false],Props.SQLTableName,ID]);
  result := EngineList(aSQL,true); // ForceAJAX=true -> '[{...}]'#10
  if result<>'' then
    if IsNotAjaxJSON(pointer(result)) then
      // '{"fieldCount":2,"values":["ID","FirstName"]}'#$A -> ID not found
      result := '' else
      // list '[{...}]'#10 -> object '{...}'
      result := copy(result,2,length(result)-3);
end;

function TSQLRestServerDB.EngineRetrieveBlob(Table: TSQLRecordClass;
  aID: integer; BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
var SQL: RawUTF8;
    Req: PSQLRequest;
begin
  result := false;
  if (self=nil) or (DB=nil) or (aID<=0) or (Table=nil) or not BlobField^.IsBlob then
    exit;

  try
    SQL := FormatUTF8('SELECT % FROM % WHERE RowID=?;',
      [BlobField^.Name,Table.RecordProps.SQLTableName]);

    DB.Lock(SQL); // UPDATE for a blob field -> no JSON cache flush, but UI refresh
    try
      Req := fStatementCache.Prepare(SQL);
      with Req^ do
      try
        Bind(1,aID);
        if (FieldCount=1) and (Step=SQLITE_ROW) then begin
................................................................................
  except
    on ESQLite3Exception do
      result := false;
  end;
end;

function TSQLRestServerDB.RetrieveBlobFields(Value: TSQLRecord): boolean;
var Static: TSQLRestServerStatic;
    SQL: RawUTF8;
    f: integer;
    data: TSQLVar;
begin
  result := false;
  if Value=nil then
    exit;
................................................................................
procedure TSQLRestServerDB.SetNoAJAXJSON(const Value: boolean);
begin
  inherited;
  if Value=NoAJAXJSON then exit;
  fDB.Cache.Reset; // we changed the JSON format -> cache must be updated
end;

function TSQLRestServerDB.EngineUpdate(Table: TSQLRecordClass; ID: integer;
  const SentData: RawUTF8): boolean;
begin
  if (self=nil) or (Table=nil) or (ID<=0) then
    result := false else
  if SentData='' then // update with no simple field -> valid no-ops
    result := true else begin
    // this SQL statement use :(inlined params): for all values
    result := EngineExecuteFmt('UPDATE % SET % WHERE RowID=:(%):;',

      [Table.RecordProps.SQLTableName,GetJSONObjectAsSQL(SentData,true,true),ID]);
    InternalUpdateEvent(seUpdate,Table,ID,nil);
  end;
end;

function TSQLRestServerDB.EngineUpdateBlob(Table: TSQLRecordClass;
  aID: integer; BlobField: PPropInfo;
  const BlobData: TSQLRawBlob): boolean;
var SQL: RawUTF8;
    AffectedField: TSQLFieldBits;
begin

  result := false;
  if (self=nil) or (DB=nil) or (aID<=0) or (Table=nil) or not BlobField^.IsBlob then
    exit;
  try
    SQL := FormatUTF8('UPDATE % SET %=? WHERE RowID=?;',
             [Table.RecordProps.SQLTableName,BlobField^.Name]);
    DB.Lock(SQL); // UPDATE for a blob field -> no JSON cache flush, but UI refresh
    try
      with fStatementCache.Prepare(SQL)^ do begin
        Bind(1,pointer(BlobData),length(BlobData));
        Bind(2,aID);
        repeat
        until Step<>SQLITE_ROW; // Execute all steps of the first statement
        result := true;
      end;
    finally
      DB.UnLock;
    end;
    Table.RecordProps.FieldIndexsFromBlobField(BlobField,AffectedField);
    InternalUpdateEvent(seUpdateBlob,Table,aID,@AffectedField);
  except
    on ESQLite3Exception do
      result := false;
  end;
end;

























function TSQLRestServerDB.UpdateBlobFields(Value: TSQLRecord): boolean;
var Static: TSQLRestServerStatic;
    SQL: RawUTF8;
    f: integer;
    data: TSQLVar;
    temp: RawByteString;
begin
  result := false;
  if Value=nil then
................................................................................
        DB.UnLock;
      end;
      InternalUpdateEvent(seUpdateBlob,PSQLRecordClass(Value)^,Value.ID,@BlobFieldsBits);
    end else
      result := true; // as TSQLRest.UpdateblobFields()
end;

function TSQLRestServerDB.EngineUpdateField(Table: TSQLRecordClass;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
var Static: TSQLRestServerStatic;
    WhereID: integer;
begin
  result := false;
  if (self=nil) or (Table=nil) then
    exit;
  Static := GetStaticDataServerOrVirtualTable(Table);
  if Static<>nil then
    result := Static.EngineUpdateField(Table,SetFieldName,SetValue,WhereFieldName,WhereValue) else
    with Table.RecordProps do 
    if Fields.IndexByName(SetFieldName)>=0 then begin
      WhereID := 0;
      if IsRowID(pointer(WhereFieldName)) then begin
        WhereID := GetInteger(Pointer(WhereValue));
        if (WhereID<=0) or not RecordCanBeUpdated(Table,WhereID,seUpdate) then
          exit; // limitation: will only check for update from RowID
      end else
        if Fields.IndexByName(WhereFieldName)<0 then
          exit;
      result := EngineExecuteFmt('UPDATE % SET %=:(%): WHERE %=:(%):',
        [SQLTableName,SetFieldName,SetValue,WhereFieldName,WhereValue]);
      if WhereID>0 then
        InternalUpdateEvent(seUpdate,Table,WhereID,nil);
    end;
end;

procedure TSQLRestServerDB.Commit(SessionID: cardinal=1);
begin
  inherited Commit(SessionID); // reset fTransactionActive + write all TSQLVirtualTableJSON
  try
    DB.Commit;
  except
    on ESQLite3Exception do
................................................................................
  result := nil;
  n := length(Tables);
  if (self<>nil) and (n>0) then
  try // will use JSON cache if available:
    aSQL := Model.SQLFromSelectWhere(Tables,SQLSelect,SQLWhere);
    if n=1 then
      // InternalListJSON will handle both static and DB tables
      result := fServer.InternalListJSON(TSQLRecordClass(Tables[0]),aSQL) else
      // we access localy the DB -> TSQLTableDB handle Tables parameter
      result := TSQLTableDB.Create(fServer.DB,Tables,aSQL,not fServer.NoAJAXJSON);
    if fServer.DB.InternalState<>nil then
      result.InternalState := fServer.DB.InternalState^;
  except
    on ESQLite3Exception do
      result := nil;







|







 







|
|
|
|
|
|

|

|

|







 







|







 







|



<
|
>

|
<






|
|
<







 







|

|
|
<
<
|
>
|
|
|
>

|



|
|
|
<
|
|
|
|
>







 







|
|







 







|


|













|
|




|

>


<
>







 







|







 







|


|

|



>
|
|



|
<
|



>
|
<
|


|












|







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

|







 







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







 







|







102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
...
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
...
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
...
639
640
641
642
643
644
645
646
647
648
649

650
651
652
653

654
655
656
657
658
659
660
661

662
663
664
665
666
667
668
...
787
788
789
790
791
792
793
794
795
796
797


798
799
800
801
802
803
804
805
806
807
808
809
810
811

812
813
814
815
816
817
818
819
820
821
822
823
...
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
....
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037

1038
1039
1040
1041
1042
1043
1044
1045
....
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
....
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119

1120
1121
1122
1123
1124
1125

1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
....
1207
1208
1209
1210
1211
1212
1213




























1214
1215
1216
1217
1218
1219
1220
....
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
      value in milliseconds
    - now handles User Defined Functions, via sqlite3_create_function_v2 and
      corresponding sqlite3_result_* functions: as sample, the MOD() function
      is defined in any database opened via TSQLDataBase (it was needed
      to have compatibility with Oracle/MySQL/MSSQL/PostGreSQL engines)
    - protect the TSQLDatabase methods called when self is nil, which could
      occur if the database is not yet initialized (could occur if only a
      TSQLRestStorage exists, like in TTestSQLite3Engine._TSQLRestClientDB)
    - new SOUNDEX() function available in SQL statements (calling SoundExUTF8)
      and associated SOUNDEXFR/SOUNDEXES for french or spanish Soundex variants
    - fixed an issue found out by WladiD about all collation functions:
      If a field with your custom collate ISO8601 is empty '' (not NULL),
      then SQLite calls the registered collate function with length 0 for s1len
      or s2len, but the pointers s1 or s2 map to the string of the previous call
    - added sqlite3_result_error() call to make wrong parameter count error
................................................................................
    // :(%): parameter values; in this case, TSQLRequest.Close must not be called
    // - expect sftBlob, sftBlobDynArray and sftBlobRecord properties
    // to be encoded as ':("\uFFF0base64encodedbinary"):'
    function GetAndPrepareStatement(const SQL: RawUTF8): PSQLRequest;
    /// reset the cache if necessary
    procedure SetNoAJAXJSON(const Value: boolean); override;
    /// overriden methods for direct sqlite3 database engine call:
    function MainEngineList(const SQL: RawUTF8; ForceAJAX: Boolean; ReturnedRowCount: PPtrInt): RawUTF8; override;
    function MainEngineRetrieve(TableModelIndex, ID: integer): RawUTF8; override;
    function MainEngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer; override;
    function MainEngineUpdate(TableModelIndex, ID: integer; const SentData: RawUTF8): boolean; override;
    function MainEngineDelete(TableModelIndex, ID: integer): boolean; override;
    function MainEngineDeleteWhere(TableModelIndex: Integer; const SQLWhere: RawUTF8;
      const IDs: TIntegerDynArray): boolean; override;
    function MainEngineRetrieveBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean; override;
    function MainEngineUpdateBlob(TableModelIndex, aID: integer;
      BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean; override;
    function MainEngineUpdateField(TableModelIndex: integer;
      const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean; override;
    /// execute one SQL statement
    // - intercept any DB exception and return false on error, true on success
    // - optional LastInsertedID can be set (if ValueInt/ValueUTF8 are nil) to
    // retrieve the proper ID when aSQL is an INSERT statement (thread safe)
    function EngineExecute(const aSQL: RawUTF8; ValueInt: PInt64=nil; ValueUTF8: PRawUTF8=nil;
      LastInsertedID: PInt64=nil): boolean; overload;
................................................................................
    // with already registered modules via RegisterVirtualTableModule()
    // - you can override this method to call e.g. DB.RegisterSQLFunction()
    procedure InitializeEngine; virtual;
    /// call this method when the internal DB content is known to be invalid
    // - by default, all REST/CRUD requests and direct SQL statements are
    // scanned and identified as potentially able to change the internal SQL/JSON
    // cache used at SQLite3 database level; but some virtual tables (e.g.
    // TSQLRestStorageExternal classes defined in SQLite3DB) could flush
    // the database content without proper notification
    // - this overriden implementation will call TSQLDataBase.CacheFlush method
    procedure FlushInternalDBCache; override;
  public
    /// initialize a REST server with a database
    // - any needed TSQLVirtualTable class should have been already registered
    // via the RegisterVirtualTableModule() method
................................................................................
      sptText:    result^.Bind(i+1,Values[i]);
      sptBlob:    result^.Bind(i+1,pointer(Values[i]),length(Values[i]));
      sptInteger: result^.Bind(i+1,GetInt64(pointer(Values[i])));
      sptFloat:   result^.Bind(i+1,GetExtended(pointer(Values[i])));
    end;
end;

function TSQLRestServerDB.MainEngineAdd(TableModelIndex: integer; const SentData: RawUTF8): integer;
var SQL: RawUTF8;
    LastID: Int64;
begin

  result := 0;
  if TableModelIndex<0 then
    exit;
  SQL := 'INSERT INTO '+fModel.TableProps[TableModelIndex].Props.SQLTableName;

  if trim(SentData)='' then
    SQL := SQL+' DEFAULT VALUES;' else
    SQL := SQL+GetJSONObjectAsSQL(SentData,false,true,
      JSONRetrieveIDField(pointer(SentData)))+';';
  if EngineExecute(SQL,nil,nil,@LastID) then begin
    result := LastID;
    InternalUpdateEvent(seAdd,fModel.Tables[TableModelIndex],result,nil);
  end;

end;

procedure InternalRTreeIn(Context: TSQLite3FunctionContext;
  argc: integer; var argv: TSQLite3ValueArray); {$ifndef SQLITE3_FASTCALL}cdecl;{$endif}
var aRTree: TSQLRecordRTreeClass;
    BlobA, BlobB: pointer;
begin
................................................................................
    on E: Exception do begin
      DB.RollBack; // will close any active Transaction
      raise;     // caller must handle exception
    end;
  end;
end;

function TSQLRestServerDB.MainEngineDelete(TableModelIndex, ID: integer): boolean;
begin
  if (TableModelIndex<0) or (ID<=0) then
    result := false else begin


    // notify BEFORE deletion
    InternalUpdateEvent(seDelete,fModel.Tables[TableModelIndex],ID,nil);
    result := EngineExecuteFmt('DELETE FROM % WHERE RowID=:(%):;',
      [fModel.TableProps[TableModelIndex].Props.SQLTableName,ID]);
  end;
end;

function TSQLRestServerDB.MainEngineDeleteWhere(TableModelIndex: Integer;
  const SQLWhere: RawUTF8; const IDs: TIntegerDynArray): boolean;
var i: integer;
begin
  if (TableModelIndex<0) or (SQLWhere='') or (IDs=nil) then
    result := false else begin
    // notify BEFORE deletion

    for i := 0 to high(IDs) do
      InternalUpdateEvent(seDelete,fModel.Tables[TableModelIndex],IDs[i],nil);
    result := EngineExecuteFmt('DELETE FROM % WHERE %',
      [fModel.TableProps[TableModelIndex].Props.SQLTableName,SQLWhere]);
  end;
end;

destructor TSQLRestServerDB.Destroy;
var i: integer;
begin
  try
    if fRegisteredVirtualTableModules<>nil then
................................................................................
      LogToTextFile('TSQLRestServerDB.EngineExecuteAll Error: '+RawUTF8(E.Message)+#13#10+aSQL);
      {$endif}
      result := false;
    end;
  end;
end;

function TSQLRestServerDB.MainEngineList(const SQL: RawUTF8; ForceAJAX: Boolean;
  ReturnedRowCount: PPtrInt): RawUTF8;
var Req: PSQLRequest;
    MS: TRawByteStringStream;
    RowCount: integer;
begin
  result := '';
  RowCount := 0;
  if (self<>nil) and (DB<>nil) and (SQL<>'') then begin
................................................................................
      DB.UnLockJSON(result,RowCount);
    end;
  end;
  if ReturnedRowCount<>nil then
    ReturnedRowCount^ := RowCount;
end;

function TSQLRestServerDB.MainEngineRetrieve(TableModelIndex, ID: integer): RawUTF8;
var aSQL: RawUTF8;
begin
  if (ID<0) or (TableModelIndex<0) or (result<>'') then
    exit;
  with Model.TableProps[TableModelIndex] do
    aSQL := FormatUTF8('SELECT % FROM % WHERE RowID=:(%):;',
      [SQL.TableSimpleFields[true,false],Props.SQLTableName,ID]);
  result := EngineList(aSQL,true); // ForceAJAX=true -> '[{...}]'#10
  if result<>'' then
    if IsNotAjaxJSON(pointer(result)) then
      // '{"fieldCount":2,"values":["ID","FirstName"]}'#$A -> ID not found
      result := '' else
      // list '[{...}]'#10 -> object '{...}'
      result := copy(result,2,length(result)-3);
end;

function TSQLRestServerDB.MainEngineRetrieveBlob(TableModelIndex, aID: integer;
  BlobField: PPropInfo; out BlobData: TSQLRawBlob): boolean;
var SQL: RawUTF8;
    Req: PSQLRequest;
begin
  result := false;
  if (aID<0) or (TableModelIndex<0) or not BlobField^.IsBlob then
    exit;
  // retrieve the BLOB using SQL
  try
    SQL := FormatUTF8('SELECT % FROM % WHERE RowID=?;',

      [BlobField^.Name,Model.TableProps[TableModelIndex].Props.SQLTableName]);
    DB.Lock(SQL); // UPDATE for a blob field -> no JSON cache flush, but UI refresh
    try
      Req := fStatementCache.Prepare(SQL);
      with Req^ do
      try
        Bind(1,aID);
        if (FieldCount=1) and (Step=SQLITE_ROW) then begin
................................................................................
  except
    on ESQLite3Exception do
      result := false;
  end;
end;

function TSQLRestServerDB.RetrieveBlobFields(Value: TSQLRecord): boolean;
var Static: TSQLRestStorage;
    SQL: RawUTF8;
    f: integer;
    data: TSQLVar;
begin
  result := false;
  if Value=nil then
    exit;
................................................................................
procedure TSQLRestServerDB.SetNoAJAXJSON(const Value: boolean);
begin
  inherited;
  if Value=NoAJAXJSON then exit;
  fDB.Cache.Reset; // we changed the JSON format -> cache must be updated
end;

function TSQLRestServerDB.MainEngineUpdate(TableModelIndex, ID: integer;
  const SentData: RawUTF8): boolean;
begin
  if (TableModelIndex<0) or (ID<=0) then
    result := false else
  if SentData='' then // update with no simple field -> valid no-op
    result := true else begin
    // this SQL statement use :(inlined params): for all values
    result := EngineExecuteFmt('UPDATE % SET % WHERE RowID=:(%):;',
      [Model.TableProps[TableModelIndex].Props.SQLTableName,
       GetJSONObjectAsSQL(SentData,true,true),ID]);
    InternalUpdateEvent(seUpdate,Model.Tables[TableModelIndex],ID,nil);
  end;
end;

function TSQLRestServerDB.MainEngineUpdateBlob(TableModelIndex, aID: integer;

  BlobField: PPropInfo; const BlobData: TSQLRawBlob): boolean;
var SQL: RawUTF8;
    AffectedField: TSQLFieldBits;
begin
  if (aID<0) or (TableModelIndex<0) or not BlobField^.IsBlob then
    result := false else

  with Model.TableProps[TableModelIndex].Props do
  try
    SQL := FormatUTF8('UPDATE % SET %=? WHERE RowID=?;',
             [SQLTableName,BlobField^.Name]);
    DB.Lock(SQL); // UPDATE for a blob field -> no JSON cache flush, but UI refresh
    try
      with fStatementCache.Prepare(SQL)^ do begin
        Bind(1,pointer(BlobData),length(BlobData));
        Bind(2,aID);
        repeat
        until Step<>SQLITE_ROW; // Execute all steps of the first statement
        result := true;
      end;
    finally
      DB.UnLock;
    end;
    FieldIndexsFromBlobField(BlobField,AffectedField);
    InternalUpdateEvent(seUpdateBlob,Table,aID,@AffectedField);
  except
    on ESQLite3Exception do
      result := false;
  end;
end;

function TSQLRestServerDB.MainEngineUpdateField(TableModelIndex: integer;
  const SetFieldName, SetValue, WhereFieldName, WhereValue: RawUTF8): boolean;
var WhereID: integer;
begin
  result := false;
  if (TableModelIndex<0) or (SetFieldName='') then
    exit;
  with Model.TableProps[TableModelIndex].Props do
  if Fields.IndexByName(SetFieldName)>=0 then begin
    WhereID := 0;
    if IsRowID(pointer(WhereFieldName)) then begin
      WhereID := GetInteger(Pointer(WhereValue));
      if (WhereID<=0) or not RecordCanBeUpdated(Table,WhereID,seUpdate) then
        exit; // limitation: will only check for update from RowID
    end else
      if Fields.IndexByName(WhereFieldName)<0 then
        exit;
    result := EngineExecuteFmt('UPDATE % SET %=:(%): WHERE %=:(%):',
      [SQLTableName,SetFieldName,SetValue,WhereFieldName,WhereValue]);
    if WhereID>0 then
      InternalUpdateEvent(seUpdate,Table,WhereID,nil);
  end;
end;

function TSQLRestServerDB.UpdateBlobFields(Value: TSQLRecord): boolean;
var Static: TSQLRestStorage;
    SQL: RawUTF8;
    f: integer;
    data: TSQLVar;
    temp: RawByteString;
begin
  result := false;
  if Value=nil then
................................................................................
        DB.UnLock;
      end;
      InternalUpdateEvent(seUpdateBlob,PSQLRecordClass(Value)^,Value.ID,@BlobFieldsBits);
    end else
      result := true; // as TSQLRest.UpdateblobFields()
end;





























procedure TSQLRestServerDB.Commit(SessionID: cardinal=1);
begin
  inherited Commit(SessionID); // reset fTransactionActive + write all TSQLVirtualTableJSON
  try
    DB.Commit;
  except
    on ESQLite3Exception do
................................................................................
  result := nil;
  n := length(Tables);
  if (self<>nil) and (n>0) then
  try // will use JSON cache if available:
    aSQL := Model.SQLFromSelectWhere(Tables,SQLSelect,SQLWhere);
    if n=1 then
      // InternalListJSON will handle both static and DB tables
      result := fServer.ExecuteList(Tables,aSQL) else
      // we access localy the DB -> TSQLTableDB handle Tables parameter
      result := TSQLTableDB.Create(fServer.DB,Tables,aSQL,not fServer.NoAJAXJSON);
    if fServer.DB.InternalState<>nil then
      result.InternalState := fServer.DB.InternalState^;
  except
    on ESQLite3Exception do
      result := nil;

Changes to SynDB.pas.

1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
....
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
....
4995
4996
4997
4998
4999
5000
5001
5002
5003
5004
5005
5006
5007
5008
5009
    // - any further call will use this internal list, so response will be
    // immediate
    // - the whole foreign key list is shared by all connections
    function GetForeignKey(const aTableName, aColumnName: RawUTF8): RawUTF8; 

    /// adapt the LIMIT # clause in the SQL SELECT statement to a syntax
    // matching the underlying DBMS
    // - e.g. TSQLRestServerStaticExternal.AdaptSQLForEngineList() calls this
    // to let TSQLRestServer.URI by-pass virtual table mechanism
    // - integer parameters state how the SQL statement has been analysed
    function AdaptSQLLimitForEngineList(var SQL: RawUTF8;
      LimitRowCount, AfterSelectPos, WhereClausePos, LimitPos: integer): boolean; virtual;
    /// determine if the SQL statement can be cached
    // - used by TSQLDBConnection.NewStatementPrepared() for handling cache
    function IsCachable(P: PUTF8Char): boolean; virtual;
................................................................................
    // & [ {"col1":val11,"col2":"val12"},{"col1":val21,... ]
    // - if Expanded is false, JSON data is serialized (used in TSQLTableJSON)
    // & { "FieldCount":1,"Values":["col1","col2",val11,"val12",val21,..] }
    // - BLOB field value is saved as Base64, in the '"\uFFF0base64encodedbinary"'
    // format and contains true BLOB data
    // - similar to corresponding TSQLRequest.Execute method in SynSQLite3 unit
    // - returns the number of row data returned (excluding field names)
    // - warning: TSQLRestServerStaticExternal.EngineRetrieve in mORMotDB unit
    // expects the Expanded=true format to return '[{...}]'#10
    function FetchAllToJSON(JSON: TStream; Expanded: boolean;
      DoNotFletchBlobs: Boolean=false): PtrInt;
    // Append all rows content as a CSV stream
    // - CSV data is added to the supplied TStream, with UTF-8 encoding
    // - if Tab=TRUE, will use TAB instead of ',' between columns
    // - you can customize the ',' separator - use e.g. the global ListSeparator
................................................................................
    result := inherited GetMainConnection;
  else
    result := nil;
  end;
end;

{
  tmBackgroundThread should handle TSQLRestServerStaticExternal methods:
  Create: ServerTimeStamp+GetFields
  BeginTransaction
  Commit
  Rollback
  InternalBatchStop: PrepareSQL+BindArray+ExecutePrepared
  EngineUpdateBlob:  PrepareSQL+Bind/BindNull+ExecutePrepared
  ExecuteDirect:     PrepareSQL+Bind+ExecutePrepared







|







 







|







 







|







1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
....
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
....
4995
4996
4997
4998
4999
5000
5001
5002
5003
5004
5005
5006
5007
5008
5009
    // - any further call will use this internal list, so response will be
    // immediate
    // - the whole foreign key list is shared by all connections
    function GetForeignKey(const aTableName, aColumnName: RawUTF8): RawUTF8; 

    /// adapt the LIMIT # clause in the SQL SELECT statement to a syntax
    // matching the underlying DBMS
    // - e.g. TSQLRestStorageExternal.AdaptSQLForEngineList() calls this
    // to let TSQLRestServer.URI by-pass virtual table mechanism
    // - integer parameters state how the SQL statement has been analysed
    function AdaptSQLLimitForEngineList(var SQL: RawUTF8;
      LimitRowCount, AfterSelectPos, WhereClausePos, LimitPos: integer): boolean; virtual;
    /// determine if the SQL statement can be cached
    // - used by TSQLDBConnection.NewStatementPrepared() for handling cache
    function IsCachable(P: PUTF8Char): boolean; virtual;
................................................................................
    // & [ {"col1":val11,"col2":"val12"},{"col1":val21,... ]
    // - if Expanded is false, JSON data is serialized (used in TSQLTableJSON)
    // & { "FieldCount":1,"Values":["col1","col2",val11,"val12",val21,..] }
    // - BLOB field value is saved as Base64, in the '"\uFFF0base64encodedbinary"'
    // format and contains true BLOB data
    // - similar to corresponding TSQLRequest.Execute method in SynSQLite3 unit
    // - returns the number of row data returned (excluding field names)
    // - warning: TSQLRestStorageExternal.EngineRetrieve in mORMotDB unit
    // expects the Expanded=true format to return '[{...}]'#10
    function FetchAllToJSON(JSON: TStream; Expanded: boolean;
      DoNotFletchBlobs: Boolean=false): PtrInt;
    // Append all rows content as a CSV stream
    // - CSV data is added to the supplied TStream, with UTF-8 encoding
    // - if Tab=TRUE, will use TAB instead of ',' between columns
    // - you can customize the ',' separator - use e.g. the global ListSeparator
................................................................................
    result := inherited GetMainConnection;
  else
    result := nil;
  end;
end;

{
  tmBackgroundThread should handle TSQLRestStorageExternal methods:
  Create: ServerTimeStamp+GetFields
  BeginTransaction
  Commit
  Rollback
  InternalBatchStop: PrepareSQL+BindArray+ExecutePrepared
  EngineUpdateBlob:  PrepareSQL+Bind/BindNull+ExecutePrepared
  ExecuteDirect:     PrepareSQL+Bind+ExecutePrepared

Changes to SynSelfTests.pas.

427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
...
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
...
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
....
7163
7164
7165
7166
7167
7168
7169
7170
7171
7172
7173
7174
7175
7176
7177
....
7320
7321
7322
7323
7324
7325
7326
7327
7328
7329
7330
7331
7332
7333
7334
....
7932
7933
7934
7935
7936
7937
7938
7939
7940
7941
7942
7943
7944
7945
7946
....
8111
8112
8113
8114
8115
8116
8117
8118
8119
8120
8121
8122
8123
8124
8125
....
8166
8167
8168
8169
8170
8171
8172
8173
8174
8175
8176
8177
8178
8179
8180
8181
8182
8183
8184
8185
8186
8187
8188
8189
8190
....
8456
8457
8458
8459
8460
8461
8462
8463
8464
8465
8466
8467
8468
8469
8470
    // - the JSON content generated must match the original data
    // - a VACCUM is performed, for testing some low-level SQLite3 engine
    // implementation
    // - the SortField feature is also tested
    procedure _TSQLTableJSON;
    /// test the TSQLRestClientDB, i.e. a local Client/Server driven usage
    // of the framework
    // - validates TSQLModel, TSQLRestServer and TSQLRestServerStatic by checking
    // the coherency of the data between client and server instances, after
    // update from both sides
    // - use all RESTful commands (GET/UDPATE/POST/DELETE...)
    // - test the 'many to many' features (i.e. TSQLRecordMany) and dynamic
    // arrays published properties handling
    // - also test FTS implementation if INCLUDE_FTS3 conditional is defined
    // - test dynamic tables
................................................................................
  // implemented in the mORMotSQLite3 unit, i.e. the SQLite3 engine itself,
  // with a file-based approach
  TTestFileBased = class(TTestSQLite3Engine);

  /// this test case will test most functions, classes and types defined and
  // implemented in the mORMotSQLite3 unit, i.e. the SQLite3 engine itself,
  // with a memory-based approach
  // - this class will also test the TSQLRestServerStatic class, and its
  // 100% Delphi simple database engine
  TTestMemoryBased = class(TTestSQLite3Engine)
  published
    /// check RTREE virtual tables
    procedure _RTree;
  end;

................................................................................
    /// check the SQL auto-adaptation features
    procedure AutoAdaptSQL;
    /// check the per-db encryption
    // - the testpass.db3-wal file is not encrypted, but the main
    // testpass.db3 file will (after the first 1024 bytes)
    procedure CryptedDatabase;
    /// test external DB implementation via faster REST calls
    // - will mostly call directly the TSQLRestServerStaticExternal instance,
    // bypassing the Virtual Table mechanism of SQLite3
    procedure ExternalViaREST;
    /// test external DB implementation via slower Virtual Table calls
    // - using the Virtual Table mechanism of SQLite3 is more than 2 times
    // slower than direct REST access
    procedure ExternalViaVirtualTable;
    {$ifndef CPU64}
................................................................................
  public
    property GUID: TGUID read fGUID write fGUID;
    property GUIDStr: TGUID read fGUIDStr write fGUIDStr;
  end;

  // class hooks to force DMBS property for TTestExternalDatabase.AutoAdaptSQL
  TSQLDBConnectionPropertiesHook = class(TSQLDBConnectionProperties);
  TSQLRestServerStaticExternalHook = class(TSQLRestServerStaticExternal);

class procedure TSQLRecordCustomProps.InternalRegisterCustomProperties(Props: TSQLRecordProperties);
begin
  Props.RegisterCustomFixedSizeRecordProperty(self,sizeof(TGUID),'GUID',
    @TSQLRecordCustomProps(nil).fGUID);
end;

................................................................................
var Props: TSQLDBConnectionProperties;
    SQL: RawUTF8;
begin
  Props := TSQLDBSQLite3ConnectionProperties.Create(SQLITE_MEMORY_DATABASE_NAME,'','','');
  try
    Check(VirtualTableExternalRegister(fExternalModel,TSQLRecordPeopleExt,
      Props,'SampleRecord'));
    with TSQLRestServerStaticExternalHook.Create(TSQLRecordPeopleExt,nil) do
    try
      SQL := SQLOrigin;
      TSQLDBConnectionPropertiesHook(Props).fDBMS := aDBMS;
      Check((Props.DBMS=aDBMS)or(aDBMS=dUnknown));
      Check(AdaptSQLForEngineList(SQL)=AdaptShouldWork);
      Check(IdemPropNameU(SQL,SQLExpected)or not AdaptShouldWork);
    finally
................................................................................
{$ifndef LVCL}
    VO: TSQLRecordPeopleObject;
{$endif}
    FV: TFV;
    ModelC: TSQLModel;
    Client: TSQLRestClientDB;
    Server: TSQLRestServer;
    aStatic: TSQLRestServerStaticInMemory;
    Curr: Currency;
    DaVinci, s: RawUTF8;
    Refreshed: boolean;
    J: TSQLTableJSON;
    i, n, nupd, ndx: integer;
    IntArray, Results: TIntegerDynArray;
    List: TObjectList;
................................................................................
  end;
end;
{$endif}
procedure TestVirtual(aClient: TSQLRestClient; DirectSQL: boolean; const Msg: string;
  aClass: TSQLRecordClass);
var n, i, ndx: integer;
    VD, VD2: TSQLRecordDali1;
    Static: TSQLRestServerStatic;
begin
  Client.Server.StaticVirtualTableDirect := DirectSQL;
  Check(Client.Server.EngineExecuteAll(FormatUTF8('DROP TABLE %',[aClass.SQLTableName])));
  Client.Server.CreateMissingTables(0);
  VD := aClass.Create as TSQLRecordDali1;
  try
    if aClient.TransactionBegin(aClass) then
................................................................................
      Check(aClient.TableRowCount(aClass)=1001);
      aClient.Commit; // write to file
      // try to read directly from file content
      Static := Client.Server.StaticVirtualTable[aClass];
      if CheckFailed(Static<>nil) then
        exit;
      if Static.FileName<>'' then begin // no file content if ':memory' DB
        (Static as TSQLRestServerStaticInMemoryExternal).
          UpdateFile; // force update (COMMIT not always calls xCommit)
        Static := TSQLRestServerStaticInMemoryExternal.Create(aClass,nil,Static.FileName,
          aClass=TSQLRecordDali2);
        try
          Check(TSQLRestServerStaticInMemory(Static).Count=n);
          for i := 1 to n do begin
            ndx := TSQLRestServerStaticInMemory(Static).IDToIndex(i);
            if CheckFailed(ndx>=0) then
              continue;
            VD2 := TSQLRestServerStaticInMemory(Static).Items[ndx] as TSQLRecordDali1;
            if CheckFailed(VD2<>nil) then
              continue;
            Check(VD2.ID=i);
            Check(IdemPChar(pointer(VD2.FirstName),'SALVADOR'));
            Check(VD2.YearOfBirth=1904+i);
            Check(VD2.YearOfDeath=1989+i);
          end;
................................................................................
        Server := TSQLRestServerTest.Create(ModelC,false);
        try
          Server.NoAJAXJSON := true;
          DeleteFile('People.json');
          DeleteFile('People.data');
          Server.StaticDataCreate(TSQLRecordPeople,'People.data',true);
          JS := Demo.ExecuteJSON('SELECT * From People');
          aStatic := Server.StaticDataServer[TSQLRecordPeople] as TSQLRestServerStaticInMemory;
          Check(aStatic<>nil);
          aStatic.LoadFromJSON(JS); // test Add()
          for i := 0 to aStatic.Count-1 do begin
            Check(Client.Retrieve(aStatic.ID[i],V),'test statement+bind speed');
            Check(V.SameRecord(aStatic.Items[i]),'static retrieve');
          end;
          // test our 'REST-minimal' SELECT statement SQL engine







|







 







|







 







|







 







|







 







|







 







|







 







|







 







|

|


|

|


|







 







|







427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
...
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
...
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
....
7163
7164
7165
7166
7167
7168
7169
7170
7171
7172
7173
7174
7175
7176
7177
....
7320
7321
7322
7323
7324
7325
7326
7327
7328
7329
7330
7331
7332
7333
7334
....
7932
7933
7934
7935
7936
7937
7938
7939
7940
7941
7942
7943
7944
7945
7946
....
8111
8112
8113
8114
8115
8116
8117
8118
8119
8120
8121
8122
8123
8124
8125
....
8166
8167
8168
8169
8170
8171
8172
8173
8174
8175
8176
8177
8178
8179
8180
8181
8182
8183
8184
8185
8186
8187
8188
8189
8190
....
8456
8457
8458
8459
8460
8461
8462
8463
8464
8465
8466
8467
8468
8469
8470
    // - the JSON content generated must match the original data
    // - a VACCUM is performed, for testing some low-level SQLite3 engine
    // implementation
    // - the SortField feature is also tested
    procedure _TSQLTableJSON;
    /// test the TSQLRestClientDB, i.e. a local Client/Server driven usage
    // of the framework
    // - validates TSQLModel, TSQLRestServer and TSQLRestStorage by checking
    // the coherency of the data between client and server instances, after
    // update from both sides
    // - use all RESTful commands (GET/UDPATE/POST/DELETE...)
    // - test the 'many to many' features (i.e. TSQLRecordMany) and dynamic
    // arrays published properties handling
    // - also test FTS implementation if INCLUDE_FTS3 conditional is defined
    // - test dynamic tables
................................................................................
  // implemented in the mORMotSQLite3 unit, i.e. the SQLite3 engine itself,
  // with a file-based approach
  TTestFileBased = class(TTestSQLite3Engine);

  /// this test case will test most functions, classes and types defined and
  // implemented in the mORMotSQLite3 unit, i.e. the SQLite3 engine itself,
  // with a memory-based approach
  // - this class will also test the TSQLRestStorage class, and its
  // 100% Delphi simple database engine
  TTestMemoryBased = class(TTestSQLite3Engine)
  published
    /// check RTREE virtual tables
    procedure _RTree;
  end;

................................................................................
    /// check the SQL auto-adaptation features
    procedure AutoAdaptSQL;
    /// check the per-db encryption
    // - the testpass.db3-wal file is not encrypted, but the main
    // testpass.db3 file will (after the first 1024 bytes)
    procedure CryptedDatabase;
    /// test external DB implementation via faster REST calls
    // - will mostly call directly the TSQLRestStorageExternal instance,
    // bypassing the Virtual Table mechanism of SQLite3
    procedure ExternalViaREST;
    /// test external DB implementation via slower Virtual Table calls
    // - using the Virtual Table mechanism of SQLite3 is more than 2 times
    // slower than direct REST access
    procedure ExternalViaVirtualTable;
    {$ifndef CPU64}
................................................................................
  public
    property GUID: TGUID read fGUID write fGUID;
    property GUIDStr: TGUID read fGUIDStr write fGUIDStr;
  end;

  // class hooks to force DMBS property for TTestExternalDatabase.AutoAdaptSQL
  TSQLDBConnectionPropertiesHook = class(TSQLDBConnectionProperties);
  TSQLRestStorageExternalHook = class(TSQLRestStorageExternal);

class procedure TSQLRecordCustomProps.InternalRegisterCustomProperties(Props: TSQLRecordProperties);
begin
  Props.RegisterCustomFixedSizeRecordProperty(self,sizeof(TGUID),'GUID',
    @TSQLRecordCustomProps(nil).fGUID);
end;

................................................................................
var Props: TSQLDBConnectionProperties;
    SQL: RawUTF8;
begin
  Props := TSQLDBSQLite3ConnectionProperties.Create(SQLITE_MEMORY_DATABASE_NAME,'','','');
  try
    Check(VirtualTableExternalRegister(fExternalModel,TSQLRecordPeopleExt,
      Props,'SampleRecord'));
    with TSQLRestStorageExternalHook.Create(TSQLRecordPeopleExt,nil) do
    try
      SQL := SQLOrigin;
      TSQLDBConnectionPropertiesHook(Props).fDBMS := aDBMS;
      Check((Props.DBMS=aDBMS)or(aDBMS=dUnknown));
      Check(AdaptSQLForEngineList(SQL)=AdaptShouldWork);
      Check(IdemPropNameU(SQL,SQLExpected)or not AdaptShouldWork);
    finally
................................................................................
{$ifndef LVCL}
    VO: TSQLRecordPeopleObject;
{$endif}
    FV: TFV;
    ModelC: TSQLModel;
    Client: TSQLRestClientDB;
    Server: TSQLRestServer;
    aStatic: TSQLRestStorageInMemory;
    Curr: Currency;
    DaVinci, s: RawUTF8;
    Refreshed: boolean;
    J: TSQLTableJSON;
    i, n, nupd, ndx: integer;
    IntArray, Results: TIntegerDynArray;
    List: TObjectList;
................................................................................
  end;
end;
{$endif}
procedure TestVirtual(aClient: TSQLRestClient; DirectSQL: boolean; const Msg: string;
  aClass: TSQLRecordClass);
var n, i, ndx: integer;
    VD, VD2: TSQLRecordDali1;
    Static: TSQLRestStorage;
begin
  Client.Server.StaticVirtualTableDirect := DirectSQL;
  Check(Client.Server.EngineExecuteAll(FormatUTF8('DROP TABLE %',[aClass.SQLTableName])));
  Client.Server.CreateMissingTables(0);
  VD := aClass.Create as TSQLRecordDali1;
  try
    if aClient.TransactionBegin(aClass) then
................................................................................
      Check(aClient.TableRowCount(aClass)=1001);
      aClient.Commit; // write to file
      // try to read directly from file content
      Static := Client.Server.StaticVirtualTable[aClass];
      if CheckFailed(Static<>nil) then
        exit;
      if Static.FileName<>'' then begin // no file content if ':memory' DB
        (Static as TSQLRestStorageInMemoryExternal).
          UpdateFile; // force update (COMMIT not always calls xCommit)
        Static := TSQLRestStorageInMemoryExternal.Create(aClass,nil,Static.FileName,
          aClass=TSQLRecordDali2);
        try
          Check(TSQLRestStorageInMemory(Static).Count=n);
          for i := 1 to n do begin
            ndx := TSQLRestStorageInMemory(Static).IDToIndex(i);
            if CheckFailed(ndx>=0) then
              continue;
            VD2 := TSQLRestStorageInMemory(Static).Items[ndx] as TSQLRecordDali1;
            if CheckFailed(VD2<>nil) then
              continue;
            Check(VD2.ID=i);
            Check(IdemPChar(pointer(VD2.FirstName),'SALVADOR'));
            Check(VD2.YearOfBirth=1904+i);
            Check(VD2.YearOfDeath=1989+i);
          end;
................................................................................
        Server := TSQLRestServerTest.Create(ModelC,false);
        try
          Server.NoAJAXJSON := true;
          DeleteFile('People.json');
          DeleteFile('People.data');
          Server.StaticDataCreate(TSQLRecordPeople,'People.data',true);
          JS := Demo.ExecuteJSON('SELECT * From People');
          aStatic := Server.StaticDataServer[TSQLRecordPeople] as TSQLRestStorageInMemory;
          Check(aStatic<>nil);
          aStatic.LoadFromJSON(JS); // test Add()
          for i := 0 to aStatic.Count-1 do begin
            Check(Client.Retrieve(aStatic.ID[i],V),'test statement+bind speed');
            Check(V.SameRecord(aStatic.Items[i]),'static retrieve');
          end;
          // test our 'REST-minimal' SELECT statement SQL engine