You are not logged in.
Hey Y'All,
NOTE: I'm duplicating this thread from the Lazarus forums because, maybe, I can get a quicker answer here
https://forum.lazarus.freepascal.org/in … 476.0.html
While embarking on the task of upgrading the backend side of OPM, I've decided I'm going to FINALLY start using mORMot 2.0!!
I've started my experimentation and I got to the point where I'm serving some content from a temporary SQLite database.
The idea is to use PostgreSQL in production.
I'm a bit surprised by some of the conventions that mORMot 2.0 imposes. I was counting on it being opinionated, like any other framework, but
nonetheless, some of the opinions are rather strange from someone that is used to CakePHP, that takes the same opinionated ideas from Ruby on Rails.
All this long winded blah-blah to get to my immediate questions:
1. How do I force the table names for each model?
I much prefer that both the tables and the fields be in lower case. I also like my models to be in the singular, but my tables to be in the plural.
Yeah, I know, I'm a bloody whiner, but I'm old and I have my own conventions
2. How do I change the case for the JSON that is served?
I'm rather spoiled by the fact that in Go I can define the JSON name for each field, and I'm wondering if I can do the same with mORMot 2.0
3. Is there a way to do database migrations outside the main application?
I ask this because I got, kinda, spoiled from using CakePHP's migrations. With these you can do up/down quite easily, and all those migrations
are just another source file you add to the source control. On top of that, they are timestamped and some info is stored in the database itself, to
track which ones have been applied.
It would be rather nice to have something of this nature to use.
I think this is enough for now. When I have some other doubts, I'll keep adding to this thread.
Cheers,
Gus
Offline
It is difficult to understand what you really need.
Are you willing to use interface-based services?
Then, you could use existing SQL easily, returning the JSON as you require.
The use of mORMot ORM is not mandatory at all, especially if it does not match your existing DB schema.
Offline
Hey Arnaud,
It is difficult to understand what you really need.
Yeah, I guess it is!!
I'm throwing at you stuff from CakePHP and Go, which is a bit confusing if I'm just mentioning it because I'm a whiner
Are you willing to use interface-based services?
Then, you could use existing SQL easily, returning the JSON as you require.
The use of mORMot ORM is not mandatory at all, especially if it does not match your existing DB schema.
That's a bit too advanced at the moment for me. I quite like that mORMmot 2.0 does it's own database schema, and I would like it to continue like that!! I HATE writing SQL!!!
Like I said on the message, I'm coming into mORMmot 2.0 with a ton of bad habits that I picked up from Ruby on Rails based frameworks like CakePHP.
I would like to be able to use those bad habits, but if it's too much fussing about, I'll just bite the bullet and conform to the new norms that are the mORMmot 2.0 conventions.
I'm not willing to mess with the auto-magic that comes with mORMmot 2.0. That, in my humble opinion, would be contrary to a speedy code production.
I'll have to live with these new norms. In the end, it's not that much of a hassle!!
What about the question about database migration?
Anything, remotely, resembling the CakePHP ones?
Or is it done differently on this side of the fence?
Many thanks for your attention and time, Arnaud!!
It's IMMENSELY appreciated !!
Cheers,
Gus
Offline
Hey Arnaud,
Complete forgot to address this:
The use of mORMot ORM is not mandatory at all, especially if it does not match your existing DB schema.
To me it doesn't make sense to embark on creating a completely new backend for OPM, if it's not done in Object Pascal.
And the biggest boy in terms of Object Pascal and RESTful is mORMmot 2.0!!!
So, it's pretty much a no brainer
Even the frontend is being done with fp-web as the core of it.
I'm also gonna use HTMX and possibly Alpine.js. Welp, not me, but the friend that is taking part of the frontend. I'm backend material
Cheers,
Gus
Offline
Some remarks:
If your database starts from scratch, then you can use mORMot ORM.
Then the tables or columns names could be "mapped" using the external ORM feature.
Switching from one DB to another is just a matter of initialization at runtime.
A good practice, when using interface-based-services, is to use:
1) a "Clean" architecture with a application-level API: do not leak your domain objects
2) define some DTOs for this app API - it is easy to use records with TRttiMap from mormot.core.rtti
Offline
Hey Arnaud,
If your database starts from scratch, then you can use mORMot ORM.
Then the tables or columns names could be "mapped" using the external ORM feature.
Switching from one DB to another is just a matter of initialization at runtime.
Yes, indeed I'm starting from scratch!!
And I want mORMmot to take care of all the SQL as necessary.
I want to only do ANY SQL at all, when there's an impossibility of doing it via mORMmot!!
A good practice, when using interface-based-services, is to use:
1) a "Clean" architecture with a application-level API: do not leak your domain objects
2) define some DTOs for this app API - it is easy to use records with TRttiMap from mormot.core.rtti
Hummm... I'm guessing that I'll probably have to plunge into this at some time in the future, once the entire things gets a level of complexity that the basics can't handle.
At the moment this is waaayyy above my head.
Nonetheless, I'm glad to have some tips on the subject. Thank you so very much Arnaud!!
My intentions are to create your run of the mill RESTful API based on CRUD.
The website/frontend will consume it's data from the above API, and will, serve it as HTML.
In the future, if anyone wants to create a mobile app or anything that is not the main site, they can always consume the RESTful API.
This is basically what the big boys are doing, so I'm pinching their ideas
Cheers,
Gus
Offline
Hey Arnaud,
I've left the migrations question on the back burner of my brain and I guess I sussed it out...
In mORMmot, every time we launch a new version of the executable, it will migrate the database to reflect the current state of the TOrm* classes.
This means it will add, update, and hopefully remove, fields and field types according to the TOrm* class current fields.
If this is true, it will at least fill in my requirements of having pseudo migrations that will go up/down by themselves and are tracked via the version control.
The only thing I'm unsure is if I remove a TOrm* class all together, will it DROP that table on the database?
I guess that dropping it by default may be undesirable, due to accidental loss of data... right... something to ponder about
I started with the TAuth* tables when I had security on. Then I switch the security off, and the tables remained... I guess that answers a bit of my question...
So, yeah, dropping tables is always a no-no, at least when it's done automagically!!
I think this is kinda settled, now.
And, if I'm not mistaken, that takes care of all my initial questions!!
I'm gonna have some questions about CLI tools using the bare minimum to access the database and do cronjobs in the future, maybe.
I guess I can stop being a lazy bastard and bang my head against the wall for a change. Less answering dumb questions on your side
Cheers,
Gus
Offline
Adding fields is supported.
Update the name or remove a field is not.
If you don't put an TOrm class to the TOrmModel, it won't indeed drop the table in the database.
Note that if you define the database, also consider using variant fields, and store JSON in it using TDocVariantData.
Consider reading the "aggregate" pattern in the documentation: it fits well the mORMot ORM patterns.
All this is documented in https://synopse.info/files/html/Synopse … 01.18.html
Offline
Hey Arnaud,
Adding fields is supported.
Update the name or remove a field is not.
If you don't put an TOrm class to the TOrmModel, it won't indeed drop the table in the database.
Good to know !!!
I'm now a bit sad to loose the database migration freedom from CakePHP, but it is what it is and I can't keep whining about it
Hummm... had a brain fart:
There are 2 directions on how an ORM app works:
1. The code adapts to the SCHEMA.
2. The SCHEMA adapts to the code.
Since PHP is a dynamic language, adapting the code to the SCHEMA is rather easy, so they use that direction.
But... and here's the but: In compiled land, we cannot do that, so we have to do the opposite, the SCHEMA adapts to the code.
Which means that I've been whining about something that is not even possible, or it's rather complicated, to do with the compiled approach!! DOH !!!!!!
OK, NOW it makes more sense!!!
Note that if you define the database, also consider using variant fields, and store JSON in it using TDocVariantData.
Consider reading the "aggregate" pattern in the documentation: it fits well the mORMot ORM patterns.All this is documented in https://synopse.info/files/html/Synopse … 01.18.html
OK, at this present moment, this kinda goes over my head. Nonetheless, I promise to read the relevant parts of the docs about the aggregate pattern!!
I'm also quite tired right now, so I'll leave it for after I've had some rest!!
For the time being I've been following the code suggested in my session with ChatGPT and some small bits from the ex/MVC example.
Between the two, I was able to get where I am now.
Alas, the MVC example produces HTML pages. My WEB API will only produce JSON...
Still, I need to dig deeper into that example because there might be some gems that I'm missing!!
Again, cannot be said enough: Many thanks for dedicating some of your precious time entertaining a complete noob in mORMmot!!!
I really appreciate every moment of it !!
Cheers,
Gus
Offline
You can use a TPersistent (or TSynPersistent) as a variable schema field. TProductDetails is stored as JSON and can be changed at any time.
TProductDetails = class(TPersistent)
private
fUFI : RawUTF8;
//fLabInfo : TLabInfo;
published
property UFI : RawUTF8 read fUFI write fUFI; // format : UFI: N1QV-R02N-J00M-WQD5
//property LabInfo : TLabInfo read fLabInfo;
end;
TOrmProduct = class(TOrm)
private
fCode : TMainField;
fBrand : RawUTF8;
fDetails : TProductDetails;
public
procedure InternalCreate;override;
destructor Destroy; override;
published
property Code : TMainField index 30 read fCode write fCode stored AS_UNIQUE;
property Brand : RawUTF8 read fBrand write fBrand;
property Details : TProductDetails read fDetails write fDetails;
end;
Offline
hi gcarreno,
Alas, the MVC example produces HTML pages. My WEB API will only produce JSON...
Append `/json` at the end of the url to see what happen (it was primarly made for debugging purpose, but it can be consumed as is too).
What about the question about database migration?
I wrote some parts of rock migration, a module used by ProcessWire community (I am pretty sure you know it) so I can clearly see what you have in mind. As you already mentioned, things turn a bit different with a compiled approach but not that much I think.
I've just published a very minimal example here showing how to compare current model table with external table fields to remove orphaned columns. Then, imagine you set up a proper ci/cd pipelnie for example with jenkins and fpc, you could write migration files to be consumed by your flow, compile, deploy. And as mormot is able to serialize any class/record into a json object, writing a migration module similar to what you’d have on cakephp to keep or rollback your db/api definition should be ok. Anyway, nothing native yet
Hope this points you in more or less the right direction
Last edited by flydev (Yesterday 08:45:14)
Offline
Don't look at the MVC sample.
It is for HTML output.
You may tweak it with /json at the end of each URI but it is clearly a hack.
Use interface-base services instead, for your project.
Offline
Hey flydev,
Hope this points you in more or less the right direction
I had a look at the code and it might give me some ideas for the future.
Thank you so very much for that!!
At the moment I'm still trying not to drown with the deluge of info being thrown at me, so I'll pin your message for a future investigation!!
Cheers,
Gus
Offline
Hey Arnaud,
Don't look at the MVC sample.
It is for HTML output.
You may tweak it with /json at the end of each URI but it is clearly a hack.
If I'm perfectly honest, the only thing I took from the MVC example was some code structure and the logging use.
The interface thing, they have there, got me a bit thrown off, so I didn't look into that further.
Use interface-base services instead, for your project.
You've mentioned this more than once. Is there a code example that you can point me to, please?
At the moment I have two very simple endpoints:
- /api/v1/packages
- /api/v1/categories
That's about the amount of progress I've gone through.
You have already pointed me to the aggregate pattern from the docs. I'm sorry, but I haven't had the opportunity to look at that yet.
Nonetheless, a simple-ish example of interface based models would be quite appreciated!!
Cheers,
Gus
Last edited by gcarreno (Today 02:12:43)
Offline
Hey DonAlfredo,
You can use a TPersistent (or TSynPersistent) as a variable schema field. TProductDetails is stored as JSON and can be changed at any time.
While this still goes over my head at my present understanding of what mORMmot can do and what is it's advanced usage, I do appreciate the tip!!
I'm sure that this piece of the puzzle will click into place, further along my journey into mORMmot.
Thank you ever so much for it!!
You see... My problem is that I'm very familiar with Go and PHP framework with their ORM/CRUD/RESTful features.
Alas, this does not translate 1:1 onto mORMmot. This is getting pretty obvious to me, now.
This means that I'm gonna try really hard to, unconsciously, map a feature from what I'm used to, into what mORMmot has to offer.
This will probably fail most of the time, mainly because my past experience has not been in mORMmot, but in frameworks that are not this huge.
They give simpler routes to success.
Of course, they are not as powerful as mORMmot, hence the mentioned simplicity
Cheers,
Gus
Offline
Hey Y'All,
My plan is to have the frontend be kind'of a proxy to the backend I'm working on.
The frontend is being designed with MVC in mind, so the Models for said frontend will be the pieces of code that will consume data from the backend.
As I mentioned above, I've only got to the point of setting up 2 endpoints. REMEMBER that I'm at step 1 or 2 of what will be a multi tens of thousands of kilometres of a journey!!!
Since I'm still kinda feeling in the dark to get my bearings in mORMot, those endpoints are the most basic thing you can do:
- Write the published properties.
- Feed it to the ORM engine.
- Set the HTTPRest and RestDB servers up to get some JSON out.
Next I need to have the ManyToMany model between Packages and Categories thrown into the mix.
And I think the previously mentioned MVC example can help me, since they have TOrmArticles, TOrmTags, and TOrmArticlesTags, which maps perfectly into my Packages and Categories.
I've even turned off security all together, in order to just have quicker testing conditions.
But in the future, the frontend will proxy all login, logout and data reading/modification into the backend.
This means, that by that time, I'll need to have good Authentication, but most importantly, good Authorisation!!!
There's the added headache of how to accept input from the frontend and then route it to the backend, instead of doing it directly, but that's a bridge my partner and I will have to cross in the future. And yes, either one of us will probably be coming here to bug you!!!
After getting the ManyToMany relationship done, I need to work on a CLI tool to slurp the current package data and drop it into the new mORMot models. This will be temporary, just to keep the new database up to date with the new packages, while we're still building stuff.
This will give me the knowledge of how to use, only, the ORM part of mORMot. It will probably give me knowledge of the mORMot JSON libs. I'm used to fp-json, but why use that when I can keep within the framework.
And if mORMot has an HTTPClient, maybe I can use that too instead of TFPHTTPClient. Like I said, keep it inside the framework
In the future, this will allow me to do all sorts of cron jobs for database maintenance, or even other time related stuff.
SOOOOOOO, while I DO appreciate all the hints all of you have been offering, I would like to ask if you can dumb it down a lill And, believe you me, I do appreciate all the help.
It's just that at my current understanding of all things mORMot, it's been more confusing than helpful.
I'm not ashamed to say it: I need a bit of hand holding here!!
Especially with the advanced stuff. Basic I think I can manage, but the advanced stuff is, maybe, even above my past knowledge on other frameworks.
Yeah, keep the tips coming, but please hand them to me gently
Cheers,
Gus
Offline
Hey Arnaud,
Please correct me if I'm wrong, but while exploring the Third Party Examples, I found Example #4 from Martin Doyle, and this looks like the Interface based services you were talking about, right?
Martin Doyle - Example 4 - Interfaced Based Services
I think this allows me more control over the bare bones that mORMot offers from the start. If I'm reading it correctly, since all endpoints will be the methods listed in the Interface.
Ok, found that, but I'll put a pin on it until I progress a bit more into the code.
Like I said: Very overwhelmed at the moment. Especially trying to migrate old concepts and conciliating them with mORMot.
I'll get there...
Cheers,
Gus
Last edited by gcarreno (Today 07:06:53)
Offline
Look at the mORMot 1 documentation about interface-based services.
It almost did not change (but some naming).
https://synopse.info/files/html/Synopse … ml#TITL_63
Offline
Hey Arnaud,
First of all, let me thank you for being polite and not shout at me: READ THE EFFING MANUAL!!!
I guess, I AM being that lame, sorry!!
Look at the mORMot 1 documentation about interface-based services.
It almost did not change (but some naming).
https://synopse.info/files/html/Synopse … ml#TITL_63
Okies, I've started reading that, and with the example from Martin Doyle, I think I have my work cut out for the next 2 or 3 hours.
I'll try and read the aggregate pattern next. Maybe that'll give me more insight into what I SHOULD do!!
Again, thanks for being polite and not explode when I'm asking such dumb and basic questions that have answers in the docs!!
Cheers,
Gus
Offline
Hey Arnaud,
Before I attempt a HasManyThrought or ManyToMany, I'm trying a simple HasOne.
I'm getting a bit frustrated, because I'm not sure if it works on class procedure TORMPackages.InitializeTable...
This is my Packages interface:
TORMPackages = class(TORMTimeStamped)
private
FName: RawUtf8;
FDescription: RawUtf8;
FCategory: TORMCategories;
protected
public
class procedure InitializeTable(const Server: IRestOrmServer;
const FieldName: RawUtf8; Options: TOrmInitializeTableOptions); override;
published
property Name: RawUtf8
read FName
write FName;
property Description: RawUtf8
read FDescription
write FDescription;
property Category: TORMCategories
read FCategory
write FCategory;
end;
And this is the implementation of InitializeTable:
class procedure TORMPackages.InitializeTable(const Server: IRestOrmServer;
const FieldName: RawUtf8; Options: TOrmInitializeTableOptions);
var
index: Integer;
ormPackage: TORMPackages;
ormCategory: TORMCategories;
begin
inherited InitializeTable(Server, FieldName, Options);
ConsoleWrite('Init Packages', ccRed);
// Temporary, just to have some stuff to look at
if FieldName = '' then // New table, seed some data
begin
ormPackage:= TORMPackages.Create;
try
for index:= 1 to 100000 do
begin
ormPackage.Name:= Format('Sample Package %d', [index]);
ormPackage.Description:= Format('Description for Sample Package %d', [index]);
ormCategory:= nil;
if Server.Retrieve(1, ormCategory) then
ConsoleWrite(['Category: ', ormCategory.Name], ccRed);
Server.Add(ormPackage, True);
end;
finally
ormPackage.Free;
end;
end;
end;
What am I doing wrong, since I get a false when calling the Retrieve?
And I made sure that the Categories is running the same InitializeTable BEFORE Packages.
Cheers,
Gus
Offline
Do not try to use the relational model like "many to many" or such.
Consider using the "Aggregate" pattern as documented.
That is, do not think in terms of tables, but as business object.
The idea is to put everything you need in a given business context in a single object, with some high-level fields like TDocVariant/variant, TSynPersistent or dynamic array of TSynPersistent.
Those complex fields will be stored as JSON in the database, and it will be very efficient.
In your code, also look at Retrieve() documentation: the ormCategory should not be nil, but an existing TOrmCategory instance.
Offline
Hey Arnaud,
Okidokes... After having a better look at example 4 from Martin Doyle, I know understand why you recommended Interface Based:
-> If I had a mORMot client, it would be super awesome to consume the stuff on the server via the Interfaced Object.
I can see on that example, that the client pretty much just needs to call the Add method, and BOOM, you've added a new entry.
Same with the Find method, to get the server to do SELECT * FROM sample WHERE ....
Alas, this is not the route I want to go!!
I'm planning on having the frontend not being mORMot based at all.
Just plain TFPHTTPClient, or the REST client that comes somewhere in the Free Pascal packages, if I'm not mistaken.
I don't want to tie my client to mORMot.
This will also allow other non Object Pascal clients to consume the API.
While I do appreciate you mentioning the Interfaced Based approach, I now realise that it would only be beneficial when using it with a mORMot client.
I don't think I need to go that route if my plans don't include a mORMot client.
Nonetheless, and again, many thanks for that suggestion !!! If I ever embark on a Server/Cleint both with mORMot, I'll surely use the Interface Based approach!!
Cheers,
Gus
Offline
Hey Arnaud,
After a bit of a chat with ChatGPT and having a look at the comment above TOrmMany, I'm now at this point...
The TORMPackages Interface:
TORMPackages = class(TORMTimeStamped)
private
FName: RawUtf8;
FDescription: RawUtf8;
FCategories: TORMPackagesCategories;
protected
public
class procedure InitializeTable(const Server: IRestOrmServer;
const FieldName: RawUtf8; Options: TOrmInitializeTableOptions); override;
published
property Name: RawUtf8
read FName
write FName;
property Description: RawUtf8
read FDescription
write FDescription;
property Categories: TORMPackagesCategories
read FCategories
write FCategories;
end;
The TORMPackagesCategories Interface:
TORMPackagesCategories = class(TOrmMany)
private
protected
FPackage: TORMPackages;
FCategory: TORMCategories;
public
published
property Source: TORMPackages
read FPackage;
property Dest: TORMCategories
read FCategory;
end;
Adding Packages:
class procedure TORMPackages.InitializeTable(const Server: IRestOrmServer;
const FieldName: RawUtf8; Options: TOrmInitializeTableOptions);
var
index: Integer;
ormPackage: TORMPackages;
begin
inherited InitializeTable(Server, FieldName, Options);
// Temporary, just to have some stuff to look at
if FieldName = '' then // New table, seed some data
begin
ormPackage:= TORMPackages.Create;
try
for index:= 1 to 100000 do
begin
ormPackage.Name:= Format('Sample Package %d', [index]);
ormPackage.Description:= Format('Description for Sample Package %d', [index]);
ormPackage.Categories:= TORMPackagesCategories.Create;
ormPackage.Categories.ManyAdd(Server, index, 1);
Server.Add(ormPackage, True);
end;
finally
ormPackage.Free;
end;
end;
end;
Adding it all to the OrmModel:
FORMModel:= TOrmModel.Create([
TORMCategories,
TORMPackages,
TORMPackagesCategories
], 'api/v1');
This all works fine and all 3 tables have data...
Ok, I'm now baffled has how I can retrieve the association!!
I've tried:
curl "http://localhost:2020/api/v1/packages/1?select=*/"
curl "http://localhost:2020/api/v1/packages/1?with=Categories/"
To no avail...
I know I'm missing something. One more lill step tp make the magic happen, but I'm completely baffled!!
The comment above the TOrmMany mentions a param named through, but I don't know where to use that
Many thanks in advance for still having the patience to answer my utmost noob questions!!
Cheers.
Gus
Offline