Clean architecture with mORMot

Roberto Schneiders · 2013-02-21 13:48:01

I want to start a discussion about clean architecture . This is not a discussion related to mORMot in particular.

After see this lecture from Uncle Bob i started to think about it. Take a look http://vimeo.com/43612849

Initially the architecture of my software would be this way:

We can consider the TSQLHTTPClient as a repository (In Domain Driven Design)? I'm not sure, but I believe so. The biggest problem is that in this case the repository is the infrastructure layer. That seems very wrong. What do you think?
For now I will consider that I can use as a repository.

Back to the topic of clean architecture. One of the things that Uncle Bob talks in the lecture is more or less like this: Your software (your domain) should not know the frameworks. Do not attach your software to frameworks. "Make implementation details swappable."

Considering everything that Uncle Bob said, do you think this architecture is clean?
I do not think so.

In this way the software is strongly linked to the framework. The domain objects directly accesses the framework. The entities inherit from framework classes.
What will happen with your software if you, for whatever reason, need to change the framework?
So, the architecture is not clean.

I'm new at this, I want your help to try to create a clean architecture.

How about creating a abstraction layer? like this

Note that I include in the abstraction layer even the primitive types of mORMot like RawUTF8. How is a specific type of mORMot, if I use it throughout the software when you need to change it, I'll have to change the whole system.

I created a class called TAbstractRecord. This class inherits from TSQLRecord and basically is an empty class. Does absolutely nothing right now. In case of changing the framework I just need to adapt this classes to the new framework and my entities not need to be changed. Obviously it may not be so easy, but it's a start.

Another class of abstraction is the TRepository. This class encapsulates all communication with mORMot objects like TSQLHTTPClient. If the persistence technology change, I need to adapt this class, and not the entire system.

A question about DDD. The class TSQLHTTPClient seems like a generic repository that works for all types of entities. I do not know exactly if this goes against the DDD or not. What do you think?

Say it fits the idea of DDD. Still, there will be cases in which it is better to create specific repository. Especially if you have a aggregate.

In this case I think that the architecture would be like this:

What do you think of these ideas? What do you think by clean architecture?

I still have many questions about DDD and Clean Architecture, hopefully we can resolve some here.

So...
In Domain Driven Design world, thinking about a clean architecture:
1 - We can consider the TSQLHttpClient as a repository?
1.1 - The repository can stay in the infrastructure layer?
1.1 - Is it correct have a generic repository, for all entities?
2 - You think I'm in the right way by creating abstraction layers?

Last edited by Roberto Schneiders (2013-02-21 14:07:33)

ab · 2013-02-21 15:58:55

Some thoughts on this very interesting subject.
Main idea in DDD is "stay uncoupled".
But also: "do not let your domain objects leak"!

The main point about DDD is not about abstraction (i.e. separate layers), but about separation (i.e. uncouple logic).

First of all, ensure you read the latest version of DDD presentation in the 1.18 SAD pdf, and also http://blog.synopse.info/post/2013/01/0 … and-mORMot
If something is not clear enough, feedback is welcome!

1. Considering the TSQLHttpClient as a repository

AFAIK, in DDD the "Persistence agnostic" principle forbid the repository to be implemented neither in the Domain Layer, nor in the Application Layer.
The entities stay in the Domain Layer, as business logic, and use the Persistence layer to implement a "Repository" interface.
So in "pure" DDD, you should never have direct CRUD operation on your TSQLRecord, from your end-user applications.

TSQLRecord are persistence objects, and should be implemented within the Persistence Layer, but "consumed" by the Domain Layer.
You can have a HTTP connection between the Persistence and Domain layers - or something faster like named pipes or GDI messages if both are hosted in the same PC.
In short: access the TSQLRecord instances via CRUD from the Domain layer with an abstract TSQLRest class. It could be in-process, or remote, on need.

TSQLRecord, TSQLModel, and TSQLRest are defined in the Domain layer, but storage is implemented in the Persistence Layer.

BUT...

a) You have another pattern, called CQRS, which allows direct query of the information, probably directly to the repository.
So you could publish your TSQLRecord directly from the persistence layer to your software, using a TSQLHTTPServer e.g., but with the appropriate mORMot's user rights policy, only allowing GET (=read) - direct and easy to work with.
Or you can implement it using a dedicated Query service - more work, but it could be more secure to forbid access to the SQL "where" clause for instance.

b) You may also have direct remote access to the repository, in some special cases.
mORMot gives you remote access in a "standard" way, therefore it could be sometimes better to use it, rather than publish your own services.
For instance, if you need a custom query to the database with an one-the-fly SQL "where" clause, not easily supported by the application layer services.

As a result:

1.1 - The repository should stay in the infrastructure layer - but could be directly accessed, on special needs.
1.2 - A generic repository, for all entities, is exactly what TSQLRest abstract classes gives you.

2. Creating abstraction layers

In DDD, the "don't leak your domain" principle joins the SOA's moto "stay uncoupled".
Here comes DTOs - Data Transfer Objects.
Together with the SOLID principles (see the SAD pdf), you always should rely on abstractions.

On the other hand, there are some cross-cutting elements in mORMot.
Typically, RawUTF8 is part of it. So you don't need to define your own type. You should not be afraid to use SynCommons.pas in your clients.
Also all interface-based services, and even the TSQLRest / TSQLRecord definitions can be safely shared. I would not blame you to refer to mORMot.pas in your clients.
This is one of the design principles of the framework: some units are shared, others are specialized. You should not link your clients to mORMotDB nor mORMotSQLite3, but could safely use SynCommons.pas and mORmot.pas everywhere.

For Value Objects, since they may be implemented either as TPersistent or plain records, and should be immutable, they can safely be shared over all layers, just like RawUTF8.

When you use interface-based services, you also have access to TSQLRecord operations, since both share the same TSQLRest routing class.
BUT do not forget that you can have several TSQLRest instances.

It is typical to have:
- one TSQLRest endpoint between the Persistence and the Domain Layers;
- and one another TSQLRest endpoint (with diverse - or even none - TSQLRecord) between the Domain and Application layers;
- then another TSQLRest endpoint between the Application and Client layers.
Interface-based services and DTOs are typical to be used between the Application and the Client layer. Even a CQRS pattern can benefit to be implemented with interface-based services (e.g. with the new RawJSON kind of response).

I'll add an auto-adapter pattern in mORMot.pas, ready to marshall property values from a TSQLRecord to a TPersistent, with no need of writing code.
TPersistent are indeed very good to implement DTOs.
But on the other hand, you can safely re-use the TSQLRecord as DTOs, even as interface-based method parameters.
As written in the SAD, TSQLRecord can be seen as a good translation type.
Remember that if a TSQLRecord is not part of the TSQLRest data model, it won't be used as an ORM class, just as a DTO (or value object).

Roberto Schneiders · 2013-02-21 19:19:29

I'm a little confused now. I had read your blog post about DDD, but it is unclear to me. I'm new in the DDD world.

I'm sorry, but I have much more questions now.

You say "AFAIK, in DDD the "Persistence agnostic" principle forbid the repository to be implemented neither in the Domain Layer, nor in the Application Layer."

Maybe I misunderstood, but I understand that the Repository (explained by Eric) it is an object of the domain layer. Eric says that are domain concepts. Of course the persistence with the database would be in infrastructure. But the repositories will contain links to the infrastructure layer, where persistence is implemented.
In this book (Page 54 and 56) has these statements:
"It can be noted that the implementation of a repository can be closely liked to the infrastructure, but that the repository interface will be pure domain model.
...Another way this is noted is that Factories are “pure domain”, but that Repositories can contain links to the infrastructure, e g the database."

You say "You should not be afraid to use SynCommons.pas in your clients." about the abstraction layer.
Ok. As I understand the TSQLRecord and TSQLRest already are abstraction layers. They are part of an abstraction layer of the framework. So I do not need to create my own abstraction layer. I understood correctly?

Thanks for your patience.

ab · 2013-02-21 19:53:33

The repository is used by the Domain layer, but implemented in the Persistence layer.
That is, the Repository should be "injected" in the Domain Layer, and its implementation (objects CRUD operation to DB, memory or files) is part of the Persistence Layer.
You can change the Repository by a mock or a stub - this is pretty common - to test the Domain layer.

Sorry for the confusion.

"TSQLRecord and TSQLRest already are abstraction layers" is the main point.
You may better re-use an existing abstract layer, and not reinvent the wheel: we did it for you!
Most mORMot.pas and SynCommons.pas classes are in fact some kind of "Layer Super Type":

It’s not uncommon for all the objects in a layer to have methods you don’t want to have duplicated throughout the system. You can move all of this behavior into a common Layer Supertype.

Roberto Schneiders · 2013-02-21 20:37:32

Now I began to understand!

In your opinion should I design my software as in the first diagram?

I understand that TSQLRecord and TSQLRest classes, as well as the types like RawUTF8, are a kind abstract types (Super types). That is ok. But I think we're looking at the problem from different perspectives. Of course, we're on totally different levels of knowledge, but let me try to explain why I thought in the abstraction layer.

According to the uncle bob lecture ...
My software must not know the frameworks I'll use (mORMot). The frameworks need to be plugged in the software. I must not develop my software based on a framework. If I do it'll be stuck in this framework.

So the question is, how do I design my software so that it does not know the framework? How am I going to plug the framework in my software.

I thought creating an abstraction layer could solve this.

If I use any class or type of framework directly into my software I will be tying my software to the framework. Or not?

Imagine that in 5 years we decided to change ORM (I see no reason for this, just a guess), using the mORMot.pas and SynCommons.pas units I have to refactor the whole system.
I understand that mORMot has your own abstraction layers, but these layers, willingly or not, are part of the framework.

Do you think that creating a new layer of abstraction is the wrong way?

I'm having a hard time imagining how to separate the layers using DDD and mORMot. I wanted to create a diagram (fictional) where I could see more clearly the layers and organizing things with DDD.

I know that's abuse, but maybe it would be interesting to create a diagram showing an architecture using a simple software with mORMot. Something more like the real world.
Could contain:
- A CRUD
- A report
- A business rule

I think it would be a very useful diagram showing how objects (entities and objects of value) fit with Repositories using mORMot on a project with DDD. At which layer the framework classes fit, etc...

Thank you.

ab · 2013-02-21 21:44:55

It does make sense to abstract everything... but is it worth it?

Translators should be optimized and automated, IMHO. Otherwise you will spent much time and cpu during the translation process on the server side...

Refactoring a whole system based on interfaces could be faster once your software is stabilized, than during its long maturation phase.
This is one reason why I perhaps would not spend a lot of time writing all those translators when initiating a new project.
In a Agile process, time-to-market is crucial.
You just have to ensure that you won't be stucked later.

If your software is testable, my experiment is that it will probably be refactorable.
So if the framework allows you to test your application, it will certainly force you to

A framework will give you a consistent set of inheritance and APIs, whereas too much abstraction was identified as a potential weak model, by which you may loose most DDD benefits.
Having so many translation objects may in fact lead to a more difficult refactoring and abstraction later: instead e.g. of mapping one framework consistent set of classes into a new framework, you will have to translate all your abstract objects individually to the new framework expectations.

An Open Source framework will allow you to fork it, adapt it, or even know how it works, therefore would certainly help using another framework in the future.
I hate nothing more than a "black box" approach, and external compiled libraries. This is one of the reasons I do not like nugget much in .Net, when it is used to retrieve "official" libraries compilations. Source code is always better.

Using Poco/Pojo/Podo (Plain Old Delphi Objects) is some kind of dogma...
Entity Framework 4.0 STE (Self Tracking Entities) kind of objects, which just sounds like our TSQLRecord, and are a bit more lax.

MSDN Spain DDD wrote:

STEs are suitable for N-Tier applications where we control their end-to-end development. They are not, however, suitable for applications where there is no intention of sharing real data types between the client and server; for example, pure SOA applications where we only control one end, either the service or the consumer. In these cases, where there is no possibility or permission to share data types, it is recommended to make use of DTOs in distributed services (Web Services, etc.).

This definition fits perfectly with our TSQLRecord Client-Server ORM process.

Sorry, but I 'm a lazy and pragmatic coder, so I do not like such "dogmas".
The best solution is the one you embraced, works, and is still maintainable.
:-)

mORMot Open Source

#1 2013-02-21 13:48:01

Clean architecture with mORMot

#2 2013-02-21 15:58:55

Re: Clean architecture with mORMot

#3 2013-02-21 19:19:29

Re: Clean architecture with mORMot

#4 2013-02-21 19:53:33

Re: Clean architecture with mORMot

#5 2013-02-21 20:37:32

Re: Clean architecture with mORMot

#6 2013-02-21 21:44:55

Re: Clean architecture with mORMot

Board footer