The Big Divide

Javierus · 2024-10-14 22:26:44

I've been following some comments on mORMot2 GihHub, and could not resist to comment, here

Sure, mORMot2 is not a big monolith, it's carefully architected from a RTL++ to full toolset, and then more

But from outside, to a stranger, it seems... a big monolith. Heck, nobody would think on mORMot if looking for a JSON lib, or crypto, or sockets, websockets, a million stuff. Because it's an ORM, you know, it says it on the name, mORMot

It's very sadthat such an engineering feat as mORMot is not massively used, be it for REST, for ORM, for RTTI, for JSON, for.... ... ...

Spliting it on a few repos *might* be what is needed to make it more used. The higher level repos would use the lower level ones; zero changes on the current code: just a division on more manageable chunks

Just my 2 cents

zen010101 · 2024-10-15 06:46:32

What about QuickLib? https://github.com/exilon/QuickLib

Areas of functionality:

Mapping: Map fields from a class to other class, copy objects, etc..
Config: Use your config as an object and load/save from/to file (Json/Yaml) or Windows Registry.
Serialization: Serialize objects to/from json/Yaml.
Scheduling: Schedule tasks launching as independent threads with retry policies.
Threading: Simplify run and control of multithread background tasks, Thread-safe Lists, queues, etc
Data: Flexible data interchange and storage, allowing several input-output types.
Cloud: Simplify cloud Azure/Amazon file management, send emails and more.
Querying: Indexed Lists, Searchable Lists and Linq query system for generic lists and arrays.
Benchmark: Time elapsed control and benchmark functions.
Filesystem: Process and Services control, file modify monitors and helpers, etc...
FailControl: Fail and Retry policies.
Caching:: Cache string or objects to retrieve fast later.
Templating: Simple string templating with dictionaries.
Debuging: Utils to debug your code.
Parameters: Work with commandline parameters.

So, BIG is not the problem forever .

ab · 2024-10-15 07:01:24

Discussion was https://github.com/synopse/mORMot2/issues/304

Since it is a "marketing" issue, I am completely lost.
So any input is welcome, and better than mine.
This is the rule of humanity: the one who makes can't be the one who sales.

If we create several repositories, we could create a main mORMot repository, with everything as git sub-modules, one per /src/* sub-folder.
But there are some minimal dependencies in core, e.g. mormot.lib.z is used in mormot.core.zip.
So we would need to change the folders.

The problem with sub-modules is that, if I understand correctly, we would need to change the units locations, because we can't have overlapping folders.
Of course, the unit names are genuine, so we could do that.
But we would loose the unit-name-is-location feature of the framework I like very much - i.e. mormot.lib.z.pas can be found in /src/lib and mormot.core.zip.pas in /src/core.
It could be a shame to more mormot.lib.z.pas into /src/core but we could...

Sub-modules are not straightforward to use, as anything in git - this is why prefer fossil.
https://git-scm.com/book/en/v2/Git-Tools-Submodules
It would complicate our daily work, because we need every sub-modules to be in synch.
Perhaps there is a better option than git submodules?

Perhaps what could help is that Embarcadero would finally accept to put mORMot in their GetIt system.
But they don't want because in fact, it may offend their own "Enterprise" line of products plan.
And our full compatibility and support with FPC could also be a reason of Embarcadero
But https://synopse.info/forum/viewtopic.ph … 518#p17518 still applies - you could go up to this thread to see the Embarcadero reasons not to include mORMot.

IMHO most mORMot users do still use Delphi because of mORMot, otherwise they would have switched to Java or C#.
We like Delphi, as a language and a platform. We leverage decades of (object) pascal practice.

Almost 10 years after my remark (!) I indeed personally know companies still paying for their Delphi license thanks to mORMot, not despite of mORMot.
Anyway, this is another discussion.
But having smaller repositories may help integration with GetIt. They may include the core units, or the crypto units in some countries... as you can see, I try to be open and weight the pros and the cons of "the Big Divide".

So two questions:

What about the repositories content?
What about the repositories naming?

Javierus · 2024-10-15 10:14:52

mORMot won't be ever on GetIt. It does something strictly forbidden: "adding Client Server technology to Professional"

However if it's split in smaller repos, building ones on top of the others, I believe a good part of it could be there. And that could lead those repo users to the whole mORMot 2

Javierus · 2024-10-15 10:19:05

ab wrote:

Discussion was https://github.com/synopse/mORMot2/issues/304
What about the repositories content?
What about the repositories naming?

In my opinion, the first should be identifying the repositories content. I've been checking for unit cycles, and there are very few, which means that will not be a restriction on deciding how to split
I'll keep studying your code structure to see what can I hint

ab · 2024-10-15 10:49:54

zen010101 wrote:

What about QuickLib?
So, BIG is not the problem forever .

I don't see your point here, sorry.

squirrel · 2024-10-15 11:32:37

If I may add my 2 cents...

@ab is 100% correct that this is a marketing issue, and not a code issue. Splitting mORMot into several repositories will be undesirable and create more difficulties than it solves. Imho the refractoring of the code into more logical folders and files with the move to v2 already achieved what a repository split would do. So please do not split the wonderful mORMot into different projects.

Maybe restating the issue as one of discoverability can help guide to the future. People finding out about the project need a way to quickly discover what can be done/achieved using the project and exactly where to find out how.

There are currently 4 ways to find out: mORMot has wonderful documentation and even a getting started guide on github. There are also example projects and the blog. Those are real gems that very few opensource projects have. I think what @zen010101 touched on, is to have one place that lists a few of the major things that can be done using this project, with direct links to more information. The main github project page for mORMot already does that. People just need to know what is possible before needing to find out how. And after they know what can be done, they need a direct way to get to the parts of the documentation that is relevant. So for example going directly to the documentation of how to use IDocDict without first learning how TDocVariantData is implemented.

My suggestion would be to make a very simple main/index page for this website that lists a few of the most commonly used parts of the project with links directly to the relevant part of the documentation or to the blog.

Example (note how it deliberately does not require knowledge of the project code or terms - that can be discovered later):

JSON
Quickly convert json into objects, or convert objects into json.
Here is a short 5 lines code sample. [...]
Here are the links to the relevant documentation on (un)serialization

This project already has everything, all it needs is a summary page up front

Last edited by squirrel (2024-10-15 11:37:12)

tbo · 2024-10-15 12:05:03

The first time I promoted mORMot, I would have shared the points from the opening post. Later I thought, no matter what you say, many developers don't want to rethink. Today I promote mORMot with the slogan: It's on your computer in 3 minutes and removed without a trace in 3 seconds if you don't like it. Often I show concrete source code including the uses part. So it works with copy&paste.

I therefore tend to agree with @squirrel, but I'm not someone who knows anything about marketing.

With best regards
Thomas

dcoun · 2024-10-15 12:58:09

First time I checked mormot was 5 years ago. The Mormot v1 was a huge investment to start using it. I gave up.
Two years ago, with the 2nd version, I started to read the code and it was really helpful to have different units as it is now.
The documentation is probably the main problem for a new adopter now.
Dividing mormot to different repositories is not going to help more. The increased time it will need, it is better to be invested in more documentation with more examples.

cybexr · 2024-10-15 13:35:55

dcoun wrote:

First time I checked mormot was 5 years ago. The Mormot v1 was a huge investment to start using it. I gave up.
Two years ago, with the 2nd version, I started to read the code and it was really helpful to have different units as it is now.
The documentation is probably the main problem for a new adopter now.
Dividing mormot to different repositories is not going to help more. The increased time it will need, it is better to be invested in more documentation with more examples.

me too, v1 was too difficult to work with, v2 seems more friendly. Split mormot into sub-modules may please someone's taste, or maybe not.
IMHO some more documents and more realistic examples may help newbies to mormot world! And they will be amazed by this great framework!!

ab · 2024-10-16 07:58:21

I am also concerned by the complexity - and confusion - which may come from switching to several repositories, and sub-modules.

We had already a lot of problems with the DMustache and SynPdf projects, which share some part of mORMot 1 code, so was messing up the compilation when both were installed, in several versions.
If we require the sub-modules to be always in synch, then I don't see any benefit of using them, in respect to a global repository.
If we want the divided repositories to be stand-alone, then there will be a lot of redundant code for sure.

We could do what other projects propose: a "core" repository, then a "crypt" repository, then a "net" repository, then a "db" repository, then a "rest/orm/soa" repository, then a "ui" repository... each one with the parent projects as sub-modules (or not).
And all mormot.lib.*.pas units would be not in their own folder, but appear whenever they are needed.
Just from the description, you see how complex it may become.
I am afraid it would add just more confusion.

Quick Start documents would help for sure, more than changing the code.
I am also still thinking about some simple (UI?) tools to easily create projects skeleton.

okoba · 2024-10-17 12:24:23

I suggest adding many examples instead.
There are good ones in the "ThirdPartyDemos" directory that teach the basics step by step, but they are about server stuff, and mORMot is much more.
As for test coverages, I would suggest increasing the examples' coverage to teach newcomers (or old who don't know a topic like RTTI units) how to use it.
Also, these will help people to find those features, as there are many features that are buried in the huge units that if you don't know there are, you will not find them unless you read the whole repo or see them used someplace else. The examples can play a helpful role here.

@ab, Did you forgot to post mORMot 2.3 Stable Release on the forum here? I saw it on Lazarus forum by chance but not here.

Last edited by okoba (2024-10-17 12:26:53)

pvn0 · 2024-10-19 07:20:01

Monolithic is good if you want new people to join. Downloading a single zip file containing the entire project is still the lowest entry possible for newcomers and splitting it into seperate projects/modules will not lower the entry bar at all.

vs · 2024-10-19 17:38:19

Let me add my two cents.
For me, there are three things that greatly help in understanding the framework, first two of which I try to do daily:

1. Follow the commits. Honestly, I'm just amazed by Arnaud's productivity throughout the day, especially day after day. Arnaud, please share your secret to such work efficiency!
2. Keep an eye on the discussions forum.
3. When necessary, look at the test units for examples of how to use various functions, approaches, and usage scenarios, as they cover the entire framework.

darianmiller · 2024-10-22 15:46:23

ab wrote:

Discussion was https://github.com/synopse/mORMot2/issues/304
Perhaps what could help is that Embarcadero would finally accept to put mORMot in their GetIt system.

I was under the impression that mORMot was listed in GetIt for a while but they are waiting for a contributor to update the latest version. Please reach out to Ian Barker for details on getting mORMot2 added to GetIt.

uian2000 · 2024-11-03 02:47:01

I think current repo is good for me.
Subfolder name indicates it's function, very clear.

But the document or TFM(The fuck manual) is too big to read.
To find a correct class to implement my program is hard work too.

So, maybe we could introduce some divided sample project(some "best practices" I think?), more than samples we have got now.
Then freshmen could step up more easily.

Maybe we could maintain some pratices wiki?

mORMot Open Source

#1 2024-10-14 22:26:44

The Big Divide

#2 2024-10-15 06:46:32

Re: The Big Divide

#3 2024-10-15 07:01:24

Re: The Big Divide

#4 2024-10-15 10:14:52

Re: The Big Divide

#5 2024-10-15 10:19:05

Re: The Big Divide

#6 2024-10-15 10:49:54

Re: The Big Divide

#7 2024-10-15 11:32:37

Re: The Big Divide

#8 2024-10-15 12:05:03

Re: The Big Divide

#9 2024-10-15 12:58:09

Re: The Big Divide

#10 2024-10-15 13:35:55

Re: The Big Divide

#11 2024-10-16 07:58:21

Re: The Big Divide

#12 2024-10-17 12:24:23

Re: The Big Divide

#13 2024-10-19 07:20:01

Re: The Big Divide

#14 2024-10-19 17:38:19

Re: The Big Divide

#15 2024-10-22 15:46:23

Re: The Big Divide

#16 2024-11-03 02:47:01

Re: The Big Divide

Board footer