mORMot Open Source

Vladimir

Member

Registered: 2012-12-21

Posts: 10

Advice for 3-tier doc storage based on Mongo

Hello

I need some general advice for a new app that I am building. The rough functionality it will have is this:
- store documents without a completely fixed structure. There will be a few document types that will have common fields (like number, date, description, etc) and other data structures that will not be fixed and will be different in each doc.
- store scanned files that are related to each document, but they will not be in the DB
- of course various text searches in the documents, most likely in the fixed structure data
- not a big database, probably not more than 5-10000 docs/year and 5-10 users.

Based on that, I decided to try mORMot together with MongoDB, mostly because Mongo seems suitable exactly for the doc storage without a fixed structure. I saw that with mORMot I could use an RDBMS and TDocVariant to store the unstructured data, but the Mongo ODM approach sounds better.

I made a quick test with a rich client connected to MongoDB directly with SynMongo and it worked well. However, the plan is to make a 3-tier structure like:
- MongoDB for storage of the data, as well as direct file storage of the scanned files
- An app server that will handle the logic, doc and file storage, and provide APIs for several clients. This will likely run on the same machine as the MongoDB, which will likely be Linux.
- A Windows desktop client written in Delphi. It will expose the full functionality of the API for managing the data
- An Android client that will be simpler and used mostly for viewing data. Written probably in Delphi or React Native.

This all sounds good to me, but since I don't have too much experience with Mongo or mORMot and 3-tier, I would be happy to hear some opinions about the concept.
1) Would Mongo/ODM have any advantages over the more traditional approach of SqLite/ORM combined with some TDocVariant fields for storing the unstructured data, considering that the database will not be very large? I think that the search will be harder in the SqLite/TDocVariant case.
2) mORMot is not compatible with the Delphi Linux compiler, right? So I suppose that I would have to use FPC for the app server on Linux.
3) What would be the best way to do the client-server communication for the Android and the Delphi client? Are there any samples that demonstrate either or both?
4) Would there be any advantage or disadvantage regarding the client-server API if I write the mobile client in Delphi or in React Native?
5) Any other samples that will be relative to a similar 3-tier architecture? The SynFile demo seems a good candidate.

The questions are very broad of course without the full specifications, but I would be happy to hear your opinions, even if they are subjective

Thanks in advance.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,242

Website

Re: Advice for 3-tier doc storage based on Mongo

1) TDocVariant fields is indeed the way to go. MongoDB will be remote and centralized, but we always favor a local Sqlite3 database, which uses less resource, is faster (no network trip), safer (no need to secure your MongoDB instance), and match better with the MicroService way of coding, i.e. each service its own database. Only if you expect to search within the variant fields, MongoDB may have a benefit, since you may be able to create indexes on it - but in practice, we put as regular fields the searched information (sometimes duplicated/extracted from the TDocVariant), then create a SQLite3 index on it for instant search.
Anyway, you can try both options, switching from Sqlite3 ORM to MongoDB ODM at runtime, and see what's the best for you.

2) mORMot works great with FPC on Linux - which is a much better and mature solution than the Delphi Linux compiler (no need to switch to ARC, much better RTL, stabilized since years, native debugging under Lazarus on Linux)

3) Use an interface-based service and the cross-platform client code generation

4) It is up to you. Mobile client may be faster to market (RAD is great for UI), since React Native is huge to adopt and sometimes difficult to tune.

5) Check also the DDD demo.

Note that storing files in MongoDB is far from perfect, unless they are smaller than 16MB and you can embedd them into a single document.
For bigger files, storing them in local folders may be a better solution: easy to maintain and archive, and also allowing static serving directly from the file system using https://synopse.info/files/html/Synopse … #MNDX_1564

Vladimir

Member

Registered: 2012-12-21

Posts: 10

Re: Advice for 3-tier doc storage based on Mongo

Thanks for the pointers. I will read some more and will experiment with FPC, the Linux daemon and the overall structure. Probably more questions will come up in the process