mORMot Open Source

cheemeng

Member

From: Malaysia

Registered: 2011-08-09

Posts: 61

Best Way to Store Realtime Stats?

I've been meaning to track and store server access stats.

Something like, a per-second snapshot of total calls to each Method/Interface of each TSQLRestServer. Which then can be used to create 5 min, 30 min, hourly and daily totals, for charting purposes.

What would be a good way to collect and persist this data, without affecting performance too much? I have toyed with the idea of having a thread to send data from TSQLRestServerStats to a Redis server (say, every second), but it would be great if this can all be done in a self-contained manner, with no extra dependencies. Would storing it in SQLite create too big a performance hit? Will it bulk up the size of the database too much? (I'm looking at, say, an median of 10-20 calls per minute, up to a max of 200-300 calls per second).

Anyone has experience doing this, in conjuction with the mORMot Framework? Thanks!

Last edited by cheemeng (2014-09-30 11:27:26)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,657

Website

Re: Best Way to Store Realtime Stats?

Using Redis would be not a good idea, IMHO.
Just the network access will take all performance down.

Such statistics should not be gathered in a "brute store".
It should be stored in a specific layout, known as "round robin". See http://en.wikipedia.org/wiki/RRDtool

cheemeng

Member

From: Malaysia

Registered: 2011-08-09

Posts: 61

Re: Best Way to Store Realtime Stats?

Thanks for the reply Arnaud.

I wonder if the built-in SQLite is usable enough to store such data for now, since I'm not having that much traffic? Of course, that means having to do my own presentation layer to display the stats.