Will there be TSynLog and TMemoryMapText for linux OS ?

ComingNine · 2014-05-01 07:25:11

TSynLog is a very good log framework! TMemoryMapText provides a very fast file-reading capability!

Both of them are contained in SynCommons.pas, I am therefore wondering if they can be used under Linux OS ? If so, how should one consume them ? (For example, a "Kylix" DEF can be seen in SynCommons.pas. However, when Delphi7/CrossKylix is used to compile a application using TSynLog, the process stops when compiling SynLz.)

Last edited by ComingNine (2014-05-01 07:26:36)

ab · 2014-05-01 07:47:53

Are you still using crosskylix?

Not FPC?

Our framework is not cross platform yet...
But I also still use crosskilix to write server apps.

ComingNine · 2014-05-01 08:01:19

Yes, because of the great comments in this SO post. http://stackoverflow.com/questions/3387501/

Oh! When your great framework turns cross-platform, will it be with Kylix or FPC ?

Oh! How do your server apps utilize mORMot with crosskylix ? Or your server apps only utilize your enhanced RTL ?

ab · 2014-05-01 18:12:05

We prepared the FPC support (e.g. for types, we used the FPC's PtrInt and not the Delphi NativeInt since FPC Win64 compiler was ready years before Delphi).
But it is not finished yet, due to lack of direct use on our side, and some small differences in low-level RTL.

Kylix is still used by us for several server-side apps, and we may use it still.

The enhanced RTL is totally an option, only for Delphi 7 and 2007.
And mORMot works from Delphi 6 up to XE6.

One of the main advantage of mORMot is the http.sys IOCP server, which is... Windows centric...
We would like to use mORMot on Linux - but we are not sure how to have a high-efficient multi-thread IOCP (event driven) HTTP server on Linux easily.
There is a FastCGI module for mORMot, but it won't be truly multi-thread...
Do you have any input about it?

ComingNine · 2014-05-01 20:44:14

Thank you for your time, efforts and knowledgeable comments !

Sorry that I know nothing about this topic... Please check this SO post about IOCP under Linux (In short, epoll seems to be used under Linux instead of IOCP)
http://stackoverflow.com/questions/2794535

ab · 2014-05-01 21:48:10

I just found out that fastcgi can be multi threaded.
See http://www.fastcgi.com/devkit/examples/threaded.c

So we could use mORMot over nginx or lighhtpd on Linux, with full IOCP and high performance.
:-)

ComingNine · 2014-05-01 22:16:45

Thank you for your efforts very much ! That is indeed a great news !

It probably needs mentioning that http.sys is in "kernel land" but lighhtpd is in "user land" . Does this have a consequence ?
(Note, this comment is from google/internet...)

Last edited by ComingNine (2014-05-01 22:17:26)

ab · 2014-05-02 07:38:05

Yes, kernel-mode http.sys has some advantages, but there are some tricks in modern Linux which avoid some of the user-land / kernel-land data copy (like direct writing from file to socket).
The bigger speed difference for FastCGI access would not be this user-land / kernel-land copy, but the fact that FastCGI itself comunicates via Unix Sockets with the web server: this will be one additional data communication layer.

But I guess that a mORMot's FastCGI HTTP server, with a lighttpd or nginx front-end, will be very fast.
And able to serve a lot of clients at once, since both are IOCP-driven.

And easier / cheaper to host than a Windows server.

ComingNine · 2014-05-02 09:51:22

Thank you for your time and efforts ! A linux version of mORMot framework should benefit many people !

ComingNine · 2014-05-11 10:49:22

The HTTP/IOCP part is really beyond me.

Nonetheless, could you help to comment specifically about what one could do to help to make SynCommons compilable under Linux ? Thus the goodies inside the SynCommons (TSynLog and/or TMemoryMapText) can be used under Linux...

ab · 2014-05-11 12:14:26

First of all, we have to find out a path to follow..

Is http://crosskylix.untergrund.net to be used?
I still use it, and it is closer to Delphi 7 than FPC (e.g. the RTTI is identical, and the RTL is the same).
Perhaps it would be faster to first let mORMot server part compiles with CrossKylix.

Then, we may go to http://www.freepascal.org
Either with Lazarus, or http://www.crossfpc.com ?

Main help will be for testing and feedback.
If I'm alone using the Linux port, I won't go this way in short notice.
There is some cheap Windows hosting available, which limited hardware is just a perfect fit for a mORMot server!
Here in France, we have for instance http://express.ikoula.com/serveur-virtuel
See also what Eric wrote http://www.delphitools.info/2013/11/20/ … ow-settled

What do you think?

ComingNine · 2014-05-11 14:58:54

Thank you for your comments very much !

CrossKylix is a good idea: (1) CrossKylix is convenient (2) From the FPC mail list archives, it seems that CrossKylix gives better performance and compatibility while FPC gives more targets. (3) That Delphi 7 is one of your favorite IDEs probably means the Linux/CrossKylix part will be in good sync with the Delphi part.

FPC may become useful when CrossKylix fails mainstream Linux distributions. CodeTyphon, bare Lazarus or even CrossFPC are all good choices.

That will be my pleasure! Furthermore, once the linux (Kylix/FPC) version is ready, the FPC gurus (for example, Graeme Geldenhuys who contributes to tiOPF and fpGUI, which might share equally good concepts with your great framework) may also start to use it and like it. Involvements from these experts should be be helpful and important.

ab · 2014-05-11 16:18:11

Thanks for the input.

BTW, what is your main Delphi IDE?

ComingNine · 2014-05-11 16:54:59

Two IDEs are used Delphi 7 (CrossKylix) and Delphi XE. (Non-visual code is using D7 syntax but edited with DXE.)

edwinsn · 2014-05-17 07:03:11

ab, it won't be only you'll use or want to use mORMot under Linux, the server worlds has been dominated by Linux for years now, right?

I, for one, do strongly wish mORMot will fully support Free Pascal, why? Because I can write a web app that runs on Windows, Mac OSX and Linux and expose the UI via whatever web browser the user uses! It'll be amazing!

mORMot almost contains almost everything we need to make a web app - the standalone http server, the URI routing module, it's RESTful nature, the ORM, the interfaces to many major DBMS (even including the cutting edge MongoDB), the Mustache template engine, and so on.

When it's FPC-ready, an announcement in the FPC forum will prove what I said above.

Last edited by edwinsn (2014-05-17 07:07:16)

ab · 2014-05-17 07:25:56

Perhaps we will release 1.18 soon.

Then start 2.x branch with cross-platform in mind.

edwinsn · 2014-05-17 08:00:52

ab wrote:

Perhaps we will release 1.18 soon.
Then start 2.x branch with cross-platform in mind.

@ab, that's a good news! You are always great, as usual

ComingNine · 2014-05-17 08:48:12

Very knowledgeable comments ! Very good news !

esmondb · 2014-05-18 12:51:18

re value hosting:

Speaking on the 'phone to ikoula they were very helpful but warned their 1 Euro/month windows plan could escalate to 100 Euro if you go too far over the bandwidth limit. But it looks great for a test server. I'm looking at atlantic.net which seem to offer very good value.

ab · 2014-05-18 13:09:40

I was thinking about the 13 or 20 euros plans (see my link above), not the 1 euro VM.
AFAIK they have no bandwidth restriction.

edwinsn · 2014-05-18 15:38:44

BTW, how do you use CrossFPC? Their website nor Google do not have any tutorial. I tried install it, the only I found it adds to the Delphi XE4 IDE is an option window, but I couldn't figure out how to create a project for a specified platform, say Linux, nor I couldn't find out how to specify a target platform that it supports. At the end I uninstalled it...

Any body has any hint on this?

ab · 2014-05-19 09:01:38

I never used CrossFPC, just CrossKylix.

I prefer using Lazarus directly, when I target FPC.
And codetyphon sounds like a good distribution to me.

I do not like the fact that CrossFPC ship binaries, not source.
CodeTyphon does compile everything from source, which I find much better: it is safer, and I can change/upgrade/fork/tweak/fix the source, if I need it.

ComingNine · 2014-05-19 09:56:37

Last weekend I tried SynCommons with CrossKylix and sadly failed due to my incompetence. Several problems :
(1) encoding/UTF8: MultipleByeToWideChar function (not sure NativeXmlWin32Compat.pas works as expected by mORMot)/SynAnsiConvert type/etc
(2) memory-mapped files: how to implement the same functionality using Libc.mmap function etc
(3) Windows-specific: TWindowsVersion type/GetACP function/some {$IFDEF MSWindows} in the interface section but not in the implementation section but referenced in many places/etc

Furthermore, when debugging for the Linux port: OpenSuse+Kylix and windows+Delphi7 seems to be better (more straightfoward) than Delphi7+CrossKylix to me...

Last edited by ComingNine (2014-05-19 10:00:39)

ComingNine · 2014-06-08 13:28:04

Hi Arnaud,

Please find in the zip archive an example of handling exception stack trace using Kylix (Linux) . In short, it uses JclDebug & JclHookException compatible with Kylix (Linux) .

http://mir.cr/PL9MZCGM

ComingNine · 2014-06-20 19:22:41

Could you help to provide a rough estimation when your great SynLog will turns cross-platform ?

ab · 2014-06-20 19:53:55

It is not planned nor needed here, on our side.
Just a pious wish by now.

It may be done in the close future only if someone dedicates some time (or money) to implement it.

ComingNine · 2014-06-20 20:04:57

Thanks for your comments !

Now how much money are we talking about ? If the amount is beyond me then I just have to forget about the idea for the moment.

ComingNine · 2014-08-13 07:51:04

2014-07-28
[50208921410a] now SynCommons unit compiles with Kylix - but not tested, and exception interception for logging and stack trace are disabled (user: User, tags: trunk)

Really nice work ! Kudos for your efforts ! Nevertheless, when "exception interception for logging and stack trace" are disabled, it seems that one cannot expect line number from TSynLog ?

ComingNine wrote:

Hi Arnaud,
Please find in the zip archive an example of handling exception stack trace using Kylix (Linux) . In short, it uses JclDebug & JclHookException compatible with Kylix (Linux) .
http://www.solidfiles.com/d/d76d916481/ … rKylix.zip

As shown in the quoted post, JclDebug & JclHookException that is compatible with Kylix (Linux) can be used to handle exception stack trace using Kylix (Linux). However, the workflow seems different from what SynCommons does.

Furthermore, the MultiLog has a SendException that gives stack trace (only when compiled with Free Pascal). https://code.google.com/p/luipack/ The workflow also relies on API different from the above one or the SynCommons one. Would you think it is feasible to enable SynCommons to handle exception stack trace for Delphi (on Windows), Kylix (on Linux), and Free Pascal in a unified way ?

ab · 2014-08-13 08:11:17

Good input.

I've added a reference to this forum thread to the corresponding ticket.
See http://synopse.info/fossil/tktview/b07ceddab631

ComingNine · 2014-08-13 12:31:18

Many thanks for you great efforts !

ab · 2015-01-18 21:56:22

Exception logging and Stack trace do work now on Linux with Kylix/CrossKylix!!!!

I took inspiration from the code you supplied!
Thanks!

See http://synopse.info/fossil/info/13bacab9f3

ComingNine · 2015-01-19 00:44:45

ab wrote:

Exception logging and Stack trace do work now on Linux with Kylix/CrossKylix!!!!
I took inspiration from the code you supplied!
Thanks!
See http://synopse.info/fossil/info/13bacab9f3

It is excellent, excellent work ! Thank you for your efforts very much !

edwinsn · 2015-01-20 10:12:44

Hi Arnaud,

I might be wrong, but I assume supports also Kylix will cause a lot of IDEF's, is there a reason to still support a project that's dead for ten years?

ab · 2015-01-20 14:46:39

CrossKylix has indeed several drawbacks:

1. It is a dead project, but an alive product. It still works!

2. You can not buy it any more.

3. The debugger and IDE is unusable, but it is still perfect for cross-building server executable for Linux from the Delphi IDE, and thanks to SynLog.pas, you can debug it easily remotely.

We are adding CrossKylix support for several reasons:

1. We use it since years, with great success, so we know it better than FPC.

2. It has still a better compiler than FPC, e.g. for the RTTI we need on interfaces, or even for executable size and memory use.

3. Its compilation is instant - whereas FPC is long to compile.

4. It supports FastMM4, which performs better than the FPC memory manager, from our tests.

5. Resulting executables, for mORMot purpose, are faster than FPC - based on the regression tests.

6. If the code works with Delphi 7, it will certainly work with Kylix (since it shares the same compiler and RTL), whereas FPC is compatible, but not the same. So it sounds safer to be used on production than FPC, today.

7. There is not a lot of IFDEF, but in SynCommons. Then there is a SynKylix.pas unit for several functions. User code would be the same than Delphi.

8. There is a Linux compiler in the official Embarcadero roadmap (for next year?): so we can hope that this one will be closer to Kylix than FPC... so it sounds more like a "back to the future" work...

edwinsn · 2015-01-22 07:12:51

Thanks for info, Arnaud, they make sense! I'm especially want the linux support in the future versions of Delphi.

Last edited by edwinsn (2015-01-22 07:13:11)

ComingNine · 2015-06-05 12:53:04

ab wrote:

Exception logging and Stack trace do work now on Linux with Kylix/CrossKylix!!!!
I took inspiration from the code you supplied!
Thanks!
See http://synopse.info/fossil/info/13bacab9f3

Could you help to comment whether you would provide built-in "Exception logging and Stack trace" for mORMot with FPC/(Win32,Linux) ?
If not, could you help to comment on the best alternative to enable "Exception logging and Stack trace" for mORMot with FPC/(Win32,Linux) ?

Last edited by ComingNine (2015-06-05 12:53:42)

ab · 2015-06-05 13:24:36

In fact, for FPC, I guess the standard FPC functions of exception interception and stack trace should be used.

Someone has to take a little time and include the features already available within the FPC RTL, and described at
http://wiki.freepascal.org/Logging_exceptions

ComingNine · 2015-06-05 16:20:51

ab wrote:

In fact, for FPC, I guess the standard FPC functions of exception interception and stack trace should be used.
Someone has to take a little time and include the features already available within the FPC RTL, and described at
http://wiki.freepascal.org/Logging_exceptions

After reading the "#Handling thread exceptions" section, I really hope that you, who know every detail of mORMot threading, could do the work ...

Luzcka · 2015-06-06 01:47:09

Hi! I get the lasted version from Git and I found an error in SynDBFirebird, missing SynLog unit in uses.

ab · 2015-06-06 06:19:02

SynDBFirebird is not working.

Use SynDbZeos instead.

mORMot Open Source

#1 2014-05-01 07:25:11

Will there be TSynLog and TMemoryMapText for linux OS ?

#2 2014-05-01 07:47:53

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#3 2014-05-01 08:01:19

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#4 2014-05-01 18:12:05

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#5 2014-05-01 20:44:14

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#6 2014-05-01 21:48:10

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#7 2014-05-01 22:16:45

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#8 2014-05-02 07:38:05

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#9 2014-05-02 09:51:22

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#10 2014-05-11 10:49:22

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#11 2014-05-11 12:14:26

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#12 2014-05-11 14:58:54

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#13 2014-05-11 16:18:11

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#14 2014-05-11 16:54:59

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#15 2014-05-17 07:03:11

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#16 2014-05-17 07:25:56

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#17 2014-05-17 08:00:52

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#18 2014-05-17 08:48:12

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#19 2014-05-18 12:51:18

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#20 2014-05-18 13:09:40

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#21 2014-05-18 15:38:44

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#22 2014-05-19 09:01:38

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#23 2014-05-19 09:56:37

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#24 2014-06-08 13:28:04

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#25 2014-06-20 19:22:41

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#26 2014-06-20 19:53:55

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#27 2014-06-20 20:04:57

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#28 2014-08-13 07:51:04

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#29 2014-08-13 08:11:17

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#30 2014-08-13 12:31:18

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#31 2015-01-18 21:56:22

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#32 2015-01-19 00:44:45

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#33 2015-01-20 10:12:44

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#34 2015-01-20 14:46:39

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#35 2015-01-22 07:12:51

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#36 2015-06-05 12:53:04

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#37 2015-06-05 13:24:36

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#38 2015-06-05 16:20:51

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#39 2015-06-06 01:47:09

Re: Will there be TSynLog and TMemoryMapText for linux OS ?

#40 2015-06-06 06:19:02