Penentration testing / Pen testing

erick · 2018-09-13 17:07:10

Has anyone professionally penetration-tested mORMot? We are doing that right now and would like to look at any previous reports that would support it.

You can reply to me here or privately.

erickengelke@gmail.com
Erick

ab · 2018-09-14 07:50:20

I don't have at hand any such report.

But note that on production, we usually add a nginx front-end as reverse proxy for HTTS (even on Windows).
And we disable the ORM REST feature, force either authentication or a JWT, and set a THttpServerGeneric.OnBeforeBody callback.

hnb · 2018-09-14 10:34:07

Worth to note that OnBeforeBody behaves in different way for http.sys and socket server. For the http.sys OnBeforeBody is executed only when body exist but for sockets server OnBeforeBody is executed always (even when body is empty).

Leslie7 · 2018-09-24 08:36:22

Erick,

do you have any results?

mpv · 2018-09-24 16:50:29

hnb wrote:

Worth to note that OnBeforeBody behaves in different way for http.sys and socket server. For the http.sys OnBeforeBody is executed only when body exist but for sockets server OnBeforeBody is executed always (even when body is empty).

Yes, you are right! I'm create a fix in pull request #131 Now onBeforeBody will be called on HTTP.SYS even for empty body. For example HEAD requests can be with content-length header but without actual body.

Last edited by mpv (2018-09-24 16:52:15)

pvn0 · 2018-09-25 08:04:43

I'm very interested in this subject, I wish there was more information on this.

@erick, please let us know if you plan to release your findings.

@ab, to disable ORM REST, is it enough to setup default authentication and remove all SQLAccessRights or is there another way?

ab · 2018-09-25 14:34:28

What we do to disable ORM REST is just to use a TSQLRestServerFullMemory with no ORM TSQLRecord class in the model to publish over HTTPS/HTTP.
On it, we publish the services (via methods or interfaces).

Then internal TSQLRestServerDB are used for actually access the database(s), from the service implementation side.

pvn0 · 2018-09-25 17:35:57

thanks ab, much appreciated.

erick · 2018-10-03 05:07:36

> I'm very interested in this subject, I wish there was more information on this.

@erick, please let us know if you plan to release your findings.

Sure, I was distracted for a few days, I have other responsibilities besides coding. We haven't gotten very far yet.

I'm the Director of Computing for the University of Waterloo's Engineering Faculty. UWaterloo got in the news today because one of our faculty was awarded a Nobel Prize for Physics today, and so we're all in a good mood right now.

My security guy was not happy to hear there were no former pentests known or shared in the mormot group.

Also, he had a couple of questions I'll try to relay here.

But first, my situation is this:

I have made no changes to mormot itself, STOCK CODE, but I'm not using it in the way most of you are probably doing. I'll explain what I've done, why, and will ask if anyone has suggestions to improve it or sees a problem.

My clients are all JavaScript written with Elevate Web Builder, as described in my mormot book. Elevate Web Builder (or EWB) is a combination GUI library and pascal to javascript converter. It is similar to TypeScript or CoffeeScript or Smart Mobile Studio. And since the end result is Javascript, I think it will likely be no more or less secure than those other JavaScript based systems. Sure, the result can be 'compiled' to unreadable obfuscated JavaScript, but then I know better.... I was one of many contributors to the Undocumented DOS book series, so I know people love a challenge.

Rather than use mormot passwords, however, I use Centralized Autentication Server (CAS) which is a common SSO Single Sign On system popular at univiersities. Probably shortly I will convert to using OIDC Open ID Connect. Both of CAS and OIDC are enterprise or inter-enterprise type Single Sign On facilities, more appropriate for an enterprise, also used by google and others.

The goal there: we don't handle passwords. Since no passwords are gathered, they also cannot be leaked. Furthermore, it means the system can work with MultiFactor Authentication, such as DUO, and does not require a separate login between sites.

So, upon a login, I pass the client a token to present to mormot going forward, that token's SHA256 is placed into the Mormot password facility. So we then treat that token like a disposable password. At the end of the session, the account's cookie is set to an impossible value, so the session token no longer works.

The rest relies on mormot's own user accounting, group memberships, etc.

I hide mormot on an private address space, IPsec'd connection to the private addresds MariaDB database and another IPsec connection to the Apache web server that exposes it to the world.

Security is all about risk analysis and risk aversion. So we've tried to take many positive steps, such as avoiding handling or keeping passwords, forcing user controls, APIs implementing business logic rather than pure REST access to data, principles of least privilege, etc.

Questions:

Does anyone have evidence that doing the mormot handshake is any less secure when done by JavaScript?

Has anyone done a cryptoanalysis of the mormot handshake protocol.

More of a comment, the avoidance of cookies may not have been a good thing. We will likely add in cookies for session checking, since browser cookies have advanced in the last few years so they can be issued but taught not to be browser readable - they can be set to just pass a opaque block of data.

The reason cookies are desired is a big concern in web apps is the possibility of cross site scripting, something which is obviously less of a problem for compiled binaries. But I believe the present and future of applications like ours are web based applications that run on computers/phones/TVs without recompiling.

I may come back with more questions, but we do hope to plug any holes we find and report back to the community.

I can be reached at erickengelke@gmail.com

Erick

erick · 2018-10-03 05:17:52

Oh, if you're interested in seeing what it looks like, the student part (the simplest part) is demonstrated here: https://www.youtube.com/watch?v=3JzjlOpC_vo&ar=1
Much more complex screens are available for staff to manage the students.

E

ab · 2018-10-03 09:58:14

The mORMot handshake is fully secure only over an encrypted connection, like HTTPS or binary encrypted WebSockets.
So from JavaScript, if your server uses HTTPS, it should be safe.

Note that Google or others do send plain passwords over HTTPS, so the mORMot handshake protocol is more secure for sure - at least you can never see the password, and can't replay an old signature!

pvn0 · 2018-10-03 10:37:09

Thanks erick for your detailed input, the video looks awesome btw and gratz to your uni.

erick wrote:

More of a comment, the avoidance of cookies may not have been a good thing. We will likely add in cookies for session checking, since browser cookies have advanced in the last few years so they can be issued but taught not to be browser readable - they can be set to just pass a opaque block of data.
The reason cookies are desired is a big concern in web apps is the possibility of cross site scripting, something which is obviously less of a problem for compiled binaries. But I believe the present and future of applications like ours are web based applications that run on computers/phones/TVs without recompiling.

Have you considered JSON Web Tokens (JWT) ? documentation.

erick wrote:

My security guy was not happy to hear there were no former pentests known or shared in the mormot group.

erick wrote:

Has anyone done a cryptoanalysis of the mormot handshake protocol.

This comes down to funding outside auditors and white hackers, maybe the synopse community would be interested in a joint funding project for research like this. I know I am.

Last edited by pvn0 (2018-10-03 10:38:45)

erick · 2018-10-03 11:26:41

pvn0 wrote:

Thanks erick for your detailed input, the video looks awesome btw and gratz to your uni.
erick wrote:
More of a comment, the avoidance of cookies may not have been a good thing. We will likely add in cookies for session checking, since browser cookies have advanced in the last few years so they can be issued but taught not to be browser readable - they can be set to just pass a opaque block of data.
The reason cookies are desired is a big concern in web apps is the possibility of cross site scripting, something which is obviously less of a problem for compiled binaries. But I believe the present and future of applications like ours are web based applications that run on computers/phones/TVs without recompiling.
Have you considered JSON Web Tokens (JWT) ? documentation.

Absolutely. OpenID Connect (which is our next step), is based on JWT.

Erick

hnb · 2018-10-03 16:06:44

ab wrote:

Note that Google or others do send plain passwords over HTTPS, so the mORMot handshake protocol is more secure for sure - at least you can never see the password, and can't replay an old signature!

the only what is possible is usage of current/latest signature to logout user ^^!

ab · 2018-10-04 00:18:50

If the user logged out properly, the associated session ID will be rejected.

And the signature is per uri.

hnb · 2018-10-04 08:10:07

yes but "hacker/attacker" is able to logout user without user knowledge. AFAIK this is still actual : https://synopse.info/forum/viewtopic.php?id=2996

pvn0 · 2018-10-04 09:45:04

hnb wrote:

yes but "hacker/attacker" is able to logout user without user knowledge. AFAIK this is still actual : https://synopse.info/forum/viewtopic.php?id=2996

Based on thread you linked , attacker needs to know session signature, which only happens if either client device is compromised or your connection is not encrypted, in either case you have bigger problems then someone logging you out. But still a nice catch, I guess it's valid if it still exists.

Last edited by pvn0 (2018-10-04 09:50:53)

ab · 2018-10-05 08:19:32

There is no thing as "session signature": there is an "URI signature" for a given session.
This is why our TSQLRestServerAuthenticationDefault is in fact more refine than a session cookie/JWT/bearer.
Please check how TSQLRestServerAuthenticationDefault works in https://synopse.info/files/html/Synopse … ml#TITL_98 - what is described in this thread makes no sense to me.

@hnb I don't understand this thread.
The closing URI has a diverse URI value than others, so there is no way to know the signature but to replay an existing URI which closed the session - and in this case the connection is already closed...

@pvn0 You are plain right: if the https certificate is compromised, then you have bigger problems for sure!

erick · 2018-10-10 22:43:51

I figured I should give an update.

As I said before, using externalized authentication CAS or OpenID Connect allows us to prevent leakage of passwords and use smart cards and cell phone apps to authetnicate. Mormot's SHA256 password stores are sort of, maybe, okay today, but not for much longer as CPU power is always increasing, and not a good solution in a world that needs multifactor authentication to prevent users from leaking passwords.

When writing web applications, ie. Javascript/HTML5/AJAX, using cookies with the SECURE and HTTPONLY flag set is typically required to prevent cross site scripting (XSS) and cross-site request forgery (XSRF). If you are writing a web app, you should not be just using mormot's otherwise good security, because someone in javascript can read the copy of the password in the browser's memory that you have to keep around for message signing. And if you don't know, XSS and XSRF are big attack vectors that took out Facebook and others.

The HTTPONLY flag on a cookie means the browser does not expose that cookie to javascript. Attacking programs cannot steal the cookie (except if the browser is super buggy), and so they cannot present your credenitals. The secure flag means it will never be exposed using HTTP, whereas HTTPS or better or allowed.

In my implementation I use CAS or soon OpenID Connect to indicate logins to a server, then set a secure httponly cookie to the value of a JSON Web Token (JWT). Before any packet is passed to any RPC's, my code uses Mormot to evaluate the JWT. Encoded in the JWT is the userid, the time the token was issued, and the expiry time of the token, as well as some other info, and hashed with a mormot-supported crypto-hash function. If the hash works using our secret, the cookie was set by a trusted server. Otherwise, I return an HTTP error. Offhand, I think I return 403. Maybe there is a better choice.

After the expiration time, mormot automatically fails the JWT token, and the connection is cut off, no more requests can make it through with out re-authenticating.

If you can imagine, the Javascript app is oblivious to the JWT knowledge, it cannot read it (due to httponly flag). So it actually has to call an RPC of the mormot app called WhoAmI that tells it which userid it is logged in as. That is advisory only, the actual login happend back when CAS or OpenID Connect were called.

There is also logic to allow logouts, because you need to register the logout or you would just be autologged in again, and could never change userids until the timeout happens. So that was another hour of work.

My work is still being evaluated by a fellow with >10 years pentesting experience. He usually finds flaws with everything, so I realistically expect him to find flaws with my work too. Security testing is a humbling experience for the developer.

After we pass our tests (hopefully), I will bundle up what I've done.

Erick

erick · 2018-11-06 20:59:17

Is there a way to tell mormot to encrypt the MySQL connection. I'm currently using ZEOS as the Mysql driver, is that still a good choice.

Thanks in advance. I still intend to write a summary here when we are done pen testing.

Erick

ab · 2018-11-07 12:55:15

I don't use MySQL but I guess TLS could be used for communication, if it is enabled on the server.
Of course, it requires a private/public key pair to have been setup first.
Another option (which I have seen) is to have a VPN or SSH tunnel defined.

It is more a question for the Zeos/ZDBC forum support...

erick · 2018-11-15 05:01:32

ab wrote:

I don't use MySQL but I guess TLS could be used for communication, if it is enabled on the server.
Of course, it requires a private/public key pair to have been setup first.
Another option (which I have seen) is to have a VPN or SSH tunnel defined.
It is more a question for the Zeos/ZDBC forum support...

Thanks. Setting it up is apparently a bit more challenging.

I think we may end up doing IPsec between the various points. IPsec between two homogenous systems works great, but between different OS's is more of a challenge. I think we might have to use the open source swan thing.

Thanks for your comments, Ab.

Erick

erick · 2018-12-07 03:01:28

FYI, the end-to-end software penetration testing is being done against my system for about five days from Dec 6 to Dec 11th, with a report to me to follow sometime later. They are using the IBM AppScanner automated testing solution. Of course, I will share results with this community.

https://www.ibm.com/us-en/marketplace/a … header-top

pvn0 · 2018-12-18 10:17:24

thanks erick, I've learned a lot from this thread. Wish you all happy holidays.

erick · 2018-12-21 04:07:23

I got the security audit report yesterday. I spent Wednesday patching up the few remaining issues.

The good news is that:
1. Mormot marshalling of data appeared to work perfectly
2. The IBM scanners could not get at data not intended for it, nor could they break the database
3. Momot application survived an onslaught of hacking wisdom

But to get to that state I had to harden it beyond what is stock or it would be vulnerable to a large number of attacks.

The things I had to address before and/or after the audit were XSS, XSRF, content policies, XSS headers, cookies, etc. etc. I'm updating my examples to include all the tidbits I've learned. I will complete this over the Christmas holidays.

I will document them in the 3rd edition of my book. And I will, as always, send a copy to AB. Copies of the software will be available without requiring a purchase, but I encourage one to buy the Ebook to gain full knowledge, and also to help sponsor the work.

Erick

pvn0 · 2018-12-21 07:32:46

That's great news! Looking forward to reading your book.

rdevine · 2020-05-08 13:26:29

Hi Erick - are you still planning to publish a 3rd edition with the security overview?

Cheers, Bob

erick · 2020-05-20 13:51:11

Hi, I have completed a new book, but it's more focused on a specific situation. I felt there is now ample free material about mormot, but where I've invested a lot of time in the last few years was building a really good dev environment using Elevate Web Builder with mormot.

The benefits of using EWB is that it gives you a totally Pascal environment, which compiles to JavaScript. The days of cross compiling to Android/IOS/OS/X LInux, etc are gladly mostly gone for me. And the distribution nightmare. EWB produces nice Web apps (or PWAs if you want to install on a desktop/android/etc/) and my users are preferring them to having to install native clients. It also means I know everyone is running the current version.

With a modern web environment, you have something that produces an experience very similar to usual applications. But without the nightmares.

In the book I cover external authentications (SAML, OpenID Connect, etc.) when used with EWB + Mormot. Also how to avoid most web attacks (Cross Site Scripting, Cross SIte request Forgery) which most Mormot example JS scripts are vulernable to.

You can read more about it at https://www.erickengelke.com

But the average Momrot user looking to make Thick PC/Android/IOS/MacOS clients will not benefit from it.

Erick

Junior/RO · 2020-05-20 19:09:31

@erick

Is Tim Young/Elevate Software updating Elevate Web Builder?

macfly · 2020-05-20 19:45:39

I have a little pessimistic thinking about using pascal to create web applications (javascript / html).

The work they are doing with pas2js is very good.

But who is going to study pascal to develop Web applications?
So this is restricted to those who already have knowledge of the language.

Tools like eslint help a lot to avoid errors in javascript.

And there is TypeScript that has become standard for who need a typed language to make the transpiler.

Using proprietary software such as TMS Web Core and EWB makes us dependent on updates, and the resources they provide.

I believe that the best way is to create the front directly in Javascript / HTML. There are many options, like Vue, React, etc.

And you will always get professionals to work with these tools.

erick · 2020-05-21 02:44:46

Junior/RO wrote:

@erick
Is Tim Young/Elevate Software updating Elevate Web Builder?

Yes, there is a version 3 coming out sometime, it's in beta right now. I'm not sure how long before it will be released.

Erick

erick · 2020-05-21 02:46:29

Junior/RO wrote:

@erick
Is Tim Young/Elevate Software updating Elevate Web Builder?

erick · 2020-05-21 03:09:59

macfly wrote:

I have a little pessimistic thinking about using pascal to create web applications (javascript / html).
Tools like eslint help a lot to avoid errors in javascript.
And there is TypeScript that has become standard for who need a typed language to make the transpiler.
Using proprietary software such as TMS Web Core and EWB makes us dependent on updates, and the resources they provide.
I believe that the best way is to create the front directly in Javascript / HTML. There are many options, like Vue, React, etc.
And you will always get professionals to work with these tools.

Sure Coffeescript and others are big improvements over JS. I think anyone trying to do a big project in just pure JS is out to lunch, just like doing Asssembler for the whole project, it doesn't make sense. JS programmed code is too buggy, and not typed, etc. The proejct will not scale well. You need a typed language and the benefits of a well designed library. Save the hand coding for the few bits that are hard to do or time critical, but use a good language and toolkit for the rest.

You didn't say it, but I think you are hinting at marketshare. Which is kind of funny on a forum of Pascal developers.
Consider is Java... remember when it was the coming king and everyone was bowing down to it. Oracle bought the company and turned against everyone. All your Java code and skills... suddenly with the new costing model it lost Google's support and others. Wrote your whole app in Java, sorry to hear that

The life cycle of so many development kits is short. Microsoft has abandoned several over my lifetime. Silverlight, MFC, I cannot remember all the false starts and promises....

We developers sometimes dump on Delphi, but i've got 30 year old Delphi code which has been recompiled and is being reused to this day.

Just as Delphi code from 30 years ago is largely reusable today, knowledge you have of the VCL/LCL is largely portable to EWB and that saves you effort today, and the EWB library itself has been quite stable for a number of years.

Is it viable for the long term? There is no guarantee for any technology being both maintainable and long term supported. JS itself is not maintainable. Certiainly PHP is not stable - we're constantly rewriting PHP code to support new versions. Java doesn't offer cost stability. And half the environments people use today will be gone in 5 years.

I'm not trying to disprove you, I'm only saying nobody knows the right anwer.

macfly · 2020-05-21 14:03:41

@erick

I don't know if it was clear, but my post was exclusively about using pascal as transtype of javascript.

I have no problem with pascal, I use it in desktop applications and servers.

I have an Delphi7 application with the same code base since 2006, that I keep until today, running on Windows XP until 10.
Delphi is incomparable in this point.

Yes, I fully agree with Java and PHP.

But using pascal to create JS / HTML clients is really not something I would do.

It is not just a question of the marketshare. It's a development cycle that I don't want to have:

Proprietary editor -> Pascal -> proprietary JS library + Commum JS = HTML APP.

rdevine · 2020-05-25 17:11:59

Hi Erick - that's great news, looking forward to it. I'm specifically interested in the security aspects of your web apps - having gone through Pen Testing it sounds like it'll be an invaluable resource for mORMot web apps. I used EWB many years ago, then Smart Mobile Studio and am now on TMS Web Core (although still subscribe to SMS). Combined with the FNC components Web Core really is in a different league, and the pas2js compiler is very impressive.

Cheers, Bob

erick · 2020-05-26 20:51:58

Hi Bob,

I purchased SMS for two years too, and did a bake-off to decide which I prefered before settling on EWB. But my observations from that time are dated wouldn't be a fair comparison as each product has grown significantly since my evaluation time. I know TMS also do nice things, I did own licenses for several Delphi components of theirs for a spell until I wrote my own replacement components.

I've known people who use each of these with RemObjects or Mormot, and many other open tools. The interoperability and source code-access let you pick something that fits your budget, your scabability, your comfort zone, etc. And that's great, more power to you.

There's so many good development models today - yet so many people are sure only THEIR method is officially sanctioned by the Gods, or Microsoft, or whomever. I just got an earful from someone telling the client that anything not based on .Net is not valid. Idiot! Sure it's one choice, but to dismiss all others as invalid...

Erick

mORMot Open Source

#1 2018-09-13 17:07:10

Penentration testing / Pen testing

#2 2018-09-14 07:50:20

Re: Penentration testing / Pen testing

#3 2018-09-14 10:34:07

Re: Penentration testing / Pen testing

#4 2018-09-24 08:36:22

Re: Penentration testing / Pen testing

#5 2018-09-24 16:50:29

Re: Penentration testing / Pen testing

#6 2018-09-25 08:04:43

Re: Penentration testing / Pen testing

#7 2018-09-25 14:34:28

Re: Penentration testing / Pen testing

#8 2018-09-25 17:35:57

Re: Penentration testing / Pen testing

#9 2018-10-03 05:07:36

Re: Penentration testing / Pen testing

#10 2018-10-03 05:17:52

Re: Penentration testing / Pen testing

#11 2018-10-03 09:58:14

Re: Penentration testing / Pen testing

#12 2018-10-03 10:37:09

Re: Penentration testing / Pen testing

#13 2018-10-03 11:26:41

Re: Penentration testing / Pen testing

#14 2018-10-03 16:06:44

Re: Penentration testing / Pen testing

#15 2018-10-04 00:18:50

Re: Penentration testing / Pen testing

#16 2018-10-04 08:10:07

Re: Penentration testing / Pen testing

#17 2018-10-04 09:45:04

Re: Penentration testing / Pen testing

#18 2018-10-05 08:19:32

Re: Penentration testing / Pen testing

#19 2018-10-10 22:43:51

Re: Penentration testing / Pen testing

#20 2018-11-06 20:59:17

Re: Penentration testing / Pen testing

#21 2018-11-07 12:55:15

Re: Penentration testing / Pen testing

#22 2018-11-15 05:01:32

Re: Penentration testing / Pen testing

#23 2018-12-07 03:01:28

Re: Penentration testing / Pen testing

#24 2018-12-18 10:17:24

Re: Penentration testing / Pen testing

#25 2018-12-21 04:07:23

Re: Penentration testing / Pen testing

#26 2018-12-21 07:32:46

Re: Penentration testing / Pen testing

#27 2020-05-08 13:26:29

Re: Penentration testing / Pen testing

#28 2020-05-20 13:51:11

Re: Penentration testing / Pen testing

#29 2020-05-20 19:09:31

Re: Penentration testing / Pen testing

#30 2020-05-20 19:45:39

Re: Penentration testing / Pen testing

#31 2020-05-21 02:44:46

Re: Penentration testing / Pen testing

#32 2020-05-21 02:46:29

Re: Penentration testing / Pen testing

#33 2020-05-21 03:09:59

Re: Penentration testing / Pen testing

#34 2020-05-21 14:03:41

Re: Penentration testing / Pen testing

#35 2020-05-25 17:11:59

Re: Penentration testing / Pen testing

#36 2020-05-26 20:51:58

Re: Penentration testing / Pen testing

Board footer