Relay server scenario

sakura · 2020-01-14 09:57:36

Hi,

imagine the following setup. You have your mORMot server running at the office. And you want to access it with a client from another place, like home or a coffee shop. The user cannot connect directly into the office, so we need a relay server to send to requests into the office server and the answers back to the client.

Now, one idea here is, to make this into an add-on feature, and yes, let the customer pay for the service.

Communication shall be encrypted end-to-end. However, data packets going through our server must be identified, eg: who sends them, to which customers server do they need to be routed. Servers and clients connect to our relay server alike. Now down to the question:

Where would be the best point to hook into the communication chain.

The relay server cannot open TCP/IP to the server, so the server could open a web socket into the relay server to be informed when a request arrives. Pick it up and process it.

On the client, the whole call should be wrapped, so its request can be identified by the relay server and routed to the correct office server.

Those is a very early concept and any ideas leading me the right way are welcome. No line of code done yet. First a proof of concept. :-D

Regards,
Daniel

ehkhalid · 2020-01-14 10:35:33

In theory, companies uses VPN to resolve this, as many companies allow remote work from home to its salaries, they only use vpn to allow secure connection from outside to local server !

sakura · 2020-01-14 10:39:46

ehkhalid wrote:

In theory, companies uses VPN to resolve this, as many companies allow remote work from home to its salaries, they only use vpn to allow secure connection from outside to local server !

Yes, but many customers here do not like the burden of a VPN setup and rather look for another, for them, simple solution. VPN was my first though too, but this is considered undesirable here.

xalo · 2020-01-14 12:33:21

But, if I've understood it correctly, why not simply open a port in the office redirected to the server using an httpS protocol (end-to-end encryption as it was designed) and use the mORMot secure RESTful authentication?

What it would be the actual mission of the 'relay' for you?

Last edited by xalo (2020-01-14 12:35:04)

sakura · 2020-01-14 12:41:14

Due to the hardware setup, many of our customers cannot open ports to the outside (special security hardware), however, it would be possible to connect via HTTPS to our servers, which we could announce through the company, that setups the hardware.

I am not fond of this approach myself, however, this would be the only way for us to provide that possibility at all. Due to all those, currently, customers have two installations and manually import and export their database onto thumb drives to work at home and in the office. :-o

oz · 2020-01-14 13:10:29

Hi Daniel,

without going into too much detail, I would implement it as such:

- Define relay-user-accounts on your relay server. Store the info to which in-office-server that account should connect to.
- Let in-office-server connect to your relay server.
- Establish a web-socket connection to relay-server.
- Let home-clients connect to that relay-server.
- Intercept the http request from the client, store the request in the relay-session, let the client wait here in blocking mode
- Send websocket-notification to the in-office-server
- Load the original client-request from relay-server, process/reassign http headers if required an let the in-office-server handle that request
- Intercept the http reply and redirect it to the relay-server
- Send reply to home-client.

Nice to see that you are evaluating mORMot for that task, i'm sure it could be done using mORMot.

Cheers,
Orest.

ab · 2020-01-14 14:52:56

Did you check https://synopse.info/files/html/Synopse … l#TITL_147 ?

At least the following configuration is close to what you expect:
https://synopse.info/files/html/Synopse … #TITLE_123

sakura · 2020-01-15 11:17:27

Hi Arnaud,

right now I cannot see, how this would solve the problem. Let me try to explain in other words.

Simple work model is easy. Client App (possibly a web interface, possibly a GUI client) connects to the mORMot server, as just about any of the demo apps to demonstrate. Easy, no problem.

The mORMot server is the *only* server having direct access to the data, no replication involved. Now, for various reasons and restraints, when out of office, the client app needs to access the server. The client cannot use a VPN, nor connect to the mORMot server directly. There is simply no way inbound connections into the server are available. Outbound are though. Think of it, like TeamViewer can map sessions around most firewalls, by providing an HTTPS server to route all traffic.

When running the mORMot server will connect with our HTTPS server and announce, that it is available.
When being out of office, the users client will connect to an HTTPS server provided by us, identify and get the correct destination server assigned.
From here on all requests made by the client are sent to our HTTPS server and this one must forward it to the correct mORMot server.
The real problem for me is, that mORMot usually receives request via POST/GET/DELETE/..., however, as there are no inbound connections allowed, the server has to pick up those requests (HTTP GET) and unwrap the original request and process it as it would, when the client connects directly.

So my question here would be, do you have an idea where best to hook into the framework to make that work. Yes, the process would be slower, than connecting directly, however, as of now, we do not see a better approach. However, a better approach is just as welcome. However, we must keep in mind, while data must be transfered, the database must not be replicated.

macfly · 2020-01-15 12:58:08

So in the end what you need is a server that translates GEST requests to the final server, correct?

You can create a simple server for this that receives requests via Get, and call the "real" server in the appropriate GET / POST / PUT format, etc.

Perhaps passing the method on the URL as:

https://myproxy-server/post/adduser/?name=Jhon

This server receive this request, and call via POST:

https://myservice-server/root/adduser/?Name=Jhon
Or
https://myservice-server/root/adduser/
Post Data: {name: Jhon}

In this scenario the actual server that owns the services not need to be changed to accept only GET.
Also because I believe this will be an arduous task.

Last edited by macfly (2020-01-15 12:59:46)

ab · 2020-01-15 13:41:38

Instead of reinventing the wheel, use somehing existing.

Perhaps the easiest may be to open the firewall in the main office, and let connect to the mORMot server throught its IP.
Or use something light like a https://www.freelan.org

sakura · 2020-01-15 13:46:53

@ab : that is my wish too, however, in the environment of most customers (medical networks) those possibilities are not within their power. All we can arrange for, is a connection to our server. Installing other VPN solutions is simply not possible. Firewalls are not controlled by our customers in this network. And we are talking about a lot of customers...

sakura · 2020-01-15 13:48:02

macfly wrote:

lso because I believe this will be an arduous task.

Oh yeah, it will be and I am not sure we would like to do it. But for now, this is a small research project, so I am fine with this.

emk · 2020-01-16 18:18:22

What @sakura is asking is the same methodology implemented by TeamViewer, GoToMyPC, etc:

The client is trying to connect directly to server but if it fails, will try by connecting to a Relay-server (which is maintained by TeamViewer company) to which, the desired server is already connected as a client. So relay server acts as a forwarder between the client and the server(which is connected as a client to relay server) without making any hardware changes.
This is the most useful setup when direct connection fails and that's why software like TeamViewer works every time no matter how complicated infrastructure exists or how many firewalls are between the peers.

In my opinion is better to use a full TCP/IP connection between Office-server and Relay-server(use Indy maybe), but only when is needed because on the Relay-server side it uses one thread per connection.

I would go with this scenario:
1. Office-server is connected to Relay server by Websocket protocol (only for notification of first request of a session).
2. Client sends request to Relay.
3. On first request, Relay sends notification by websocket to Office-server and waits for full-TCP/IP-connection initiated by Office-server.
4. Office-server opens full-TCP/IP-connection to Relay.
5. Relay picks-up the full TCP/IP connection and sends the client request to Office-server, waits for processing, gets the response and send back to client, release the full-TCP/IP-connection handle without closing.
6. If there is no traffic on TCP/IP full connection for let's say 15 min, Relay closes TCP/IP full connection.
On subsequent requests only steps 2. and 5. would happen.

Or, another possibility is to use message-brokers like RabbitMQ, where Client and Office-server are connected as clients to RabbitMQ-server(Relay).

Last edited by emk (2020-01-16 21:26:04)

ttomas · 2020-01-18 18:13:43

Interesting project
https://blog.alexellis.io/raspberry-pi- … l-gateway/

ab · 2020-01-23 19:53:21

You may be interested in https://synopse.info/fossil/info/1a1bc2a087

This implements "pure mORMot" tunnelling system for WebSockets communication.
The public relay could be hosted on any cheap Linux box.

I will make a sample (code + binary) which may work with just a config file.
Also plain HTTP(S) is not supported yet, only WebSockets. It will be in the future.

sakura · 2020-01-24 07:25:57

Hi, this week Bernd Ua is visiting us, but next week I'll have time again to look at it. Thanks a bunch already, you are simply awesome.

ab · 2020-01-24 09:51:15

Say hello to Bernd from me!

Still some work to do on the feature (some bugs and unfinished features), but it may save a lot of time and efforts to mORMot users willing to run a server locally on site, without the ability to tweak the IT firewall.

sakura · 2020-01-24 10:41:59

Oh yes, it will. May it be, that you forgot to commit sample 38, you mentioned it in the comments, but it does not show in the rep.

Thx

ab · 2020-01-24 17:35:41

It is not written yet...

Leslie7 · 2020-01-25 11:54:18

ab,

this is excellent stuff! I wanted to do the same thing for security reasons. Even when accessing the server would not be a problem it is much more secure if the servers with sensitive data are not accessible directly from the outside world at all.
I am eagerly waiting for the addition of http(s) tunneling.

sakura · 2020-02-17 14:24:02

Have tested it and it works. Now I'll have to extend it for our scenario.

I would be happy to try to implement HTTP/HTTPS relay. However, do you have a pointer, as to what could be the most performant way?

Thanks,
Daniel

ab · 2020-02-17 14:43:27

Put a nginx server as reverse proxy on the Linux box where the Public relay instance is.
So it will handle all TLS stuff (including e.g. Let's Encrypt certificate) and buffering/security for you.

The Private relay could connect directly to the Public relay with no TLS, since it will already use encrypted WebSockets for its communication.

sakura · 2020-02-17 19:11:26

Will not quite work, as different clients need to be connected to different host servers, depending on their credentials. With your web sockets relay classes I have a great blueprint for the same setup for our needs. Will get that done this week.

mORMot Open Source

#1 2020-01-14 09:57:36

Relay server scenario

#2 2020-01-14 10:35:33

Re: Relay server scenario

#3 2020-01-14 10:39:46

Re: Relay server scenario

#4 2020-01-14 12:33:21

Re: Relay server scenario

#5 2020-01-14 12:41:14

Re: Relay server scenario

#6 2020-01-14 13:10:29

Re: Relay server scenario

#7 2020-01-14 14:52:56

Re: Relay server scenario

#8 2020-01-15 11:17:27

Re: Relay server scenario

#9 2020-01-15 12:58:08

Re: Relay server scenario

#10 2020-01-15 13:41:38

Re: Relay server scenario

#11 2020-01-15 13:46:53

Re: Relay server scenario

#12 2020-01-15 13:48:02

Re: Relay server scenario

#13 2020-01-16 18:18:22

Re: Relay server scenario

#14 2020-01-18 18:13:43

Re: Relay server scenario

#15 2020-01-23 19:53:21

Re: Relay server scenario

#16 2020-01-24 07:25:57

Re: Relay server scenario

#17 2020-01-24 09:51:15

Re: Relay server scenario

#18 2020-01-24 10:41:59

Re: Relay server scenario

#19 2020-01-24 17:35:41

Re: Relay server scenario

#20 2020-01-25 11:54:18

Re: Relay server scenario

#21 2020-02-17 14:24:02

Re: Relay server scenario

#22 2020-02-17 14:43:27

Re: Relay server scenario

#23 2020-02-17 19:11:26

Re: Relay server scenario

Board footer