Enhanced URI routing for method-based services

oz · 2017-11-06 12:24:05

Hi,

i need to build some kind of "external api" for my server. I need to enhance custom URI routing for logically grouping method-based services by functionality using TSQLRestServerDB.ServiceMethodRegister.
Example:

ServiceMethodRegister('MyMethodBasedApiMethod_Number_1',MyMethodCallback1,true);

This results in following URLs:

localhost/root/MyMethodBasedApiMethod_Number_1

But those URIs should rather be accessible as:

localhost/root/api/Method/Number1

using:

ServiceMethodRegister('api/Method/Number1',MyMethodCallback1,true);

I got this working by changing following method:

function TSQLRestServerURIContext.URIDecodeREST: boolean;
...
  slash := PosEx(RawUTF8('/'),URI);
  if slash>0 then begin
    URI[slash] := #0;
    Par := pointer(URI);
    InternalSetTableFromTableName(Par);
    inc(Par,slash);
    if (Table<>nil) and (Par^ in ['0'..'9']) then
      // "ModelRoot/TableName/TableID/URIBlobFieldName"
      TableID := GetNextItemInt64(Par,'/') else
      TableID := -1; // URI like "ModelRoot/TableName/MethodName"
    URIBlobFieldName := Par;
    if Table<>nil then begin
      j := PosEx('/',URIBlobFieldName);
      if j>0 then begin // handle "ModelRoot/TableName/URIBlobFieldName/ID"
        TableID := GetCardinalDef(pointer(PtrInt(URIBlobFieldName)+j),cardinal(-1));
        SetLength(URIBlobFieldName,j-1);
      end;
    end;
    if (Table=nil) and (slash>0) then   // <----------- do not truncate URI if there was a slash, but no Table.
      URI[slash]:='/' else
      SetLength(URI,slash-1);
...

The idea is to not truncate URI if no Table is found. My tests show that this is working properly. Are there any hidden side-effects which i'm not aware of?

Edit:
Looks like if i'd been to fast... static MVC URIs are broken after this patch, maybe even more. So this is not working.

Last edited by oz (2017-11-06 13:01:14)

ab · 2017-11-06 14:35:52

You should NOT change the basic TSQLRestServerURIContext implementation, but INHERIT from TSQLRestServerURIContext and override with your own routing scheme.

edwinsn · 2022-05-13 08:01:58

@ab, Can overriding TSQLRestServerURIContext.URIDecodeREST retrieve the following URI resolving?

http://server.com/[language]/[companyname]/[contact or teams or overview]?

Note, there is no root nor method name are specified in the above URI.
If not, how can I achieve the above URI resolving? Thanks.

ab · 2022-05-13 08:32:31

The problem is about no root...
The mORMot HTTP server requires root values, because a single HTTP server can publish diverse REST servers...

I will look into adding extended routing to the TRestServer, and perhaps allow no root.
But in mORMot 2 only.

pvn0 · 2022-05-13 08:51:49

tbh the no root is better to be handled by whatever front-end reverse/balancing proxy you are using. It's trivial to do this in nginx.

edwinsn · 2022-05-13 10:18:23

ok, maybe pvn is right, to handle it with the reverse proxy like IIS or nginx.

I guess the reverse proxy can also hide the method name? I mean, currently for method-based service in mORMot, the URI will be like:
http://server.com/root/methodname

Can I remove both the root and method name? I use IIS and I wonder if can point to to some technical article or tell me which keywords to use for googling

pvn0 · 2022-05-13 10:30:20

You can with nginx, I don't use IIS. I don't know why you would want to remove method name, with mORMot you can dynamically link a method to any string using TRestServer.ServiceMethodRegister(...).

edwinsn · 2022-05-13 12:31:27

pvn0 wrote:

You can with nginx, I don't use IIS. I don't know why you would want to remove method name, with mORMot you can dynamically link a method to any string using TRestServer.ServiceMethodRegister(...).

Consider a website that's mostly for representing products and you might have the following pages:

http://server.com/en/product1/overview
http://server.com/en/product1/reviews
...
http://server.com/de/product2/overview
http://server.com/de/product2/reviews

The URIs above is obviously cleaner than something like:
http://server.com/root/www/en/product1/overview

And that's why I want to remove both the root name uri and method name (www in the above example) in this case.
Hope it makes sense.

igors233 · 2022-05-13 14:20:03

Why not make language a root, so you'll have en root and de root?

If you do not want to use IIS/nginx, perhaps you can achieve redirection with TRestStorageRemote.

ab · 2022-05-13 16:26:03

Or you have the THttpServerGeneric.OnBeforeRequest event which allows to change the URI on the fly before it is actually sent to the REST server(s).

I have just made THttpServerRequest URI/Method fields writable - could be customized in this OnBeforeRequest event handler.
See https://synopse.info/fossil/info/02c03e4147
and https://github.com/synopse/mORMot2/commit/898858e9

I did not test this method, but I guess this is soon enough to customize whatever input you need, and make internal redirection.
My guess is that this may be at this level that I may add some custom URI routing in mORMot 2.

ab · 2022-05-13 17:13:25

I looked at https://linuxhint.com/url_rewriting/
This is amazing how everything could be done with RegEx, but with so much pain...

Here is my proposal for mORMot 2.
- We will implement a simple rewrite/routing mechanism at the HTTP server level.
- Input will be parsed, and $ placeholders will be extracted into a new URI.

So you could redirect e.g. http://server.com/en/product1/overview by something like

  HttpServer.Route('$/$/overview', 'restroot/overview?lng=$&product=$');

then overview may be either a method-based service named

procedure Overview(Ctxt: TRestServerUriContext);

Or

  HttpServer.Route('$/$/overview', 'restroot/products.getoverview?lang=$&product=$');

would do the trick for an interface-based service defined as such:

 type IProducts = interface(IInvokable)
  function GetOverview(const Lang: RawUtf8; unused: integer; const Product: RawUtf8): RawJson;

And of course, we could add some optional parameter to the Route() function, so that it also can convert and redirect specific HTTP methods (GET, POST, PUT, DELETE).

What do you think?
Does it make sense to you?

ttomas · 2022-05-13 22:11:11

@ab,
Excellent idea, it make sense to me for MVC application where user see URL.

Another idea

HttpServer.Route('$/$/overview', 'http://dynamicserver1.net/restroot/overview?lng=$&product=$');

Converting httpserver to proxy/gateway where dynamicserver1.net can dynamically add route to httpserver on start.
I know this is more complex change. Just idea.

Last edited by ttomas (2022-05-13 22:14:52)

edwinsn · 2022-05-14 03:24:58

igors233 wrote:

Why not make language a root, so you'll have en root and de root?

Because everything about the languages and product names are dynamic.

edwinsn · 2022-05-14 03:29:51

ab wrote:

Or you have the THttpServerGeneric.OnBeforeRequest event which allows to change the URI on the fly before it is actually sent to the REST server(s).
I have just made THttpServerRequest URI/Method fields writable - could be customized in this OnBeforeRequest event handler.
See https://synopse.info/fossil/info/02c03e4147
and https://github.com/synopse/mORMot2/commit/898858e9
I did not test this method, but I guess this is soon enough to customize whatever input you need, and make internal redirection.

Thanks! From what I read and understand, I think the changes will help achieve the URL directing I want.

edwinsn · 2022-05-14 03:39:38

@ab,
What your proposed above for mORMot2 is a very clean but useful idea I think! And it just meets the requirement I suggested above.

edwinsn · 2022-05-14 03:42:17

ttomas wrote:

@ab,
Another idea
HttpServer.Route('$/$/overview', 'http://dynamicserver1.net/restroot/overview?lng=$&product=$');
Converting httpserver to proxy/gateway where dynamicserver1.net can dynamically add route to httpserver on start.
I know this is more complex change. Just idea.

If what you are proposing is possible with mORMot, when we can write a 'load balancer' with mORMot?

ttomas · 2022-05-14 12:29:01

edwinsn wrote:

If what you are proposing is possible with mORMot, when we can write a 'load balancer' with mORMot?

Yes why not
THttpGatewayServer as descended, with methods or interface for remote AddRoute, DeleteRoute
In case of

HttpServer.AddRoute('$/$/overview', 'http://dynamicserver1.net/restroot/overview?lng=$&product=$');
HttpServer.AddRoute('$/$/overview', 'http://dynamicserver2.net/restroot/overview?lng=$&product=$');
HttpServer.AddRoute('$/$/overview', 'http://dynamicserver3.net/restroot/overview?lng=$&product=$');

you will have load balancer
Very useful in case of bugfix update/release, Just run new bugfix server and shutdown buggy server
Or dynamically update version, if you running AddRoute('/api/v1/func1...')
Just run new release with AddRoute('/api/v2/func1...'), update all clients to v2 and shutdown v1 server
Nice feature will be integrated MVC admin panel where you can monitor and dynamically add or delete working servers

THttpGatewayClientServer(THttpServer)
Just configure GW URI and server will register/AddRoute all routes on start. On shutdown DeleteRoutes.
Shutdown can be invoked remotely by GW Server

Only sky is the limit

Last edited by ttomas (2022-05-14 12:42:53)

ab · 2022-05-14 13:56:26

I would rather use nginx or haproxy as load balancer: they are more optimized for this purpose, and safe and used on production since years.
But you are right, for dynamic balancing, a mORMot balancer could make a difference.
We have to think about it, but not with high priority, I guess.

ab · 2022-05-17 10:39:18

For the routing, perhaps

HttpServer.AddRoute('<lang>/<product>/overview', 'restroot/products.getoverview');

seems easier to work with.
Idea is to let the routing mechanism generates the parameters for the resulting

restroot/products.getoverview?lang=<lang>&product=<product>

URI with no $ needed.

And I guess existing parameters to the initial query will be kept intact. Only new parameters, like lang and product above, would be added to any existing parameters.

We could add some GET/POST members routing too, for a better REST parametrization:

HttpServer.AddRoute('GET <lang>/<product>/overview', 'restroot/products.getoverview'); 
HttpServer.AddRoute('POST <lang>/<product>/overview', 'restroot/products.setoverview');

seems fine to work with.
Here the GET is identical as above. But the POST parameters will execute restroot/products.setoverview as

 type IProducts = interface(IInvokable)
  procedure SetOverview(const Lang: RawUtf8; unused: integer; const Product: RawUtf8; const Body: RawJson);

Here Body parameter is the default be we could define the root as such to rename e.g. as const NewValue: RawJson:

HttpServer.AddRoute('POST<newvalue> <lang>/<product>/overview', 'restroot/products.setoverview');

for

  procedure SetOverview(const Lang: RawUtf8; unused: integer; const Product: RawUtf8; const NewValue: variant);

I would like to implement this first at the HTTP server level, then we could add some routing features at the interface-based services definition level.

Then, perhaps something for load balancing... but I am still not convinced, unless we can redirect the balancing to a nginx proxy frontend.

edwinsn · 2022-05-17 15:12:08

@ab,

I think both routing syntax is OK as long as it's flexible enough and can also route to a method-based service

mORMot Open Source

#1 2017-11-06 12:24:05

Enhanced URI routing for method-based services

#2 2017-11-06 14:35:52

Re: Enhanced URI routing for method-based services

#3 2022-05-13 08:01:58

Re: Enhanced URI routing for method-based services

#4 2022-05-13 08:32:31

Re: Enhanced URI routing for method-based services

#5 2022-05-13 08:51:49

Re: Enhanced URI routing for method-based services

#6 2022-05-13 10:18:23

Re: Enhanced URI routing for method-based services

#7 2022-05-13 10:30:20

Re: Enhanced URI routing for method-based services

#8 2022-05-13 12:31:27

Re: Enhanced URI routing for method-based services

#9 2022-05-13 14:20:03

Re: Enhanced URI routing for method-based services

#10 2022-05-13 16:26:03

Re: Enhanced URI routing for method-based services

#11 2022-05-13 17:13:25

Re: Enhanced URI routing for method-based services

#12 2022-05-13 22:11:11

Re: Enhanced URI routing for method-based services

#13 2022-05-14 03:24:58

Re: Enhanced URI routing for method-based services

#14 2022-05-14 03:29:51

Re: Enhanced URI routing for method-based services

#15 2022-05-14 03:39:38

Re: Enhanced URI routing for method-based services

#16 2022-05-14 03:42:17

Re: Enhanced URI routing for method-based services

#17 2022-05-14 12:29:01

Re: Enhanced URI routing for method-based services

#18 2022-05-14 13:56:26

Re: Enhanced URI routing for method-based services

#19 2022-05-17 10:39:18

Re: Enhanced URI routing for method-based services

#20 2022-05-17 15:12:08

Re: Enhanced URI routing for method-based services

Board footer