mORMot Open Source

squirrel

Member

Registered: 2015-08-13

Posts: 155

Difference in URL parsing between mORMot1 and 2

I notice a difference between how urls are parsed between mORMot 1 and 2 and was wondering if there is a setting to get the original behaviour back.  Sample code:

  TMyRestServer = class(TSQLRestServerFullMemory)
  published
    procedure Method1(Ctxt: TSQLRestServerURIContext);
  end;
...
procedure TMyRestServer.Method1(Ctxt: TSQLRestServerURIContext);
begin
  Ctxt.Returns('{"value":"hello"}');
end;

When using mORMot1, accessing all these urls will give the same response:
https://127.0.0.1:8084/test/Method1
https://127.0.0.1:8084/test/Method1/
https://127.0.0.1:8084/test/Method1/someval

{"value":"hello"}

But with mORMot2, all of the above works except for https://127.0.0.1:8084/test/Method1/ which returns:
{"errorCode":400,"errorText":"Invalid URI"}

Is this the expected behaviour?

tbo

Member

Registered: 2015-04-20

Posts: 353

Re: Difference in URL parsing between mORMot1 and 2

Is this the expected behaviour?

I can't reproduce your observation with this example. Please compare what you do differently.

PS: Sorry, I missed the slash at the end. Yes, with slash 400 is returned.

With best regards
Thomas

Last edited by tbo (2023-10-27 12:25:45)

squirrel

Member

Registered: 2015-08-13

Posts: 155

Re: Difference in URL parsing between mORMot1 and 2

The example-03 does not compile, but I've created my own tiny example dpr here which shows the difference between mORMot 1 and 2 and can be used to highlight/test the issue:
https://paste.ee/p/tKeqf

tbo

Member

Registered: 2015-04-20

Posts: 353

Re: Difference in URL parsing between mORMot1 and 2

The example-03 does not compile, ...

The example compiles for me without errors, warnings or hints. You do not specify which Delphi version you are using, nor which mORMot commit.

If the function name is expanded with a slash, the log shows following:

GET Files/GetAllFileNames/=400
mormot.rest.server.TRestServerRoutingRest {  "errorCode":400,  "errorText":"Invalid URI"  }

The following is written in help:

function TServiceCalculator.Add(n1, n2: integer): integer;
will accept such requests:
- URL='root/Calculator.Add' and InBody='[ 1,2 ]'
- URL='root/Calculator.Add?+%5B+1%2C2+%5D' // decoded as ' [ 1,2 ]'
- URL='root/Calculator.Add?n1=1&n2=2'      // in any order, even missing

If the help is correct, it is the expected result.

PS: If you put a breakpoint in function TRestServer.Uri() at position "node := fRouter.Lookup(ctxt);", you can trace it.

With best regards
Thomas

Last edited by tbo (2023-10-27 14:15:47)

squirrel

Member

Registered: 2015-08-13

Posts: 155

Re: Difference in URL parsing between mORMot1 and 2

I'm using Delphi 10.3.

example-03:
in u_SharedTypes, the following line causes [dcc32 Error] u_SharedTypes.pas(75): E2034 Too many actual parameters
      pmCheckedFileName^ := TPath.Combine(dirName, fileName, False)
Removing the ,false parameter will cause it to compile.

I hope there is a way to restore the mORMot1 uri parsing that allows the trailing /, since I am porting projects to mORMot 2 that depends on it being consistent.

tbo

Member

Registered: 2015-04-20

Posts: 353

Re: Difference in URL parsing between mORMot1 and 2

example-03:
in u_SharedTypes, the following line causes [dcc32 Error] u_SharedTypes.pas(75): E2034 Too many actual parameters
      pmCheckedFileName^ := TPath.Combine(dirName, fileName, False)
Removing the ,false parameter will cause it to compile.

You are right, in Delphi 10.3 the definition still looks like this: "class function TPath.Combine(const Path1, Path2: string): string;".

With best regards
Thomas

squirrel

Member

Registered: 2015-08-13

Posts: 155

Re: Difference in URL parsing between mORMot1 and 2

If it is possible to get the OnBeforeURI to trigger for when the uri has a trailing / then I can do a redirect or handle it somehow.

squirrel

Member

Registered: 2015-08-13

Posts: 155

Re: Difference in URL parsing between mORMot1 and 2

Is there any way to get OnBeforeURI to trigger in this scenario in mORMot 2 like it does in 1?

htits2008

Member

Registered: 2015-03-25

Posts: 34

Re: Difference in URL parsing between mORMot1 and 2

en，I have same trouble。and it take me afew day try to read forum and change my code。

      //Server.RootRedirectToUri('/static/web/'); //old code ,
      Server.RootRedirectToUri('/static/web');
      server.Route.Get('/static/web','/static/web/index.html',urmGet);
      server.Route.get('/static/assets/*','/static/web/assets/*',urmGet); 

squirrel

Member

Registered: 2015-08-13

Posts: 155

Re: Difference in URL parsing between mORMot1 and 2

Unfortunately this doesn’t only affect my root function, but All my method functions. It is breaking bookmarked links at the clients and 3rd party integrators to the extent that upgrading to mormot 2 is now blocked.

squirrel

Member

Registered: 2015-08-13

Posts: 155

Re: Difference in URL parsing between mORMot1 and 2

Thanks to @htits2008 for pointing me in the right direction.  For anybody else affected by the same issue, here is a procedure to fix it.  Just remember to add all new methods where you call it, otherwise they will still be affected by it.  Maybe there is a way to get a list of all the method based functions (not the interface functions) and step through them at create time so that this procedure can be called without parameters?  Will check later when more time is available:

procedure FixURIRoutingMess(const HttpServer: TRestHttpServer; const methods: array of string);
var
  I, R: Integer;
  J: TUriRouterMethod;
  tmpURI: string;
begin
  for I := 0 to Length(methods) -1 do
    for R := 0 to HttpServer.RestServerCount -1 do
    begin
      tmpURI := '/' + HttpServer.RestServer[R].Model.Root + '/' + methods[I];
      for J := urmGet to urmHead do
        HttpServer.Route.Rewrite(J, tmpURI + '/', J, tmpURI);
    end;
end;

This can be called something like this (based on the pastee sample above):

  FixURIRoutingMess(aHttpsServer, ['method1']);

Last edited by squirrel (2023-11-08 07:32:19)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Difference in URL parsing between mORMot1 and 2

I will try to restore it back to what mORMot 1 does.

squirrel

Member

Registered: 2015-08-13

Posts: 155

Re: Difference in URL parsing between mORMot1 and 2

If it is possible, that will be wonderful!  Thanks @ab!

Last edited by squirrel (2023-11-08 08:00:06)

squirrel

Member

Registered: 2015-08-13

Posts: 155

Re: Difference in URL parsing between mORMot1 and 2

If there is a way to get OnBeforeURI to be triggered in this scenario, then I can do a redirect or handle it in many ways.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Difference in URL parsing between mORMot1 and 2

Please try https://github.com/synopse/mORMot2/commit/ffa5d754

squirrel

Member

Registered: 2015-08-13

Posts: 155

Re: Difference in URL parsing between mORMot1 and 2

Thanks @ab.  Looks like it is working now.

ttomas

Member

Registered: 2013-03-08

Posts: 135

Re: Difference in URL parsing between mORMot1 and 2

Migrate mormot2 to trunk and my production service fail.
I have TMyRestServer = class(TRestServerFullMemory) , without interface or published method, only OnBeforeUri.
OnBeforeUri never call!
procedure TRestServer.Uri(var Call: TRestUriParams) return Error in line 7553  // 4. decode..
never reach //6.  line 7589 OnBeforeUri
Older version of mormot2 from beginning of 2023 work OK.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Difference in URL parsing between mORMot1 and 2

The regression tests seem to run just fine.
https://luti.tranquil.it/listfolder/bc1 … b2cb437039

Line 7753 of TRestServer.Uri is about ctxt.Redirect(RootRedirectGet) - could you tell us a bit more why it returns?

ttomas

Member

Registered: 2013-03-08

Posts: 135

Re: Difference in URL parsing between mORMot1 and 2

Sorry for bad line numbers not from trunk. I don't use router. TMyRestServer receive different aRoot on creation 'newroot/webhook', BeforeUri response to POST request on 'newroot/webhook' and GET request to 'newroot/webhook/test' for testing that service is alive. In mormot1 and mormot2 before router, OnBeforeUri is called for all 'newroot/webhook*' requests.

  TMyRestServer = class(TRestServerFullMemory)
  private
    function BeforeUri(Ctxt: TRestServerUriContext): boolean;
  public
    constructor CreateWithOwnModel(aModelServer: ThkOrmRestServerDB;
      aProps: TSqlDBConnectionPropertiesThreadSafe;
      aShApiConfig: ThkConfigSh;
      const aRoot: RawUtf8;
      aKorBr, aMagBr, aDrzBr, aLoyArtPay: integer;
      aMapPay, aMapShip: ThkMapDynArray); reintroduce; overload;
  end;

implementation

constructor TMyRestServer.CreateWithOwnModel(
  aModelServer: ThkOrmRestServerDB;
  aProps: TSqlDBConnectionPropertiesThreadSafe;
  aShApiConfig: ThkConfigSh; const aRoot: RawUtf8;
  aKorBr, aMagBr, aDrzBr, aLoyArtPay: integer;
  aMapPay, aMapShip: ThkMapDynArray);
begin
  inherited CreateWithOwnModel([], false, aRoot);
  ...
  OnBeforeURI:=BeforeURI;
end;

Correct trunk line numbers in mormot.rest.server :-), line 7544 node := fRouter.Lookup(ctxt);, node is nil and never reach line 7584 OnBeforeUri(ctxt).
Lookup line 5978 result := pointer(TRestTreeNode(fTree[Ctxt.Method].Root).Lookup(p, Ctxt));, result is nil
I workaround comment lines 7543..7548, just to build new version of my service and everithing is OK now. Just workaround.

Last edited by ttomas (2024-01-18 10:38:35)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Difference in URL parsing between mORMot1 and 2

I am a bit confused about how you make the routing.
Try to find out what is registered in TRestRouter.Setup for you application, and why your requests are not routed.

If you don't use the REST routing, you could use the HTTP server routing, or HTTP interception.

ttomas

Member

Registered: 2013-03-08

Posts: 135

Re: Difference in URL parsing between mORMot1 and 2

I am a bit confused about how you make the routing.

By parsing URI in OnBeforeURI event.

Try to find out what is registered in TRestRouter.Setup for you application, and why your requests are not routed.
If you don't use the REST routing, you could use the HTTP server routing, or HTTP interception.

As I said before this RestServer doesn't have any ORM or SOA interface or method, just OnBeforeURI event and nothing is registered in Router. Adding:

  aWebhookRestServer.Router.Setup(mGET, 'test', rnMethod);
  aWebhookRestServer.Router.Setup(mPOST, '', rnMethod);

I solve problem for this project and now OnBeforeUri is called. I use MyRestServer just to separate code(baseness logic) and define newroot needed for reverse proxy. I will create two published methods and use router.rewrite instead of OnBeforeURI.

As this topic title said I have 2-3 years old mormot1 project broken to migrate to m2. This project replace some Magento(PHP)  eCommerce image processing URI, where I start using OnBeforeURI. RestServer root is 'Assets' and need to process:
http://localhost/Assets/image/Variable/P/a/t/h/image1.jpg
as static server serve original image and
http://localhost/Assets/img/<func>/<x>/<y>/Variable/P/a/t/h/image1.jpg
http://localhost/Assets/img/resize/200/200/Variable/P/a/t/h/image1.jpg
On first call resize original image to 200x200 and keep/cache on disk, next call serve image as static server.

I know this is not standard use of OnBeforeURI in RestServer and don't insist any change. This URI serving can be moved to HTTP server interception.
One possible change will be line 7546 if (node=nil) to try to call OnBeforeURI before ctx.Error('Invalid URI'...

Last edited by ttomas (2024-01-19 13:54:02)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Difference in URL parsing between mORMot1 and 2

You don't expect any ORM or SOA interface/method to be called, then you expect OnBeforeUri() to be called?
This was previously working by chance (by mistake).
The official documentation of OnBeforeUri() is exactly how it is implemented: "event trigerred when Uri() is about to execute an ORM/SOA command".

So instead of using OnBeforeUri, just use the routing available at the THttpServer level.
You will be able to trigger an event even before the TRestServer class is involved.

Sorry for the confusion.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Difference in URL parsing between mORMot1 and 2

Note that for
http://localhost/Assets/img/<func>/<x>/<y>/Variable/P/a/t/h/image1.jpg
the HTTP server routing would allow to specify the <func> <x> and <y> place holders and directly retrieve them in the implementation method.

I mean, you could specify the routing format as:
  http://localhost/Assets/img/<func>/<int:x>/<int:y>/<*>

Then retrieve the values using Ctxt.RouteUtf8('func'), Ctxt.RouteInt64('x'), Ctxt.RouteInt64('y') and Ctxt.RouteUtf8('path')

ttomas

Member

Registered: 2013-03-08

Posts: 135

Re: Difference in URL parsing between mORMot1 and 2

Thanks @ab, new router feature is great and very powerful addon.
Just one note, if I create SOA method <func>, without using any router.rewrite/setup, OnBeforeUri is called with all URI:
http://localhost/root/func/*
http://localhost/root/func/Variable/P/a/t/h/...
Is this as designed or working by chance (by mistake)?

Last edited by ttomas (2024-01-25 12:35:19)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: Difference in URL parsing between mORMot1 and 2

I guess it is as designed.

The SOA method-based service can have any kind of URI after its 'root/func/' base URI.
But it is up to the method-based service to parse the end of this URI.