mORMot Open Source

larand54

Member

Registered: 2018-12-25

Posts: 104

urlEncode don't convert the "tilde" character

Tries to use OAUTH2 in a project but a secret code  contains '~'-characters and they are not replaced by the function.
While debugging I could se that this character belongs to the "unreserved characters" and will be left unchanged.

The OUTH2 service to get a token don't accept this character.

What can I do about it?
I no INDY has that function and that works but would like to make it without adding more libraries.

---

Delphi-11, WIN10

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: urlEncode don't convert the "tilde" character

The reference charset is the following:

    if c in ['_','-','.','~','0'..'9','a'..'z','A'..'Z'] then
      include(TEXT_CHARS[c], tcURIUnreserved);

and it indeed includes the '~'.

It was as requested by https://tools.ietf.org/html/rfc3986#section-2.3

2.3.  Unreserved Characters

   Characters that are allowed in a URI but do not have a reserved
   purpose are called unreserved.  These include uppercase and lowercase
   letters, decimal digits, hyphen, period, underscore, and tilde.

      unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"

This was our reference material - as quoted in the source code.

The problem is that this RFC 3986 from 2005 - 15 years ago! - is not followed.
Most code still follows https://tools.ietf.org/html/rfc1738 from 1994 - 26 years old.

So it is clearly a mess...

But http://jkorpela.fi/tilde.html seems to argue - with good background - that the tilde should be encoded, whatever the "new" RFC states...

Some users and code will expect '~' to be escaped...
I am just afraid that it may break existing code. It will break some of our regression tests for sure.

larand54

Member

Registered: 2018-12-25

Posts: 104

Re: urlEncode don't convert the "tilde" character

Ok, I understand.
Well, good to know why it is like that, then it makes it more easy to accept.
In my case, for the moment at least, it won't be much of a problem as it is a fixed string that will not change often if ever so I only have to replace the '~':s with "%7E" manually.

---

Delphi-11, WIN10

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: urlEncode don't convert the "tilde" character

Since no one reacted, we will escape the "tilde", trusting the practice, and not the last RFC.

Check https://synopse.info/fossil/info/bbcf6fdbdd

macfly

Member

From: Brasil

Registered: 2016-08-20

Posts: 374

Re: urlEncode don't convert the "tilde" character

Had not seen the blog post

I've seen and used addresses with ~.
(It is useful to access a server account (linux + apache) until the IP has propagated.)

I believe that if ~ is allowed in URLs it should be kept that way.
In the initial post it was not very clear how he is passing this secret parameter, is via Get?  Would this be correct?

In any case, Speaking specifically of mORMot, this change does not affect my development.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 14,655

Website

Re: urlEncode don't convert the "tilde" character

Thanks for the feedback.