urlEncode don't convert the "tilde" character

larand54 · 2020-08-10 10:56:09

Tries to use OAUTH2 in a project but a secret code contains '~'-characters and they are not replaced by the function.
While debugging I could se that this character belongs to the "unreserved characters" and will be left unchanged.

The OUTH2 service to get a token don't accept this character.

What can I do about it?
I no INDY has that function and that works but would like to make it without adding more libraries.

ab · 2020-08-10 12:41:43

The reference charset is the following:

    if c in ['_','-','.','~','0'..'9','a'..'z','A'..'Z'] then
      include(TEXT_CHARS[c], tcURIUnreserved);

and it indeed includes the '~'.

It was as requested by https://tools.ietf.org/html/rfc3986#section-2.3

2.3. Unreserved Characters
Characters that are allowed in a URI but do not have a reserved
purpose are called unreserved. These include uppercase and lowercase
letters, decimal digits, hyphen, period, underscore, and tilde.
unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"

This was our reference material - as quoted in the source code.

The problem is that this RFC 3986 from 2005 - 15 years ago! - is not followed.
Most code still follows https://tools.ietf.org/html/rfc1738 from 1994 - 26 years old.

So it is clearly a mess...

But http://jkorpela.fi/tilde.html seems to argue - with good background - that the tilde should be encoded, whatever the "new" RFC states...

Some users and code will expect '~' to be escaped...
I am just afraid that it may break existing code. It will break some of our regression tests for sure.

larand54 · 2020-08-10 16:15:55

Ok, I understand.
Well, good to know why it is like that, then it makes it more easy to accept.
In my case, for the moment at least, it won't be much of a problem as it is a fixed string that will not change often if ever so I only have to replace the '~':s with "%7E" manually.

ab · 2020-08-21 16:55:30

Since no one reacted, we will escape the "tilde", trusting the practice, and not the last RFC.

Check https://synopse.info/fossil/info/bbcf6fdbdd

macfly · 2020-08-21 17:21:40

Had not seen the blog post

I've seen and used addresses with ~.
(It is useful to access a server account (linux + apache) until the IP has propagated.)

I believe that if ~ is allowed in URLs it should be kept that way.
In the initial post it was not very clear how he is passing this secret parameter, is via Get? Would this be correct?

In any case, Speaking specifically of mORMot, this change does not affect my development.

ab · 2020-08-21 19:56:20

Thanks for the feedback.

mORMot Open Source

#1 2020-08-10 10:56:09

urlEncode don't convert the "tilde" character

#2 2020-08-10 12:41:43

Re: urlEncode don't convert the "tilde" character

#3 2020-08-10 16:15:55

Re: urlEncode don't convert the "tilde" character

#4 2020-08-21 16:55:30

Re: urlEncode don't convert the "tilde" character

#5 2020-08-21 17:21:40

Re: urlEncode don't convert the "tilde" character

#6 2020-08-21 19:56:20

Re: urlEncode don't convert the "tilde" character

Board footer