Pages: 1

#1 2024-10-02 07:53:55

zen010101
Member
Registered: 2024-06-15
Posts: 66

Accessing this URL using TWinHttp will report an error

Regardless of whether OpenSSL is used as the TLS layer, accessing https://z.cn using TWinHttp on Windows will cause winhttp.dll to report an error. 

winhttp.dll error 2EF1 (The operation has been canceled)

Whose fault is this?

  HttpClient := TSimpleHttpClient.Create;
  HttpClient.Tls^.IgnoreCertificateErrors := True;
  if HttpClient.Request('https://z.cn', 'GET') = 200 then
    ConsoleWrite('Length(HttpClient.Body): %', [Length(HttpClient.Body)], ccGreen)
  else
    ConsoleWrite('LastError: %', [HttpClient.LastError], ccRed);

Offline

#2 2024-10-02 09:06:31

profh
Member
Registered: 2010-07-02
Posts: 161

Re: Accessing this URL using TWinHttp will report an error

i got similar error before, after i modified code in file mormot.net.client.pas as follow, now it works:

function TWinHttp.InternalGetProtocols: cardinal;
...
result := WINHTTP_FLAG_SECURE_PROTOCOL_TLS1;
...

modified to:

function TWinHttp.InternalGetProtocols: cardinal;
...
  result := WINHTTP_FLAG_SECURE_PROTOCOL_TLS1 or
            WINHTTP_FLAG_SECURE_PROTOCOL_SSL2 or
            WINHTTP_FLAG_SECURE_PROTOCOL_SSL3;
...

Offline

#3 2024-10-02 12:06:12

zen010101
Member
Registered: 2024-06-15
Posts: 66

Re: Accessing this URL using TWinHttp will report an error

profh wrote:

i got similar error before, after i modified code in file mormot.net.client.pas as follow, now it works:

...

modified to:

function TWinHttp.InternalGetProtocols: cardinal;
...
  result := WINHTTP_FLAG_SECURE_PROTOCOL_TLS1 or
            WINHTTP_FLAG_SECURE_PROTOCOL_SSL2 or
            WINHTTP_FLAG_SECURE_PROTOCOL_SSL3;
...

1. I found this in the code:

  // WINHTTP_FLAG_SECURE_PROTOCOL_SSL2 and WINHTTP_FLAG_SECURE_PROTOCOL_SSL3
  // are unsafe, disabled at Windows level, therefore never supplied

2. Even if I add SSL2 and SSL3, my program still prompts the same error.

Offline

#4 2024-10-02 13:16:18

profh
Member
Registered: 2010-07-02
Posts: 161

Re: Accessing this URL using TWinHttp will report an error

zen010101 wrote:

1. I found this in the code:

  // WINHTTP_FLAG_SECURE_PROTOCOL_SSL2 and WINHTTP_FLAG_SECURE_PROTOCOL_SSL3
  // are unsafe, disabled at Windows level, therefore never supplied

1、sometimes the server i want to access applies the unsafe PROTOCOL i have to accord to.

2、with https WINHTTP used to be complex in different windows version, so in mormot.defines.inc file,
     i define FORCE_OPENSSL and take libssl-3-x64.dll and libcrypto-3-x64.dll together.

{$define FORCE_OPENSSL}

3、usually it is ok, but some other time it is not enough, perhaps the dependence is needed, 
    for instance VC_redist with related version.

Offline

#5 2024-10-02 14:47:46

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,655
Website

Re: Accessing this URL using TWinHttp will report an error

I don't think this is a SSL 2 or SSL 3 compatibility issue.

The https://z.cn website properly uses TLS 1.3:

~$ curl --verbose https://z.cn
*   Trying 54.222.60.252:443...
* Connected to z.cn (54.222.60.252) port 443 (#0)
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):

and even disable SSL 2 ans SSL 3:
https://www.ssllabs.com/ssltest/analyze.html?d=z.cn

Which version of Windows are you using?
There seem to be some problems with TLS 1.3 support on some Windows 11, with our SChannel TLS layer.

Offline

#6 2024-10-02 19:06:19

zen010101
Member
Registered: 2024-06-15
Posts: 66

Re: Accessing this URL using TWinHttp will report an error

ab wrote:

Which version of Windows are you using?
There seem to be some problems with TLS 1.3 support on some Windows 11, with our SChannel TLS layer.

Windows 11 23H2 22631.4169 and Windows Server 2016. Both not work.

Offline

#7 2024-10-04 13:43:27

flydev
Member
From: France
Registered: 2020-11-27
Posts: 73
Website

Re: Accessing this URL using TWinHttp will report an error

On my side and also on Windows 11 23H2 22631.4169, test of mormot v2.2.8610 (commit 34b4c7c from 2024-10-02) fail at fetching mustache specs from github (in test.core.data).

!  - Mustache renderer: 5 / 62 FAILED  1m36

Running microsoft curl.exe with `curl --verbose https://raw.githubusercontent.com/mustache/spec/master/specs/interpolation.json` fail

* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
* schannel: failed to decrypt data, need more data

but with original curl.exe, there is no error (it use embedded CA bundle)..

*   Trying 185.199.110.133:443... (githubusercontent.com)
* ALPN: curl offers h2,http/1.1
...
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / [blank] / UNDEF
* ALPN: server accepted h2

On a embedded computer running on win7, the mustache test passed.

Note, without hijacking op thread, an os exception was triggered on first run and a second run returned a different result:
- first run log: https://pastebin.com/raw/PFQxFmez
- second run: https://pastebin.com/raw/xDLU21V0

Windows 7 SP1 (6.1.7601) [WinAnsi 1.7GB 1DB10A01]    4 x Intel(R) Celeron(R) CPU J1900 @ 1.99GHz [1MB] (x86)    on iEi H788 V1.00

Offline

#8 2024-10-10 04:46:17

zen010101
Member
Registered: 2024-06-15
Posts: 66

Re: Accessing this URL using TWinHttp will report an error

Update:

Tested on the latest Windows 11 update, the issue remains.


Version Windows 11 Home Chinese Version
Version number 23H2
Installation Date 2024/9/8
Operating System Version 22631.4317

Offline

#9 2024-10-10 06:16:41

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,655
Website

Re: Accessing this URL using TWinHttp will report an error

!  - Mustache renderer: 5 / 62 FAILED  1m36

They should pass with yesterday's trunk. I disabled SChannel to retrieve the content.

On a embedded computer running on win7, the mustache test passed.
Note, without hijacking op thread, an os exception was triggered on first run and a second run returned a different result:

Sounds like if this CPU is way too slow to generate the RSA keys without a timeout.

Offline

#10 2024-10-10 09:28:21

flydev
Member
From: France
Registered: 2020-11-27
Posts: 73
Website

Re: Accessing this URL using TWinHttp will report an error

@ab, confirmed: All tests passed successfully.

Sounds like if this CPU is way too slow to generate the RSA keys without a timeout.

This makes sense, thanks you.

Offline

#11 2024-10-10 16:23:41

zen010101
Member
Registered: 2024-06-15
Posts: 66

Re: Accessing this URL using TWinHttp will report an error

I tried the latest trunk, the winhttp.dll exception remains.

Here is the call stack:

#0 FPC_RAISEEXCEPTION at :0
#1 RaiseLastModuleError($0000000100130B58^: 'winhttp.dll', ExceptClass($000000010012F898)) at mormot.core.os.windows.inc:634
#2 EWinHttp.RaiseFromLastError(^$__vtbl_ptr_type($000000010012F898)) at mormot.net.client.pas:3827
#3 TWinHttp.InternalQueryDataAvailable(TWinHttp($0000000001587310)) at mormot.net.client.pas:3803
#4 TWinHttpApi.InternalRetrieveAnswer(TWinHttpApi($0000000001587310), 'HTTP/1.1 301 Moved Permanently'#$0D#$0A'Connection: keep-alive'#$0D#$0A'Date: Thu, 10 Oct 2024 16:19:24 GMT'#$0D#$0A'Transfer-Encoding: chunked'#$0D#$0A'Content-Type: text/html; charset=iso-8859-1'#$0D#$0A'Location: http://www.amazon.cn/ref=z_cn?tag=zcn0e-23'#$0D#$0A'Server: Server'#$0D#$0A'Strict-Transport-Security: max-age=47474747; includeSubDomains; preload'#$0D#$0A#$0D#$0A, '', '', '') at mormot.net.client.pas:3533
#5 THttpRequest.Request(THttpRequest($0000000001587310), '', 'GET', 10000, '', '', '', 'HTTP/1.1 301 Moved Permanently'#$0D#$0A'Connection: keep-alive'#$0D#$0A'Date: Thu, 10 Oct 2024 16:19:24 GMT'#$0D#$0A'Transfer-Encoding: chunked'#$0D#$0A'Content-Type: text/html; charset=iso-8859-1'#$0D#$0A'Location: http://www.amazon.cn/ref=z_cn?tag=zcn0e-23'#$0D#$0A'Server: Server'#$0D#$0A'Strict-Transport-Security: max-age=47474747; includeSubDomains; preload'#$0D#$0A#$0D#$0A, '') at mormot.net.client.pas:3366
#6 TSimpleHttpClient.Request(TSimpleHttpClient($0000000001581DF0), TUri (True; nlTcp; 'https'; 'z.cn'; '443'; ''; ''; ''), 'GET', '', '', '', 10000) at mormot.net.client.pas:4530
#7 THttpClientAbstract.Request(THttpClientAbstract($0000000001581DF0), 'https://z.cn', 'GET', '', '', '', 10000) at mormot.net.client.pas:4425
#8 $main at test.lpr:56
#9 SYSTEM_$$_MAIN_WRAPPER$POINTER$POINTER$$INT64+6 at :0
#10  at :0

I noticed that the code below is not being executed at all:

  if SChannelEnableTls13 and // should be explicitly enabled
     ((OSVersion in [wEleven, wEleven_64]) or
      (OSVersion >= wServer2022_64)) then

Offline

#12 2024-10-10 18:15:28

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,655
Website

Re: Accessing this URL using TWinHttp will report an error

Yes, TWinHttp does not use this code.
It directly wraps the windows API.

But the stack trace helps.
The failing WinHttp API call is InternalQueryDataAvailable().

I have identified several potential issues.
Especially https://github.com/synopse/mORMot2/comm … e786a27368
Please try with the latest trunk.

Offline

#13 2024-10-11 17:42:17

zen010101
Member
Registered: 2024-06-15
Posts: 66

Re: Accessing this URL using TWinHttp will report an error

This time no exceptions were raised, instead of an error message: "Failed to retrieve the webpage. Status: 301".

I wonder why it doesn't automatically redirect to the new address? After all, in the TSimpleHttpClient.Create method, I read:

fConnectOptions.RedirectMax := 4; //seems fair enough

Last edited by zen010101 (2024-10-11 17:43:02)

Offline

#14 2024-10-11 18:28:25

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,655
Website

Re: Accessing this URL using TWinHttp will report an error

In the WinHttp API, redirections are not done automatically.
We may add them.

Happy to see that it was indeed a connection closed which triggered the initial unexpected error.

Offline

#15 2024-10-12 15:53:06

zen010101
Member
Registered: 2024-06-15
Posts: 66

Re: Accessing this URL using TWinHttp will report an error

ab wrote:

In the WinHttp API, redirections are not done automatically.
We may add them.

Very much looking forward to this feature coming online .

Offline

#16 2024-10-15 12:42:41

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,655
Website

Re: Accessing this URL using TWinHttp will report an error

Please try
https://github.com/synopse/mORMot2/commit/f9e6cf26

Offline

#17 2024-10-16 08:35:58

zen010101
Member
Registered: 2024-06-15
Posts: 66

Re: Accessing this URL using TWinHttp will report an error

ab wrote:

All works but TWinHttp.Get('https://z.cn', '', True{IgnoreTlsCertificateError}, nil, nil, 3000)) not, it seems not reading the RedirectMax property value.

Neither HttpGet('https://z.cn', ''); .

Last edited by zen010101 (2024-10-16 08:52:48)

Offline

#18 2024-10-16 16:30:04

zen010101
Member
Registered: 2024-06-15
Posts: 66

Re: Accessing this URL using TWinHttp will report an error

We have several entries in mormot.net.client.pas to initiate an HTTP GET Request, but their default RedirectMax is a bit confusing:

1. IHttpClient/TSimpleHttpClient: 4 times, can be changed before calling Request
2. TWinINet.Get(): automatic, unknown max times value
3. TWinHttp.Get(): 0 times
4. OpenHttpGet() function: 5 times
5. HttpGet() function: 0 times if use winhttp or 5 times as same as OpenHttpGet
6. THttpClientSocketWGet.WGet/WGet() function: 0 times default, can be changed when called

It might be more friendly if we could unify the default values of RedirectMax for different objects or functions written in different periods.  smile

Offline

Pages: 1

Board footer

Powered by FluxBB