getHTTP problem

esmondb · 2013-01-09 00:55:10

I'm trying to use the getHTTP function in SynCRTSock like this:

memo1.Text := HttpGet('www.ecb.europa.eu', '80', '/stats/eurofxref/eurofxref-daily.xml');

It's returning 200 OK but the content is empty (while a web browser works). Any suggestions on what's wrong - could it be because it's fetching an xml file?

esmondb · 2013-01-09 08:17:12

Just tried this again with Delphi 2007 and it works. But still have the problem above with XE2

ab · 2013-01-09 08:19:32

Could you step in the function using the debugger, and find out what is the content length retrieved, or if there is an error which stops the download?

esmondb · 2013-01-09 09:18:03

The content length is -1

It looks like the content should be read in at line 2439 of SynCRTSock but this just loops 3 times reading a empty string.

esmondb · 2013-01-21 16:32:35

Looking at this again I've found the procedure THttpSocket.GetHeader in SynCrtSock is not returning any headers under Delphi XE2 (called by THttpClientSocket.Request). With Delphi 2007 a full set of headers, including content-length, is received.

Both XE2 and 2007 receive the first line 'HTTP/1.1 200 OK' but with XE that's all and if the http request is run a second time without restarting the app even this first line isn't received.

Any tips on how to de-bug this?

Thanks

ab · 2013-01-22 09:10:52

Could you use the debugger and find out where no data is returned?

Reading should be made from function InputSock() in this case.

Perhaps we have some issue with the low-level record alignment in SynWinSock.pas, due to some regressions (features?) of Delphi XE2.

esmondb · 2013-01-22 12:16:18

It stops returning data the after reading the first header line. THttpSocket.GetHeader doesn't work as readln returns an empty string in SockRecvLn(s) when it should return the first line header line after 'HTTP/1.1 200 OK'

I'm a bit puzzled by InputSock(). The HTTP response is 2070 bytes long so I expected it to be called 3 times with 1k chucks but it gets called 5 times with Recv being passed 0 as the size the last two times. With Delphi 2007 it only gets called once but works. How many times should InputSock get called?

ab · 2013-01-22 13:40:21

It should be called only once, if the HTTP header size fits in one 1KB chunk (which sounds to be the case).

Then HTTP body is then retrieved at once, directly from the socket.

Is F.BufSize=0 in XE2?

esmondb · 2013-01-22 14:22:28

F.BufSize=1024 in XE2.

InputSock() is only called once before THttpSocket.GetBody. The rest of the calls are from GetBody.

One strange thing I've noticed is F.BufPtr gets truncated to 124 characters after the first call to InputSock when looked at at the start of the the InputSock function.

noobies · 2013-02-22 12:04:15

i have some problem. (XE2 and XE3 on latest source)
may be you write small sample of httpget:
1) get xml
2) get jpeg

Last edited by noobies (2013-02-25 05:59:05)

ab · 2013-06-10 15:09:26

It is now fixed by http://synopse.info/fossil/info/91cfffdaf8

Thanks for the report.

esmondb · 2013-06-20 08:10:09

Thanks, just tried it and it working fine.

noobies · 2014-02-04 06:15:29

hi again, it work fine for normal sites, but if i load slow sites i raised exception class ECrtSocket with message 'SockRecvLn 1'.
site slow load in browser 6-10 second, but load!
http://kobsme.dzo-kostroma.ru/index.php?page_id=3
my project - free infomat for medical institutions, please help how load data from slow site

noobies · 2014-02-04 09:07:27

i solve my problem, but may be add HttpGet TimeOut parameter?

function HttpGet(const server, port: AnsiString; const url: RawByteString): RawByteString;
var Http: THttpClientSocket;
begin
  result := '';
  Http := OpenHttp(server,port); <--- next see OpenHttp declaration
  if Http<>nil then
  try
    if Http.Get(url)=200 then
      result := Http.Content;
  finally
    Http.Free;
  end;
end;

function OpenHttp(const aServer, aPort: AnsiString): THttpClientSocket;
begin
  try
    result := THttpClientSocket.Open(aServer,aPort); <--- next see Open declaration
  except
    on ECrtSocket do
      result := nil;
  end;
end;
function Open(const aServer, aPort: AnsiString): TCrtSocket;
begin
  try
    result := TCrtSocket.Open(aServer,aPort);
  except
    on ECrtSocket do
      result := nil;
  end;
end;

constructor Open(const aServer, aPort: AnsiString; aLayer: TCrtSocketLayer=cslTCP; aTimeOut: cardinal=5000); <--- bingo find aTimeOut parameters and set 15000 and work fine.

Last edited by noobies (2014-02-04 09:07:51)

noobies · 2014-02-18 13:26:58

hi, i again get problem, in my work on win xp, win 7 work fine, no problem.
but on infomat pc (win 7 maximal) httpget dont work.
all computers work in same proxy. code for test:

ShowMessage(HttpGet('www.ecb.europa.eu', '80', '/stats/eurofxref/eurofxref-daily.xml'));

in module syncrtsock i place message for test, after message size sin = 16, long wait and then message connect failed and empty message.

function CallServer(const Server, Port: RawByteString; doBind: boolean; aLayer: TCrtSocketLayer): TSocket;
...
  showmessage('try connect');
  showmessage('size sin = ' + IntToStr(SizeOfVarSin(Sin))); // size sin = 16
  if Connect(result,Sin)<>0 then begin
     CloseSocket(result);
     showmessage('connect failed');

Last edited by noobies (2014-02-18 13:27:28)

ab · 2014-02-18 14:53:18

Try to use either TWinINet.Get() or TWinHTTP.Get() methods.
They are more stable in some cases (e.g. TWinHTTP is the default for a mORMot's TSQLHttpClient).

noobies · 2014-02-19 05:12:13

i dont understand, try all variant but no one not work.

ShowMessage(HttpGet('koscrb.dzo-kostroma.ru', '80', 'index.php?page_id=1'));
ShowMessage(TWinHTTP.Get('koscrb.dzo-kostroma.ru/index.php?page_id=1'));
ShowMessage(TWinINet.Get('koscrb.dzo-kostroma.ru/index.php?page_id=1'));

my program use chromium and in chromium this link koscrb.dzo-kostroma.ru/index.php?page_id=1 work fine, no problem connection and in any browser on this pc (ie, chrome) link also works well.

I'd like to use mormot HttpGet, very simple command, but this bug puzzles me.

i install last version OverbyteIcs, try use and this code works

  With HttpCli1 do begin
    URL            := 'http://koscrb.dzo-kostroma.ru/index.php?page_id=1';
    Proxy          := '10.44.1.163';
    ProxyPort      := '3128';
    RequestVer     := '1.1';
    RcvdStream := TMemoryStream.Create;
    try
        Get;
    except
      RcvdStream.Free;
      Exit;
    end;
    RcvdStream.Seek(0,0);
    Memo1.Lines.LoadFromStream(RcvdStream);
    RcvdStream.Free;
  end;

Last edited by noobies (2014-02-19 05:44:58)

ab · 2014-02-19 10:00:25

If I understand well, the difference between computers is also the proxy definition?

AFAIK TWinINet.Get() will transparently use the system (aka Internet Explorer) proxy settings.

noobies · 2014-02-19 15:59:34

ab wrote:

If I understand well, the difference between computers is also the proxy definition?
AFAIK TWinINet.Get() will transparently use the system (aka Internet Explorer) proxy settings.

no, all computers have some proxy. i know about proxy setting and they are the same everywhere.
I do not understand why HttpGet not working on 4 computers.
In my home pc on win 8.1 work fine without problem.

mORMot Open Source

#1 2013-01-09 00:55:10

getHTTP problem

#2 2013-01-09 08:17:12

Re: getHTTP problem

#3 2013-01-09 08:19:32

Re: getHTTP problem

#4 2013-01-09 09:18:03

Re: getHTTP problem

#5 2013-01-21 16:32:35

Re: getHTTP problem

#6 2013-01-22 09:10:52

Re: getHTTP problem

#7 2013-01-22 12:16:18

Re: getHTTP problem

#8 2013-01-22 13:40:21

Re: getHTTP problem

#9 2013-01-22 14:22:28

Re: getHTTP problem

#10 2013-02-22 12:04:15

Re: getHTTP problem

#11 2013-06-10 15:09:26

Re: getHTTP problem

#12 2013-06-20 08:10:09

Re: getHTTP problem

#13 2014-02-04 06:15:29

Re: getHTTP problem

#14 2014-02-04 09:07:27

Re: getHTTP problem

#15 2014-02-18 13:26:58

Re: getHTTP problem

#16 2014-02-18 14:53:18

Re: getHTTP problem

#17 2014-02-19 05:12:13

Re: getHTTP problem

#18 2014-02-19 10:00:25

Re: getHTTP problem

#19 2014-02-19 15:59:34

Re: getHTTP problem

Board footer