CRITICAL - Load balancer & client IP address problem

mpv · 2016-11-16 12:42:50

Because our service has reached a critical load we put a Load balancer (nginx with realip_module) and run a several instances of application (HTTP.SYS based server). Some part of our application (encryption, user rights management, logging) require a knowledge about real client IP. BUT current THttpApiServer implementation (and all other HTTP servers) determinate a client IP address from a socket. In case of load balance the preferred way is to determinate a Client IP from a HTTP header, ballancer will add for us. In most case this is a X-Real-IP or X-Forwarded-For

I think we should add a property THttpServerGeneric.RealClientIPHeader: SockString and in case it set to something <> '' retrieve a real client IP address from value of RealClientIPHeader header.

In this case if I know I deploy my service behind the load balance, a will set a RealClientIPHeader property to the value, specified by my balancer and got a real client IP as expected. If not - the old realisation must work (to prevent a attack when hacker add a X-Forwarded-For by hand)

For a while I patch a HTTP API server RetrieveHeaders implementation (as a temporary solution) as such:

begin
  assert(low(KNOWNHEADERS)=low(Request.Headers.KnownHeaders));
  assert(high(KNOWNHEADERS)=high(Request.Headers.KnownHeaders));
  // MPV start X-Forwarded-For hack
  P := Request.Headers.pUnknownHeaders;
  RemoteIP := '';
  if P<>nil then
    for i := 1 to Request.Headers.UnknownHeaderCount do begin
      if (P^.NameLength = 15) and (StrLComp(P^.pName, 'X-Forwarded-For', 15) = 0) then begin
        SetString(RemoteIP, P^.pRawValue, P^.RawValueLength);
        break;
      end;
    end;
  if (RemoteIP = '') and (Request.Address.pRemoteAddress<>nil) then
 //MPV End X-Forwarded-For hack
    GetSinIPFromCache(PVarSin(Request.Address.pRemoteAddress)^,RemoteIP);
  // compute headers length

May be anybody know better solution? AB - can I add a THttpServerGeneric.RealClientIPHeader to the current trunc?

Last edited by mpv (2016-11-16 13:04:08)

mpv · 2017-11-23 16:54:43

@AB - I see in latest commit you improve a client IP detection. But usage of a constant HEADER_REMOTEIP_UPPER = 'REMOTEIP: ' is invalid in case servers are behind a reverse proxy. Is it possible to add a property THttpServerGeneric.RealClientIPHeaderUpper: SockString and in case it set to something <> '' retrieve a real client IP address from value of RealClientIPHeader header? This feature is critical for me.....

ab · 2017-11-23 21:54:09

Please check https://synopse.info/fossil/info/398681f7ca

It should work with http.sys and also with the socket server.

mpv · 2017-11-24 13:28:13

Thanks a lot!

edwinsn · 2021-05-20 09:38:39

I guess the value of TSQLHttpServer.HttpServer.RemoteIPHeader should be set to 'X-Forwarded-For' by default?
Because according to the Mozilla docs:

The X-Forwarded-For (XFF) header is a de-facto standard header for identifying the originating IP address of a client connecting to a web server through an HTTP proxy or a load balancer.

ab · 2021-05-20 12:06:08

The default is to retrieve the info from the socket itself, i.e. it is ''.

Idea is to change it only if a reverse proxy is used, to avoid an unneeded search via FindNameValue(headers).

Chaa · 2021-05-21 03:26:39

And in case if reverse proxy is not used, it is mandatory that RemoteIPHeader to be empty.
In other case client can arbitrarily change RemoteIP, that is possible security risk.

edwinsn · 2021-05-21 06:31:17

@ab and @chaa, Thanks for the explanation.

Last edited by edwinsn (2021-05-21 14:37:54)

ab · 2021-05-21 12:47:42

@Chaa
The security argument is the main one, for sure!

mpv · 2021-05-23 08:28:17

As noted above dafault value is '', and this is OK.
Be careful with X-Forwarded-For - it can contains several IP addresses in case of several reverse proxies, so better to use X-Real-IP and calculate it on the last reverse proxy - most of them know how to work with multiple IPs in X-Forvarded-For.

In case of nginx valid address can de retrieved using $remote_addr variable (and a proxies chain builds using $proxy_add_x_forwarded_for)

 
proxy_set_header    X-Real-IP  $remote_addr;
proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;

and in case of many proxies additional configuration is required to get a real IP form intermediate proxy we trust

  # use client real IP as reported by intermediate proxy
  real_ip_header    X-Forwarded-For;
  # A proxy we trust (address, hostname or mask)
  set_real_ip_from  b.b.b.b;

mORMot Open Source

#1 2016-11-16 12:42:50

CRITICAL - Load balancer & client IP address problem

#2 2017-11-23 16:54:43

Re: CRITICAL - Load balancer & client IP address problem

#3 2017-11-23 21:54:09

Re: CRITICAL - Load balancer & client IP address problem

#4 2017-11-24 13:28:13

Re: CRITICAL - Load balancer & client IP address problem

#5 2021-05-20 09:38:39

Re: CRITICAL - Load balancer & client IP address problem

#6 2021-05-20 12:06:08

Re: CRITICAL - Load balancer & client IP address problem

#7 2021-05-21 03:26:39

Re: CRITICAL - Load balancer & client IP address problem

#8 2021-05-21 06:31:17

Re: CRITICAL - Load balancer & client IP address problem

#9 2021-05-21 12:47:42

Re: CRITICAL - Load balancer & client IP address problem

#10 2021-05-23 08:28:17

Re: CRITICAL - Load balancer & client IP address problem

Board footer