Robust detection of HTTP client IP address including connections through reverse proxy, load balancer, web acceleration and application firewall

The Hypertext Transfer Protocol (HTTP), the protocol which is driving the World Wide Web under the hood, inherently supports detection of the other end’s IP address by examining the Transmission Control Protocol (TCP) connection properties.

Why does one need to perform anything special to accurately detect the client IP address?

Here is why: examining TCP connection properties for the client endpoint IP address only works well when the connection between the client and the server is direct, meaning no reverse proxy servers, application firewalls, load balancers, web accelerators or content delivery networks (CDNs) (such as Limelight, Cotendo, Akamai and many others) are involved in between (all involving a reverse proxy of some kind that way or another)

What is a reverse proxy?

A reverse proxy (in a nutshell) is a web server acting as a proxy (meaning forwarding the requests of or executing requests on behalf of) the client, moreover, more than one reverse proxy server might be involved in delivering and serving an HTTP request, which means that the TCP client endpoint which the server sees, is the originating IP address of the last reverse proxy server in the chain and not the address of the original client.

image

This property of proxy servers is widely used for anonymization (hiding one’s Internet activity traces).

So, how does one know the IP address of the real client?

Since IP addresses are such a crucial piece of information for debugging, analysis and other intelligence, a mechanism has been devised to carry the client IP address from the first reverse proxy server in the chain to the web server servicing the request which is using HTTP request header for transport.

Various HTTP header fields control various aspects of the HTTP request and response, and the greatest part about them is that the information they carry is extensible, meaning anyone can add headers to the request as long as they conform to the HTTP protocol specification.

X-Forwarded-For saves the day

The X-Forwarded-For (XFF) HTTP header field is a de facto standard for identifying the originating IP address of a client connecting to a web server through an HTTP proxy or load balancer.

The general format of the field is:

X-Forwarded-For: client, proxy1, proxy2

where the value is a comma+space separated list of IP addresses, the left-most being the original client, and each successive proxy that passed the request adding the IP address where it received the request from. In this example, the request passed through proxy1, proxy2, and then proxy3 (not shown in the header). proxy3 appears as remote address of the request.

By examining the X-Forwarder-For HTTP header field value, you can track the real IP address of a client on the Internet accessing your web server in a reverse proxy scenario, even if your web server is not routable from the Internet.

Security considerations

Since it is easy to forge an X-Forwarded-For field the given information should be used with care. The last IP address is always the IP address that connects to the last proxy, which means it is the most reliable source of information. X-Forwarded-For data can be used in a forward or reverse proxy scenario.

You should NOT trust all X-Forwarded-For information in this scenario as you may have received bogus information from the Internet. As such a trust list should be used to make sure that proxy IP addresses in the X-Forwarded-For field are trusted by you.

Code sample

For convenience, I have created a C# class which implements the above information for determining the client IP address.

Here’s a short usage example for this class:

using System;
using System.Collections.Generic;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Net;

public partial class _Default : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
        // HttpClientInfo contains the information regarding the         // current client IP address
        HttpClientInfo client =             HttpClientInfo.Create(HttpContext.Current);

        // The first member of the ClientIPAddresses array is the         // client IP address
        // (either delivered through a reverse proxy, or directly)
        IPAddress ip = client.ClientIPAddresses[0];

        Response.Write("Real client IP address: " + ip.ToString());
    }
}

The HttpClientInfo class

The code sample above would be useless without the actual HttpClientInfo class implementation.

Here is the code listing for the class:

using System;
using System.Collections.Generic;
using System.Web;
using System.Net;
using System.Collections.Specialized;

/// <summary>
/// Http client information with reverse proxy support.
/// </summary>
public class HttpClientInfo
{
    // Private data members
    private IPAddress[] m_clientIPAddresses = new IPAddress[] { };
    private string[] m_reverseProxyChain = new string[] { };

    /// <summary>
    /// Client IP address chain, including reverse proxies.
    /// 
    /// The first IP address in the chain will be the address of the real client.
    /// </summary>
    public IPAddress[] ClientIPAddresses
    {
        get
        {
            return m_clientIPAddresses;
        }
    }

    /// <summary>
    /// Reverse proxy information chain.
    /// </summary>
    public string[] ReverseProxyChain
    {
        get
        {
            return m_reverseProxyChain;
        }
    }

    /// <summary>
    /// Private constructor to prevent instances of this class to be created directly.
    /// </summary>
    private HttpClientInfo()
    {
    }

    /// <summary>
    /// Create client info object from HttpRequest.
    /// </summary>
    /// <param name="request">HTTP request</param>
    /// <returns>HTTP client information</returns>
    public static HttpClientInfo Create(HttpRequest request)
    {
        HttpClientInfo result = new HttpClientInfo();

        result.m_clientIPAddresses = GetClientIPs(request);
        result.m_reverseProxyChain = GetReverseProxyServers(request);

        return result;
    }

    /// <summary>
    /// Create client info object from HttpContext.
    /// </summary>
    /// <param name="context">HTTP context</param>
    /// <returns>HTTP client information</returns>
    public static HttpClientInfo Create(HttpContext context)
    {
        return Create(context.Request);
    }

    /// <summary>
    /// Create client info object from current HTTP context.
    /// </summary>
    /// <returns>HTTP client information</returns>
    public static HttpClientInfo Create()
    {
        return Create(HttpContext.Current.Request);
    }

    /// <summary>
    /// Get reverse proxy servers chain information.
    /// </summary>
    /// <param name="context">HTTP request</param>
    /// <returns>Array of proxy servers descriptions</returns>
    private static string[] GetReverseProxyServers(HttpRequest request)
    {
        List<string> result = new List<string>();

        // Via and X-Via are de facto standard headers for proxy server chain identification
        result.AddRange(GetValues(request.Headers, "X-Via"));
        result.AddRange(GetValues(request.Headers, "Via"));

        return result.ToArray();
    }

    /// <summary>
    /// Get client IP address chain including reverse proxies.
    /// </summary>
    /// <param name="context">Request HTTP context</param>
    /// <returns>Array of client IP addresses</returns>
    private static IPAddress[] GetClientIPs(HttpRequest request)
    {
        List<IPAddress> result = new List<IPAddress>();

        // There might be several X-Forwarded-For headers
        foreach (            string ipValues in            GetValues(request.Headers, "X-Forwarded-For")        )
        {
            // IP addresses can be comma-delimited
            foreach (                string ip in                ipValues.Split(                    new char[] { ',', ' ', '\t', '\r', '\n' },                    StringSplitOptions.RemoveEmptyEntries                )            )
            {
                result.Add(IPAddress.Parse(ip.Trim()));
            }
        }

        // Either real client IP or the last reverse        // proxy server in the chain
        result.Add(IPAddress.Parse(request.UserHostAddress));

        return result.ToArray();
    }

    /// <summary>
    /// Get all values for specified NameValueCollection keys    //// if they exist.
    /// </summary>
    /// <param name="nameValueCollection">Input name/value collection</param>
    /// <param name="keys">Keys to look for</param>
    /// <returns>Array of values</returns>
    private static string[] GetValues(NameValueCollection nameValueCollection, params string[] keys)
    {
        List<string> result = new List<string>();

        foreach (string key in keys)
        {
            string[] values = nameValueCollection.GetValues(key);

            if (values != null)
            {
                result.AddRange(values);
            }
        }

        return result.ToArray();
    }
}
Advertisements

Generating a cryptographically secure random number in Microsoft SQL Server T-SQL

Among the applications for random number generators are gambling, statistical sampling, computer simulation and cryptography (the latter probably being the most significant application of all).

The diversity of applications has led to the development of several methods of generating random numbers.

A true random number generator measures some physical phenomena (such as thermal noise, or user interaction delays, etc.) and uses the measurement results to generate random numbers, while a pseudo-number generator (found in most programming environments) uses a deterministic algorithm to generate a sequence of numbers which appear random. This sequence is not truly random since it is determined by initializing what is known as a “seed.”  A seed is a random event, such as the current time of day in milliseconds, that is used to simulate a different sequence every time it is used.

A pseudo-random number generator cannot be regarded as a “true” or cryptographically secure random number generator since it’s output is predictable.

Microsoft SQL Server T-SQL language provides two methods for generating random numbers:

  1. The RAND() function, which generates pseudo-random numbers
  2. The CRYPT_GEN_RANDOM() function, which generates a cryptographically secure random number sequence of the specified length

To generate a sequence of 10 random bytes, CRYPT_GEN_RANDOM() can be used as follows:

declare @random_bytes varbinary(10)

set @random_bytes = CRYPT_GEN_RANDOM(10)

— @random_bytes now contains a sequence of 10 random bytes

However, the documentation does not explain how to generate a usable random number within the specified range. Let’s start with an arbitrary random number:

declare @random_int int
declare @random_bigint bigint

set @random_int = convert(int, CRYPT_GEN_RANDOM(4))
— @random_int now contains an arbitrary random integer

set @random_bigint = convert(bigint, CRYPT_GEN_RANDOM(8))
— @random_bigint now contains an arbitrary random bigint

To limit the range of the generated number, it is enough to calculate it’s modulus after dividing by the range limit + 1:

declare @random_int int
declare @random_bigint bigint

set @random_int = ABS(convert(int, CRYPT_GEN_RANDOM(4))) % 10
— @random_int now contains a random integer between 0 and 9

set @random_bigint = ABS(convert(bigint, CRYPT_GEN_RANDOM(8))) % 10
— @random_bigint now contains a random bigint between 0 and 9

Notice the use of ABS() function used to limit the result to positive numbers

Under the hood, CRYPT_GEN_RANDOM() uses the Windows API CryptGenRandom() function which is used by various security components in the system, which in turn allows for multiple processes to contribute to a system-wide seed, which is also combined with various system and user data such as the process ID and thread ID, the system clock, the system time, the system counter, memory status, free disk clusters and the hashed user environment block.

Authenticating with Google Service Account in C# (JWT)

To support scenarios where an unattended application accesses Google data, Google introduced the concept of Service Accounts which allows for unattended log in using JWT (JSON Web Token).

Having fought with the somewhat incomplete documentation and code samples, I decided to summarize and explain the working code here for the benefit of all.

To use the sample code provided here you will need:

  • A Google Account
  • A Google Service Account client e-mail address which is obtained from the Google API Console. The client e-mail address looks like:
    XXX@developer.gserviceaccount.com
  • A Google Service Account private key file (privatekey.p12) which is also obtained from the Google API Console
  • A C# compiler

For simplicity, I created a .NET class called GoogleJsonWebToken with a public static method GetAccessToken which performs the authentication:

public static dynamic GetAccessToken(string clientIdEMail, string keyFilePath, string scope)

The method returns a dictionary with the Google access token accessible by the key “access_token”:

var auth = GoogleJsonWebToken.GetAccessToken(
    clientIdEmail,
    keyFilePath,
    GoogleJsonWebToken.SCOPE_ANALYTICS_READONLY);

Console.WriteLine("Google access token: {0}", auth["access_token"]);

This token can later be returned as part of the standard Google OAuth2 API GetAuthentication delegate result IAuthorizationState:

AuthorizationState result = new AuthorizationState();

result.AccessToken = auth["access_token"];
result.AccessTokenExpirationUtc = DateTime.UtcNow.AddMinutes(30);

return result;

For your convenience I include the full source code of the GoogleJsonWebToken C# class:

using System;
using System.Text;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Web.Script.Serialization;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
using System.Net;

public class GoogleJsonWebToken
{
    public const string SCOPE_ANALYTICS_READONLY = "https://www.googleapis.com/auth/analytics.readonly";

    public static dynamic GetAccessToken(string clientIdEMail, string keyFilePath, string scope)
    {
        // certificate
        var certificate = new X509Certificate2(keyFilePath, "notasecret");

        // header
        var header = new { typ = "JWT", alg = "RS256" };

        // claimset
        var times = GetExpiryAndIssueDate();
        var claimset = new
        {
            iss = clientIdEMail,
            scope = scope,
            aud = "https://accounts.google.com/o/oauth2/token",
            iat = times[0],
            exp = times[1],
        };

        JavaScriptSerializer ser = new JavaScriptSerializer();

        // encoded header
        var headerSerialized = ser.Serialize(header);
        var headerBytes = Encoding.UTF8.GetBytes(headerSerialized);
        var headerEncoded = Convert.ToBase64String(headerBytes);

        // encoded claimset
        var claimsetSerialized = ser.Serialize(claimset);
        var claimsetBytes = Encoding.UTF8.GetBytes(claimsetSerialized);
        var claimsetEncoded = Convert.ToBase64String(claimsetBytes);

        // input
        var input = headerEncoded + "." + claimsetEncoded;
        var inputBytes = Encoding.UTF8.GetBytes(input);

        // signiture
        var rsa = certificate.PrivateKey as RSACryptoServiceProvider;
        var cspParam = new CspParameters
        {
            KeyContainerName = rsa.CspKeyContainerInfo.KeyContainerName,
            KeyNumber = rsa.CspKeyContainerInfo.KeyNumber == KeyNumber.Exchange ? 1 : 2
        };
        var aescsp = new RSACryptoServiceProvider(cspParam) { PersistKeyInCsp = false };
        var signatureBytes = aescsp.SignData(inputBytes, "SHA256");
        var signatureEncoded = Convert.ToBase64String(signatureBytes);

        // jwt
        var jwt = headerEncoded + "." + claimsetEncoded + "." + signatureEncoded;

        var client = new WebClient();
        client.Encoding = Encoding.UTF8;
        var uri = "https://accounts.google.com/o/oauth2/token";
        var content = new NameValueCollection();

        content["assertion"] = jwt;
        content["grant_type"] = "urn:ietf:params:oauth:grant-type:jwt-bearer";

        string response = Encoding.UTF8.GetString(client.UploadValues(uri, "POST", content));

        var result = ser.Deserialize<dynamic>(response);

        return result;
    }

    private static int[] GetExpiryAndIssueDate()
    {
        var utc0 = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc);
        var issueTime = DateTime.UtcNow;

        var iat = (int)issueTime.Subtract(utc0).TotalSeconds;
        var exp = (int)issueTime.AddMinutes(55).Subtract(utc0).TotalSeconds;

        return new[] { iat, exp };
    }
}