WANTED: Single High-Volume Web Service Seeks Compatible Web Proxy

We have 3 web servers we can't get rid of. Can you help?

One day at the Rubicon Project we looked up and realized that we have at least three web servers, and somehow we just can’t get rid of any of them. We’re using:

Apache 2.2 for mod_rewrite and our mod_proxy_http_sla hack
HAProxy as a proxy
nginx as a cache

This is ridiculous! But try as we might, we just can’t replace any of these features with a half-baked implementation given the huge scale at which we operate. What we’d like is a single open source web proxy that has the best of all worlds. Something that has:

HAProxy’s small resource footprint
a simple web cache that works
mod_rewrite’s awesome url rewrite support
mod_proxy_http_sla‘s millisecond proxy timeouts
mod_proxy_http_sla‘s ability to serve default content rather than 50x errors on backend failures
a simple API to add/remove backend hosts without editing configuration files

We can’t be the only web service looking for such a thing. What say you, internets? Can it be done?

Tags: Apache, haproxy, nginx, open source, web proxy, web server, web service