Hosting multiple Heroku apps on a single domain

In this guest engineering post from Plane's CEO Matt Pelc, he discusses how a single application on Heroku can have any number of domains assigned to it, but you can only add a domain to one app.

Matt Pelc

PEO vs EOR office building against a purplish sky

A single application on Heroku can have any number of domains assigned to it, but you can only add a domain to one app. This means that by default you can’t serve example.com from the example-1 while example.com/blog is served from example-2.We ran into this problem with plane.com recently where we have a constellation of apps (pilot-co, pilot-blog, pilot-stories, etc.) which we wanted to host under a single domain.We found a way to do that by putting a custom HAProxy instance, also hosted on Heroku, in front of all other Heroku apps we use.

1. Set up

Let’s say you have two apps on Heroku already:

example-com running https://example.com

example-blog running https://blog.example.com

We will need a new app for your load balancer:

$ mkdir load-balancer

$ cd load-balancer

$ git init .

Then create an app on Heroku:

$  heroku apps:create example-lb

Creating example-lb... done, stack is cedar-14

http://example-lb.herokuapp.com/ | https://git.heroku.com/example-lb.git

Git remote heroku added

2. Installing Docker

You will deploy it to Heroku using Docker. We found it to be easier to manage than creating a custom buildpack.Luckily, installing Docker on your machine is easy. Get Docker Toolbox and follow its setup instructions.To verify that you have a working Docker installation, open your terminal and run:

docker ps

CONTAINER ID        IMAGE               COMMAND ...

$ docker-compose --version

docker-compose version: 1.4.0

To deploy a Docker container to Heroku you will need heroku-docker:

$ heroku plugins:install heroku-docker

Heroku requires an app.json and Procfile manifests to be able to run your app.

  "name": "Pilot Load Balancer",

  "description": "A load balancer for pilot.co",

Your Procfile should look something like this:

web: sbin/haproxy -f haproxy.cfg

Then initialise Docker assets for the app:

$ heroku docker:init

Wrote Dockerfile

Wrote docker-compose.yml

3. Configuring HAProxy

Your Dockerfile is where we add instructions for Heroku on how to compile HAProxy:

FROM heroku/cedar:14

RUN mkdir -p /app/user

WORKDIR /app/user

# Install HAProxy

RUN apt-get update && apt-get install -y libssl1.0.0 libpcre3 --no-install-recommends && rm -rf /var/lib/apt/lists/*

ENV HAPROXY_MAJOR 1.5

ENV HAPROXY_VERSION 1.5.14

ENV HAPROXY_MD5 ad9d7262b96ba85a0f8c6acc6cb9edde

# see http://sources.debian.net/src/haproxy/1.5.8-1/debian/rules/ for some helpful navigation of the possible "make" arguments

RUN buildDeps='curl gcc libc6-dev libpcre3-dev libssl-dev make' \

	&& set -x \

	&& apt-get update && apt-get install -y $buildDeps --no-install-recommends && rm -rf /var/lib/apt/lists/* \

	&& curl -SL "http://www.haproxy.org/download/${HAPROXY_MAJOR}/src/haproxy-${HAPROXY_VERSION}.tar.gz" -o haproxy.tar.gz \

	&& echo "${HAPROXY_MD5}  haproxy.tar.gz" | md5sum -c \

	&& mkdir -p /app/user/src/haproxy \

	&& tar -xzf haproxy.tar.gz -C /app/user/src/haproxy --strip-components=1 \

	&& rm haproxy.tar.gz \

	&& make -C /app/user/src/haproxy \

		TARGET=linux2628 \

		USE_PCRE=1 PCREDIR= \

		USE_OPENSSL=1 \

		USE_ZLIB=1 \

    PREFIX=/app/user \

		all \

		install-bin \

	&& rm -rf /app/user/src/haproxy \

	&& apt-get purge -y --auto-remove $buildDeps

COPY haproxy.cfg /app/user/haproxy.cfg

One last thing we need to do is configure HAProxy to route requests from our main app (called frontend) to all other apps (called backends).HAProxy’s configuration manual is relatively easy to understand, and after some fine-tuning you should end up with something like this:

global

    maxconn 256

defaults

    mode http

    timeout connect 5000ms

    timeout client 50000ms

    timeout server 50000ms

frontend http

    bind 0.0.0.0:$PORT

    option forwardfor

    # Force SSL

    redirect scheme https code 301 if ! { hdr(x-forwarded-proto) https }

    # Redirect all requests to /blog* to the example-blog app.

    use_backend example-blog if { path_beg /blog }

    # And all other requests to example-com.

    default_backend pilot-com

backend pilot-com

    http-request set-header X-Forwarded-Host example.com

    http-request set-header X-Forwarded-Port %[dst_port]

    reqirep ^Host: Host:\ example-com.herokuapp.com

    server example-com example-com.herokuapp.com:443 ssl verify none

backend example-blog

    http-request set-header X-Forwarded-Host example.com

    http-request set-header X-Forwarded-Port %[dst_port]

    reqirep ^Host: Host:\ example-blog.herokuapp.com

    server example-blog example-blog.herokuapp.com:443 ssl verify none

You can verify your setup locally by starting Docker:

$ docker-compose up web

and opening the browser:

$ open "http://$(docker-machine ip default):8080"

4. Deploying your load balancer to Heroku

If you’re satisfied with the outcome, it’s time to deploy it to Heroku:

heroku docker:release

heroku open

After you verified that your new setup works on https://example-lb.herokuapp.com, you can remove the example.com domain from example-com and attach it to example-lb.

5. After you’re done

Requests to https://example.com will go through example-lb and be served from example-com.

Requests to https://example.com/blog will also go through example-lb but be served from example-blog instead.

All this will be completely hidden from your users. At no point they should see example-blog.herokuapp.com or any domain other than example.com.

If you’re using SSL (which this guide assumes you were) you can safely remove the SSL add on from all apps other than example-lb. Traffic between Heroku apps will be encrypted using their *.herokuapp.com certificate.