UPDATE 2016/08/10: AWS improved their ELB and they now support websocket and http/2 protocols. More on this link https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/.

AWS ELB does not support WSS protocol on its HTTPS endpoints. If you are using it for load balancing this becomes a blocker for scaling your service. Hopefully there is a way to overcome this limitation.

Switching ELB protocols to TCP/SSL will not be enough as the server will not receive X-Forwarded-For header anymore.

To solve this you will need to

  • Add ProxyProtocol policy to ELB so it starts using proxy_protocol
  • Enable proxy_protocol support on nginx (Play unfortunatelly cannot be configured to understand proxy protocol at this moment :( )

Proxy protocol adds additional header to requests to pass server client’s ip which can be used if there is a load balancer between your server and clients.

How it looks?

PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP + single space + PROXY_IP + single space + CLIENT_PORT + single space + PROXY_PORT + "\r\n"

Configuring ELB

Adding policy to ELB is for now only available through aws-cli. You can download it from here.

1. Add Policy

aws elb create-load-balancer-policy \
    --load-balancer-name <AWS_ELB_NAME> \
    --policy-name My-proxy-protocol \
    --policy-type-name ProxyProtocolPolicyType \\
    --policy-attributes AttributeName=ProxyProtocol,AttributeValue=True

2. Configure ELB

AWS ELB SSL configuration

3. Configure nginx on your instance

Enable proxy_protocol support on nginx on your instance (and use it as a template for any other instance in your scaling pool).

# ELB PROXY protocol handler
server {
        listen 81 proxy_protocol;
        listen [::]:81 proxy_protocol;

        server_name _;

        root /var/www/html;

        set_real_ip_from 172.31.48.0/20; # ELB CIDR

        real_ip_header proxy_protocol;
        proxy_set_header X-Real-IP       $proxy_protocol_addr;
        proxy_set_header X-Forwarded-For $proxy_protocol_addr;

        location / {
                proxy_pass http://127.0.0.1:9000/;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $host;
        }
}

Important notes:

  • We use Nginx here to pass requests to Play running on port 9000 on the same instance.
  • Port 80 is kept here to redirect requests to https and it cannot use proxy protocol as ELB will not enable proxy protocol on HTTP/HTTPS listeners (which is a must on port 80)

Here is a sample nginx configuration for port 80

# Redirect everything to https
server {
        listen 80 default_server;
        listen [::]:80 default_server;

        server_name _;

        return 301 https://$host$request_uri;
}

4. Attach policy to ELB

Attach policy to ELB so it adds proxy protocol to all requests that are passed to port 81 on the instance

aws elb set-load-balancer-policies-for-backend-server \
    --load-balancer-name <ELB_NAME> \
    --instance-port 81 \
    --policy-names My-proxy-protocol

5. Open port 81 for EC2 security group :)

Just a reminder :)

6. Test it out

Test your setup by using ‘wss://’ in your requests. Websocket.org gives you an easy way to test websockets.