Saturday 10 August 2024

Using Apache as a Reverse Proxy with Tailscale backend server

Apache as a Reverse Proxy

I started to use Tailscale to access my computers externally after moving to an ISP who use CGNAT. This enabled backups to and from my NAS from external servers I rented. I decided against opening up any of the machine within the Tailnet to full worldwide access, as that seemed too "open".

This works well, but I still had one website that I hosted on a RaspberryPi at home which I needed web access to. I originally just added my phone to the Tailnet, but there were problems with re-issuing LetsEncrypt certificates, and I wanted to retain my own domain name for the website, rather than use the .ts.net ones.

I had an external server on my Tailnet already running Apache, and decided the simple solution was to use that existing service to sit in front of the RaspberryPi and forward HTTP requests to it. After all, I only was using this website; it wasn't used by anyone else. This proved a little trickier than hoped, but eventually I got there, and these are the steps I went through...

Add VirtualHost to Apache

On the external server I added an website definition to Apache (i.e. in /etc/apache2/sites-available) which just consisted of the VirtualHost definition for the website domain:

<VirtualHost *:80>
    ServerName mydomain.com

    ProxyPass         "/" "http://tailnetname.xxxx-yyyy.ts.net/"
    ProxyPassReverse  "/" "https://mydomain.com/"
</VirtualHost>

where "mydomain.com" is the domain name you are exposing to the outside world, and "tailnetname.xxxx-yyyy" is the Tailnet name of the server actually hosting the website (in my case the RaspberryPi running locally).
Enable that site (a2ensite) and restart Apache.

Note that I'm using Apache to handle the SSL connection, and talking to the backend server over HTTP. This is just as secure, as all tailnet traffic is encrypted, and the tailnet name (.ts.net) is not exposed to the web (it has a CGNAT IP address, as do all machines in a tailnet). You could use HTTPS to get to the backend, but that seems a pointless overhead to me.

Issues I had later on were due to originally setting the ProxyPassReverse to the tailnet name, rather than mydomain.com - you want Apache to add headers to retain the external name of the website, not the tailnet name.

Adding LetsEncrypt Certificate

With the definition in Apache, use certbot to create your SSL certificate. certbot was already installed, so it was just a case of running "sudo cetbot certonly --apache" and letting certbot offer me the website to add the certificate to. I don't recall if this worked without any changes to the backend server, but I think it did. I manually updated the site definition in Apache, but you can let certbot do that by removing the 'certonly' parameter. Either way, you end up with the site definition amended with the following lines:

<VirtualHost *:80>
    ServerName mydomain.com

    RewriteEngine on
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

<VirtualHost *:443>
SSLCertificateFile /etc/letsencrypt/live/mydomain.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/mydomain.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf

- which are the standard lines to redirect any HTTP traffic to HTTPS, and point the site at the SSL certificates just created.

Changing the backend server

At this point, requests to https://mydomain.com should go via the external server, over your tailnet to the correct backend server (providing you have set the tailnet access rules appropriately - I have fairly open rules so nothing needed changing).

Although the connection was working, I kept getting 404 errors in the browser. This was caused by a number of issues, which needed fixing.

Firstly, the website definition was still setup to handle SSL traffic from the outside world, (as above) so the HTTP requests were being redirected to HTTPS.
The mod_rewrite rules and VirtualHost for port 443 needed removing.

Secondly, the VirtualHost definition still included mydomain.com as the ServerName. This meant Apache (on the RaspberryPi) was confused by the response pointing at mydomain.com, as it could be served locally, I think. In short I needed to just have the ServerName set as "tailnetname.xxxx-yyyy.ts.net" and remove any references to mydomain.com.

The final issue was I had the flask-talisman module installed in the web application - this does its own redirect of HTTP to HTTPS requests, and this was the final cause of the 404 responses (as there was no local handler for port 443 on the website). Maybe I could have avoided two of these issues by sticking with SSL on the backend, but there we are. I initially fixed this by just removing talisman, but eventually just changed the options, as I still wanted it to add the CSP headers, and so on.

Additional Changes

As the backend is now always receiving requests via the proxy, logging needs to handle the different HTTP headers, to record the actual external details, rather than those of the proxy.






No comments: