← writing featured

The Safest Login Page is the One I Never Published

I finally got tired of not knowing whether anyone reads this thing, so I spent a Saturday afternoon standing up Umami in the homelab. The install was the easy part; the instructions are dead simple.

Then I got to the bit where you expose it to the internet. I went along with the advice everyone gives, stopped halfway and decided it wasn’t good enough for me. So, I did what I normally do and stubbornly wandered off to my own path.

Quick bit of context for anyone who hasn’t gone down this particular hole. Umami is a self-hosted analytics platform: a small Node app with a PostgreSQL database to maintain persistence sans the cookies and third parties slupring up all the data. You drop one script tag on your site, it posts a little blob of JSON on each page view and you get a dashboard. Couldn’t be easier. My install runs in an LXC on a member node of my homelab’s Proxmox cluster.

I landed on it for the “normal” reasons. Cloudflare’s Analytics are free but limited, they don’t give me precisely what I’m looking for and they’re a bit too slow for my liking. And, I’m not even going to give Google a single thought; they have enough of my data already.

Which leaves the obvious question: how does a script tag on a public website safely and securely reach a container in my house? Without punching a hole in my router?

The answer everyone gives you.

The standard answer is a Cloudflare Tunnel. A small daemon, cloudflared, runs next to your service and dials out to Cloudflare. No dicking around with portforwarding, firewall rules or opening up your router to the world. Traffic arrives at Cloudflare, goes down the pipe your daemon already opened and lands on your service. It’s genuinely lovely and it costs nothing. And since I already use Cloudflare to currently host my static websites and domains it was just the pragmatic option. We’ll see how long I’m comfortable with this, but moving on…

The second half of the standard answer is where I nearly went wrong.

Umami has an admin interface. An admin interface typically has a login page. So every guide, quite reasonably, tells you to put something like Cloudflare Access in front of it. Access is an authentication layer at the edge. Basically, someone hits your hostname, Cloudflare intercepts, they sign in against an identity provider and only then does the request continue to your box.

I had the tab open, I was about to wire it to my IDP aaaand then I actually stopped to read my own ingress config. What followed was a very slow eyebrow raise.

Tunnels match on path.

The important little detail I clumsily skimmed past was that a tunnel’s ingress rules do not just match on hostname. They match on path.

I had written this, which is the shape everyone starts with:

HCL
ingress = [
  {
    hostname = "analytics.nightcity.network"
    service  = "http://umami:3000"
  },
  {
    service = "http_status:404"
  },
]

A single hostname and everything on it goes straight to the app while anything else gets a 404. Ok.

But my analytics service only needs to expose two things to the public internet; the tracker script and the collector endpoint. That is the entirety of the desired contract. Nobody on the internet needs to see /login or the dashboard, the settings, the user management or the API the dashboard talks to. So, why was I publishing them and then buying a lock to place in front of it?

HCL
ingress = [
  {
    hostname = "analytics.nightcity.network"
    path     = "^/bundle\\.js$"
    service  = "http://umami:3000"
  },
  {
    hostname = "analytics.nightcity.network"
    path     = "^/api/v1/data$"
    service  = "http://umami:3000"
  },
  {
    service = "http_status:404"
  },
]

I only need precisely two paths out while everything else gets met with a lovely 404. Not “protected” or hidden behind a challenge or login screen; just absent.

My admin interface still exists, of course ( I am not typing SQL to read my own page views ). It sits on my home network behind my own reverse proxy, where it has always been, reachable from the couch and over the VPN I already run for everything else. I did not lose anything. I just stopped handing the entirety of it to the internet and then slapping a gate over it.

Drawn out, the whole arrangement is two roads that happen to share a name:

asked from the internet

asked from my LAN

104.21.x.x

10.0.0.200

/bundle.js

/api/v1/data

everything else

analytics.nightcity.network

Cloudflare DNS

Technitium

Cloudflare edge

My reverse proxy

cloudflared

Umami

404

cloudflared is the one deciding what gets through and it’s only aware of two paths. My side of the picture never touches Cloudflare, or the rest of the internet, at all. The fork at the very top is split horizon DNS: one name, two answers, depending on where you ask from. Hold that thought, because it comes back to bite me later.

Why I think this is the better trade.

I want to be fair to Access here, because it’s a good product and there are plenty of setups where it’s the right call. If you genuinely need to reach an admin panel from anywhere on a machine you don’t control, without a VPN, Access is exactly the kind of tool that’ll get you there.

But if you don’t need that, compare what you’re actually buying.

With Access, /login is on the internet. It returns a challenge instead of a form, which is much better than nothing, but the surface is still there. There’s an auth flow to configure, an IDP to keep working and an additional sign-in every time. Umami doesn’t consume the Access token, so you authenticate to Cloudflare and then authenticate again to Umami. Nobody has time for that nonsense.

With path scoping, /login returns a 404. There is no flow to misconfigure because there is no flow. There is no IDP to keep alive and manage. There is nothing to leave accidentally open when you change something else eighteen months from now.

Elimination is the ultimate form of hardening.

Pretty much every cybersecurity professional.

The thing I keep coming back to: an attack surface you removed cannot be misconfigured later. A control you added can. This is all about shrinking the attack surface, which is a far older idea than my Saturday afternoon. We’re not doing anything new here. But a standard installation path is built to get the typical user running quickly and quick doesn’t necessarily mean appropriate for a setup like this.

There’s a small bonus, too. The default Umami tracker lives at /script.js and posts to /api/send and those exact strings are what content blockers match on. You can rename both with a config value, but because I only published the renamed paths, the default ones now 404 for the whole internet as a side effect. I didn’t plan that, but I’ll take it.

If you want a sense of how indiscriminate that matching is: my own resolver blocks umami.is. Not the tracker endpoint. The homepage. Of the analytics tool I chose, installed and run myself. I found out while trying to link to it from this post and that link up there still doesn’t work from my desk. Because I’m lazy and haven’t updated by Technitium blocklists yet.

Does it work though?

The nice part about this setup is that you can test the public path without leaving the house. curl --resolve lets you skip your own DNS and dial the Cloudflare edge directly, while still sending the right hostname, so the request takes the same road a real visitor does.

Bash
curl -sI --resolve analytics.nightcity.network:443:104.21.x.x \
  https://analytics.nightcity.network/login | head -1

Run that against a handful of paths and you get:

/                  404
/login             404
/bundle.js         200
/script.js         404
/api/v1/data       405
/api/send          404

That 405 is the collecter endpoint telling me it only accepts POST, which is exactly what I want to see from a GET. It does admit the endpoint exists, where a 404 would not, but that one has to be findable anyway. It’s sitting in the script on every page. Everything else gets a polite middle finger.

Meanwhile, from inside the house, the same hostname gives me the full dashboard. The same two roads as before and the split falls out of the DNS I already run for network-wide adblock via a Technitium cluster.

Something worth pointing out.

I flipped the ingress config, re-ran my checks and /script.js came back 200. Which was alarming, because I had just watched it 404.

It was cached. Umami serves its tracker with a 24 hour cache header. My earlier testing had pulled it through the edge and Cloudflare was dutifully serving me the copy it already had. Which, to be fair, is exactly what it should normally be doing. A request with a junk query string came back 404 immediately, which confirmed the origin was doing its job and the edge was just being helpful at the worst possible moment.

Worth knowing if you ever unpublish something: the edge does not find out until its copy expires. Cache invalidation is my passion.

Hark! A plot twist!

I shipped the script tag, confirmed real traffic was landing, felt very clever about the whole thing and then Firefox showed me this:

Go home Firefox, you are drunk.

Sir, this is my own site from my machine on my home network. How dare you?

So, I run split horizon DNS. Inside my network, analytics.nightcity.network resolves to a private address on my LAN so I get the dashboard directly. Outside, where you are reading from, the same name resolves to Cloudflare and goes down the tunnel. That’s what makes the two-faced thing work. I mentioned this before in my previous article My Blog Now Ships From My Homelab! and it’s one of my favourite aspects about my setup.

But when I load my public blog from my own house, the browser sees a page served from a public origin trying to load a subresource from 10.x.x.x. Browsers have started treating that as exactly the attack it usually is: a website on the internet quietly poking at your router, your printer or your NAS. Firefox calls it Local Network Access and blocks it by default. Chrome is heading the same way.

So the one place on earth where my analytics silently do not work is the chair I’m sitting in.

I sat with that for a minute before realising it’s correct behaviour on every level. The browser is right to block it. My DNS is right to answer the way it does. And analytics that ignore my own visits are analytics that aren’t polluted by me hammering refresh on my own blog. This is effectively all three of these systems disagreeing correctly.

If it ever bothers me, the fix is to give the tracker its own hostname in a domain my internal resolver doesn’t answer for, so it goes out to the edge from everywhere including my lounge room. I have not bothered. I’m all too happy to skip my own page views.

The one thing path scoping does not fix.

There is one limitation here and I’d rather name it than let you find it on your own.

Moving the admin interface off the internet does nothing for the collecter endpoint. It can’t. That endpoint has to accept anonymous POSTs from every visitor’s browser, or it isn’t an analytics endpoint. Umami rejects requests carrying a website ID it doesn’t recognise, which stops idle nonsense, but my real website ID is sitting in the page source of every post I’ve ever written. Literally anyone can read it.

So the only real control is applying some kind of rate limit. I’ll admit up front this is more of a speed bump rather than a wall. Cloudflare’s free tier, which I’m currently on because I’m cheap and actually trying to reduce my subscription costs, permits one rule at a ten second window and it insists on counting per data centre rather than globally. It’ll stop something stupid. But, it will not stop something determined.

The honest summary is that path scoping shrinks the surface enormously and then stops. All that’s left is the stuff that’s supposed to be open.

We’ll see how this goes, but worst case I can always stick Nginx between Umami and cloudflared with something like the following:

Nginx
... boilerplate ...

location = /api/v1/data {
    limit_req        zone=collect burst=20 nodelay;
    limit_req_status 429;
    proxy_pass       http://umami:3000;
}

... more boilerplate ...

Running out of disk space with garbage data points would really be the worst case scenario here. Which would no-shit make me laugh.

In closing …

None of this is Cloudflare-specific either, by the way. Any reverse proxy has a path-scoped location block and any tunnel worth the name matches on more than a hostname. The vendor is incidental. If you want to use Tailscale, Pangoline or Netbird, then go for your life. The idea is just to publish only what has to be public.

None of this is novel. Tunnels have matched on path since forever, it’s in the docs and I’m certain plenty of people are already doing exactly this. I just hadn’t thought about it properly, because the well-trodden advice is “put an auth layer in front of it” and this kind of consensus is usually good enough that you stop thinking.

But “protect the admin panel” and “publish the admin panel and then protect it” are not the same sentence and you should be aware of the difference.

If you’re running anything through a tunnel right now, it’s worth five minutes with your ingress config. Ask yourself which paths actually need to leave your network. For a lot of self-hosted things the honest answer is “fewer than all of them”. The best answer is “none”.

And if you’d rather find out than wonder, that curl --resolve trick works against any hostname, including yours. Start with /login and see what comes back.

And if you’re about to tell me the browser permission dialog is a bug, it isn’t. It’s three correct systems arguing and I’ve decided that’s a feature. So, there.

If you want to pull on any of these threads …

I kept the theory light in the post, because naming a principle tends to make a small idea sound like it needs the backup and it kind of goes against the vibe of the site and my own casual style of writing. But the ideas underneath this are old and much better argued elsewhere, so here’s where I’d start.

  • The Protection of Information in Computer Systems, Saltzer and Schroeder, 1975. The source of fail-safe defaults ( deny by default, allow by exception, which is exactly what that catch-all 404 rule is ) and economy of mechanism ( fewer moving parts means fewer parts to get wrong ). It’s a fifty year old paper and it reads like one, but section I is short and it has aged as disgracefully as I have.
  • Attack Surface Analysis, OWASP. The plain-English version of the whole post, minus my anecdotes. Useful if you want a structured way to ask “what am I actually exposing” about something larger than a script tag.
  • Tunnel configuration file, Cloudflare. The ingress rules reference. This is the page I had skimmed past twice before noticing that path was sitting right there next to hostname.
  • Local Network Access, WICG. The spec behind the permission dialog that ambushed me. If you’d rather have prose than a spec, Chrome’s New permission prompt for Local Network Access covers the same ground and explains why browsers decided this needed fixing.

If you only read one, make it the Saltzer and Schroeder. Almost everything I thought I worked out that evening turns out to be in there, described better, by people who got to it first and who are far smarter than me.

Comments

3 revisions since publication

  • 89ce545 +0 −0 update og image for accuracy
  • 2d82463 +13 −13 some more edits to the latest thing
  • d6d7b32 +17 −4 ok, good enough for me; shipit!
  • 1edf020 +204 −0 published