For awhile I’ve liked the idea of using a VPS for “critical” services. Currently looking at running:
- Authentik
- Komodo (with periphery agents on local boxes)
- Uptime Kuma
- NTFY
- Panglolin (or Cosmos Cloud?)
So, first of all, to folks already using a VPS, do you think it’s worth it? Do you think I’m missing anything? Happy to discuss/research alternatives, too. I’ve thought about TinyAyuth+PocketID in place of Authentik. While I think Authentik is probably more complex (and likely overkill), it’s a single solution. That said, I haven’t played with TinyAuth/PockedID.
Second, I was pretty interested in Pangolin until I saw Cosmos Cloud mentioned elsewhere. It seems like it actually ticks a lot of boxes:
- Built-in authentication
- Reverse Proxy
- VPN (At least for local-to-VPS connection, but possibly also for external clients?)
- Docker management(?): They have an “app store” that’s all docker images, so there’s some docker capability built-in. Not sure yet if it can handle multiple hosts like Komodo.
- DNS (I would still keep at least 1 local pi-hole instance)
Looking at the doc for chaining proxies and hiding IP, here, it mentions creating an A record for services hosted on a different server. I’m curious to know if this means Cosmos will only manage DNS for services hosted on the same box. Honestly this seems kind of odd, unless I’m misunderstanding how proxy servers work.
Anyway, I know this was a bit of a meandering post. Curious to know thoughts on my original plan, but also if anyone has played with Cosmos, I’d like to hear your thoughts.
Lastly: This morning, I found this interesting write-up to manage container updates using Forgejo, Renovate, and Komodo. Another rabbit hole to explore!
EDITS:
- Spelling
I remember the first Linux server I stood up on a VPS. It got thoroughly hacked almost immedietly. Not only did they hack the server, they set up attack vectors on other servers…aaaand a bitcoin miner. Got up that morning, checked mail, and there was a nastygram from my host wanting to know WTF over. Since then, I did a ton of reading, took a couple basic online courses for my own edification. I now tend to go overboard on security now days if that is possible. I’ve been told my set up is way over engineered. However, it’s been ticking along these many, many years now without issue. Also, since I am the only user of my network, it’s a little easier to lock down. Users create complexities and complexities cause issues.
I’m sure you have done the leg work in bolstering your knowledge base in setting up your first VPS server, but as others have said, beware. It reminds me of the movie Constantine, where just beyond light, in the shadows, lurk thousands and thousands of demons. They are sophisticated bots too, and are quite autonomous.
Authentik
In my reading, tho I don’t run it, VoidAuth is supposed to be lighter than Authentik. Do you have a directive or purpose sketched out for your server? What you want to accomplish, etc?
VPN (At least for local-to-VPS connection, but possibly also for external clients?)
Tailscale is my choice for my VPN overlay on the server. I also use the evil Cloudflare Tunnel/Zero Trust. All devices also run their own VPN.
I have played around with Cosmos. Pretty neat little package, especially for someone just starting out. I can’t speak to it’s performance, but I read glowing reviews. YunoHost would be in that category as well, with a very large app catalog.
Looks like you are heading in the right direction.
reminds me of my first mail server, accidentally set up an open relay and got a lot of abuse reports from mail providers saying they blocked my server due to it. Took forever to get fixed again.
I mean, there can be some serious consequences, especially if your server starts attacking other servers. They don’t take that shit lightly.
oh for sure, it made sense that they wanted to make sure was fixed. Just was super alarming the speed it was advertised that the relay was there!
I like what you are doing, but I want to tell a little story for educational purposes:
Once there was an IT pro (like in “it’s their job to do stuff like that for money, but with a team and funds”) who thought he’s smarter than the internet and started selfhosting on a VPS for the family. Nothing dramatic, just some nextcloud stuff and games for the kids. Everything was secure and always up-to-date, backups to multiple locations, the works.
This worked for years, but one night Mr. Smartypants made a tiny config change that needed to be reversed for full security. But he forgot about it. And it took only 3 hours for a bad actor to exfil all data and burn the VPS to the ground. And that was before AI started to roam the web…
You get the idea: you can do everything right, but then you will make that one tiny mistake or forget an update. With the internet at its actual state you can asume all your data will be compromised or gone at some point. Just make sure that’s ok for you.
If you put any data anywhere, assume it will be contained in a breach in the future. Blue teaming is hard. You have to be perfect every time. Red teaming is easy. You just have to wait till the blue team makes a simple mistake…
Mails sent to a company? Their or your mail account will be breached one day.
Account details on a webpage? Their user database will be leaked.
Your cloud drive, ect.
Even your data on your NAS at home or on your PC could get accessed in one way or another, you’re just a zero day and an unfortunate click away from disaster.
On the upside, as long as you do not have a target on your back, patch your stuff in a timely manner and keep some hygiene in configs, secops … You should be fine, as most automated attacks aren’t that high level and target the low hanging fruits. But that doesn’t make you completely safe.
Blue teaming is hard.
After 20 years in the field I’d say it’s an impossible job. We are always 5 steps behind. Now with AI, 27 steps. I quit IT to keep my sanity.
And don’t listen to the promises of big companies with billion-funds, they cook with water like the rest of us. See " Amazon infiltrated by North-Korea ".
And don’t listen to the promises of big companies with billion-funds, they cook with water like the rest of us.
Oh, yeah, I remember when one of our super important core systems was migrated to SaaS. My system was interfacing with it, so I got notified about the migration to make a plan. I basically told them: “We’re using $API to connect to your system. Tell me the new hostnames and IP ranges and ports and I will configure the firewall on our end. Also, our connector for your migrated system will be deployed in $IP-range so please allow these inbound connections in your firewall.”
Half a year later I got a message: “Yeah, we just found out, that $SaaS-provider never configured the firewall after our tickets and everything is reachable from the public internet. We’re forcing them to lock down the system now. Can you please tell us again from where you connect to us?”
That N Korea story was wild!
NGL, it’s scary! I still haven’t put anything of consequence on the VPS. Right now, I have everything on 1 system going through swag. I’ve long been a proponent of not fixing what isn’t broken, so I might look for a way to scale my current setup to more than the one system.
I host a couple of small services for more than just me, but I have no plans to provide any cloud services to anyone else but my wife (if she wants).
Thanks for the story, though. Sorry you went through that. I’ve been lucky so far. I’m hoping things stay that way, but obviously trying to plan for when they don’t!
Sorry you went through that.
Me? Who said that was me? I never said that. How rude. Nothing like that could ever happen to a brilliant guy like me. No, you shut up!
I still haven’t put anything of consequence on the VPS
Maybe keep it that way? It’s just not worth it in the end. If you just want to play with new tech on a VPS, have at it. But maybe without important data, and make sure to tell every user of your services about the risks. Because in the end, you are responsible. You are not Microsoft, a company that was never held accountable for billions in corporate damages through cyber-securitty bullshit.
Cosmos sounds kind of interesting, if I’m understanding correctly the vps only job would be the lighthouse. And the fact once they are connected they don’t have to go through the lighthouse anymore sounds good. However the cost to get to features is concerning to me, I don’t mind paying but I want to make sure it’s what I want first .
I can’t speak to cosmos, but I have Pangolin, Kuma, and vaultwarden on a hetzner vps and I love it. I run everything in docker compose files on a Debian host. Right now I have authentik on one of my sites, but I think I will be switching to voidauth also hosted on the vps in the near future
Im on the same boat, Pangolin on a vps with some other services. Really inpressed with pangolin and usung pocket id for oauth witg great success
Hadn’t heard of voidauth. Will check it out. Thanks!
