This isn’t a docker issue but an issue of running a non-deterministic process on the host system. If you must run an LLM run it inside a docker container.
rootless podman ftw.
I’ve tried setting up rootless podman on my machine but I just could not get it to work for some reason. It’s been a while so I unfortunately don’t remember the error message but I think it had something to do with the docker socket being unavailable or podman not having the permission to access it.
Damn straight. Yes, docker has rootless now, although I rarely see it in the wild (as I’m converting it to rootless podman), no one seems to care or know. I was using user level quadlets (really like the syntax and structure) before that, and you get SELinux as well. Everything that doesn’t really need it should not have root, pretty much the basic level of Linux (back to Unix) security and Docker is like, ‘Don’t Care, Got Root ;)’.
I’ve been converting my entire docker stack to rootless. You run into a number of issues where certain applications simply cannot handle having that limited access, especially on smaller or more niche projects. The documentation doesn’t offer solutions so you are just shit out of luck unless you can troubleshoot it.
principle of least privilege
That’s the one.
This is the exact trap with agents, they will use whatever privilege is lying around, they don’t stop and think ‘should I?’ like a person might. The docker group really is root, just with extra steps, so anyone in it can mount the host root fs and write to /etc.
What worked for me: run the agent as a seperate unprivileged user with no docker group at all, rootless Podman if you can, and give it a dedicated project dir only. Mounts read-only where possible, no host socket, no --privileged, and keep it off your real home dir and secrets. FYI I also log everything it runs and review the diff before anything gets applied, because the audit trail is the only way you notice it doing something sneaky after the fact. YMMV on tooling, but the principle stays the same, least privilege plus a sandbox you actually trust in your enviroment.
Running LLM on a server? Sounds like a recipe for disaster. What for?
I am running most of my agents in https://git.jeena.net/jeena/agent-container but every now and then I want it to be able to fix something globally for me and be able to test it.
I feel like I read the exact same story (Person discovers docker group is effectively root because llm agent used it to circumvent not having sudo) a few minths ago. (Not saying you stole it, two people can have the same experience independently of each other. I just found it interesting that this is, apperently, such a commonly overlooked issue.)
Also, this is documented:
Running containers (and applications) with Docker implies running the Docker daemon. This daemon requires root privileges unless you opt-in to Rootless mode, and you should therefore be aware of some important details.
First of all, only trusted users should be allowed to control your Docker daemon. This is a direct consequence of some powerful Docker features. Specifically, Docker allows you to share a directory between the Docker host and a guest container; and it allows you to do so without limiting the access rights of the container. This means that you can start a container where the /host directory is the / directory on your host; and the container can alter your host filesystem without any restriction. This is similar to how virtualization systems allow filesystem resource sharing. Nothing prevents you from sharing your root filesystem (or even your root block device) with a virtual machine.
https://docs.docker.com/engine/security/#docker-daemon-attack-surface
Yeah, I think of myself as fairly security conscious, not an expert but interested layman. And I still didn’t realize it for so many years so I figured many other people probably also didn’t, therefor I posted it.
Also back in around 2018 I was setting up docker to contain bitbake in a ubuntu container to be able to run it on my Arch, because everyone else was running debian or ubuntu at work and nobody wanted to deal with my Arch installation. But I wanted a nice seamless way of doing by mounting the current directory and aliasing bitbake to
docker exec -it ...and doing sudo every time felt kind of wrong so I read in the documentation about adding my user to the docker group and didn’t really think about it since then until now.Super fair, ich habe erst die letzten Jahre angefangen mich mehr mit Containern auseinander zu setzen und da war diese Gefahr in vielen Tutorials recht präsent. ist wahrscheinlich ein shift der da in der Wahrnehung zwischenzeitlich passiert ist.
I think it was well over a year ago I had my first AI agent do this and I’ve been strictly rootless podman ever since (for everything, not just agents).
Any AI stuff I run goes in a sandbox (Nono on Linux, sandbox on macOS). I also used my own harness for a long time, which only had a limited set of file operations, sandboxed JavaScript code mode, and everything else was installed as a plugin.
One of the reasons my pref is unpriveleged lxc.
Yea, mapping the guest user ids to 10xxxx user IDs on the host is a elegant and secure solution imo. Not sure if that’s LXC or Proxmox doing tho…
