Docker -> Podman

Software & Operating Systems Open Source & Web-Based
1

Are there people who are familiar with Podman that can help me transition from Docker compose?

So I run lots of self-hosted services on my own hardware that is open to the public internet. Most of them use Docker containers. This includes Vaultwarden (Bitwarden), Jellyfin, Joplin server (if I can ever get it working again), and some kind of Document sync solution like Nextcloud. Anyway, after the latest Docker debacle, I began looking into Podman. Here I noticed some really cool features about Podman that I like, or at least think I like. The one I like the most is rootless containerization. I have tried, but to no avail, to run several of my docker containers rootless.

So here’s my dilemma. I use docker compose for all my docker containers. I find it to be more elegant than Docker CLI, and it gives me a record of what I did so that I can easily replicate it. I have discovered, that while Podman is a drop-in replacement for Docker CLI, this isn’t the case for Docker compose. I have found several references to tools to bridge this gap including podman-compose, direct support for docker compose within podman itself, or something like Kubernetes (I think). I tried the docker compose support first but to no avail because I discovered that it doesn’t support rootless mode; then I found out about podman-compose after a little more digging. I almost succeeded with that, but still failed. I am not having much luck here, so I was hoping that someone might have insight for helping me transition to Podman. I would like to give podman-compose another go - this time in a staging environment so that I can avoid more downtime with my docker containers and also so I don’t mess up my “server”.

But I am also open to using Podman with kubernetes. The problem is that every time I try to figure out what Kubernetes is, and how to get started; I get inundated with jargon and my eyes glaze over so I move on.

Also the stuff that I am finding about Podman suffers from the Ubuntu wiki problem and is like 2 to 3 years old and is therefore outdated.

1 Like
2

I don’t really have any suggestions on the Podman front, but with regard to moving away from Docker, the issue was with Docker Hub, not Docker itself.

So if you insulate yourself from Docker Hub, then that takes away much of the urgency of switching.

There are a number of services that can be self-hosted and will cache docker images from upstream repositories. That way, if an upstream source goes away, you still have access to the images you are using. I am using Sonatype Nexus for this, since I already had it set up previously.

Although long term, I do expect that I would switch to Podman as well, but I’m waiting until it is a bit closer of a drop in replacement for my docker compose files.

1 Like
3

You can run docker rootless (most of it), you can run podman rootless.

rootless just means that you can run stuff in containers that thinks it’s uid=0, but in reality it’s actually yourself… A theoretical benefit is that uid based security in the kernel is easier for people to understand and therefore it’s less likely to become exploited by some botnet if/when exposed to the internet, or if/when one of your machines is taken over and infected.

Practical benefit of rootless containers is that you can more conveniently use them for your own development projects to have self-contained dev environments, and also if you’re building your own images or want to play with things that traditionally require some root privilege or capability — you no longer need sudo to mess with stuff.

I run a mix of docker and podman now at home, I use docker and docker-compose for my home network services (dockerd running as root, with a bunch of containers that don’t run s6 overlay and don’t require root to run - they just run as a specific uid).

And for playing around with containers I use podman, and I have various build.sh and test.sh and run.sh short scripts that more or less do what docker-compose kind-of would’ve done.

I’ve briefly considered moving some of my home services to systemd calling podman run , but I kind of gave up for now. … I’ll look into this in the future.

4

Does anyone have any thoughts on using Kubernetes for a home services server?

5

If you’re thinking about docker swarm, then probably better to go down kubernetes route instead. Otherwise, I’d say stick to docker/podman.

1 Like
6

Care to expand?

8

I don’t even understand the concept of Kubernetes. I started reading the documentation for it, but since this is a personal project I haven’t gotten far. I only mentioned it because some of the articles and stuff I read about Docker compose compatibility in Podman mentioned it. So if Docker Swarm is its counterpart, then I guess my question is answered in part.

I guess I sense a problem with Docker in general. I just feel like it may be time to go ahead and jump ship before they try something else. The way they treat the open source community is hypocritical to say the least, and their current attempts for generating revenue worry that they are gonna make it hard for even for “home” users to use their software without gouging them.

10

Red hat/IBM track record ain’t stellar either…

Anyway, definitely avoid a big bang migration, setup a separate VM to run all your tests with more time, and attack one problem at a time,
Then come back with specific things you are unable to achieve so that we can help you?

As for Kubernetes, unless you need it for your job I would let it go, and even if I needed to learn how the ecosystem works for my job, i probably would not bother using it at home …
It is like installing an oracle cluster database to replace a SQLite one…

2 Likes
11

Both docker and kubernetes run container images and use kernel namespace containerization.

In docker on one machine, you can optionally constrain your container in terms of resources to give it some ram/CPU shape, and you can optionally define additional networking options - typically to expose a port, and for storage you can mount directories and volumes typically, and some other stuff.

Also, ok one machine docker, you can go a step further, and define health-checks, so that docker automatically restarts your container when a periodic healthcheck fails. You can also define your own network drivers, to move physical interfaces into a container, or create and configure custom interfaces e.g OpenVPN or wireguard or vxlan. Same with storage… there’s a pluggable API you can use, but 99.99% of people don’t bother with this pluggable API.

Docker swarm adds some plugins, and it adds a facade you can talk to, that’ll schedule your container on one of the machines you have.

… enter kubernetes …

Turns out building blocks are the same (conceptually at least, implementation is not same) as a one machine docker, but plugin architecture in k8s relative to docker is roided up to 9000 then turned up to 11.

Similarly to how podman is great for software development, so are minikube and k3s; if you’re developing stuff that’ll eventually run on Kubernetes, you can play with those two to get to a working proof of concept for whatever distributed service you’re building.

K8s comes from a place that runs hundreds of 20k machine Borg cells (not VM machines, physical machines, each). You can run minikube or k3s in your homelab, but it’s not something that’s very useful for a single person at home… (kind of like Kerberos and AD).

Unless it’s for learning…

… for learning, I’d recommend you played with ceph first. … try spinning up a network of a few dozen VMs, and see if you can get clustered storage going.

… lots of folks use NFS to provide storage to k3s or K8s for playing around, and that works for playing around.

… in general clustered/HA storage is essential for any kind of clustered compute you want to build on top of it. (real world deployments would do petabytes of flash as cache in front of exabytes of spinning rust) but you can start with a few dozen VMs for learning, or even use ceph instead of ZFS if you feel like it without VMs.

You probably don’t want either docker swarm or k3s/k8s to actually run stuff in your homelab… probably. (There’s simpler ways to get redundant pfSense and DNS and so on).

12

Kubernetes, or k8s, is a clustering software for what is commonly called OCI containers (Open Cloud Initiative). OCI is an organization formed from Docker Inc, Google, Red Hat I believe and others. The general idea behind OCI is that they make rules for how container technology should be created.

The original one was Docker. Podman is a Red Hat counterpart. K8s was initially using docker IIRC, but Google needed something better, so they made their own stuff. Docker itself was originally running on LXC (I’ll get to that later), but then made its own thing.

K8s, again, is a clustering software, like Docker Swarm which someone mentioned. Certain K8s stacks can run in single mode. Examples of K8s stacks include: Google Kubernetes, k3s, k0s and Canonical’s microk8s.

Google k8s is hard to get into and there’s even certifications for it. Completely overkill for home and requires quite the setup. It is the OpenStack of OCI world. People at home don’t run OpenStack, they just use Proxmox, XCP-ng, TrueNAS, or virt-manager among others.

On the k8s stacks, a lot of people praise SUSE / RancherLabs’ Rancher. Supposedly it is a really easy way to run k3s. All of the 3 mentioned, k3s, k0s and microk8s are lightweight OCI containers clustering software.

Think of Docker and Podman like of Oracle VirtualBox (:face_vomiting:) and Virt-Manager. These are meant to run OCI containers in only one machine. The k8s stacks are like OpenStack or OpenNebula, you are meant to run stuff on multiple servers (although OpenNebula and certain k8s stacks can run in single server mode).

I know microk8s is the easier to deploy, for better or worse. But I don’t think it has a rootless mode, which is what you are looking for.

I know k3s has an experimental rootless option, but it’s not yet ready for prime time. In the site above, none of the k8s stacks appear, because their management software (probably) is running as root. In a k8s stack, you generally want to be able to more easily control stuff like networking, known ports binding and more, which is why root is required and why big companies don’t care about rootless stuff. Adding these features to a rootless software adds complexity, which eventually you need to leverage root for anyway.

As you may or may not know, there are 2 general types of containers nowadays: application containers (generally all defined as OCI containers) and OS containers (of which LXC is part of). LXC got mentioned in the link as being able to run rootless, but funnily enough, not LXD, which leverages LXC (I have to give that a read). OS containers are kinda what they say they are: entire operating systems in a single environment, meant to run one or more programs in tandem - they are a replacement for VMs (for the most part).

Other software praised are Portainer. Portainer is the easiest to deploy with microk8s, but it’s supposedly not that hard to use with k3s either.

For properly running rootless (OCI) containers, Podman is the only one that is ready for use, because it does not require more OS level stuff (like access over the network) which require root privileges. So you are looking in the right direction.

I only run LXD, my knowledge on OCI stuff is lacking. Podman-compose is probably never going to be 100% compatible with docker-compose. It is a decent alternative, but if you are interested in running podman, you should learn the way podman handles stuff: podman, buildah, skopeo and friends. Just my $0.02.

The original idea of this post was to explain k8s, but I deviated a bit lot (not unexpected of me).

IMO, switching cold-turkey and fixing everything is the way to really do a migration. Of course, don’t do it on production or “homeprod” directly, but do it in a VM and see how it goes, what needs to change, what things you can adapt by changing to scripts instead of compose and so on. Then, with the knowledge you gathered now documented, start moving (home)prod. At least that is the way I found the motivation to change stuff (when everything works, you got no real reason to change, but if things are inaccessible and you want them, you’ll figure it out in probably 2 to 3 days of after-dayjob work).

13

If you are comfortable with docker-compose, but want to wet your feet with podman. I suggest using docker-compose with podman.

Avoid podman-compose, it just doesn’t work well.

Rather, you can use the docker-compose binary and point it to the podman socket:

# as a non-root user
systemctl --user start podman.socket
# or to make it persisstent
systemctl --user enable --now podman.socket

export DOCKER_HOST=unix:///run/user/$(id -u)/podman/podman.sock

# then use docker-compose as normal

I use this for many services and it works great.

One of the biggest hurdles you’ll faces is that many container images out of the box don’t work well with podman rootless. In particular all the linuxserver.io images explicitly do not work with rootless.

1 Like