This seems like a really weird decision. If base images are duplicated for every image you have, that will add up quickly.
However, the tedium of the reply chain reminds me why I tend to focus most energy on internal projects rather than external open source...
Docker may have been built for a specific type of use case that most developers are familiar with (e.g. web apps backed by a DB container) but containerization is useful across so much of computing that are very different. Something that seems trivial in the python/DB space, having one or two different small duplicates of OS layers, is very different once you have 30 containers for different models+code, and then ~100 more dev containers lying around as build artifacts from building and pushing, and pulling, each at ~10GB, that the inefficient new system is just painful.
The smallest PyTorch container I ever built was 1.8GB, and that was just for some CPU-only inference endpoints, and that took several hours of yak shaving to achieve, and after a month or two of development it had ballooned back to 8GB. Containers with CUDA, or using significant other AI/ML libraries, get really big. YAGNI is a great principle for your own code when writing from scratch, but YAGNI is a bit dangerous when there's been an entire ecosystem built on your product and things are getting rewritten from scratch, because the "you" is far larger than the developer making the change. Docker's core feature has always been reusable and composable layers, so seeing it abandoned seems that somebody took YAGNI far too extreme on their own corner of the computing world.
I'm also in the process of building a BuildKit builder, I'm seeing large improvements on the speed of exporting images. The same image that takes Docker >3 minutes to export and push takes me under a minute. https://github.com/clipper-registry/benchmarks/actions/runs/...
So much that containerization in general predates Linux, and UNIX, all the way back to System 360.
Also it got introduced into Tru64, HP-UX, BSD and Solaris, before landing into Linux.
Inference, development cycles, any of the application domains of PyTorch that don't involve training frontier models... all of those are complicated by excessive container layers.
But mostly dev really sucks with writing out an extra 10GB for a small code change.
For some problems you might even be able to get away with single digit numbers of training points (classic example of this regime being Physics-Informed Neural Networks)
Due to that, a careless installation of a few new dev-systems under the new docker version immediately blew up storage usage on the root-disk, while happily ignoring hundreds of gigabytes on a volume on /var/lib/docker.. because that's where it needs the storage, right? A few older systems also were upgraded but didn't, which was quite confusing at first.
Sorry for being salty, but that was a pretty hectic afternoon with those new agents trashing builds, and now we have a pretty annoying migration plan to plan for the rest. And yes yes it's just a reinstallation, but we have other things to do as well.
> The containerd image store uses more disk space than the legacy storage drivers for the same images. This is because containerd stores images in both compressed and uncompressed formats, while the legacy drivers stored only the uncompressed layers.
Why ?
This means `/var/lib/docker` is no longer "hermetic": images and container snapshots are located in `/var/lib/containerd` now.
More info about the switch: https://www.docker.com/blog/docker-engine-version-29/
To configure this directory, see https://docs.docker.com/engine/storage/containerd/.
To keep both /var/lib/{containerd,docker} in sync, I use a single ZFS dataset ("custom filesystem volume" in Incus parlance) and mount subpaths inside the container:
incus storage volume create local docker-data
incus config device add docker docker disk pool=local source=docker-data/docker path=/var/lib/docker
incus config device add docker containerd disk pool=local source=docker-data/containerd path=/var/lib/containerd
How bad did we fall with the ship often, ship early and fix later idea? Make a major change, release it and the migration feature is experimental and not recommended.
My reasoning is simply that I don't really want to swap out one overly complicated thing for another. I'm sure Podman is fine and amazing. But I'm just not in the IBM/Red Hat ecosystem and I have some reservations their generally a bit overly complex solutions. There's a reason IBM is involved, just saying. And as I'm not planning to use podman in production I see no reason to have it on my laptop.
As for Rancher, that seems to me a bit like moving the problem than solving it as it seems to be a for profit solution around an OSS core with its own complexity and potentially similar risks to Docker Desktop down the road.
With colima, it's all open source and easy to install/upgrade via homebrew. Nice simple wrapper around qemu. There's no UI, and I don't really miss having one. Lazydocker works fine as a TUI if you crave a UI and so do other generic docker UIs/IDEs. I mainly use docker and docker compose on the command line and that works fine for me. It has Kubernetes support as well if you need that but that's not something I use or need.
Meanwhile, the basic stuff like caching doesn't work properly.
I can't believe Docker finally shit the bed. Time to replace Docker with Podman.... sigh
docker system prune -a -f
docker volume prune -a -f"Regularly" = when you're running out of space because of a bunch of built up old stuff.
Ref: https://docs.docker.com/reference/cli/docker/system/prune/
The 'system' context captures networks; much to my dismay, this has been a problem for no fewer than three employers. It's painfully common for things to expect the networks to persist. They don't really consume resources, so I see no reason to invite the systematic heartburn.
When? When there's disk pressure. Maybe some longer term (weekly, monthly?) to keep a lid on things. The image cache provides a benefit, no sense fighting it. At our rate, daily pruning means I might lose hours (through a week) repeatedly pulling the same images.