Uncategorized on scratch

Run containers without pulling images

Thu, 24 Oct 2019 18:37:23 +0000

CRFS is a Google project that aims at running a container without pre-pulling the image first. The key insight is that in practice a container process only accesses a small fraction of the files in its image, so fetching the entire image before startup wastes both time and disk space. CRFS achieves this through the stargz (Seekable tar.gz) format, which restructures each compressed layer so that individual files can be fetched on demand rather than requiring the entire tarball to be downloaded and extracted upfront.

Crun moved to github.com/containers

Mon, 12 Aug 2019 09:54:25 +0000

The giuseppe/crun github project was moved under https://github.com/containers/crun. Moving to the containers organization means the project is no longer a personal experiment but a community-maintained component of the container stack, alongside tools like Podman, Buildah, and fuse-overlayfs. This makes it easier to coordinate changes across the ecosystem and signals that crun is a supported alternative OCI runtime for production use.

Similarly libocispec, used internally by crun for parsing the OCI configuration file was moved to https://github.com/containers/libocispec

Rootless resources management with Podman on Fedora 30

Sun, 12 May 2019 20:36:59 +0000

I have finally opened some PRs for conmon and libpod that enable resources management for Podman rootless containers on Fedora 30 when using crun. This builds on the cgroups v2 delegation support added to crun earlier: Fedora 30 ships a kernel and systemd new enough to support the unified cgroup hierarchy, so with a single kernel command-line option and a small systemd drop-in, unprivileged users can now set memory and CPU limits on their containers without root access.

Resources management with rootless containers and cgroups v2

Tue, 26 Feb 2019 21:22:10 +0000

cgroups v2 will finally allow unprivileged users to manage a cgroup hierarchy in a safe manner without requiring any additional permission. In the cgroups v1 model, writing to cgroup control files requires root, which means rootless containers cannot enforce memory limits or CPU quotas. The unified cgroups v2 hierarchy introduces a delegation mechanism where systemd can hand ownership of a subtree to a user process, enabling the OCI runtime to configure resource limits directly without any privileged helper.

Rootless containers @ devconf.cz

Sun, 24 Feb 2019 22:26:15 +0000

The video of the rootless containers talk from Devconf.cz 2019 is finally available on YouTube. The talk covers how user namespaces, fuse-overlayfs, and slirp4netns come together to allow running containers entirely as an unprivileged user, without any setuid helpers beyond newuidmap and newgidmap, and discusses the remaining challenges around cgroup resource management and overlay storage performance that still need to be addressed for rootless containers to reach full feature parity.

SUID binaries from a user namespace

Thu, 10 Jan 2019 21:49:30 +0000

Additional IDs that are allocated to a user through /etc/subuid and /etc/subgid must be considered as permanently allocated and never reused for any other user. The reason is that a setuid binary created inside a user namespace can retain access to any UID that was mapped in that namespace, even after the namespace is destroyed. If the same UID range is later assigned to a different user, that new user would inherit access to files owned by the old user’s containers.

Disposable rootless sessions

Wed, 09 Jan 2019 22:01:08 +0000

Would be nice to have a way to “fork” the current session and be able to revert all the changes done, without any leftover on the file system. With fuse-overlayfs, a user-space overlay filesystem that unprivileged users can mount, this turns out to be surprisingly straightforward: mount the entire root filesystem as the lower layer of an overlay, point the upper layer at a temporary directory, and every write is captured there and can be discarded at the end of the session, leaving the underlying system untouched.

An Emacs mode for Rust

Tue, 18 Dec 2018 20:57:11 +0000

I was looking for an Emacs mode that could help me to hack on Rust. The built-in rust-mode provides syntax highlighting and basic indentation, but for a language with a complex type system and borrow checker it is useful to have editor integration that can navigate to definitions, show type information, and offer completions. This post covers setting up racer-mode, which drives the racer code-completion engine to provide those features inside Emacs.

Rust-mode itself has not enough features to help me with a language I am not really proficient with yet.

Rootless Podman from upstream on CentOS 7

Fri, 12 Oct 2018 09:14:21 +0000

This is the recipe I use to build podman from upstream on Centos 7 and use rootless containers. We need an updated version of the shadow utils as newuidmap and newgidmap are not present on Centos 7. The shadow utils are installed using “make install” which is not the clean way to install packages and it also overwrites the existing binaries, but it is fine on a development system. Podman is already present on Centos 7 and in facts we install it so we don’t have to worry about conmon and other dependencies.

Network namespaces for unprivileged users

Sun, 05 Aug 2018 13:54:44 +0000

A couple of weekends ago I’ve played with libslirp and put together slirp-forwarder. The challenge with network namespaces for unprivileged users is that creating TAP or TUN devices requires privileges in the host network namespace. SliRP sidesteps this by emulating a full TCP/IP stack entirely in user space, so the helper process can forward traffic to the outside world using only normal socket operations, without needing any elevated capability.

SliRP emulates in userspace a TCP/IP stack. It can be used to circumvent the limitation of creating TAP/TUN devices in the host namespace for an unprivileged user. The program could run in the host namespace, receive messages from the network namespace where a TAP device is configured, and forward them to the outside world using unprivileged operations such as opening another connection to the destination host. Privileged operations are still not possible outside of the emulated network, as the helper program doesn’t gain any additional privilege that running as an unprivileged user.

Become-root in a user namespace

Thu, 19 Jul 2018 08:28:06 +0000

I’ve cleaned up some C files I was using locally for hacking with user namespaces and uploaded them to a new repository on github: https://github.com/giuseppe/become-root. The tool creates a new user namespace and maps the caller to UID 0 inside it, while also mapping additional UIDs and GIDs from the ranges allocated in /etc/subuid and /etc/subgid. This is the foundation needed for rootless containers, which require a full UID/GID mapping — not just the single-UID mapping that unshare -r provides — to correctly represent file ownership inside container images.

Fuse-overlayfs moved to github.com/containers

Fri, 13 Jul 2018 22:00:59 +0000

The fuse-overlayfs project I was working on in the last weeks was moved under the github.com/containers umbrella. fuse-overlayfs is a user-space implementation of the overlay filesystem that can be mounted without root privileges, which is essential for rootless containers. With Linux 4.18 introducing the ability to mount FUSE filesystems inside user namespaces, this makes overlay-based storage finally usable by unprivileged container runtimes such as Podman.

With Linux 4.18 it will be possible to mount a FUSE file system in an user namespace. fuse-overlayfs is an implementation in user space of the overlay file system already present in the Linux kernel, but that can be mounted only by the root user. Union file systems were around for a long time, allowing multiple layers to be stacked on top of each other where usually the last one is the only writeable.
Overlay is an union file system widely used for mounting OCI image. Each OCI image is made up of different layers, each layer can be used by different images. A list of layers, stacked on each other gives the final image that is used by a container. The last level, that is writeable, is specific for the container. This model enables different containers to use the same image that is accessible as read-only from the lower layers of the overlay file system.

Current status (and problems) of running Buildah as non root

Sun, 25 Feb 2018 13:59:14 +0000

Having Buildah running in a user namespace opens the possibility of building container images as a non-root user. I’ve done some work to get Buildah running inside a user container, where it can still create and modify container images without any elevated privileges on the host. This is useful for CI environments and shared systems where granting root or setuid access is not acceptable.

There are still some open issues to get it fully working. The biggest open one is that overlayfs cannot be currently used as non root user. There is some work going on, but this will require changes in the kernel and the way extended attributes work for overlay. The alternative is far from ideal and it is to use the vfs storage driver, but it is a good starting point to get things moving and see how far we get. (Another possibility that doesn’t require changes in the kernel would be an OSTree storage for Buildah, but that is a different story).

New COPR repository for crun

Wed, 15 Nov 2017 19:25:46 +0000

I made a new COPR repository for crun so that it can be easily tested on Fedora without having to build from source. crun is a lightweight OCI container runtime written in C, intended as a faster and lower-overhead alternative to runC. The COPR repository tracks the upstream development branch, making it straightforward to try out new features and report issues before they land in a distribution package.

https://copr.fedorainfracloud.org/coprs/gscrivano/crun/

To install crun on Fedora, it is enough to:

System containers presentation

Mon, 30 Jan 2017 18:41:37 +0000

Here are the slides for the Atomic System Containers talk I gave at Devconf.cz 2017. System containers are a way to run infrastructure services — such as etcd and Flannel — outside of Docker, managed directly by runc and systemd, which removes the circular dependency that arises when a container runtime depends on components that must themselves be running inside containers.

http://scrivano.org/static/system-containers-demo/

If you are interested in the video, it is on YouTube:

Facebook detox?

Tue, 27 Dec 2016 20:52:47 +0000

I have been using Facebook for the last years to fill every dead time:waiting for the bus, ads on TV, compiling, etc. The quality of the information coming from Facebook is inferior to any other social network, at least to my experience (it can be I follow/know the wrong people), though the part of the brain that controls procrastination seems addicted to this lower quality information and the chattering there. Also, I don’t want to simply delete my Facebook account and move on, most of the people I know are present only there, neither I want to be more “asocial”.

Refactoring a function name across several patches with git rebase

Fri, 22 Apr 2016 17:32:23 +0000

git rebase is one of my favorite git commands. It allows to update a set of local patches against another git branch and also to rework, through the -i flag, some previous patches. A lesser-known capability is the –exec flag, which runs an arbitrary shell command after each patch is applied during the rebase. Combined with the -X theirs merge strategy to silently resolve conflicts, this makes it straightforward to apply mechanical transformations — such as a symbol rename — across an entire patch series without manual intervention.

System containers for Atomic

Thu, 24 Mar 2016 15:41:50 +0000

The main reason behind system containers was the inability to run Flannel in a Docker container as Flannel is required by Docker itself. CoreOS solved this chicken and egg problem by using another instance of Docker (called early-docker) that is used to setup only Etcd and Flannel. Atomic system containers take a different approach: instead of a second Docker daemon, they are managed directly by runc and systemd, so the dependency on Docker is removed entirely and the chicken-and-egg problem simply does not arise.

Uncategorized on *scratch*