Strategy contianer in container with contianerd

[kameshsampath]

What is the best and recommended strategy to do container in container ( docker in docker) kind of thing with containerd. Typically I wish to start a container from within another container.

My Environment:

I am using Rancher Desktop on macOS with containerd runtime.

[kameshsampath]

I was able to run a container within the nerdctl image after building it, but then was wondering how I can share the storage and namespaces form the underlying containerd so that when running container in container I don't need to download the images every time.

[AkihiroSuda]

Sharing the storage is not supported.

Please create a private registry or just use nerdctl save/nerdctl load tarballs to avoid redundant downloading.

[kameshsampath]

@AkihiroSuda sorry about the confusion there, when I should meant the socket not the storage :( its my mistake

I was able to connect or start a container from within another container started using nerdctl, I have detailed the steps in the gist https://gist.github.com/kameshsampath/c61493e9a7946ab99a745fb075a9ae62.

[kameshsampath]

Create a nerdctl container image

Dockerfile(Dockerfile.nerdctl)

FROM alpine

ARG NERDCTL_VERSION=0.20.0
RUN echo http://dl-cdn.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories

RUN apk upgrade \
    && apk update \
    && apk add --update wget curl \
	&& cd /usr/local/bin \
	&& wget -c https://github.com/containerd/nerdctl/releases/download/v${NERDCTL_VERSION}/nerdctl-${NERDCTL_VERSION}-linux-arm64.tar.gz -O - | tar -xz \
	&& cd -

Note: Update the registry quay.io/kameshsampath/nerdctl if required

Build the image using the command

nerdctl build --platform=linux/amd64,linux/arm64 -t quay.io/kameshsampath/nerdctl -f Dockerfile.nerdctl .

Push the image to the registry

nerdctl push --all-platforms quay.io/kameshsampath/nerdctl

With Rancher Desktop Running on your local machine you can run the following command to connect to RD containerd socket from within the nerdctl container,

nerdctl run -it \
  -v /run/k3s/containerd/containerd.sock:/run/containerd/containerd.sock \
  quay.io/kameshsampath/nerdctl nerdctl images

If all went well the above command should list all the images from default namespace.

[fahedouch]

@kameshsampath mounting socket is not enough because some nerdctl logic are in containerd side, I think @cpuguy83 is working on this topic. By the way this issue may help you #272

[fahedouch]

You tired this #272 (comment) ?

[kameshsampath]

no @fahedouch I did not tried that. That's something I plan to do today.

[kameshsampath]

I just tried running the following,

this fails as there is not /var/lib/containerd on the host (RD vm)

  -v /run/k3s/containerd:/run/containerd \
  -v /var/lib/containerd:/var/lib/containerd \
  -v /var/lib/nerdctl:/var/lib/nerdctl \
  -v /tmp:/tmp \
   quay.io/kameshsampath/nerdctl sh

This starts the contianer, but form within the container I try doing nerdctl run -it busybox echo "Hello World" it fails with no such file or directory

nerdctl run -it --privileged \
  -v /run/k3s/containerd:/run/containerd \
  -v /var/lib/nerdctl:/var/lib/nerdctl \
  -v /var/lib/cni:/var/lib/cni \
  -v /tmp:/tmp \
   quay.io/kameshsampath/nerdctl sh

[fahedouch]

you should have a running containerd on the host
the default root is /var/lib/containerd, but may be it is replaced plz check you config.toml (default is /etc/containerd/config.toml)

#root = "/var/lib/containerd"
#state = "/run/containerd"
#subreaper = true
#oom_score = 0

[dongzhiwei-git]

Did you succeed? Can you provide me with a demo

[dongzhiwei-git]

controlplane $ cat >> config. toml << EOF
> root = "/var/lib/containerd"
> state = "/run/containerd"
> subreaper = true
> oom score = 0
> EOF controlplane $ ls config. toml
io.containerd. runtime. v1. linux
io. containerd. content. v1. content io.containerd. runtime. v2. task io.containerd.grpc.v1.cri io.containerd.metadata.v1. bolt
io. containerd.snapshotter.vi.aufs
io.containerd. snapshotter. v1. native
10.containerd.snapshotter.v1.overlayfs tmpmounts
lo.containerd.snapshotter.v1.btrfs
controlplane $ docker run -it -v /var/lib/containerd:/var/lib/containerd
--privileged -v / run/containerd/: / run/contai
nerd/ -v /opt/cni:/opt/cni docker. io/hubdongzhiwei/nertctl:v1 root@615dfaaf6ff:/opt/apps# nerdctl
-namespace=k8s. io
--insecure-registry=true run --runtime=io.containerd. runtime.v
1. linux docker. 10/hubdongzhiwei/nertctl:v1
FATA 0000] OCI runtime create failed: run create failed: unable to start container process: error during container init error mounting "/var/lib/nerdctl/1935db59/containers/k8s. io/5b24102c1467ca55935d4cd4ed521f5c447bc30e633b7deef59792d405 e0dab8/resolv.conf" to rootfs at "/etc/resolv.conf": stat /var/lib/nerdctl/1935db59/containers/k8s. io/5b24102c1467 ca5593
5d4cd4ed521f5c447bc30633b7deef59792d405e0dab8/resolv.conf: no such file or directory: unknown
rootab615dfaaf6ff:/opt/apps#

[MondayCha]

Hello, Excuse me. Is there any follow-up regarding this case?

I hope to connect to the containerd of the host from the privileged container (daemonset), so as to support the commit operation on the running containers.

[AkihiroSuda]

I hope to connect to the containerd of the host from the privileged container (daemonset),

This is unsupported.

Workarounds:

• Jailbreak to the host (https://github.com/justincormack/nsenter1)
• Or, SSH into the host

[fahedouch]

@AkihiroSuda how Jailbreak will fix the issue ? we still need to get containerd fs from host

[fahedouch]

ah Sorry! got it => by access the host mount ns

[MondayCha]

ah Sorry! got it => by access the host mount ns

Thank you very much! I later successfully connected from a privileged container (DaemonSet) to the host's containerd, enabling us to dynamically save images of running pods using nerdctl.