Fails to Unmount Image in MicroK8s on Hyper-V

General Discussions microk8s
1

I am running MicroK8s on an Ubuntu VM hosted on Hyper-V. When trying to pull certain images using microk8s.ctr , I encounter an error related to unmounting. The problem seems to be specific to certain images, as I can pull some images without any issues, but others result in the following error:

ctr: failed to extract layer sha256:74c0af6e02274b54b88f851843ae69880a234694dede8ff9fb93bfa076af45ed: failed to unmount /var/snap/microk8s/common/var/lib/containerd/tmpmounts/containerd-mount2689904121: failed to unmount target /var/snap/microk8s/common/var/lib/containerd/tmpmounts/containerd-mount2689904121: device or resource busy: unknown

Environment:

  • Host OS: Windows (Hyper-V)
  • Guest OS: Ubuntu 20.04
  • MicroK8s version: microK8s v1.28.2 revision 6085
  • containerd version: v1.6.15

Additional Information:

  • I’ve verified that there’s sufficient storage space on both the host and the VM.
  • There are no active security scans or antivirus software on the Ubuntu VM that could be causing the issue.
  • Other images can be pulled without any problems.

Any assistance or insights into this issue would be greatly appreciated.

images

image
image883×440 26.5 KB

4 Likes
2

Did you solve this?
We are experiencing the same problem.

3

Same here. Apparently, it was Microsoft Defender Endpoint scanning the temporary mount and preventing containerd from unmounting it, resulting in the error. Since both ctr and microk8s use containerd, the error is the same. Directory and mount were created just seconds ago, so there is no reason any other software should access it and interfere (unless AV is monitoring fresh mounts for scanning purposes).

There is more discussion here: containerd issue 5538 on github

Command to add exclusion to MS Defender:
mdatp exclusion folder add --path “/var/snap/microk8s/common/var/lib/containerd/tmpmounts/*”