# Use NFS for Persistent Volumes on MicroK8s **URL:** \ **Category:** microk8s\ **Tags:** docs\ **Created:** [February 10, 2022, 12:21pm UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035 "2022-02-10T12:21:33Z")\ **Posts on this page:** 14\ **Page:** 2
**Author:** ![telecomspec](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/telecomspec/32/12808_2.png) [@telecomspec](https://discuss.kubernetes.io/u/telecomspec)\ **Post date:** [August 23, 2023, 4:29pm UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/22 "2023-08-23T16:29:21Z")
May be this link will help you in finding some points for your questions. > **[Network File System (NFS) | Ubuntu](https://ubuntu.com/server/docs/service-nfs)** > > Ubuntu is an open source software operating system that runs from the desktop, to the cloud, to all your internet connected things. ---
**Author:** ![shibu\_luoge](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/shibu_luoge/32/13485_2.png) [@shibu\_luoge](https://discuss.kubernetes.io/u/shibu_luoge)\ **Post date:** [October 17, 2023, 12:00pm UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/23 "2023-10-17T12:00:17Z")
what is microk8s command in this part ? ---
**Author:** ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)\ **Post date:** [October 17, 2023, 1:09pm UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/24 "2023-10-17T13:09:57Z")
Hi, sorry I’m not sure about the question. Which part are you having difficulty with? ---
**Author:** ![shibu\_luoge](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/shibu_luoge/32/13485_2.png) [@shibu\_luoge](https://discuss.kubernetes.io/u/shibu_luoge)\ **Post date:** [October 18, 2023, 7:57am UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/25 "2023-10-18T07:57:18Z")
i mean that all this discuss, there many commands begin with microk8s , such microk8s kubectl describe pvc my-pvc or microk8s kubectl apply -f - \< pvc-nfs.yaml. so what is this command, how i install this command in my cluster? ---
**Author:** ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)\ **Post date:** [October 19, 2023, 3:18pm UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/26 "2023-10-19T15:18:17Z")
Hi, this discussion particularly relates to using NFS with MicroK8s. If you [install MicroK8s](https://microk8s.io/docs/getting-started) the `microk8s`command will be available. ---
**Author:** ![joselbr2099](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/joselbr2099/32/13655_2.png) [@joselbr2099](https://discuss.kubernetes.io/u/joselbr2099)\ **Post date:** [November 13, 2023, 12:28am UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/27 "2023-11-13T00:28:13Z")
Error in pvc-nfs from installation [guide](https://microk8s.io/docs/addon-nfs) ```auto [node@master test]$ kubectl describe pvc my-pvc Name: my-pvc Namespace: default StorageClass: nfs-csi Status: Pending Volume: Labels: Annotations: volume.beta.kubernetes.io/storage-provisioner: nfs.csi.k8s.io volume.kubernetes.io/storage-provisioner: nfs.csi.k8s.io Finalizers: [kubernetes.io/pvc-protection] Capacity: Access Modes: VolumeMode: Filesystem Used By: Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Provisioning 52s (x7 over 2m55s) nfs.csi.k8s.io_node5_865e9280-866f-4bb6-a2b8-484cf33adb80 External provisioner is provisioning volume for claim "default/my-pvc" Warning ProvisioningFailed 42s (x7 over 2m45s) nfs.csi.k8s.io_node5_865e9280-866f-4bb6-a2b8-484cf33adb80 failed to provision volume with StorageClass "nfs-csi": rpc error: code = Internal desc = failed to mount nfs server: rpc error: code = Internal desc = mount failed: exit status 32 Mounting command: mount Mounting arguments: -t nfs -o hard,nfsvers=3 10.164.151.1:/srv/nfs/kube /tmp/pvc-5de49579-65e4-456c-b0d9-36fe91f0aebf Output: /usr/sbin/start-statd: 10: cannot create /run/rpc.statd.lock: Read-only file system mount.nfs: rpc.statd is not running but is required for remote locking. mount.nfs: Either use '-o nolock' to keep locks local, or start statd. Normal ExternalProvisioning 12s (x13 over 2m56s) persistentvolume-controller waiting for a volume to be created, either by external provisioner "nfs.csi.k8s.io" or manually created by system administrator ``` but I’ve tried only command and it works: ```auto [node@master ~]$ sudo mount -t nfs -o hard,nfsvers=3 10.164.151.1:/srv/nfs/kube nfs [node@master ~]$ mount | grep nfs sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) 10.164.151.1:/srv/nfs/kube on /home/node/nfs type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.164.151.1,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.164.151.1) [node@master ~]$ ``` Anyone can help me please? I’ve tried this [add-on](https://microk8s.io/docs/addon-nfs) with NFS too and works normal, but y need to use my nfs server: ```auto [node@master test]$ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE my-pvc Pending nfs-csi 11m pvc-nfs Bound pvc-53fc6f1e-13a2-41ae-ba3a-7d527a049e33 100Mi RWX nfs 8s [node@master test]$ [node@master test]$ kubectl describe pvc pvc-nfs Name: pvc-nfs Namespace: default StorageClass: nfs Status: Bound Volume: pvc-53fc6f1e-13a2-41ae-ba3a-7d527a049e33 Labels: vol=pvc-nfs Annotations: pv.kubernetes.io/bind-completed: yes pv.kubernetes.io/bound-by-controller: yes volume.beta.kubernetes.io/storage-provisioner: cluster.local/nfs-server-provisioner volume.kubernetes.io/storage-provisioner: cluster.local/nfs-server-provisioner Finalizers: [kubernetes.io/pvc-protection] Capacity: 100Mi Access Modes: RWX VolumeMode: Filesystem Used By: Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ExternalProvisioning 25s (x2 over 25s) persistentvolume-controller waiting for a volume to be created, either by external provisioner "cluster.local/nfs-server-provisioner" or manually created by system administrator Normal Provisioning 24s cluster.local/nfs-server-provisioner_nfs-server-provisioner-0_56c197ea-5d92-4dd2-9132-6e02ac77c306 External provisioner is provisioning volume for claim "default/pvc-nfs" Normal ProvisioningSucceeded 23s cluster.local/nfs-server-provisioner_nfs-server-provisioner-0_56c197ea-5d92-4dd2-9132-6e02ac77c306 Successfully provisioned volume pvc-53fc6f1e-13a2-41ae-ba3a-7d527a049e33 ``` ---
**Author:** ![telecomspec](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/telecomspec/32/12808_2.png) [@telecomspec](https://discuss.kubernetes.io/u/telecomspec)\ **Post date:** [December 19, 2023, 6:10pm UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/29 "2023-12-19T18:10:14Z")
Do we have any solution for this?? If yes cloud you suggest. Now the mounted storage’s owner is nobody:nobody (UID=65534) with permission 775. And in the pod, I cannot change owner of the storage. I hope to use non-root user for processes in the pod for security. Resultingly, processes in pod don’t have permission to write on NFS storage. For this case, is there any way to avoid this permission problem? May I have your opinion for it? Assume that I can not change NFS server side configuration (owner=nobody and permssion=775) for security. And pod side have to use non-root user for processes as well. ---
**Author:** ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)\ **Post date:** [February 20, 2024, 2:13pm UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/30 "2024-02-20T14:13:38Z")
Hi, sorry for the delayed response, for some reason I didn’t see this mention. The driver is an upstream project. I think the idea of non-anonymous access has been brought up before but I don’t believe it has been implemented. e.g. > > > \*\*Is your feature request related to a problem?/Why is this needed\*\* > Right now …it seems that this driver can only mount NFS volumes if server allow anonymous access. > 2 years ago, on https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/issues/136#issuecomment-907710848 was said that this feature could have been implemented here but I wasn't able to find anything related. > > \*\*Describe the solution you'd like in detail\*\* > A way to set some kind of credentials to perform authentication (eg Kerberos, sys) > > \*\*Additional context\*\* > none ---
**Author:** ![Martin\_Colley](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/martin_colley/32/13048_2.png) [@Martin\_Colley](https://discuss.kubernetes.io/u/Martin_Colley)\ **Post date:** [March 28, 2024, 4:47pm UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/31 "2024-03-28T16:47:27Z")
Hi Thanks for this article. I found it, and the discussion very helpful in my troubleshooting efforts, specifically the posts regarding dynamic provisioning vs static nfs pvc. I am posting this comment in the hope that it may assist others. With static provisioning, I am getting an error regarding the /var/snap/ directory being read only: ```plaintext Warning FailedMount 3m32s (x448 over 14h) kubelet MountVolume.SetUp failed for volume "pvc-1c0b7728-5ebd-4bb0-85ba-0f0877ab9080" : rpc error: code = Internal desc = mkdir /var/snap: read-only file system ``` I see from the driver source [https://github.com/kubernetes-csi/csi-driver-nfs/blob/master/charts/latest/csi-driver-nfs/templates/csi-nfs-node.yaml](https://github.com/kubernetes-csi/csi-driver-nfs/blob/master/charts/latest/csi-driver-nfs/templates/csi-nfs-node.yaml) that the _kubeletDir/pods_, _kubeletDir/plugins/csi-nfsplugin_ and _kubeletDir/plugins\_registry_ objects are referenced, but all only accessible by root. On the other hand, I did not experience this with dynamic provisioning. A couple of other comments I can make based on what I have found during troubleshooting: - I know this may be nfs server specific, but I found (man 5 exports on ubuntu 22.04) that _sync_ and _no\_subtree\_check_ are both defaults, so in my case I can remove redundant options in the /etc/exports entry - My installation of microk8s (–channel=1.28/stable --classic) placed a symbolic link at _/var/lib/kubelet_ to _/var/snap/microk8s/common/var/lib/kubelet_, making (in my case) the overriding of kubeletDir redundant. Neither of these would be expected to cause a problem, but simplification is possible. Thanks again Kind Regards Martin ---
**Author:** ![bobesz](https://avatars.discourse-cdn.com/v4/letter/b/e495f1/32.png) [@bobesz](https://discuss.kubernetes.io/u/bobesz)\ **Post date:** [May 14, 2024, 1:51pm UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/32 "2024-05-14T13:51:40Z")
```auto root@vmi1703919:~# microk8s kubectl logs --selector app=csi-nfs-node -n kube-system -c nfs I0514 13:34:04.546980 1 server.go:117] Listening for connections on address: &net.UnixAddr{Name:"//csi/csi.sock", Net:"unix"} I0514 13:34:05.197659 1 utils.go:109] GRPC call: /csi.v1.Identity/GetPluginInfo I0514 13:34:05.197683 1 utils.go:110] GRPC request: {} I0514 13:34:05.199538 1 utils.go:116] GRPC response: {"name":"nfs.csi.k8s.io","vendor_version":"v4.7.0"} I0514 13:34:05.326910 1 utils.go:109] GRPC call: /csi.v1.Identity/GetPluginInfo I0514 13:34:05.326956 1 utils.go:110] GRPC request: {} I0514 13:34:05.326987 1 utils.go:116] GRPC response: {"name":"nfs.csi.k8s.io","vendor_version":"v4.7.0"} I0514 13:34:06.328361 1 utils.go:109] GRPC call: /csi.v1.Node/NodeGetInfo I0514 13:34:06.328384 1 utils.go:110] GRPC request: {} I0514 13:34:06.328800 1 utils.go:116] GRPC response: {"node_id":"vmi1703919"} I0514 13:34:52.604752 1 server.go:117] Listening for connections on address: &net.UnixAddr{Name:"//csi/csi.sock", Net:"unix"} I0514 13:34:53.385357 1 utils.go:109] GRPC call: /csi.v1.Identity/GetPluginInfo I0514 13:34:53.385437 1 utils.go:110] GRPC request: {} I0514 13:34:53.392769 1 utils.go:116] GRPC response: {"name":"nfs.csi.k8s.io","vendor_version":"v4.7.0"} I0514 13:34:53.443139 1 utils.go:109] GRPC call: /csi.v1.Identity/GetPluginInfo I0514 13:34:53.443295 1 utils.go:110] GRPC request: {} I0514 13:34:53.443447 1 utils.go:116] GRPC response: {"name":"nfs.csi.k8s.io","vendor_version":"v4.7.0"} I0514 13:34:53.622122 1 utils.go:109] GRPC call: /csi.v1.Node/NodeGetInfo I0514 13:34:53.622176 1 utils.go:110] GRPC request: {} I0514 13:34:53.622298 1 utils.go:116] GRPC response: {"node_id":"vmi1703920"} I0514 13:34:28.413032 1 server.go:117] Listening for connections on address: &net.UnixAddr{Name:"//csi/csi.sock", Net:"unix"} I0514 13:34:28.987993 1 utils.go:109] GRPC call: /csi.v1.Identity/GetPluginInfo I0514 13:34:28.988013 1 utils.go:110] GRPC request: {} I0514 13:34:28.992196 1 utils.go:116] GRPC response: {"name":"nfs.csi.k8s.io","vendor_version":"v4.7.0"} I0514 13:34:29.146458 1 utils.go:109] GRPC call: /csi.v1.Identity/GetPluginInfo I0514 13:34:29.146568 1 utils.go:110] GRPC request: {} I0514 13:34:29.146701 1 utils.go:116] GRPC response: {"name":"nfs.csi.k8s.io","vendor_version":"v4.7.0"} I0514 13:34:29.718305 1 utils.go:109] GRPC call: /csi.v1.Node/NodeGetInfo I0514 13:34:29.718513 1 utils.go:110] GRPC request: {} I0514 13:34:29.718658 1 utils.go:116] GRPC response: {"node_id":"vmi1703921"} root@vmi1703919:~# microk8s kubectl logs --selector app=csi-nfs-controller -n kube-system -c nfs E0514 13:43:41.934825 1 utils.go:114] GRPC error: rpc error: code = Internal desc = failed to mount nfs server: rpc error: code = Internal desc = mount failed: exit status 32 Mounting command: mount Mounting arguments: -t nfs -o hard,nfsvers=4.1 10.0.0.4:/srv/nfs /tmp/pvc-2e7970ff-fe9a-4401-81cc-b2a134ebda1f Output: mount.nfs: No route to host I0514 13:43:49.938104 1 utils.go:109] GRPC call: /csi.v1.Controller/CreateVolume I0514 13:43:49.938149 1 utils.go:110] GRPC request: {"capacity_range":{"required_bytes":5368709120},"name":"pvc-2e7970ff-fe9a-4401-81cc-b2a134ebda1f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-2e7970ff-fe9a-4401-81cc-b2a134ebda1f","csi.storage.k8s.io/pvc/name":"my-pvc","csi.storage.k8s.io/pvc/namespace":"default","server":"10.0.0.4","share":"/srv/nfs"},"volume_capabilities":[{"AccessType":{"Mount":{"mount_flags":["hard","nfsvers=4.1"]}},"access_mode":{"mode":7}}]} I0514 13:43:49.938711 1 controllerserver.go:462] internally mounting 10.0.0.4:/srv/nfs at /tmp/pvc-2e7970ff-fe9a-4401-81cc-b2a134ebda1f I0514 13:43:49.938789 1 nodeserver.go:132] NodePublishVolume: volumeID(10.0.0.4#srv/nfs#pvc-2e7970ff-fe9a-4401-81cc-b2a134ebda1f##) source(10.0.0.4:/srv/nfs) targetPath(/tmp/pvc-2e7970ff-fe9a-4401-81cc-b2a134ebda1f) mountflags([hard nfsvers=4.1]) I0514 13:43:49.938856 1 mount_linux.go:218] Mounting cmd (mount) with arguments (-t nfs -o hard,nfsvers=4.1 10.0.0.4:/srv/nfs /tmp/pvc-2e7970ff-fe9a-4401-81cc-b2a134ebda1f) root@vmi1703919:~# ping 10.0.0.4 PING 10.0.0.4 (10.0.0.4) 56(84) bytes of data. 64 bytes from 10.0.0.4: icmp_seq=1 ttl=64 time=2.43 ms 64 bytes from 10.0.0.4: icmp_seq=2 ttl=64 time=0.479 ms 64 bytes from 10.0.0.4: icmp_seq=3 ttl=64 time=0.505 ms 64 bytes from 10.0.0.4: icmp_seq=4 ttl=64 time=0.473 ms ^C --- 10.0.0.4 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3027ms rtt min/avg/max/mdev = 0.473/0.972/2.434/0.843 ms root@vmi1703919:~# ``` ---
**Author:** ![mlb5000](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/mlb5000/32/15139_2.png) [@mlb5000](https://discuss.kubernetes.io/u/mlb5000)\ **Post date:** [July 29, 2024, 5:32pm UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/33 "2024-07-29T17:32:54Z")
I actually have this exact same problem. Is there a solution to this @evilnick ? Currently, I can’t seem to get `nfs-csi` PVs to work when the running service needs to change the filesystem owner at startup (like RabbitMQ’s Cluster Operator, for example) ---
**Author:** ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)\ **Post date:** [August 7, 2024, 11:04am UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/34 "2024-08-07T11:04:53Z")
Sadly that issue for the driver is still open. If anyone has found a useful workaround I’m happy to include it in the docs ---
**Author:** ![mlb5000](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/mlb5000/32/15139_2.png) [@mlb5000](https://discuss.kubernetes.io/u/mlb5000)\ **Post date:** [August 7, 2024, 2:32pm UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/35 "2024-08-07T14:32:22Z")
So I “got around it”, but I don’t 100% remember how except a bunch of initial trial and error. I believe I had to change permissions on the PVC directory **after** it was created by K8S. Setting it on the root directory of the NFS mount path (e.g. `/srv/nfs`) doesn’t look to be enough. ---
**Author:** ![resurrexi](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/resurrexi/32/17968_2.png) [@resurrexi](https://discuss.kubernetes.io/u/resurrexi)\ **Post date:** [January 9, 2026, 12:49am UTC](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035/36 "2026-01-09T00:49:07Z")
I was able to get everything working with the only addition of adding `no_root_squash` for the share folder in `/etc/exports`. [Previous page](https://discuss.kubernetes.io/t/use-nfs-for-persistent-volumes-on-microk8s/19035.md?page=1)