# Good storage provisioner for on-premises (non-cloud) cluster?

**URL:** <https://discuss.kubernetes.io/t/good-storage-provisioner-for-on-premises-non-cloud-cluster/4258>\
**Category:** General Discussions\
**Created:** [January 16, 2019, 10:12pm UTC](https://discuss.kubernetes.io/t/good-storage-provisioner-for-on-premises-non-cloud-cluster/4258 "2019-01-16T22:12:22Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![dvree](https://avatars.discourse-cdn.com/v4/letter/d/dbc845/32.png) [@dvree](https://discuss.kubernetes.io/u/dvree)\
**Post date:** [January 16, 2019, 10:12pm UTC](https://discuss.kubernetes.io/t/good-storage-provisioner-for-on-premises-non-cloud-cluster/4258/1 "2019-01-16T22:12:22Z")

</div>

We have an simple on-premises kubernetes cluster here and are looking for the best way to provision persistent volumes. The cluster runs on KVM – not OpenStack. Currently we use the nfs provisioner here [https://github.com/kubernetes-incubator/external-storage](https://github.com/kubernetes-incubator/external-storage) which has been working well for us, but for databases, the performance over NFS is like 6X slower than a local disk.

Is there a better solution? What do non-cloud kubernetes people use?

I found this “local” provisioner:

> **[kubernetes-sigs/sig-storage-local-static-provisioner](https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner)**
>
> Static provisioner of local volumes. Contribute to kubernetes-sigs/sig-storage-local-static-provisioner development by creating an account on GitHub.

but it is static only – no dynamic provisioning. Less than ideal, but workable.

Any suggestions are appreciated.

---

<div class="post-metadata">

**Author:** ![ntolia](https://avatars.discourse-cdn.com/v4/letter/n/e36b37/32.png) [@ntolia](https://discuss.kubernetes.io/u/ntolia)\
**Post date:** [January 23, 2019, 1:21am UTC](https://discuss.kubernetes.io/t/good-storage-provisioner-for-on-premises-non-cloud-cluster/4258/2 "2019-01-23T01:21:46Z")

</div>

Unfortunately, the answer is always going to be “It depends.” That said, how large do you expect your clusters to be, how many applications do expect to run, and what is your expected storage footprint in terms of size? There are open-source block storage solutions that could run within your cluster (Nexenta, OpenEBS, Ceph/Rook, etc.) but there are management overheads there.

We know folks using local storage for their Kubernetes clusters but then you do want to use tools like Kanister ([https://github.com/kanisterio/kanister](https://github.com/kanisterio/kanister)) to extract data for backup/recovery/migration/etc.

Hope this helps but happy to answer additional questions if needed.

---

<div class="post-metadata">

**Author:** ![llarsson](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/llarsson/32/9193_2.png) [@llarsson](https://discuss.kubernetes.io/u/llarsson)\
**Post date:** [January 25, 2019, 12:11pm UTC](https://discuss.kubernetes.io/t/good-storage-provisioner-for-on-premises-non-cloud-cluster/4258/3 "2019-01-25T12:11:06Z")

</div>

Since you are on KVM and not on OpenStack or vSphere or something more sophisticated like that, I definitely recommend you take a look at Rook, which seems specifically developed for this use case.

[https://rook.io/](https://rook.io/)

Please report back with your findings and experience, as this is something that quite a few people in the community would be interested in.

---

<div class="post-metadata">

**Author:** ![dvree](https://avatars.discourse-cdn.com/v4/letter/d/dbc845/32.png) [@dvree](https://discuss.kubernetes.io/u/dvree)\
**Post date:** [January 25, 2019, 7:19pm UTC](https://discuss.kubernetes.io/t/good-storage-provisioner-for-on-premises-non-cloud-cluster/4258/4 "2019-01-25T19:19:54Z")

</div>

We will be taking a hard look at rook – thank you both for your feedback.

---

<div class="post-metadata">

**Author:** ![mohideen](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/mohideen/32/3111_2.png) [@mohideen](https://discuss.kubernetes.io/u/mohideen)\
**Post date:** [January 29, 2020, 10:09pm UTC](https://discuss.kubernetes.io/t/good-storage-provisioner-for-on-premises-non-cloud-cluster/4258/5 "2020-01-29T22:09:19Z")

</div>

@dvree We are in a similar position. Can you share what worked for you?

---

<div class="post-metadata">

**Author:** ![dvree](https://avatars.discourse-cdn.com/v4/letter/d/dbc845/32.png) [@dvree](https://discuss.kubernetes.io/u/dvree)\
**Post date:** [January 29, 2020, 11:11pm UTC](https://discuss.kubernetes.io/t/good-storage-provisioner-for-on-premises-non-cloud-cluster/4258/6 "2020-01-29T23:11:46Z")

</div>

Rook - Ceph that utilize KVM pass-through SSD drives.

Steep learning curve, but works very well

---

<div class="post-metadata">

**Author:** ![larrydevops](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/larrydevops/32/4105_2.png) [@larrydevops](https://discuss.kubernetes.io/u/larrydevops)\
**Post date:** [January 30, 2020, 3:37pm UTC](https://discuss.kubernetes.io/t/good-storage-provisioner-for-on-premises-non-cloud-cluster/4258/7 "2020-01-30T15:37:56Z")

</div>

+1 for rook

---

<div class="post-metadata">

**Author:** ![lmacdonald](https://avatars.discourse-cdn.com/v4/letter/l/90db22/32.png) [@lmacdonald](https://discuss.kubernetes.io/u/lmacdonald)\
**Post date:** [March 4, 2021, 4:30pm UTC](https://discuss.kubernetes.io/t/good-storage-provisioner-for-on-premises-non-cloud-cluster/4258/8 "2021-03-04T16:30:24Z")

</div>

Hello. I am fairly new to Kubernetes and have setup a on premise 3 node cluster on VMs in VMware. I want to use dynamic provisioning on local storage. I tried Azure storage but since the Ubuntu version 20.04 that K8 is installed on doesn’t have support for Azure-Cli I didn’t get very far. My question is generally the same as above. I have had no luck with what dvree used. I am unable to install it, having difficulty following the instructions. If nothing else is there a fast way to create persistent volumes instead of writing the yaml file like 20 times. I would like to pre-create a bunch. Or if there is a new easy to follow document on some sort of dynamic provisioning I’d love that link. Thanks.
