Good storage provisioner for on-premises (non-cloud) cluster?


#1

We have an simple on-premises kubernetes cluster here and are looking for the best way to provision persistent volumes. The cluster runs on KVM – not OpenStack. Currently we use the nfs provisioner here https://github.com/kubernetes-incubator/external-storage which has been working well for us, but for databases, the performance over NFS is like 6X slower than a local disk.

Is there a better solution? What do non-cloud kubernetes people use?

I found this “local” provisioner:

but it is static only – no dynamic provisioning. Less than ideal, but workable.

Any suggestions are appreciated.


#2

Unfortunately, the answer is always going to be “It depends.” That said, how large do you expect your clusters to be, how many applications do expect to run, and what is your expected storage footprint in terms of size? There are open-source block storage solutions that could run within your cluster (Nexenta, OpenEBS, Ceph/Rook, etc.) but there are management overheads there.

We know folks using local storage for their Kubernetes clusters but then you do want to use tools like Kanister (https://github.com/kanisterio/kanister) to extract data for backup/recovery/migration/etc.

Hope this helps but happy to answer additional questions if needed.


#3

Since you are on KVM and not on OpenStack or vSphere or something more sophisticated like that, I definitely recommend you take a look at Rook, which seems specifically developed for this use case.

https://rook.io/

Please report back with your findings and experience, as this is something that quite a few people in the community would be interested in.


#4

We will be taking a hard look at rook – thank you both for your feedback.