I’m trying to figure out the best approach for managing storage across multiple cloud environments in Kubernetes, and I’m running into some challenges with syncing and access.
Cluster information:
Kubernetes version: v1.28
Cloud being used: AWS + GCP
Installation method: kubeadm
Host OS: Ubuntu 22.04
CNI and version: Calico v3.26
CRI and version: containerd 1.7
Problem:
I have workloads running in different clusters across AWS and GCP, and I need to:
Sync data between them reliably
Avoid manual downloads/uploads
Keep storage costs optimized
Right now I’ve tried:
Native cloud sync tools → too limited across providers
Custom scripts → hard to maintain
What I’m exploring:
I recently started testing a multi-cloud storage management tool that connects different providers and allows direct transfer/sync between them.
But I’m unsure if this is the right architectural approach for production use.
Have you given a thought to using a PV like NFS or something similar, where you can host the storage system server in one cloud or a neutral location and mount it as a Persistent Volume (PV) in both clusters?
Also, you can try using Ceph with Rook. Its a great way to fulfil your use case, but check the documentation to see how it fits in your requirements.
Or GlusterFS is also something that could solve your use case, but I am not 100% certain, but a quick test would help you identify the key parameters.
I think the above are better solutions, instead of looking for solutions that would sync the storage.
You should ideally have a storage that’s ouside the cluster (to a third location) and let your workload mount those volumes/storage, this way all your resources can use it regardless of cloud provider.
You can even do application level setup, like an object Storage (S3/GCS), instead of using block storage. Design the application to read/write to a multi-region object storage bucket (like an AWS S3 bucket with cross-region replication).