Cluster information:
Kubernetes version: 1.14.
Cloud being used: bare-metal
Installation method:
Host OS: Centos 7
CNI and version: Calico
CRI and version:
Hi,
I want to use glusterfs-kubernetes to build my cluster storage.
It looks like the following:
kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
glusterfs-dynamic-21bb60f8-8907-11e9-9cee-0050568c1261 ClusterIP 10.107.212.208 <none> 1/TCP 6d8h
glusterfs-dynamic-58869a26-8c1e-11e9-9cee-0050568c1261 ClusterIP 10.99.79.113 <none> 1/TCP 2d9h
glusterfs-dynamic-78c5fed9-8df4-11e9-8273-0050568c1261 ClusterIP 10.104.170.211 <none> 1/TCP 114m
heketi ClusterIP 10.99.107.115 <none> 8080/TCP 121m
heketi-storage-endpoints ClusterIP 10.101.193.240 <none> 1/TCP 121m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 15d
The heketi service is defined as following:
kubectl describe service heketi
Name: heketi
Namespace: default
Labels: glusterfs=heketi-service
heketi=service
Annotations: description: Exposes Heketi Service
Selector: glusterfs=heketi-pod
Type: ClusterIP
IP: 10.99.107.115
Port: heketi 8080/TCP
TargetPort: 8080/TCP
Endpoints: 192.168.198.184:8080
Session Affinity: None
Events: <none>
my storage class is defined as followed:
kubectl describe sc standard
Name: standard
IsDefaultClass: No
Annotations: kubectl.kubernetes.io/last-applied-configuration={"allowVolumeExpansion":true,"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"standard"},"parameters":{"restauthenabled":"false","resturl":"http://10.99.107.115:8080","restuser":"notusedyet","restuserkey":"notusedyet","volumetype":"replicate:2"},"provisioner":"kubernetes.io/glusterfs"}
Provisioner: kubernetes.io/glusterfs
Parameters: restauthenabled=false,resturl=http://10.99.107.115:8080,restuser=notusedyet,restuserkey=notusedyet,volumetype=replicate:2
AllowVolumeExpansion: True
MountOptions: <none>
ReclaimPolicy: Delete
VolumeBindingMode: Immediate
Events: <none>
Now, here comes my problem. As the above configuration works fine at the beginning, it stopped working after a full cluster restart of kubernetes. I noticed that the clusterIP had changed.
But when I change the IPhttp://10.99.107.115:8080 to the service name http://heketi:8080 I cannot deploy an persistent volume claim any longer. Error message is that the server send empty response. But the url is working if I curl for it from another pod.
here is the yaml with the non working service name:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: standard1
provisioner: kubernetes.io/glusterfs
parameters:
#resturl: "http://192.168.198.149:8080"
resturl: "http://heketi:8080"
restauthenabled: "false"
restuser: "notusedyet"
restuserkey: "notusedyet"
volumetype: replicate:2
allowVolumeExpansion: true
Any Idea how to solve that issue?
Thanks, Andreas