Prometheus pod is not coming up on Azure Kubernetes. Getting the following error:
ts=2020-09-17T09:43:08.601Z caller=main.go:764 err=“opening storage failed: found unsequential head chunk files /data/chunks_head/000068 (index: 68) and /data/chunks_head/000070 (index: 70)”
Upon checking this in Google, lot of users are facing this issue and the solution given was to delete out of sequence folders.
But we are unable to get into the STorage class in Azure and see the folders as well.
Basically we are stuck. Can anyone help?
Cluster information:
Kubernetes version:
Cloud being used: Azure Kubernetes version
Installation method:
Host OS: Ubuntu
CNI and version: Not sure
CRI and version: Not sure