Problem about reaching the prometheus usi at the new stable chart

General Discussions
1

I tried installing the new stable prometheus-operator helm chart. I changed clusterIP part to NodePort, set storageSpecs for persistent volumes and disable grafana installation. But the uis is not reachable somehow. Any idea?

# kubectl get svc -n monitoring
NAME                                           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
alertmanager-operated                          ClusterIP   None            <none>        9093/TCP,6783/TCP   68m
prometheus-operated                            ClusterIP   None            <none>        9090/TCP            67m
prometheus-operator-alertmanager               ClusterIP   10.233.7.172    <none>        9093/TCP            68m
prometheus-operator-kube-state-metrics         ClusterIP   10.233.51.240   <none>        8080/TCP            68m
prometheus-operator-operator                   ClusterIP   10.233.30.240   <none>        8080/TCP            68m
prometheus-operator-prometheus                 NodePort    10.233.61.13    <none>        9090:31090/TCP      68m
prometheus-operator-prometheus-node-exporter   ClusterIP   10.233.8.8      <none>        9100/TCP            68m
`
2

Does the service have any valid targets? - You can use kubectl describe service prometheus-operator-prometheus to do a quick check.

3

No

kubectl describe service prometheus-operator-prometheus -n monitoring

Name: prometheus-operator-prometheus
Namespace: monitoring
Labels: app=prometheus-operator-prometheus
chart=prometheus-operator-4.3.3
heritage=Tiller
release=prometheus-operator
Annotations:
Selector: app=prometheus,prometheus=prometheus-operator-prometheus
Type: NodePort
IP: 10.233.61.13
Port: web 9090/TCP
TargetPort: 9090/TCP
NodePort: web 31090/TCP
Endpoints:
Session Affinity: None
External Traffic Policy: Cluster
Events:

All the pods are running though

kubectl get pods -n monitoring

NAME READY STATUS RESTARTS AGE
alertmanager-prometheus-operator-alertmanager-0 2/2 Running 0 31h
prometheus-operator-kube-state-metrics-78fb6c979-2q4ll 1/1 Running 0 31h
prometheus-operator-operator-7cdd54b449-fvcrh 1/1 Running 0 31h
prometheus-operator-prometheus-node-exporter-c2scq 1/1 Running 0 31h
prometheus-operator-prometheus-node-exporter-kcwxl 1/1 Running 0 31h
prometheus-operator-prometheus-node-exporter-v8v2g 1/1 Running 0 31h
prometheus-operator-prometheus-node-exporter-vkzxw 1/1 Running 0 31h
prometheus-operator-prometheus-node-exporter-w9xdg 1/1 Running 0 31h
prometheus-operator-prometheus-node-exporter-wpc7g 1/1 Running 0 31h
prometheus-operator-prometheus-node-exporter-xmtvs 1/1 Running 0 31h

4

I don’t see prometheus itself in that list. You should have something like prometheus-kube-prometheus-0 in there. Double check the statefulset was created.