Cluster information:
Kubernetes version: 1.30
Cloud being used: eks
My understanding was that the probe would start (initialDelaySeconds + periodSeconds) after that container was started (when initialDelaySeconds greater than periodSeconds)
However, in my time managing Kubernetes, I’ve seen things that seem to prove my knowledge wrong, and I’ve tested them
Here’s the first pod we tested.
apiVersion: v1
kind: Pod
metadata:
name: test-probe
labels:
app: test-probe
spec:
initContainers:
- name: init
image: nginx
command:
- /bin/sh
- -c
- "echo 'started and wait 10 seconds' && sleep 10"
containers:
- name: nginx
image: nginx
readinessProbe:
httpGet:
path: "/"
port: 80
initialDelaySeconds: 20
periodSeconds: 15
successThreshold: 1
failureThreshold: 2
timeoutSeconds: 1
[logs]
## init container
[2025-01-02T16:44:26] started and wait 10 seconds
[2025-01-02T16:44:38] /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
...And another nginx startup logs...
And the pod status at this point is
[pod status]
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2025-01-02T16:44:26Z"
status: "True"
type: PodReadyToStartContainers
- lastProbeTime: null
lastTransitionTime: "2025-01-02T16:44:37Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2025-01-02T16:44:24Z"
message: 'containers with unready status: [nginx]'
reason: ContainersNotReady
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2025-01-02T16:44:24Z"
message: 'containers with unready status: [nginx]'
reason: ContainersNotReady
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2025-01-02T16:44:24Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://03e72a8fe53d70f428891ba8dabe3f70f51c71dd8c3298163cfc1c124d4c37bd
image: docker.io/library/nginx:latest
imageID: docker.io/library/nginx@sha256:42e917aaa1b5bb40dd0f6f7f4f857490ac7747d7ef73b391c774a41a8b994f15
lastState: {}
name: nginx
ready: false
restartCount: 0
started: true
state:
running:
startedAt: "2025-01-02T16:44:38Z"
initContainerStatuses:
- containerID: containerd://d6f69d93a01ed286903e12d23dfcbeb2c12752036a28e287c48d712e6a74518b
image: docker.io/library/nginx:latest
imageID: docker.io/library/nginx@sha256:42e917aaa1b5bb40dd0f6f7f4f857490ac7747d7ef73b391c774a41a8b994f15
lastState: {}
name: init
ready: true
restartCount: 0
started: false
state:
terminated:
containerID: containerd://d6f69d93a01ed286903e12d23dfcbeb2c12752036a28e287c48d712e6a74518b
exitCode: 0
finishedAt: "2025-01-02T16:44:36Z"
reason: Completed
startedAt: "2025-01-02T16:44:26Z"
phase: Running
startTime: "2025-01-02T16:44:24Z"
Based on my existing knowledge, I thought the first probe would occur at around 16:45:13, which is 16:44:38(containerStatuses[0].state.running.startedAt) + 20s(initialDelaySeconds) + 15s(periodSeconds)
However, the actual first probe occurred at 16:45:09 (event also has no history of failed probes)
(looks like, status.startTime + 10(delay) + 20(init) + 15(period))
... [02/Jan/2025:16:45:09 +0000] "GET / HTTP/1.1" 200 615 "-" "kube-probe/1.30+" "-"
Other test pods don’t make sense either
apiVersion: v1
kind: Pod
metadata:
name: test-probe
labels:
app: test-probe
spec:
initContainers:
- name: init
image: nginx
command:
- /bin/sh
- -c
- "echo 'started and wait 10 seconds' && sleep 10"
containers:
- name: nginx
image: nginx
readinessProbe:
httpGet:
path: "/"
port: 80
initialDelaySeconds: 0
periodSeconds: 20
successThreshold: 1
failureThreshold: 2
timeoutSeconds: 1
[logs]
## init container
[2025-01-02T16:59:56] started and wait 10 seconds
[2025-01-02T17:00:08] /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
...And another nginx startup logs...
...[02/Jan/2025:17:00:09 +0000] "GET / HTTP/1.1" 200 615 "-" "kube-probe/1.30+" "-"
[status]
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2025-01-02T16:59:57Z"
status: "True"
type: PodReadyToStartContainers
- lastProbeTime: null
lastTransitionTime: "2025-01-02T17:00:07Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2025-01-02T17:00:09Z"
status: "True"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2025-01-02T17:00:09Z"
status: "True"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2025-01-02T16:59:55Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://dc44abdee3e7545612efd11da9893dddc09300df9752670406921cbbc7b90652
image: docker.io/library/nginx:latest
imageID: docker.io/library/nginx@sha256:42e917aaa1b5bb40dd0f6f7f4f857490ac7747d7ef73b391c774a41a8b994f15
lastState: {}
name: nginx
ready: true
restartCount: 0
started: true
state:
running:
startedAt: "2025-01-02T17:00:08Z"
initContainerStatuses:
- containerID: containerd://ce4f4aa69bdb26785169f234fbc9eab42194852b3a6c931bf6a360233405bd4e
image: docker.io/library/nginx:latest
imageID: docker.io/library/nginx@sha256:42e917aaa1b5bb40dd0f6f7f4f857490ac7747d7ef73b391c774a41a8b994f15
lastState: {}
name: init
ready: true
restartCount: 0
started: false
state:
terminated:
containerID: containerd://ce4f4aa69bdb26785169f234fbc9eab42194852b3a6c931bf6a360233405bd4e
exitCode: 0
finishedAt: "2025-01-02T17:00:06Z"
reason: Completed
startedAt: "2025-01-02T16:59:56Z"
phase: Running
qosClass: BestEffort
startTime: "2025-01-02T16:59:55Z"
Please explain when the first probe starts when there is an initContainer.