This is resolved, so I’ll close this question.
For those who hit the same issue, this was troubleshooting
- I’ve looked at the service code (java) and didn’t see any logic that can handle SIGTERM
- That rings issue immediately to pull the image (or you can build a new one) and run locally
- I’ve baked a custom compose to bring what service needs to start in
dev
mode - As soon as composing started I did the following
docker kill -s SIGTERM <continer-name>
- what happens there is that service didn’t respond as I expected at all
- I’ve baked a custom compose to bring what service needs to start in
- That rings issue immediately to pull the image (or you can build a new one) and run locally
You can see below that if I send SIGKILL that will produce 137 as k8s logs showed above on the traces.
I’ve done a test on the redis
with the same SIGTERM and as you can see it gracefully shut down with the right exit code
<removed logs of the service>
api exited with code 137
redis | 1:signal-handler (1721988133) Received SIGTERM scheduling shutdown...
redis | 1:M 26 Jul 2024 10:02:13.970 # User requested shutdown...
redis | 1:M 26 Jul 2024 10:02:13.970 * Saving the final RDB snapshot before exiting.
redis | 1:M 26 Jul 2024 10:02:13.972 * DB saved on disk
redis | 1:M 26 Jul 2024 10:02:13.972 # Redis is now ready to exit, bye bye...
redis exited with code 0
NOTE: This was a new service to me so I wasn’t aware of this missed configuration before.