Memory management, how?

Hello all,
I have a general question regarding memory management. Most probable I’m doing something wrong, or simply it is lack of knowledge.
I’m running an application, that in one container is running few subprocesses.
pod is created with memory limit - but in some cases it is not fitting into it…
By default, if one of subprocess will cause exceeding the limit - whole container is killed.
I’m considering implementing in my application monitoring of used memory, and act before k8s will kill me, but before I would like to ask if killing whole pod is the only way k8s knows. I was looking thru the documentation but did not find anything more precise. What would make me happy is that instead of kill, simply allocation of memory will fail - then subprocess will quit, and others subprocesses will be saved. Or my app will receive signal that I will be able to handle and kill biggest subprocess. Or simply do nothing, if worker node still have lot of memory - so tie all of containers of my app in kind of budget - but then it is a problem what to do when it is reaching limit - but for sure probability that all containers will exceed limit is much lower than one of many containers will do so.
how you are dealing with memory? are you really happy with sigkill?
thanks in advance for any advice!!