Problems about Unmanaged Pod and Master Racing

Hello, I am a newcomer to Kubernetes, we are investigating to deploy our computing platform on Kubernetes. Our computing cluster is a master-slave architecture, the master takes charge of managing all the master and slave pods in a low-level semantics, said not by Deployment or ReplicaSet that Kubernetes provides. Also, the master needs to keep state for High-Availability mode. I have two questions:

  1. If we send a request to Kubernetes master to create a Pod with a specified label, is that possible that in some periods we can’t find that Pod via successfully polling the Kubernetes API with that label?
  2. If we set the RestartPolicy of the master pod to “OnFailure”, is it possible that in some extream situation or edge exception cases that the old master pod is not deleted but a new one is deployed to run(), so there are are two master pods running at the same time for a short period? I concern about it because, in our design, we must ensure that at any time there could be at most one master pod running to avoid the master racing condition.