Hello all 
,
we need to fulfill the requirement to have a redundant Kubernetes system.
This requirement is for our small edge devices, where we have very minimalistic clusters.
We have the following two scenarios there:
- 3 Master cluster, which should still operate healthy, if two masters are down
- 2 Master cluster, which should still operate if one master is down (this is probably already covered if finding a solution for the first scenario)
In this minimalistic environments the Master nodes are also acting as Worker nodes and are the only existing servers in the cluster.
I know officially with embedded etcd we could set up a 3 Master cluster but then only 1 Master is allowed to have an outage, I also saw the answer here: High Availability Host Numbers - #2 by meyay
Also I am aware, that this is because of the quorum of etcd, that it requires it, but I am wondering if we can somehow manipulate the quorum, so that we can maybe vote ourself 2 times.
Maybe an alternative to etcd which would work in this scenarios, avoiding a split-brain. But no matter if using etcd or an other external database, all those components still need to run on the existing servers, no extra hardware can be added.
I also found so far one company which is selling some solution for this scenario here: https://www.evidian.com/products/high-availability-software-for-application-clustering/kubernetes-the-simplest-high-availability-cluster-with-synchronous-replication-and-failover-between-two-redundant-servers/
But I am searching some open source solution for it which we can better integrate in our existing software architecture.
Any help is welcome, let me know in case anything is unclear or you need more informations 
Cluster information:
Cloud being used: bare-metal
Host OS: CentOS 7
You can format your yaml by highlighting it and pressing Ctrl-Shift-C, it will make your output easier to read.