Network issue - Time to first byte

Cluster information:

Kubernetes version: 1.14.2
Cloud being used: bare-metal
Installation method:
Host OS:
CNI and version: weave 2.5.2
CRI and version: Docker 18.09.1

HI everybody!

I have notices some strange network performance problems with my cluster.
It seems that our K8s Cluster, with 1 Master and 4 Nodes is slower as Minikube on my local machine.
When I do a Test on an application like Wirecloud I have massive network time delays.
As example:

For a better understanding I separated the problem description in 2 Phases.

Phase 1:

  • It takes about 4-5 seconds until the application begins to load
    (I think this is called time to first byte)

Phase 2:

  • Application loads 2-3 seconds

I see major differences in Phase 1. As example in our production and test environment I have timeslots from 4-5 seconds. When I install Minikube the time to the first byte is only 0-1 seconds. All the server are installed with Cent Os 7.x.latest.

We are using the weave-net pod version 2.5.2 and Kubernetes Version 1.13.4 (Prod) and 1.14.2) on the PreProd. The System is installed in an VM environment managed via VMWare.

I did some research about different tuning profiles, but in our situation this is set by default
- virtual-guest - Optimize for running inside a virtual guest

So for my understanding the difference is only in the network, has anyone a quick shot or a lucky finding for me ?
Cheers Mchoeti

Can you profile your application to see if your app is taking so long or if some other hop?

Maybe your app does a DNS request, and in the cluster it takes time to resolve (or times out), for example?