Windows nodes drop network connection on pod termination

Environment:
K8 Version: v1.18.2
Windows Version: Server 1909
Docker Version: 19.3.8
Network Type: L2Bridge
Network Plugin: flannel

Wanting to know if anyone else has experienced this.
We’re running a mixed OS k8 cluster, about 85 nodes, half ubuntu 18.04.4, half Windows Server 2019 v1909.
The nodes are all virtual machines running in hyper-v locally in our datacenter.
We’re experiencing issues with the windows nodes specifically, when a pod terminates on one of the nodes, the network connections for all windows nodes will start to freak out.
For about a minute after the pod termination, either the latency will go through the roof and pings will reply at over 1000ms or they will disconnect from the network completely for a second or two.
We have linux nodes on the same subnet, same VLAN, running on the same Hyper-V host that experience absolutely no issues, no network latency, no network drops.

Our assumption is that it’s something happening with windows HNS on the windows nodes and have a support ticket opened with microsoft but wanted to see if anyone else in the k8 community has experienced this