Communication between pods on pods from different remote networks

Cluster information:

Kubernetes version: 1.11.1
Cloud being used: bare-metal (running on GCE, but installed from scratch)
Installation method:
Host OS: Ubuntu 16.04
CNI and version: Canal 3.8
CRI and version: Docker 18.06.1-ce

Problem descirption

I have a following problem:

I have setup a k8s cluster locally using kubeadm. Now I want to join a node from a remote network in that local cluster.
I was able to join the node with the appropriate kubeadm init ...(with correct public IP addresses of master node for flags --apiserver-advertise-address and --apiserver-cert-extra-sans).

But now, pods on that remote node cannot be reached.
Eg. I try, kubectl --namespace=kube-system logs canal-b9dhq calico-node where canal-b9dhq is a pod on a remote node, but I get:

Error from server: Get dial tcp i/o timeout

where is an local IP address of a remote node in its own local network.

All ports are open in both networks. I’m using NodePort and Canal (kubectl apply -f

Actually, both machines are VMs in GCP, in separate VPCs.
I cannot even debug this since, as can be seen from the error above, I cannot even get the logs of the pods on a remote node …

But, I can manually ssh to a remote node and see the logs using docker logs … For Calico pod, there are a lot of INFO logs which I assume confirms that there is communication between the master and the remote node.

Can anyone help?