Local kubernetes cluster network problem/config

Cluster information:

Kubernetes version: v1.24.2
Cloud being used: bare-metal
Installation method: kubeadm
Host OS: ubuntu 22.04
CNI and version: calico 0.3.1
CRI and version: containerd 1.6.6

I installed via kubeadm: 3x control panels, 3x workers, external etcd, HA via external haproxy.

My network:
ISP router (192.168.1.1) →
PfSense VLANs (10.0.10.1, 10.0.50.1, …) →
K8s Cluster (10.0.50.31…10.0.50.36) →
Cluster network (10.0.80.0)

Installed metallb as LB, traefik as ingress controller.

I deployed a test nginx:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: nginx
  name: nginx-deploy-main
spec:
  replicas: 1
  selector:
    matchLabels:
      run: nginx-main
  template:
    metadata:
      labels:
        run: nginx-main
    spec:
      containers:
      - image: nginx
        name: nginx
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: nginx
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`nginx.example.com`)
      kind: Rule
      services:
        - name: nginx-deploy-main
          port: 80
  tls:
    certResolver: letsencrypt

I can curl the nginx from inside the network (the certificate is properly created).
I am pointing the domain (nginx.example.com - changed to real domain) to the ISP router’s IP via cloudflare, but I can’t access it from outside, the traffic seems to stop before reaching the service.

I should have proprely port-forwarded HTTP and HTTPS traffic from both the ISP router and the pfSense to traefik.
Should I port-forward to the single service instead?

Thank you,

Thanks,

How do you have metallb configured? Is the IP range allocated to it routable on your host network?

The network (10.0.50.0) is a vlan managed by pfsense.
PfSense has dhcp enabled on a different range from metallb’s range.
All other devices/machines on that network are visible, it’s the same network the k8s hosts are.

How do I see of that particular range is routable or not?

Traefik Loadbalancer IP is 10.0.50.100, on the same network the cluster hosts are, for example.

Thanks,

Can pfsense curl or nc 10.0.50.100? by “inside the network” do you mean on k8s hosts? or other systems also on the same network?

Adding the service domain to pfsense pointing to traefik, all the network sees and makes curl of nginx.

I also modified metallb from L2 to BGP.

Still if I try to curl from outside the network (internet), going through cloudflare.
Cloudflare says it can’t reach the server.

So I think it must be the ISP router or pfSense blocking outside requests, not forwarding them.

Thing is I can use a vpn from outside, so some traffic is passed, I need to pass http/https.

Ok, managed to properly port-forward http/https traffic to traefik!

Thank you for your time, you actually pointed me in the right directions.