Being pretty new to microk8s and kubernetes in general, it also took me hours to get cert-manager working with microk8s because I had to change the class name to public.
I think it would be good if the microk8s docs would mention this clearly.
1 Like
Trying to expose 4001/TCP+UDP does not seem to work. Can only choose one of them.
spec:
containers:
- name: nginx-ingress-microk8s
image: 'k8s.gcr.io/ingress-nginx/controller:v0.44.0'
args:
- /nginx-ingress-controller
- '--configmap=$(POD_NAMESPACE)/nginx-load-balancer-microk8s-conf'
- >-
--tcp-services-configmap=$(POD_NAMESPACE)/nginx-ingress-tcp-microk8s-conf
- >-
--udp-services-configmap=$(POD_NAMESPACE)/nginx-ingress-udp-microk8s-conf
- '--ingress-class=public'
- ' '
- '--publish-status-address=127.0.0.1'
ports:
- name: http
hostPort: 80
containerPort: 80
protocol: TCP
- name: https
hostPort: 443
containerPort: 443
protocol: TCP
- name: prox-tcp-4001
hostPort: 4001
containerPort: 4001
protocol: TCP
- name: prox-udp-4001
hostPort: 4001
containerPort: 4001
protocol: UDP
Once I apply these changes, the DaemonSet seems to accept it. But removes the UDP section internally from the config. Same, when I add the UDP section first, then TCP is removed.
How to expose both TCP+UDP for the same port?
@monky, you may have hit this kubernetes bug:
If you want both to work, they have to be created that way, updates will only give you one or the other because of the way key merging works
Hey, the Ingress docs page should reference the Metallb docs explaining how to set up Ingress with Metallb IPs. It took my a while and some help on Slack to figure this out. I noticed no svc of type loadbalancer was created so I tried to define the svc myself. Itās not obvious and shouldnāt be expected that you check Metallb docs page to find out how to have ingress-nginx add on use a load balancer.
Text suggestion:
āTo use Ingress with a load balancer, see Metallb docs.ā
After using the command microk8s enable ingress, I then wanted to check whether that had worked. I used the following commands to check:
$ k get pod -A | grep nginx
ingress nginx-ingress-microk8s-controller-wfg66 1/1 Running 0 29s
This shows me a pod, which is good.
$ k get svc -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 5m11s
This did not show me a service, which is confusing in comparison to the Helm install method, shown below.
By contrast, I also used:
helm upgrade --install ingress-nginx ingress-nginx \
--repo https://kubernetes.github.io/ingress-nginx \
--namespace ingress-nginx --create-namespace
From which I saw:
k get svc -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 14m
ingress-nginx ingress-nginx-controller-admission ClusterIP 10.152.183.27 <none> 443/TCP 2m43s
ingress-nginx ingress-nginx-controller LoadBalancer 10.152.183.2 <pending> 80:30704/TCP,443:31906/TCP 2m43s
$ k get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-node-hmfm9 1/1 Running 0 14m
kube-system calico-kube-controllers-f6b5877b8-qxplq 1/1 Running 0 14m
ingress-nginx ingress-nginx-controller-77f4468d76-c9j5h 1/1 Running 0 2m52s
The apparent discrepancy with the k svc -A output led me to doubt whether the microk8s enable ingress had in fact worked as expected.
I would like to suggest that the documentation be enhanced with commands that can be used to show that the nginx ingress controller is properly installed. Additionally, some indication of why the different install techniques seem to result in different outcomes would help.
Many thanks
Nathan
With respect to the default value of ingressClass=public, also see: #253 - nginx ingress microk8s - ` Service "default/gitea-http" does not have any active Endpoint.` - helm-chart - Gitea: Git with a cup of tea and kubernetes - Simple ingress from host with microk8s? - Stack Overflow
What would be good is a very explicit mention of this unexpected change, and also a means to set the ingressClass=nginx on the CLI, for example something like microk8s enable ingress:ingressClass=nginx.
To install nginx such that it works with the ingressClass=nginx use:
#https://kubernetes.github.io/ingress-nginx/deploy/
helm upgrade --install ingress-nginx ingress-nginx \
--repo https://kubernetes.github.io/ingress-nginx \
--namespace ingress-nginx --create-namespace
I would add to the doc something along the lines of:
You can confirm the addon is ready using:
sh -c "until microk8s.kubectl rollout status daemonsets/nginx-ingress-microk8s-controller -n ingress -w; do sleep 5; done"
hi, i have created a wordpress using this command
microk8s helm3 install wordpressnodeport bitnami/wordpress --set service.type=NodePort
after that i can access it via https://localhost:32284/
*attachment top image
then i enabled ingress via
microk8s enable ingress
then create a yaml like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: default
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/configuration-snippet: |-
proxy_ssl_server_name on;
proxy_ssl_name $host;
name: wordpressnodeport
spec:
ingressClassName: public
rules:
- host: wordpressnodeport.localhost
- http:
paths:
- path: /*
pathType: Prefix
backend:
service:
name: wordpressnodeport
port:
number: 443
and applied it:
microk8s kubectl apply -f ingress-controller-wordpressnodeport.yaml
now when i try to access https://localhost/wordpressnodeport, it can access but seems like resources arenāt loaded
*attachment bottom image
any idea what could be wrong in my case?
solution: i used the example in this link networking - How to enable default-http-backend in Kubernetes in order to make Ingress work? - Stack Overflow
and managed to fix the problem, now it works https://wordpressnodeport.localhost also for a second page https://wordpress2.localhost
example for wordpressnodeport in case anyone got similar problem like me:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
defaultBackend:
service:
name: wordpressnodeport
port:
number: 80
rules:
- host: wordpressnodeport.localhost
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: wordpressnodeport
port:
number: 80
for wordpress2, just replace wordpressnodeport with wordpress2. and replace name:ingress to wordpress2
example
metadata:
name: wordpress2
so one can spam as much controllers as one wish i think.
using
microk8s kubectl describe ingress
I get a lot of old configuration, anyone knows how to delete it? or how to get the original yaml when they are created?
As a new and inexperienced Kubernetes user, I was stumped to find that there was not an Nginx Ingress pod running. A mention of the ingress namespace would be nice along with maybe an example on how to troubleshoot the instance (microk8s kubectl logs -n ingress [pod name here]). Looking through the discussion here I am not the only person who was confused by this.
If Iām being honest, this page as a whole was not useful in the slightest to me beyond the installation command, as it just dumps a block of code in front of you with āthis is how you can create an Ingressā. I was still clueless at that point. Pointers to at least the Kubernetes documentation would have been helpful, or a description of what this block achieves. Also, how do I edit those config maps? Why would I do this over creating a rule?
2 Likes
Hello! I recently formed a microk8s cluster with 3 ubuntu nodes + 2 windows 2019 nodes. It seems that everything is working great and up and running. Also, thanks for @balchua1 's help on the nodeselector for different OS. However, I just found out that the microk8s addon ingress auto-scheduled 5 pods on each node. But those 2 pods on Windows are not working and are under āContainerCreatingā. Any chance I could custom make an enhancement on this? Thanks!
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/nginx-ingress-microk8s-controller 5 5 3 5 3 102m
NAME READY STATUS RESTARTS AGE
pod/nginx-ingress-microk8s-controller-zpk5g 0/1 ContainerCreating 0 102m
pod/nginx-ingress-microk8s-controller-tsr7n 1/1 Running 0 102m
pod/nginx-ingress-microk8s-controller-9rpxz 1/1 Running 0 99m
pod/nginx-ingress-microk8s-controller-tdr2q 1/1 Running 0 89m
pod/nginx-ingress-microk8s-controller-ck2hp 0/1 ContainerCreating 0 68m
Yes definitely. PRs are welcome.
Microk8s team has now split the addons to a different repo.
The ingress addon is part of the core addons
Thank you and I will try this out soon.
Indeed! itās very basic information that would be nice to have on the Add-On documentation page.
and microk8s enable ingress,pod status is ImagePullBackOff,I found out that I cannot access registry.k8s.io/ingress-nginx/controller, but I can access Google Cloud console. How to solve this? Can I modify the configuration file of ingress? Where can I modify
Can I use this but with nginx plus instead of the open source ?
@nguyenphu123 possibly. You would have to build your own add-on and call it something different, then replace whatever images and settings you needed from
Thanks for the reply, I have tried this and it seem working good. Now I just need to figure out how the pod can pull image from private registry instead of public.
Hi, do you have any example for nginx-ingress configmap, I tried to change the nginx.conf via edit nginx-config.yaml but it not applied.
Here is my nginx-config.yaml file:
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-conf
namespace: nginx-ingress
data:
nginx.conf: | #my nginx.conf configs are here
Here is my nginx-plus-ingress.yaml file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ingress
namespace: nginx-ingress
spec:
replicas: 2
selector:
matchLabels:
app: nginx-ingress
template:
metadata:
labels:
app: nginx-ingress
app.kubernetes.io/name: nginx-ingress
#annotations:
#prometheus.io/scrape: "true"
#prometheus.io/port: "9113"
#prometheus.io/scheme: http
spec:
serviceAccountName: nginx-ingress
automountServiceAccountToken: true
securityContext:
seccompProfile:
type: RuntimeDefault
volumes:
- name: nginx-conf
configMap:
name: nginx-conf
items:
- key: nginx.conf
path: nginx.conf
# - name: nginx-etc
# emptyDir: {}
# - name: nginx-cache
# emptyDir : {}
# - name: nginx-lib
# emptyDir: {}
# - name: nginx-log
# emptyDir: {}
containers:
- image: my nginx plus image
imagePullPolicy: IfNotPresent
name: nginx-ingress
ports:
- name: http
containerPort: 8082
- name: https
containerPort: 443
- name: readiness-port
containerPort: 8081
- name: prometheus
containerPort: 9113
- name: service-insight
containerPort: 9114
readinessProbe:
httpGet:
path: /nginx-ready
port: readiness-port
periodSeconds: 1
resources:
requests:
cpu: "100m"
memory: "128Mi"
#limits:
# cpu: "1"
# memory: "1Gi"
securityContext:
allowPrivilegeEscalation: false
# readOnlyRootFilesystem: true
runAsUser: 101 #nginx
runAsNonRoot: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
# volumeMounts:
# - mountPath: /etc/nginx
# readOnly: false
# name: nginx-conf
# subPath: nginx.conf
# name: nginx-etc
# - mountPath: /var/cache/nginx
# name: nginx-cache
# - mountPath: /var/lib/nginx
# name: nginx-lib
# - mountPath: /var/log/nginx
# name: nginx-log
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
args:
- -nginx-plus
- -nginx-configmaps=$(POD_NAMESPACE)/nginx-conf
- -report-ingress-status
- -external-service=nginx-ingress
#- -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
#- -enable-cert-manager
#- -enable-external-dns
#- -enable-app-protect
#- -enable-app-protect-dos
#- -v=3 # Enables extensive logging. Useful for troubleshooting.
#- -enable-prometheus-metrics
#- -enable-service-insight
#- -global-configuration=$(POD_NAMESPACE)/nginx-config
# initContainers:
# - image: my nginx image
# imagePullPolicy: IfNotPresent
# name: init-nginx-ingress
# command: ['cp', '-vdR', '/etc/nginx/.', '/mnt/etc']
# securityContext:
# allowPrivilegeEscalation: false
# readOnlyRootFilesystem: true
# runAsUser: 101 #nginx
# runAsNonRoot: true
# capabilities:
# drop:
# - ALL
# volumeMounts:
# - mountPath: /mnt/etc
# name: nginx-etc