Apply 4000 rewrite rules in Ingress

General Discussions
1

Asking for help? Comment out what you need so we can get more information to help you!

Cluster information:How to apply 4000 rewrite rules in ingress resource under server-snippet.

After adding 1000 rewrite rules it start giving error. K8S version 1.21.14
Error : The Ingress “app-k8s” is invalid: metadata.annotations: Too long: must have at most 262144 bytes

nginx.ingress.kubernetes.io/server-snippet: |
  location = /abc.html { return 301 http://ayx.com; }
  location = /zyz.html { return 301 http://funtou.com; }
  <need to add 4000 rewrite rules here>

Kubernetes version:
Cloud being used: (put bare-metal if not on a public cloud)
Installation method:
Host OS:
CNI and version:
CRI and version:

You can format your yaml by highlighting it and pressing Ctrl-Shift-C, it will make your output easier to read.

1 Like
2

I am also facing the same issue.

3

Annotations have a size limit of 256kB

With so many entries you might be hitting this normally, but if you created it with kubectl apply the previous version of the object is stored as an additional annotation: kubectl.kubernetes.io/last-applied-configuration. You can see some examples of this here: Declarative Management of Kubernetes Objects Using Configuration Files | Kubernetes

You can work around this by using kubectl replace instead of apply,

As a sidenote, you REALLY should upgrade. 1.21 has been EOL’ed and out of support for some time, and if you’re relying on upstream (deploying with kubeadm etc) you will not be able to upgrade (easily) forever. The k8s.gcr.io endpoint has been replaced with registry.k8s.io and older images may be pruned from the old one.

Newer versions also have features like Server Side Apply that work around some of the limitations you’ve come across.

4

Thanks for the reply.
The remedy was attempted to be implemented in K8s version 1.23.5, however the fault persisted.

5

Did you use kubectl replace or did you upgrade and use apply? Is kubectl.kubernetes.io/last-applied-configuration still there?

Either way you have a hard limit on the size of annotations, even on the latest version of K8s, trying to put 4000 configs in there hit the limit alone.

6

Also as an FYI - this version will also be EOL’ed at the end of the month:

7

Thanks for the reply.

For the 4000 rule, we have used the “kubectl replace” command in k8s version 1.21, yet the problem persists.
Then, using a combination of the following techniques, we tried K8s version 1.23.5. Issue is still persists in version k8s 1.23.5

kubectl apply -f abc.yml
kubectl replace -f abc.yml

8

It will persist across all versions then, that’s just too many annotations and you’re hitting the hard limit.

You may want to create multiple ingresses and setup the rewrite rules that way or look into loading the config directly into it via configmap+file mount.