URL getting rewritten when using Kubernetes ingress alongside Flask

General Discussions
1

Cluster information:

Kubernetes version: v1.33.4
Cloud being used: bare-metal
Installation method: command line
Host OS: Xubuntu 22.04 (both master and worker VMs), Windows 11 (host)
CNI and version: 10-calico-conflist
CRI and version: idk, I use Docker

I want to access two services with my Kubernetes ingress, and both services are based upon a Flask app. Each Flask app is made of a main.py script and a index.html web page, which is rendered using render_template library. Each service should let me get an answer from a Large Linguage Model (LLM) using Groq, and each service differs from the other only because of the model it uses, so I will show you the code of just one of them.

main.py

from flask import Flask, render_template, request
from groq import Groq

model_id="qwen/qwen3-32b"
groq_api_key = "<--my API key :)-->"

# Initialize the Groq client with the API key
client = Groq(api_key=groq_api_key)

app = Flask(__name__)

# Home route to display
@app.route('/')
def index():
    return render_template('index.html')

# Route to handle form submission
@app.route('/answer', methods=['POST'])
def answer():
    input_text = request.form.get('input_text')
    if not input_text:
        return "Please provide input text.", 400
    try:
        completion = client.chat.completions.create(
            model=model_id,
            messages=[
                {"role": "system", "content": "User chatbot"},
                {"role": "user", "content": input_text}
            ],
            temperature=1,
            max_tokens=1024,
            top_p=1,
            stream=True,
            stop=None,
        )
        # Collect the streamed response
        result = ""
        for chunk in completion:
            result += chunk.choices[0].delta.content or ""
    except Exception as e:
        return f"An error occurred: {e}", 500
    # Render the index.html template with the results
    return render_template('index.html', input_text=input_text, result=result)

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

index.html

<form action="{{ url_for('answer') }}" method="POST">
  <label for="input_text">Enter Input Text:</label>
  <br>
  <textarea id="input_text" name="input_text" rows="4" cols="50" required></textarea>
  <button type="submit">Submit</button>
</form>
{% if result %}
  <div class="result-container">
    <p><strong>Input Text:</strong> {{ input_text }}</p>
    <p><strong>Result:</strong> {{ result }}</p>
  </div>
{% endif %}

Similarly, for each Flask app there is a Deployment and a Service. The Deployment uses a simple Docker image with the Flask app and its dependencies inside. Since the two Deployments and the two Services are similar, I will show you the code of just one of them (each).

Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: groq-app
spec:
  selector:
    matchLabels:
      app: groq-app
  template:
    metadata:
      labels:
        app: groq-app
    spec:
      containers:
      - name: groq-app
        image: <--my DockerHub username :)-->/groq-test:v2
        ports:
        - containerPort: 5000

Service

apiVersion: v1
kind: Service
metadata:
  name: groq-app-service
spec:
  type: NodePort
  selector:
    app: groq-app
  ports:
  - name: http
    protocol: TCP
    port: 8080
    targetPort: 5000
    nodePort: 30008

Now the fun part: the ingress. And yes, I have an Ingress Controller (nginx) and it works fine (I use MetalLB to get external IP and load balancer).

Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-groq-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - host: test.com
    http:
      paths:
      - path: /qwen
        pathType: Prefix
        backend:
          service:
            name: groq-app-service
            port:
              number: 8080
      - path: /llama
        pathType: Prefix
        backend:
          service:
            name: llama-app-service
            port:
              number: 9090

The problem I need help for: when I try to access one of the two services via its URL, for example http://test.com/qwen, everything is fine, but when I type the input and press Submit, what I get is the URL http://test.com/answer and, obviously, a 404 NOT FOUND error. The URL I’d like to see, with the corresponding web page, is http://test.com/qwen/answer. Obviously, I want something similar for http://test.com/llama.

What does the rewrite, the Ingress or Flask? And how I fix it?

What I tried & I can tell so far:

  • Flask app works well when executed in any other way: using python or flask commands from both master and worker nodes, while inside a Docker container using docker run, while inside the pod accessing it directly and while inside the pod accessing it through the nodePort. The only problem is the URL rewrite.

  • I tried to use nginx.ingress.kubernetes.io/rewrite-target and regular expressions, but I don’t know the right way to do it. I don’t even know if it’s because of the ingress!

  • I tried to modify the subpath used in URLs by the Flask app, but without results. I don’t even know if it’s because of Flask!

Thank you in advance.

2

I’ve found the solution: a middleware inside my main.py based on this class. The full code of my main.py is now the following.

from flask import Flask, render_template, request
from groq import Groq

# App configuration
model_id = "qwen/qwen3-32b"
groq_api_key = "<--my API key :)-->"
PREFIX = "/qwen"

# PrefixMiddleware auxiliary class
class PrefixMiddleware:
    def __init__(self, app, prefix):
        self.app = app
        self.prefix = prefix

    def __call__(self, environ, start_response):
        path = environ.get('PATH_INFO', '')
        if path.startswith(self.prefix):
            environ['SCRIPT_NAME'] = self.prefix
            environ['PATH_INFO'] = path[len(self.prefix):] or '/'
        return self.app(environ, start_response)

# App definition
app = Flask(__name__)
app.wsgi_app = PrefixMiddleware(app.wsgi_app, PREFIX)
client = Groq(api_key=groq_api_key)

# Flask routes
@app.route('/')
    def index():
    return render_template('index.html')

@app.route('/answer', methods=['POST'])
    def answer():
    input_text = request.form.get('input_text')
    if not input_text:
        return "Please provide input text.", 400
    try:
        completion = client.chat.completions.create(
            model=model_id,
            messages=[
                {"role": "system", "content": "User chatbot"},
                {"role": "user", "content": input_text}
            ],
            temperature=1,
            max_tokens=1024,
            top_p=1,
            stream=True,
            stop=None,
        )
        result = ""
        for chunk in completion:
            result += chunk.choices[0].delta.content or ""
    except Exception as e:
        return f"An error occurred: {e}", 500

    return render_template('index.html', input_text=input_text, result=result)

# __main__
if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

In the Kubernetes Ingress, I also removed the annotation. The other files are the same.