Node.js with NGINX ingress controller; no images are shown

General Discussions
1

Good day all,

I have a small node.js program that gets it’s images from the default public directory. When I expose it using a service of type NodePort the applications is accesibPreformatted textle and shows it’s images. When I use a nginx ingress deployed using a daemonset I can access the application but none of my images are shown.

The ingress rules used are:
> apiVersion: extensions/v1beta1
> kind: Ingress
> metadata:
> name: visitor
> annotations:
> nginx.ingress.kubernetes.io/rewrite-target: /$1
> nginx.ingress.kubernetes.io/ssl-redirect: “false”
> kubernetes.io/ingress.class: visitor
> spec:
> rules:
> - http:
> paths:
> - path: /hello/?(.*)
> backend:
> serviceName: visitor-hello
> servicePort: 8080

The logs from the nginx controller

> 10.8.62.8 - [10.8.62.8] - - [17/Feb/2019:18:06:13 +0000] "GET /hello/ HTTP/1.1" 200 162 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36" 454 0.001 [visitor-visitor-hello-8080] 192.168.3.196:8080 162 0.001 200 8afd95c4d2e13c73dade53370c8c9aed
> 10.8.62.8 - [10.8.62.8] - - [17/Feb/2019:18:06:13 +0000] "GET /hello.svg HTTP/1.1" 404 143 "http://kub14n04:30100/hello/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36" 342 0.001 [visitor-visitor-hello-8080] 192.168.3.196:8080 143 0.001 404 f43b075fcc5d8ba73ed8c04496d94af7

The relevant nodejs code

> app.use(express.static('public')) ;
> 
>   res.send('<img src="/roxit.png" width=200px><H1>Hello World!</H1><br>Serving you from '+hostname+' <br>You are visitor: #'+visitorCount+' and coming from '+visitorAddr+'<br>') ;
>   res.end() ;

My question:

How can I setup the ingress rules in such a way that the images, js etc are becoming ‘visible’ in my nodejs application?

I will document the issue and the solution for others.

Kind regards,

Pascal van Dam

2

The 404 for the image is the one you expect that is not showing? That log is from the ingress, right? What does you application log?

If the file exist on the path, it would probably be served. I bet there is something with the nginx redirection you are doing that gets in the way.

Can you check you app logs and see what request arrives to the app and if it makes sense that a file does not exist? Or disabling the redirect too.

3

Good atfernoon Rodrigo,

Thanks for your quick reply.

Looking at the logs of one of my nodejs pods:

app listening on port 8080!
GET
{ host: 'kub14n02:30100',
  'x-request-id': '21bddf9c135c98442317ea26fe41cff0',
  'x-real-ip': '10.8.62.216',
  'x-forwarded-for': '10.8.62.216',
  'x-forwarded-host': 'kub14n02:30100',
  'x-forwarded-port': '80',
  'x-forwarded-proto': 'http',
  'x-original-uri': '/cs',
  'x-scheme': 'http',
  'upgrade-insecure-requests': '1',
  'user-agent':
   'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36',
  accept:
   'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
  'accept-encoding': 'gzip, deflate',
  'accept-language': 'en-US,en;q=0.9' }
/
** catch all **
GET
{ host: 'kub14n02:30100',
  'x-request-id': '7b8ed26f90600abb9c998b46a69da1c9',
  'x-real-ip': '10.8.62.216',
  'x-forwarded-for': '10.8.62.216',
  'x-forwarded-host': 'kub14n02:30100',
  'x-forwarded-port': '80',
  'x-forwarded-proto': 'http',
  'x-original-uri': '/cs.svg',
  'x-scheme': 'http',
  'user-agent':
   'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36',
  accept: 'image/webp,image/apng,image/*,*/*;q=0.8',
  referer: 'http://kub14n02:30100/cs',
  'accept-encoding': 'gzip, deflate',
  'accept-language': 'en-US,en;q=0.9' }
/.svg

Ergo, it looks like the ingress controller is eating my cs from cs.svg
Any idea how to prevent this? I am using the 0.22 version of the Ingress controller.

Thanks in advance.

Kind regards,

Pascal van Dam

4

Sorry, I must be missing something. But where there you see the app responded with a 200 for the image request?

I’m on my phone, maybe it’s there :smiley:

5

Please paste the output from curl -vvv ${YOUR_SITE_HERE}, as it will show exactly what is going on and which server (NGINX Ingress or your Express one) is actually making the response.

6

Hi Llarson, thanks for your reply, herewith the output of curl -vvv

[pascal@kub14n01 log]$ curl -vvv http://kub14n01:30100/cs
* About to connect() to kub14n01 port 30100 (#0)
*   Trying 10.8.63.119...
* Connected to kub14n01 (10.8.63.119) port 30100 (#0)
> GET /cs HTTP/1.1
> User-Agent: curl/7.29.0
> Host: kub14n01:30100
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx/1.15.8
< Date: Sat, 23 Feb 2019 13:46:38 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 157
< Connection: keep-alive
< X-Powered-By: Express
< ETag: W/"9d-7KaeLZLJoUQNG/XR3mNa9eSk1W4"
< 
* Connection #0 to host kub14n01 left intact
<img src="/cs.svg" width=200px><H1>Hello World!</H1><br>Serving you from visitor-cs-765f8bbbc9-ltvdg <br>You are visitor: #3 and coming from 192.168.3.18<br>[pascal@kub14n01 log]$
7

Looking very carefully at the rule you’ve specified (very similar to the official example from the docs), let’s go through what happens when we try to access /hello.svg:

  • Does /hello/?(.*) match /hello.svg? Yes, it does. There’s no slash (optional), so the (.*) part gets filled with .svg and stored for later use in $1.
  • Your rewrite-target rule says to rewrite $1 into /$1, which essentially means that your NGINX Ingress will make a request to your express app looking like: /.svg, which is obviously not right, and you get a 404.

This is exactly what your log output says, too.

So the good news is that everything is working as instructed, and you just need to fix your regex and rewrite rule.

Do you have to include the slash? Does it bring benefit somehow? Because if you would just remove the slash from the rewrite rule, I am pretty sure the routing (and image serving) would work.

8

Good afternoon…

I think I need the / cause alle the apps that are at the backend work from /
After experimenting a little bit I see the following:

When I put a trailing / to the URL; it works. I see the app and it’s images without issues:
E.g.: http://kub14n01:30100/at/

This results in the logs

1 92.168.0.1 - [192.168.0.1] - - [24/Feb/2019:10:52:18 +0000] “GET /at/ HTTP/1.1” 200 157 “-” “Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:65.0) Gecko/20100101 Firefox/65.0” 336 0.016 [visitor-visitor-at-8080] 192.168.1.251:8080 157 0.016 200 88f9317393a7c27ffec86b26baff2351
192.168.0.1 - [192.168.0.1] - - [24/Feb/2019:10:52:18 +0000] “GET /at/at.jpg HTTP/1.1” 200 28169 “http://kub14n01:30100/at/” “Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:65.0) Gecko/20100101 Firefox/65.0” 288 0.005 [visitor-visitor-at-8080] 192.168.1.251:8080 28169 0.005 200 1d673bcae8b1e439c89402e72788db33

from the ingress-controller

The curl output:

192.168.0.1 - [192.168.0.1] - - [24/Feb/2019:10:52:18 +0000] "GET /at/ HTTP/1.1" 200 157 "-" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:65.0) Gecko/20100101 Firefox/65.0" 336 0.016 [visitor-visitor-at-8080] 192.168.1.251:8080 157 0.016 200 88f9317393a7c27ffec86b26baff2351
192.168.0.1 - [192.168.0.1] - - [24/Feb/2019:10:52:18 +0000] "GET /at/at.jpg HTTP/1.1" 200 28169 "http://kub14n01:30100/at/" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:65.0) Gecko/20100101 Firefox/65.0" 288 0.005 [visitor-visitor-at-8080] 192.168.1.251:8080 28169 0.005 200 1d673bcae8b1e439c89402e72788db33

The ingress rules:

  - path: /at/?(.*)
    backend:
      serviceName: visitor-at
      servicePort: 8080

If I omit the trailing /

Eg: http://kub14n01:30100/at

Then I do not see a image and actually the loading of the page stalls (no image is een, the text is displayed)

Logs from the nginx-controller

192.168.0.1 - [192.168.0.1] - - [24/Feb/2019:11:00:09 +0000] "GET /at HTTP/1.1" 200 156 "-" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:65.0) Gecko/20100101 Firefox/65.0" 335 0.001 [visitor-visitor-at-8080] 192.168.1.251:8080 156 0.001 200 f17066c116affa5b95e189feef60e0a0
2019/02/24 11:01:09 [error] 47#47: *3788 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.0.1, server: _, request: "GET /at.jpg HTTP/1.1", upstream: "http://192.168.1.251:8080/.jpg", host: "kub14n01:30100", referrer: "http://kub14n01:30100/at"

curl debug output:

[pascal@kub14n01 visitor]$ curl -vvv http://kub14n01:30100/at
* About to connect() to kub14n01 port 30100 (#0)
*   Trying 10.8.63.119...
* Connected to kub14n01 (10.8.63.119) port 30100 (#0)
> GET /at HTTP/1.1
> User-Agent: curl/7.29.0
> Host: kub14n01:30100
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx/1.15.8
< Date: Sun, 24 Feb 2019 11:03:02 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 155
< Connection: keep-alive
< X-Powered-By: Express
< ETag: W/"9b-Y1J0fBXdrZzAmZr5Y8IIP8yhBQ4"
< 
* Connection #0 to host kub14n01 left intact
<img src="at.jpg" width=200px><H1>Hello World!</H1><br>Serving you from visitor-at-567dcd84f8-vjcv2 <br>You are visitor: #4 and coming from 192.168.2.8<br>

The rewrite rule in both cases is still:

    nginx.ingress.kubernetes.io/rewrite-target: /$1
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    # https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#enable-cors
    #nginx.ingress.kubernetes.io/enable-cors: "true"
    kubernetes.io/ingress.class: visitor

Question now is: hot to configure the rules so that I do not have to specify the trailing /

Thanks for helping!

Kind regards,

Pascal van Dam