Nginx 403 Forbidden Error while using NFS-Shared volume on kubernetes

Hi Team ,

I am testing out the Shared Persistent Volumes on my Cluster . I have created an EKS cluster on AWS , I have installed nfs server on one of the nodes & created PV& PVC for the same . I am able to mount the volumes with multiple applications (i am able to see the data in nfs-server, application pod & nginx pod ) .But i am unable to access the html files from nginx server as it throws 403 Forbidden Error .
Below are the Screenshot of my NFS -Sever ,nginx where my data mounted on it with permissions

Kindly help me with where i am doing wrong

Thank you in advance

Please find the nginx screenshot which shows the data is actually being mounted to Nginx pod

What’s in the nginx error logs?

When I checked out the container, the nginx image is built to direct all error output to stderr:

$ kubectl run nginx --rm -it --image=nginx:latest -- /bin/bash
If you don't see a command prompt, try pressing enter.
root@nginx:/# ls -lah /var/log/nginx/error.log
lrwxrwxrwx 1 root root 11 May 25 15:43 /var/log/nginx/error.log -> /dev/stderr

So this output should land in the container logs. You can check with this command:

kubectl logs pod/nginx-pod-name -c container-name-here

The -c container-name-here is not necessary if this is a single container pod.

Hi @protosam Thank you for the reply , below are the logs for my pod
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2021/06/07 13:42:14 [notice] 1#1: using the “epoll” event method
2021/06/07 13:42:14 [notice] 1#1: nginx/1.21.0
2021/06/07 13:42:14 [notice] 1#1: built by gcc 8.3.0 (Debian 8.3.0-6)
2021/06/07 13:42:14 [notice] 1#1: OS: Linux 5.4.117-58.216.amzn2.x86_64
2021/06/07 13:42:14 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2021/06/07 13:42:14 [notice] 1#1: start worker processes
2021/06/07 13:42:14 [notice] 1#1: start worker process 30
2021/06/07 13:42:14 [notice] 1#1: start worker process 31
2021/06/07 13:42:14 [notice] 1#1: start worker process 32
2021/06/07 13:42:14 [notice] 1#1: start worker process 33
2021/06/07 13:42:14 [notice] 1#1: start worker process 34
2021/06/07 13:42:14 [notice] 1#1: start worker process 35
2021/06/07 13:42:14 [notice] 1#1: start worker process 36
2021/06/07 13:42:14 [notice] 1#1: start worker process 37
2021/06/07 13:44:15 [error] 30#30: *1 directory index of “/usr/share/nginx/html/” is forbidden, client: 127.0.0.1, server: localhost, request: “GET / HTTP/1.1”, host: “localhost”
127.0.0.1 - - [07/Jun/2021:13:44:15 +0000] “GET / HTTP/1.1” 403 153 “-” “curl/7.64.0” “-”

What’s the output from ls -lah /usr/share/nginx/html/ inside that container?

Same as you got when you ran the command

Oh, I missed your 2nd post. Looks like you don’t have an index file.

i do have .html files inside the folders test & test1

Your screenshot shows mysql, test, and test1 inside of /usr/share/nginx/html.

I don’t see an index file in /usr/share/nginx/html.

Hi @protosam , Thank you for the reply . FYI : I am using NFS Shared volume between Minio & Nginx . Nginx (/usr/share/nginx/html) will have minio buckets & respective files(files are inside the test folder & test 1folder )
Nginx should show those files on it servers but it throws 403 forbidden .

I hope this give u clarification on my scenario

Here’s a demo of how the nginx pod works. I started off with a plain nginx pod, replicated the 403 Forbidden error, and fixed it. Hope this helps clarify. :slight_smile:

protosam@github-storage $ cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: nginx-demo
spec:
  containers:
  - name: nginx
    image: nginx:latest
EOF
pod/nginx-demo created

protosam@github-storage $ kubectl exec -it nginx-demo -- bash
root@nginx-demo:/# curl localhost
    <!DOCTYPE html>
    <html>
    <head>
    <title>Welcome to nginx!</title>
    <style>
        body {
            width: 35em;
            margin: 0 auto;
            font-family: Tahoma, Verdana, Arial, sans-serif;
        }
    </style>
    </head>
    <body>
    <h1>Welcome to nginx!</h1>
    <p>If you see this page, the nginx web server is successfully installed and
    working. Further configuration is required.</p>

    <p>For online documentation and support please refer to
    <a href="http://nginx.org/">nginx.org</a>.<br/>
    Commercial support is available at
    <a href="http://nginx.com/">nginx.com</a>.</p>

    <p><em>Thank you for using nginx.</em></p>
    </body>
    </html>

root@nginx-demo:/# cd /usr/share/nginx/html

root@nginx-demo:/usr/share/nginx/html# ls -lah
    total 16K
    drwxr-xr-x 2 root root 4.0K May 25 15:43 .
    drwxr-xr-x 3 root root 4.0K May 25 15:43 ..
    -rw-r--r-- 1 root root  494 May 25 12:28 50x.html
    -rw-r--r-- 1 root root  612 May 25 12:28 index.html

root@nginx-demo:/usr/share/nginx/html# mv index.html removed.html

root@nginx-demo:/usr/share/nginx/html# curl localhost
    <html>
    <head><title>403 Forbidden</title></head>
    <body>
    <center><h1>403 Forbidden</h1></center>
    <hr><center>nginx/1.21.0</center>
    </body>
    </html>

root@nginx-demo:/usr/share/nginx/html# mv removed.html index.html

root@nginx-demo:/usr/share/nginx/html# ls -lah
    total 24K
    drwxr-xr-x 1 root root 4.0K Jun 10 19:29 .
    drwxr-xr-x 1 root root 4.0K May 25 15:43 ..
    -rw-r--r-- 1 root root  494 May 25 12:28 50x.html
    -rw-r--r-- 1 root root  612 May 25 12:28 removed.html

root@nginx-demo:/usr/share/nginx/html# curl localhost
    <!DOCTYPE html>
    <html>
    <head>
    <title>Welcome to nginx!</title>
    <style>
        body {
            width: 35em;
            margin: 0 auto;
            font-family: Tahoma, Verdana, Arial, sans-serif;
        }
    </style>
    </head>
    <body>
    <h1>Welcome to nginx!</h1>
    <p>If you see this page, the nginx web server is successfully installed and
    working. Further configuration is required.</p>

    <p>For online documentation and support please refer to
    <a href="http://nginx.org/">nginx.org</a>.<br/>
    Commercial support is available at
    <a href="http://nginx.com/">nginx.com</a>.</p>

    <p><em>Thank you for using nginx.</em></p>
    </body>
    </html>

root@nginx-demo:/usr/share/nginx/html# exit
    command terminated with exit code 130

protosam@github-storage $ kubectl delete pod nginx-demo
    pod "nginx-demo" deleted