Upstream prematurely closed connection while reading response header from upstream

General Discussions
1

Hello!

Cluster:
“gcr.io/google-containers/hyperkube-amd64:v1.11.3”
“KernelVersion”: “4.4.0-135-generic”,
“OSImage”: “Ubuntu 16.04.5 LTS”,
“ContainerRuntimeVersion”: “docker://17.3.2”,
“KubeletVersion”: “v1.11.3”,
“KubeProxyVersion”: “v1.11.3”,

Question:
We have external nginx proxy. And we tested our site from different ips. The whole http trafic send to nginx proxy and proxy send trafic to kubernetes NodePort(we do not use ingress). Proxy has upstream that point to 2 ips kube master. And we send about 300 requests to proxy in seccond and in logs proxy i see next messages: “upstream prematurely closed connection while reading response header from upstream”.
Can be that kube iptables blocked trafic from nginx proxy ?
Where i can see logs the my problem ?

Thanks.

1 Like
2

The problem is (most often) related to timeouts. Especially if your application on the backend isn’t able to handle requests in an orderly fashion. I’d first increase your timeouts using something as simple as:

location / {
    proxy_read_timeout 300;
    proxy_connect_timeout 300;
    proxy_pass http://localhost:3000;
}

If you’re still having issues please provide your nginx configuration(s).

Good luck!

3

Thank you.
My proxy settings.
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_intercept_errors on;
proxy_max_temp_file_size 0;
proxy_connect_timeout 300;
proxy_send_timeout 120;
proxy_read_timeout 1800s;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_temp_path /tmp 1 2;
proxy_ignore_headers Set-Cookie;
proxy_cache off;
proxy_cache_lock on;
proxy_cache_lock_age 5s;
proxy_cache_lock_timeout 5s;
proxy_cache_methods GET HEAD;
proxy_cache_min_uses 1;
proxy_cache_revalidate on;
proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
proxy_pass_header Set-Cookie;
proxy_cache_valid 200 10m;
#proxy_cache_background_update on;

4

hii did this problem fix I too have same error
. If it is fixed can you let me know the solution

5

resolved by adding this annotation nginx.ingress.kubernetes.io/bckend-protocol: “HTTPS”