Truenas Scale, Nextcloud App, Nginx Reverse Proxy, External FQDN No Workie

Hey All,

Posting here because I am afraid of the Truenas forums.

I am running TrueNas Scale Beta 2 with Nextcloud running as an app (container) with a virtualized Ubuntu VM running Nginix to reverse proxy external WAN traffic back into Nextcloud. I had this working in ESXi but have since moved it all to TrueNas. It is not the weird VM’s cannot talk to the host or other VM’s thing in TrueNas, I fixed that by using multiple NICs and gateways, but I cannot get the reverse proxy to work.

I get this error:

502 Bad Gateway

nginx/1.18.0 (Ubuntu)

I know I haven’t given enough info, I have made a slew of changes in Nginx and Nextcloud to no avail, it’s like Nextcloud is refusing the proxy requests.

Has anyone done this successfully? Anyone feel like helping a noob out here?

Thanks,
Ben

Post your nextcloud/config/config.php. (sans secrets of course)

Specifically, interested in the trusted_domains section. It should have at least two entries for your local subnet where the server lives and the fqdn of your public url.

E.g:

'trusted_domains' =>
  array (
    0 => '192.168.1.*',
    1 => 'cloud.fqdn.com',
  ),

Also please post your nginx config as well.

And when you do a request, please post the tail -n 50 /var/log/nginx/error.log

Thanks! I put the info into separately named files. Looking forward to knowing how I’ve messed this up! :slight_smile:

NextCloudConfig.txt (1.2 KB)
nginxSitesAvailable.txt (1.3 KB)
NginxTail.txt (16.1 KB)

Config file looked fine.

For your sites available you should remove the proxy_pass directive from your server block hosting on port 80. (the second one)

Your reverse proxy log says that it fails with the SSL handshake to the upstream server (nextcloud).

Your nginx proxy is trying to connect over https to your 10.0.0.5 on port 9001. If you don’t have an internal SSL certificate (self-signed is fine), then you’re connection is being refused because you’re probably running your nextcloud only on http.

So you can either:

Once you do either option, just issue an sudo nginx -s reload and then your nextcloud service should work.

Thanks, that makes sense.

I plan on the SSL Cert method, once I figure out how! :slight_smile:

However, in the meantime, of course I want to try option 2.

I made the change below, as you can see.

Ran the sudo nginx -s reload

But I get the same thing. Did I misinterpret what you instructed?

server {
        server_name cloud.supervarelas.com;

        access_log /var/log/nginx/reverse-access.log;
        error_log /var/log/nginx/reverse-error.log;

        location / {
                    proxy_pass http://10.0.0.5:9001;
                    # re-send the host header - this may not be necessary
                    proxy_set_header Host $host;
                    # set the X-Forwarded-For header, so that the public IP of the client is available to the backend server
                    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/cloud.supervarelas.com-0002/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/cloud.supervarelas.com-0002/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}
server {
    if ($host = cloud.supervarelas.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


        listen 80;
       server_name cloud.supervarelas.com;
       location / {
                  #proxy_pass http://10.0.0.5:9001;
       }
     return 404; # managed by Certbot```

This

location / {
                  #proxy_pass http://10.0.0.5:9001;
       }

Can be entirely commented out. The whole block. Because at the top it runs an ‘if’ and never executes the below statement. However, for completeness/tidiness is why I recommended it.

But yes, other than that you look good and it should work now. So long as you don’t have port 9001 blocked by a firewall on that host.


Do this for me. If you get a successful response you should see a successful connection and the output of a web page.

curl http://10.0.0.5:9001

Is this helpful?

PS C:\Windows\system32> curl http://10.0.0.5:9001                                                                       

StatusCode        : 200
StatusDescription : OK
Content           : <!DOCTYPE html>
                    <html class="ng-csp" data-placeholder-focus="false" lang="en" data-locale="en" >
                        <head
                     data-requesttoken="MsY9oOd9xxkdL7lbkDEzD8SvN/sZc/4oO+dIwuj5i24=:Xqp7lIQV81xpYdQy/GcYeJHlT9AoAIw...
RawContent        : HTTP/1.1 200 OK
                    Referrer-Policy: no-referrer
                    X-Content-Type-Options: nosniff
                    X-Download-Options: noopen
                    X-Frame-Options: SAMEORIGIN
                    X-Permitted-Cross-Domain-Policies: none
                    X-Robots-Tag: none
                    X-...
Forms             : {}
Headers           : {[Referrer-Policy, no-referrer], [X-Content-Type-Options, nosniff], [X-Download-Options, noopen],
                    [X-Frame-Options, SAMEORIGIN]...}
Images            : {}
InputFields       : {@{innerHTML=; innerText=; outerHTML=<input id="initial-state-core-loginUsername" type="hidden"
                    value="IiI=">; outerText=; tagName=INPUT; id=initial-state-core-loginUsername; type=hidden;
                    value=IiI=}, @{innerHTML=; innerText=; outerHTML=<input id="initial-state-core-loginAutocomplete"
                    type="hidden" value="dHJ1ZQ==">; outerText=; tagName=INPUT;
                    id=initial-state-core-loginAutocomplete; type=hidden; value=dHJ1ZQ==}, @{innerHTML=; innerText=;
                    outerHTML=<input id="initial-state-core-loginThrottleDelay" type="hidden" value="MA==">;
                    outerText=; tagName=INPUT; id=initial-state-core-loginThrottleDelay; type=hidden; value=MA==},
                    @{innerHTML=; innerText=; outerHTML=<input id="initial-state-core-loginResetPasswordLink"
                    type="hidden" value="IiI=">; outerText=; tagName=INPUT;
                    id=initial-state-core-loginResetPasswordLink; type=hidden; value=IiI=}...}
Links             : {@{innerHTML=Nextcloud; innerText=Nextcloud; outerHTML=<a class="entity-name"
                    href="https://nextcloud.com" target="_blank" rel="noreferrer noopener">Nextcloud</a>;
                    outerText=Nextcloud; tagName=A; class=entity-name; href=https://nextcloud.com; target=_blank;
                    rel=noreferrer noopener}}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 11880```

yup status code is 200: success.

Which proves that the issue is somewhere between your nginx proxy host talking to your web server host.

Could you tail that nginx-reverse log again?

Below is the latest log.

For clarity on my part, when you mention web host is it like this:

[Client] <–> [My Router] <–> [Nginx] <–> [Web Server/Host/TrueNas] <–> [Docker/NextCloud] ?

bvdrax@nginix:/etc/nginx/sites-available$ sudo tail -50 /var/log/nginx/reverse-error.log
2021/12/14 21:24:24 [error] 1182#1182: *231 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:24:56 [error] 1182#1182: *233 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:25:03 [error] 1182#1182: *235 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:25:28 [error] 1182#1182: *237 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:25:39 [error] 1182#1182: *239 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET / HTTP/1.1", upstream: "http://10.0.0.5:9001/", host: "cloud.supervarelas.com"
2021/12/14 21:26:00 [error] 1182#1182: *241 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:26:05 [error] 1182#1182: *243 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:26:32 [error] 1182#1182: *245 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:27:04 [error] 1182#1182: *247 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:27:07 [error] 1182#1182: *249 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:27:36 [error] 1182#1182: *251 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:28:08 [error] 1182#1182: *253 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:28:09 [error] 1182#1182: *255 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:28:40 [error] 1182#1182: *257 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:29:11 [error] 1182#1182: *259 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:29:12 [error] 1182#1182: *261 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:29:44 [error] 1182#1182: *263 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:30:13 [error] 1182#1182: *265 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:30:16 [error] 1182#1182: *267 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:30:48 [error] 1182#1182: *269 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:31:15 [error] 1182#1182: *271 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:31:20 [error] 1182#1182: *273 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:31:52 [error] 1182#1182: *275 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:32:17 [error] 1182#1182: *277 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:32:24 [error] 1182#1182: *279 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:32:56 [error] 1182#1182: *281 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:33:19 [error] 1182#1182: *283 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:33:28 [error] 1182#1182: *285 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:34:00 [error] 1182#1182: *287 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:34:21 [error] 1182#1182: *289 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:34:32 [error] 1182#1182: *291 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:35:04 [error] 1182#1182: *293 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:35:23 [error] 1182#1182: *295 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:35:36 [error] 1182#1182: *297 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:36:08 [error] 1182#1182: *299 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:36:25 [error] 1182#1182: *301 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:36:40 [error] 1182#1182: *303 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:37:12 [error] 1182#1182: *305 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:37:27 [error] 1182#1182: *307 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:37:44 [error] 1182#1182: *309 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:38:15 [error] 1182#1182: *311 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET / HTTP/1.1", upstream: "http://10.0.0.5:9001/", host: "cloud.supervarelas.com"
2021/12/14 21:38:16 [error] 1182#1182: *311 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /favicon.ico HTTP/1.1", upstream: "http://10.0.0.5:9001/favicon.ico", host: "cloud.supervarelas.com", referrer: "https://cloud.supervarelas.com/"
2021/12/14 21:38:16 [error] 1182#1182: *314 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:38:18 [error] 1182#1182: *311 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /index.php/apps/photos/service-worker.js HTTP/1.1", upstream: "http://10.0.0.5:9001/index.php/apps/photos/service-worker.js", host: "cloud.supervarelas.com", referrer: "https://cloud.supervarelas.com/index.php/apps/photos/service-worker.js"
2021/12/14 21:38:29 [error] 1182#1182: *317 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:38:47 [error] 1182#1182: *311 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /index.php/apps/photos/service-worker.js HTTP/1.1", upstream: "http://10.0.0.5:9001/index.php/apps/photos/service-worker.js", host: "cloud.supervarelas.com", referrer: "https://cloud.supervarelas.com/index.php/apps/photos/service-worker.js"
2021/12/14 21:38:48 [error] 1182#1182: *320 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:39:20 [error] 1182#1182: *322 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:39:31 [error] 1182#1182: *324 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.0.0.1, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"
2021/12/14 21:39:52 [error] 1182#1182: *326 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 23.121.225.73, server: cloud.supervarelas.com, request: "GET /status.php HTTP/1.1", upstream: "http://10.0.0.5:9001/status.php", host: "cloud.supervarelas.com"

Dynamic_Gravity, thanks for your help on this. I looked into how to do the certificates piece, and how I was attempting to do this was all wrong. I have been playing with this for months but your suggestions gave me the nudge I needed to figure this out.

Just in case anyone else runs into this, if you are running an application in TrueNas scale and want to reverse proxy external traffic into that application you need to use a certificate on the TrueNas host, Traefic and follow some setup. 07 - Adding Lets-Encrypt Certificates - TrueCharts

Thanks,
Ben

1 Like

Glad you got it working!

Congrats on figuring it out.

Enjoy your nextcloud. :grin: