Traefik, Mattermost, and Docker (Oh My....)

Hello everyone!

I am calling upon the wizards to retrieve some assistance with Traefik, Mattermost, Docker, and getting some ports exposed. I currently have a working instance of Jellyfin that is reachable over the Internet (locked down to my home IP address only right now) and confirms to be functioning. The docker-compose.yml file is below for review.

```version: "2.1"
services:
  jellyfin:
    image: lscr.io/linuxserver/jellyfin:latest
    container_name: jellyfin
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=America/New_York
    networks:
      - proxy
    volumes:
      - /mnt/HDD_01/Jellyfin:/config
      - /mnt/HDD_01/:/data
        # ports:
        # - 8096:8096
        # - 8920:8920
        # - 7359:7359/udp
        # - 1900:1900/udp
    restart: unless-stopped
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.jellyfin.entrypoints=http"
      - "traefik.http.routers.jellyfin.rule=Host(`REDACTED`)"
      - "traefik.http.middlewares.jellyfin-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.jellyfin.middlewares=portainer-https-redirect"
      - "traefik.http.routers.jellyfin-secure.entrypoints=https"
      - "traefik.http.routers.jellyfin-secure.rule=Host(`REDACTED`)"
      - "traefik.http.routers.jellyfin-secure.tls=true"
      - "traefik.http.routers.jellyfin-secure.service=jellyfin-svc"
      - "traefik.http.services.jellyfin-svc.loadbalancer.server.port=8096"
      - "traefik.docker.network=proxy"
networks:
  proxy:
    external: true

I have ZERO issues with this container at this time, so I believe it is set up in the ideal manner. However, I am trying to spin up a Mattermost server as well and I cannot figure out why it won’t work. Below is the docker-compose.yml file, and if further explanation is needed please let me know.

I will say this, the issue with the Mattermost docker container is that even though it launches, is accessible internally, it refuses to forward the correct port out to my Cloudflare instance. Thank you in advance!!


version: "2.4"

services:
  postgres:
    image: postgres:${POSTGRES_IMAGE_TAG}
    restart: ${RESTART_POLICY}
    security_opt:
      - no-new-privileges:true
    pids_limit: 100
    read_only: true
    networks:
      - proxy
    tmpfs:
      - /tmp
      - /var/run/postgresql
    volumes:
      - ${POSTGRES_DATA_PATH}:/var/lib/postgresql/data
    environment:
      # timezone inside container
      - TZ

      # necessary Postgres options/variables
      - POSTGRES_USER
      - POSTGRES_PASSWORD
      - POSTGRES_DB

  mattermost:
    depends_on:
      - postgres
    image: mattermost/${MATTERMOST_IMAGE}:${MATTERMOST_IMAGE_TAG}
    restart: ${RESTART_POLICY}
    security_opt:
      - no-new-privileges:true
    pids_limit: 200
    read_only: ${MATTERMOST_CONTAINER_READONLY}
    networks:
      - proxy
    tmpfs:
      - /tmp
    volumes:
      - ${MATTERMOST_CONFIG_PATH}:/mattermost/config:rw
      - ${MATTERMOST_DATA_PATH}:/mattermost/data:rw
      - ${MATTERMOST_LOGS_PATH}:/mattermost/logs:rw
      - ${MATTERMOST_PLUGINS_PATH}:/mattermost/plugins:rw
      - ${MATTERMOST_CLIENT_PLUGINS_PATH}:/mattermost/client/plugins:rw
      - ${MATTERMOST_BLEVE_INDEXES_PATH}:/mattermost/bleve-indexes:rw
      # When you want to use SSO with GitLab, you have to add the cert pki chain of GitLab inside Alpine
      # to avoid Token request failed: certificate signed by unknown authority
      # (link: https://github.com/mattermost/mattermost-server/issues/13059 and https://github.com/mattermost/docker/issues/34)
      # - ${GITLAB_PKI_CHAIN_PATH}:/etc/ssl/certs/pki_chain.pem:ro
    environment:
      # timezone inside container
      - TZ

      # necessary Mattermost options/variables (see env.example)
      - MM_SQLSETTINGS_DRIVERNAME
      - MM_SQLSETTINGS_DATASOURCE

      # necessary for bleve
      - MM_BLEVESETTINGS_INDEXDIR

      # additional settings
      - MM_SERVICESETTINGS_SITEURL
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.mattermost.entrypoints=http"
      - "traefik.http.routers.mattermost.rule=Host(`REDACTED`)"
      - "traefik.http.middlewares.mattermost-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.mattermost.middlewares=portainer-https-redirect"
      - "traefik.http.routers.mattermost-secure.entrypoints=https"
      - "traefik.http.routers.mattermsot-secure.rule=Host(`REDACTED`)"
      - "traefik.http.routers.mattermost-secure.tls=true"
      - "traefik.http.routers.mattermost-secure.service=mattermost-svc"
      - "traefik.http.services.mattermost-svc.loadbalancer.server.port=8065"
      - "traefik.docker.network=proxy"
networks:
  proxy:
    external: true

# If you use rolling image tags and feel lucky watchtower can automatically pull new images and
# instantiate containers from it. https://containrrr.dev/watchtower/
# Please keep in mind watchtower will have access on the docker socket. This can be a security risk.
#
#  watchtower:
#    container_name: watchtower
#    image: containrrr/watchtower:latest
#    restart: unless-stopped
#    volumes:
#      - /var/run/docker.sock:/var/run/docker.sock```

By, “accessible internally” do you mean you can hit the container port for mattermost, or you can hit the port for traefik and get to mattermost?

I can access the port of the container so I know the service has spun up. I apologize for the lack of information on that!

Check the traefik logs and dashboard. Logs might not be enabled by default, in which case you can enable them in the settings.

Ahh yeah, this is the thing about traefik that I just hate. It’s such a pain to get working correctly.

I had so much difficulty with it that I just wound up giving up and switching to boring nginx, when I was using docker.

There does not seem to anything obviously wrong with traeffic labels in mattermost strackfile ( ie. lb port is correct), what is the actual observed issue?

I.e what do you see when trying to access mattermost via set host url? What does the mattermost container emit to logs during the attempt?

Key elements to verify by observation:

  • is the mattermost stack functioning at all? Verify if service is accesible by manual port exposure, before troubleshooting traeffic (mattermost and postgresql container logs
  • are requests to <mattermost_uri> correctly forwarded from host network via proxy network to mattermost container? (traefic container logs and mattermost container logs)

I’ll mark sure to do so @quilt!

I am finding that out, but thankfully, it’s not SUPER bad so far, I just need to get this portion figured out. I’ll see if some further logs will help.

@greatnull It basically says that there is no running service on that port, it puts up a 404 error. However, when I actually visit my internal IP address with the port (8065) it loads up just fine.

Well I am sad (happy?) to say that it was all due to a typo, yikes… I appreciate the suggestions that everyone had!

All right, so new issue now! I am able to successfully caonnect to my Mattermost server through the domain name, etc. However, I am receiving the below error when trying to start a call.

msg=Failed to start SCTP: DTLS not established plugin_id=com.mattermost.calls origin=webrtc/v3.(*PeerConnection).startSCTP github.com/pion/webrtc/[email protected]/peerconnection.go:1472

If I try to start the call from the iOS app I receive the following: Error unable to connect to the voice call: timed out waiting for peer connection

I do not receive anything in the logs on my server meaning that it appears the app is not even communicating with the call server.

Any thoughts are appreciated!

mattermsot vs. mattermost ? Insidious and happens to best of us. I did succesfully take down test service by service name collision by mistake too.

      - "traefik.http.routers.mattermost.middlewares=portainer-https-redirect"
      - "traefik.http.routers.mattermost-secure.entrypoints=https"
      - "traefik.http.routers.mattermsot-secure.rule=Host(`REDACTED`)"
      - "traefik.http.routers.mattermost-secure.tls=true"
      - "traefik.http.routers.mattermost-secure.service=mattermost-svc"

Check what ports you actually need, you just exposed internal port 8065 to external 443 via set hostname.

If webrtc needs to establish connection via multiple ports, then it will fail. Documentation points that way. You need to able to communicate also over UDP/8443. Easiest way would be direct port exposure at docker host.


https://docs.mattermost.com/configure/calls-deployment.html

Otherwise you can decouple rtc handling from mattermost container by deploying rtcd per documentation. This might be also solution.

2 Likes