Return to Level1Techs.com

Netbox IPAM / DCIM guide

linux
tutorial
#1

Part I - Netbox

Introduction

This document is a guide to building a Netbox IPAM / DCIM server to hold information about the state of your network. I found Netbox after looking for a FOSS IPAM solution to replace three horrible spreadsheets that were used where I work for the following purposes.

  1. VLAN / IP address / switch port allocation
  2. Physical and VM server details
  3. Rack plans

The website for the application is here https://netbox.readthedocs.io/en/latest/

So why am I writing a guide for this when you can just go to the link above? While I have nothing but admiration and gratitude for the creator of this awesome piece of software I do feel that the documentation could be a little more expansive. Also this guide goes into additional into areas not covered in the documentation.

Netbox is not a network scanner. It is how your network should be, not how it is.

Requirements

  1. A virtual Linux x86 server
  2. A static IP address
  3. An internet connection
  4. External NAS storage for backups

Optional

  1. A second virtual Linux x86 server as a replica

In this guide I am using a virtual server as the requirements are so low. You can use a single vCPU, 1GB of RAM and 8GB of disk space. As it is I went with the following specifications on Ubuntu 16.04 LTS

2 vCPU
2GB RAM
16GB hard disk

If you want to complete the optional section you will of course need a second server.

Objective

The objective is to be store all the relevant information about your network infrastructure, devices and virtual machines in a dedicated built for purpose application.

Assumptions

  1. You possess a separate internet connected computer
  2. You possess hardware capable of running virtual machines 24/7
  3. You know how to setup Ubuntu 16.04 as a VM and access it via SSH or PuTTY

Commands, configuration files and examples

Just for clarity terminal commands will be in bold.

sudo apt-get upgrade

While configuration files will be shown as code blocks.

insert your configuration here

The local user on the server is ‘netbox’ and is a member of the sudo’ers group.

The example domain being used is ‘planetexpress.net

The first server hostname is ‘netbox-primary’ and has an IP of 192.168.0.20

The second server hostname is 'netbox-secondary and has an IP of 192.168.0.21

3 Likes

#2

Part II - Netbox install

So please SSH into your brand spanking new server and apply all pending updates. Then also install htop, tree and nfs-common with

sudo apt-get install htop tree nfs-common

PostgreSQL

Netbox uses PostgreSQL as it requires some features which exists in this database which do not exist in MySQL / MariaDB.

sudo apt-get install -y postgresql libpq-dev

Once the install is complete please login to PostgreSQL with the following command

sudo -u postgres psql

We now need to create a blank database, create the netbox PostgreSQL user and grant privileges.

Enter the commands as show after postgres=# I just left that in their to indicate that we are operating inside PostgreSQL.

postgres=# CREATE DATABASE netbox;

postgres=# CREATE USER netbox WITH PASSWORD ‘YOUR-DB-PASSWORD’;

postgres=# GRANT ALL PRIVILEGES ON DATABASE netbox TO netbox;

postgres=# \q

Now we can test that the PostgreSQL user can access the netbox DB

Enter the below command to login to the netbox DB.

psql -U netbox -W -h localhost netbox

You should see

Enter the below command to exit PostgreSQL.

\q

Netbox software prerequisites

Now we install some of the Netbox prerequisites with the following commands

sudo apt-get install -y python3 python3-dev python3-setuptools build-essential libxml2-dev libxslt1-dev libffi-dev graphviz libpq-dev libssl-dev zlib1g-dev

sudo easy_install3 pip

Download Netbox

Please check you still in your home directory with

pwd

You should see

/home/netbox

Now download the application with the following command. At the time of writing this is v2.4.4. Adjust as needed for future versions.

wget https://github.com/digitalocean/netbox/archive/v2.4.4.tar.gz --no-check-certificate

The --no-check-certificate switch is useful if you are behind a proxy. If you have no proxy you can leave it off.

Now unpack the application to /opt with the below command. Adjust as needed for future versions.

sudo tar -xzf v2.4.4.tar.gz -C /opt

Now we need to create a softlink called netbox pointing at the program directory inside /opt

This is essential to make seemless upgrades possible later down the line.

cd /opt

sudo ln -s netbox-2.4.4 netbox

Now double check the folder structure using tree

tree -dL 3

It should look like this.

Python prerequisites

Now we have to install the python prerequisites using the included requirements.txt file

To make sure it is present enter the floowing command

cat /opt/netbox/requirements.txt

At the time of writing it should be identical to the file shown here https://github.com/digitalocean/netbox/blob/develop/requirements.txt

Now install the python prerequisites with the command below.

sudo pip3 install -r /opt/netbox/requirements.txt

Configuration files

Now we move onto making some changes to the applications configuration files. Make a production copy of the example configuration file.

sudo cp /opt/netbox/netbox/netbox/configuration.example.py /opt/netbox/netbox/netbox/configuration.py

Generate a secret key

Now we need to create a secret key which will go in the file we just created. These is used for encrypting secrets in the application.

sudo python3 /opt/netbox/netbox/generate_secret_key.py

Make a note of the output and store in a safe place.

Edit the configuration.py file with the following command.

sudo nano /opt/netbox/netbox/netbox/configuration.py

Change the fields below to match your installation. The IP address, NETBIOS & FQDN names are those of your server.

ALLOWED_HOSTS = ['192.168.0.20', 'netbox-primary', 'netbox-primary.planetexpress.net']

DATABASE = {
    'NAME': 'netbox',               # Database name
    'USER': 'netbox',               # PostgreSQL username
    'PASSWORD': 'YOUR-DB-PASSWORD', # PostgreSQL password
    'HOST': 'localhost',            # Database server
    'PORT': '',                     # Database port (leave blank for default)
}

SECRET_KEY = 'EnterTheSecretKeyYouJustGenerated'

All other sections in the configuration.py file are optional for the operation of the server although you may well want to consider changing the following from ‘False’ to ‘True’

LOGIN_REQUIRED = False

If it is left as false you don’t need a login to be able to see all the data via the website. All you need is the URL.

Create the DB schema

Create the application DB schema with the following command

sudo python3 /opt/netbox/netbox/manage.py migrate

Now we create the initial super user you will need to login to the netbox application via the web GUI.

sudo python3 /opt/netbox/netbox/manage.py createsuperuser

Follow the on screen prompts.

Now we need to load some files into the application.

sudo python3 /opt/netbox/netbox/manage.py collectstatic --no-input

And now load some initial example objects. This is an optional step.

sudo python3 /opt/netbox/netbox/manage.py loaddata initial_data

Create testing instance

You can now test your work so far by creating a test instance.

sudo python3 /opt/netbox/netbox/manage.py runserver 0.0.0.0:8000 --insecure

You should see the following output

195095442

and you can access the test instance with the URL provided once you substitute your own servers IP address.

Install the web server

Now we need to provide a premenant web server to run the application.

sudo apt-get install -y nginx

Now we create a site configuration file with

sudo nano /etc/nginx/sites-available/netbox

and paste the following into it, being sure to amend the server_name

server {
    listen 80;

    server_name netbox-primary.planetexpress.net;

    client_max_body_size 25m;

    location /static/ {
        alias /opt/netbox/netbox/static/;
    }

    location / {
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header X-Forwarded-Host $server_name;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
    }
}

Now remove the default config file

sudo rm /etc/nginx/sites-enabled/default

And create a soft link to your site

cd /etc/nginx/sites-enabled/

sudo ln -s /etc/nginx/sites-available/netbox netbox

Now restart Nginx with

sudo service nginx restart

Install Gunicorn

According to Wikipedia, “Gunicorn” is “Green Unicorn” a Python Web Server Gateway Interface (WSGI) HTTP server. The Gunicorn server is broadly compatible with a number of web frameworks, simply implemented, light on server resources and fairly fast.

sudo pip3 install gunicorn

Create and edit the gunicorn configuration file with

sudo nano /opt/netbox/netbox/gunicorn_config.py

Paste in the text below, then save and exit the file

command = '/usr/local/bin/gunicorn'
pythonpath = '/opt/netbox/netbox'
bind = '127.0.0.1:8001'
workers = 3
user = 'www-data'

Make the file executeable with

sudo chmod +x /opt/netbox/netbox/gunicorn_config.py

Install supervisor

Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.

sudo apt-get install -y supervisor

Now create the supervisor configuration file

sudo nano /etc/supervisor/conf.d/netbox.conf

Paste in the text below, then save and exit the file

[program:netbox]
command = gunicorn -c /opt/netbox/netbox/gunicorn_config.py netbox.wsgi
directory = /opt/netbox/
user = www-data

sudo service supervisor restart

Make sure your permissions are correct

sudo chown -R netbox:netbox /opt/netbox-2.4.4

sudo chmod -R 775 /opt/netbox-2.4.4

Restart Nginx and supervisor

sudo service nginx restart

sudo supervisorctl restart netbox

Done

If all is well you should be able to navigate to your application.

Primary

Secondary

Configure LDAP (optional)

Netbox can be configured to authenticate users via LDAP. Most commonly of course this will be Microsoft Active Directory.

First you need to install some more software pre-requisites

sudo apt-get install -y python-dev libldap2-dev libsasl2-dev libssl-dev

sudo pip install django-auth-ldap

Now you need to create an additional configuration file

sudo nano /opt/netbox/netbox/netbox/ldap_config.py

Paste in the following, make changes to reflect the domain, users and groups you are using then exit and save the file.

import ldap

AUTH_LDAP_SERVER_URI = "ldap://mydomain.local"
AUTH_LDAP_CONNECTION_OPTIONS = {
            ldap.OPT_REFERRALS: 0
            }
AUTH_LDAP_BIND_DN = "CN=svc-netbox,OU=Service Accounts,DC=mydomain,DC=local"
AUTH_LDAP_BIND_PASSWORD = "PASSWORDFORACCOUNTABOVE"
LDAP_IGNORE_CERT_ERRORS = True

from django_auth_ldap.config import LDAPSearch

AUTH_LDAP_USER_SEARCH = LDAPSearch("OU=Admin Accounts,OU=Infrastructure Admin,DC=mydomain,DC=local",
                                                    ldap.SCOPE_SUBTREE,
                                                    "(sAMAccountName=%(user)s)")

# You can map user attributes to Django attributes as so
AUTH_LDAP_USER_ATTR_MAP = {
           "first_name": "givenName",
           "last_name": "sn",
           "email": "mail"
                        }

from django_auth_ldap.config import LDAPSearch, NestedGroupOfNamesType

AUTH_LDAP_GROUP_SEARCH = LDAPSearch("DC=mydomain,DC=local", ldap.SCOPE_SUBTREE, "(objectClass=group)")

AUTH_LDAP_GROUP_TYPE = NestedGroupOfNamesType()

AUTH_LDAP_REQUIRE_GROUP = "CN=Admin-Netbox,OU=Admin Groups,OU=Infrastructure Admin,DC=mydomain,DC=local"

# Define special user types using groups. Exercise great caution when assigning superuser status.
AUTH_LDAP_USER_FLAGS_BY_GROUP = {
    "is_active": "CN=Active-Netbox,OU=Admin Groups,OU=Infrastructure Admin,DC=mydomain,DC=local",
    "is_staff": "CN=Staff-Netbox,OU=Admin Groups,OU=Infrastructure Admin,DC=mydomain,DC=local",
    "is_superuser": "CN=SU-Netbox,OU=Admin Groups,OU=Infrastructure Admin,DC=mydomain,DC=local"
}

AUTH_LDAP_FIND_GROUP_PERMS = True
AUTH_LDAP_CACHE_GROUPS = True
AUTH_LDAP_GROUP_CACHE_TIMEOUT = 3600

Lastly correct the permissions of the file.

sudo chown netbox:netbox ldap_config.py

sudo chmod +x ldap_config.py

sudo chmod 775 ldap_config.py

sudo supervisorctl restart netbox

3 Likes

#3

Part III - Netbox backup & restore

Backup

The data you are going to be putting into this application is very sensitive so you better make sure you have good backups.

The below script backups up the Netbox installation directory, database and nginx configuration file to a NAS.

This script assumes the remote share on the NAS is already mounted.

Cretae the script with the following commands

sudo su

cd /root

nano netbox-backup

Now copy in the script text below. save and quit the file.

#!/bin/bash

# Backup netbox, compress and copy
# Backup postgressql, compress and copy
# Copy Nginx netbox configuration file
# Remove files from /tmp
# Remove backups older than 28 days

tar -chzf /tmp/"netbox.$(date +%F).tar.gz" -C / opt/netbox
pg_dump -h localhost -U netbox -Fc netbox > /tmp/"netbox.$(date +%F).dump"
pg_dump -h localhost -U netbox -Fc netbox > /tmp/netbox.restore.dump
rsync /tmp/"netbox.$(date +%F).tar.gz" /mnt/FreeNAS/Backups/netbox-primary/
rsync /tmp/"netbox.$(date +%F).dump" /mnt/FreeNAS/Backups/netbox-primary/
rsync /etc/nginx/sites-available/netbox /mnt/FreeNAS/Backups/netbox-primary
rm /tmp/"netbox.$(date +%F).tar.gz"
rm /tmp/"netbox.$(date +%F).dump"
find /mnt/FreeNAS/Backups/netbox-primary/* -mtime +28 -exec rm {} \;

Make the script executable.

chmod +x netbox-backup

Be sure to create your /root/.pgpass file to allow the script to make the database backup.

The form of .pgpass is as follows

localhost:5432:netbox:netbox:DATABASEPASSWORD

The permissions on .pgpass are 0600 for the root account. The backup will not work if they are anything different.

1721645266

The script is run under the root account via cron at 03:00 and will remove any backups older than 28 days.

Restore

Generally if my production server went bang (and I had no Veeam or Rubrik VM level backups) I would spin up a new VM, run through the Netbox install process until I had a blank working system and then restore the database.

To restore the database backup to the same or a different server the commands are as follows.

sudo -u postgres psql

postgres=# DROP DATABASE netbox;

postgres=# CREATE DATABASE netbox;

postgres=# \q

pg_restore -C -d postgres /mnt/FreeNAS/Backups/netbox-primary/netbox-primary.2018-09-13.dump

Their are different commands you can use to accomplish this task but these seemed to work best for me.

Adjust the date in the path as needed.

###Update

You should also be able to restore the database with this command

pg_restore -c /tmp/netbox.restore.dump -U netbox -d netbox -h localhost

2 Likes

#4

Part IV - Netbox upgrade

In this example we are upgrading from version 2.4.3 to version 2.4.4

The initial working directory is /home/netbox

The actions are performed under the netbox account

wget https://github.com/digitalocean/netbox/archive/v2.4.4.tar.gz

sudo tar -xzf v2.4.4.tar.gz -C /opt

cd /opt

sudo ln -sfn netbox-2.4.4/ netbox

sudo cp /opt/netbox-2.4.3/netbox/netbox/configuration.py /opt/netbox/netbox/netbox/configuration.py

sudo cp /opt/netbox-2.4.3/netbox/gunicorn_config.py /opt/netbox/netbox/gunicorn_config.py

sudo cp /opt/netbox-2.4.3/netbox/netbox/ldap_config.py /opt/netbox/netbox/netbox/ldap_config.py

sudo /opt/netbox/upgrade.sh

sudo chown -R netbox:netbox /opt/netbox-2.4.4

sudo chmod -R 775 /opt/netbox-2.4.4/
 
sudo supervisorctl restart netbox

Troubleshooting

It is very easy to place files or directories at the wrong level because of the softlink and the creators use of multiple “netbox” directories on top of each other.

Errors will almost certainly be one of the three following things.

Configuration file at the wrong level.

Path inside a configuration file referring to a path or file at the wrong level.

Permissions to the netbox directory are wrong. The local netbox user should be the owner of the directory and all files within. All permissions should be set at 775.

2 Likes

#5

Part V - Netbox replica

The rule of thumb with data is that ‘It does not exist unless we have it in three places’.

So here we will have the data on the primary server, restored nightly on secondary server in a different physical location. We will also have the backups from both servers on seperate NAS devices in seperate locations from the servers.

netbox-primary on 192.168.0.20

netbox-secondary on 192.168.0.21

Run the following command on both servers.

cd /home/netbox

ssh-keygen -t rsa

Copy the contents of the /home/netbox/.ssh/id_rsa.pub created to the /home/netbox/.ssh/authorized_keys file on the other server. Repeat on both servers.

On netbox-secondary

Create the script with the following commands.

sudo su

cd /root

nano netbox-backup

Now copy in the script text below. Save and quit the file.

#!/bin/bash
scp [email protected]:/tmp/netbox.restore.dump /tmp/
pg_restore -c /tmp/netbox.restore.dump -U netbox -d netbox -h localhost

Make the script executable.

chmod +x netbox-backup

Be sure to create your /root/.pgpass file to allow the script to restore the database backup.

The form of .pgpass is as follows

localhost:5432:netbox:netbox:DATABASEPASSWORD

The permissions on .pgpass are 0600 for the root account. The backup will not work if they are anything different.

Schedule the script is run under the root account via cron at 03:30

Every morning your secondary server will have its DB restored from the primary. This method avoids two issues I found using hot standby database replication as the secondary server DB is read / write not read only. This means

Upgrading netbox is not a problem. Just do both servers on the same day.

You can set LOGIN_REQUIRED = True on both servers with no issue.

3 Likes