Personal Cloud #3 - Mailservers! Postfix, Dovecot, and OpenDKIM

Are you sick of $BigDataCorporation spying on your emails?

Let's set up our own mail server using Postfix, Dovecot, and OpenDKIM!

  • Postfix is the actual mail service itself. It receives mail and sends mail to and from external servers.
  • Dovecot is a mail connector essentially. It allows you to store and access mail from remote devices via IMAP and SMTP, like from your phone or laptop.
  • OpenDKIM is a mail signature verification system. Emails signed with DKIM tell the recipient that the email is genuine and originates from the real domain name. Google's Gmail will block all mail from unknown sources that do not have a DKIM signature, so we need to have this enabled.

-Initial Setup

First let's to install postfix, dovecot, and the MySQL connectors for them.

# apt install postfix postfix-mysql dovecot-core dovecot-imapd dovecot-lmtpd dovecot-mysql mysql-server

When postfix asks you, enter your domain name, and setup a root password for MySQL.

-MySQL

Go ahead and login to MySQL:
# mysql -u root -p

Now let’s create the MySQL databases.
This will create the mail database. Change "securepassword" to a secure password for the mail user to connect to the database:

mysql> create database mail;
mysql> grant select on mail.* to 'mail'@'localhost' identified by 'securepassword';
mysql> flush privileges;
mysql> use mail;
mysql> CREATE TABLE `virtual_domains` (
`id`  INT NOT NULL AUTO_INCREMENT,
`name` VARCHAR(50) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

This will create the users database:

mysql> CREATE TABLE `virtual_users` (
`id` INT NOT NULL AUTO_INCREMENT,
`domain_id` INT NOT NULL,
`password` VARCHAR(106) NOT NULL,
`email` VARCHAR(120) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `email` (`email`),
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

This will create the aliases database:

mysql> CREATE TABLE `virtual_aliases` (
`id` INT NOT NULL AUTO_INCREMENT,
`domain_id` INT NOT NULL,
`source` varchar(100) NOT NULL,
`destination` varchar(100) NOT NULL,
PRIMARY KEY (`id`),
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Now we can create our first email address. In this next command make sure to change "somepassword" to a secure password for your email account, and change the "[email protected]" to your name with your domain name.

mysql> INSERT INTO `mail`.`virtual_domains`
(`id` ,`name`)
VALUES
('1', 'yourdomain.com');
mysql> INSERT INTO `mail`.`virtual_users`
(`id`, `domain_id`, `password` , `email`)
VALUES
('1', '1', ENCRYPT('somepassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), 
'[email protected]');

And exit MySQL
mysql> exit

-Postfix

Start by editing the main configuration:
# nano /etc/postfix/main.cf

Delete everything in here. Hold CTRL and K to delete entire lines to make it go faster.

Put this in the file:

smtpd_tls_cert_file = /etc/letsencrypt/live/yourdomain.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/yourdomain.com/privkey.pem
smtpd_use_tls = yes
smtpd_tls_auth_only = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtp_tls_security_level = may
myhostname = vm-1.yourdomain.com
mydestination = yourdomain.com, vm-1.yourdomain.com, localhost.yourdomain.com, localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
milter_protocol = 2
milter_default_action = accept
smtpd_milters = inet:localhost:12301
non_smtpd_milters = inet:localhost:12301
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
virtual_transport = lmtp:unix:private/dovecot-lmtp
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf,
        mysql:/etc/postfix/mysql-virtual-email2email.cf

As always change "yourdomain" to your actual domain name.

Now do the same for these next few files. In these change "securepassword" to the password you used for the mail user to connect to the database earlier in the MySQL setup.
# nano /etc/postfix/mysql-virtual-mailbox-domains.cf

user = mail
password = securepassword
hosts = 127.0.0.1
dbname = mail
query = SELECT 1 FROM virtual_domains WHERE name='%s'

# nano /etc/postfix/mysql-virtual-mailbox-maps.cf

user = mail
password = securepassword
hosts = 127.0.0.1
dbname = mail
query = SELECT 1 FROM virtual_users WHERE email='%s'

# nano /etc/postfix/mysql-virtual-alias-maps.cf

user = mail    
password = securepassword
hosts = 127.0.0.1
dbname = mail
query = SELECT destination FROM virtual_aliases WHERE source='%s'

# nano /etc/postfix/mysql-virtual-email2email.cf

user = mail
password = securepassword
hosts = 127.0.0.1
dbname = mail
query = SELECT email FROM virtual_users WHERE email='%s'

And restart Postfix:
# service postfix restart

Now let’s test the connection. Run this command with the email address you created earlier. If it returns a “1” then it was successful.
# postmap -q [email protected] mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

Now let’s allow clients to connect with TLS for full encryption.
# nano /etc/postfix/master.cf

Uncomment these lines:

submission inet n       -       -       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_client_restrictions=$mua_client_restrictions
  -o smtpd_helo_restrictions=$mua_helo_restrictions
  -o smtpd_sender_restrictions=$mua_sender_restrictions
  -o smtpd_recipient_restrictions=
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

Now append this to the recipient_restrictions line before the closing quotes:

permit_mynetworks,permit_sasl_authenticated,reject

-Dovecot

First let’s make a user for it to run as (change "yourdomain"):
# useradd -m yourdomain

Edit Dovecot's main config:
# nano /etc/dovecot/dovecot.conf

Replace everything in here with this changing "yourdomain" as always:

!include_try /usr/share/dovecot/protocols.d/*.protocol
protocols = imap lmtp

service imap-login {
  inet_listener imap {
    port = 0
  }
}
service lmtp {
   unix_listener /var/spool/postfix/private/dovecot-lmtp {
           mode = 0600
           user = postfix
           group = postfix
   }
}
service auth {

  unix_listener /var/spool/postfix/private/auth {
  mode = 0666
  user = postfix
  group = postfix
  }

  unix_listener auth-userdb {
  mode = 0600
  user = yourdomain
  #group =
  }

  #unix_listener /var/spool/postfix/private/auth {
  # mode = 0666
  #}

  user = dovecot
}
service auth-worker {
  user = yourdomain
}

disable_plaintext_auth = yes
auth_mechanisms = plain login
ssl = required
ssl_cert = </etc/letsencrypt/live/yourdomain.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/yourdomain.com/privkey.pem

mail_location = maildir:/home/yourdomain/mail/%d/%n
mail_privileged_group = yourdomain
passdb {
  driver = sql
  args = /etc/dovecot/dovecot-sql.conf.ext
}
userdb {
  driver = static
  args = uid=yourdomain gid=yourdomain home=/home/yourdomain/mail/%d/%n
}

Now let’s make the folders for the domain and set the permissions on it:
# mkdir /home/yourdomain/mail
# mkdir /home/yourdomain/mail/yourdomain.com
# chown -R yourdomain:yourdomain /home/yourdomain/mail

Connect it to the MySQL database:
# nano /etc/dovecot/dovecot-sql.conf.ext

Add this into the file, changing "securepassword" to the mail database user password :

driver = mysql
connect = host=localhost dbname=mail user=mail password=securepassword
default_pass_scheme = SHA512-CRYPT
password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';

Now we set permissions on the dovecot folder itself and the auth daemon:
# chown -R yourdomain:dovecot /etc/dovecot
# chown -R yourdomain:dovecot /var/run/dovecot/auth-userdb

Restart dovecot:
# service dovecot restart

Now we need to open some ports:
# ufw allow 25
# ufw allow 587
# ufw allow 993
# ufw reload

-OpenDKIM

# apt install opendkim opendkim-tools

Edit the config:
# nano /etc/opendkim.conf

Replace everything with:

AutoRestart             Yes
AutoRestartRate         10/1h
UMask                   002
Syslog                  yes
SyslogSuccess           Yes
LogWhy                  Yes

Canonicalization        relaxed/simple

ExternalIgnoreList      refile:/etc/opendkim/TrustedHosts
InternalHosts           refile:/etc/opendkim/TrustedHosts
KeyTable                refile:/etc/opendkim/KeyTable
SigningTable            refile:/etc/opendkim/SigningTable

Mode                    sv
PidFile                 /var/run/opendkim/opendkim.pid
SignatureAlgorithm      rsa-sha256

UserID                  opendkim:opendkim

Socket                  inet:12301@localhost
OversignHeaders         From

Edit the next config:
# nano /etc/default/opendkim

And add this to the bottom:
SOCKET="inet:12301@localhost"

Now let’s make folders to store keys in (change "yourdomain"):
# mkdir /etc/opendkim
# mkdir /etc/opendkim/keys
# mkdir /etc/opendkim/keys/yourdomain.com

Now to specify trusted hosts:
# nano /etc/opendkim/TrustedHosts

Add this in the file (change "yourdomain"):

127.0.0.1
localhost
::1
*.yourdomain.com

Edit the key table:
# nano /etc/opendkim/KeyTable

Add (change "yourdomain"):

mail._domainkey.yourdomain.com yourdomain.comt:mail:/etc/opendkim/keys/yourdomain.com/mail.private

Now let’s create the keys:
# cd /etc/opendkim/keys/yourdomain.com
# opendkim-genkey -s mail -d yourdomain.com

Set ownership on the key:
# chown opendkim:opendkim mail.private

Now we need to read the key and add it to our DNS record for this domain:
# cat mail.txt

Copy from “mail” to the closing parenthesis

Edit the DNS zone for your domain name:
# nano /etc/bind/zones/db.yourdomain.com

Paste the key into the very bottom of the zone file. Remove the opening and closing parenthesis, and make sure it is all on a single line. It should look like this:

mail._domainkey IN      TXT     "v=DKIM1; k=rsa; p=AKLJBVEWBlKjbfhlkse1237098vfnkuNJHYGFASD978290NLNo920293480KBN8oy0324lnawlskn213098"

Now increment the serial number on line 6 of the zone file by 1.

Restart bind9, postfix, and opendkim:
# service bind9 reload
# service postfix restart
# service opendkim restart

-Closing notes

If everything works you can now setup your email client. Use the full email address as the username, and connect to your domain name for the server. Make sure to enable SSL/TLS for inbound messages through IMAP, and STARTTLS for outbound messages through SMTP.

Now you have your own custom configured email server!

To add more email accounts, run the same command when you added the first email account. Make sure to increase the line number each time. This is the first number on the last line of this command. In the example below it is "2" for adding a second account. To add a third account run the same command with the 3rd email address and password you want and set this number to "3".

mysql> INSERT INTO `mail`.`virtual_users`
(`id`, `domain_id`, `password` , `email`)
VALUES
('2', '1', ENCRYPT('somepassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), '[email protected]');

Thanks for reading! If you have any feedback let me know, or if you have any questions just ask.

13 Likes

I like to see a very low level approach to setting up a mail server. But i would only do it once. Updating can get annoying after some time.

Why didn't you go the route with full packages/setup scripts like mailcow, iredmail or all in one groupware solutions? I think such a video tutorial would attract a bigger audience.

BTW: You might want to get a LED ring light or panel. The lighting in the video was very grey, wolf.

From my experience with cPanel, the all in one solutions are unstable and if something breaks then you are at the mercy of their documentation to figure it out. When you use postfix and dovecot you have the entire Linux community to ask. Both of these are very old and very well tested/documented. And the other mail solutions are just scripts automating postfix and dovecot anyway, might as well use the original instead of a spin-off. Basically my take on why Debian is better than Ubuntu.

And it's not the point of this tutorial series. This is all about understanding how the core system works and building your own from the ground up.

No updating is necessary, outside of the normal apt upgrade every now and then. Once this is setup it never has to be changed.

I do have an LED panel. I played with Lumetri color grading on my other videos but can't quite seem to get it right, usually end up looking orange. So I left it raw this time to see the result.

2 Likes

Nice guide! However I would be interested to know what precautions and measures should be taken when opening this service to the internet. Much like placing a web server online and locking down access to apache.

What do you mean? With TLS encryption there isn't much more "locking-down" to do.

What sort of security measures do you take with Apache?

That is the answer i wanted to hear. :slight_smile: I still use the setup scripts, but your tutorial makes it easy to understand what happens in the background and how to do it manually. For daily admin i like to use a webui.

Hopefully. Friends of mine have lost weekends when they had to change the configs after an update. It's only 2 or 3 times in 5 years, but it is takes a long time. Especially if you want your server to be secure. (Tick all the security checkboxes!)

Any particular reason why you did not include SPF in your write up? Many servers don't check DKIM, but really want to see spf or you might end up on a block list or grey listing takes several hours.

Seriously - nice write up!

But when does part two come out? The part about keeping it from become a spam server? Kind of to building off @SudoSaibot suggestion.

Things to consider. What is monitoring? How do I monitor my mail queues and what heck is a mail queue? How am I alerted when the outbound queue suddenly jumps to 7,000 and what does that mean?

Is there a way to set a lockout policy?

Which logs are important to inspect and what to look for? How do I spot a compromised account?

How many emails can one compromised account send out in 30 minutes? How many other email servers would receive a forwarded phishing attack from an unattended email server. How might that affect another email admin with 15,000 accounts?

To not be a total dick - I'll post a simple bash script tomorrow after work, which can be useful for seeing the health of your email server (small potatoes but it's a tool none the less. Part of an arsenal) and some resources to check to make sure your email server isn't a trouble maker.

Gnigjt.

1 Like

This series is for newbies. Follow the guidelines, use proper procedure with full encryption, and most people won't have any issues.

You are free to post that if you wish, your tone was a bit hostile though. Everything you described is an advanced topic and not part of this guide's scope.

As you mentioned this may be an addendum to this initial beginner tutorial.

When I first setup a web server I later found a slew of vulnerabilities I was unaware of. And after witnessing the hoard of bots crawling IP addresses I'm more skeptical when setting up something new.

With apache, disabling options like indexing and deny all access to folders except specific directories. Removing unnecessary modules and disabling signatures. Also setting up https. I also setup pfsense and use snort and firewall rules to severely limit access to my network.

A personal email server would be sweet though, especially If I can shut it down until I'm expecting emails. Basically use it only for account verification and recovery. Or relay my server/ security equipment notifications.

I don't have time to watch this video now, but this is highly interesting for me.
I bought some VPS last year to migrate to Kolab, with the expectation of a new release in Q1 2017. That never happened.
And what I did get installed of Kolab isn't working the way the docs say it should. I don't know if the project is stalled, dying, or already dead. It doesn't feel like a vibrant community, when the forums are actually functional

I've been considering Citadel, but it doesn't have mobile sync. I suppose I could tolerate IMAP/SMTP but ugh.

The only other option is to pay $13/month/user for hosted exchange.

If you read part 1 & 2 of my guides I go through full https, only opening necessary ports, ssh keys, and full TLS encryption for email.

Other than a spam checker there isn't much to do to be more secure without other tools.

1 Like

SELECT COUNT(*) FROM virtual_users;

If this is a small email server you should know how many emails are on your sever, if there are many more suddenly, you should inspect your table for "interesting" email addresses. That would also be a big red flag that someone else has elevated system permissions - frequently checking wc -l /etc/passwd would be helpful here. Also, know your wheel group memberships.

Monitor your email server, check out icinga. Keep spam limited.

As promised here is one of my first AdminTools I wrote as a junior: It basically tells if you're on any naughty lists - DON'T CONFUSE THIS WITH MONITORING...

    #!/bin/bash
    #use your outward facing SMTP IP
    smtp1=

    echo "Are we blacklisted"

    echo "Checking $smtp1 SMTP"
    echo
    echo
    curl -k --data "domainName=$smtp1" https://www.ultratools.com/tools/spamDBLookupResult | sed 's/<[^>]\+>// g' | sed 's/ //g' | grep -B1 "^Listed"

Very nice tutorial . Good quality setup. I am curious though for the end user does proton mail not make more sense? Its encrypted and the data is in Switzerland most secure data center?

Thanks for this tutorial. However "mydestination" in postfix config file should only be localhost because in the db the domain is already mentioned (it is producing warnings in the logs about that.

Also, when using Thunderbird it tells me it cannot create the directory "Sent" when I send an email... Or when I manually create a directory it has strange characters "INBOX^test" with a square reflecting an unknown character at the end... how could we resolve this? Nothing's in the log about it :frowning:

If someone is able to gain root access to your box and create new email addresses that is an issue outside the scope of this tutorial. This means you have a fundamental security hole in your system. I cover SSH keys in another guide, use that and nobody can brute force your system.

That's how it's setup on my server and it works just fine. Might check your formatting or make sure your hostnames are set correctly in /etc/hosts.

The Thunderbird thing is new, make sure you set the permissions correctly on the mail folders in /home/yourdomain/mail

This is about hosting your own mail server and understanding it from the ground up. If you would prefer an offshore system then by all means go for it.

You never know if the service is actually encrypted if you are trusting an outside vendor to do it for you. The only way to be sure of your data's location is to host it yourself.

Do they type their password in when they log into their email or are they using cert-based auth to access their email accounts?

Like when I connect to the server from my phone - after I enter my username do I have to type my password on my phone or do I have private key on my phone for access my email account?

I was wondering the same. SPF is quite important, both for your incoming and outgoing mail. It checks whether the delivering mail-server is authorized to deliver mail in the name of the sender. So this not only protects your own users from getting easy phishing mails but also protects your own identity towards other servers so they don't put your mail domain on a blacklist because someone else delivered spam in your name.

The best part is, that SPF is as simple as a DNS TXT record for your outgoing mail and a simple python/perl script that's referenced to in your Postfix's master.cf and main.cf files for your incoming mail.

https://help.ubuntu.com/community/Postfix/SPF


The precautions here are e.g. making sure you're not an open relay (anyone can send mail via your server) and that you configured things like SPF. Spam filtering is not a must but for obvious reasons it's a great idea. "Spamassassin" is your Google keywords here.


These are two separate things. "root access" is a console-based login through SSH which is done best with certificates. Email is usually received via IMAP, where you would need a normal password-based authentication. You can use certificates for authenticating IMAP too, but I think most clients like smartphones don't support this. However, if your users are only using laptops or desktop computers to check their mail, this might be a viable option. Thunderbird for example supports this way of authentication.

1 Like

I'm not interested in root right now. If I want to connect with my smart phone, in this example, do I type my password in and then send it or is it cert-based auth for access to my account?

Also, on a stock install of what you currently have, if this were an outward facing IP what would an nmap scan show?