Full Tutorial: How to Set Up Docker Mailserver with SSL on Debian 12 VPS

This guide walks you step-by-step through setting up a fully functional mail server using docker-mailserver on a Debian 12 VPS, with self-signed SSL support.

---

✨ Prerequisites

• Debian 12 VPS with root access
• Domain name (e.g., mydomain.net)
• Hostname set to mail.mydomain.net
• Ports 25, 587, 465, 993, and 143 open

---

🧱 1. Install Docker & Docker Compose

apt update && apt upgrade -y
apt install -y docker.io docker-compose ufw curl git
systemctl enable docker --now

---

📁 2. Set Up Project Directory

mkdir -p ~/docker/mailserver
cd ~/docker/mailserver

---

📝 3. Download setup.sh

wget https://raw.githubusercontent.com/docker-mailserver/docker-mailserver/master/setup.sh
chmod +x setup.sh

---

🔒 4. Generate Self-Signed SSL Cert (Optional for testing)

mkdir -p config/ssl
openssl req -x509 -newkey rsa:4096 \
  -keyout config/ssl/key.pem \
  -out config/ssl/fullchain.pem \
  -days 365 -nodes \
  -subj "/CN=mail.mydomain.net"

---

🚀 5. Run Docker Mailserver

docker run -d --name mailserver \
  -p 25:25 -p 587:587 -p 993:993 -p 143:143 -p 465:465 \
  -v "$(pwd)/maildata":/var/mail \
  -v "$(pwd)/mailstate":/var/mail-state \
  -v "$(pwd)/config":/tmp/docker-mailserver \
  -v "$(pwd)/config/ssl":/etc/ssl/mail \
  -e SSL_TYPE=manual \
  -e SSL_CERT_PATH=/etc/ssl/mail/fullchain.pem \
  -e SSL_KEY_PATH=/etc/ssl/mail/key.pem \
  -h mail.mydomain.net \
  mailserver/docker-mailserver:latest

---

💼 6. Add Your Mail Account

./setup.sh email add user@mydomain.net

Enter and confirm your password.

---

🔍 7. Test IMAP Over SSL

openssl s_client -connect 127.0.0.1:993

Then type:

a login user@mydomain.net yourpassword

Expected: a OK Logged in

---

⚖️ 8. UFW Firewall Configuration (Optional)

ufw allow 22/tcp
ufw allow 25,465,587,993,143/tcp
ufw enable

---

📊 9. DNS Records to Add

Type	Name	Value
A	mail	VPS IP
MX	@	mail.mydomain.net (Priority 10)
TXT	@	v=spf1 mx ~all

---

🚫 10. Troubleshooting

• Can’t connect to 993:
  • Confirm Dovecot is running: docker exec -it mailserver ps aux | grep dovecot
  • Confirm port binding: docker ps
  • Recheck firewall: ufw status
  • Check logs: docker logs mailserver
• No SSL cert: Recreate config/ssl/key.pem and fullchain.pem

---

🎉 Done!

You now have a fully operational Docker-based mailserver with secure IMAP over SSL. From here, you can:

• Set up webmail (e.g., Roundcube)
• Enable DKIM/DMARC
• Add reverse DNS/PTR records in your VPS panel

Let me know if you’d like to extend this with SMTP relay, Nginx proxy, or Let’s Encrypt integration!