by j_s on 1/26/18, 1:29 PM with 325 comments
by agentultra on 1/26/18, 3:08 PM
Actually it's fine until it's not. Then your email doesn't work and you could be missing out on important communications. And then you're scrambling to figure out how the spammers managed to exploit your setup this time. And you have to learn a tonne of crap in order to manage it... and the text files! Configuration... configuration everywhere. Obscure configuration. Configuration that has real consequences and causes spooky action at a distance. Configuration that will soon be exploited in strange ways.
I was so frustrated the last time my mail server went down that I started writing an SMTP protocol handler in Haskell with the intent of writing a MTA with the goal of minimizing configuration and being secure and resistant to attacks by default. So that hopefully more people can run their own infrastructure without prematurely aging. I dunno how useful it will be to others but at least it will keep my gray hairs at bay, I hope, when it's ready for use.
Until then though we need more guides like this for us poor souls who do go down this route. There are way too many out-dated guides awash in the sea of information.
by deadbunny on 1/26/18, 2:56 PM
This for me is the deal breaker. I say this as a sysadmin.
It's all well and good running a single server but if that shits the bed then you have to deal with it immediately.
So to run something that doesn't require 24/7 support I now have to run a cluster of servers.
I'll also need something to manage those servers like salt/ansible/puppet
I also need to deal with being my IP blacklisted because of a previous owner, or just entire domains now delivering my email because they don't like the fact i'm not using $email provider
Then there is spam filtering and the constant battle that is.
Or I could just pay someone a couple of bucks a month to worry about all that shit and not worry about it.
by notinventedhear on 1/26/18, 2:36 PM
I've been running my own mail server with https://mailinabox.email for ~2 years and can heartily recommend it.
Still, even with that there are gotchas if you want to be able to send messages from your server to Google et al, eg. reverse-DNS-records, DKIM, SPF.
Not for the pressed-for-time.
by SwellJoe on 1/26/18, 4:27 PM
I think what I'm trying to say is that it's not harder than it used to be (though some problems, like spam and security requirements for safety, have gotten worse in absolute terms). With modern tools and packaging on modern Linux distributions, you can be up and running pretty quickly. My company ships a turn-key solution as part of Virtualmin, but you can build something similar without that in an afternoon or two if you're reasonably Linux-savvy and have some notion of how all the pieces fit together (maybe a couple extra afternoons if you don't know the basics; DKIM and SPF can be tricky, since you also have to know or learn you some DNS).
It's harder than a web server or DNS server, but not something you should flee in terror from. Admittedly, it's gotten cheaper in recent years to outsource it...and with microservice-based architectures, maybe it makes more sense to have some other API than SMTP (though SMTP is very easy to use from every language I've ever worked in). But, there are problems and complexities with outsourcing, as well.
by workthrowaway27 on 1/26/18, 3:17 PM
It would be great if someone wrote a program handling all of this that could be deployed as a single binary with secure defaults and limited configurability, but I don't see that happening any time soon. Email providers are good enough for almost everyone and the people who are good enough programmers to make sense of all the different protocols they'll have to deal with and get everything to interoperate nicely probably have other things to work on that people will actually pay for.
Edit: That said, this guide does look like a great resource for someone who is interested in doing this. It's interesting to learn how email works and if I had this guide when I started out I'd have saved a ton of time.
by Yetanfou on 1/26/18, 3:41 PM
I've never regretted running my own server, nor have I ever contemplated moving to a hosted solution. Spam is not a problem either, Spamassassin in combination with a greylist make for a nearly spam-free experience. The whole setup has been migrated from the original Pentium-66 via an aBit-BP6 (SMP for the masses [2], retired in 2009) to the current Intel SS-4200 (upgraded to a dual-core Pentium but still limited to 2GB). In practice a Raspberry Pi would be enough to run a viable mail server so even this rather anaemic setup does its job without breaking a sweat.
The whole setup consists of Debian (Sid) running Exim through a smarthost, feeding through Spamassassin + greylistd into Dovecot. Apart from some auto-manual intervention to cope with Microsoft/Google/... not coping with the greylisting and thus needing whitelisting it more or less just works. In other words, just go ahead and run your own server.
[1] http://shop.oreilly.com/product/9780596510299.do
by grinsekatze on 1/26/18, 4:19 PM
I have been using mailcow[0] for years now and it does all of this for me and works great. The UI is beautiful and intuitive. And setting up mailcow literally takes a few of minutes, since the project was ported to docker. Highly recommend it.
by linsomniac on 1/26/18, 3:07 PM
How to run your own mail server (for experts): Don't.
I say this as someone who has run my own mail server for 20+ years.
Now, if you absolutely have to get off Google, and the other available hosted options don't work for you, then this article looks like a good start. I'd add roundcube for web access and letsencrypt for SSL.
One thing you'll never really get, in my experience, is good spam handling. The big providers just have so much more data to work with to prevent spam.
by lucb1e on 1/26/18, 2:49 PM
That seems like an awful amount of work / overkill. This is not a good introduction for a normal geek, and die-hard open source fans will figure it out without too much trouble anyway.
Postfix is not the best choice for novices, FreeBSD is not the most well-supported/documented system just because it's not as popular as some others, and a directory service shouldn't be necessary.
We had to setup Exim, Postfix or Sendmail for school. Sendmail was universally hated the most, Postfix came in second, and Exim was... well, not exactly logical or easy, but the best of the three mainstream MTAs.
I'm running hMailServer at home. Windows-only, unfortunately, but until I find a proper replacement, I'll just keep running it in a VM. Nothing else even comes close in admin-friendliness. It's just install and run, with either a local admin interface or a web interface (using PHP, so it runs anywhere).
by Torgo on 1/26/18, 2:24 PM
If tweaking spam filters and deleting spam is your idea of fun, then run your own email server.
by codingdave on 1/26/18, 3:47 PM
I know the article is concerned with owning your own data, and I appreciate the point. But finding a mail provider that meets your needs is, IMO, a better way to spend your time than just saying "Gmail isn't good for me, so I'll do my own."
by icelancer on 1/26/18, 3:56 PM
https://www.linuxbabe.com/mail-server/ubuntu-16-04-iredmail-...
I dunno. I feel like I'm sitting on a timebomb. It's hosted on DigitalOcean and while it works great with RoundCube and gmail... eh. I dunno why I even did it in the first place.
by ef4 on 1/27/18, 1:24 AM
This just isn't true. You can host your own mail server and GMail will probably still end up hosting a large fraction of the email you read and write, because the people you correspond with are still using GMail.
(In the same vein, you can refuse to have a Facebook account but Facebook probably has a dossier on you anyway. Enough people you know have dumped their contacts into Facebook that they already know your place in the social graph.)
by twothamendment on 1/26/18, 2:32 PM
My setup is similar, but it uses MySQL instead of LDAP.
I love being able to make aliases and even better - deleting them when I'm done with them.
by digitalsin on 1/26/18, 8:48 PM
Running your primary email on these big hosting companies is taking your privacy and pissing it right down the drain.
It's really not that hard folks.
by perlgod on 1/26/18, 2:52 PM
by ankitank on 1/27/18, 11:30 AM
iRedmail - https://www.iredmail.org/ Mail in a box - https://mailinabox.email/
They allow you to setup your own mail server and yet make it easier to get started.
I have been using iRedmail and it has been working well so far.
by kazinator on 1/26/18, 10:11 PM
Exim MTA, Courier IMAPD on Debian.
Pretty easy setup; nothing complicated.
For remote access I use two things: RoundCube webmail, and K-9 Mail on Android. For sending mail from K-9, I connect home, via authenticated SMTP which is on port 587, rather than 25.
I have developed a little web app called Tamarind for generating throw-away mail aliases.
http://www.kylheku.com/cgit/tamarind/tree/
I run some mailing lists which use GNU Mailman. For archiving them, I don't use that horrible pipermail, but rather a hacked version of Lurker. I patched Lurker to pass through HTML so that HTML mails end up rendered as HTML in the archive. The HTML has to be scrubbed, so I wrote a little scrubber for that:
http://www.kylheku.com/cgit/hc/tree/
Lurker patches:
by akulbe on 1/26/18, 2:30 PM
Second, if you must... I'd recommend Mail-in-a-Box. MUCH easier to setup / maintain than this one, at least from a cursory read. https://mailinabox.email
by CodeWriter23 on 1/26/18, 5:20 PM
by jacksnipe on 1/26/18, 4:58 PM
by interfixus on 1/26/18, 4:44 PM
And then ... after you've done every damned thing exactly by the book, and DKIM'ed the dickens out of your headers, killed the spam, policed yourself off the blacklists, etc. etc. - turns out you might as well not have bothered. The googles and the microsofts (the microsofts especially!) will one day drop your outgoing mail without the slightest notification, because the ip range, or because the full Moon, or just because they can.
I ran that show for nearly fifteen years, but threw in the towel last year, and handed over to Fastmail. With regret, although their service is first rate.
Email is not a succesful federated protocol these days. The monoliths effectively killed it off.
by Tepix on 1/26/18, 4:12 PM
by teekert on 1/26/18, 2:39 PM
I also learned that having it in your basement means trouble: Someone pulls the plug, your IP changes or ends up on a spam list. For some years I ran my own server on a DO droplet. It is very cost effective when you can make as much mailboxes as you want for family (+ unlimited aliases, addresses that deliver to both you and your wife, being able to email 500 mb to familie, etc). I still don't know down what sinkhole emails to my brother-in-law's outlook.com address went down. The literal response of MS at the time: We don't manage our own spam filter, try adding more text, make it look more real...
But man, the pain, the complexity, the reverse DNS, the startTLS, the SPF record, the DKIM records. It took me a long time to understand the difference between mail servers and MTAs and why there are different ports for them. Also, few providers in the Netherlands even allow you to use port 25, luckily mine did. Email is truly an old protocol that has been hacked up-to-date (more or less) and setting up your own mail server will make this very clear to you :)
I'd recommend it though, you'll learn a lot! But to be honest, I now pay 3 euros a months to a dutch email provider because email is too important and I didn't want to go through the pain again when 16.04 came out. I might still have a go at it in the future, there is something beautiful about running your own email server :)
[0] https://arstechnica.com/information-technology/2014/02/how-t...
by jstewartmobile on 1/26/18, 4:11 PM
One nice thing about the programs he chose is that their config options are fairly stable (can't vouch for Solr). That many moving pieces would be absolutely unmanageable if the options changed frequently. Been using a similar setup for years without difficulties.
Adding something like fail2ban into the mix wouldn't hurt.
If you're going to do this, first check that your VPS / ISP allow inbound traffic to port 25/tcp. AWS allows it upon special request. GCE doesn't. Don't know about the others. In the US, most residential ISPs block 25/tcp inbound.
by osrec on 1/26/18, 8:43 PM
by locusm on 1/27/18, 12:14 AM
Its useful as quite often youll get blocked just for being in the same IP block as a spammy server. My experience to date is that once you notify them your IP isnt the culprit the block gets removed pretty quick.
by herbst on 1/26/18, 1:35 PM
Curious if you looked into other mail server options? I mean Postfix (not Postgres) easily handles the load of a single user, but it is still rather hard to configure and modify.
I recently started to work with Haraka and even thought its ment for high traffic use cases i wont ever look back to Postfix (not Postgres).
by marsrover on 1/26/18, 2:27 PM
by philrw on 1/26/18, 3:44 PM
by j45 on 1/26/18, 3:24 PM
Instead of running each component individually, I would recommend looking at something like Zimbra or another OSS mail package that handles a lot of this.
I hosted my own email for over 10 years and maintaining the bits are as painful if you don't have a plan in place.
A decent comparable for do-it-yourself hosting is the kind of luck a product like MDaemon provides - it decent job on windows of rolling all the features into a reasonably manageable server, as well as being quite affordable. I don't work for MDaemon, but tools like this make hosting email relatively trivial.
by TYPE_FASTER on 1/26/18, 3:10 PM
Fighting spam effectively is not trivial.
by andris9 on 1/26/18, 3:35 PM
by apple4ever on 1/26/18, 3:11 PM
I thought about Fastmail, but its too expensive when you want to do more than one email address or domain. I run mine on a $5 server from DigitalOcean.
by foxhop on 1/27/18, 1:02 PM
I also accept inbound mx for some of my personal domains, but I don't currently manage my own mailboxes, the mail gets aliased to my Gmail account.
If at some point I get fed up with this arrangement I can transparently change where the email ends up.
by sfilargi on 1/26/18, 10:54 PM
It's basically a single binary that has an SMTP server and webmail server. It works absolutely fine for me without much stress. I do hit a couple of bugs here and there, mainly on the mail parsing, but it's not big deal.
by JepZ on 1/26/18, 2:58 PM
by vog on 1/26/18, 2:30 PM
However, after reading this, the software developer in me feels like "These tutorials should not be necessary."
This should be a GitHub repo, trivial to fork and trivial to test out locally for anyone.
by godman_8 on 1/26/18, 2:41 PM
by Jaruzel on 1/26/18, 3:24 PM
I.E. like this:
Internet Email (SMTP) -> [MTA + Rspamd] -> [Real MTA + Inboxes]
Thanks.
by storsjt on 1/26/18, 5:08 PM
Sure it's _hosted_ in a centralised place but since you're paying for it Amazon shouldn't have an incentive to harvest your data.
by brlewis on 1/26/18, 4:21 PM
by c17r on 1/27/18, 1:10 AM
by petre on 1/26/18, 2:58 PM
by mattbillenstein on 1/26/18, 10:25 PM
by phelmig on 1/26/18, 4:06 PM
by goerz on 1/26/18, 4:20 PM
by thro1237 on 1/26/18, 5:39 PM
by mderazon on 1/26/18, 5:54 PM
Looks pretty daunting to me
by leonroy on 1/27/18, 12:23 AM
If you like email Zimbra's great - it's a fully baked mail server which you install on your Linux distro of choice and it goes off and installs all of the above for you. Everything is managed via a GUI and you have a great web interface and standards based IMAP, CalDAV support.
If you buy the paid version you can even get ActiveSync and Exchange Web Services for it.
Despite all that though, I would give anything to have all those countless hours I put into running my own mail server back. It is a colossal time sink. I can't even stress how much work it is, especially if you have anyone relying on the box for their primary mail account. It's no fun at all.
You're gonna be debugging Fetchmail for when you or your user's want POP3 accounts downloading mail locally. Procmail for filtering. SpamAssassin (gawd if ever there was something which consumed my life it's that software and its myriad libs and helpers), not to mention familiarize yourself with DNS MX records, SPF, DKIM etc. etc.
All of the above works surprisingly well and is fairly solid - until it isn't. When Google added DKIM/SPF protection and blacklisted servers which didn't was a fun weekend that I'd rather have spent with my family. When customer emails started bouncing because their IPs had hit an over zealous RBL list which Zimbra was using was a fun afternoon of debugging. When Zimbra decides to randomly let in 10-20 spam emails a day into my mailbox is another weekend project which I've yet to get round to. Thank god I'm self employed is all I can say because no employer would tolerate an employee putting the care and feeding required to maintain a personal email server!
Unless you want to nurture a career as a mail sysadmin seriously, don't host your own mail server.
Bottom line I'd recommend to anyone thinking about hosting one to either:
1. Don't
2. Use Microsoft Exchange Server
3. Use Zimbra
4. Seriously, don't - consider Fastmail, Gmail, O365 or Protonmail instead.
by bedros on 1/26/18, 4:04 PM
by rootsudo on 1/26/18, 2:35 PM
by tzs on 1/26/18, 7:27 PM
For example, suppose I have things set up so all outgoing mail from my home goes through my SMTP server. If I send an email with a from address of tzs@mydomain, then the setup in the article is perfect.
Suppose, though, I send an email from home with my from address set to tzs@employer, where "employer" is my employer's domain? Assume this email is not to an @employer address [1]. With the setup in the article (and in almost every other similar setup I've seen covered in similar articles) this might run into spam filter issues unless I've convinced my employer to add my SMTP server to their SPF record.
The way I want this to be handled is for my SMTP server to see that the mail is from an @employer address, and instead of trying to deliver it directly, relay it through employer's SMTP server.
This is similar to the common "smart host" configuration often used when you run an SMTP server at home, but want it to send all outgoing mail through your ISP's SMTP server instead of trying direct delivery. Essentially what I want is a conditional smart host based on the from address.
Postfix supports this. In fact, it seems to support it in a couple different ways. I played with it a bit but could not quite get it working.
What I'm doing for now, until I find out how to do it right, is only send work email outside of work from my desktop Mac. I took tzs@employer off the list of mail aliases for my mydomain mail account, and created a second account in Apple Mail for @employer. I set the incoming mail server to POP3 on 127.0.0.1 so that it would fail, and set the outgoing server to smtp.employer. It complained for a while that it could not contact the POP3 server, but eventually stopped complaining, and the address in the configuration dialog changed to 0.0.0.0.
With that setup Apple Mail sends mail from @employer directly to my employer's SMTP.
Sometime recently, after an OS update, that stopped working. It would no longer let me enable an account unless it could successfully talk to the incoming mail server for that account.
I did find an ugly workaround for that. I gave it the correct address for employer's POP3 server, and the correct password. Once it was happy, I went to Keychain Access, found the saved password for the POP3 server, and changed it in Keychain Access to something incorrect. Mail then complains that it cannot login to the POP3 server, but that does not cause it to disable the account. Net effect: a send only account in Apple Mail. (It is important to do the password change in Keychain Access, not in Mail, because Mail won't save the change until it sees the new password work).
(If that had not worked, I probably would have written a dummy POP3 server that always reports no mail and used that).
[1] This happens reasonably often for me, because I have my mail server set up to use fetchmail to fetch my incoming work email and deliver it via procmail. Same for any other SMTP accounts I have. That way I only have to configure mail clients to work with mail server and I get access to all my mail from all of my non-web email accounts.
by throwawayeo5 on 1/26/18, 3:26 PM
I swear, I don’t work for Fastmail, but I’d much rather use them (or any email provider that is halfway decent) than fiddle with my own mail server.
by Faaak on 1/26/18, 2:57 PM
by alasdair_ on 1/26/18, 4:25 PM
I even put together a self-contained single-DVD installer that would install and set everything up securely and solidly. (I don't think I have a copy any longer and if I did it's very out of date)
I still don't recommend doing it. Even for the security-concious. It's just not worth it.
by peterburkimsher on 1/26/18, 2:21 PM
The use case is to transfer arbitrary files over a local hotspot.
I tried installing from the Apple TV tutorial, but it put my iPhone in a boot loop.
Just a chance, but are there any iOS jailbreakers here who could advise?