Server Insecurity

This article attempts to chronicle my recent upgrade adventure installing Gentoo Linux/SElinux on my home server. I'm a technical kind of guy, so I'm likely to use highly technical references without realizing it, but I'm trying to keep my tale within reach of non-Linux-experts. My objective is to keep the article accessible to anyone interested, while illustrating why someone might be crazy enough to try to install a hardened Linux build despite the extra effort required.

Set the wayback machine to mid-1998. I was looking through my home Linux server, which was then running RedHat 5.0, if memory serves, when I discovered files on the hard disk that I didn't put there. Since no one else had access to the server (legitimately) , I had been hacked. As a result, I immediately started reading everything I could get my hands on about computer security. The first thing I learned was that the applications I was using with RedHat--SendMail, WS-FTP, and Bind, three very common server applications that were installed by default on that old version of RedHat--had some of the worst security hole records out there. I turned off two of those applications, replacing them with more secure versions: Qmail, a powerful and free email server, and ncftpd, a proprietary but also free FTP server. I didn't find a good replacement for Bind (at the time), so I patched it and crossed my fingers.

About a year later, I found another file I hadn't put there. It looked like I'd been hacked again. I dropped RedHat and switched to a little-known hardened distribution called Trustix. I also found djbdns, a Bind replacement, and managed to get rid of that last major security risk.

Fast-forward to the present: Trustix didn't upgrade to the newest Linux kernels quickly enough, so I decided to try other distributions and settled on Mandrake, which had been good to me so far; whenever I heard about a new exploit or security hole, Mandrake would supply the patch and I'd quickly update my box. Unfortunately, some patches would break things, and it would take a few hours to get things up and running. And because there was that window between when the exploit is discovered and when I patched it, I always had to wonder if someone had broken in and left a trojan on my box. As a result, every new install meant starting over practically from scratch with a virgin system (so that nothing a hypothetical hacker had left behind would persist to the new system). And with a full time job, it's not fun to have to spend that much free time rebuilding my system every year or two.

For over a year I ran (on my private home server) a Mandrake 9.0 Linux box with as much security as I could turn on while still being able to use it. The box did a lot of good work for me -- it stored my MP3 files, operated as a mail server and webmail client, served my domain names, and blocked evil attacks from the outside world from my vulnerable Windows boxes.

However, Mandrake wasn't supporting 9.0 any more, so it was time to upgrade. This time, however, I wanted it to be different. I wanted this upgrade to be the last major upgrade I would need to perform, at least for many years, and I wanted the server files to be safe without me having to worry about whether someone snuck in with an exploit leaving trojans throughout my system.

I'd read a lot about Gentoo Linux, so I thought I'd give it a try. A few features recommended it to me, including the fact that there isn't really a discrete "version" number of Gentoo--when new package versions become available, you can just upgrade--and you can rebuild packages with custom options for the exact system you're building. I also discovered that Gentoo supported SElinux, a package developed by the NSA to minimize (specifically to localize) the potential damage that an intrusion can do. For instance, if your Web server can only read web files and someone breaks in through the Web server, the most that hacker can do is read web files. An over-simplification, but that's the basic idea. Gentoo also has other options for a hardened kernel build (features that prevent some attacks to begin with), which I also turned on.

The only stumbling block I hit getting started was that the normal Gentoo Live CD (the CD that you use to boot up the first time) didn't work with the SElinux build, and it didn't tell me why it wouldn't work--but a little Google searching came up with the alternate SElinux Live CD, and I was off and running. There was also a bit of confusion where my Google search had picked up an outdated HOWTO with broken links, but once I found the correct HOWTO things went a bit more smoothly.

So with an SElinux Live CD, I got a build together and installed my favorite packages--which worked fine until I turned on the SElinux protections. I had enabled the download of "development packages" so I could get SElinux components--not realizing at first that I would get the development packages for everything. These new development packages didn't have updated SElinux rules, so it took a lot of reconfiguration to get the ultra-paranoid SElinux kernel to work with all of these packages. Worse, I had the temerity to want to install packages that had no existing SElinux configurations available at all. It took a lot of work to get all of the settings tuned.

In a nutshell, when you want to make a new service work under SElinux, it's a kind of whack-a-mole game of granting access to each specific feature that the service needs. It's not quite that easy, though, because you also need to create new "security domains" for the new service, and map out how each domain can interact with each other domain...and if you do it wrong, you've just opened up a new potential security hole. I would say that this process is not for the faint-of-heart, and certainly not for the newbie.

Once all of your services have SElinux packages, and you're using the right versions of each, it will be hard to beat the security of SElinux, though. If the Gentoo folks are interested, I'll put together a few packages with my changes in them to help out future SElinux installers.

In any case, my new server is finally up and running. I also have a lot of new features almost for free because of the great Gentoo package system: SpamAssassin helps to deflect spam from my system, ClamAV helps to block viruses before they enter my email box, and Poptop will soon allow me to VPN into my home box to get to my file shares.

I won't say that the new server is bulletproof, because that would be stupid--it would certainly attract more attention than I want trying to prove me wrong, and I would probably be wrong. But considering it's a completely uninteresting personal server without as much as a single credit card number, the level of security I have on it now is enough to quell my fears.

Besides which, just to be extra paranoid, I did a baseline checksum scan of all of the applications on the server, so if someone does someday get past all of these barriers I put up, at least I'll be able to know. Until then, I'm going to sleep a bit more easily.