Why sticking with distro packages can be (very) bad for your security

I’ve been keeping a variety of systems up to date, updating security and other bits with zealous fervor. Security is never far from my mind, as I’ve watched bad practices being used at customers resulting in any number of things … from minor probes, through (in one case, with a grad student impacted by a windows key logger), taking down a linux cluster, but not before knocking the university temporarily off the internet. Yeah, that last one was a doozy. We had warned the user what not to do , and they ignored us. Multiple times. They wound up messing up many other people in the process. And then they got angry with us when we refused to fix their problems (again) for free (again). So with this backdrop, I’ve been working on keeping all of our sites up to current best practices. I am not a security expert by any stretch … though I have the requisite paranoia … driven in many cases by the scary logs I’ve collected over the years. I pay a great deal of attention to what people whom focus upon this say. And I leverage a number of tools to test our sites. I am not doing hard core pen testing of our systems (yet), but this is probably going to be on the radar. Ok. So here is the problem. Using a distro’s most up-to-date packages, the nginx web server didn’t support TLSv1.2. Which, if you read the qualys ssl labs pages, is the only really secure protocol. The testing tools all seemed to indicate that there was a problem. As Google is your friend, I did some checking, and sure enough, people in 2012 (!!!) had noticed that the nginx version was linked against an older ssl. 0.9.8 or something like this. In 2016, its not like, I dunno … possible? I mean, with all the SSL attacks, wouldn’t a stable distro actually have links against modern kit? This is one of the reasons, but not the primary reason, why we provide out own toolchain. Long experiences with distros shipping badly outdated tools (some programming language revisions being End-of-Lifed years before the product shipped!!!) led us to start doing this. As the complexity of our needs increased, and our strong dependency upon working modern and patched languages and toolsets grew, it became harder and harder to use the distro tools. We couldn’t just replace them, as the distros often build effective dependencies upon the older broken tools. So tonight, I am looking at a low grade on a web server, after I updated the certificate. I was stumped. I had checked the ssl bits, and they were correct. The same as the servers getting the high grades. What could be the issue? A quick

ldd `which nginx`

showed me this:

...
    libssl.so.0.9.8 => /lib/x86_64-linux-gnu/libssl.so.0.9.8 (0x00007f5d93b02000)
    libcrypto.so.0.9.8 => /lib/x86_64-linux-gnu/libcrypto.so.0.9.8 (0x00007f5d93774000)
...

Oh … my. Ok. So my choices are to either use the package providers package, or build from source. Lets try the former first. A quick install, and restart later, and my server gets an excellent score. I wonder, for all the nice “stable” packages out there in the “stable” distros, what level of insecurity they are willing to tolerate? I don’t want to take ownership for lots of the distro stack … I like distros with well thought out and implemented tools (Debian and alike), well thought out overall theses (alpine). I want them to provide modern packages with up to date security. I do not want … absolutely positively … do not want … backports of “security patches” which provide an illusion of security, while not actually providing the security. This is what I was running into. The incessant updating? Check. Security? Possibly, but only because I am paranoid about it. Not because the packages have it built in. They don’t. I am starting to think that most of the packages that open ports to listen to network traffic might need to be paid very close attention to. Far more than simply updating. Personally I think the illusion of security may be as bad as, if not worse than no security. As if you believe you are secure, and you really aren’t, you may discredit reports that indicate you have issues, real issues, when you shouldn’t.