Friday, January 8, 2016

Goodbye, Debian 8.1

Goodbye, Debian 8.1 (Jessie). I tried the distro for several months because RHEL clones (Springdale Linux and CentOS, both 7.0 and 7.1) didn't like my legacy nVidia GeForce 6150 SE in this old desktop and it was a pain to fix that. The system still is good enough to do all the work I need to do and performs reasonably well. Debian seemed like a reasonable alternative. I chose to install it with LXDE as the desktop environment which is lightweight and ideal for old systems.

First, Iceweasel is not simply Firefox with the branding stripped out as is often claimed. It slows down, takes all the system resources and locks for anywhere from a few seconds to almost a minute on many websites and does this repeatedly. It did it across multiple versions. Firefox simply works without this nonsense. Of course, I can use genuine Mozilla Firefox on Debian, but only outside their package management system unless I build and maintain packages myself. Yes, other browsers offer better performance on Debian, but for some sites Firefox is my preference and sometimes I really need to test in Firefox.

Speaking of packages, their vaunted large repository would often have broken updates because of version mismatches and/or a lack of timely dependency updates. I also had to give up on Iceweasel language packs for a while because the browser was updated but the language packs were not. I had the choice of a browser with a known vulnerability that wouldn't upgrade or ripping out the language packs. Granted, I don't need browser menus in another language, but a lot of people do and there are things I wanted to test in a localized environment. I've seen this sort of really poor repository/package management in other distros, of course. I just haven't seen it much recently.

Despite the large package selection I was surprised that some very ordinary things I use regularly that are found in lots of other distros weren't in Debian. No matter what distro I use I seem to end up building packages. Debian is no different.

Then there is PulseAudio. Yes, it works. I couldn't get it to remember that I wanted line out as the default, not headphones, so I kept having to change it. Sure, that's a really minor annoyance, but it just works on most distros. User error? In this particular case quite possibly. I just couldn't be bothered to research it. I shouldn't have to spend the time to do so on something that just works in literally every other distro I've tried in recent years.

Performance was decent, but a distro designed to be lightweight can be faster on older equipment. I went back to an old favorite, Vector Linux, who have a very solid release in 7.1, and my system is faster. I'm not supporting Debian for work at the moment so there was no reason to keep it. Look, it's not a bad distro. Most of what I've described is relatively minor and entirely fixable. I don't want to tinker and I do want performance on legacy equipment. Debian 8.1 is simply not the best choice for me.

Monday, October 19, 2015

List of Linux System Hardening Resources

My recent post about how quickly newly commissioned Linux systems can be attacked and possibly compromised led to a bunch of e-mail queries about resources which explain how to lock down a variety of Linux distributions. Most such guides are distribution specific because, while the basic principles are always the same, there are significant differences between distributions and even versions of the same distribution that make writing a generic guide difficult at best.

I did compile a list which I added to the comments. However, based on the number of questions I've received I thought it would be best to publish the list as a blog post, something people could easily find and bookmark, with some additions to what I originally posted. I've limited this list to distributions commonly used in businesses (large and small), academia, and in non-profits. I have not included specialized distributions, including those designed for use by security professionals. Most of these distributions also are excellent choices for personal use.

Red Hat Enterprise Linux / Centos / Scientific Linux / Springdale Linux




Wednesday, October 7, 2015

Linux Security: Lock Down a New System Immediately

PCWorld recently published an article about Linux botnets launching DDoS attacks. The attackers find and exploit poorly secured Linux systems. Some Linux users have a fairly cavalier attitude about security, assuming the supposedly superior design of the OS somehow protects them. It doesn't. Now that Chromebooks outsell Windows laptops and Amdroid devices are ubiquitous the days when Linux was a secondary target for malware are long gone. Linux' prominence in both the server room and on consumer devices make it a prime target.

How quickly will a newly commissioned system be attacked? Here's my recent experience. I did a build of a webserver with CentOS 7.1 and Drupal 8. That went smoothly. Then I installed all the latest patches. On to configuring and hardening the server. The first time I issued a su command to allow me to configure sudo I got a message about failed logins. I looked at the logs and sure enough, someone was already trying to break in with a scripted brute force attack. I immediately configured ssh to disallow root login and to disconnect after three failed attempts. That slowed but did not stop the attacks. OK, I checked the IP address they were coming from, knowing full well it might be spoofed, and found the owner was a French ISP. To satisfy my curiosity I put in a temorary firewall rule to disallow that IP range and the attack stopped and did not resume. So... either the IP address was not spoofed and some script kiddie was trying to break in from their actual connection or else the attack came through a proxy server with an IP address in that range. Cute.

Anyway, I finished my usual tasks for hardening a server and Apache, uploaded my backed up files and put them in the appropriate places with the appropriate permissions. My idea of hardening Apache includes both mod_evasive and mod_security with a current rule set. My idea of securing a server includes blocking pretty much all inbound connections not needed by either the web server, software used on the website or SSH.

I should also add that IP address blocking, up to and including geolocking, is generally not an effective defense. Knowledgeable hackers use proxies. Many change their apparent IP addresses frequently. Blocking IPs, even at the firewall level, is a lot like playing Whac-a-Mole. The attack simply continues from a different IP in short order. The brute force attacks I'm still experiencing are trying to login as root and since remote root logins are disabled that simply won't work. However, the excess traffic to my server continues and there is relatively little I can do about it.

How long between my rebuild and the start of the attacks? A few minutes. Such is life on the Internet today. The first thing anyone should do with a new system, any system, is lock it down.

Friday, May 30, 2014

32-bit Enterprise Linux Still Matters

I've been testing the Red Hat Enterprise Linux 7 Release Candidate. One thing that stuck out right away was the lack of a 32-bit x86 build. In last week's DistroWatch Weekly Jesse Smith questioned the need for such a build, which is only useful on legacy hardware, in the enterprise. He wrote:
"Something which caught my attention while reading this question was the requirement for a 32-bit operating system with newer software than Red Hat Enterprise Linux 6 offers. It seems unusual that someone would want new software versions, enterprise support and a 32-bit operating system. New software and legacy hardware (or new software and enterprise environments) rarely go together and it might be worth looking into whether these criteria are really necessary."
While I certainly understand Jesse's point about 32-bit being legacy hardware, there are still many use cases where 32-bit and current enterprise quality software and OS are necessary. Many current Linux apps are still very light and can run very well on rather old hardware, both in the server room and on the desktop.

I've done a lot of support of government servers and they run for about forever, as in until they serve no further use. Even retired, old servers are often repurposed and put back into service due to budget restrictions and/or long lead times to order new equipment under the required procedures for government procurement. In the United States this is especially true at the state level. When a server is repurposed it is usually reloaded with the current enterprise standard Linux distrubution release and applications, not legacy releases. That's one common use case.

Non-profits and small businesses often get by with older equipment as well, and in the case of non-profits it may even be donated second hand equipment that was no longer useful in it's former commercial enterprise home. Once again, a 32-bit OS and current software makes sense in cases like this.

My personal hope is that the free enterprise Linux clones will take Red Hat's 64-bit sources and create a 32-bit version. It isn't hard to do but it is time consuming. CentOS has already made clear they will release a 32-bit build(see comment by developer Johnny Hughes below), which leaves Scientific Linux and Springdale Linux.

[Note: This article was expanded from my comments on DistroWatch Weekly, Issue 560.]

Friday, April 25, 2014

The Lucrative Linux Job Offer I Turned Down

I haven't written about sexism in the Information Technology field in six and a half years. Any time I have ever written about gender issues, sexism or discrimination in IT, and particularly when I wrote for O'Reilly, there would be many comments by men who would get all defensive, tell me it's all in my pretty little head or if I would just get tougher and ignore it it would all go away. Others would fall back on sexist stereotypes, claiming women are just not interested in computing or are simply not as good at anything related to math, science and engineering than their male counterparts. Those who inevitably argue that the problem doesn't exist demonstrate just how pervasive the problem is with their comments. Never mind that IT is still dominated by young, white men plus a smattering of young Asian men. Other minorities and women are grossly underrepresented. Never mind that women have left IT in droves over the past 14 years and when interviewed they cite the hostile workplace that is the reality of many if not most IT shops. Nope, there's nothing wrong at all.

Last month I went through an interview process with a large company in a nearby state. I turned down what would have been the highest rate of my career as a Senior Linux Engineer. It paid about $10K more than I earned in a contract in Texas last year. Why did I walk away from an offer like that? I was told there are 400 people in IT, all male. I would have been woman #1 in 2014. When I asked about whether I might have problems in that environment the gentleman who would have been my boss said, "I'm the manager. I'm in a position to make sure you have no problems." That's when I knew that I had to walk away. If he had said, "I know my people. You won't have problems." I might still have taken the job. To think he can dictate attitudes and corporate culture told me that a lot more was wrong than even I saw in my interview process, and I had a bad feeling about this position pretty early on.

It's 2014, isn't it? I've been in IT since 1980. I have never seen an all male IT department before. Even worse: I worked for one of this company's direct competitors years ago and the ratio of men to women in my group was 60/40. Sexism in IT hasn't gotten better in recent years. It's gotten far, far worse. Still, I expect lots of comments from people, mostly men, in denial about this. After all, things are just fine for them.

I have pretty much decided to continue to build my consulting business even if it means less money in the short run than a corporate or government position. At least in my own business I have some control of the environment, even if I have to work a lot harder for every dollar.

Friday, May 24, 2013

It Seems I Won't Be Writing For Linux Advocates After All

Last week I had announced in the LXer forums that I would be a contributing author to Linux Advocates. That was followed by a post on the site announcing that I would be joining their team. I was honestly excited about this. I felt that writing for Linux Advocates would add credibility to my stories and bring me back some of the wider audience I had when I wrote for O'Reilly Media. The additional exposure would help me market my consulting business which brings Linux and FOSS solutions to businesses and organizations looking to reduce IT costs and enhance the reliability, stability and security of their IT infrastructure.

Today it became clear that I wouldn't be writing for Linux Advocates after all. I've learned a lot in the past week and I've come to the conclusion that this is for the best.

First, a number of prominent writers and developers in the Linux community tried to get me to reconsider. The big issue for them was what they saw as heavy handed moderation by Dietrich Schmitz, including banning a number of them from the site entirely. I've argued that website owners have the right to moderate and control the content on their sites. I've made clear that such editorial control is most definitely not censorship as some have claimed. The dispute between Mr. Schmitz and those who felt they were unfairly treated, including several former Linux Advocates writers, spilled over into five different threads in the LXer forums and several Google+ pages and included a great deal of rather heated language.

After reading all the comments back and forth I decided to go ahead as planned and I began an article on systemd to be published as my first post for Linux Advocates. Unfortunately little things like earning a living plus one day where I was a bit under the weather got in the way of my finishing the article until today. Following instructions sent to me by Mr. Schmitz I added myself as a contributor to the site. So far, so good.

Only then did I really read and digest everything else Mr. Schmitz wanted me to do, like setup an account in his personal domain so that I could have a Linux Advocates e-mail address. He also wanted me to install a Zemanta plugin for Chrome. I have deliberately chosen not to use Chrome (an article about that soon) and wasn't happy about that at all. I went to install the Zemanta plugin for Firefox instead and was presented with a Microsoft-style End User License Agreement (EULA). As many in the open source community would expect, that set off all sorts of red flags for me.

In the seven years since my identity was stolen and used for criminal purposes I've become increasingly paranoid about data security. I already feel I've left way too much information about myself out on the net. I'm very concerned about what companies like Google, Facebook and Microsoft do when they collect data about me. I'm very concerned about who they might sell or give that data to. Suddenly, I found myself reading a license agreement for a proprietary piece of software that explicitly had terms relating to collecting and retaining data.

I felt deeply uncomfortable and didn't go through with the installation. It's one thing to be forced to use proprietary software to service a client or do work required by an employer. It's quite another for someone who is getting my labor at no cost and benefiting from it to demand I install something on my own system. I first asked if Zemanta was mandatory and then wrote a follow-up e-mail making clear that I wasn't about to install it.

Mr. Schmitz' response was direct and to the point. If I can't accommodate how he chooses to run his site then I should go elsewhere. Once again, he was getting writing from me on a voluntary basis on a website were he is currently begging for money to make ends meet. This is a Linux advocacy site. You'd think he'd be the one to accommodate an aversion to proprietary tools that aren't in any way necessary for him to publish my writing. I guess not.

So.. no, sorry, Mr. Schmitz, I won't be accommodating you. I'll find ways to bring traffic to my blog which don't require sacrificing my security, privacy or principles. I still have other outlets who would like me to write for them as well. I wish Dietrich Schmitz all the best with his website. I just won't be a part of it.

Friday, May 17, 2013

Linux, Standards and the Enterprise: Why Red Hat Enterprise Linux Remains the Best Choice

Dietrich Schmitz, writing for the Linux Advocates website, posted an article yesterday about how Red Hat's adherence to the Linux Standards Base (LSB) guarantees stability and reduces costs in the enterprise. While I agree with Mr. Schmitz wholeheartedly, from my perspective the reasons by Red Hat Enterprise Linux remains both the leader and the best choice in business, government and non-profit spaces goes far beyond the LSB.

I've been a professional UNIX/Linux systems administrator for 18 years now. I've had to implement, maintain and support servers from all of the enterprise distributions and a few distributions not generally used in the enterprise as well for my employers and customers over the years. I'm a big advocate for Red Hat and the various free clones (CentOS, Scientific Linux and Springdale Linux) as the best solution for most organizations. First, it's exceptionally stable as Mr. Schmitz points out. Second, it offers the longest support period at 10 years. Third, they have excellent and professional support.* Fourth, they do the best job at insuring compatibility with both FOSS and commercial apps during the full 10 year release cycle.

My big issue with SUSE Linux Enterprise (SLES/SLED) is that they do push major version changes of the kernel, tool chain and apps in what they euphemistically call Service Packs. The Service Packs are actually major releases recently and have been known to cause major breakage and pain. My experience with their support organization here in the U.S. has been less than satisfactory, particularly the time needed to respond to and resolve issues.

Canonical (Ubuntu) has much shorter support periods than either Red Hat or SUSE. They also don't backport additional hardware support or patches into their kernel, forcing you to either do the SUSE style update gamble even more frequently than with SUSE or else to run without needed support and/or vulnerabilities.

The free clones of Red Hat Enterprise Linux I mentioned earlier are not permitted to name their source, referring merely to "the upstream provider," but pretty much everyone in the Linux community knows precisely what they mean. They represent a real advantage to Red Hat (the distribution if not the business) in that they allow businesses to try before they buy. They provide the opportunity to run a test bed or non-critical system at reduced cost. The clones also allow non-profits and cash strapped small businesses to forgo commercial support, at least for a time, and still use software that is entirely compatible with the leading enterprise Linux distribution. As organizations grow and their needs change converting a server or workstation running a clone to a genuine, supported Red Hat system is a simple process.

Finally, I'm sure fans of Debian and Slackware packaging will disagree with me, but keeping to standards, specifically the LSB, also goes a long way to insure application compatibility. I think it's vital that all enterprise distros follow standards.

*= Disclaimer: I was part of the support team for seven months as a consultant in 2005. I no longer am affiliated with Red Hat in any way, shape or form. [NOTE: This article originally appeared as a comment on in abbreviated form.]