CS Majors Need Not Apply

Usually, I like Coding Horror. I just read a post, though, where he argues that CS majors should be taught more software engineering. He quotes CS students who have never been formally exposed in their entire undergraduate program to things that the professional field lives and dies by, such as deployment management and revision control. But then, he goes one step too far, right off the cliff:

If we aren't teaching fundamental software engineering skills like deployment and source control in college today, we're teaching computer science the wrong way.

Sorry Jeff, but I'm going to have to call you on this one. If you're teaching fundamental software engineering skills like deployment and source control, we're not teaching computer science at all - we're teaching software engineering!

Think about cars, and the people who design them. Typically, they went through a Mechanical Engineering degree. While this means that they did get a solid grounding in some underlying physics, such as heat transfer, stress transfer, and material analysis, the focus is on how to apply those areas to the real world. Complicated, precise physics formulas are replaced for approximations and tables designed to quickly and easily give an answer that may be less accurate, but errs on the side of safety.

On the other end are the actual guys who do the real hard-core science - physics. Mostly done on chalkboards and computers, these guys only delve into the real world to gather data or test out a hypothesis. There are quite a few good physicists out there who could explain to you in great detail how and why a tire has a particular amount of traction on asphalt, but couldn't actually change a tire if their live depended on it.

The important point here is that even though ME and physics of the properties of physical things, there is still a distinction between the abstract, research oriented side, and the dirty, messy, practical side. This is a distinction which most of the computer "science" majors out there seem to pretend doesn't exist.

Most of true computer science doesn't even have anything to do with computers. Take Big O notation. In computer science, if an algorithm takes an hour, a day, or a mon, as long as they scale linearly as the size of the input goes up, they're all O(n). Try to argue to a customer that they should be considered equal in any way, though, is likely to make for a short career as a programmer.

The harsh reality is that most companies advertising for computer science majors don't really want computer science majors. Sure, they want someone with a good knowledge of algorithms, but - as Jeff pointed out - the ability to use version control is at least as important. Grungy skills, such as creating crash dumps that allow you to get good diagnostics info about customer problems without having to ship them custom builds, while utterly boring from a pure CS standpoint, are worth their weight in gold outside of academia.

This isn't to say that software engineers shouldn't have a grounding in CS theory. There's going to be a lot of overlap. The difference is one of focus. Once we accept that there are really two majors trying to fit into one curriculum in most schools, we can start the process of trying to make a one size fits all, and stop trying to turn out physics majors that we expect to be able to design a camshaft.

Typing Puppet Strings Onto Your Servers

Just like a good Perl programmer, a system administrator should strive for a certain degree of laziness.

Now, this is not the kind of laziness that leads one to think "Eh, I'm not going to bother installing that update." No, this is the form of efficient laziness that says "I could download and install that update, but there's got to be a way to get it done automatically without wasting my time." These are the kind of people who have libraries of shell scripts and packed cron jobs.

Now, those libraries of shell scripts are great, but they can be an awful lot of work to write and maintain. Not very lazy at all! So, rather than going that route, I've been working with (and on) a package called Puppet.

Puppet is a client/server package written in Ruby. Essentially, you configure the server with the configuration settings you want all of your machines to look like. The clients get pointed at the server, pull all of their settings down, and make them happen.

It's got a decent library of native types (such as packages, files, users, etc) right out of the box. If you need something that's not covered, it's fairly straightforward to write your own custom code (assuming you know Ruby) that allows you to extend what kinds of files and setting Puppet is able to directly manage. Thanks to some good helper libraries, I was able to whip up a custom module that allows me to manage entries in /etc/sysctl.conf is only 59 lines of code!

Some of the cooler features of Puppet:

• All communication is XML-RPC based, making it easier to write custom programs that communicate with Puppet
• Collections of facts about client systems (OS, OS version, etc) are reported back to the server and can be stored in a database
• Defines and Exec allow you to create complex configurations without writing any Ruby code
• ERb templating system (the same one of Ruby on Rails fame) allows you to generate complex configuration files with per-host settings

Ask anyone who manages big numbers of systems - hundreds, or thousands - and they'll tell you that the ability to automatically manage systems from provision to decommission without manual intervention is absolutely essential. Whether it's built in, like GPO in Windows, or an add-on package like Puppet, trying to manage any more than one or two systems without this kind of help is just making more work for yourself.

And that's not very lazy at all.

Coding and Complexity

First off, let me just make a quick confession - while my undergraduate degree was stamped with "Computer Science" as my major, I don't really consider myself to primarily be a programmer. Sure, I do actually spend a good number of my days mucking around with writing code (usually Perl, occasionally Ruby), but my job is really IT support, specifically networking. I deal with switches, routers, wireless, VPN, and a handful of Linux servers supporting the network with DNS, DHCP, etc. The code writing that I do is almost exclusively to support everything else, such as working on a host registration system or device monitoring scripts. The software I write is to directly address a need, rather than to be sold to address someone else's need.

That said, when I just read Steve Yegge's latest rant, Code's Worst Enemy, it struck a chord with me.

I happen to hold a hard-won minority opinion about code bases. In particular I believe, quite staunchly I might add, that the worst thing that can happen to a code base is size.

Now, as someone who does not consider himself a spectacular coder by any means, I would certainly feel quite daunted by tackling a 500k line codebase by myself. On the other hand, as a professional coder, Stevey ought to be able to casually fling around great swaths of code, using advanced software repositories and indexing tools, right? But no - he feels that, all other things being equal, less is more.

One feature of large code bases that I think he gave short shrift to was the idea of complexity. He talks a little bit about how complexity certainly makes a given code base harder to work on, and also that some of the automatic tools, such as refactoring, that try to deal with it just make the problem worse by bloating the code base even more.

This is something significant in his argument, I think. In this example, we have two code bases, before and after being run through the automatic refactoring tool. The initial state has a given level of functionality, size, and (for lack of a better word), "goodness". The final state greater size, and therefore less goodness, but identical functionality! This mirrors his stated goal of taking his existing game, and rewriting it with identical functionality but less than half the lines of code.

I think the explanation boils down to this: we can only fit so much in our brains at a time. Great programmers can mentally swap in more of the big picture at once, but everyone has their limit. This limit is why we decompose programs down into manageable subroutines, each of which can be understood (at least partially) in isolation from the rest. It is why we hide massive chunks of functionality behind a handful of calls into a library. The smaller chunk size we're working on, the more likely we are to be able to fully understand it and not screw up.

From here, the trick to making sense of Stevey's size argument is realizing that there are two completely different kinds of complexity at play here. If you're writing code to do, say, an FFT, you've got to know the math behind it and how it works. That's a fair bit of complexity that you've got to hold in your heard, and it's going to remain constant regardless of whether you're developing in Java, Ruby, C++, Assembly or BF.

This invariant portion of the complexity is what I call inherent complexity. (Please don't tell me if that term isn't original; I know if probably isn't, but I like to pretend.) It's the piece that you can't get away from, since it's what defines the actual problem you're trying to get that hunk of copper and silicon to solve for you. It's the tax code embodied in Quicken, the rules of mathematics in Mathematica, the graph theory in Garmin and TomTom. Remove the inherent complexity from a problem, and all you've got left is a very complex, boring video game with executables instead of high scores and compiler errors instead of health damage.

If the inherent complexity were all there was to it, then knowledge of the problem domain would be all that's required. You wouldn't need a programmer to write Mathematica, just a mathematician to sit down and tell the computer everything she knows about math. Easy, right?

Sadly (or fortunately, if you make a living as a programmer) this is not the case. The person coding has to know extra details that are outside of the problem domain, like the fact that the number 0.1 cannot be represented with absolute precision in a floating point number. Or that if you accidentally tell a computer to loop forever, it will do so. Or that each of these three different sort routines will produce the same final product, but the memory and time requirements can vary by an order of magnitude or more - and not always in the same order, depending on the data set. Not to mention nitty language details, like dealing with pointers in C or "bless" in Perl.

All of these other layers upon layer of crap that gets wrapped around the real problem is just extraneous complexity. I mean, let's be honest - learning objected oriented design or unit testing may help you write code faster and with fewer bugs, but won't help with bullet point one of the design requirements for an ERP (or online order system, or factory automation, or... ). It's all work that is, in the end, unquestionably important to creating a finished product, but any time spent working on that extraneous complexity is time not spent on the inherit complexity.

Or, to put it more bluntly, any time you spend appeasing your programming environment is time that you're not spending on solving the actual problem.

Based on this, the best development languages are ones that are fairly thin, succinct, and in general just get the hell out of your way and let you work. Go back a few decades, and compared to the alternatives of the time, this is what C was. The book that was for many years the definitive guide to C was under 300 pages long, and let the programmer almost completely ignore the messy details of things like programming in assembly. Loops and conditionals were suddenly a simple, easy mnemonic syntax.

More recently, I think this "thinness" is a huge portion of the success of Ruby on Rails. Starting from a database schema, you can literally create a functional skeleton application in minutes with just a few commands, with all of the components already laid out neatly organized and slots already created for niceties such as porting to different databases, unit testing, and version control.

Sure, it's all stuff that any competent programmer can easily handle, but automating it frees up that many more brain cells to do whatever it is the client or employer wants to give you money for.

Frank's Law of Foreign Key Constraints

While bouncing around between a handful of typical LAMP style applications, I've come to a harsh realization of a brutal truth:

Those who do not learn proper foreign key constraints are doomed to create an incomplete, buggy implementation of them in their application.

Minus 50 million points to MySQL for creating an entire generation of web programmers who have only a vague, fuzzy idea of what constraints are by shipping a version that either didn't have them, or defaulted to a table type without them, for so long.

Blacklists and You

Blacklists. Whether they're for virus signatures, firewall rules, or spam filters, every security guy who's spent more than 15 minutes in the business knows then, loves them, and hates them. Coding Horror has a mostly right article up summing it up titled, quite simply, Blacklists Don't Work.

On the one hand, all of the downsides he lists are dead on. Most of the reason that we frantically run around installing anti-virus software on Windows boxes are directly traceable to horribly shortsighted design decisions made as far back as MS-DOS. (Heck, search around, and you'll still occasionally find people having problems due to 8.3 filename restrictions!) And yes, blacklists are horribly inefficient, a royal pain the maintain, and often easily bypassed. After all, there's nothing whatsoever stopping our Evil Virus Author from taking his latest malware and running it through the dozen most popular virus scanners to make sure it slips by all of them.

But really, what are the other options? Are we to truly believe that there is some magic silver bullet waiting in the wings, parked next to the car that runs on water and an eclipse plugin that can tell when you typed ">" but meant ">="? Jeff puts forward the same idea that Microsoft has been painfully pushing in for years - forcing users to run as regular users instead of as administrators all of the time. Now, to be sure, this is absolutely something worth pursuing, both for security and general reliability issues. Ask anyone who maintains an open lab on a college campus how much fun it is trying to keep the right printer drivers installed and working when anyone can do anything they want on the machines!

Even this idea falls short, though. Most of those lab computers and corporate desktops, where you have site administrators who can hoard admin privs to themselves, aren't the real problem. Those computers are the ones with people babying them already, making sure passwords are strong, patches are up to date, and virus scanners are running. Sadly, it falls short when applied to Aunt Millie. She will gleefully open that email from her anonymous new best friend, follow the directions to open the encrypted zip virus, and do whatever is necessary to firmly embed the virus deep in her computer.

Even if you take away administrative rights, in a few months those same hackers will quickly start installing programs in My Documents, and use the same startup mechanisms that legit apps do. After all, it's not like you really need full system control to send spams or participate in a DoS attack. And if you do, once you get a program running on the computer, there are usually plenty of privilege escalation bugs and attacks that can get you the rest of the way, regardless of what level the user launched the program at.

The problem is that, as bad as they are, it's not quite fair to say unconditionally that blacklists don't work. They're slow, annoying, have lots of holes - in other words, they work quite horribly - and, like democracy, also happen to work better than any other workable solution out there right now. I'll agree 100% that we need to start building systems where security is just as important a design goal as reliability and profitability, but until we figure out a way to divine the intent of a given program, some form of blacklisting will always be with us.

Shared dedicated or dedicated shared?

I like having internet at home. Sure, it's not quite the same as multiple 30M+ pipes at work, but it's plenty fast enough to waste time on youtube and settle arguments with wikipedia. These days, most people have pretty much two options for home connections with decent speed: DSL over phone lines, or cable modem over CATV lines. (At this point, I'm not really counting FIOS yet.)

Now the primary thing that you want from an ISP is a reliable, fast internet connection. All of the other fluffy, feel good benefits like more free email addresses, little bits of web storage, etc don't really count for much if your web pages take minutes to load. One of the little canards that DSL providers love to throw that really, really bugs me is the "DSL is dedicated! Cable is shared!"

I'm a network guy. I build and maintain 'em for a living. Now, it's true that with cable modems, the bandwidth is shared per coaxial segment among all of the customers on that segment, while each DSL customer gets to use all available bandwidth on that particular dedicated pair of lines. But guess what all those dedicated lines do? That's right, they go into a set of equipment (routers and uplinks) that are - horrors! - shared.

There isn't a network on this planet that doesn't do some level of oversubscription. Cable modem providers simply have to allocate enough bandwidth to each neighborhood loop to satisfy the actual demands, just like DSL providers have to do with their aggregation points.

Now, when an ISP starts advertising with promises of no hidden bittorent filters, secret P2P filters, or anti-criticism termination clauses - in short, the things that the Net Neutrality people have been lobbying for - then I'll care.

Swatting (computer security) flies with a bazooka

In every field, there are some bad ideas that just won't die. In medicine, you have ideas such as curing cancer by pushing bones around with chiropracty. In the audio world, people get suckered into searching for "oxygen free" cables to somehow make that song, recorded in 1973 onto an 8 track, sound perfect. In the field of computer security, one of my personal favorite bad ideas is to make a magic virus that, instead of going around doing bad things, will slip in through existing security holes and clean out other viruses, install patches, and turn on firewall rules for the poor uneducated users. Dan Geer is the latest one to tie some strings onto this zombie and make it dance around.

Now, to someone who's only vaguely familiar with computers, or is used to dealing with one or two systems, this may sound like a good idea. I mean, who wouldn't love to have a magic program swoop in and clean house like an uber-l33t Mary Poppins with a keyboard? For added fun, Dan has added in an extra twist. Rather than the usual infection vector of scanning around the network (just like the viruses that people don't like), Dan proposes that secure web sites should ask users if they want security. If they say "yes", pretend that they're obviously competent and should be trusted, and run as normal. If they say "no", pretend that they're obviously idiots and fling the magic pixie dust back at their computer that keeps the Big Bad Scary Hackers hiding under the bed.

Let's start with us poor slobs stuck actually supporting the reality of computers, often in the hundreds or thousands, rather than in the idealistic realm pundits love to live in. To us, the idea of some random local bank or knick-knack vendor actually running arbitrary code on machines we have to keep going is downright terrifying. Writing this kind of code is hard - really hard. Don't believe me? Just ask Microsoft, who managed to release a silent, unblockable patch to the automatic update system that in some casesstopped updates from being installed. And that was an update only applied to Windows XP machines at a minimum patch level - imagine trying to make something so complex perfectly reliable and secure on all patch levels of Windows 98, 2000, XP, and Vista, not to mention Mac OS and Linux!

What shall we poke at next? I know! How about assumption that this code that gets downloaded to the poor computer is somehow safe itself? I mean, the whole purpose of this magic program is to make things safe on already infected machines - easy, right? Hah! Just ask the folks who spent millions creating the content protection scheme used in Blu-Ray about the impenitrability of BD+. (I'll give you a hint - it's been cracked.) Fundamental computer security 101 - once the OS is compromised, it's pretty much game over for any other programs running on it. Half the viruses out there disable the most popular virus scanners; if this magical security bit becomes at all popular, there's no reason to think it won't be targeted as well.

Okay, I think we have time for one more, so let's make it a good one. Let's assume for a moment that Mr. Geer manages to hire Tinkerbell, ensuring an adequate supply of pixie dust to the magical program work as designed. How much would that actually accomplish?

• That single transaction - secured.
• Any time the user logs into another site with the same password - unsecured!
• Executables sent in email or instant message links - unsecured!
• Phishing emails telling users to type their passwords into malicious sites - unsecured!
• Malicious sites lurking on common typos of legitimate domains - unsecured!
• Users picking bad passwords - unsecured!

I could go on listing other things that this idea wouldn't protect against, but I think you can see the pattern. Even if this idea could somehow be made to work completely properly, it's pretty doubtful that it would make a substantial dent in the problem of securing the computers of unskilled end users.

Well, it's time for me to head off to bed, so I'll leave you with this closing thought. Let's stick with the assumption for a moment that someone does come up with some magic <make-it-secure> HTML tag you can stick onto any web page. Instead of trying to use a yes/no dialog box as an ouija board to guess whether the computer is secure or not, why not just use the damn thing on every sensitive page?