Swatting (computer security) flies with a bazooka

In every field, there are some bad ideas that just won't die. In medicine, you have ideas such as curing cancer by pushing bones around with chiropracty. In the audio world, people get suckered into searching for "oxygen free" cables to somehow make that song, recorded in 1973 onto an 8 track, sound perfect. In the field of computer security, one of my personal favorite bad ideas is to make a magic virus that, instead of going around doing bad things, will slip in through existing security holes and clean out other viruses, install patches, and turn on firewall rules for the poor uneducated users. Dan Geer is the latest one to tie some strings onto this zombie and make it dance around.

Now, to someone who's only vaguely familiar with computers, or is used to dealing with one or two systems, this may sound like a good idea. I mean, who wouldn't love to have a magic program swoop in and clean house like an uber-l33t Mary Poppins with a keyboard? For added fun, Dan has added in an extra twist. Rather than the usual infection vector of scanning around the network (just like the viruses that people don't like), Dan proposes that secure web sites should ask users if they want security. If they say "yes", pretend that they're obviously competent and should be trusted, and run as normal. If they say "no", pretend that they're obviously idiots and fling the magic pixie dust back at their computer that keeps the Big Bad Scary Hackers hiding under the bed.

Let's start with us poor slobs stuck actually supporting the reality of computers, often in the hundreds or thousands, rather than in the idealistic realm pundits love to live in. To us, the idea of some random local bank or knick-knack vendor actually running arbitrary code on machines we have to keep going is downright terrifying. Writing this kind of code is hard - really hard. Don't believe me? Just ask Microsoft, who managed to release a silent, unblockable patch to the automatic update system that in some casesstopped updates from being installed. And that was an update only applied to Windows XP machines at a minimum patch level - imagine trying to make something so complex perfectly reliable and secure on all patch levels of Windows 98, 2000, XP, and Vista, not to mention Mac OS and Linux!

What shall we poke at next? I know! How about assumption that this code that gets downloaded to the poor computer is somehow safe itself? I mean, the whole purpose of this magic program is to make things safe on already infected machines - easy, right? Hah! Just ask the folks who spent millions creating the content protection scheme used in Blu-Ray about the impenitrability of BD+. (I'll give you a hint - it's been cracked.) Fundamental computer security 101 - once the OS is compromised, it's pretty much game over for any other programs running on it. Half the viruses out there disable the most popular virus scanners; if this magical security bit becomes at all popular, there's no reason to think it won't be targeted as well.

Okay, I think we have time for one more, so let's make it a good one. Let's assume for a moment that Mr. Geer manages to hire Tinkerbell, ensuring an adequate supply of pixie dust to the magical program work as designed. How much would that actually accomplish?

• That single transaction - secured.
• Any time the user logs into another site with the same password - unsecured!
• Executables sent in email or instant message links - unsecured!
• Phishing emails telling users to type their passwords into malicious sites - unsecured!
• Malicious sites lurking on common typos of legitimate domains - unsecured!
• Users picking bad passwords - unsecured!

I could go on listing other things that this idea wouldn't protect against, but I think you can see the pattern. Even if this idea could somehow be made to work completely properly, it's pretty doubtful that it would make a substantial dent in the problem of securing the computers of unskilled end users.

Well, it's time for me to head off to bed, so I'll leave you with this closing thought. Let's stick with the assumption for a moment that someone does come up with some magic <make-it-secure> HTML tag you can stick onto any web page. Instead of trying to use a yes/no dialog box as an ouija board to guess whether the computer is secure or not, why not just use the damn thing on every sensitive page?

Too much logging is almost enough

Not too long ago, I stumbled across Log Everything All the Time on the site High Scalability. The short version is that the classic method of turning on debug logging when needed to track down a problem is useless because it implies that logging was off when the problem actually happened. Anyone who's dealt with an intermittent problem (only happens two or three times day, or one out of a thousand transactions, or if at least six people are wearing blue jackets) in a complex system can attest to the futility of trying to divine which combination of the tens or hundreds of thousands of events flowing by per day actually blew up. This is the deadly and elusive Heisenbug, an easily startled creature which runs away from the debug switch like a cockroach from a light switch. This particular article was written from the perspective of writing a large scale application. Building a system in this way has the huge advantage that you (where you is the programmer) get to drop debug and analysis code in at any point in the program flow. Suspect that data structures aren't getting initialized right? Print out a dump of the data structure right before the initialization routine sends it back. Garbage strings showing up in the database? Trace the entire lifecycle of the data that go into it, with a before and after every piece of code that touches them. Now that's all well and good for you lucky guys are busy actually building systems on general purpose computers from code up. But what about us poor schmucks who are running the networks that let those web 2.0 apps fling AJAX across the globe? By comparison, we're fighting with one hand tied behind our back while blindfolded.

• Most network devices are, for all intents and purposes, embedded devices. This means the code running on them is far more rigidly fixed than on, say a Linux server. Open source apps can be patched and recompiled, PHP pages can have extra print statements thrown in, and even fussy vendor binaries can be spied on with strace and tweaked with library preloading. With your typical network device, though, unless you've got a really good service contract, you're pretty much stuck with whatever you've got.
• The built in even logging on most network devices tends to be a bit on the anemic side. At best, you'll typically get reports of what the device considers to be unusual events. At worst, you'll get nothing at all. I've worked with at least one switch that taunts you by by just logging when an event occurs. No details, not even what even occurred - just a timestamp.
• The state of a network is almost all transitory. Transactions on servers, even without logging, tends to leave various breadcrumbs behind - database entries, file timestamps, emails, etc. Once a state table entry for a stateful firewall expires, though, no trace is left behind.

The best remaining option is to constantly squeeze snapshots of the state of the network and track them. Then, later on, when someone comes to you with a report of a problem from three days ago, you at least stand a fighting chance of piecing together the half dozen otherwise normal events that line up to point at the real problem. For example, we once had a professor who's machine kept falling off the network over weekends. All of the usual troubleshooting techniques - look for errors on the port, test out the cabling, etc - showed everything perfectly normal. Looking through our more extensive state monitoring logs, we found two sets of interesting events, however. On Saturday morning, the port briefly lost a link, and a different MAC address started showing up on the port. Saturday evening, the link flickered again, and the original MAC address showed up again. Looking up the address showed that it belonged to a grad student working for the professor. The professor's own assistant was unplugging his machine! Without logging all of the "normal" state of the network, we'd probably never have found the real cause. So what information should you log? Here are what we log for network troubleshooting.

Switch FDB

This gives you the record of where all of your machines have been. More importantly, it can tell whether or not a machine was talking on a port at a specific time, rather than relying on a users guestimate.

ARP Tables

Utterly indispensable for catching misconfigurations. We've all had that bozo who set his IP address to the local gateway, or a nearby server. With this info, you already have a record of the offender, rather than tracking it down manually.

DHCP Leases

If you don't have fixed addresses everywhere, then without tracking DHCP leases, it's much harder to tell what machine a given IP address belonged to at a given time.

DHCP Fingerprints

By capturing extra information about DHCP lease requests, such as the option request list and the VCID, it is possible to do passive OS identification using techniques like those used by PacketFence. This can be particularly useful when tracking down disallowed or problematic devices, such as SOHO NAT routers.

IP Flows

An invaluable record of who talked to who when. You can either grab it from your routers directly with flow exports, capture it using span ports and a package like Argus, or even better, both. By capturing snapshots of IP flows from multiple points and looking for discrepancies, you can narrow down exactly which one of the three routers and five switches in the data path threw away a few packets. Also, your security guys will love this for helping to track down patient zero in virus outbreaks.

SNMP Traps

These are often your best insight into what the network devices think are going on. BGP session flapping, port security violations, failed login attempts - often times you can find events like these being generated in traps that don't even show up in the device log files.

The final thing to remember is however you end up collecting this information (along with whatever other data sources you can come up with for your network), make sure they all go into a repository where you can easily drill down and jump around between data sources. You'll want to be able to look at a switch port, find the MAC address on it, look up it's OS and IP in the DHCP data, and get a list of all other machines it's talked to in the last 12 hours. Make sure you can easily browse through your data like this, and as you start playing around with it, you'll end up finding solutions to problems you didn't even know you had.

Privacy without anonymity?

So by now, anyone who cares has no doubt heard about the comments from Donald Kerr, deputy director of national intelligence. His claim is that nowadays, the American public has no choice but to abandon the idea of anonymity, and instead simply place their faith in the government and corporations to handle their private data properly.

Give me a break! If there's a more classic example of the foxes guarding the henhouse that doesn't involve an actual farm, I certainly can't think of it.

Let's ignore for a moment the fact that so far, these foxes have proved to be horribly incapable of properly handling data. Forget for a moment that from Choicepoint selling data to thieves, to TJX utterly failing to secure their own systems, to the countless laptops and backup tapes lost or stolen, the IT industry as a whole has not given consumers much reason to sleep soundly.

An essential aspect of free speech is the ability to speak your mind without fear of reprisal. While it's a wonderful ideal to be able to simply declare you can't be punished for speaking something unpopular, the harsh reality is that sometimes it can't be avoided. Look, for example, at the whistle blower protection laws. The fact that you can report illegal activities that a company you work for without that company knowing you did the reporting is absolutely essential. Would you turn if your boss if it was going to be months or years before results happened, and he would know what you did?

Kerr's statement is the worst kind of lie - one hidden in the middle of a bundle of truths. Do we all need to adjust our expectations and behavior regarding privacy in the age of Google? Clearly. Should people be able to expect that their government and companies will take care of their data properly? Absolutely.

But to suggest that these are an acceptable substitute for anonymity is both foolish and dangerous.

DRM vs Consumers...

Ah, so it begins. For some time, consumer rights advocates have been concerned about the power that Digital Rights Management (or, as some more accurately call it, Digital Restrictions Management) grants to conent producers over consumers. Now, MLB has apparently decided that it really doesn't care about screwing over customers who bought DRM locked content.

In short, MLB has decided to change the technical details in how they protect any content they sell (in this case, videos of old games you can download and burn to disc). Who cares, right? Well, it turns out that the old DRM they use checks back in to a central server every time you go to play it. And, as part of the change, MLB sorta removed the magic server bits that convinced the player to show you what you paid for. All of those files you downloaded - and paid for! - are now just encrypted noise taking up space on your hard drive.

Oops.

I'd bet that, given the noise that's starting up, and the complete idiocy of what MLB has done, they'll bashfully find some what to at least look like they're making up for it - gift cards for screwed customers towards repurchasing the content (still under DRM, of course), or something equally feeble. A more interesting question is, what lessons will be learned from this on the industry side?

Microsoft Vista is shoving DRM features deeper and deeper into the core OS. Heck, the volume licensing arrangement already includes a component that talks to a server every 30 days, and being out of contact for too long will cripple your computer. The issues with performance, stability, application compatibility, and overall quirkiness have been enough to make resellers revolt and continue selling XP.

In both of these cases, the guys selling stuff have a de facto monopoly. No one but Microsoft gets the final say in what happens with new versions of Windows, and hanging out at your local high school baseball games just isn't the same as watching Don and Remy at Fenway. In the end, they'll be able to push through a lot of this kind of crap, and still come out reasonably well. This doesn't mean it's a good thing to build a business model around pissing off your consumers, however. Apple has had enough pressure that they're even starting to sell music without any DRM, and plenty of people have talked about Radiohead and the unencumbered downloads.

So the question now is, which lesson will the industry learn? That if you have enough momentum to not care about your customers, you can get away with DRM, or that if you offer your customers enough value, you won't need to bother with the expense of DRM in the first place?