If It Weren't For My Horse...

Lewis Black has this great bit he does, where he describes being out and about one day, and hearing a stray phrase from a nearby woman grab his attention like flypaper grabs a mosquito:

If it weren’t for my horse, I wouldn’t have spent that year in college.

Well, today, while doing a little research on network monitoring packages to see what's new out there, I ended up taking a peek at how Big Brother is doing these days. On their page, in the License section, lay another equally enigmatic phrase:

Big Brother is distributed under our Better than Free license. Clause 2 from that license determines whether you need to buy a Commercial license.

I'm sorry - what? It's better than free, but I have to worry about buying (as in, non-free) a license in some conditions. I really have no idea of how to reconcile those two sentences together without application of drugs. I recommend you don't try too hard, either, or you risk at minimum a migraine.

Let's just look at the first half. Better than free? So unless they're paying me to use their software, then not only is it free, but they throw in something else, too, like cake. Now, it is true that you can download and start using the software at no charge, but there are some strings attached. To be fair, let's compare their version of free to that famous poster child of free software, the GPL.

	Better than Free	GPL
Duration	30 days, and then you have to buy a commercial license.	Perpetual, until the terms are violated.
Source Code	Only if they feel like giving it to you.	If you obtain a copy of a GPL binary, you are entitled to the source code that generated it.
Derivative Works	Standard commercial no modifications, no reverse engineering, "hands off!" clause.	Ensuring that each user has full rights to create, modify, and distribute derivative works is the entire purpose of the GPL.
Termination	License is terminated on breach, or also with a 30 day notification on the web site.	Only on breach of terms.

Uh-huh. So you've only got 30 days, you're not guaranteed the right to delve into and modify the software, you can't give it or any changes you make to anyone else, and they can change the terms on you whenever they like. And no cake.

If that's "better", then I'll stick with plain old GPL and boring old "just free".

Microsoft and "Vista Ready"

There have already been oodles of articles out there talking about the fiasco that is the "Vista Ready" vs "Vista Capable" fiasco. Simply put, there's evidence (including internal emails) that Microsoft lowered the standards required to meet Vista Capable. This resulted in machines with the Capable sticker barely being able to run Vista at all, let along certain advanced features, most noticeably the Aero interface, and much user annoyance and confusion.

This gives me a good opportunity to point out a common misconception about Microsoft. One that most people have, and which leads to a great deal of confusion about why Microsoft does what it does.

The confusion, simply stated, is that people think Microsoft makes software. It doesn't.

Now, now, I know what you're all thinking. What about Windows? and Office? and SQL Server, and MS Money, and all of the other Microsoft products lining the shelves at Best Buy? Okay, so Microsoft also makes advertisements. So is it an ad company? How about a payroll company, since it pays its employees?

My point is, those boxes of bits are, when you really get right down to it, in the same category as ads and pay stubs - nothing more than a means to an end. And that end is, of course, money. (The green paper stuff, that is, not the aforementioned program.) In other words, Microsoft doesn't make software; it makes money through its expertise at making software.

So why should you as a random consumer of Microsoft care? Because each and every decision that makes will have an implied footnote, a hidden subtext that reads like a banner out of Office Space: "Is this good for the company?" Each potential action will be weighted based on how much money it makes, or loses.

Sure, there will be plenty of consideration about what's good for customers, but let's face it - if Microsoft went out of its way to screw over consumers, it would have a difficult time convincing those same customers to give it money. Beyond that, Microsoft is a big company with a lot of people in it, and no doubt quite a few of them really do try to do right by their customers.

But in the end, like every other publicly traded corporation, Microsoft has to answer to it's shareholders. And each and every decision is evaluated, not on how popular it is, or on technical merit, or if it follows standards, or even ethics - but what it does for the bottom line.

Energy

Looking back at the small bits of noise I've added to that gossip factory we call the Internet, I can see that I've only really bothered to talk about tech stuff, and somewhat esoteric bits at that. So, I've decided to change bandwagons mid-stream (to quote Eric Raymond, I like my metaphors shaken, not stirred) and talk a little bit about environmental issues.

More specifically, I'd like to talk about what is, at least in the long run, the single most important issue: energy.

Everything we do, from research, to cooking, to transportation, to taking a walk around the room requires energy. As our societies become larger and more technologically sophisticated, we require our energy be delivered both in greater total quantities, and in higher density packaging. A millennium ago livestock, crops, and farm animals were enough; a century ago modest amounts of petroleum products and electricity sufficed; now it's all we can do to keep energy production matching pace with ever increasing demand.

One of hot topics that obviously flows from this discussion is where we should be squeezing all of that energy from. So what choices do we have?

1. Oil
2. Coal
3. Wind
4. Water (hydroelectric dams)
5. Nuclear
6. Hydrogen
7. Solar

Now all of these various sources have advantages and disadvantages. Water and wind are clean and renewable, but hydrocarbons have far higher energy densities. But what happens when we step back and take a longer term view? And when I say "longer", I'm not talking about the "not one, but two quarters ahead!" view that seems to dominate most companies and public debates, but a true, seven generations out view.

Let's start with hydrogen. Hydrogen does occur naturally, but not in huge quantities. The big challenge with hydrogen isn't using it, it's creating it. Hydrogen is not an energy source, it's an energy transport. It's one solution to a huge piece of the problem, but it's not a source. Since we're only talking about energy sources, let's vote it off the island and see who's left.

1. Oil
2. Coal
3. Wind
4. Water (hydroelectric dams)
5. Nuclear
6. Solar

Let's go pick on nuclear power next. Now, if handled right, nuclear could potentially offer up quite a bit of power for quite some time. A string of properly set up breeder reactors can pass material down from one to the next, extracting additional energy at each stage. Even assuming that somehow, someone could muster up the political and financial capitol to make it happen, there is still only a finite amount of glowing rocks laying around to throw in the reactors. Once those pockets are used up, it's done, so let's throw it off our list too.

1. Oil
2. Coal
3. Wind
4. Water (hydroelectric dams)
5. Solar

Now let's go all "green" here and go after the Big Bad Carbon Producers: oil and coal. When you get right down to it, they're nothing more than dinosaur and plant extract. And where did the stored energy that we pour into our gas tanks every day come from? Why, the sun, of course, as any middle school level science textbook could show you with one of those near little diagrams with arrows pointing in circles, and a picture of the sun off to one side pumping energy into the picture of plants. So since they're really just pockets of condensed sunshine (energy-wise) let's consolidate the list further by taking those fossil fuels off.

1. Wind
2. Water (hydroelectric dams)
3. Solar

That list is getting might short, isn't it? But hey! At least what's left are all clean, renewable resources, right? But wait a minute. Wind and water are great, but what makes them move? What drives them? Or, as an actor would say, what's their motivation? Well, for wind, it's heating and cooling caused by - c'mon, guess - that's right! It's the sun again.

Water? We get energy out of water falling downhill, but something has to push that water uphill in the first place to store up that kinetic energy. More specifically, on the scale we're talking about, something has to evaporate it so it can condense into rain that lands at a higher altitude than it evaporated from. Which implies heating, which... yes, yes, it's the sun again. So, now our continually shortened list.

1. Solar

And there you have it. All of the various energy sources we argue over, discriminate between, and tweak to squeeze more out of, are either solar, concentrations of stored solar, or finite resources doomed to run out, short of mining other planets in the solar system.

Now, I'm quite aware that solar power, as it currently exists, has problems. It's output is heavily influenced by weather conditions, efficiencies are still relatively low (especially when compared against the energy in a gallon of gasoline), and it's only in the last decade or so that a solar cell could be expected to produce more energy in its entire lifetime than it took to manufacture it. It's output is also limited to electrical or simple raw heat, and again, we don't have any kind of batteries that can compare to the energy transportation and storage of petroleum products.

(Some companies are looking at ways of making hydrogen more easily used by binding it up with other elements to make it more stable, such as carbon. Which gets you volatile hydrocarbons - aka, petroleum products!)

So in the end, we really don't have much of a choice. We can take advantage of little caches of energy, stored in plutonium or crude oil. We can pick the path we take to get to solar energy, whether it's through an intermediate, such as manufacturing hydrocarbons, or direct, such as boiling water or solar cells. But in the end, the sun is really the only source of energy that's going to hang around long enough for us to pretend that it's going to last forever.

(At this point, the pedants out there will point out that eventually, the sun will let us down by expanding out and destroying the earth, rather than providing us with a gentle stream of life giving radiation. I, for one, fervently hope that the human race is around long enough to have to worry about this.)

(At Least) One Of These Things Is Not Like The Others...

I just recently read Freaknomics, an interesting book on economics with interesting ideas put forth by a guy who economists claim is more of a sociologist, while sociologists claim he's an economist. The book throws out and makes a stab at answering bizarre questions about seemingly unrelated topics, like "how are teachers like sumo wrestlers?"

So, in the same spirit, I'll start off this post with the same kind of question: What do Freakonomics, Netflix, and my last hospital visit all have in common?

Now, unless you've been stalking me, I really wouldn't expect you to guess how my last hospital visit comes into play, so I'll give you a hint. My hospital is well into the process of converting from thick, massive folders of paper records, over to digital records with a PC in every exam room. While the nurse was going through medical records and scheduling procedures, she apologized for taking so long, and complained that the software layout made no sense for her field, and obviously wasn't designed by someone who knew it.

Figured it out yet? One more hint - the challenge Netflix is currently running to find a better movie recommendation looks like it just might be won, not by some MIT team of CS majors, but by a phsycologist!

Every field, be it computer science, psychology, or medicine, has boundaries. Various ideas and concepts get sorted into the right field based on those lines. An algorithm for traversing a graph? CS. A study of the effects of a new drug? Medicine, further narrowed down by specialty.

There's a problem with those lines, though. No one told the problems we're trying to solve about them!

In the first two examples, exceptional results were found by doing work that happily straddled those lines. In the Netflix example, without psychology, he likely never would have had the insight required, and without CS, he never would have been able to actually implement it.

Likewise, in the hospital example, the fact that the software engineers who created the software weren't intimately familiar with the actual job created a system that didn't match the workflow. Instead of helping the nurses, they end up stumbling around looking for options and fighting the system.

We've all heard the joke about a bunch of blind men who stumble across an elephant, and try to figure out what it is by feeling it: "It's a snake!" "No, it's a tree trunk!" "No, it's a wall!" Well, guess what? We're all a little guilty of it now and then. It's only human to try to look for solutions within the one or two fields we're comfortable in.

So what should we do about it? Recognize that the sum of human knowledge may be sorted by the Dewey Decimal System, but it is not defined by it. Read outside of your field, and see what kind of tricks those guys who went to college in a different building may have up their sleeve. Working in a college myself, I can tell you that it's not too uncommon for someone who's sacrificed any pretense of breadth for incredible depth in one field to struggle with a problem solved decades ago in an apparently unrelated field.

And in the end, ask yourself which one you'd rather be - the guy winning a Netflix prize by fusing together two superficially unrelated fields, or the software engineer who gets yelled at because he used the wrong kind of chicken guts when divining a nurses workflow?

Windows World (Slowly) Learning From Unix History

The excellent Coding Horror blog has a short article up about one way of categorizing software: UsWare vs. ThemWare. The idea is simple enough. ThemWare is software that's only used by "Them" - ie, none of the users are also developers of it. UsWare is software that is used by the developers as well as others.

Jeff comes to the conclusion - which I happen to agree with 100% - that creating software as UsWare will, all other things being equal, lead to vastly higher quality than software created as ThemWare. To help this process along, he encourages his software developer readers to work to gain the user perspective, to eat their own dogfood. This is certainly a good idea.

But as I thought about it a bit, I realized that this is only half the picture. The focus here is to give the programmers more of a user perspective so they meet user needs better. But what if things could go the other way? What if users could get more of a programmer perspective, so they actually could communicate their needs effectively? And maybe, in the case of users who have some programming experience, be allowed to help out and contribute bits of code that demonstrate what they want with far more precision than any prose description. Either way, the end result is to break down barriers, and blur the line between developer and user.

Oh, wait. There's a name for that already - open source.

That phenomenon called open source software hasn't really caught on too strongly in the Windows world, in no small part because Microsoft does everything in its power to keep all of its source code under heavy lock and key. With how much Microsoft depends on license keys to enforce paying for software, there really isn't much of an alternative for them. Even more important, I believe, is the fact that Microsoft began from square zero by selling software to non-programmers. The people using those original DOS systems didn't want computers for their own sake, they just wanted them to run their business.

In the Unix world, however, things began completely different. While Microsoft was busy trying to sell computers to people who didn't want to know anything more about them than they had to, Unix was a programmers playground. Researchers used Unix, and often had to create their own applications, and were able to with compilers being commonplace on Unix systems. Unix was an environment created by programmers for programmers, and the result is that once you begin to feel a little comfortable as a Unix user, the bar to becoming a Unix programmer is fairly low.

As the Free Software and OSS movements had propelled Linux systems as the successor to the Unix heritage, this trend has only become more pronounced. These days, a typical Linux system will have two or three programmer-friendly editors, an IDE, compilers for C, C++, and possibly Fortran, lisp (if you count Emacs), java (Sun, or an open source alternative), and a handful of powerful scripting languages such as Perl, Python, and Ruby.

And that's just the typical stuff! For the Linux user truly interested in becoming a programmer, there are debuggers, Ada, Smalltalk, Rexx, Haskell, and countless other languages and development aids just a Freshmeat search away. With all those tools just waiting to be picked up, each and every open source user is a potential contributor, of anything from a bug fix, to feature enhancement, to documentation, all the way up to becoming a full fledged maintainer.

Jeff is absolutely right that programmers who learn what it's like to be users will end up producing higher quality software. But as long as you freeze out your users from becoming contributors, you're throwing away valuable resources that you often couldn't buy if you wanted to. And that's why Linux will always have an edge over Windows, no matter how many animations they add to Aero.

Plan For Failure

Vista "enhancements" include removing the ability to do repair installs. Screw Windows up a little too badly, and your only option is to reformat and reinstall.

Rim has an undisclosed problem with servers off in Canada, and suddenly every Blackberry everywhere goes offline.

Congress starts ramping up surveillance and blanket data retention, but never seems to worry about the fact that those same tools are equally useful for criminals.

What do these three disparate events all have in common? Simple. All of the design was built around what happens when things go right, not wrong. All three cases display a horrific lack of preemptive failure analysis.

Failure analysis is something that is taught to more established professions, such as mechanical or civil engineering. In these professions, where a screw up frequently can mean people die, worrying about when happens when - not if - something breaks is beaten into students until they think about it the way a deep sea diver thinks about his air supply.

When a civil engineer designs a bridge, he can easily end up putting in thousands of pieces. Some pieces, when they fail, are rather unimportant. If the dedication plaque rusts or falls off, a donor may be upset, but the operation of the bridge isn't compromised. On the other hand, if a rivet or weld holding a support in place cracks, then the engineer who signed off on the design is going to be very interested in what will happen. Will the bridge hold for a year? Six months? A day?

Every part has an MTBF. Just as important as knowing when that part is likely to fail is every bit as important as knowing what will happen when it does fail. Often times, an early analysis can find hidden critical dependencies that can be fixed or mitigated with simple design changes.

Take the Vista removal of recovery restores. Strictly speaking, removing this feature didn't add any failure modes. Unlike a new driver or filesystem, it didn't add any new ways for an existing Windows system to break. What it does, is ensure that once a failure beyond a threshold does happen, the impact will go from being recoverable, to being a death sentence for that copy of Windows. Without adding any new failure modes, the number of critical failures just went up.

Now if you ask the people who put these systems together, I highly doubt that they intended for these systems to fail. This seems obvious... But it's also the problem.

Every system out there will have a failure sooner or later. Let's be fair to Microsoft, by giving them a plus side. All Blackberries have their data go through Rim servers, despite having a perfectly good data connection from the cell provider. This adds a wonderful single point of failure. By contrast, Microsoft based smart phones don't need any such assistance. They're perfectly capable of talking on their own, without an extra translator.

Microsoft could take their entire infrastructure offline, and the phones wouldn't care. By keeping their own servers out of the data path, they've reduced the number of failure modes of Windows Mobile phones out in the wild.

If we programmers and IT guys want to be taken seriously, we absolutely have to start planning for failure. Throwing redundant servers at problems reduces the likelihood of failure, but doesn't reduce it to zero. RAID protects you against a single hard drive failure, but not multiples.

We have to start asking ourselves, with each and every component we build or install, what will happen when this system breaks? That's how you notice things like a pair of high end servers both plugged into the same $4.95 ValuePak power strip. That's how you put in exception handlers that, when that exception that can't possibly happen happens, at least ensure the program goes down gracefully instead of exploding with a corrupted database.

That's how we can start building systems where a single, simple stupid failure doesn't turn into a headline generating, career limiting fiasco. Then maybe those civil and ME guys will stop snickering whenever one of us calls himself a software "engineer".

WTF is Google Thinking?

Google. The projects they do, the reactions they provoke, even the cooking in the cafeteria - whatever they do, almost always ends up being big. Unfortunately, with their latest "It seemed like a good idea at the time!" they're most likely about to piss of even more IT staff than when Google Desktop started copying files onto Google servers indiscriminately.

The description from the press release sounds innocuous enough:

Google (NASDAQ: GOOG) today announced Google Apps Team Edition as the simplest and fastest way for groups of employees and students to collaborate within an organization using Google Apps.

But then they go on:

Once users verify their business or school email address, they can instantly share documents and calendars securely without burdening IT for support.

ARS Technica had it right when they described this as Google trying to "sneak Team Edition suite past IT help desk". To those IT help desks Google is referring to, this is roughly like working to bring new an exciting drugs to market without burdening the FDA, or opening a new restaurant without burdening those poor health inspectors.

The problem is, Google is offering to host some set of end user data, but those end users quite simply lack the ability to evaluate whether or not Google is a suitable custodian of that data. Random end users shouldn't be expected to make those kinds of evaluations on their own. After all, why should an accountant worry about going over technical details of colocation and outsourcing details, such as key escrow management, encryption, etc, when you already have an IT department to worry about them?

In any decent sized company, this is how things are supposed to work. The business side of the house sets the priorities, then passes the goals and requirements off to the IT of the house, who picks the best solution on suitability and technical merit. Management sets the why and what, IT decides the how.

Google, on the other hand, appears to be trying to take that away. Now, I'll be the first to say that expanding the online Google tool suite is great. And adding in collaboration features is a pretty obvious next step.

But damnit all, Google has a responsibility to make sure this loaded gun is at least pointed in the right direction! If you want to sell liquor, fine - but that doesn't mean you should open up shop across the street from a high school. The last story that I heard of where users decided to go off and create a working solution on their own, the end results included an SSL free commerce web site and credit card numbers were tossed around in plain text email to be typed in. Collaboration definitely sounds like a powerful tool in the right hands, but IT still has to have a prominent role in picking which tool to use and how to use it.

Now I'm sure that the good folks at Google never intended to have sensitive data, like business plans or credit card numbers, passed around. The problem is, to an ordinary user, only moderately technically literate, the only difference between storing that top secret business plan on a secured server and Google docs is which bookmark they click on.

In a a managed corporate IT environment, the IT and business sides of the house have a close working relationship. The IT side understands enough of the business side to create a working system. At Boeing, the IT staff understand that plans for new airplanes are highly sensitive, and so can set up servers and encryption to protect it, and train users in how to use it to protect data. With Google, however, you get what they offer, and that's it. If Google apps doesn't meet your needs, you either end up with a hole that Google apps can't fill, or even worse, leaving data inadequately protected.

So the next time that someone who has no chance of understanding the implications of the fine print in the acceptable use policy goes off and leaks the company crown jewels by clicking the wrong checkbox in a Google app, will Google accept any of the blame? Or even more importantly, any of the responsibility of cleaning up the resulting mess? Tracing the extent of data leaks? Buying credit protection for identity theft victims?

Somehow I suspect that Google won't mind burdening the IT help desk with that half of the job.