Lies, Damned Lies, and Marketing: A Plea to Netflix

I work in IT. Not surprisingly, this means I get to do a fair amount of support for broken computers. In my case, it's mostly for a handful servers from a particular well-known vendor. Since we pay a premium for the top-level support, I tend to be pretty satisfied when calling in for failed components. The calls basically tend to consist of "What's broken?", "Let's run a quick diagnostic to make sure", "Do you want a technician or just parts?", and "Do you want it there tomorrow, or this afternoon?".

Then one day, I had to make a call in for a desktop. Same vendor, still had one of the higher level support contracts, and still quite obviously a hardware failure.

Unfortunately, this meant that instead of getting routed to a bunch of IT-savvy techs determined to keep my downtime to a minimum, I got to deal with the general home user support group.

Now, I do enough end user support to be able to sympathize with quite a bit of what these guys go through. I really don't mind them asking me really basic questions like "Is the computer on fire?"; I've had users who would neglect to mention this when asking why we turned off their Internet. I completely understand them strongly wanting to get an error code back before they'd start shipping replacement parts; I wouldn't be surprised if they've had users who didn't understand that you need to put a blank CD in before they can make a mix CD of their pirated MP3s. I won't pretend to like these things, but I understand they're necessary and don't hold it against the poor people at the other end of the line.

No, what bugs the hell out of me when they keep claiming they're "sorry". Yes, that's right, every time I talk to a new person, and every time I mention something that's a problem, they rattle off, all in one quick, unconvincing, insincere, scripted breath, "Oh-I'm-terribly-sorry-sir-I-feel-really-bad-about-that-I-hope-that-we-can-fix-the-problem-and-I'm-sorry-for-the-inconvenience" .

Oh, really? You feel personally bad about every annoying user with a broken coffee cup holder who can't tell you if it's plugged in because the power's out? Bull. After the fourth or fifth time, I'm actually far more annoyed than if you just said "Okay" and punted me off to the tech in line, because it's quite obvious that you're lying to me. I'm paying the extra support money for tech support on the product. If I wanted someone to talk to and empathize with me, I'll to find a qualified therapist and talk about my childhood, thank you very much.

Where was I going with this? Oh yes, Netflix.

As I'm sure that anyone who has a Netflix account, reads techie news sites, has an Internet connection, or uses electricity has heard by now, Netflix is removing the profiles feature, which lets you split up a single account into separate queues and preferences. This lets multiple people share a single account, rather than each buying their own - perfect for households with more than one person.

Now, the canceling of this feature is bad enough. My wife and I use this, and let me tell you, it's a lot easier than trying to come up with ratings that accommodate chick flicks, romantic comedies, sci-fi, and anime. I mean, seriously, how many people really like all of those categories?

As if that weren't bad enough, though, Netflix had to take it one more step. They decided to just give all their profile users a father-knows-best pat on the head, and tell 'em "It's for you own good." Like the tech who personally fells the pain of each and every of the thousand customers per day, Netflix has spun a falsehood that is insultingly transparent:

As a Netflix product manager I'm tasked with the wonderful job of helping members find movies they'll love. But today my job is more challenging as we've decided to terminate the profiles feature on September 1. Please know that the motivation is solely driven by keeping our service as simple and as easy to use as possible. Too many members found the feature difficult to understand and cumbersome, having to consistently log in and out of the website.

Let me get this straight. You have a feature that, while perhaps not wildly popular, is strongly loved by those who do use it. "Some" people allegedly find it "confusing" (we'll assume for the moment that Netflix has legitimate data to back this claim up), so rather than, oh, I don't know, fixing the problem, you just decide to nuke it completely. How does that "help" users?

Now, where Dad could give 5 stars to Goldfinger, Mom could give 5 stars to Pretty Woman, and Junior could give 5 stars to Shrek, Netflix will be trying to analyze a single person that would give 5 stars to all three movies. I can only imagine the bizarre recommendations for such split personality victim! How does that "help" users?

Before, each member of the household would have their own queue, and would get their own next movie for each one sent back. Now they'll have to carefully shuffle the queue each time one goes back to make sure that the right next movie goes back, or else Junior sending the cartoon he just watched back will land Julia Robert's latest movie in the mailbox. How does that "help" users?

If you're going to pull out some backend code that implements this feature, fine - but I doubt there's a software engineer on the planet who thinks it's a good idea to pull a feature away before you have something more compelling to convince your customers to give you money.

If maintaining the feature is taking up too much time, or is getting you stuck in some expensive patent war, then tell us you can't afford the feature and we'll probably understand and get over it.

But please, please, please - don't just rip the feature out of our hands and tell us it's for our own good. It's a blatant lie of the worst kind - a marketing lie - and once your customers think that you're lying to them, they're quite liable to take their money off to one of your competitors in a hurry.

You can trust me on that.

What Time Is It Anyway?

I can't believe that I feel I need to write this post. Really. But, I do, so here it goes.

Start by asking yourself a question: what time is it? A simple enough question, with a simple enough answer.

Now pretend for a moment that you had 100 people scattered around the globe on a conference call. Now ask them all, at precisely same moment, what time it is.

(Hands down, all you physics majors out there. We're ignoring relativity, since this is all make believe anyway, so everyone agrees it happens at the same instant in time.)

Now all of a sudden the answer to your question isn't quite so straightforward anymore, is it? You have to worry about dealing with multiple timezones, the international date line, and daylight savings time. The only way to deal with this is to use dates that explicitly include the timezone. Trying to deal with dates and times missing timezones is like trying to use latitudes without longitudes, or an email address without a domain. As soon as the scope expands beyond a very tiny size, it breaks down quickly.

Now, the fact that just about any standard formatted timestamp includes this information seems like it would make this pretty obvious. Email, HTTP, filesystems - they all either include a timestamp, or are universally defined as relative to a fixed timezone that you can easily base off of.

So will someone tell me why, in this day and age, the derby database chose to define timestamp columns that are missing the timezone? You wouldn't forget to make numerical types with floating point support, would you? Or strings that didn't support storing lower case? Or... well, you get the general idea.

So come on, guys. It's a big world. Databases are all about sharing information, and these days even a modest open source project can easily be sharing between half a dozen timezones across three continents. At this point, you're just making yourselves look silly.

Successful Failures and Faulty Successes

Anyone who attempts to generate random numbers by deterministic means is, of course, living in a state of sin. — John von Neumann

So unless your job has nothing to do with IT, or you've been living under a rock somewhere out of Blackberry range, you've no doubt heard about the utterly terrifying Debian OpenSSL vulnerability which left all those vast 4096 bit private keys into effectively 15 bit keys. (For those of you who don't speak crypto, this means that the bad guys can guess your private key in about 32 thousand guesses - pretty trivial for any modern computer.

The problem, ironically enough, came about when Debian developers attempted to fix some compiler warnings about a function in OpenSSL that was using uninitialized memory. While normally a horrible idea, OpenSSL was using this as an additional source of entropy, to make the private keys it generated more random. Unfortunately, the actual result was to remove nearly all entropy, leaving only 15 bits behind from the PID.

I think it's pretty safe to say that this was a catastrophic failure. Now Debian has done an admirable job of releasing a fix to the tool that generates the weak certificates (though you still have to go back and replace already existing ones), the question remains - how in the world did OpenSSL exist in this blatant failure mode, completely undetected, for two years?

This problem is a beautiful illustration of part of why cryptography, and security in general, is so devilishly difficult to get right. In normal software testing, successes are successes, and they're good, and failures are failures, and they're bad. Simple enough, right?

When you're testing security, though, things are different. In security, you also have to make sure test for what I like to call successful failures, and faulty successes.

Take a firewall system, such as iptables or pfw, for example. Without one, if a client attempts to make a TCP connection, it expects to succeed. If the connection succeeds, the test succeeds; if it fails, the test fails. Once the firewall is in place and configured to block that connection, though, that connection damn well better fail! That counts as a successful failure - a case where you succeeded in selectively making something like a TCP connection fail in a case where it would be undesirable. Likewise, when a file is encrypted, unauthorized attempts to read it (or at least, extract meaningful data from it) by anyone without the appropriate key is expected to fail.

Likewise, you also have to test for the inverse case. Back to our firewall example, let's say that the admin carelessly mistyped the mask, leaving our service unprotected from ranges that we don't want to have access. Connection attempts will all of a sudden start succeeding where we don't want them to. We now have a faulty success. Even worse, we won't notice unless we happen to test from the tiny sliver of IP addresses that were erroneously granted access. Back in the crypto realm, this is what happened to OpenSSL. It failed to prevent success, where success means a request that should have been prevented was not blocked.

These additional twists on defining success and failures help to make testing security software and configurations devilishly difficult. The OpenSSL bug didn't cause any visible changes in the test results. Everything still compiled; the output was still valid; data was encrypted and decrypted properly; no regression tests failed.

Hopefully someone someday will figure out a more reliable way to test this kind of code than the current method of having people who've forgotten more about math than most of us ever even heard of stare at it until drops of blood appear on their foreheads. Until then, we'll just have to be ready to roll out patches, scramble passwords, and revoke certificates when the next inevitable vulnerability or system compromise happens.

If It Weren't For My Horse...

Lewis Black has this great bit he does, where he describes being out and about one day, and hearing a stray phrase from a nearby woman grab his attention like flypaper grabs a mosquito:

If it weren’t for my horse, I wouldn’t have spent that year in college.

Well, today, while doing a little research on network monitoring packages to see what's new out there, I ended up taking a peek at how Big Brother is doing these days. On their page, in the License section, lay another equally enigmatic phrase:

Big Brother is distributed under our Better than Free license. Clause 2 from that license determines whether you need to buy a Commercial license.

I'm sorry - what? It's better than free, but I have to worry about buying (as in, non-free) a license in some conditions. I really have no idea of how to reconcile those two sentences together without application of drugs. I recommend you don't try too hard, either, or you risk at minimum a migraine.

Let's just look at the first half. Better than free? So unless they're paying me to use their software, then not only is it free, but they throw in something else, too, like cake. Now, it is true that you can download and start using the software at no charge, but there are some strings attached. To be fair, let's compare their version of free to that famous poster child of free software, the GPL.

	Better than Free	GPL
Duration	30 days, and then you have to buy a commercial license.	Perpetual, until the terms are violated.
Source Code	Only if they feel like giving it to you.	If you obtain a copy of a GPL binary, you are entitled to the source code that generated it.
Derivative Works	Standard commercial no modifications, no reverse engineering, "hands off!" clause.	Ensuring that each user has full rights to create, modify, and distribute derivative works is the entire purpose of the GPL.
Termination	License is terminated on breach, or also with a 30 day notification on the web site.	Only on breach of terms.

Uh-huh. So you've only got 30 days, you're not guaranteed the right to delve into and modify the software, you can't give it or any changes you make to anyone else, and they can change the terms on you whenever they like. And no cake.

If that's "better", then I'll stick with plain old GPL and boring old "just free".

Microsoft and "Vista Ready"

There have already been oodles of articles out there talking about the fiasco that is the "Vista Ready" vs "Vista Capable" fiasco. Simply put, there's evidence (including internal emails) that Microsoft lowered the standards required to meet Vista Capable. This resulted in machines with the Capable sticker barely being able to run Vista at all, let along certain advanced features, most noticeably the Aero interface, and much user annoyance and confusion.

This gives me a good opportunity to point out a common misconception about Microsoft. One that most people have, and which leads to a great deal of confusion about why Microsoft does what it does.

The confusion, simply stated, is that people think Microsoft makes software. It doesn't.

Now, now, I know what you're all thinking. What about Windows? and Office? and SQL Server, and MS Money, and all of the other Microsoft products lining the shelves at Best Buy? Okay, so Microsoft also makes advertisements. So is it an ad company? How about a payroll company, since it pays its employees?

My point is, those boxes of bits are, when you really get right down to it, in the same category as ads and pay stubs - nothing more than a means to an end. And that end is, of course, money. (The green paper stuff, that is, not the aforementioned program.) In other words, Microsoft doesn't make software; it makes money through its expertise at making software.

So why should you as a random consumer of Microsoft care? Because each and every decision that makes will have an implied footnote, a hidden subtext that reads like a banner out of Office Space: "Is this good for the company?" Each potential action will be weighted based on how much money it makes, or loses.

Sure, there will be plenty of consideration about what's good for customers, but let's face it - if Microsoft went out of its way to screw over consumers, it would have a difficult time convincing those same customers to give it money. Beyond that, Microsoft is a big company with a lot of people in it, and no doubt quite a few of them really do try to do right by their customers.

But in the end, like every other publicly traded corporation, Microsoft has to answer to it's shareholders. And each and every decision is evaluated, not on how popular it is, or on technical merit, or if it follows standards, or even ethics - but what it does for the bottom line.

Energy

Looking back at the small bits of noise I've added to that gossip factory we call the Internet, I can see that I've only really bothered to talk about tech stuff, and somewhat esoteric bits at that. So, I've decided to change bandwagons mid-stream (to quote Eric Raymond, I like my metaphors shaken, not stirred) and talk a little bit about environmental issues.

More specifically, I'd like to talk about what is, at least in the long run, the single most important issue: energy.

Everything we do, from research, to cooking, to transportation, to taking a walk around the room requires energy. As our societies become larger and more technologically sophisticated, we require our energy be delivered both in greater total quantities, and in higher density packaging. A millennium ago livestock, crops, and farm animals were enough; a century ago modest amounts of petroleum products and electricity sufficed; now it's all we can do to keep energy production matching pace with ever increasing demand.

One of hot topics that obviously flows from this discussion is where we should be squeezing all of that energy from. So what choices do we have?

1. Oil
2. Coal
3. Wind
4. Water (hydroelectric dams)
5. Nuclear
6. Hydrogen
7. Solar

Now all of these various sources have advantages and disadvantages. Water and wind are clean and renewable, but hydrocarbons have far higher energy densities. But what happens when we step back and take a longer term view? And when I say "longer", I'm not talking about the "not one, but two quarters ahead!" view that seems to dominate most companies and public debates, but a true, seven generations out view.

Let's start with hydrogen. Hydrogen does occur naturally, but not in huge quantities. The big challenge with hydrogen isn't using it, it's creating it. Hydrogen is not an energy source, it's an energy transport. It's one solution to a huge piece of the problem, but it's not a source. Since we're only talking about energy sources, let's vote it off the island and see who's left.

1. Oil
2. Coal
3. Wind
4. Water (hydroelectric dams)
5. Nuclear
6. Solar

Let's go pick on nuclear power next. Now, if handled right, nuclear could potentially offer up quite a bit of power for quite some time. A string of properly set up breeder reactors can pass material down from one to the next, extracting additional energy at each stage. Even assuming that somehow, someone could muster up the political and financial capitol to make it happen, there is still only a finite amount of glowing rocks laying around to throw in the reactors. Once those pockets are used up, it's done, so let's throw it off our list too.

1. Oil
2. Coal
3. Wind
4. Water (hydroelectric dams)
5. Solar

Now let's go all "green" here and go after the Big Bad Carbon Producers: oil and coal. When you get right down to it, they're nothing more than dinosaur and plant extract. And where did the stored energy that we pour into our gas tanks every day come from? Why, the sun, of course, as any middle school level science textbook could show you with one of those near little diagrams with arrows pointing in circles, and a picture of the sun off to one side pumping energy into the picture of plants. So since they're really just pockets of condensed sunshine (energy-wise) let's consolidate the list further by taking those fossil fuels off.

1. Wind
2. Water (hydroelectric dams)
3. Solar

That list is getting might short, isn't it? But hey! At least what's left are all clean, renewable resources, right? But wait a minute. Wind and water are great, but what makes them move? What drives them? Or, as an actor would say, what's their motivation? Well, for wind, it's heating and cooling caused by - c'mon, guess - that's right! It's the sun again.

Water? We get energy out of water falling downhill, but something has to push that water uphill in the first place to store up that kinetic energy. More specifically, on the scale we're talking about, something has to evaporate it so it can condense into rain that lands at a higher altitude than it evaporated from. Which implies heating, which... yes, yes, it's the sun again. So, now our continually shortened list.

1. Solar

And there you have it. All of the various energy sources we argue over, discriminate between, and tweak to squeeze more out of, are either solar, concentrations of stored solar, or finite resources doomed to run out, short of mining other planets in the solar system.

Now, I'm quite aware that solar power, as it currently exists, has problems. It's output is heavily influenced by weather conditions, efficiencies are still relatively low (especially when compared against the energy in a gallon of gasoline), and it's only in the last decade or so that a solar cell could be expected to produce more energy in its entire lifetime than it took to manufacture it. It's output is also limited to electrical or simple raw heat, and again, we don't have any kind of batteries that can compare to the energy transportation and storage of petroleum products.

(Some companies are looking at ways of making hydrogen more easily used by binding it up with other elements to make it more stable, such as carbon. Which gets you volatile hydrocarbons - aka, petroleum products!)

So in the end, we really don't have much of a choice. We can take advantage of little caches of energy, stored in plutonium or crude oil. We can pick the path we take to get to solar energy, whether it's through an intermediate, such as manufacturing hydrocarbons, or direct, such as boiling water or solar cells. But in the end, the sun is really the only source of energy that's going to hang around long enough for us to pretend that it's going to last forever.

(At this point, the pedants out there will point out that eventually, the sun will let us down by expanding out and destroying the earth, rather than providing us with a gentle stream of life giving radiation. I, for one, fervently hope that the human race is around long enough to have to worry about this.)

(At Least) One Of These Things Is Not Like The Others...

I just recently read Freaknomics, an interesting book on economics with interesting ideas put forth by a guy who economists claim is more of a sociologist, while sociologists claim he's an economist. The book throws out and makes a stab at answering bizarre questions about seemingly unrelated topics, like "how are teachers like sumo wrestlers?"

So, in the same spirit, I'll start off this post with the same kind of question: What do Freakonomics, Netflix, and my last hospital visit all have in common?

Now, unless you've been stalking me, I really wouldn't expect you to guess how my last hospital visit comes into play, so I'll give you a hint. My hospital is well into the process of converting from thick, massive folders of paper records, over to digital records with a PC in every exam room. While the nurse was going through medical records and scheduling procedures, she apologized for taking so long, and complained that the software layout made no sense for her field, and obviously wasn't designed by someone who knew it.

Figured it out yet? One more hint - the challenge Netflix is currently running to find a better movie recommendation looks like it just might be won, not by some MIT team of CS majors, but by a phsycologist!

Every field, be it computer science, psychology, or medicine, has boundaries. Various ideas and concepts get sorted into the right field based on those lines. An algorithm for traversing a graph? CS. A study of the effects of a new drug? Medicine, further narrowed down by specialty.

There's a problem with those lines, though. No one told the problems we're trying to solve about them!

In the first two examples, exceptional results were found by doing work that happily straddled those lines. In the Netflix example, without psychology, he likely never would have had the insight required, and without CS, he never would have been able to actually implement it.

Likewise, in the hospital example, the fact that the software engineers who created the software weren't intimately familiar with the actual job created a system that didn't match the workflow. Instead of helping the nurses, they end up stumbling around looking for options and fighting the system.

We've all heard the joke about a bunch of blind men who stumble across an elephant, and try to figure out what it is by feeling it: "It's a snake!" "No, it's a tree trunk!" "No, it's a wall!" Well, guess what? We're all a little guilty of it now and then. It's only human to try to look for solutions within the one or two fields we're comfortable in.

So what should we do about it? Recognize that the sum of human knowledge may be sorted by the Dewey Decimal System, but it is not defined by it. Read outside of your field, and see what kind of tricks those guys who went to college in a different building may have up their sleeve. Working in a college myself, I can tell you that it's not too uncommon for someone who's sacrificed any pretense of breadth for incredible depth in one field to struggle with a problem solved decades ago in an apparently unrelated field.

And in the end, ask yourself which one you'd rather be - the guy winning a Netflix prize by fusing together two superficially unrelated fields, or the software engineer who gets yelled at because he used the wrong kind of chicken guts when divining a nurses workflow?