First, I want to underscore Shane Ross's outstanding, super-critical advice on upgrading your operating system. Although he wrote it in the context of upgrades to Leopard affecting your Final Cut Pro installation, it applies much more broadly of course. What he recommends is the right thing to do for any OS update, on either platform.
(Yo, Mac users...of whom I'm one. Following Shane's directions, the upgrade to Vista is smooth as silk. Certainly no more disruptive than OS 10.2, less disruptive than the migration to Intel architecture, and FAR less disruptive than the first OS X....which doesn't even hold a candle to the transition from System 6 to System 7, the worst OS update EVER. If you were using Macs then, you know I'm right.
BTW, this is neither a pro Vista or anti Mac observation. I'm just saying.)
Since Shane covered the OS, I'm going to focus on applications.
I don’t know if this is true, but I’ve heard that Thomas Jefferson said something along the lines that nobody who loves them should ever watch the making of sausages or democracy.
The same is generally true of software. It can be fun, but it’s always messy. My observations below don’t necessarily reflect anything I saw at the companies I worked for. I've had product manager pals of both software and hardware in a variety of industries for 10 years. They've told me stories far worse than these.
These are bad enough.
1) Nobody has tested your configuration.
Think about what you've got at the absolute core of your system. Drives and drivers. IO, QuickTime. OS, plug-ins... You think anybody inside a company has a configuration exactly like yours? I can tell you now THEY DON'T.
I'll go in reverse order, and start with the computer. Companies don't get these for free. They MIGHT get a loaner when they're testing the computer itself...but don't count on it. The absolute worst, worst WORST for this is Apple.
Which means that most third-party vendors, Windows or Mac, has to BUY EVERY configuration they want to test. Every computer, every array, every IO. What do you think the odds are that they've tested a system even a LITTLE like yours? Trust me, they probably haven't even gotten close.
No need to ask whether they’ve tested every camera and deck. You know the answer
2) Nobody has tested your workflow.
Yes, there are plenty of things every workflow has in common….and those are pretty much the only things you can guarantee have been tested: cut, copy, paste, save, open, close, capture, render, output.
Every format? Every frame rate? Every conversion between them? If you live in Europe but your software is developed in the US, I'm telling you now: PAL gets nowhere near the testing NTSC does, and yes, PAL-only bugs pop up plenty. I've never even heard of SECAM being tested...but surely somebody does, right? Right?
The other obvious thing every conscientious NLE developer tests is the export/import tango with Adobe After Effects. Beyond that, maybe yes, maybe no.
We’re mostly talking about NLEs here, but let me add a speedy note about OS software for pro users. Not registered users of the big pro software packages, which is north of a million each. I’m talking about real live editing pros – no more than a few tens of thousands, maybe a couple of hundred thousand tops, for everyone combined.
Any of you kids beta tested an OS? If it was Windows, you might have. Love those public betas, baby.
But Mac or Win, you think they’ve gone through everything you might do with their shiny new operating system?
3) Nobody has tested your features.
Shhhhh. Not every new feature is tested in every new release. The big ones, yes. One hundred percent, in every possible way they might be used? Uhm, not so much.
You'd be amazed how often you hear the phrase, "It should just work" bouncing around the software world. We don't need to test everything because "It should just work."
So let's pretend that all the new features get tested. The new ones get most of the attention. Everybody loves shiny babies. But what about the gawky adolescents known as old features? Which is to say, THE FEATURES YOU USE MOST.
Maybe the old features work with the new ones. Maybe not. Early adopters will find out the hard way if older features get broken by new ones. They do. Do you want to be the one who first tries to get it fixed?
Which brings us to,
4) New software can break your old stuff.
This one truly doesn't happen much at all, but when it does, it's heinous. I've never heard of things like entire projects getting torched, but presets? Absolutely. Keyframes? It’s happened with software I’ve bought. Thank goodness it’s rare, but it really is a heinous experience.
Don't forget: if things get hairy in your new version and you decide that the only way you can finish your project is in the old software, you're hosed. Opening new projects in an old version is a no can do.
Bailing on a new OS to go back to the old one that worked? Truly ugly, unless you followed Shane's absolutely critical advice for upgrading.
OS or app, back up everything. Trust no one.
5) Where'd I put that thing?
Unless you need a new feature or a specific old problem is fixed, adding new software is a guaranteed productivity killer.
Among the best-tested, most-reliable applications ever is Photoshop....which is the WORST application about moving features, changing toolbars, changing what's on menus. That would all be survivable if keyboard shortcuts stayed the same. Oops. Photoshop is the worst ever about this, too.
(Love ya, P-shop. I’m just saying.)
It’s just one example to illustrate the rule of Upgrade Inverse Proportion: the importance of the software is inversely proportional to the speed with which you should upgrade it.
In other words, don't upgrade if you need to get work done NOW.
6) Testers? What testers?
The industry's dirty little secret: many applications barely get any testing at all. One Very Famous Product has had outside testers in the single digits.
A product manager from the same Very Famous Company bragged to me about a release that “nearly 24” testers. Nearly?!? He clearly didn't want to just come out and tell me the real number...but he thought I should be impressed.
I was DEpressed that anyone thought this was a good idea. See numbers 1, 2, 3, 4 and 5, above.
BTW, if you're reading this, you almost certainly use this Very Famous Software.
(No, not that one. Not that one either. Yep, THAT one.)
7) There are lots of reasons to ship a product. The product being ready isn't always one of them.
Did you ever say to yourself, "Wow, who tested this stuff?"
My observation from many, many companies, is that even the best beta testers are nowhere near as diligent and thorough as the company’s internal QA (quality assurance) folks. They find a ton of problems that beta testers don't.
(You’ll also find a ton of problems they won’t. See numbers 1, 2 and 3, above.
There's even sophisticated, AI software to automate the testing of every single button and slider, as well as stability across hundreds of hours. In fact, they generate a number called MTBF, or Mean Time Between Failures. They almost know EXACTLY when and where things are going to go wrong.
Which is why I’ve seen QA folks practically chain themselves to the loading dock to try to keep software from shipping.
You should let developers off the hook a little here. I'll bte that everything you've ever "finished" -- term papers, projects for clients, mopping the kitchen -- could have benefited from more "finishing." Sometimes you have to say "enough already" and move on.
As they say in the game, "Shipping is a feature." Maybe shipping is what will give the company money to actually finish development.
How badly do you want to find out?
8) Third parties get the final product when you do.
There’s the software itself, then there’s all the people besides you who rely on it to create drivers, plug-ins, and all that good stuff. When they ship their compatible products, they need them to actually BE compatible.That’s why the REAL testing for those applications begins when the shrinkwrapped version of the software or hardware hits the streets. There's simply no other way development and testing can happen with a realistic chance of success.
Yet another reason to avoid overly-early adoption.
[edit, 10/30] A number of sites (here's one) report that Apple first released its developer build on 10/26....the same day YOU got it.
As mentioned above, there are obvious exceptions, but not many...and for Apple, far fewer than with other companies. Not passing judgment here, because it really is true for everyone. I’m just saying.
9) They don't always tell you all the bugs they know about. But they tell you plenty.
This one isn't as bad as it sounds. There really truly are edge cases, with combinations of genuinely obscure circumstances. They hope that you don't have them. Odds are better than even that it's not in the release notes.
There are other bugs that they hope to have fixed not long after shipping. The odds are about even you'll see it in the release notes.
If you don’t read the release notes, thoroughly, TWICE, before you even THINK about installing an upgrade – well, as my man Scooby Doo would say, “Ruh-roh.”
The truth, the whole truth and nothing but the truth.
The actual process is scary, but you know what? Things mostly turn out okay. Developers are almost all very smart people full of the best intentions. I know it can seem hard to believe when you’re on the receiving end, but they have more at stake in the product’s success than you do.
Upgrades usually go fine. As long as you’re only changing one thing at once.
Upgrading an OS, though? That's a whole lot of moving targets to hit.
You can avoid almost all of the pain by doing two things. The first is following Shane’s advice to the letter.
The other way to avoid the pain? Keep reading the Cow, and learn from all the people who’ve ignored all this.
PS. I'm about to install Leopard, and install Vista into Boot Camp. Two new OSes on the same day, on the same computer. Woo-hoo!!! Can't wait. Obviously. Stay tuned.
Digg it