In the olden days of programming, back when you had to code by candlelight, it was considered acceptable to release beta versions that contained tons of bugs. For the most part, people avoided using beta software because they knew it could bite them. The only people who tried beta versions were hard-core geeks who knew what to expect. They expected show-stopping bugs, and plenty of ’em.
Those days are over, though. The mainstreaming of software coupled with the increasing prevalence of beta versions has resulted in the widespread usage of pre-release software. People no longer consider "beta" to mean "use at your own risk." And given the years-long beta status of web applications like Gmail, who could blame them?
These days, a lot of people get their first impression of your product by trying out a beta version. It doesn’t matter if you warn people about problems – they’ll still forever think of your work as low-grade crap if your beta isn’t polished. If they run into any annoying bugs, they may steer clear of your software in the future.
So unlike the carefree code-slinging days of my youth, today I’ve got to make sure my beta versions are as solid as possible. I have to treat public betas with almost as much care as the final release.
In some ways I view this as a good thing, because it forces me to pay more attention to potential problems as I’m coding. But it’s also a bad thing, because it’s flat-out impossible to test desktop software with every possible combination of hardware and software that exists in the wild. I have to release a public beta in order to uncover bugs involving specific configurations, but quite often users who encounter these problems never report them – they simply uninstall the software and move on.
Just do the proper thing and invite power/long time users of your application into a private beta group. It worked pretty well to sort out bigger bugs in Klient – my relation to that application is being such a private beta tester and a happy user of that IRC client :)
Have you thought about bundling the betas with bug reporting software much like Firefox? What are the downsides to that, if any?
@Björn That’s basically what I’ve done with TopStyle. It first gets “previewed” by a small set of private testers before going into a public beta. The risk with this approach, though, is that private testers tend to be power users, and their needs may not match those of most customers.
@Cody The latest versions of TopStyle and FeedDemon both have built-in error reporting, and this has really helped to track down configuration-specific bugs. But again the problem is that if people encounter bugs in beta software, they may simply uninstall it and move on.
I think the best approach is to ensure your software can report on its own bugs.
The downside to the end-user, then, is slower performance (presumably, you’d have more instrumentation enabled in your beta), but as long as you’re not talking molasses-slow, that’ll be ok during the beta… plus, it’ll be one more improvement you can trumpet when you finally release! You’ve improved performance!
:)
Most of us are used to the “Can we collect anonymous statistics” question in even our production code, now… part of your beta terms could be that they don’t get to opt out of that, and even that it doesn’t have to be anonymous. Those are reasonable constraints for the opportunity to participate in a beta, I think.
Nick,
Is this not also an issue of simply doing a good job of managing expectations? Isn’t it also that now we just call what used to be called a beta an alpha?
Nick,
first:As a longtime user of your programs let me say that (almost all of) your betas are better than (= have less bugs than) standard-releases of other many companies..
second: in most cases there was a prompt response, when there were serious bugs, which means there was a corrected beta out soon
so no problem in trying your betas :-)
Thanks, Kurt – I appreciate that.
I agree with you on most parts. It’s true that more non-technical users are going to use beta’s, but it’s still far from everyone.
Most of the time, people have some software installed, and install a newer version when the program asks for it. Those people will never touch a beta, unless it’s something like a website that is in beta.
The a-bit-more technical users, will see that there is a checkbox for beta versions, or visit the site of that program, and like to check out beta’s. Next to that, you have beta hunters that simply install all betas.
The reason why mosts bugs are gone already, is because the huge standards that company already have launched. If they create a program, they most of the time (hope so) use the .NET Framework. Then it’s simply saying what the program should do and thats it. The show stoppers are more like ideas that the layout or usage of the programs are completely different then the user thought it would be.
It’s with software like drivers and the anti-virus programs that might give some big issues, or afcorse a windows beta.
A public Beta is an interesting breed. At one end, the developer gets people to test their product free, thereby eliminating quite a bit of in-house QA. At the other end, many users see the use of a beta as a way to see new features, especially in a near ubiquitous product like FeedDemon.
It’s a careful dance between the author and user–no show-stoppers to dissuade the user, yet an early enough release to iron out issues with dissimilar platforms.
I’ve been with this stuff since 1952 and am familiar with programmers self-convidence hype. Beta is still beta. If it exicuted properly there would be no need for BETA. However, very much like NewsGator.
Louis
You are so right about the term beta has changed with some things like windows defender in a constant beta!
still if there is a beta of some of my fave software I often will try it out. But if it messes up that software or my computer, its my own fault for trying it. I do believe though that for trying out that software version its only polite to report anything odd or weird. Its so wrong not to say if you have a problem or bug!
Maybe there needs to be a new term in between alpha and beta release which is for testing but is like the old beta definition :)
If there is a new FeedDemon I do always try the betas. But don’t try beta versions of windows and major software on the system. (not that newsgator is not major :))
its funny you should mention all of this…
some time ago, i purchase a great domain name, inbetaforever.com
in the 15 years ive been doing this, last 10 with coldfusion and homesite, homesite+, ive noticed that most every web app is kinda truly in BETA for
a long long time… they can be molded, every day, changed with customer requests
etc, etc, etc… yanno, its like thats what makes me love what i do…
the fact that we can in fact use a single point of distribution to relegate what
people use in their daily lives… and with the elegance of that notion, we
are given the ability to deliver up to the minute updates to everyone who uses
what we build… therefore enabling us to rewrite the definition of what people
expect REAL BETA to be… i like it…. as you can tell :) i never got around
to building the blog of my new apps that inbetaforeve.com was going to be, but
i thought it would be a cool domain name, for a cool blog, of always in beta
web applictions… maybe ill do that… who knows, this post might have inspired me…
anyway, thanks dude, homesite rocks.
ill never go to dreamweaver.
never.
later
tony
If you’re writing this article in reference to TopStyle 4 I’ll beta test it so long as it doesn’t cause my computer to start on fire or kill my dog! ha!
I love TopStyle and been an owner of the app for *years* from the 2.x versions.