I was invited to be a beta tester recently, and it struck me that I haven’t been seeing calls for beta testers being put out any more.
It might be that the beta testing period has been subsumed by Lean methodologies. After all, beta testing is about the feedback loop which is implicit in the lean process.
It might be that beta testing is just that much harder on mobile than web, and it was just left behind because it didn’t fit with the ecosystem any more.
But then the feedback loop isn’t visible to me these days. At least in the young startups I’ve been following. From the outside the process most startups seem to follow is: announce idea on Launchrock -> launch MVP -> growth hacking (simplified obviously). As a user of these services I am not asked for feedback, and I don’t usually see any obvious feedback collection mechanisms on the app. Is everyone relying on metrics these days?
Did beta testing lose its value because early beta testers are a very niche segment of the market, or not even a customer segment at all?
When I worked at Tangler.com, we had a dedicated beta testing forum where startups would put out the call and beta testers would jump in, test the products and let loose with their feedback. It wasn’t always easy for the startups to handle it – we had a bunch of very early adopters who had technical backgrounds so they tended to have really high expectations and …. let’s call it brutally honest feedback.
In addition, there’d always be the inevitable drop off in usage. Beta testers would flock to the new shiny, test it and move on. From a startup’s perspective this was a roller coaster ride. They’d get a spike in sign ups and activity, and then have to deal with the lull. Well that was a bummer….
There was one shining example of a success story however. A bunch of us were invited to test out Weewar.com, an asynchronous, multi-player strategy game, and in the process of testing the game, we were all hooked. A bunch of us brought our non-tester friends in – just so we could keep playing while waiting turns on other games! The founders of Weewar were very active in responding to feedback and this was also evident in the product’s evolution. Weewar was eventually acquired by EA Games.
What I think is a big loss with not actively running in a beta testing phase is the feedback loop between startups and users. (I’m not using the term ‘customers’ on purpose.)
There is an implicit social contract at play – as users we will use your product and provide feedback, as startups we will listen and consider the feedback. There is a lot more to this than obvious at first glance. As a user I am more inclined to provide feedback if I know there is a purpose for it. That it will be considered by someone. As a startup it affects how I run my communications strategy, whether or not I include a feedback mechanism into my product, how I manage my backlog.
Beta testing is a state of mind for all involved.
So startups, look at yourselves, evaluate whether running a beta testing phase is a fit, and then do it. You will get invaluable insights into your product’s UX, features and potentially even its future.
And beta testers, instead of throwing out a whinge on Twitter (GUILTY!) hit up the startup’s contact page and make a valuable contribution.
Oh, and the startup that kicked this post off – Formula Legend. Make sure to check em out and sign up if it’s your thing.
Interesting thoughts Rai. Yes, I think the whole concept has changed, with many new products opting for the “waiting list” approach, where the beta testing is now done “live” and not in a restricted/closed circle. I have noticed more of the traditional “beta” testing done with iOS apps, where the developer would then send you builds before they are out on the AppStore and less so with web applications.
Ah, I didn’t know that Bruno. Have only seen one or 2 mobile apps in testing mode.
I don’t think there needs to be a difference between closed betas or live testing. I do think however saying explicitly that a site is in the beta testing phase helps the process. As opposed to just putting something out there and trying to figure out what the metrics might be indicating.