Beta testing – an obsolete concept?

September 24th, 2013 § 2 comments § permalink

I was invited to be a beta tester recently, and it struck me that I haven’t been seeing calls for beta testers being put out any more.

We want you to test WordPress

It might be that the beta testing period has been subsumed by Lean methodologies. After all, beta testing is about the feedback loop which is implicit in the lean process.

It might be that beta testing is just that much harder on mobile than web, and it was just left behind because it didn’t fit with the ecosystem any more.

But then the feedback loop isn’t visible to me these days. At least in the young startups I’ve been following. From the outside the process most startups seem to follow is: announce idea on Launchrock -> launch MVP -> growth hacking (simplified obviously). As a user of these services I am not asked for feedback, and I don’t usually see any obvious feedback collection mechanisms on the app. Is everyone relying on metrics these days?

Did beta testing lose its value because early beta testers are a very niche segment of the market, or not even a customer segment at all?
When I worked at, we had a dedicated beta testing forum where startups would put out the call and beta testers would jump in, test the products and let loose with their feedback. It wasn’t always easy for the startups to handle it – we had a bunch of very early adopters who had technical backgrounds so they tended to have really high expectations and …. let’s call it brutally honest feedback.
In addition, there’d always be the inevitable drop off in usage. Beta testers would flock to the new shiny, test it and move on. From a startup’s perspective this was a roller coaster ride. They’d get a spike in sign ups and activity, and then have to deal with the lull. Well that was a bummer….

There was one shining example of a success story however. A bunch of us were invited to test out, an asynchronous, multi-player strategy game, and in the process of testing the game, we were all hooked. A bunch of us brought our non-tester friends in – just so we could keep playing while waiting turns on other games! The founders of Weewar were very active in responding to feedback and this was also evident in the product’s evolution. Weewar was eventually acquired by EA Games.

What I think is a big loss with not actively running in a beta testing phase is the feedback loop between startups and users. (I’m not using the term ‘customers’ on purpose.)
There is an implicit social contract at play – as users we will use your product and provide feedback, as startups we will listen and consider the feedback. There is a lot more to this than obvious at first glance. As a user I am more inclined to provide feedback if I know there is a purpose for it. That it will be considered by someone. As a startup it affects how I run my communications strategy, whether or not I include a feedback mechanism into my product, how I manage my backlog.
Beta testing is a state of mind for all involved.

So startups, look at yourselves, evaluate whether running a beta testing phase is a fit, and then do it. You will get invaluable insights into your product’s UX, features and potentially even its future.
And beta testers, instead of throwing out a whinge on Twitter (GUILTY!) hit up the startup’s contact page and make a valuable contribution.

Oh, and the startup that kicked this post off – Formula Legend. Make sure to check em out and sign up if it’s your thing.


Don’t ask me what my New Year’s resolutions are. I don’t make em.

January 5th, 2011 § 0 comments § permalink

No really, I don’t. Actually, I vehemently oppose the practice.

To understand why, we need to go back a bit. When I was a kid, every New Year’s Eve I’d be asked about my resolutions, and I’d loudly proclaim them, having thought about them in preparation.

By the time I was 10 or 12, I was completely disillusioned with the whole thing. Each year started with such promise, such optimism, yet a few months in, I’d feel like I was letting myself and people around me down.

So I stopped.

And here’s the thing – many years later, after being suspended from university and having to essentially start building from scratch, I came to two conclusions.

  1. We are made up of habits.
  2. I suck at lofty but well-intended, noble goals.

This wasn’t a sudden realisation. Oh no, not at all. It came from a dark time, the biggest failure I had faced, a shattered ego and sense of self, unforgiving intense inquiry into my self, my actions, the causes and reasons for getting where I was.

But that first point – that was a major lightbulb moment when I put it together. We get used to acting the way we act, reacting the way we do, living the way we live. We don’t even think about these things on a daily basis. Setting one huge goal for the sake of it blinds us from all the little tiny changes we need to make in order to get there. We set a result without mapping the route. In startup terms – we talk about exit strategies without having done the gruelling product and customer grind.

What works for me is having a vague goal coupled with piecemeal chipping away and constant vigilance. And experimentation. Oh, the glorious data collection! Touching up and tweaking as we go along. Knowing that the 31st of Dec isn’t a deadline. The fun ends when I die dammit!

I’ve been reflecting on and investigating the past year a lot (more on that soon…ish) and have been making use of these ‘processes’ a lot. Processes are consistent slow-release happy pills. That’s much much more than resolutions have ever been for me.

Where Am I?

You are currently browsing entries tagged with process at dek's blog.