Ask HN: How willing are you to alpha/beta test a product (before release) | Hacker News Reader

Ask HN: How willing are you to alpha/beta test a product (before release)

9 pointsjmonegro9 comments

I'm thinking about a few subjects and a question came up: how willing are people to alpha/beta test a product before it's release to the general public?

I thought I'd ask Hacker News first.

View on HN

If you launch a product on HN, the link is publicly viewable, and archived by Google shortly thereafter.

You have, by definition, launched publicly.

No one wants to debug someone else's broken crap. So deliver something worth an educated stranger's time with the caveat that it might be unpolished. People expect sites to get better with time.

Depends on how useful/interesting seems the product to me.

An holographic display? Bring it on.

Bingo card generator? Not so much.

The most awesome bingo card generator? The same. Isn't about your product. Is about me.

We released one of our products as a public beta in June last year, because we felt it would be useful to people, but not quite good enough yet – by our own standards – to charge people for.

This worked out great, within a very short timespan we had thousands of beta users, who apparently agreed with us that our product was useful. With all the email we received from these people we were able to assess what kind of features people wanted added to our product (and which bugs really needed to be fixed right away) to be 'good enough' in their opinions. We mixed in the feedback with the improvements we had in mind already to release 8 more betas to incrementally get the product to a 'good enough' sweet spot that was being redefined/refined with every release.

Finally, we released a 1.0 (for pay) version of the product 5 monhts later.

Our application now has a large, very loyal and mostly happy group of customers and it won an Apple Design Award last month. I'd say our ideas about how we took it from beta to 1.0 worked out pretty well.

So, I'll echo what others have said here: Make something useful, release that, then work your ass off to turn it from something that's just useful (which is already a hard criterion to achieve) into something great.

If you just need 'testing': Hire someone to be a tester, or employ some friends to play with a private beta and give you feedback. We did this too before taking the beta public, and thereby prevented a lot of annoyances for our users and a ton of nearly identical emails in our support inbox for issues we really ahould've fixed before we released.

Even with this precaution in place, we still received multiple hundreds of emails a day just after the public beta release about other issues that we could have fixed before shipping but didn't yet know about. Releasing prematurely can not only turn away your earliest users, but it can also overwhelm you with a support burden you don't want to deal with when what you really should be doing is continuing development towards a solid 1.0.

Something to note: Hacker News is probably not a reliable sample group if you expect the "people" in your question to == "general public" or "average users" unless HN is the target audience for your product.

I am of the opinion that this demographic enjoys poking at things for the sake of poking at things... but, as jp_sc, if it's totally orthogonal to our interests, our motivation drops dramatically.

Moreover, lsb makes an important point that bears repeating: no one* wants to debug someone else's broken crap. Someone released a beta link for a dating site within the last few days, and judging by the volume of criticism, it probably qualified as such. (No offense... it just wasn't ready. Some projects take time.)

*by and large

I think its largely personal interests. There are tons of projects just getting into beta and no doubt they took many man hours to create; yet if the text on the homepage does not entice me in the 4 seconds I give it, I'm pretty much gone for good. No disrespect to the code, the coders, the concept, the vision, I just value my time very highly. Don't we all?

It's not about "testing". I will start to use a product if I think it will be useful. Being alpha or beta will reduce my expectation that it will be useful, but not significantly. Being easy to get started with, having the right features, and looking responsive to feedback will more than make up for it.

It totally depends on the situation. If I know the product is from someone who is good at what they do, I will be fine doing testing if it is benefiting me. Otherwise, no way.

Totally depends on the product. I love beta testing games since it saves me paying for them later and figuring out they're bad.

What's the context?

For financial transactions or security concerns? Not so much. For creating work product I can evaluate and decide whether to use? Sure. Can I get my information back out, so that I've not lost all my time putting it into the product?

How easy is bug reporting?

Can I document and submit bugs quickly? I mean, I will put in the effort to be thorough, accurate, and concise in my descriptions. But can I do so without fighting the tracking software interface (and/or its host)? Can I easily and quickly track my issues, with update notifications so I don't have to remember to check status manually?

So, is the risk manageable, the effort streamlined, and the feedback responsive?

And, of course, does the product offer compelling features.

Making bug reporting easy seems under-rated to me. Many times, I've found a bug, to give up on reporting it because the process is too cumbersome. And response to submitted bugs motivates me, as well. I appreciate being able to help a communal effort. I don't expect a lot of time and effort returned unless I've found something significant; a status change in the tracker may be sufficient. But shoving bug reports into a black hole of status quickly becomes demotivating.

Reply on news.ycombinator.com