If your software is almost ready for release, you might be considering, and you are probably right, launching a beta version first, to make sure the market’s response is positive, and make adjustments if it is not. Here a few tips to help you make the best out of your beta testing!
But before we get started, what is a beta version of a software? Any software goes through an alpha-testing phase, that is usually carried out internally, to make sure that all major bugs and glitches are identified and corrected. However, in many cases, the number of possibilities and use cases for a software is too high for a small team to cover it all by itself. Releasing a second version of the software, free of the major bugs identified at first, to a larger group of users, will allow developers to collect more feedback both on possible remaining bugs, but more importantly on the overall user experience.
Beta – work in progress
What do you want to do with your beta?
Launching a beta version of a software has now become quite common, but before you do it too, take some time to think about what you will want your testers to actually test. By the time you launch the beta, the software will be running pretty smoothly globally, but you might have in mind some features for which you need real customer feedback, or want to check if everything keeps on working okay when an important number of users connect simultaneously to an online service.
Identifying your main objectives for your beta testing will help you planify the differents steps of your testing period, but most importantly, it will make the testers’ job easier as they will know precisely what they should focus on during their time with your product.
Choosing your group of testers
You would probably want your beta testing to be short and easy, but unfortunately, it will probably take a few months, since you will need to have each new beta version you put out tested again to make sure all the adjustments and improvements are working the way users wanted them to, and the way you tried to implement them.
You will be spending a lot of time with your beta testers, and, hopefully, hearing a lot from them, so getting the right group of beta testers is the first thing you should be working on. Here a few things to keep in mind:
- Public or private? Having a public beta (available to anyone) might seem like a good idea to collect more data, but too much data will probably prevent the most important issues from emerging clearly. It will also be harder to stay connected in a personal way with your beta testers to keep them involved in the process, so you might feel like the number of feedbacks is the declining as you move forward.
- If you decide to go with a private beta, you might get a lot of applications (hey, who doesn’t like getting early access to a cool new software?) from which you will need to extract those that will actually help you gather useful feedback.
- Target the people you want: they might be those you think will be your core users, or a new segment whose reaction to your software you want to test, but keep in mind that the data, and feedback, you will get will depend on who is testing your product.
- Remember to let people know what’s expected of them as beta testers. They should feel enticed to providing some feedback, and not only using your software before other people. You can try to quantify objectives (time spent on the software, number of feedbacks…) or simply incentive feedbacks, but if you’ve built your pool of testers carefully, that might not be necessary.
Note on NDAs: if your beta is private, and you have not launched your product yet, you might want to look into Non-Disclosure Agreements (NDAs). The NDA will reinforce the importance of secrecy among your testers, and give you legal protection should anything leak.
Collecting data
People like to provide feedback in very different ways. Think about yourself: maybe you have never sent a bug report when your app or software crashed, maybe you have. Maybe you prefer writing emails once in a while to cover a long period of testing, or you’d rather send tiny pieces of feedback every time you get the chance. Chances are the same thing will happen for your beta testers, so letting them use whichever way they want will increase your chances of getting feedback.
Can your testers contact you? Email, Skype, Facebook, Twitter… They should be able to reach out to you in any way that is more convenient for them. Any effort you don’t put in making their lives simpler is an additional effort they will not make to send you the feedback you’re looking for. Whether they want to send a weekly recap of all the glitches they have encountered, and all the improvements they think would be useful, or just notify you instantly of what is going through their mind, your beta testers should not have to wonder if they can contact you through this or that way, as the answer should always be YES.
Want to get feedback on something more specific? A particular level in your game, or a question to which you would like to get an answer from all or part of your testers? An in-app prompt to answer a question is the surest way to find out what users are thinking. They have just finished a level you still have doubts about? Ask them what they think about! They have not been using a feature, or have been overusing an other? Ask them why, and find out what your users are really thinking. With polljoy, you can ask your testers anything you want, and customize your questions to match your software’s look and feel. Whether you ask multiple choice questions or open ones, you will have instant access to all the data to identify immediately the most important issues your users are facing.
Remember how you always skip those automatic bug reports (I know I do…)? Well, it turns out that some people don’t! Implementing those in your beta version comes at no (or little) extra cost, and it’s one effort people will not have to make if they want to report a crash. Just hit send, and voilà.
All this data, and now what?
Keep your testers involved! They are (hopefully) providing a ton of feedback that will make your product better in the end, so the least you can do is to let them know how their efforts have benefited you. Here a few ways you can keep your testers involved:
- Updates. Let your testers know how it’s going. They might feel like things are not changing that much even though they’re actively providing feedback. So while you’re working on making your product better, inform your testers you’re doing so. Let them know which changes you are making, and how their work has led you to making these adjustments. A weekly update, whether it’s an email, or in a closed group on any social platform you like (I hear Facebook is still pretty useful for that) is a simple gesture that should be well received. Share your successes, but also your failures or disappointments. Keeping the relationship between you and your testers human is a great way for them to stay motivated.
- New builds. In the end, the goal of the beta test is to release new builds that take into account the feedbacks you have been getting from your users, correcting what they did not like or adding the most requested features. Every new build is proof to your testers that their feedbacks are implemented as the development goes along, and that their efforts actually have an impact. And, if you ask me, that’s a pretty cool reward!
- Rewards. Speaking of rewards, some of your testers might be expecting a little gesture from you at some point in the process, and most probably at the end. Promising a reward before the beta test begins will get you more applications beforehand, but remember that you will have to hold on to that promise. If you’re not too sure what you’ll be able to offer in the end, better not to make any hazardous promise, and surprise your users with an unforeseen reward. It could be a free version of the software they have been testing, or a reward inside that software if it is free, or anything you can come up with that sounds cool enough to you (a visit to your office? Dinner with the development team? Really, anything!).
Angry people.
Unfortunately, and despite your best efforts to put up a great testing team, some people will subscribe to the beta as a way to get early access to a software they are interested in, with no or little intention to provide feedback.
You probably wish the only angry things in your surroundings were those birds in your smartphone, but being surrounded with beta testers, the truth might be a little different. Some of your testers might not exactly understand that a beta version of a software is, by essence, imperfect and get pretty, um, harsh to you whenever something goes wrong. If you’ve picked your testers carefully, and let them know precisely what the situation was in the beginning, there shouldn’t be too many of those, but if they are, you can do either two things. If the feedback is valuable, you might want to smooth things out and remind the tester that bugs and glitches are inherent to beta testing. If harsh feedbacks keep on coming, remember you can always choose to exclude a tester from the process. However irritating it might be, keep cool – you do not want your software to come into market with detractors already waiting for you around the corner.
Being responsive both to positive and negative feedbacks will keep your testers participating, while censorship, even to overtly negative feedbacks, might not be very well received and insert a bias in the testing process.
Have you been beta testing recently? Share your experience with us at hi@polljoy.com or on Twitter, @polljoy!
