{"id":12467,"date":"2021-06-11T06:48:34","date_gmt":"2021-06-11T04:48:34","guid":{"rendered":"https:\/\/usersnap.com\/?p=12467"},"modified":"2025-08-30T17:53:20","modified_gmt":"2025-08-30T15:53:20","slug":"beta-test-example-pocket-prep","status":"publish","type":"post","link":"https:\/\/usersnap.com\/blog\/beta-test-example-pocket-prep\/","title":{"rendered":"How to beta test, a robust example from award-winning learning app Pocket Prep"},"content":{"rendered":"\n

Surely all software companies are familiar with beta testing<\/a>, but how well<\/em> do you actually do it?<\/p>\n\n\n\n

If you\u2019re not all that confident with answering this question, hey, that\u2019s cool too, because you\u2019ve found the right article to get some practical ideas! <\/p>\n\n\n\n

We talked to Colin Ulin, senior software engineer at Pocket Prep, about their low-effort high-outcome beta testing strategy and design-thinking development process. <\/p>\n\n\n\n\n\n\n\n

Pocket Prep<\/a> is an e-learning<\/a> SaaS company providing all sorts of exam preparation courses via mobile and online at reasonable prices. Every day, 50,000 students, trainees and educators benefit from the platform. The award-winning team consists simply of 5 developers and 13 other talented staff members.<\/p>\n\n\n\n

Now that you know the basic background of the product and team, let\u2019s jump right into their beta test setup.<\/p>\n\n\n

Try Usersnap for Beta Testing<\/h2>Try Usersnap Now<\/a><\/div>\n\n\n\n

A feedback-driven example of beta testing<\/h2>\n\n\n\n

When a new product is ready to beta test, the first task is to invite testers. Instead of employing random software testers to go through different test scenarios, Pocket Prep gets actual customers to try out the new application. They would have a dozen schools, think of them as accounts, and around 50 users in the testing environment. <\/p>\n\n\n\n

To collect feedback, the first method is regular meetings between business development managers and customers. <\/p>\n\n\n\n

The second, and the more self-service way, is that customers can send screen captures within the application with their real-time experiences via Usersnap\u2019s widget.<\/p>\n\n\n\n

It\u2019s just as important to ensure the UX of the feedback process as it is to ensure the quality of your testing product. The on-screen annotation tools and simple feedback form by Usersnap<\/a> help users effortlessly highlight issues and present suggestions. Like this:<\/p>\n\n\n\n

\"Simple<\/figure>\n\n\n\n

It takes one minute to send a feedback with a customer feedback software. Compared to writing an extensive email or filling out a technical bug report that takes more than 10 minutes every time.<\/p>\n\n\n

Here’s a way to speed up beta testing feedback by 10x @usersnap<\/div>
<\/i>Click to Tweet<\/a><\/span><\/div><\/div>\n\n\n

Now, let\u2019s talk about the timeframe. <\/p>\n\n\n\n

Many software companies nowadays tend to go for the \u201cdo now, apologize later\u201d approach for shipping new features and products. But at Pocket Prep, the beta test and improvement process take 2 to 3 quarters. <\/p>\n\n\n\n

\n

\u201cWe are keen on doing it right the first time. If we ship fast but not great, it will only upset our users. And this is also how we\u2019ve built strong customer relationships.\u201d<\/p>\n– Colin Ulin, senior software engineer<\/cite><\/blockquote>\n\n\n\n

Listening to feedback and building smart solutions to those pain points takes time. But in return, your product can harvest memorable UX and customer loyalty.<\/p>\n\n\n\n

What kind of feedback should you be looking for?<\/h2>\n\n\n\n

During the beta test stage, the most common feedback you can expect are bug reports and feature requests. This helps the development team and management team to understand if the product is bug-proof and feature complete.<\/p>\n\n\n\n

For Pocket Prep, to uphold the trust their users have in them, it\u2019s essential the product launch is on par with users\u2019 expectations. <\/p>\n\n\n\n

Beta testing feedback goes beyond standard bug reports and feature requests. They are looking for UX defects and identifying missing elements to close the expectation gap. <\/p>\n\n\n\n

\n

\u201cWe get feedback from beta users about what they think is broken, when in fact it’s just the feature not working as they expected. This helps us understand that the feature is not ready yet.\u201d <\/p>\n– Colin Ulin, senior software engineer<\/cite><\/blockquote>\n\n\n\n

And because Usersnap\u2019s tools allow users to easily share feedback instantly anytime, users won\u2019t hesitate to click on the feedback button. <\/p>\n\n\n\n

Once launching the feedback widget<\/a>, a toolbar appears on the screen for making annotations on the screen capture. You can draw, highlight and\/or pin comments directly on the screenshot. <\/p>\n\n\n\n

Metadata, such as URL, browser data, device info and javascript errors, are automatically recorded and attached to the ticket.<\/p>\n\n\n\n

\"Collecting<\/figure>\n\n\n\n

Getting the right context empowers Colin and the team to find the fix and solve issues faster. In addition, when the users don\u2019t need to worry about entering a long list of \u201cboring\u201d details every time they send feedback, the happier they are to share. <\/p>\n\n\n\n

And with more feedback, the QA process becomes more robust.<\/p>\n\n\n\n

When is the beta test phase done?<\/h2>\n\n\n\n

Although Usersnap is great for collecting beta testing feedback, the issues won\u2019t fix themselves. <\/p>\n\n\n\n

The development and design teams at Pocket Prep work together in refining the new flows and adding new requirements to features. They tend to ship a couple of versions to test and gather more user feedback. <\/p>\n\n\n\n

If after some iterations, they get positive feedback or stop receiving complaints, then it\u2019s champagne time. But if similar feedback continues, then they would know the solution(s) did not work.<\/p>\n\n\n\n

There is no exact cutoff point in beta testing. However, the amount and type of user feedback can be taken as references along the way.<\/p>\n\n\n\n

How you decide when beta testing is complete also depends on your objective. Pocket Prep adheres to a meticulous approach because they believe making studying and exams fun means their app and UX also need to be enjoyable.<\/p>\n\n\n

2-3 quarters of #BetaTesting – worth it! Here’s how @pocketprep cultivates user love, a case study by @usersnap<\/div>
<\/i>Click to Tweet<\/a><\/span><\/div><\/div>\n\n\n

Customer feedback as the source of product solutions <\/h2>\n\n\n\n

Some customer feedback is very tangible and actionable. For example, a screen recording feedback of a non-loading CTA button. But others require more digging and analysis, such as the product onboarding process.<\/p>\n\n\n\n

To explore the root problem and evaluate a holistic perspective, Pocket Prep holds \u201cexpert panels\u201d where different departments are invited to join forces in the design thinking process. <\/p>\n\n\n\n