{"id":1286,"date":"2013-08-07T13:35:14","date_gmt":"2013-08-07T11:35:14","guid":{"rendered":"https:\/\/usersnap.com\/?p=1286"},"modified":"2025-02-25T08:36:06","modified_gmt":"2025-02-25T07:36:06","slug":"ab-testing","status":"publish","type":"post","link":"https:\/\/usersnap.com\/blog\/ab-testing\/","title":{"rendered":"The dos and don&#8217;ts in A\/B testing"},"content":{"rendered":"\n<p>As someone making a living in the startup world, one can not have missed the rise of A\/B tests, greatly boosted by <a title=\"Eric Ries\" href=\"http:\/\/www.startuplessonslearned.com\/\" target=\"_blank\" rel=\"noopener\">Eric Ries<\/a>\u2019 book <a title=\"The Lean Startup\" href=\"http:\/\/www.amazon.com\/The-Lean-Startup-Entrepreneurs-Continuous\/dp\/0307887898\" target=\"_blank\" rel=\"noopener\">The Lean Startup<\/a>.<\/p>\n\n\n\n<p> But what is this A\/B testing all about? And how do you make sure you get a data-driven approach to product development right for your website or web application?<\/p>\n\n\n<div class=\"acf-cta\" style=\"background-image: url(https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2025\/02\/Group-1000004194.svg); width: 100%;\"><h2>Try Usersnap for Various Testings<\/h2><a href=\"https:\/\/usersnap.com\/signup\" class=\"cta-button\">Try Usersnap Now<\/a><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">What is A\/B testing?<\/h2>\n\n\n\n<p>A\/B testing requires to have two different versions of a page, one being your current version, and the other is the version you want to change the page to. Every A\/B experiment starts with a little hypothesis. For instance: in order to drive more traffic towards our signup page we need a friendly green button, instead of the blue one we have currently. To research and justify your changes, you route half your visitors to the first page and half to the second. Next, you monitor how many of the visitors perform the desired action (like sign up for your service) on each page, and you calculate the conversion rate for the old and new page. The page with the highest conversion rate is probably the one you should use. <\/p>\n\n\n\n<!--more-->\n\n\n\n<p>For a funny intro on A\/B testing, go check <a title=\"intro on A\/B testing\" href=\"https:\/\/visualwebsiteoptimizer.com\/what-is-ab-testing\/\" target=\"_blank\" rel=\"noopener\">this presentation<\/a> by Visual Website Optimizer. You\u2019ll find more serious (and useful &#8211; who sells fish online anyway?!) slides by <a title=\"Ben Tilly\" href=\"https:\/\/plus.google.com\/114613808538621741268\/posts\" target=\"_blank\" rel=\"noopener\">Ben Tilly<\/a> on the topic on <a title=\"Ben Tilly - effective A\/B testing\" href=\"http:\/\/elem.com\/~btilly\/effective-ab-testing\/\" target=\"_blank\" rel=\"noopener\">elem.com<\/a>. Just one of the take-aways: A\/B tests do not substitute for talking to users, usability tests, acceptance tests, unit tests or \u2018thinking\u2019.<\/p>\n\n\n\n<p><a title=\"Benjamin Plesser\" href=\"https:\/\/twitter.com\/bjpless\" target=\"_blank\" rel=\"noopener\">Benjamin Plesser<\/a> (<a title=\"EngineHere - Coding as Conversation\" href=\"http:\/\/www.enginehere.com\/\" target=\"_blank\" rel=\"noopener\">EngineHere<\/a>) describes \u2018<a title=\"The A\/B testing life cycle\" href=\"http:\/\/benplesser.com\/2012\/09\/20\/denial-anger-bargaining-depression-and-acceptance-the-ab-testing-lifecycle\/\" target=\"_blank\" rel=\"noopener\">The A\/B testing life cycle<\/a>\u2019 in a striking post:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>To be considered during sprint planning, a feature normally must relate back to a testable hypothesis which attempts to predict its impact on key company metrics. If the upside of a hypothesis is promising enough, we will move quickly to prototype the feature, utilizing a combination of internal and third party A\/B testing tools.<\/p>\n<\/blockquote>\n\n\n\n<p>All too often he has seen people blaming some \u2018broken JavaScript\u2019 or Google\u2019s algorithm for unexpected outcomes of A\/B tests. Or questioning the validity of HTML. \u201c<em>Did our acceptance testing miss something with IE(6|7|8)?<\/em>\u201d Sounds familiar? It sure does to me. At a former company testing a new design, the \u2018go\u2019 would sometimes be backed by the outcome of an A\/B test, while at other times it just \u2018fits the new vision\/strategy\u2019 (read: the A\/B test did in no way turn out in favor of concept B)&#8230; Benjamin ends his article with: \u201c<em>The most important lesson that I have learned in my life has been that you should train yourself to avoid results-oriented thinking at all costs.<\/em>\u201d And I couldn\u2019t agree more.<\/p>\n\n\n\n<p><a title=\"Joel Lewenstein on Twitter\" href=\"https:\/\/twitter.com\/joellewenstein\" target=\"_blank\" rel=\"noopener\">Joel Lewenstein<\/a>&#8216;s (product designer at Quora) on <a title=\"unexpected negative outcome for your A\/B test\" href=\"https:\/\/www.quora.com\/Design\/How-do-designers-at-Twitter-Facebook-and-other-tech-companies-balance-their-design-instinct-and-judgment-vs-the-engineers-impulse-to-use-A-B-test-data-for-every-design-change\/answer\/Joel-Lewenstein\" target=\"_blank\" rel=\"noopener\">handling an unexpected negative outcome for your A\/B test<\/a>:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>When data confirms intuition, it&#8217;s an easier outcome to feel good about and move forward with. Harder to reconcile is when the data disagrees with intuition, i.e. Scenarios II and III. Underneath the inherent frustration and bewilderment that comes with these outcomes, there can be tremendous learning opportunities.<\/p>\n\n\n<div class=\"acf-cta\" style=\"background-image: url(https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2025\/02\/Group-1000004194.svg); width: 100%;\"><h2>Try Usersnap for Various Testings<\/h2><a href=\"https:\/\/usersnap.com\/signup\" class=\"cta-button\">Try Usersnap Now<\/a><\/div>\n<\/blockquote>\n\n\n\n<p>Unexpectedly bad data forces a reconsideration of assumptions:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Really thoughtful consideration of these results, without biases or assumptions, creates fertile ground for brainstorming new ideas with potentially high impact.<\/p>\n<\/blockquote>\n\n\n\n<p>To really reap the benefits of this data-driven approach to product development, Joel claims (at least) two things to be critical. Test groups should be conceived to test as few variables as possible. And those variables should have clear, testable names like \u2018I think people will sign up for a service that seems affordable, so I&#8217;m going to put our price information on the homepage\u2019.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">When A\/B testing turns out to be A\/A testing<\/h2>\n\n\n\n<p>Stavros Korokithakis recently wrote a great blog on <a title=\"Stavros Korokithakis on the pitfalls of A\/B testing\" href=\"http:\/\/www.stavros.io\/posts\/pitfalls-b-testing\/?new\" target=\"_blank\" rel=\"noopener\">the pitfalls of A\/B testing<\/a>. Stavros:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>The main problem with A\/B testing is that you can&#8217;t really be sure that the results were actually statistically significant, rather than a fluke. If you show one person the first page and they sign up, and show one person the second page and they don&#8217;t sign up, does this mean that the first page is better? Clearly not, since it might be pure luck. So, we need a way to tell whether or not the difference in rate is due to randomness. (&#8230;) Wanting to verify that my A\/B testing code was implemented properly, I gave it two variations of the page that were exactly the same, and which shouldn&#8217;t have produced any significant difference. To my great surprise, I saw that my tests reported conversions being up by a whole 30%, with 99.8% confidence, on one of the two (identical) versions! How could it be 99.8% sure that one page is 30% better, when there was no change between the two?<\/p>\n<\/blockquote>\n\n\n\n<p>Many online A\/B testing frameworks let you automatically stop or conclude your test at the \u2018moment of significance\u2019, and there is very little discussion on false positive rates. In academic research, however, failing to control for testing errors and false positive rates are considered a huge faux-pas. Mats Einarsen knows this trade-off all too well and writes the following on Booking.com dev blog: <a title=\"Mats Einarsen on the Booking.com dev blog\" href=\"http:\/\/blog.booking.com\/is-your-ab-testing-effort-just-chasing-statistical-ghosts.html\" target=\"_blank\" rel=\"noopener\">Is your A\/B testing effort just chasing statistical ghosts?<\/a>:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>For anyone running A\/B tests in a commercial setting there is little incentive to control your false positives. Why make it harder to show successful changes just to meet a standard no one cares about anyway? (&#8230;) It actually matters, and matters a lot if you care about your A\/B experiments and what you learn from them.<\/p>\n\n\n<div class=\"acf-cta\" style=\"background-image: url(https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2025\/02\/Group-1000004194.svg); width: 100%;\"><h2>Try Usersnap for Various Testings<\/h2><a href=\"https:\/\/usersnap.com\/signup\" class=\"cta-button\">Try Usersnap Now<\/a><\/div>\n<\/blockquote>\n\n\n\n<p>To demonstrate how much it matters, he ran a simulation of how much impact you should expect repeated testing errors to have on your success rate. You can<a title=\"A\/B testing script\" href=\"https:\/\/gist.github.com\/anonymous\/2945361\" target=\"_blank\" rel=\"noopener\"> download the script<\/a> and alter the variables to fit your metrics. The outcome? 1.000 controlled experiments where it\u2019s known that there is no difference between the variants showed that 771 experiments out of 1.000 reached 90% significance at one point or another. And moreover: 531 experiments out of 1.000 reached 95% significance at some point.<\/p>\n\n\n\n<p>The conclusion being that if you use an A\/B testing package that automatically turns on experiments based on significance alone, you&#8217;ll probably see your success rate go up regardless of the quality of your changes.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><a href=\"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2013\/08\/AB.jpg\" rel=\"lightbox\"><img decoding=\"async\" width=\"700\" height=\"831\" src=\"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2013\/08\/AB.jpg\" alt=\"A\/A testing\" class=\"wp-image-1292\" srcset=\"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2013\/08\/AB.jpg 700w, https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2013\/08\/AB-252x300.jpg 252w, https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2013\/08\/AB-117x140.jpg 117w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><\/a><figcaption class=\"wp-element-caption\">Photo: <a href=\"http:\/\/www.flickr.com\/photos\/18091975@N00\/2963986468\/\" target=\"_blank\" rel=\"lightbox noopener\">Eoin Gardiner<\/a><\/figcaption><\/figure>\n<\/div>\n\n\n<h2 class=\"wp-block-heading\">How many subjects are needed for an A\/B test?<\/h2>\n\n\n\n<p>The question of how many subjects are needed to achieve relevant results from an A\/B test is one that puzzles many startup founders, as they usually have too little of a user base of visitors rate to call significance.<\/p>\n\n\n\n<p>There are tools that help you figure out your sample size like Evan Miller\u2019s <a title=\"Sample Size Calculator\" href=\"http:\/\/www.evanmiller.org\/ab-testing\/sample-size.html\" target=\"_blank\" rel=\"noopener\">Sample Size Calculator<\/a>. In statistics, <a title=\"G-tests\" href=\"http:\/\/en.wikipedia.org\/wiki\/G-test\" target=\"_blank\" rel=\"noopener\">G-tests<\/a> are likelihood-ratio for your statistical significance tests. You can find a G-test Calculator on <a title=\"Test Calculator\" href=\"http:\/\/elem.com\/~btilly\/effective-ab-testing\/g-test-calculator.html\" target=\"_blank\" rel=\"noopener\">elem.com<\/a>.<\/p>\n\n\n\n<p>Evan Miller wrote an industry standard article on \u2018<a title=\"how not to run an A\/B test\" href=\"http:\/\/www.evanmiller.org\/how-not-to-run-an-ab-test.html\" target=\"_blank\" rel=\"noopener\">how not to run an A\/B test<\/a>\u2019 on April 18, 2010:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>If you run experiments: the best way to avoid repeated significance testing errors is to not test significance repeatedly. Decide on a sample size in advance and wait until the experiment is over before you start believing the \u201cchance of beating original\u201d figures that the A\/B testing software gives you. \u201cPeeking\u201d at the data is OK as long as you can restrain yourself from stopping an experiment before it has run its course.<\/p>\n<\/blockquote>\n\n\n\n<p>Set up your sampling size before you start with your A\/B tests and don&#8217;t trust the significance calculations provided by your A\/B tool blindly. Some of them show you: &#8220;It&#8217;s significant now&#8221;, but that&#8217;s actually not true. Reach your sample size first, then (and only then) have a look at the data. To cite our very own <a title=\"Erik Bovee on Twitter\" href=\"http:\/\/www.twitter.com\/ebovee\" target=\"_blank\" rel=\"noopener\">Erik Bovee<\/a> (SpeedInvest):<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>The plural of anecdote is probably not data.<\/p>\n<\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\">Get a data-driven approach to product development right for your company<\/h2>\n\n\n\n<p>A\/B tests are great to justify small iterations on your site if you get the following prerequisites right:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Your sample size (and thus user base) should be large enough. Testing a green versus a blue button with a hundred users will never result in an outcome you should act upon.<\/li>\n\n\n\n<li>Start your experiment with a clear (and small) hypothesis. \u201cThe green button will make everything better\u201d is <strong>not<\/strong> a clear hypothesis.<\/li>\n\n\n\n<li>Don\u2019t &#8211; ever &#8211; stop your experiment prematurely.<\/li>\n\n\n\n<li>Evaluate your tests to fix testing errors and avoid false positives.<\/li>\n\n\n\n<li>Don\u2019t take it personally when data disagrees with intuition. And certainly don\u2019t just go for option B anyway, regardless of the outcome of your A\/B tests. What\u2019s next? Anarchy?<\/li>\n<\/ul>\n\n\n\n<p>Happy testing!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>As someone making a living in the startup world, one can not have missed the rise of A\/B tests, greatly boosted by Eric Ries\u2019 book The Lean Startup. But what is this A\/B testing all about? And how do you make sure you get a data-driven approach to product development right for your website or [&hellip;]<\/p>\n","protected":false},"author":5,"featured_media":4498,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":true,"inline_featured_image":false,"ub_ctt_via":"","footnotes":""},"categories":[8],"tags":[],"class_list":["post-1286","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-web-development-blog"],"acf":[],"featured_image_src":"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2016\/02\/ab-testing-startups.gif","author_info":{"display_name":"Floor Drees","author_link":"https:\/\/usersnap.com\/blog\/author\/floor\/"},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.6 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>The dos and don&#039;ts in A\/B testing - Usersnap Blog<\/title>\n<meta name=\"description\" content=\"What is this A\/B testing all about? And how do you make sure you get data-driven approach to product development right for your website or web application?\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/usersnap.com\/blog\/ab-testing\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The dos and don&#039;ts in A\/B testing - Usersnap Blog\" \/>\n<meta property=\"og:description\" content=\"What is this A\/B testing all about? And how do you make sure you get data-driven approach to product development right for your website or web application?\" \/>\n<meta property=\"og:url\" content=\"https:\/\/usersnap.com\/blog\/ab-testing\/\" \/>\n<meta property=\"og:site_name\" content=\"Usersnap Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/usersnap\" \/>\n<meta property=\"article:published_time\" content=\"2013-08-07T11:35:14+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-02-25T07:36:06+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2016\/02\/ab-testing-startups.gif\" \/>\n\t<meta property=\"og:image:width\" content=\"640\" \/>\n\t<meta property=\"og:image:height\" content=\"386\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/gif\" \/>\n<meta name=\"author\" content=\"Floor Drees\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@FloorDrees\" \/>\n<meta name=\"twitter:site\" content=\"@usersnap\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Floor Drees\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/usersnap.com\/blog\/ab-testing\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/usersnap.com\/blog\/ab-testing\/\"},\"author\":{\"name\":\"Floor Drees\",\"@id\":\"https:\/\/usersnap.com\/blog\/#\/schema\/person\/192527bfcdea3007f77595befffaf5c4\"},\"headline\":\"The dos and don&#8217;ts in A\/B testing\",\"datePublished\":\"2013-08-07T11:35:14+00:00\",\"dateModified\":\"2025-02-25T07:36:06+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/usersnap.com\/blog\/ab-testing\/\"},\"wordCount\":1451,\"publisher\":{\"@id\":\"https:\/\/usersnap.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/usersnap.com\/blog\/ab-testing\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2016\/02\/ab-testing-startups.gif\",\"articleSection\":[\"Web Development\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/usersnap.com\/blog\/ab-testing\/\",\"url\":\"https:\/\/usersnap.com\/blog\/ab-testing\/\",\"name\":\"The dos and don'ts in A\/B testing - Usersnap Blog\",\"isPartOf\":{\"@id\":\"https:\/\/usersnap.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/usersnap.com\/blog\/ab-testing\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/usersnap.com\/blog\/ab-testing\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2016\/02\/ab-testing-startups.gif\",\"datePublished\":\"2013-08-07T11:35:14+00:00\",\"dateModified\":\"2025-02-25T07:36:06+00:00\",\"description\":\"What is this A\/B testing all about? And how do you make sure you get data-driven approach to product development right for your website or web application?\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/usersnap.com\/blog\/ab-testing\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/usersnap.com\/blog\/ab-testing\/#primaryimage\",\"url\":\"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2016\/02\/ab-testing-startups.gif\",\"contentUrl\":\"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2016\/02\/ab-testing-startups.gif\",\"width\":643,\"height\":388,\"caption\":\"ab testing for startups\"},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/usersnap.com\/blog\/#website\",\"url\":\"https:\/\/usersnap.com\/blog\/\",\"name\":\"Usersnap Blog\",\"description\":\"Learn more about how to collect user feedback and build better products with the magic power of feedback.\",\"publisher\":{\"@id\":\"https:\/\/usersnap.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/usersnap.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/usersnap.com\/blog\/#organization\",\"name\":\"Usersnap\",\"url\":\"https:\/\/usersnap.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/usersnap.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/usersnap.com\/wp-content\/uploads\/2020\/08\/Usersnap-Updated-Logo.png\",\"contentUrl\":\"https:\/\/usersnap.com\/wp-content\/uploads\/2020\/08\/Usersnap-Updated-Logo.png\",\"width\":136,\"height\":26,\"caption\":\"Usersnap\"},\"image\":{\"@id\":\"https:\/\/usersnap.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/usersnap\",\"https:\/\/x.com\/usersnap\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/usersnap.com\/blog\/#\/schema\/person\/192527bfcdea3007f77595befffaf5c4\",\"name\":\"Floor Drees\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/usersnap.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/398c7a57bc95c84ec629ed2fba1408402fe66b76f9b56d8169829ad06d77e907?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/398c7a57bc95c84ec629ed2fba1408402fe66b76f9b56d8169829ad06d77e907?s=96&d=mm&r=g\",\"caption\":\"Floor Drees\"},\"sameAs\":[\"https:\/\/x.com\/FloorDrees\"],\"url\":\"https:\/\/usersnap.com\/blog\/author\/floor\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The dos and don'ts in A\/B testing - Usersnap Blog","description":"What is this A\/B testing all about? And how do you make sure you get data-driven approach to product development right for your website or web application?","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/usersnap.com\/blog\/ab-testing\/","og_locale":"en_US","og_type":"article","og_title":"The dos and don'ts in A\/B testing - Usersnap Blog","og_description":"What is this A\/B testing all about? And how do you make sure you get data-driven approach to product development right for your website or web application?","og_url":"https:\/\/usersnap.com\/blog\/ab-testing\/","og_site_name":"Usersnap Blog","article_publisher":"https:\/\/www.facebook.com\/usersnap","article_published_time":"2013-08-07T11:35:14+00:00","article_modified_time":"2025-02-25T07:36:06+00:00","og_image":[{"url":"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2016\/02\/ab-testing-startups.gif","width":640,"height":386,"type":"image\/gif"}],"author":"Floor Drees","twitter_card":"summary_large_image","twitter_creator":"@FloorDrees","twitter_site":"@usersnap","twitter_misc":{"Written by":"Floor Drees","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/usersnap.com\/blog\/ab-testing\/#article","isPartOf":{"@id":"https:\/\/usersnap.com\/blog\/ab-testing\/"},"author":{"name":"Floor Drees","@id":"https:\/\/usersnap.com\/blog\/#\/schema\/person\/192527bfcdea3007f77595befffaf5c4"},"headline":"The dos and don&#8217;ts in A\/B testing","datePublished":"2013-08-07T11:35:14+00:00","dateModified":"2025-02-25T07:36:06+00:00","mainEntityOfPage":{"@id":"https:\/\/usersnap.com\/blog\/ab-testing\/"},"wordCount":1451,"publisher":{"@id":"https:\/\/usersnap.com\/blog\/#organization"},"image":{"@id":"https:\/\/usersnap.com\/blog\/ab-testing\/#primaryimage"},"thumbnailUrl":"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2016\/02\/ab-testing-startups.gif","articleSection":["Web Development"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/usersnap.com\/blog\/ab-testing\/","url":"https:\/\/usersnap.com\/blog\/ab-testing\/","name":"The dos and don'ts in A\/B testing - Usersnap Blog","isPartOf":{"@id":"https:\/\/usersnap.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/usersnap.com\/blog\/ab-testing\/#primaryimage"},"image":{"@id":"https:\/\/usersnap.com\/blog\/ab-testing\/#primaryimage"},"thumbnailUrl":"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2016\/02\/ab-testing-startups.gif","datePublished":"2013-08-07T11:35:14+00:00","dateModified":"2025-02-25T07:36:06+00:00","description":"What is this A\/B testing all about? And how do you make sure you get data-driven approach to product development right for your website or web application?","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/usersnap.com\/blog\/ab-testing\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/usersnap.com\/blog\/ab-testing\/#primaryimage","url":"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2016\/02\/ab-testing-startups.gif","contentUrl":"https:\/\/usersnap.com\/blog\/wp-content\/uploads\/2016\/02\/ab-testing-startups.gif","width":643,"height":388,"caption":"ab testing for startups"},{"@type":"WebSite","@id":"https:\/\/usersnap.com\/blog\/#website","url":"https:\/\/usersnap.com\/blog\/","name":"Usersnap Blog","description":"Learn more about how to collect user feedback and build better products with the magic power of feedback.","publisher":{"@id":"https:\/\/usersnap.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/usersnap.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/usersnap.com\/blog\/#organization","name":"Usersnap","url":"https:\/\/usersnap.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/usersnap.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/usersnap.com\/wp-content\/uploads\/2020\/08\/Usersnap-Updated-Logo.png","contentUrl":"https:\/\/usersnap.com\/wp-content\/uploads\/2020\/08\/Usersnap-Updated-Logo.png","width":136,"height":26,"caption":"Usersnap"},"image":{"@id":"https:\/\/usersnap.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/usersnap","https:\/\/x.com\/usersnap"]},{"@type":"Person","@id":"https:\/\/usersnap.com\/blog\/#\/schema\/person\/192527bfcdea3007f77595befffaf5c4","name":"Floor Drees","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/usersnap.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/398c7a57bc95c84ec629ed2fba1408402fe66b76f9b56d8169829ad06d77e907?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/398c7a57bc95c84ec629ed2fba1408402fe66b76f9b56d8169829ad06d77e907?s=96&d=mm&r=g","caption":"Floor Drees"},"sameAs":["https:\/\/x.com\/FloorDrees"],"url":"https:\/\/usersnap.com\/blog\/author\/floor\/"}]}},"_links":{"self":[{"href":"https:\/\/usersnap.com\/blog\/wp-json\/wp\/v2\/posts\/1286","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/usersnap.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/usersnap.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/usersnap.com\/blog\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/usersnap.com\/blog\/wp-json\/wp\/v2\/comments?post=1286"}],"version-history":[{"count":0,"href":"https:\/\/usersnap.com\/blog\/wp-json\/wp\/v2\/posts\/1286\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/usersnap.com\/blog\/wp-json\/wp\/v2\/media\/4498"}],"wp:attachment":[{"href":"https:\/\/usersnap.com\/blog\/wp-json\/wp\/v2\/media?parent=1286"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/usersnap.com\/blog\/wp-json\/wp\/v2\/categories?post=1286"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/usersnap.com\/blog\/wp-json\/wp\/v2\/tags?post=1286"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}