Naologic
Naologic
  • Apps
  • Products
    USE CASES FOR COMPANIES
    USE CASES FOR PARTNERS
    Startups
    Startups
    Small- and medium-sized companies from pre-seed startups to growth build visually
    Enterprises
    Enterprises
    Build your backoffice in-house for your large-scale operations and processes
    Enterprise Consultants
    Enterprise Consultants
    Deploy private projects and apps to your clients` workspaces
    Marketplace Partners
    Marketplace Partners
    Create apps, list them in the Naologic App Store and earn from every install
  • No-Code Builder
  • Pricing
  • Blog
USE CASES FOR COMPANIES
Startups
Startups
Enterprises
Enterprises
USE CASES FOR PARTNERS
Enterprise Consultants
Enterprise Consultants
Marketplace Partners
Marketplace Partners
AppsPricingNo-Code BuilderDocumentationSupportBlog
Close
  • Blog
  • Product
Naologic on Twitter
Oliver Andersen
Oliver Andersen
Chief Product Officer @ Naologic
Categories
Product

The Chronicles of A/B testing

Oliver Andersen
Oliver Andersen
Chief Product Officer @ Naologic
22 Nov 2018

With so many different approaches and methodologies out there, it can be hard to see through the smoke-screen and decipher the silver lining of conducting a/b testing

When used correctly, a/b testing can be a very powerful tool that could, and should, dictate not only whether to use ‘good’ or ‘great’ on a CTA, but also which priorities you, as a company, should make in your product roadmap.


One of the common A/B testing missteps I frequently see in startups (and some larger companies too), is to base their testing off microscopic changes to the object tested and pray that this small change will hold the key to triple their growth and securing their next round of funding. Results that are born from random variation (read: no statistical significance) can be the recipe for disaster, if they’re believed to be the winning combo and implemented. Fast-forward to when you’ve built the “winning feature”, dedicated time, resources and capital towards it, and you realize that your initial sample size was way too small and that your product or feature is about to fail miserably.


This is, of course, not to say that any seemingly significant findings should be looked upon with skepticism, sometimes your tests yield significance because you’ve hit the nail on the head, in which case, go build!

A/B testing as a discipline is interesting and can in many ways be just as unhelpful for your company if done wrong, as it can be helpful when conducted correctly. Plenty of people have analyzed the way NOT to conduct a/b testing, I want to briefly shed light on key questions we’ve had success with asking ourselves internally before conducting experiments in the past:

  1. Do you have a clearly defined success metric for your test
  2. Considering your traffic, what is an appropriate length of time to be conducting this test and at which point do you feel that results are sufficiently robust and reliable to potentially be the cornerstones of future product decisions?

If you have answered these questions before conducting your test (and stick to them during the actual testing period), you’ve taken a solid step towards ensuring that your test(s) will produce worthwhile results.


Finally, remember that we’re living in a brave new world of extremely rapid hypothesis-driven validation that is testable with low effort through interactive prototyping tools like invision, sketch, marvel, balsamiq or even a hand-sketch.


After all, it’s better to discredit your product hypothesis for a new shiny feature after a short stint of prototyping and testing, as opposed to after having committed a team of five designers and developers towards building it for one of more full sprints, only to realize that your users or customers have zero interest in what you thought sounded like a cool idea after a (too) long-week in the office (oh yes, this happens more frequently than you’d care to believe!).

Want to take Naologic for a spin?
Launch a free trial of our products in minutes.
Try for free
Join Naologic today
Join Naologic today
Help us change the way software is built and join the rocketship today!
View open roles
Have questions or comments?
Have questions or comments?
We’d love to hear from you!
Contact us
ABOUT NAOLOGIC
Naologic is an AI-enabled no-code platform for all your non-customer facing software. Build Apps without coding from templates or launch ready-made solutions for niche verticals in minutes.
PRODUCTS
  • For Startups
  • For Enterprises
  • For Enterprise Consultants
  • For Marketplace Partners
  • App Store
  • Pricing
  • No-Code Builder
INDUSTRY SAAS SOLUTIONS
  • B2B Ecommerce
  • Capital Management
  • NDC Healthcare
COMPANY
  • Contact
  • Blog
  • Careers
  • Architecture
  • In the News
  • Integrations
Partners
  • Become an Enterprise Consultant
  • Become a Marketplace Partner
  • Partner Guides
RESOURCES
  • Documentation
  • Inventory Management Blog
  • Help Center
  • Release Notes
  • Open Source
  • Guides & Ebooks
  • Media Kit
  • Privacy Policy
  • Cookie Policy
  • Terms
  • Glossary
  • Competitors
  • Sitemap
CONNECT
  • Linkedin
  • Twitter
  • Github
  • Reddit
  • Github
  • Facebook
  • Youtube
  • Quora
COMPANY
  • Contact
  • Blog
  • Integrations
  • Careers
  • Architecture
  • In the News
CONNECT
  • Linkedin
  • Twitter
  • Github
  • Reddit
  • Facebook
  • Youtube
  • Quora
© Naologic 2017-2022