---
title: One Basic Statistical Method Every Marketer Should Know for Simple A/B Testing
description: Most marketers use A/B testing. Learn how to calculate whether A/B testing results are statistically significant with a Two-Sample Proportions Test in Minitab.
image: https://blog.minitab.com/hubfs/B2BML%20-%20NEW%20FILE%20MANAGER%20STRUCTURE/Blog%20Images/Imported_Blog_Media/man%20working%20on%20laptop-1-Jul-06-2025-12-18-00-8994-AM.jpg
---
English
- [English](https://blog.minitab.com/en/?hsLang=en)
- [Français](https://blog.minitab.com/fr/?hsLang=en)
- [Deutsch](https://blog.minitab.com/de/?hsLang=en)
- [Português](https://blog.minitab.com/pt/?hsLang=en)
- [Español](https://blog.minitab.com/es/?hsLang=en)
- [日本語](https://blog.minitab.com/ja/?hsLang=en)
- [한국어](https://blog.minitab.com/ko/?hsLang=en)
- [中文（简体）](https://blog.minitab.com/zh/?hsLang=en)
- [English](https://blog.minitab.com/en/blog/basic-statistical-method-every-marketer-should-know-for-simple-a/b-testing)
- [Chinese](https://blog.minitab.com/zh/blog/basic-statistical-method-every-marketer-should-know-for-simple-a/b-testing)
- [Deutsch](https://blog.minitab.com/de/blog/statistische-methode-fuer-einfache-a-b-marketing-tests)
- [Français](https://blog.minitab.com/fr/blog/une-methode-statistique-elementaire)
- [한국어](https://blog.minitab.com/ko/blog/basic-statistical-method-every-marketer-should-know-for-simple-a/b-testing)
- [日本語](https://blog.minitab.com/ja/blog/basic-statistical-method-every-marketer-should-know-for-simple-a/b-testing)
- [ Partners](https://www.minitab.com/en-us/company/partners/)
- [ Contact](https://www.minitab.com/en-us/company/contact-us/)
- [ Talk to Minitab](https://www.minitab.com/en-us/talk-to-minitab/)
- [ My Account](https://licensing.minitab.com/account/manage)
[](https://www.minitab.com/en-us/)
- Products
Back
[Products](https://www.minitab.com/en-us/products/)
[All Products](https://www.minitab.com/en-us/products/)
- [Minitab Solution Center](https://www.minitab.com/en-us/products/minitab-solution-center/)
- [Minitab Statistical Software](https://www.minitab.com/en-us/products/minitab/)
- [Minitab Connect](https://www.minitab.com/en-us/products/connect/)
- [Minitab Model Ops](https://www.minitab.com/en-us/products/model-ops/)
- [Minitab Education Hub](https://www.minitab.com/en-us/products/education-hub/)
- [Minitab Engage](https://www.minitab.com/en-us/products/engage/)
- [Minitab Workspace](https://www.minitab.com/en-us/products/workspace/)
- [Real-Time SPC](https://www.minitab.com/en-us/products/real-time-spc/)
- [Prolink Data Collection & SPC](https://www.minitab.com/en-us/products/prolink)
- [Simul8 Discrete Event Simulation](https://info.minitab.com/en/new-to-simul8?hsLang=en)
- [SPM](https://www.minitab.com/en-us/products/spm/)
- Solutions
Back
[Solutions](https://www.minitab.com/en-us/solutions/)
[All Solutions](https://www.minitab.com/en-us/solutions/)
- [Analytics](https://www.minitab.com/en-us/solutions/analytics/)
- [Statistics & Predictive Analytics](https://www.minitab.com/en-us/solutions/analytics/statistical-analysis-predictive-analytics/)
- [Data Science & Machine Learning](https://www.minitab.com/en-us/solutions/analytics/data-science-machine-learning/)
- [Business Analytics & Intelligence](https://www.minitab.com/en-us/solutions/analytics/business-analytics-intelligence/)
- [Statistical Process Control](https://www.minitab.com/en-us/solutions/analytics/statistical-process-control/)
- [Quality Analytics](https://www.minitab.com/en-us/solutions/analytics/quality-analytics/)
- [Live Analytics](https://www.minitab.com/en-us/solutions/analytics/live-analytics/)
- [Reliability & Life Data Analysis](https://www.minitab.com/en-us/solutions/analytics/reliability/)
- [Discrete Event Simulation](https://info.minitab.com/en/new-to-simul8?hsLang=en)
- [Process Mining](https://info.minitab.com/en/new-to-simul8?hsLang=en)
- [Key Capabilities](https://www.minitab.com/en-us/solutions/key-capabilities/)
- [Continuous Improvement](https://www.minitab.com/en-us/solutions/key-capabilities/continuous-improvement/)
- [Data Integration & Data Prep](https://www.minitab.com/en-us/solutions/key-capabilities/data-integration-data-prep/)
- [Diagramming & Mind Mapping](https://www.minitab.com/en-us/solutions/key-capabilities/diagramming-mind-mapping/)
- [Digital Twins](https://info.minitab.com/en/new-to-simul8?hsLang=en)
- [Model Deployment & ML Ops](https://www.minitab.com/en-us/solutions/key-capabilities/model-deployment-ml-ops/)
- [Innovation & Project Management](https://www.minitab.com/en-us/solutions/key-capabilities/innovation-project-management/)
- [Process Excellence: Detect, Correct & Prevent](https://www.minitab.com/en-us/solutions/key-capabilities/process-excellence/)
- [Featured Industries](https://www.minitab.com/en-us/solutions/industry/)
- [Academic](https://www.minitab.com/en-us/solutions/industry/academic/)
- [Energy & Natural Resources](https://www.minitab.com/en-us/solutions/industry/energy-resources/)
- [Government & Public Sector](https://www.minitab.com/en-us/solutions/industry/government/)
- [Healthcare](https://www.minitab.com/en-us/solutions/industry/healthcare/)
- [Insurance](https://www.minitab.com/en-us/solutions/industry/insurance/)
- [Manufacturing & Industrial](https://www.minitab.com/en-us/solutions/industry/manufacturing-industrial/)
- [Services](https://www.minitab.com/en-us/solutions/industry/service-industries/)
- [Software & Technology](https://www.minitab.com/en-us/solutions/industry/software-technology/)
- [Construction](https://www.minitab.com/en-us/solutions/industry/construction/)
- [Featured Roles](https://www.minitab.com/en-us/solutions/role/)
- [Engineering](https://www.minitab.com/en-us/solutions/role/engineering/)
- [Business Analysis](https://www.minitab.com/en-us/solutions/role/business-analysts/)
- [Information Technology](https://www.minitab.com/en-us/solutions/role/information-technology/)
- [Supply Chain](https://www.minitab.com/en-us/solutions/role/supply-chain/)
- [Customer Service & Contact Center](https://www.minitab.com/en-us/solutions/role/customer-service-contact-center/)
- [Human Resources](https://www.minitab.com/en-us/solutions/role/human-resources/)
- [Marketing](https://www.minitab.com/en-us/solutions/role/marketing/)
- [Research and Development](https://www.minitab.com/en-us/solutions/role/research-and-development/)
- Resources & Services
Back
[Resources & Services](https://www.minitab.com/en-us/resources-services/)
[All Resources & Services](https://www.minitab.com/en-us/resources-services/)
- [Resources](https://www.minitab.com/en-us/resources-services/resources/)
- [Case Studies](https://www.minitab.com/en-us/resources-services/resources/case-studies/)
- [eBooks & White Papers](https://www.minitab.com/en-us/resources-services/resources/ebooks-and-white-papers/)
- [Blog](https://blog.minitab.com/blog/?hsLang=en)
- [Data Sets](https://www.minitab.com/en-us/resources-services/resources/data-set/)
- [Webinars & Events](https://www.minitab.com/en-us/resources-services/resources/events/)
- [Education Hub](https://www.minitab.com/en-us/resources-services/resources/education-hub/)
- [Services](https://www.minitab.com/en-us/resources-services/services/)
- [Training](https://www.minitab.com/en-us/resources-services/services/training/)
- [Deployment](https://www.minitab.com/en-us/resources-services/services/deployment/)
- [Consulting](https://www.minitab.com/en-us/resources-services/services/statistical-consulting/)
- [Self-Paced Learning](https://www.minitab.com/en-us/resources-services/services/self-paced-learning/)
- [Continuing Education](https://www.minitab.com/en-us/resources-services/services/continuing-education/)
- Support
Back
[Support](https://www.minitab.com/en-us/support/)
[Technical Support](https://www.minitab.com/en-us/support/)
- [Subscriptions & Activation](https://www.minitab.com/en-us/support/subscriptions-and-activation/)
- [Minitab Quick Start](https://www.minitab.com/en-us/support/quick-start/)
- [Training](https://www.minitab.com/en-us/resources-services/services/training/)
- [Installation Support](https://www.minitab.com/en-us/support/installation/)
- [Support Videos](https://www.minitab.com/en-us/support/videos/)
- [Support Documentation](https://www.minitab.com/en-us/support/documents/)
- [Software Updates](https://www.minitab.com/en-us/support/software-updates/)
- [Product Downloads](https://www.minitab.com/en-us/support/downloads/)
- [Support Policy](https://www.minitab.com/en-us/support/policy/)
- Company
Back
[Company](https://www.minitab.com/en-us/company/)
[Company](https://www.minitab.com/en-us/company/)
- [About Us](https://www.minitab.com/en-us/company/our-company/)
- [Leadership Team](https://www.minitab.com/en-us/company/leadership-team/)
- [Partners](https://www.minitab.com/en-us/company/partners/)
- [Careers](https://www.minitab.com/en-us/company/careers/)
- [Contact](https://www.minitab.com/en-us/company/contact-us/)
- [News](https://www.minitab.com/en-us/company/press-releases/)
[Try/Buy](https://www.minitab.com/en-us/try-buy/)
- [Statistics](https://blog.minitab.com/en/blog/tag/statistics)
- [Data Analysis](https://blog.minitab.com/en/blog/tag/data-analysis)
-
- [Quality Improvement](https://blog.minitab.com/en/blog/tag/quality-improvement)
- [Subscribe](https://info.minitab.com/blog/subscribe?hsLang=en)
## Notice
You are now leaving minitab.com.
Click Continue to proceed to:
Continue
Go Back
# One Basic Statistical Method Every Marketer Should Know for Simple A/B Testing
[Joshua Zable](https://blog.minitab.com/en/blog/author/joshua-zable) | 4/5/2022
Topics: [Minitab Statistical Software](https://blog.minitab.com/en/blog/tag/minitab-statistical-software)
- [Share](https://www.linkedin.com/shareArticle?mini=true&url=https://blog.minitab.com/en/blog/basic-statistical-method-every-marketer-should-know-for-simple-a/b-testing)
- [Share](https://www.facebook.com/sharer.php?u=https://blog.minitab.com/en/blog/basic-statistical-method-every-marketer-should-know-for-simple-a/b-testing)
In the past, some marketers may have shuddered at hearing the word “statistics.” But today, if you’re not incorporating statistics and analytics into your marketing strategy you’re going to be left behind.
The good news is that companies like Minitab make analyzing data easy. As you’ll see in the A/B testing example below, dipping your toe in the statistical “pool” is easy as 1-2-3.
## What is A/B Testing?
A/B testing, also known as split testing or bucket testing, refers to a randomized experimentation process wherein two or more versions of a variable (web page, page element, email etc.) are shown to different audiences at the same time to determine which version drives the most impact. Some examples of this are sending multiple emails to see which one generates more engagement, using different advertisements to measure click through rates, or split URL testing, which splits website traffic between a control group (often the original web page) and a variation (such as a new design).
## **A Simple Method for Simple A/B Testing**
Most marketers use some form of A/B testing. Unfortunately, despite best efforts to determine the content that drives the highest engagement, most marketers do not run a hypothesis test to determine whether their results are *actually* statistically significant.
There are different types of A/B testing, including multivariate testing, which alters multiple components at the same time. Today we’re looking at simple A/B testing comparing two groups. For this purpose, the most appropriate tool is an easy-to-use hypothesis test called a Two-Sample Proportions Test.
## **Example 1: Running a Proportion Test for A/B Testing of an Email Campaign**
Imagine a marketer runs a monthly email promoting training courses. The goal of these emails is to generate customer interest and awareness about training. She’s had reasonably good success in the past, but still wants to see if she can improve her performance.
She decides to run an A/B test. She wants to target 2,329 customers. She decides to break up the customers into two groups: Email 1 and Email 2. For Email 1, she is sticking with the same email she sent last month and sends this to approximately 50% of her customers. The other 50% are sent Email 2.
She sends the emails. Email 1 gets a 12% open rate (140 opens) and Email 2 gets a 9.8% open rate (115 opens). Most marketers would then say Email 1 performed better, correct?
We can leverage analytics to decide if this 2.2% (12% - 9.8%) difference is statistically significant. But many people wouldn’t, because it is too difficult to know where to go to help us answer this question in a quick and easy way. In three clicks and four numbers, you can know if the difference is statistically significant!
The method is a Two-Sample Proportion Test in Minitab. It will tell you if these two proportions are statistically different, based on the sample sizes, level of confidence and observed open rates. By going to **St**at > Basic Statistics > **2 Proportions**, Minitab presents a dialogue window for me to make a quick calculation of my summarized data. I simply plug in my data (with events corresponding to opens and sample corresponding to number of emails) and Minitab does the rest!
## **Conclusion**
Looking at the results below I see that the true difference between the groups, with a 95% confidence interval, can just as likely be 0 as .022 (or 2.2%). In stats speak, this means we “Fail to reject the null hypothesis which states that the two proportions are equal.” In other words, the two groups are NOT statistically different at the 95% confidence level. Despite one email getting a 12% open rate, and another one getting 9.8% open rate – there is not a statistical difference between the performance of the emails. Concluding that these emails result in different open rates could be a mistake.
Comparing two proportions is a useful method to ensure you fully benefit from your A/B testing strategy. By incorporating analytics into your marketing work and improving the data literacy of your team, you can ultimately drive better decision making.
### See how Minitab Statistical Software can transform your data with a 30-day free trial
[](https://cta-redirect.hubspot.com/cta/redirect/3447555/bb8468ed-94dd-4d77-9a79-69eb9a4d268d)
## You Might Also Like
[Confidence in Every Calculation: How Minitab Supports FDA-Regulated Validation](https://blog.minitab.com/en/blog/confidence-in-every-calculation-how-minitab-supports-fda-regulated-validation?hsLang=en)
[Minitab Solution Center](https://blog.minitab.com/en/blog/tag/minitab-solution-center) | 4 Minute Read
[Seedling Series #3: Simplifying Your Data Story with Minitab](https://blog.minitab.com/en/blog/seedling-series-3-simplifying-your-data-story-with-minitab?hsLang=en)
[Minitab Statistical Software](https://blog.minitab.com/en/blog/tag/minitab-statistical-software) | 3 Minute Read
[4 Solutions Minitab Can Help with in Agriculture](https://blog.minitab.com/en/blog/4-solutions-minitab-can-help-with-in-agriculture?hsLang=en)
[Minitab Connect](https://blog.minitab.com/en/blog/tag/minitab-connect) | 3 Minute Read
---
- [Careers](https://www.minitab.com/en-us/company/careers/)
- [Trust Center](https://www.minitab.com/en-us/trust-center/)
- [Suppliers](https://www.minitab.com/en-us/suppliers/)
- [Site Map](https://www.minitab.com/en-us/site-map/)
- [](https://www.linkedin.com/company/minitab/)
- [](https://www.facebook.com/Minitab/)
- [](https://www.youtube.com/user/MinitabInc)
- [](https://www.instagram.com/minitab/)
---
© 2025 Minitab, LLC. All Rights Reserved.
- [Terms of Use](https://www.minitab.com/en-us/legal/terms-of-use/)
- [Privacy Policy](https://www.minitab.com/en-us/legal/privacy-policy/)
- [Legal](https://www.minitab.com/en-us/legal/)
- Cookies Settings