# AB Test Sample Size Calculator MCP for AI Agents MCP

> The AB Test Sample Size Calculator MCP helps data teams nail down the statistical foundations of any experiment. It calculates exactly how many users you need per variant and projects the precise duration your test must run. Plus, it assesses your peeking risk, so you can confidently declare a winner without risking false positives.

## Overview
- **Category:** analytics
- **Price:** Free
- **Endpoint:** https://edge.vinkius.com/vk_preview_r2w7FicRhYLb2yLUrh65lss0cEO4Bcdo5mApGeZ1/mcp
- **Tags:** ab-testing, statistics, data-science, experimentation, conversion-rate

## Description

Running an A/B test isn't just about flipping a switch; it’s about statistics. This MCP provides the foundational tools to ensure your experiments are actually reliable. You tell your agent what your baseline conversion rate is and how big of an effect you want to detect, and the tool figures out the exact sample size required per variant. From there, you can get a solid projection on how many days your test needs to run based on current traffic. The most crucial part is checking for peeking risk; this helps prevent you from making calls too early that are just statistical noise. If you're already using Vinkius as your central catalog, connecting this MCP gives your AI client access to essential CRO math right where you need it.

## Tools

### calculate_required_sample_size
Figures out how many users you need in each group for a statistically sound A/B test setup.

### estimate_test_duration
Provides an estimate of how long your experiment must run based on your site's daily visitor traffic.

### assess_peeking_risk
Warns you if analyzing the data too early increases the chance of a false positive result.

## Prompt Examples

**Prompt:** 
```
I need to test if changing our headline increases conversions by 8%. What sample size should I aim for?
```

**Response:** 
```
**A/B Test Sample Size Calculation**

Based on your inputs (4.2% baseline CR, 0.5% MDE, 95% confidence):

*   **Required Users per Variant:** ~18,500 users
*   **Total Required Users:** ~37,000 users

This is the minimum user count needed to reliably detect that lift.
```

**Prompt:** 
```
If we need 60,000 total users and our site usually gets 2,000 visitors a day, how long should the test run?
```

**Response:** 
```
**Estimated Test Timeline**

*   **Target Users:** 60,000
*   **Daily Traffic:** 2,000
*   **Calculated Duration:** 30 days

We recommend scheduling for a minimum of 30 full days to account for weekend variations.
```

**Prompt:** 
```
Our test has been running for 10 days out of the planned 28. Should I stop it already?
```

**Response:** 
```
**Peeking Risk Assessment**

*   **Current Status:** 10/28 days complete.
*   **Risk Level:** Medium-High.
*   **Recommendation:** Do not make a final call yet. While the results look promising, continuing until day 25 will significantly lower your false positive risk and give you more statistical certainty.
```

## Capabilities

### Determine required user counts
Calculates the minimum number of users needed in each test group based on your expected conversion rates and target effect size.

### Estimate experiment timeline
Projects the necessary duration for an A/B test, using your site's current average daily traffic.

### Gauge risk of early analysis
Evaluates how high your probability of a false positive is if you stop analyzing the data before the planned end date.

## Use Cases

### Need to test a new checkout flow
A Product Manager asks their agent, 'If our current conversion rate is 4% and I want to detect at least a 10% lift, how big does this A/B test need to be?' The agent uses the required sample size tool and provides the necessary user count per variant.

### Testing seasonal changes with limited traffic
A Growth Marketer asks their agent for a timeline: 'Our site only gets 1,000 visitors daily right now; if I need 50,000 users, how long will the test take?' The agent uses the duration tool and gives a precise number of days.

### Deciding whether to wrap up an experiment
A Data Analyst checks their running test results and asks the agent about statistical safety. Using the risk assessment tool, they get immediate feedback: 'High peeking risk; continue for another week.'

### Validating multiple simultaneous experiments
The team needs to run three concurrent tests (CTA change, image update, pricing model). The agent runs the sample size tool for all three, ensuring they don't over-allocate resources or under-test critical variables.

## Benefits

- Stop basing product decisions on gut feelings. The `calculate_required_sample_size` tool tells you exactly how many users are needed, guaranteeing your results matter.
- Avoid running tests that never finish. Use `estimate_test_duration` to set realistic timelines and manage stakeholder expectations immediately.
- Eliminate false positives. By using the `assess_peeking_risk` function, you'll know when it’s safe to call an end date on your experiment.
- Your agent can handle complex statistical inputs—like baseline conversion rates and desired power levels—in a single query, saving manual spreadsheet work.
- The MCP keeps all your core CRO math centralized. You connect once via Vinkius and get access to the full suite of testing tools.

## How It Works

The bottom line is: it moves you past guessing game statistics and gives you actionable timelines for reliable data analysis.

1. Input your known metrics: Provide the baseline conversion rate, desired Minimum Detectable Effect (MDE), and confidence level.
2. The MCP processes these inputs using statistical formulas to calculate the necessary sample size per group and project the required test duration based on your traffic volume.
3. Finally, you receive a risk assessment that tells you if continuing the experiment is critical or if you're safe to analyze the results.

## Frequently Asked Questions

**How does the AB Test Sample Size Calculator MCP determine how many users I need per test?**
The tool calculates the minimum number of participants required for each group. It uses your baseline conversion rate and the effect size you want to detect, ensuring that if a real change happens, your test has enough power to prove it.

**Can I use this MCP to figure out how long my A/B test must run?**
Yes. You provide the total user count needed and your site's average daily traffic. The calculator then gives you a precise, data-backed estimate of the minimum number of days required.

**What is 'peeking risk,' and how does this MCP help me avoid it?**
Peeking risk is the danger of stopping a test early because the numbers look good. This MCP assesses that risk, telling you if you must wait for the full planned duration to prevent making false conclusions.

**Do I have to know my baseline conversion rate to use the AB Test Sample Size Calculator?**
Yes, knowing your current performance (the baseline CR) is essential. The tool needs this starting point to accurately calculate how large of a difference you need to detect.

**Is this MCP useful for testing different marketing channels?**
Absolutely. Whether the traffic comes from search, social media, or email campaigns, this MCP uses your aggregate daily traffic numbers to provide accurate test duration estimates.