Can the AI run the actual A/B test for me?

No. The AI only calculates the required sample size. You must set up and run the experiment in your own testing platform.

Does the AI decide the significance level?

No. You must tell the AI what significance level and power you want to use for the calculation.

Can the AI change my existing conversion rates?

No. The AI only reads the rates you provide to perform the math; it cannot modify your live website or database.

What happens if my required sample size is too large?

The AI will give you the number. It is up to you to decide if you have enough traffic to actually run a test of that scale.

Can the AI calculate sample sizes for multivariate tests?

The tool is designed for standard A/B testing. For complex multivariate setups, you should provide the specific parameters for each variation.

STATISTICSCONFIDENCEANALYSISHUMAN

STATISTICS
CONFIDENCE
ANALYSIS
HUMAN

How to calculate required sample size with your AI

Connect your AI to this MCP to run statistical power calculations. Once connected, you just provide your target metrics and the AI handles the math.

Ask AI about this page

Short answer

How do I find the right sample size for an A/B test?

Give your AI your current conversion rate, the minimum detectable effect you want to catch, and your desired confidence level. It will use the calculate_required_sample_size tool to tell you exactly how many users you need per group. You'll get the specific numbers required to reach statistical significance.

The outcomes

What you get from the calculation

The AI processes your experimental parameters to produce these specific outputs.

STATISTICS

Required user counts

The tool calculates the exact number of participants needed for each test group. You get a clear number to use when setting up your experiment.

CONFIDENCE

Statistical power levels

Your AI determines the sample size needed to hit your specific confidence intervals and power targets. This ensures your results aren't just noise.

ANALYSIS

MDE verification

The AI checks if your requested Minimum Detectable Effect is realistic given your current traffic. It tells you if the test is even possible.

HUMAN

Experimental design

The AI provides the math, but you decide if the sample size is practical for your actual traffic. You make the final call on whether to run the test.

The workflow

What your AI does when the parameters arrive.

The AI takes your raw testing goals and converts them into a mathematical requirement.

Parameter ingestion
Your AI gathers your baseline conversion rate, the target lift, and your significance level from your chat prompt.
calculate_required_sample_size
Statistical computation
The agent runs the math to solve for N based on your specific power and alpha constraints.
calculate_required_sample_size
Feasibility check
The AI compares the required sample size against your known traffic patterns to see if the test is viable.
calculate_required_sample_size
Result delivery
The AI presents the final numbers and the logic used to reach them in a readable format.
calculate_required_sample_size

Try it

Copy these to start.

Use these prompts to trigger the calculation immediately.

Starting points

Swap out the percentages and conversion rates in these prompts to match your actual data.

AB Test Sample Size Calculator Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_r2w7FicRhYLb2yLUrh65lss0cEO4Bcdo5mApGeZ1/mcp

Claude Desktop

Follow the steps below to connect in seconds.

1In Claude Desktop, open Settings → Connectors.
2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
3Click Add and start a new chat — AB Test Sample Size Calculator capabilities are ready to use.

Configuration · claude_desktop_config.jsonCopy

{
  "mcpServers": {
    "ab-test-sample-size-calculator-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_r2w7FicRhYLb2yLUrh65lss0cEO4Bcdo5mApGeZ1/mcp"
    }
  }
}

Copy into chat04

Calculate the sample size needed for a conversion rate of 5% if I want to detect a 10% lift with 95% confidence.
How many users do I need for an A/B test if my baseline is 2% and I want to see a 0.5% absolute increase?
Run a sample size calculation for a 12% conversion rate with a 5% significance level and 80% power.
I have a baseline conversion of 10%. How many users per group do I need to detect a relative lift of 15%?

Claude
ChatGPT
Cursor
VS Code
Windsurf
Claude Code
JetBrains
Cline

Start here

Connect AB Test Sample Size Calculator once, then ask.

One click connects this MCP to your preferred AI client. Your credentials stay encrypted and the math tools are ready to use instantly.

Connect AB Test Sample Size Calculator to your AI

FAQ

How this task behaves

01
Can the AI run the actual A/B test for me?
No. The AI only calculates the required sample size. You must set up and run the experiment in your own testing platform.
02
Does the AI decide the significance level?
No. You must tell the AI what significance level and power you want to use for the calculation.
03
Can the AI change my existing conversion rates?
No. The AI only reads the rates you provide to perform the math; it cannot modify your live website or database.
04
What happens if my required sample size is too large?
The AI will give you the number. It is up to you to decide if you have enough traffic to actually run a test of that scale.
05
Can the AI calculate sample sizes for multivariate tests?
The tool is designed for standard A/B testing. For complex multivariate setups, you should provide the specific parameters for each variation.
More questions about AB Test Sample Size Calculator? The Connector page answers them. See everything the AB Test Sample Size Calculator Connector can do

Connect AB Test Sample Size Calculator to Claude, Cursor, ChatGPT & more