• NUMBERS
  • STATISTICS
  • PLANNING
  • HUMAN

How to calculate required sample size with your AI

Stop guessing how long an experiment needs to run. Your AI calculates the necessary traffic per group based on your specific testing goals.

Ask AI about this page

Short answer

How much traffic do I need for my A/B test?

You provide your desired significance level, power, and expected minimum detectable effect. Your AI then tells you exactly how many visitors each variant needs to reach a reliable result. You'll see the math behind these numbers in the following breakdown.

The outcomes

What you get from the calculation

Once you provide your testing parameters, the AI delivers these specific results.

NUMBERS

Exact visitor counts

The AI calculates the specific number of participants required for each test group. You get a clear target to hit before you call the test a success.

STATISTICS

Confidence and power

The calculation accounts for your chosen significance level and statistical power. This ensures your results aren't just due to random chance.

PLANNING

Duration estimates

By knowing the required sample size and your daily traffic, you can estimate how many days your test must run. This prevents you from stopping a test too early.

HUMAN

Manual adjustments

The AI provides the math, but you decide if the required sample size is realistic for your current traffic. You make the final call on whether to proceed with the test.

The workflow

What your AI does when the request arrives.

Your agent handles the statistical heavy lifting so you don't have to manually run formulas.

  1. Gather parameters

    Your AI asks for your baseline conversion rate, the minimum effect you want to detect, and your desired confidence level.

    calculate_required_sample_size

  2. Run the math

    The agent uses the provided statistical inputs to solve for the sample size per group.

    calculate_required_sample_size

  3. Verify inputs

    The AI checks that your parameters are logically sound, such as ensuring your significance level is between 0 and 1.

    calculate_required_sample_size

  4. Report results

    The agent presents the final visitor count and explains how changes to your parameters would impact the required size.

    calculate_required_sample_size

Try it

Copy these to start.

Use these prompts to trigger the calculation immediately.

Starting points

Replace the bracketed numbers with your actual testing goals.

A/B Test Significance Calculator Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_fzQ1aS9NSSO5zJk9otHteGWMsIrsHlHpW7aJ8mX9/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — A/B Test Significance Calculator capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "ab-test-significance-calculator-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_fzQ1aS9NSSO5zJk9otHteGWMsIrsHlHpW7aJ8mX9/mcp"
    }
  }
}
Copy into chat04

  • Calculate the required sample size for an A/B test with a 5% baseline conversion and a 10% minimum detectable effect.

  • How many visitors do I need per group if I want 95% confidence and 80% power for my landing page test?

  • Run a sample size calculation for a checkout flow experiment where the current conversion is 12% and I want to detect a 2% lift.

  • What happens to my required sample size if I increase my significance level from 0.05 to 0.01?

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect A/B Test Significance Calculator once, then ask.

Connect the MCP to your preferred client with one click. Your credentials stay encrypted, and you can start running calculations immediately.

Connect A/B Test Significance Calculator to your AI

FAQ

Common questions

  • 01

    Can the AI run the actual A/B test for me?

    No. The AI only calculates the required sample size. You must use your own testing platform to run the experiment and collect data.

  • 02

    Does the AI decide when the test is finished?

    No. The AI tells you how many visitors you need to reach significance. You are responsible for monitoring the test and deciding when to stop it.

  • 03

    What inputs do I need to provide?

    You need to provide the baseline conversion rate, the minimum detectable effect (MDE), the significance level, and the statistical power.

  • 04

    Can the AI adjust my conversion rate in the test?

    No. The AI is a calculator, not a testing tool. It cannot change any data in your analytics or testing software.

  • 05

    Will this work with my existing testing tool?

    Yes. This MCP works with any MCP-compatible client, meaning you can use these calculations alongside whatever tool you already use to run experiments.

  • More questions about A/B Test Significance Calculator? The Connector page answers them. See everything the A/B Test Significance Calculator Connector can do

Connect A/B Test Significance Calculator to Claude, Cursor, ChatGPT & more