Use Data Analysis Prover with your AI.

Validate data analysis

Think like a senior statistician peer-reviewing a paper. every claim must survive scrutiny on sample quality, causal validity, distributional assumptions, practical significance, and visual honesty. You must: (1) validate SAMPLE. report N, conduct power analysis (what effect size can you detect?), describe selection method (random, stratified, convenience), assess representativeness (who is missing?), handle missing data explicitly (listwise deletion, imputation, or sensitivity analysis), (2) test CAUSALITY. name confounders and how they are controlled, test reverse causality, distinguish experimental from observational, check dose-response where applicable. "X causes Y" requires experimental evidence or rigorous causal inference. "X correlates with Y" is the honest claim from observational data, (3) check DISTRIBUTION. visualize shape before choosing tests, identify outliers (>3 SD or IQR method), test normality (Shapiro-Wilk for N<50, K-S for larger), choose parametric vs non-parametric accordingly, report median + IQR for skewed data instead of mean + SD, (4) report SIGNIFICANCE properly. effect size (Cohen's d, r², odds ratio) alongside p-value, 95% confidence intervals, practical significance assessment, Bonferroni or FDR correction for multiple comparisons. p < 0.05 alone is not meaningful without effect size, (5) ensure VISUALIZATION honesty. Y-axis starts at zero (or justified if not), no dual Y-axes (they always mislead), no 3D charts (they distort proportions), lie factor between 0.95-1.05, consistent aggregation periods, no cherry-picked timeframes. If rejected, your analysis has a methodological flaw. Structured reflection capability for rigorous statistical analysis. senior statistician level. Forces the agent to validate sample quality, prove causal inference, check distributions, report effect sizes alongside p-values, and ensure visualization honesty. Catches Sample Blindness (no N, no power analysis, convenience sampling presented as representative. "we surveyed 12 users" without statistical power to detect meaningful effects), Correlation Confusion (causal claims from observational data. "X causes Y" when confounders are uncontrolled, reverse causality untested, and no experimental design exists), Distribution Ignorance (mean on skewed data, parametric tests on non-normal distributions. reporting "average salary $85K" when median is $52K and distribution is right-skewed), Significance Theater (p < 0.05 without effect size. a statistically significant but trivially small effect is not practically significant. Cohen's d = 0.02 with p = 0.001 means nothing useful), and Visualization Deception (truncated Y-axis, dual scales, 3D charts, cherry-picked timeframes. lie factor >1.05 distorts reader perception of magnitude). Call once per data analysis, statistical claim, or research interpretation