01 · Mean Comparison
Compare continuous metrics such as revenue, order value, or time on page.
02 · Proportion Comparison
Compare conversion rates and other binary outcomes.
03 · Sample Size
Estimate how many observations you'll need before running a test.
04 · Statistical Power
Understand whether your experiment has enough power to detect a difference.
Three examples, three different moments where a test goes wrong.
Each one takes a decision that looked settled, and shows where the doubt should have come in — reading the result, sizing the test, or choosing how many places to look.
Example 01 — When “Not Significant” Doesn’t Mean “No Effect”
The moment you read the result.
Example 02 — Was 8 More Sign-Ups a Real Result?
The day you picked the sample size.
Example 03 - Would You Trust the Store That “Worked”?
The number of places you looked before you found it.
Together they cover the three questions worth asking before acting on any test result: Is the difference real? Could this test have seen it? How many did you look at?
Explore More from Grace Ge
Built by Grace Ge
Building systems for better decisions.