#A/B Testing

A/B Testing Statistical Significance: How Long to Actually Run a Test

The direct answer: calculate your required sample size before you start, based on your baseline conversion rate and the smallest effect worth detecting, then run until you hit it. Stopping early on a good-looking p-value is how most A/B test results turn out to be noise.