P-Hacking in Startups: Avoiding Statistical Traps

2025-06-21
P-Hacking in Startups: Avoiding Statistical Traps

The pressure to ship fast in startups often leads teams to report any result that looks like an improvement, resulting in p-hacking. This article dissects three common scenarios: multiple comparisons without correction, reframing metrics post-hoc, and running experiments until a hit is achieved. It emphasizes the importance of pre-registering hypotheses and metrics, avoiding post-hoc data dredging, using corrections for multiple comparisons, and applying appropriate thresholds for early peeking. The article advocates for celebrating definitive negative results, arguing that rigorous statistical practices accelerate learning by preventing the release of noise and building a true understanding of user behavior.

Read more
Development A/B testing

Linear Regression and Gradient Descent: From House Pricing to Deep Learning

2025-05-08
Linear Regression and Gradient Descent: From House Pricing to Deep Learning

This article uses house pricing as an example to explain linear regression and gradient descent algorithms in a clear and concise way. Linear regression predicts house prices by finding the best-fitting line, while gradient descent is an iterative algorithm used to find the optimal parameters that minimize the error function. The article compares absolute error and squared error, explaining why squared error is more effective in gradient descent because it ensures the smoothness of the error function, thus avoiding local optima. Finally, the article connects these concepts to deep learning, pointing out that the essence of deep learning is also to minimize error by adjusting parameters.

Read more