Debunking the Myth of High-Degree Polynomials in Regression

The common belief that high-degree polynomials are prone to overfitting and difficult to control in machine learning is challenged in this article. The author argues that the problem isn't high-degree polynomials themselves, but rather the use of inappropriate basis functions, such as the standard basis. Experiments comparing the standard, Chebyshev, and Legendre bases with the Bernstein basis in fitting noisy data demonstrate that the Bernstein basis, with its coefficients sharing the same 'units' and being easily regularized, effectively avoids overfitting. Even high-degree polynomials yield excellent fits using the Bernstein basis, requiring minimal hyperparameter tuning.