Hoeffding’s lemma is an elementary yet pivotal result in empirical process theory that allows the construction of exponential measure concentration bounds such as Hoeffding’s inequality. The standard proof of the lemma does not require sophisticated tools but, in my opinion, it’s not very elegant either. In fact, the proof uses... [Read More]

The frequentist confidence interval is a fundamental tool in statistics that provides a way to make inferences about the values of parameters of interest whenever there is a (quantifiable) degree of uncertainty in the studied process. However, it is common to have misconceptions about how it should be interpreted. We’ll... [Read More]

The discussion in the previous post can be generalized to the case where a linear model [Read More]

This is one of the simplest examples of a machine learning approach to solve a particular problem. Assume that you are trying to model an unknown process \(f\colon \mathbb{R}^n \to \mathbb{R}\) and that you have collected a bunch of input/output pairs \(\left(\boldsymbol{x}_1, y_1\right), \ldots, \left(\boldsymbol{x}_k, y_k\right)\) from that process, that... [Read More]