Hilbert Spaces

A Hilbert space is an inner product space that is also “complete” w.r.t the inner product - i.e. no sequence of elements in the set converges to an element outside the set. This is a generalization of the Euclidean vector space to infinite dimensions.

The key intuition behind this is to treat functions as infinite dimensional vectors, which have inner products and norms just like Euclidean vectors. This allows us to generalize many concepts from vectors to apply to functions.

Generalized Fourier Series

A fourier series¹ is a way of expressing a perioding function as a weighted sum of sines and cosines of different frequencies.

For functions in $\mathbb{L}^2$, the inner product allows us to define the concepts of orthogonality and basis vectors. This in turn lets us express these functions as a weighted sum of of a series of orthogonal basis functions that span the space. This allows us to apply concepts from Linear Algebra such as the Gram-Schmidt process² to the space of functions. This is how we get orthogonal basis functions such as Chebyshev polynomials ³ and Legendre polynomials⁴.

References