Given a random variable X with expectation E[X], one generally has E[1/X] \neq 1/E[X], due to the nonlinear nature of the inverse. A similar phenomenon holds for random matrices, and this fundamental inversion bias has important implications for modern statistical and numerical methods.

In this talk, I will discuss how this bias arises in a variety of randomized sketching techniques (including random sampling and random projections) which are commonly used in large-scale machine learning (ML) and randomized numerical linear algebra (RandNLA). Drawing on joint work with Michał Dereziński (Michigan), Edgar Dobriban (UPenn), Michael Mahoney (UC Berkeley), and Chengmei Niu (HUST), we exploit leave-one-out techniques from random matrix theory (RMT) to precisely characterize this inversion bias and show (in some cases at least) that it can be corrected with ease. Using these technical results, we establish problem-independent local convergence rates for sub-sampled Newton methods.