Identifying parameters in partial differential equations (PDEs) represent a very broad class of applied inverse problems. Usually, these problems are addressed through optimization approaches, which are then discretized for practical numerical implementation using finite difference, finite element, or neural network approximations, with the latter often referred to as unsupervised learning in this context. A key challenge in this context is deriving a priori error estimates for the numerical reconstruction of the target parameter. In this talk, we present our recent work on establishing convergence rates for finite element methods in recovering a diffusion coefficient in an elliptic equation. This is achieved by carefully exploiting relevant stability results. Moreover, the approach can be extended to unsupervised learning methods using fully connected neural networks, as well as to multi-parameter identification problems with applications in hybrid physics imaging.