In this paper, we study contragredient duals and invariant bilinear forms for modular vertex algebras (in characteristic p). We first introduce a bialgebra H and we then introduce a notion of H-module vertex algebra and a notion of (V;H)-module for an H-module vertex algebra V . Then we give a modular version of Frenkel-Huang- Lepowsky's theory and study invariant bilinear forms on an H-module vertex algebra. As the main results, we obtain an explicit description of the space of invariant bilinear forms on a general H-module vertex algebra, and we apply our results to affine vertex algebras and Virasoro vertex algebras. This is joint work with Poefessor Haisheng Li.