Originated from the algebraic study of differential equations by Ritt a century ago, differential algebra has evolved into a well-established area with rich theory and broad applications. On the other hand, Novikov algebra merged from Balinsky-Novikov's work on Poisson brackets of hydrodynamic type in early 1980s, and can be used to construct infinite-dimensional Lie algebras. Gelfand gave a natural construction of Novikov algebras from differential algebras. Recently, multi-Novikov algebras emerged from the regularity structures of stochastic PDEs initiated by Hairer, and can be constructed from differential algebras with multiple derivations. The constructions of Balinksky-Novikov and Gelfand have also been extended to bialgebras. This talk reviews the background and recent progress relating differential algebras and Novikov algebras. This is joint work with Chengming Bai and Yanyong Hong.