Decentralized optimization algorithms save remarkable communication overheads in distributed deep learning since each node averages locally with neighbors. The network topology connecting all nodes determines communication efficiency and local averaging effectiveness. The key to making decentralized algorithms efficient is to seek sparse topologies that realize effective local averaging with little communication. However, exiting common topologies either suffer expensive per-iteration communication or slow consensus rates. In this talk, we will propose several sparse and effective topologies that endow decentralized algorithms with state-of-the-art balance between communication efficiency and convergence. We will also discuss BlueFog, an open-source python library for straightforward, high-performance implementations of diverse topologies and decentralized algorithms.