Recently, the 4D Einstein-Gauss-Bonnet gravity has received a lot of attention. Remarkably, it possesses an exact vacuum solution that deviates from general relativity. I will discuss the important features of black holes and neutron stars in this theory. In particular, for very compact objects, the sequence of neutron stars matches asymptotically to the black hole limit, closing the mass gap between neutron stars and black holes of same radius. Compact objects also offer the best environment to constrain the single parameter that controls the deviations of this theory with respect to general relativity.