We present a unified framework for understanding the compaction of colloidal gels under their own weight. The dynamics of the collapse are determined by the value of the gravitational stress sigma(g), as compared to the yield stress sigma(Y) of the network. For sigma(g)<sigma(Y), gels collapse poroelastically, and their rate of compression decays exponentially in time. For sigma(g)>sigma(Y), the network eventually yields, leading to rapid settling. In both cases, the rate of collapse is backflow limited, while its overall magnitude is determined by a balance between gravitational stress and network elastic stress.