Domain authority is just Moz scoring system of the quality of the backlinks, I guess you have been building poor quality links. Remove them is the answer and stop doing it in the future.
However, Google doesn't use Moz's scoring system so providing your rankings are good then it doesn't really matter does it.