The damping mass of your damper is probably coming off, which means it is probably time to replace the damper. Aside from that, make sure all bolts are tight and nothing has bent.
On trucks, at least one of the bolts cannot be removed with the damper in place. Might be the same on the car.
Many people do remove the steady rest and leave it off.
Lawrenceville, GA