If your panel is buzzing, make sure your cover panel is secure, as well as the door. It's normal for some humming depending on what brand of circuit breaker you have, but AC current does create magnetic fields at 60Hz and can cause loose equipment to buzz. One time I installed a new breaker panel and the cover panel was loose, but it didn't become noticeable until one of the major appliances was running enough current to make the buzz VERY loud. Scared the heck out of me!
Your deadfront should not buzz because it's loose. That would have to be a structural issue.