Andy never carried a gun.
Barney Fife wasn’t the only one who didn’t carry a gun, but for different reasons. Andy didn’t carry one either. There was a reason for it, and it wasn’t a because he was clumsy. Sheriff Andy wanted the people of Mayberry to respect him, not fear him.