i would put it this way: there is no non-violent part of government. all aspects of government rely on the concentration of power and the monopoly of violence.
without that authority/monopoly, no one would have to listen to anything any government official says. with that authority, even if that branch doesn't have armed officers, we're encouraged not to resolve our own problems, but to rely on The Authorities to solve them. this leads to a whole raft of problems.