The main use case for the unvote button is for mobile, and on a spotty mobile connection it might take a lot longer than 30 seconds until the action registered.
Sometimes I'll vote on a comment, and then as I'm reading the replies and clarifications, or other comments elsewhere in the discussion, I'll realize I misunderstood something and will want to go back and change my vote.
