I know and because of this something nice looking in one font becomes something completely useless when not using that font. (like the Github character in my previous post) It's exactly what I dislike about the entire practice. I don't want to have to use a certain font to make sure your site doesn't break.
I like the consistency of having the same fonts that I like and find readable in my browser as I have in every other window on my computer. Browsers give me the chance to aim for that consistency by picking one font for all sites, but then some sites look terrible because they rely on some particular font to show non standardized icons.