I don’t think it’s just the rarity. It’s also trustless, secure and easy to transfer. For ex, with gold, you’d have to carry blocks of it to move it around. And even then, you’d still don’t exactly trust it’s source 100% unlike bitcoin, where everything is in the ledger.