I guess it's more the acceptance that buyer/seller is an adversarial relationship so you need to at least be able to trust the system or framework to be impartial and enforce honesty.

> Isn't escrow basically the opposite of trust?

No, escrow is trust-as-a-service. That is, it's a means of removing the need for trust between the transaction principals by hiring a mutually-trusted third party.