I get what you're saying. But to me he is simply being honest.

If he did not wonder to himself how he would have felt about a scraggly, hippy knocking on his front door in the evening, I would have been suspect.