If you are concerned about Ansible SSH’ing in (which means you are concerned about any person SSH’ing in), you just do the standard SSH hardening.
Because you (or some intern hired a week ago) are 1 typo away from making a disaster.
For the record, security includes availability and this is an availability risk.
If you are concerned about Ansible SSH’ing in (which means you are concerned about any person SSH’ing in), you just do the standard SSH hardening.