It was easy to take an existing open source terraform module [0], modify it a little for our purposes, and deploy runners that provision job executors in our tightly controlled VPC. It is very simple and everything is open source so you can really understand what it is doing. In our setup, all secrets are issued from our private vault instances. Most of them are short lived or one time use when possible. We are looking at moving our stuff to EKS now as well.
Overall, it was pretty easy to get going but we have the resources to do this. I could see why a small startup would outsource this to someone like Circle CI.
I second GitLab runner, used it a lot on bare metal CI servers, because we have some really exotic hardware requirements. Happy customer for 3 years now, and the CI part costs us nothing.
Set up within seconds using a few lines of cloud-init: https://gitlab.com/21analytics/gitlab-runner-cloud-init
Most of the time, it's also cheaper and maintenance is close to zero.