In my experience, most small startups, especially successful ones, rely on SaaS for as much as possible. The goal is to do as little as possible themselves.
A local copy helps you migrate to a new provider but that's still a lot of effort and downtime. You lose ticketing, pull requests, CI/CD pipeline, artifact storage, etc.
Everything will bite you in the ass eventually. Nothing is ideal. Everything has risks and eventual failure modes. Good business is about managing those risks rather than eliminating them.