>submarine startups out there sitting on top of full downloads of Common Crawl, archive.org, etc. who are only too happy to let OpenAI be the first penguin off the iceberg.

Not sure any startup would be ok with sitting on a potentially gamechanging model.

Google/Facebook? Maybe?

Amazon's LLM efforts are pretty much in shambles