GPT-3 has 175 billion parameters, and is not quite human-level. Cerebras is talking about building out clusters that can handle 100 trillion parameters, about a thousand times bigger than GPT-3.
This hypothetical GPT-4 would be big enough that it shouldn't have the context window or BPE token problems of GPT-3. It would be superhumanly good at predicting token sequences-- generating text. What that would look like, exactly, is not entirely clear. (What does it mean to be twice as good as a human at writing an essay?) If it had the same architecture as previous GPTs then it wouldn't be "conscious" or be goal directed, but it would be able to correlate more information and draw conclusions that we couldn't. (Would we understand these conclusions is another open question.)
It would also be quite expensive to run in a capital-amortized sense, at first, maybe hundreds of dollars per word. The commercial use of such a thing would be limited. Large tech companies seem to be building AI because of how self-evidently useful it will be... eventually.
The economic case for high-cost NNs is the opposite of most automation, which started from the bottom up. If a net is expensive to train and run then you have to pursue the Tesla strategy-- start from the top down. So you need to target high-price knowledge work in terms of dollars per hour, but is tolerant of small variance, since the results are still probabilistic. In the near future you won't see NNs designing jet engines, producing complete software systems from requirements documents, or even standard grunt-level software engineering work of wiring two systems together, since you can never be quite sure GPT-programmer is producing the right work.
Similarly, GPT-lawyer or GPT-CEO would be tough to do. GPT-hollywood would be interesting, if you could get it to crank out a complete Marvel movie with superhumanly good CGI. More terrifyingly, GPT-advertiser, if it can produce super-appealing ads, would be able to pay for its own runtime at the cost of a machine hijacking human minds to plug money into gacha games or the equivalent in the 2030s.
