Well, yes, but Not all models need to be "super large." Smaller models, specialized in specific tasks, working together - and then reporting to a slightly larger model is the way to go.
Think of everything being connected to a "Home Computer" in those "Future House of 2020" videos that were out there in 70s or what not.
Another example (very rough) would be something like "Weather data gets to a small model via an API, model looks at it, updates the home dashboard, also sees if there's any alerts, if so, adds x or y to home dashboard appropriately as to what it thinks best."
We can probably achieve the latter example today. (without any significant 'coding' on anyone's part except the API owner)
> Well, yes, but Not all models need to be "super large." Smaller models, specialized in specific tasks, working together - and then reporting to a slightly larger model is the way to go.
I want to believe, but I'm still yet to see this kind of set up being anywhere near GPT-4 level.
The weather example seems quite contrived. Why not just display the alerts for your area? Why is a complex system of smaller models reporting up to a slightly larger model necessary?
Because "Flood warning on roads" is very different than "hey there is a possible tornado/aftershocks/*" and I'd like the 2nd one to take up the entire home-assistant dashboard.
I can either code it myself, or let the model figure it out after passing it the YAML file for the dashboard.
>Why is a complex system of smaller models reporting up to a slightly larger model necessary?
because as the other poster said, cost and speed. thousands of queries every day (potentially) at 30 cents per million tokens is very different than 15 dollars per million tokens.
Because cost and speed. Smaller models can run on your phone for free, or on the cloud for pennies. An API call for a large LLM with a lot of context can cost orders of magnitude more and incur network latency
Think of everything being connected to a "Home Computer" in those "Future House of 2020" videos that were out there in 70s or what not.
Another example (very rough) would be something like "Weather data gets to a small model via an API, model looks at it, updates the home dashboard, also sees if there's any alerts, if so, adds x or y to home dashboard appropriately as to what it thinks best."
We can probably achieve the latter example today. (without any significant 'coding' on anyone's part except the API owner)