The models are trained to respond with “guesses” too confidently, due at at least three aspects of using online discussions as examples:
1. The obvious one: Lots of examples of humans with no clue, talking like they are absolute experts.
2. Humans without anything to contribute to discussions are not pressured to say something. They may be “there” in the discussion, but the model can’t learn from their silence.
Unlike how we are forcing these uninformed outputs from the models.
They need balanced examples of verbally expressed knowledge modesty to do this better.
3. Human’s implicit factoid omniscience, while they are online, given Google access, creates expectations that the non-Googling model can’t match.
> They need balanced examples of verbally expressed knowledge modesty to do this better.
This is generally pretty easy, with ChatGPT at least, in my experience.
Langchain uses this prompt: "The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know."
Also, just don't ask it for things it couldn't possibly know without making it up like medical citations.
This is more for where you want your "AI" assistant to say it does not know how to book a reservation rather than pretend that it can.
> Langchain uses this prompt: "The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know."
There seem to be a lot of cases where people claim the models can't do something, but with minimal direction, they can.
Like avoiding bias. Or doing long digit math carefully, instead of guessing, etc.
I don't know a human that doesn't need any feedback either.
Obviously, many of these things would be better handled in the learning stage, so bias avoidance, careful serial thought processes, etc., were its baseline thinking.
When that happens I would expect fewer mistakes, but also an overall increase in the quality of responses, and ability to handle greater complexity. Clear careful thinking reduces error in each step, making longer chains of reasoning more viable.
1. The obvious one: Lots of examples of humans with no clue, talking like they are absolute experts.
2. Humans without anything to contribute to discussions are not pressured to say something. They may be “there” in the discussion, but the model can’t learn from their silence.
Unlike how we are forcing these uninformed outputs from the models.
They need balanced examples of verbally expressed knowledge modesty to do this better.
3. Human’s implicit factoid omniscience, while they are online, given Google access, creates expectations that the non-Googling model can’t match.