Unfortunately the most probable response to a question is an authoritative answer, so that’s what usually comes out of them. They don’t actually know what they do or don’t know. If they happen to describe themselves accurately, it’s only because a similar description was in the training data, or they where specifically instructed to answer that way.
More like a collection of descriptions in the training data. It’s not a copy and paste thing, they successfully generalise and extend concepts together quite often.
It’s a good thing if the LLM’s can actually say “I can’t perform that task” instead of just saying yes and hallucinate something.
Unfortunately the most probable response to a question is an authoritative answer, so that’s what usually comes out of them. They don’t actually know what they do or don’t know. If they happen to describe themselves accurately, it’s only because a similar description was in the training data, or they where specifically instructed to answer that way.
More like a collection of descriptions in the training data. It’s not a copy and paste thing, they successfully generalise and extend concepts together quite often.
Yes, I just glossed over that detail by saying “similar to”, but that is a more accurate explanation.