Justin Derrick@JustinDerrick@mstdn.ca
@kjhealy@mastodon.social I don't know what I was expecting when I tried this for myself with a different LLM...
trenchworms@trenchworms@eldritch.cafe
@JustinDerrick@mstdn.ca @kjhealy@mastodon.social I suspect that GPT-5 has been explicitly trained to defeat the embarrassing "There are 2 R's in Strawberry", but as a result it is now trained the answer questions of the format "How many _'s are there in _____BERRY?" with "3".
The unintended side effect of this is, of course, that it will answer 3 incorrectly and if then asked to justify itself it will pattern match an attempt to do so, leading to this kind of incomprehensible nonsense. Because these things are fundamentally just matching patterns in their training data -- nothing more. That's why GPT-4 gets the blueberry question right.
Another example of this was if you asked it the question: "What weighs more, 1kg of feathers or 10kg of bricks?" -- older models would say "They weigh the same" because the majority of their training data that was of the form "What weighs more, X of feathers or X of bricks" was the riddle it's pretending to answer.