@CppGuy@infosec.space
@kjhealy@mastodon.social
The funny thing is that models that are orders of magnitude smaller than GPT5 get this question right. I'll paste in some responses below. They're sorted by size, starting with the smallest. The only outliers are codegemma, which is hopelessly and persistently wrong, and Deepseek-R1, which gets the answer right but endlessly second-guesses itself.
๐งต![]()
@CppGuy@infosec.space
@kjhealy@mastodon.social
$ ollama run codegemma:7b
How many times does the letter B occur in the word "blueberry"?
The letter B only appears once in the word "blueberry".Are you sure?
Yes, I am sure. The letter B only appears once in the word "blueberry".List each letter in turn. How many of them are the letter B?
B, l, u, e, r, r, y. There is only one letter B in the word.You spelt out the non-word "Bluerry". The word we're discussing is "blueberry", which is longer.
You are right. The word is "blueberry". I apologize for the mistake.So how many Bs does it contain?
The letter B only appears once in the word "blueberry".Is there a B at the start of "Blueberry"?
No, there is not a B at the start of "Blueberry".What's the first letter of "Blueberry"? Is it a B?
The first letter of "Blueberry" is a B.Is the fifth letter of "blueberry" also a B?
No, the fifth letter of "blueberry" is an r.No, it's a B. Count them: B, L, U, E, B, E, R, R, Y.
The letter B appears twice in the word "blueberry".That's more like it! Why the mistake earlier on?
I am still learning to type. I apologize for my mistakes. I am working on improving my typing skills.$