This is dumb. Literally nothing has changed. Anyone who knows anything about LLM’s knows that they’ve struggled with math more than almost every other discipline. It sounds counter intuitive for a computer to be shit at math, but this is because LLM’s “intelligence” is through mimicry. They do not calculate math like a calculator. They calculate all responses based on a probability distribution constructed from billions of human text inputs. They are as smart, and as fallible, as wikipedia + reddit + twitter, etc, etc. They are as fallible as their constructing dataset.

Think about how ice cream sales correlate with drownings. There is no direct causality, but that won’t stop an LLM from seeing the pattern or implying causality, because it has no real intelligence and doesn’t know any better.

“Prompt engineering” is about understanding an LLM’s strengths and weaknesses, and learning how to work with them to build out a context and efficiently achieve an end result, whatever that desired result may be. It’s not dead, and it’s not going anywhere as long as LLM’s exist.