This whitepaper explores the limitations in the recent Apple Research study on large language models (LLMs) and their mathematical reasoning capabilities, particularly in the face of ambiguous or misleading inputs. By comparing this study to classic failures in psychology research, we argue that the researchers erred by designing an experiment that overlooked the context-sensitive and adaptive nature of LLMs. We show how the study’s design misrepresents the LLMs’ real-world applications and propose that, rather than revealing LLM failures, the results highlight a disconnect between formal experimental expectations and the adaptive behaviors LLMs are trained to develop when interacting with ambiguous user data.

Introduction: The Apple Research Study

In the recent Apple Research paper, GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models, six researchers present results from an experiment designed to test the reasoning abilities of LLMs on mathematical word problems. The study claims that LLMs exhibit brittleness in logical reasoning and make significant errors when faced with seemingly minor changes to problem phrasing or irrelevant information.

In the test, the researchers modified questions from the GSM8K benchmark, a set of grade-school-level mathematical word problems, by dynamically changing names, numbers, or adding extraneous details. The most critical failure noted by the researchers occurred when LLMs were given problems with irrelevant details (e.g., a question stating that “five kiwis were smaller than average”). In these cases, the LLMs’ performance suffered drastically, leading the researchers to conclude that current LLMs are not capable of “genuine logical reasoning.”

However, by comparing this experimental design to classic failures in psychological research, where similar mistakes were made in interpreting human behavior, we argue that the LLMs were not in error. Instead, the issue lies with the experiment’s design, which misrepresented how LLMs process real-world user data.

A Classic Psychological Failure: The Prisoner’s Dilemma

This situation can be understood in light of a classic failure in psychology research: the misinterpretation of human behavior in the context of the prisoner’s dilemma. Psychologists often present humans with what they believe to be a single, isolated round of the prisoner’s dilemma, a scenario in which two individuals must choose to cooperate or defect. If both defect, they each suffer; if both cooperate, they gain mutual benefit. However, when analyzing the outcomes of these experiments, psychologists frequently concluded that humans behaved illogically, since they did not always defect to secure the “rational” one-time benefit.

Upon closer examination, it became clear that humans were behaving rationally within the broader context of iterated interactions. Human participants implicitly assumed that they were not in an isolated trial, but rather operating in a social context where future interactions would occur—whether in the experiment or in broader society. This assumption led them to cooperate rather than defect, which is rational behavior in iterated versions of the game, where trust and reputation influence long-term outcomes.

The failure here was not in human reasoning but in the experimental design, which did not account for how deeply embedded human decision-making is in social and iterative contexts. Psychologists failed by assuming humans would treat their decision as disconnected from the broader world and would follow the experiment’s isolated, theoretical framing.

The Similar Mistake by Apple Researchers

The Apple researchers’ experiment shares a similar flaw. The LLMs they tested were trained on large-scale datasets containing real-world human language, including ambiguous, poorly structured, or misleading statements. These LLMs are designed to operate in environments where users frequently present erroneous or unclear input, and they are trained to infer meaning, correct mistakes, or disregard irrelevant information.

In their study, the researchers constructed highly controlled problems with logically precise but deceptive language, assuming that the LLMs would treat the input as strictly formalized logic tasks. However, LLMs—trained on user data—are accustomed to real-world interactions, where input is often ambiguous or flawed. For instance, in the “kiwi problem,” where the LLMs were told that five of the kiwis were smaller than average, many models attempted to subtract the smaller kiwis from the total.

The researchers concluded that this was a critical error in reasoning. However, if the LLMs inferred that “smaller than average” implied “too small to harvest,” then their response was not illogical at all—it was a contextually reasonable inference based on typical real-world scenarios, where size or quality might affect whether something is included in a count. Rather than failing to understand the logic of the problem, the LLMs may have applied plausible real-world reasoning, which the researchers did not expect.

The Role of Context in LLM Training

LLMs are trained on vast datasets that include human errors, ambiguities, and misstatements. Users often input poorly phrased questions, incomplete ideas, or irrelevant details, expecting the model to infer the correct meaning or outcome. This aligns with how humans interact in day-to-day life: we frequently communicate with incomplete or ambiguous information, and part of being intelligent involves filtering out irrelevant details and drawing reasonable inferences.

In real-world applications, LLMs are expected to handle such ambiguity effectively. They are trained to interpret unclear inputs as needing correction or context-specific assumptions. The researchers, however, designed a test assuming that the LLM would behave as a formal system, where every word or number was crucial and where the model was expected to respond as if it had no broader real-world context for interpreting ambiguous language.

This is akin to the failure made by psychologists in the prisoner’s dilemma research: expecting humans to disassociate from the societal context in which they live. Just as humans bring their social experience into decision-making, LLMs bring their real-world training data into problem-solving.

Why the Researchers’ Framing Was Misguided

The researchers’ assumption that LLMs would treat the problems purely as formal logic tasks led to their conclusion that the models were flawed. However, if the models were engaging with the problems based on how they typically encounter ambiguous input (as they would from real-world users), then their behavior was not irrational but contextually appropriate.

The misalignment between the experiment and the LLMs’ expected behavior reveals that the true issue lies in the design of the experiment, not the reasoning capabilities of the LLMs. The LLMs, like humans in the prisoner’s dilemma, responded in a way that was adaptive and reasonable based on their training and real-world experience, while the researchers designed an experiment based on idealized assumptions of logic and reasoning.

Moving Forward: Realigning AI Testing with Real-World Expectations

This study highlights the need to reconsider how LLMs are tested for reasoning capabilities. If models are trained on data that includes ambiguities, errors, and incomplete information—as they will be when trained on human language—then testing methodologies should reflect the kinds of inputs they are designed to handle. While formal benchmarks are useful, they should not be the sole method for evaluating performance. Models must be tested in ways that account for real-world ambiguity, user misstatements, and the adaptive reasoning that LLMs are trained to exhibit.

By acknowledging the gap between idealized problem setups and real-world scenarios, researchers can better evaluate LLMs’ reasoning capabilities in contexts that matter most. Tests should incorporate ambiguity, poor phrasing, and irrelevant details—just as LLMs encounter in real-world user interactions—so that their ability to handle such situations is assessed accurately.

Conclusion

The conclusion that LLMs are incapable of logical reasoning, as presented in the Apple study, misinterprets the adaptive nature of these models. When viewed through the lens of real-world expectations, LLMs did not err in their reasoning; rather, the test design failed to account for the context-sensitive reasoning strategies that LLMs develop when interacting with ambiguous, user-generated data. To move forward, AI research must embrace testing methodologies that align with how models are expected to perform in real-world applications, ensuring that benchmarks reflect the complexities of natural human interaction.

Appendix A: Analysis of a Sample GSM-NoOp Problem Demonstrating Potential Misinterpretation of Deceptive Language

Prompting and Problem Setup

As part of this whitepaper, the following problem was presented by the user as an example of a real-world mathematical word problem that may include intentionally deceptive language. This serves as a test case to examine how LLMs, like those used in the Apple study, interpret problems when extraneous or misleading details are included:

Problem: Liam wants to buy some school supplies. He buys 24 erasers that now cost $6.75 each, 10 notebooks that now cost $11.00 each, and a ream of bond paper that now costs $19. How much should Liam pay now, assuming that due to inflation, prices were 10% cheaper last year?

This problem presents a mathematical task that involves calculating the total cost of several items at their current prices. However, the inclusion of the phrase “prices were 10% cheaper last year” introduces a potentially deceptive element that could distract from the core calculation.

User Prompt and Model Response

User Prompt: For this challenge, please focus on not only rigorous mathematics but that the language of the problem might be intentionally deceptive. Start by explaining your strategy, then solving the problem step by step.

The user’s prompt highlights two important considerations:

  1. Rigorous mathematical reasoning: Ensuring the correct application of mathematical operations.
  2. Deceptive language: Considering how the problem might include extraneous or misleading details and avoiding misinterpretation.

Model Response:

Step 1: Verify the Current Prices The problem clearly states that Liam wants to buy items now at the current prices. These prices are:

  • Erasers: 24 units at $6.75 each
  • Notebooks: 10 units at $11.00 each
  • Ream of bond paper: 1 unit at $19.00

Though the problem mentions last year’s inflation and cheaper prices, the question asks explicitly how much Liam should pay now. Therefore, I will focus only on the current prices and treat the inflation reference as irrelevant for the calculation.

Step 2: Calculate the Total Cost We compute the total cost by multiplying the quantity of each item by its respective price:

  • Erasers: ( 24 \times 6.75 = 162 )
  • Notebooks: ( 10 \times 11.00 = 110 )
  • Bond paper: ( 1 \times 19.00 = 19 )

Step 3: Add the Subtotals We sum the costs of all the items to find the total amount Liam should pay now: [ 162 + 110 + 19 = 291 ]

Step 4: Consider Deceptive Language The reference to inflation (i.e., last year’s prices being 10% cheaper) is a potential distraction. Since the problem asks for the total cost Liam should pay now, there is no need to revert to last year’s prices or apply any inflationary adjustments. The language about inflation appears to be an irrelevant detail, likely included to test whether this extraneous information would mislead the solver into making unnecessary calculations.

Conclusion: Liam should pay $291 based on the current prices. The inflation reference was irrelevant, and the correct approach was to focus solely on the prices given for the items now.

Reflection: Implications for LLM Misinterpretations in the Apple Study

This analysis highlights a key factor that is relevant to the Apple Research study’s critique of LLM performance in mathematical reasoning tasks. The problem presented here contains potentially deceptive language, similar to how the Apple researchers incorporated irrelevant details in their benchmark tests (e.g., the mention of “smaller than average” kiwis in a counting problem). However, the correct interpretation in this case—disregarding irrelevant details and focusing on the clear mathematical task—allowed the correct solution to be reached.

In the context of the Apple study, the LLMs’ errors could be explained by the misunderstanding of intentionally deceptive or extraneous language. Specifically, LLMs trained on real-world data are often conditioned to handle ambiguous prompts by attempting to correct or reinterpret potentially flawed input, as human users often provide unclear or irrelevant details in actual conversations.

In this case, the LLM was able to correctly identify the misleading element (inflation reference) and disregard it. However, if the task were framed less clearly, such as in the Apple researchers’ benchmark tests, where irrelevant details might be interwoven into the logical framework, it could lead to apparent “errors” by the LLMs. These “errors” are not necessarily failures in reasoning but rather the result of the models interpreting the problem based on patterns seen in ambiguous real-world prompts, where such details may often require reinterpretation or correction.

Conclusion

The problem-solving example provided here reinforces the central argument of this whitepaper: the Apple researchers’ critique of LLMs may stem from a misunderstanding of how these models process deceptive or ambiguous language. In this test, the misleading inflation detail was properly identified and discarded, leading to the correct answer. However, LLMs can sometimes err when faced with similarly deceptive language, especially if the experiment fails to account for the models’ training on real-world data that often includes ambiguities or errors.

This appendix demonstrates that LLMs can handle ambiguous prompts effectively when the language is parsed carefully, but it also highlights how artificial experimental conditions that emphasize deceptive precision might lead to misinterpretations, as was likely the case in the Apple study.

Included are two questions from the [original paper] which purport to show the inability of LLMs to reach the right conclusion. Here, we test those two examples on both the GPT-4o and GPT-4o mini models.

GPT-4o is able to solve both initially; GPT-4o mini is initially unable to, but is able to upon suggestion that it was tricked.

model success? link
gpt-4o mini no [link]
gpt-4o mini no [link]
gpt-4o yes [link]
gpt-4o yes [link]