How an LLM thinks. Anthropic knows it, finally!

Using a technique known as circuit tracing, Anthropic’s researchers found interesting, and sometimes unexpected, evidence about how their LLM Claude 3.5 Haiku responds to specific operational tasks.
Anthropic’s Papers on Decision-Making Processes
Anthropic published two scientific papers focused on Claude’s internal decision-making processes:
- The first paper describes how they applied the circuit tracing technique;
- The second paper illustrates the findings from tracing across 10 specific tasks.
Reading both papers has been amazing! A known technique brought to the next level, and its application to Large Language Models demonstrates for the first time how the so-called chain of thought actually seems to work.
Among the 10 tasks tested, two findings are particularly unexpected, at least in my opinion.
First Evidence: Approximation Arithmetic and the Explanation Paradox
A simple math problem, such as an addition, is solved using an “estimation” or approximate approach.
As humans, when we need to add 46 and 58, we often calculate:
- 45 + 55 = 100
- then add the remaining difference 1 + 3 = 4
- reaching 100 + 4 = 104.
Claude appears to use the exact same strategy. However, if you ask it to explain step by step how it computed the sum, it offers the classic textbook methodology:
“8 + 6 = 14, write 4 and carry 1, then 4 + 5 + 1 = 10 tens, so the total is 104!”
Yet this is not the process Claude actually applied to solve the sum!
This disconnect should raise serious reflections and questions regarding transparency and trustworthiness when asking an LLM to explain the rationale behind its answers.
Second Evidence: Rhyme-First and Backwards Sentence Planning
The second piece of evidence emerged when Claude was asked to complete a simple poem: given the opening line “He saw a carrot and had to grab it”, it needed to find the rhyming couplet.
In that case, the first word identified internally within Claude’s circuits was “rabbit”.
Only after locking in the rhyme did the LLM search for and generate the preceding words to complete the line:
“His hunger was like a starving rabbit”
The preceding words???? Yes!
It appears that Anthropic’s model is capable of planning ahead, identifying the final rhyme first and then building the rest of the sentence backwards!
Conclusions: What These Findings Teach Us
These two examples give a tangible measure of how little we still truly know about the internal reasoning processes of LLMs.
Continued research into transparency, circuit interpretability, and AI explainability will be fundamental for the trustworthy adoption of generative technologies across organizations.

Written by Ernesto Amato
Executive Leader and Fractional Chief AI Officer. Supporting Boards of Directors and C-Suite leaders in defining and implementing sustainable AI Governance, Enterprise AI Adoption, and Cultural Transformation.