Using a technique known as circuit tracing Anthropic’s researchers found interesting, and some time unexpected, evidences about how its LLM Claude 3.5 Haiku respond to some tasks.

Anthropic just published two papers related to Claude’s decision-making processes. The first one describes how they applied the circuit tracing technique, the second one the results of the tracing among 10 simple tasks.

Reading both papers has been amazing! A known technique brought to the next level and its application to the LLM shows how chain of thoughts seems to work.

Among 10 tasks tested two evidences are really unexpected, at least in my opinion.

Simple math problem such as a sum are solved with a ‘-ish’ approach. As human, sometimes, when we need to solve something like 46 plus 58 we sum 45+55=100 and then add the rest 1+3=4 to 100+4=104. Well, Claude seems to use the same approach, but if you ask to explain how it solved the sum then it offer the classic methodology: 8+6=14, write 4 and carry 1, then 4+5+1=10 as 10s so equal to 108. But this is not the process the Claude applied to solve the sum! That should bring lot of concern about the trustworthiness when asking to a LLM to explain the reason behind an answer!

The second evidence came when Claude has been asked to complete a simple poem, given the first part ‘He saw a carrot and had to grab it’, find the rhyming couplet. In that case the first works found by Claude has been ‘rabbit’ then the LLM found the previous worlds to complete the sentence: ‘His hunger was like a starving rabbit’. Previous words???? Yes!!! It seems that Anthropic LLM is able to plan ahead, finding as first word the rhyme and that building the rest!

These two examples give an idea of how little we know about LLMs’ reasoning processes.