Visualizing the Thought Process of a Large Language Model (R1)
2025-02-17
Researchers visualized the 'thought process' of a large language model, R1, by saving its chains of thought as text, converting them into embeddings using the OpenAI API, and plotting them sequentially with t-SNE. By calculating cosine similarity between consecutive steps, they observed a potential three-stage process: 'search,' 'thinking,' and 'concluding.' Ten diverse prompts were used, ranging from describing how a bicycle works to designing new transportation. The researchers provide methods for accessing the chain-of-thought data and code.