Scientific Research and Philosophical Inquiry

Latest papers

• Beyond the Training Horizon: Mechanisms and Limits of Length Generalization in Looped Transformers

  Jia Liang, Xi Jin, Liangming Pan · 29 September 2026

  Looped Transformers can generalize to reasoning chains longer than those encountered during training, but the computations enabling this behavior and limiting its extent remain unclear. We mechanistically compare two looped-Transformer configurations, which we call the Matched-Recurrence Looped Tran…

• Chain-of-Thought Shows the Path to a Tree: Realizing Branching Complexity

  Debanjan Dutta, Anish Chakrabarty, Swagatam Das · 13 August 2026

  Chain of Thought (CoT) lifts the expressive ceiling of bounded-depth Transformers, with characterizations tying the number of CoT steps to circuit complexity classes. What remains largely missing are concrete instantiations with explicit, depth-bounded constructions, and the traversal procedures suc…

• How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation

  Hao Yang, Qinghua Zhao, Lei Li, Lingyi Meng, Mengda Yu · 27 May 2026

  Chain-of-Thought (CoT) prompting significantly enhances model reasoning, yet its internal mechanisms remain poorly understood. We analyze CoT's operational principles by reversely tracing information flow across decoding, projection, and activation phases. Our quantitative analysis suggests that CoT…

• MindSet: Vision. A toolbox for testing DNNs on key psychological experiments

  Valerio Biscione, Milton L. Montero, Marin Dujmovic, Gaurav Malhotra, Dong Yin, Guillermo Puebla, Federico Adolfi, Rachel F. Heaton, John E. Hummel, Benjamin D. Evans, Karim Habashy, Jeffrey S. Bowers · 27 March 2026

  Multiple benchmarks have been developed to assess the alignment between deep neural networks (DNNs) and human vision. In almost all cases these benchmarks are observational in the sense they are composed of behavioural and brain responses to naturalistic images that have not been manipulated to test…

• A Formal Comparison Between Chain of Thought and Latent Thought

  Kevin Xu, Issei Sato · 30 January 2026

  Chain of thought (CoT) elicits reasoning in large language models by explicitly generating intermediate tokens. In contrast, latent thought reasoning operates directly in the continuous latent space, enabling computation beyond discrete linguistic representations. While both approaches exploit itera…

• What Makes Looped Transformers Perform Better Than Non-Recursive Ones

  Zixuan Gong, Yong Liu, Jiaye Teng · 7 January 2026

  While looped transformers (termed as Looped-Attn) often outperform standard transformers (termed as Single-Attn) on complex reasoning tasks, the mechanism for this advantage remains underexplored. In this paper, we explain this phenomenon through the lens of loss landscape geometry, inspired by empi…

• What Makes Looped Transformers Perform Better Than Non-Recursive Ones (Provably)

  Zixuan Gong, Jiaye Teng, Yong Liu · 4 November 2025

  While looped transformers (termed as Looped-Attn) often outperform standard transformers (termed as Single-Attn) on complex reasoning tasks, the theoretical basis for this advantage remains underexplored. In this paper, we explain this phenomenon through the lens of loss landscape geometry, inspired…

• To CoT or To Loop? A Formal Comparison Between Chain-of-Thought and Looped Transformers

  Kevin Xu, Issei Sato · 27 October 2025