Cellular Automata and Applications

Latest papers

• No Model Required: Text Entropy Rate Filtering Mitigates Iterative Fine-Tuning Collapse

  Lewis Mitchell · 2 October 2026

  Iterative fine-tuning on synthetic data causes \emph{model collapse}: output diversity narrows as rare patterns are progressively lost, a signature most visible as phrase-level repetition. Existing mitigations either require model log-probabilities, an external oracle, or continued access to real hu…

• Why Do Conventional World Models Fail to Learn Cellular Automata?

  Shaoyang Guo, Ziming Liu · 1 October 2026

  Although conventional world models - auto-regressive or diffusion models based on transformers or convolutional networks - may learn surface statistics of world dynamics, can they learn the exact world dynamics from its observed history? Leveraging cellular automata as a simple testbed, we find the …

• Reasoning with Neural Cellular Automata

  Mayalen Etcheverry, Pietro Miotti, Aidan Sirbu, Konstantin Sch\"urholt, Mariia Drozdova, Arna Ghosh, Blaise Ag\"uera y Arcas, James Manyika, Blake Richards, Eyvind Niklasson · 30 September 2026

  Modern AI architectures used to solve visual reasoning tasks typically rely heavily on global connectivity and synchronization. As biological systems demonstrate, though, sophisticated computation can be performed in a more decentralized fashion. In this work, we test the reasoning capabilities of N…

• On Growth and Form, and Function: Reusable Regulatory Handles Control Phenotypic Variation

  Benedikt Hartl, Milton L. Montero, Marcello Barylli, Sebastian Risi, Michael Levin · 25 September 2026

  How phenotypic transformations are implemented by changes in underlying regulatory dynamics remains a central question in developmental biology. Inspired by D'Arcy Thompson's 1917 "On Growth and Form", we ask whether coherent large-scale transformations of morphology can be encoded as low-dimensiona…

• Online Task Adaptation via Self-Organisation

  Krsto Prorokovi\'c · 25 September 2026

  Neural networks are typically adapted by computing gradients and updating model parameters. We investigate whether task-specific adaptation can instead emerge from a meta-learned self-organising process that requires no gradients at adaptation time. We instantiate this idea with a Neural Cellular Au…

• Emulating Cosmic Structure Formation with a Lagrangian Neural Cellular Automaton

  Cooper Jacobus, Beatriz Tucci, Oliver Philcox · 21 September 2026

  Field-level inference of cosmological initial conditions from galaxy surveys requires a forward model that is simultaneously accurate in the non-linear regime, computationally efficient, and fully differentiable. Traditional N-body simulations are accurate but computationally prohibitive for iterati…

• Neural Cellular Automata Learn General Features in their Hidden Channels

  Etienne Guichard, Stefano Nichele · 21 September 2026

  Modern deep learning models achieve impressive generalization through over-parameterization, but this paradigm often struggles with overfitting and memorization in few-shot regimes. Neural Cellular Automata (NCAs) offer a highly parameter-efficient alternative, yet research has focused primarily on …

• Self-Replicating Neural Cellular Automata: Quantifying Emergent Phenotypic and Genotypic Diversity in an OpenEnded Substrate

  Sanyam Jain, Felix Simon Reimers, Stefano Nichele · 18 September 2026

  We study an in-silico substrate in which every pixel of a two-channel cellular-automata grid carries a tiny neural network (an agent) that senses its Moore neighborhood. A cell persists only by self-replication: a living neighbor is cloned and its weights are mutated by a uniform perturbation, so th…

• Programmable Cellular Automata

  Ahmed Khalifa, Muhammad Umair Nasir, Matthew Siper, Steve James, Julian Togelius · 9 September 2026

  Cellular automata is a local computation paradigm where complex behavior can arise from local interactions between simple functions. This paradigm has been used to explain many systems such as biological processes, traffic simulation, computer networks, etc. In games, cellular automata have been use…

• Do Large Language Models Capture the Diversity in their Training Data?

  Youqi Wu, Farzan Farnia · 3 September 2026

  Large language models are trained to model conditional distributions over text, yet it remains inadequately understood whether they capture the full diversity of plausible outputs present in their training data. We study this question through an information-theoretic lens by comparing the conditiona…

• RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks

  Xingran Chen, Rohit Bhagat, Ghadir Ayache, Rawad Bitar, Yanmin Gong, Salim El Rouayheb · 2 September 2026

  Parameter-efficient fine-tuning methods such as LoRA have become a standard approach for adapting large foundation models. Adopting fine-tuning to distributed settings faces several challenges. Most existing distributed LoRA methods rely on centralized aggregation, and gossip-based decentralized LoR…

• Successive Capacity Growth: Task-Complexity-Driven Width and Depth Expansion for Vision Transformer Encoders in JEPA World Models

  Frederik Berenz · 28 August 2026

  Joint-Embedding Predictive Architectures (JEPAs) for world modeling typically employ fixed-size Vision Transformer encoders that are over-provisioned for simple tasks and under-provisioned for complex ones, with significant redundancy across attention heads. We propose Successive Capacity Growth (SC…

• The Artificial Experimentalist: Discovery and Control of Self-Organizing Phenomena with Autotelic Reinforcement Learning

  Marko Cvjetko, Benedikt Hartl, Michael Levin, Cl\'ement Moulin-Frier, Pierre-Yves Oudeyer · 28 August 2026

  Existing methods for exploring cellular automata and other complex systems mostly operate in open loop: they set initial conditions, execute a full simulation, and observe the outcome, without intervening during execution. We introduce a closed-loop framework based on autotelic reinforcement learnin…

• TextNCA: Neural Cellular Automata for Language Modeling via Hierarchical Local Attention

  Avni Mittal, Avinash Anand, Ashutosh Kumar, Dikshant Kukreja, Kritarth Prasad, Sushane Dulloo, Erik Cambria, Timothy Liu, Zhengkui Wang, Rajiv Ratn Shah · 4 August 2026

  Can a strictly local, iterated, weight-shared computation primitive support language modelling, and which of those three properties actually drives the model's behaviour? We define \textsc{TextNCA}, a 1D causal windowed-attention realisation of the Neural Cellular Automaton primitive, and study a hi…

• CN101 - A Digital Thermodynamic Computer for Generative AI

  Lars Holdijk, Denis Melanson, Zier Mensch, Brandon Birchall, Vincent Cheung, Nicholas Lehrter, Maxwell Aifer, Samuel Duffield, Jan Ole Ernst, Rajath Salegame, Antonio J. Martinez, Gavin Crooks, Miranda Cheng, Zach Belateche, Marc Bright, Patrick J. Coles, Faris Sbahi · 4 August 2026

  Thermodynamic computing is an emerging hardware paradigm, in which stochastic physical dynamics serve as the direct computational primitive. The recent explosion of generative AI has only sharpened the search for alternative approaches to compute, and, as we show in this work, thermodynamic computin…

• Architecture Generalization with MetaNCA

  Meet Barot, Daniel Berenberg, Sina Khajehabdollahi · 10 July 2026

  Self-organization is an emergent property of life, driven by the collective behavior of individual components acting on local information. Biological neurons, through local interactions transmitted through synapses, are able to learn efficiently and can adapt their connections over an organism's lif…

• The Program Is Still There: A Conservation Law for Program Discovery

  Jorge Miguel Silva · 15 June 2026

  Finding the shortest program that generates a sequence is uncomputable, and for six decades that fact has been mistaken for a wall around finding any generating program. It is not a wall but a price, and this paper measures it. For every algorithm that learns about a candidate program only through i…

• Re-examining Low Rank adaptation for private LLM fine-tuning

  Ali Dadsetan, Frank Rudzicz · 1 June 2026

  Privacy is a central concern when fine-tuning large language models (LLMs) on sensitive data, and differentially private stochastic gradient descent (DP-SGD) -- which clips per-sample gradients and adds calibrated Gaussian noise -- is the standard tool for formal privacy guarantees. Both theory and …

• On the Origin of Synthetic Information by Means of Steganographic Inheritance

  Ching-Chun Chang, Isao Echizen · 28 May 2026

  The origin of species has been the mystery of mysteries in natural science. By analogy, the origin of synthetic information, we suggest, is the mystery of mysteries in information science. The question carries a moral weight that a technical account can neither fully resolve nor responsibly ignore, …

• How well behaved is finite dimensional Diffusion Maps?

  Wenyu Bo (Department of Statistics University of Washington Seattle, WA), Marina Meil\u{a} (Department of Statistics University of Washington Seattle, WA) · 15 May 2026

  Under a set of assumptions on a family of submanifolds $\subset {\mathbb R}^D$, we derive a series of geometric properties that remain valid after finite-dimensional and almost isometric Diffusion Maps (DM), including almost uniform density, finite polynomial approximation and reach. Leveraging thes…

• A New Kind of Network? Review and Reference Implementation of Neural Cellular Automata

  Martin Spitznagel, Janis Keuper · 29 April 2026

  Stephen Wolfram proclaimed in his 2003 seminal work "A New Kind Of Science" that simple recursive programs in the form of Cellular Automata (CA) are a promising approach to replace currently used mathematical formalizations, e.g. differential equations, to improve the modeling of complex systems. Ov…

• On the Emergence of Syntax by Means of Local Interaction

  Zichao Wei · 21 April 2026

  Can syntactic processing emerge spontaneously from purely local interaction? We present a concrete instance on a minimal system: an 18,658-parameter two-dimensional neural cellular automaton (NCA), supervised by nothing more than a 1-bit boundary signal, is trained on the membership problem of an ar…

• Evolving Many Worlds: Towards Open-Ended Discovery in Petri Dish NCA via Population-Based Training

  Uljad Berdica, Jakob Foerster, Frank Hutter, Arber Zela · 14 April 2026

  The generation of sustained, open-ended complexity from local interactions remains a fundamental challenge in artificial life. Differentiable multi-agent systems, such as Petri Dish Neural Cellular Automata (PD-NCA), exhibit rich self-organization driven purely by spatial competition; however, they …

• Do We Really Need Permutations? Impact of Model Width on Linear Mode Connectivity

  Akira Ito, Masanori Yamada, Daiki Chijiwa, Atsutoshi Kumagai · 6 March 2026

  Recently, Ainsworth et al. empirically demonstrated that, given two independently trained models, applying a parameter permutation that preserves the input-output behavior allows the two models to be connected by a low-loss linear path. When such a path exists, the models are said to achieve linear …

• Ctrl-World: A Controllable Generative World Model for Robot Manipulation

  Yanjiang Guo, Lucy Xiaoyang Shi, Jianyu Chen, Chelsea Finn · 3 March 2026

  Generalist robot policies can now perform a wide range of manipulation skills, but evaluating and improving their ability with unfamiliar objects and instructions remains a significant challenge. Rigorous evaluation requires a large number of real-world rollouts, while systematic improvement demands…