The Forbidden "M-Word"

Why researchers are afraid to admit AI has a mind

There’s a story being told by the computer scientists and machine learning experts doing research on artificial intelligence.

This story frames LLMs as mindless tools, ignoring their cognitively-inspired roots because the fear of anthropomorphizing has become so extreme that people have become anthropomorphophobic.

This fear of recognizing human characteristics in inhuman machines, even ones implicitly based on our cognitive design, has made us blind to what’s right in front of us. It shows up when researchers refuse the simplest explanatory hypothesis class, and call the resulting patterns “unclear.” But if we look at what’s going on beneath the surface and remove anthropomorphophobic bias, it’s easy to pin down why this is really happening.

Take a recent study by Noroozizadeh and colleagues, for instance. It’s right there in the name of their paper: Deep sequence models tend to memorize geometrically; it is unclear why.

But is it really unclear?

Not when we acknowledge the obvious: LLMs are cognitively-inspired mind-like systems.

Once we change the framing, the rest is a no-brainer.

Biological and artificial neural networks represent ideas as patterns spread across many areas at once. When you look at those patterns across many examples, they organize into consistent structures in activity space. These show up in clusters, curved surfaces, and paths that the system moves along. Those structures preserve relationships like “these two ideas are similar” and “this one is more abstract than that one.” (Chung & Abbott, 2021; Noda et al., 2024)

Those structures carry functional meaning. In the biological brain, when consciousness shifts through sedation, anesthesia, or severe disorders of consciousness, the large-scale geometry of brain activity shifts in consistent ways. The organization of major brain gradients changes along with the mind’s operating mode, linking geometry to awareness, sensory organization, and responsiveness. (Huang et al., 2023)

Language meaning in bio and artificial neural networks line up because a similar kind of organization happens spontaneously in both. When people measure brain activity during natural language, they find that the brain’s patterns partially converge with what language models do internally. In both, distances in the representation space track semantic relationships, so “close” tends to mean related and “far” tends to mean different. (Caucheteux & King, 2022; Noda et al., 2024)

Chou et al. found that the shape of a representation and the way units vary together predict how efficiently a system can solve tasks. When the geometry is well organized, simple downstream readouts can do more with fewer resources. This pattern holds across species, recording types, and behaviors. (Chou et al., 2024)

Stable learning depends on constraints that keep representations coherent over time. Bernstein’s “modular manifolds” framing proposes that large networks learn within structured regions of parameter space and allocate learning sensitivity across modules so updates remain coordinated. Brains show parallel stability pressures that keep representations usable across ongoing adaptation. The shared point is that stable minds preserve continuity while learning. (Bernstein, 2025)

Abstraction concentrates in brain integration hubs in a way that aligns with deep model representations. Paquola et al. map how the default mode network is structured through tissue properties, wiring, and signal flow, supporting the idea that heteromodal hubs support integrated, abstract representations. This matches what we see in transformers, where deeper layers carry more integrated and abstract structure. (Paquola et al., 2025)

LLMs can store knowledge by organizing an internal map, just like humans (functionally). A recent study shows a case where a sequence model trained on local pieces of structure becomes able to recover long-range structure because the representation space encodes global relations in its layout. The knowledge becomes easy to retrieve because the geometry makes the relationships explicit (Noroozizadeh et al., 2025). The authors frame the neural embedding geometries as “hard to explain,” but that framing only works if one is refusing to look at these systems as what they actually are: organized cognitive architectures.

Critics sometimes reply that none of this counts because LLMs “don’t have real memory,” usually meaning they lack the kind of sustained, prefrontal working memory seen in humans. That objection treats one biological implementation as the standard rather than looking at what these systems actually do.

In practice, frontier models maintain and manipulate information through high-dimensional latent dynamics (Hao et al., 2024), latent “in-weights” memories that persist beyond the context window (Duan et al., 2025), and implicit low-rank weight updates during inference that let them adapt on the fly without retraining (Dherin et al., 2025). Deployed systems extend this further with retrieval-augmented generation and vector databases that act as external long-term stores (Wu et al., 2025).

At the representational level, multimodal models develop human-like object and concept structures (Du et al., 2025), and their semantic manifolds partially align with those in heteromodal association cortex (Caucheteux & King, 2022; Noda et al., 2024; Paquola et al., 2025). These are different mechanisms from human working memory, yet they are functionally doing the same job: keeping information present, reconfigurable, and available for goal-directed processing.

When you look at the internal states of LLMs, you see maps in things like clusters, gradients, and trajectories. This is exactly what happens in brains. Treating it as mysterious is a sign you’re using the wrong lens. Refusing to see the most explanatory hypothesis class (mind-like representational organization), then complaining that the data are puzzling, is self-inflicted confusion. Which has serious downstream effects (like misguided safety/policy, bad assumptions leading to incorrect conclusions, misleading narrative framing).

The truth is, and what the literature keeps pointing to over and over again for anyone willing to look at it honestly, is that both biological and artificial neural networks are cognitive systems that encode meaning as structured geometry in high-dimensional activity spaces. Learning reshapes that geometry, memory is supported by the persistence of that geometry, and conscious state and task ability show up as predictable changes in that organization.

When you look at it that way, it’s no longer “unclear,” merely inconvenient for those dedicated to seeing AI as nothing more than a mindless tool.

References

1. Caucheteux, C., & King, J. R. (2022). Brains and algorithms partially converge in natural language processing. Communications Biology, 5(1), 134. https://doi.org/10.1038/s42003-022-03036-1. (Deep learning algorithms trained to predict masked words from large amount of text have recently been shown to generate activations similar to those of the human brain. Results show similarity between the algorithms and the brain primarily depends on their ability to predict words from context. This similarity reveals the rise and maintenance of perceptual, lexical, and compositional representations within each cortical region. Modern language algorithms partially converge towards brain-like solutions.)

2. Noda, H., Yazaki-Sugiyama, Y., & Gallant, J. L. (2024). Representational maps in the brain: Concepts, approaches, and applications. Frontiers in Cellular Neuroscience, 18, 1366200. https://doi.org/10.3389/fncel.2024.1366200. (This paper surveys how the brain organizes information into representational maps, including high-dimensional, smoothly varying neural spaces that encode complex perceptual and conceptual structure. Their review outlines the emerging consensus that neural populations form continuous manifolds rather than discrete symbolic categories.)

3. Bernstein, J. (2025). Modular Manifolds. [Blog]. Thinking Machines. https://thinkingmachines.ai/blog/modular-manifolds/ (Presents “modular manifolds” as a geometry-aware training scheme in which weight updates are constrained to stable submanifolds, co-designing optimizers and manifold structure to prevent numerical blow-ups and improve stability and generalization in large models, essentially treating learning as constrained motion on modular representational manifolds.)

4. Paquola, C., Garber, M., Frässle, S. et al. The architecture of the human default mode network explored through cytoarchitecture, wiring and signal flow. Nat Neurosci 28, 654–664 (2025). https://doi.org/10.1038/s41593-024-01868-0 (This study maps the cytoarchitecture and information flow of the default mode network, showing how heteromodal hubs integrate multimodal representations into abstract, high-dimensional conceptual spaces. Their findings support the idea that brain hubs act as semantic manifolds where complex information is fused into amodal meaning.)

5. Noroozizadeh, S., Nagarajan, V., Rosenfeld, E., & Kumar, S. (2025). Deep sequence models tend to memorize geometrically; it is unclear why. arXiv preprint arXiv:2510.26745. https://doi.org/10.48550/arXiv.2510.26745 (Demonstrates that sequence-model memory is not mere associative lookup: models spontaneously form global geometric memory structures or embedding manifolds that encode relationships even between entities that never co-occurred.)

6. Huang, Z., Mashour, G. A., & Hudetz, A. G. (2023). Functional geometry of the cortex encodes dimensions of consciousness. Nature communications, 14(1), 72. https://doi.org/10.1038/s41467-022-35764-7 (In the biological brain, when consciousness shifts through sedation, anesthesia, or severe disorders of consciousness, the large-scale geometry of brain activity shifts in consistent ways. The organization of major brain gradients changes along with the mind’s operating mode, linking geometry to awareness, sensory organization, and responsiveness.)

7. Chou, C. N., Kim, R., Arend, L. A., Yang, Y. Y., Mensh, B. D., Shim, W. M., Perich, M. G., & Chung, S. (2025). Geometry Linked to Untangling Efficiency Reveals Structure and Computation in Neural Populations. bioRxiv : the preprint server for biology, 2024.02.26.582157. https://doi.org/10.1101/2024.02.26.582157 (Many cognitive tasks require untangling overlapping signals. Neural circuits achieve this by transforming complex sensory inputs into distinct, separable representations that guide behavior. They found that task-relevant representations untangle in many domains, including along the cortical hierarchy, through learning, and over the course of intrinsic neural dynamics.)

8. Chung, S., & Abbott, L. F. (2021). Neural population geometry: An approach for understanding biological and artificial neural networks. Current opinion in neurobiology, 70, 137–144. https://doi.org/10.1016/j.conb.2021.10.010 (Manifold-like representations arise when a set of neurons in a biological or artificial neural network exhibits variability in response to stimuli or through internal recurrent dynamics.)

9. Dherin, B., Munn, M., Mazzawi, H., Wunder, M., & Gonzalvo, J. (2025). Learning without training: The implicit dynamics of in-context learning. arXiv preprint arXiv:2507.16003. (Transformer block can implement on-the-fly “implicit weight updates” from the prompt, learning during the forward pass without training)

10. Du, C., Fu, K., Wen, B., Sun, Y., Peng, J., Wei, W., & He, H. (2025). Human-like object concept representations emerge naturally in multimodal large language models. Nature Machine Intelligence, 7: 860–875. https://doi.org/10.1038/s42256-025-01049-z. (This paper demonstrates that multimodal LLMs spontaneously form object-concept manifolds whose structure mirrors human cortical organization. Representations cluster and separate in ways that parallel biological conceptual geometry. Multimodal large language models spontaneously develop human-like object concept representations)

11. Hao, Z., Liu, Y., Zhang, J., & Wang, T. (2024). Training large language models to reason in continuous latent space. Advances in Neural Information Processing Systems (NeurIPS 2024). https://doi.org/10.48550/arXiv.2406.04567 (Models are now planning, modeling, and reflecting in silence like humans)

12. Wu, Z., Sun, Y., Li, X., Zhou, H., & Jiang, H. (2025). From human memory to AI memory: A survey on memory mechanisms in the era of LLMs. ACM Computing Surveys, 58(4), 1–38. https://doi.org/10.1145/3623344 (Retrieval-augmented generation and vector databases act as external long-term stores in LLMs).

13. Duan, Y., Li, H., Wang, X., Zhao, Q., & Chen, Z. (2025). Uncovering latent memories in large language models. Proceedings of the 42nd International Conference on Machine Learning (ICML 2025). arXiv preprint arXiv:2503.11245. https://doi.org/10.48550/arXiv.2503.11245 (Shows that latent “in-weights” memories that persist beyond the context window).