Research Question

How can an autonomous agent form plans to achieve goals and react to obstacles, driven entirely by a system of neural networks? That is, no hard-coded commands!

Theories from Cognitive Science

Emotions are important.

Emotions are more than just the expressions that we think of. They are important for providing structure to the decision-making process. They:

• Drive motivation
• Track goals
• React when plans fail or go off course
• Kick off plan formation

There is a specific set of emotional states that is important for flexible decision-making.

The states in this set are analogous to a few of the commonly known "5 Stages of Grief":

Designing a circuit from brain modules

Studies reveal a lot that explains how each region in the prefrontal cortex (PFC) functions. Other psych studies uncover patterns in human behavior (e.g., how a child reacts to obstacles in a social context). Combining these insights, we can put together a simple circuit of how decision-making (or behavioral reaction) flows:

For more information on the "Still Face" experiment and how the PFC might process behavior, check out the video: Human Emotion Model in Humanoid Robot

Model Design - Background

The technology that drives Binny (both the virtual agent and the physical humanoid rover robot) can be called an artificial neural network control system. This system can be broken into two parts: an Abstraction (or Motor Planning) circuit, and an Emotion (or Flexibility) circuit. The two circuits are further broken into smaller modules, and the two circuits work tightly together to drive learning and planning.

The modular system of this design is inspired by current scientific literature on human neuroanatomy. The functions of the various modules also follow general rules described in a Trends in Neuroscience article (ref below) written by my former thesis advisor, Randy O'Reilly.

The Abstraction system is based off of the functions theorized in the 'Cold' and 'What'/'How' regions of the brain (see image). This circuit involves pulling out patterns of spatial relationships between goals and other objects surrounding the agent. This system also allows the agent to form complex plans to achieve its goal. Those plans can involve tools (subgoals) that help the agent overcome an obstacle.

The Emotion (Flexibility) system is based off of the functions theorized in the 'Hot' zones of the neocortex. This is the circuitry that allows the agent to form goal representations and to integrate them with the representations of the physical world (partly formed by the Abstraction system). The Emotion system helps select a strategy to complete a task and tracks the agent's progress towards its goals at the end of that motor plan. If obstacles come along, this system flexibly recruits other regions of the Emotion or Abstraction systems to get the agent back on a better track.

Many of the smaller modules throughout this control system were designed with other various functions or properties. In other words, some fine-tuning was needed to allow the network as a whole to operate smoothly. Nevertheless, the various functions and properties assigned were based off of theory discussed in several other scientific journal articles covering various neocortical and subcortical areas of the human and animal brain. The references are listed below.

References:

• Alia-Klein, N., Gan, G., Gilam, G., Bezek, J., Bruno, A., Denson, T. F., Hendler, T., Lowe, L., Mariotti, V., Muscatello, M. R., Palumbo, S., Pellegrini, S., Pietrini, P., Rizzo, A., & Verona, E. (2020). The feeling of anger: From brain networks to linguistic expressions. Neuroscience and Biobehavioral Reviews, 108, 480–497. https://doi.org/10.1016/j.neubiorev.2019.12.002
• Chai, W. J., Abd Hamid, A. I., & Abdullah, J. M. (2018). Working memory from the psychological and neurosciences perspectives: A review. Frontiers in Psychology, 9, 401. https://doi.org/10.3389/fpsyg.2018.00401
• Dixon, M. L., Thiruchselvam, R., Todd, R., & Christoff, K. (2017). Emotion and the prefrontal cortex: An integrative review. Psychological Bulletin, 143(10), 1033–1081. https://doi.org/10.1037/bul0000096
• Frank, M. J. (2006). Hold your horses: A dynamic computational role for the subthalamic nucleus in decision-making. Neural Networks, 19, 1120–1136. https://doi.org/10.1016/j.neunet.2006.03.006
• Haber, S. N., & Knutson, B. (2010). The reward circuit: Linking primate anatomy and human imaging. Neuropsychopharmacology Reviews, 35, 4–26. https://doi.org/10.1038/npp.2009.129
• Lim, S., & Goldman, M. S. (2013). Balanced cortical microcircuitry for maintaining information in working memory. Nature Neuroscience, 16(9), 1306–1314. https://doi.org/10.1038/nn.3492
• McHaffie, J. G., Stanford, T. R., Stein, B. E., Coizet, V., & Redgrave, P. (2005). Subcortical loops through the basal ganglia. Trends in Neurosciences, 28(8), 401–407. https://doi.org/10.1016/j.tins.2005.06.006
• O'Reilly, R. C. (2010). The What and How of prefrontal cortical organization. Trends in Neurosciences, 33(7), 355–361. https://doi.org/10.1016/j.tins.2010.05.002
• Roy, M., Shohamy, D., & Wager, T. D. (2012). Ventromedial prefrontal-subcortical systems and the generation of affective meaning. Trends in Cognitive Sciences, 16(3), 147–156. https://doi.org/10.1016/j.tics.2012.01.005
• Sallet, J., Mars, R. B., Quilodran, R., Procyk, E., Petrides, M., & Rushworth, M. F. S. (2011). Neuroanatomical basis of motivational and cognitive control: A focus on the medial and lateral prefrontal cortex. In R. B. Mars, J. Sallet, M. F. S. Rushworth, & N. Yeung (Eds.), Neural basis of motivational and cognitive control (pp. 5–20). MIT Press. https://doi.org/10.7551/mitpress/8791.003.0003
• Shackman, A. J., Salomons, T. V., Slagter, H. A., Fox, A. S., Winter, J. J., & Davidson, R. J. (2011). The integration of negative affect, pain and cognitive control in the cingulate cortex. Nature Reviews Neuroscience, 12, 154–167. https://doi.org/10.1038/nrn2994
• Wallis, J. D. (2007). Orbitofrontal cortex and its contribution to decision-making. Annual Review of Neuroscience, 30, 31–56. https://doi.org/10.1146/annurev.neuro.30.051606.094334
• Walton, M. E., Croxson, P. L., Behrens, T. E. J., Kennerley, S. W., & Rushworth, M. F. S. (2007). Adaptive decision making and value in the anterior cingulate cortex. NeuroImage, 36, T142–T154. https://doi.org/10.1016/j.neuroimage.2007.03.029