Bio-inspired active sensing using reinforcement learning

Explore how reinforcement learning can drive sensory-motor integration in distributed systems inspired by the whisker barrel cortex, training on images with movable sensors, and potential 3D object recognition.

The objective of this thesis is to model sensory-motor integration in distributed systems, using reinforcement learning (RL), inspired by the whisker barrel cortex in rodents. The whisker barrel cortex, known for its modular architecture and sensory-motor processing, provides a biological foundation for distributed processing models. This study will develop an artificial model that mirrors the whisker barrel cortex's structure, allowing sensory processing and movement control through a message-passing ensemble of neural modules. The research explores how RL can be applied to this domain, leveraging its ability to optimize policies in dynamic environments. The system will be trained on images with movable sensors, simulating sensory interaction with visual data. Additionally, the project aims to extend the model's capabilities to recognize 3D objects, further exploring the interaction between sensory data and movement in a distributed network. The goal is to compare the behavior, scalability, and fault tolerance of RL-driven systems to static systems. Special attention will be given to how sensory and movement data interact and evolve during training, with a focus on advantages and potential challenges that arise during sensory-motor integration. This research will provide insights into artificial sensory-motor processing inspired by the modular nature of the neocortex, pushing the boundaries of distributed control systems in artificial intelligence.

Figure: (a) A rodent uses its whiskers to interact with and identify an object (a cup). The whisker barrel cortex processes sensory information, where each barrel corresponds to a specific whisker, forming a distributed network. This network collaboratively interprets sensory input, leading to object recognition by moving its whiskers. (b) In a computational model, movable sensors scan over an image, and the sensory input is processed by a distributed computational substrate. This setup mimics the message-passing behavior of the barrel cortex, allowing sensory data to be integrated for object identification by moving the sensors.

Goal

The goal of this thesis is to develop and evaluate a reinforcement learning-based model that mimics sensory-motor integration in distributed systems, inspired by the whisker barrel cortex. The system will be trained on images with movable sensors, with the potential to extend toward recognizing 3D objects. The research will assess the model’s behavior, scalability, and fault tolerance, exploring how sensory input and movement interact in a distributed network.

Learning outcome

• Reinforcement Learning: Acquire knowledge of reinforcement learning algorithms and how they can be applied to model sensory-motor integration in distributed systems similar to swarm or multi-agent systems.
• Biological Inspiration and Modeling: Gain insights into the structure and function of the whisker barrel cortex in rodents, and learn how to translate biological concepts into artificial neural network models.
• Technical Skills in Model Development: Develop practical skills in designing, implementing, and training reinforcement learning models with movable sensors, including potential extensions to 3D object recognition.
• Analysis of Distributed Systems: Learn how to assess and evaluate the behavior, scalability, and fault tolerance of distributed artificial intelligence systems.

Qualifications

Need to have:

• Basic Knowledge of Machine Learning and Reinforcement Learning: Understanding of core machine learning concepts, algorithms, and how reinforcement learning works.
• Programming Skills: Proficiency in Python, with experience in machine learning libraries such as TensorFlow or PyTorch.
• Mathematics and Statistics: Foundation in linear algebra, calculus, probability, and optimization techniques.
• Neuroscience Interest: A genuine interest in neuroscience, particularly in the structure and function of neural systems, such as the whisker barrel cortex.

Nice to have:

• Experience with Distributed Systems or Cellular Automata: Prior exposure to or understanding of distributed computing or neural cellular automata.
• Knowledge of Computer Vision: Familiarity with image processing techniques and tools, particularly relevant to training systems on image data and extending models to 3D object recognition.
• Understanding of Biological Neural Systems: Basic knowledge of computational neuroscience or neurobiology, particularly concepts related to sensory-motor systems in the brain.
• Familiarity with Simulation Environments: Experience working with simulation platforms such as MuJoCo or gym environments where reinforcement learning models interact with virtual sensors or objects.

Supervisors

• Mikkel Elle Lepperød
• Sidney Pontes-Filho
• Mia-Katrin Kvalsund

Collaboration partners

• University of Oslo