An approach to neural network modeling on the LEGO Mindstorms robotics platform is presented. The method provides a simulation tool for invertebrate neuroscience research in both the research lab and the classroom. This technique enables the investigation of biomimetic robot control principles.
We present a method to use the commercially available LEGO Mindstorms NXT robotics platform to test systems level neuroscience hypotheses. The first step of the method is to develop a nervous system simulation of specific reflexive behaviors of an appropriate model organism; here we use the American Lobster. Exteroceptive reflexes mediated by decussating (crossing) neural connections can explain an animal’s taxis towards or away from a stimulus as described by Braitenberg and are particularly well suited for investigation using the NXT platform.1 The nervous system simulation is programmed using LabVIEW software on the LEGO Mindstorms platform. Once the nervous system is tuned properly, behavioral experiments are run on the robot and on the animal under identical environmental conditions. By controlling the sensory milieu experienced by the specimens, differences in behavioral outputs can be observed. These differences may point to specific deficiencies in the nervous system model and serve to inform the iteration of the model for the particular behavior under study. This method allows for the experimental manipulation of electronic nervous systems and serves as a way to explore neuroscience hypotheses specifically regarding the neurophysiological basis of simple innate reflexive behaviors. The LEGO Mindstorms NXT kit provides an affordable and efficient platform on which to test preliminary biomimetic robot control schemes. The approach is also well suited for the high school classroom to serve as the foundation for a hands-on inquiry-based biorobotics curriculum.
Neurophysiological investigations over the past 100 years have broadened tremendously our knowledge of nervous system structure and function. However, the majority of nervous system research to date has relied on the use of isolated preparations or restrained subjects. While there have been many successful efforts to record neural activity from freely behaving animals2-5, the biorobotic approach provides a valuable tool to allow for nervous system manipulation in order to test systems level neuroscience hypotheses6. Simulated nervous systems operating on robots can be experimentally manipulated and allow for the extension of software modeling to the physical world. This approach has been well implemented in the academic world7,8 but the process of building a biomimetic robot for hypothesis testing can be expensive and time consuming. We present a method to execute the biorobotic approach using a commercially available robotics kit (LEGO Mindstorms NXT 2.0). The goal of this method is to provide a rapid and efficient way to test systems level neuroscience hypotheses on robotic9 or bio-hybrid10 embodied neural network simulations. Accelerating the process from hypothesis to experiment improves research productivity. The simple LEGO Mindstorms platform provides a test bed for biomimetic sensors and neural networks which we demonstrate using the American Lobster (Homarus americanus) as a model organism. The method also provides a powerful hands-on educational tool in the classroom as students can design and manipulate nervous systems for their own robots11.
1. Building the Robot Model
2. Programming the Nervous System
3. Nervous System Simulation Testing
Inputs from a lobster’s claws into its nervous system mediate obstacle negotiation in a novel environment. Figure 1 shows a screenshot of the video used to analyze the behavior of a LEGO robot (Figure 1A) and a lobster (Figure 1B) in the test arena. The test arena was unmodified between animal and robot tests except that the water was emptied from the tank for the robot trials.
Video tracking results are displayed in Figure 2. Animal and robot travel paths were automatically tracked using MouseLabTracker20 in MATLAB. Representative paths for lobster (black lines) and LEGO robot trials (colored lines) are shown. In the robot trials, synaptic strength from the claw bump sensory neurons into the nervous system was varied and represented by different colored paths (red, low excitation; blue, medium excitation; green, high excitation). The hypothetical nervous system with mid level excitatory connections from the claw bump sensory neurons produced similar behaviors to the animal. Quantitative parameters can be extracted from the data such as path length and average walking speed (Table I).
Figure 1. Overhead video frames of the test arena for LEGO robot (A) and lobster (B) trials. Conditions were kept identical except that water was added to the tank for the animal trials. The high contrast video image facilitates automatic tracking using MATLAB software.
Figure 2. Tracking of lobster (black lines) and LEGO robot (colored lines) paths through the test arena shown in Figure 1. On the robot, the synaptic strength from claw bump sensor neurons was varied (red, low excitation; blue, medium excitation; green, high excitation). The * symbol denotes the starting position and filled black shapes show the location of obstacles. Since head position was being tracked, apparent obstacle contact varies due to variability in claw position.
Average path distance (cm) | Average travel speed (cm) | |
Lobster | 177.37 | 1.03 |
LEGO Robot (high excitation) | 162.72 | 0.94 |
LEGO Robot (mid excitation) | 165.32 | 0.85 |
LEGO Robot (low excitation) | 51.02 | 0.69 |
Table I. Averaged parameters from analysis of video behavior data for lobsters and for LEGO robots with varied strengths of input synapses from the claw bump sensory system.
When initiating biorobotic nervous system simulation experiments, there are a few important guidelines to follow. Choosing the right model organism is critical: pick an organism that is easy to obtain and maintain. Invertebrates are ideal because they do not usually require institutional approval for experimentation and their husbandry needs are often less demanding than those of vertebrates. From a scientific perspective, it is beneficial to choose an animal that has an established neuroethological literature record from which to draw hypotheses for testing21.
Just as important as choosing a suitable organism is the selection of appropriate behavioral reflexes for study. The LEGO NXT platform can only handle four sensors without custom modifications so selecting a relatively simple sensory system with limited sensors is important. Simple behavioral reflexes are often well described and provide more flexibility in the hypotheses that can be tested. The computing power of the NXT is a limiting factor too so efforts should be made to reduce the scope of the nervous system simulation. Scientists often publish proposed neural networks explaining specific behaviors and it is easiest to select one of these. Simple taxes and kineses are well suited to this biorobotic approach22.
While this approach is useful for rapid testing of nervous system hypotheses, there are several limitations to keep in mind. The scope of testing possible is limited. Complicated sensory systems cannot be used with this platform. For example, complex visual processing is beyond the capability of the NXT processor but related questions could be addressed by investigating simpler components of the visual system, such as optical flow processing. The sensor suite for the NXT provides some inherent limitations as to the scope of investigations available. While custom sensors can be developed and used on the NXT, this is a time consuming process that requires electronics expertise, possibly negating the advantages of the platform for rapid experimental implementation.
While there are limitations on the range of hypotheses that can be tested, this approach provides a unique tool to early stage implementation of embodied nervous system simulations. The LEGO Mindstorms NXT platform can serve as a stepping stone to a more thorough testing of systems level hypotheses as demonstrated by such robots as RoboLamprey9 and Barbara Webb’s robotic cricket23. Simple components of extensive biomimetic control architectures can be tested in a rapid and straightforward manner. The method can also advance bio-hybrid simulations that control virtual agents to include embodied vehicle control24, and it can serve as a biomimetic control framework to interface between brain and machine25.
Using the LEGO Mindstorms platform allows for the transfer of this biorobotic approach to a broader audience in schools and laboratories outside of robotics. The tools have been used by high school and college students11 and provide an exciting inquiry-based method to teach neuroscience, robotics, and the value of interdisciplinary science. Laboratories without the equipment or background needed to build robots can use this well-supported mainstream robotics kit to use robots as a tool in their work. The biorobotic approach could apply to research in a range of fields, from neuroscience and ethology to biomechanics and social behavior.
The authors have nothing to disclose.
We thank Dr. Chris Rogers (Tufts University) for programming and manuscript suggestions. We thank Alex Giuliano and Deborah Lee for video production support.
Funding provided by an NSF Graduate Research Fellowship and an ONR MURI in Synthetic Biology.