DOYA Kenji
Computational Neuroscience Laboratories
Metalearning and Neuromodulation Research Director, CREST, JST


　　Cyber Rodents are a robotic platform for studying the necessary functions of adaptive agents under the constraints of self-preservation and self-reproduction, which are the defining features of life. We are exploring the mechanisms of learning and evolution by using these rodent-like robots, which are capable of capturing battery packs and exchanging "genes" through infrared communication.

1. Can a robot have emotions?
　　The creation of robots that are able to recognize human emotions and express their own emotions is a major dream for robotic researchers. To realize that goal, studies on the visual recognition of facial expressions and the mechanical implementation of expressive faces using subtle eyebrow and mouth movements are under way. Will it be possible, then, to make a humanoid robot that actually posses human-like emotions by precisely replicating human rules of recognition and expression of emotions?
　　When we talk about emotion, we first think about rich expressions like smiling and crying, and vivid subjective feelings like joy and pain. But what are our emotions really for? Are they just a privilege given to humankind to make life more complex and dramatic?
Let us consider emotion as a biological mechanism that supports two of the basic principles of life: self-preservation and self-reproduction. Strong emotional feelings, like hunger, thirst, and pain promote the survival of individuals, while others, like sexual drive, love, and sorrow aid in reproduction and protection of the genetic community. Human life is full of more complex emotions and desires, like the quest for fame and money, and excitement with curiosity and ambition, but they can somehow be traced back to basic emotions for self-survival and self-reproduction.

2. Cyber Rodents
　　In trying to understand the mechanisms required for adaptive creatures that have the same constraints as biological creatures, we started out to build Cyber Rodents (Figure 1). Cyber Rodents are meant to resemble rats or mice, about which there are a wealth of neuroethological studies, but they are rather large, at 22 cm in body length. Cyber Rodents forage for battery packs and recharge from them for self-survival. Although it is difficult to achieve reproduction in hardware, Cyber Rodents can copy their 'genes' through infrared (IR) communication to reproduce in software. Each Cyber Rodent is endowed with a variety of sensory inputs, including a wide-angle C-MOS camera, an IR range sensor, seven IR proximity sensors, gyros, and accelerometers (Figure 2). Its motion system consists of two wheels and a magnetic jaw that latches onto battery packs. A Cyber Rodent's 'brain' consists of a SuperH-4 CPU chip and an FPGA chip for real-time visual processing, which allow fully on-board, real-time learning and control. It also has a speaker, two microphones, a three-color LED, and an IR communication port for audio-visual communication and the hardware implementation of evolutionary algorithms. It is further equipped with USB and wireless communication ports for connection with a host computer, so that hardware experiments and software simulations can be seamlessly integrated.


Figure 1. Cyber Rodents and battery Packs.





Figure 2. The sensors (green), actuators (blue), and communication devices (red) of a Cyber Rodent.

3. Learning and Evolution
　　Are our behaviors and characters determined by our genes, or learned from experience? This is a frequently debated issue, but in reality, they are mutually dependent; how much we can learn depends on genetic predispositions, and what genes are selected depends on the result of learned behaviors. Cyber Rodents are an ideal experimental platform for studying the interactions between learning and evolution. It is not easy, of course, to reproduce the course of millions of years of evolution in a laboratory, but we may be able to witness some aspects of evolution by appropriately combining robotic hardware and simulation software.
　　In one of our recent experiments, we tested how evolution can help in the learning of battery-capturing behaviors. A Cyber Rodent is 'rewarded' when it successfully catches a battery pack that is positioned randomly in a field and recharges itself. It initially wanders by sending random commands to the wheels, but by associating the rewards and preceding sensory inputs and motor outputs, it gradually becomes able to approach the nearest battery pack in a straight maneuver. This kind of learning process is regulated by a number of 'metaparameters,' such as the speed of memory update and the width of random exploration. In the Cyber Rodents, these metaparameters are encoded in their 'genes.' After an initial learning period, each Cyber Rodent is evaluated by how many battery packs it captures in a reproductive period. The genes of the best individuals are selected and reproduced with crossovers and random mutations, and then used as the parameters of learning in the next generation. By repeating this process in simulation, metaparameters that fit the properties of the environment were found. We verified that those metaparameters are helpful in the hardware learning experiments of Cyber Rodents (Figure 3).


Figure 3. Experiments for learning to capture battery packs.

4. Towards Understanding of the Mind
　　Will a robot like a Cyber Rodent turn out to be of practical use, for example, for housekeeping or caring for patients? Probably not in the near future, although some hobbyists may enjoy watching how robots learn and evolve on their own, like some kind of high-tech bonsai art. What we are aiming for in the Cyber Rodent project is to understand how our minds and emotions work and evolve. Such an understanding would eventually lead us to fundamental principles for designing robots that can understand the human mind.
　　If a robot is supposed to behave just as it was programmed, there's no room or necessity for it to have anything like emotion. Robots have the need for, and the possibility of gaining, something like emotion only when they are equipped with highly autonomous learning capabilities, and when their performance affects whether they are switched on or off, and whether the production of their 'species' will be continued or terminated.
　　These days, it is a must for intelligent robots to have a wireless internet connection. And with their brains directly wired to the internet, fancy operations that are not possible in humans - for ethical or technical reasons - such as telepathy, brain transplants, and cloning, are easily realized. What sort of power will robots have with such capabilities? How fantastic, or how dangerous, could they be? It is now time for us researchers to foresee the various possibilities, not simply with SF-like thought experiments, but through realistic technical assessments.

References
Eriksson, A., Capi, G., and Doya, K (2003). Evolution of meta-parameters in reinforcement learning algorithm. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2003).
Elfwing, S., Uchibe, A., and Doya, K. (2003). An evolutionary approach to automatic construction of the structure in hierarchical reinforcement learning. Genetic and Evolutionary Computation Conference (GECCO2003)