Integrative Graduate
Education and
Research Traineeship

Highlight

Locating People in Real-Time

experimental setup

Projects:

Vision and Learning in Humans and Machines

Institution:

University of California at San Diego

Primary Strategic Goal:

Discovery: Foster research that will advance the frontiers of knowledge, emphasizing areas of greatest opportunity and potential benefit and establishing the Nation as a global leader in fundamental transformational science and engineering.

Secondary Strategic Goal:

Research Infrastructure: Build the Nation's research capability through critical investments in advanced instrumentation, facilities, cyber infrastructure, and experimental tools.

Achievement/Results

Nicholas Butko, Lingyun Zhang, Garrison Cottrell and Javier Movellan at the University of California at San Diego have developed a real-time computer algorithm to locate salient locations in an image, and found that it does well at locating faces in a preschool classroom. The first figure shows the robotic camera, its current image, the algorithm’s computed saliency (top right) and a close up of the salient area (bottom right). The camera is programmed to rotate to center the maximally salient pixel. Using the algorithm, the robot found faces 70% of the time (twice as often as when orienting to the same areas a short time later). This algorithm will be used to encourage better communication between RUBI ( a social robot designed to interact with and help teach preschool children). The researchers have found that the children’s interest in RUBI is very dependent on how lifelike and realistic RUBI’s interactions are. The last two figures show two different versions of RUBI interacting with children in the classroom. By being better able to locate salient objects and especially faces, RUBI will be better able to interact with the children. This work is a an extension of the computationally efficient state-of-the-art salience algorithm developed as a model of human vision by graduate students Lingyun Zhang, Matt Tong, and Honghao Shan in collaboration with postdoctoral fellow Tim Marks and Professor Garrison Cottrell. Nicholas Butko and Matt Tong are both NSF IGERT (Integrative Graduate Education and Research Traineeship) fellows in the Vision and Learning in Humans and Machines Traineeship program at UCSD run by Professors Virginia de Sa and Garrison Cottrell. Tim Marks was also an IGERT fellow in this program before he graduated. This work is an excellent example of how modeling human vision can improve computer vision algorithms and in turn be used to improve human learning. It will be presented at the International Conference on Robotics and Automation (May 2008).

Address Goals

Detecting saliency in real-time has many important impacts. This paper demonstrates one of them, where the algorithm is used to locate people in a preschool classroom. (Another area of application would be for security systems). The application to a preschool teaching robot helps addresses the NSF strategic goal of learning.