Achievement

One IGERT team developed a low-cost multimodal full-body gesture recognition system using multiple video cameras and a pressure-sensing floor. The system is view-independent and recognizes actions of single subjects or collaborative actions of two people, providing a fully non-intrusive solution to gesture recognition. Using floor pressure sensing, it distinguishes gestures sharing similar/symmetric shapes with different floor pressure distributions. Results show the superior recognition performance of our system compared to state-of-the-art video-based gesture recognition systems. Our system has many applications in human-computer interaction, such as gesture communication, rehabilitation, sports training, and video games. The interdisciplinary nature of the IGERT team allows for selecting and collecting data for important gestures across different areas for system training and testing, ranging from daily activities, to dance movement, and to exercises in movement rehabilitation.