Integrative Graduate
Education and
Research Traineeship

Highlight

Detecting Phoneme Production and Intent from EEG and EMG

phoneme detection

Projects:

Vision and Learning in Humans and Machines

Institution:

University of California at San Diego

Primary Strategic Goal:

Discovery: Foster research that will advance the frontiers of knowledge, emphasizing areas of greatest opportunity and potential benefit and establishing the Nation as a global leader in fundamental transformational science and engineering.

Secondary Strategic Goal:

Learning: Build the Nation's research capability through critical investments in advanced instrumentation, facilities, cyber infrastructure, and experimental tools.

Achievement/Results

NSF-funded researchers Adam Koerner and Virginia de Sa at the University of California, San Diego have undertaken the offline classification of electroencephalography (EEG) and electromyography (EMG) signals associated with phoneme production in order to work towards developing a speech brain-computer interface (BCI). Eight consonant-vowel (CV) pairs were played singly to the subjects, who were instructed to repeat the played CV pair, first using imagined motor movement and then using overt motor movement. During imagined movement the subject was instructed to focus on articulator movement, rather than silently repeating it to themselves. Using a combination of feature extraction methods combined with a simple linear classifier, they were able to achieve significant pairwise vowel classification accuracy for both imagined and overt speech (with and without vocalization), and significant pairwise consonant classification for overt speech. In addition, in some subjects it was possible to classify pairwise consonant production for imagined speech. For imagined motor movement, the EMG data were analyzed in order to ensure that there was no overt articulator activity. These results demonstrate that it is possible to extract discriminable information associated with speech production for a potential future application in a speech BCI. Adam Koerner is an NSF IGERT (integrative Graduate Education and Research Traineeship) fellow in the Vision and Learning in Humans and Machines Traineeship program at UCSD run by Professors Virginia de Sa and Garrison Cottrell. This is an example of using machine learning to improve human imaging and to improve our knowledge of fine motor movements.

Address Goals

Brain-computer interfaces could greatly improve communication abilities (and thus the quality of life) of individuals with motor disabilities. This also has the potential to greatly increase (broaden) participation in science (and other fields) by increasing the ability for individuals with motor disabilities to participate. This research is the dissertation research of Adam Koerner, an African-American graduate student.