Humans perform physical tasks with abundant multimodal information from visual, auditory, tactile, and more senses. Mainstream computers still offer very limited input and output capabilities when interacting with the users (e.g., multitouch gestures, speech input). This course provides project-based knowledge on human sensory perception & cognition and technologies that accommodate these, including topics on eye-tracking, auditory, robotic touch interfaces. Students will conduct user studies and develop software-hardware systems that explore and leverage multimodal interaction in these projects. Later in the course, students will explore an open-ended research project guided by the instructor. This course will take place in Make Reality Space. No prerequisite needed while programming skills are required.

- Analyze users’ perception and cognition of multimodal stimuli through conducting user studies. - Analyze the principles, benefits, and limitations of multimodal systems. - Exploit multimodal interaction in designing new technical applications. - Explore a topic of interest and design pilot studies that kickstart a potential research.

Students MUST come to the first class in order to get/keep enrolled in this class.