Multimodal Interfaces
Sharon Oviatt

1. Introduction to multimodal interaction and systems
   • History and current state of the art
   • Overview of mature sub-areas of multimodal research
   • Basic multimodal interface concepts and terminology
2. Goals and advantages of multimodal interface design
3. Cognitive science underpinnings of multimodal interface design
   • Multimodal versus unimodal communication and language
   • Integration and synchronization of input modes
   • Interpersonal versus human-computer multimodal interaction
   • Individual differences in multimodal interaction
   • Implications for multimodal system design
4. Methodological approaches to multimodal system design and development
   • High-fidelity simulations
   • Testing with fully functional systems
5. Multimodal language and dialogue processsing techniques
6. Architectural approaches to multimodal processing
   • Architectural requirements of multimodal system processing
   • Multi-agent architectures and multimodal processing flow
   • Approaches to multimodal information fusion
   • Hybrid symbolic/statistical architectures
7. Influence of multimedia output on multimodal input
   • Impact of visual display characteristics on multimodal input
   • Impact of speech and non-speech audio on multimodal input
   • Implications for the overall design of robust multimodal input-output systems
8. Emerging multimodal application areas
9. Current limitations of multimodal systems
10. Future Directions
    • New multimodal interface design concepts
    • Adaptive processing for successful field and mobile use
    • Multimodal language, dialogue processing, and error handling techniques
    • New hybrid symbolic/statistical architectures for multimodal fusion
    • Multimodal interfaces that support diverse users and universal access
    • Tools for corpus collection, prototyping and iterating new multimodal systems
11. Conclusion
12. References