The role of multimedia in human-computer interaction
Alistair Sutcliffe

1. Background
   • Definitions
     • physical formats- limited coverage of MPEG, .avi. mp3 etc.
     • media as perceived by users
     • media and modalities
     • creation or capture of a medium
     • taxonomies of media (Bernsen, Heller, ISO 14915)
   • Applications
     • Education and training
     • Information delivery
     • Decision support
     • Task support and process control
   • Design issues and dilemmas
     • Rich representation but confusing complexity
     • Information overloading or integrated representation
     • Large information space and conceptual disorientation
     • Exciting or distracting presentations
     • Matching the message to the media
     • Making the theme stand out from the noise
   • Design process
     • Multidisciplinary teams- within media design- video, graphics, etc
     • Creative design and/or. systematic engineering
     • Rapid prototyping
2. Cognitive Implications
   • Input channels and Multimedia information processing
   • Processing visual information- reading and viewing
     • Eye tracking studies
   • Processing audio information- hearing, speech and language
   • Information from other channels -haptics, olfaction, gustation
   • Media integration at perceptual and cognitive levels
   • Use of Human Information Processing Models to reason about resource limitations, priming effects
   • Selective attention- dynamic and static media
   • Arousal and motivation- interaction of media and content
   • Principles for multimedia design (ISO 14915 part 1)
3. Selecting Media for the Message
   • User requirements and communication goals
   • Information and data models
   • Domain and context models- MM application environment
   • Media selection heuristics and guidelines
     • Mappings from information types to media types (ISO 14915)
     • Heuristics for educational multimedia (Boyle, Heller, Kozuma)
     • Interactive multimedia microworlds and simulations
     • Use of media in warnings and safety critical applications
   • Selecting content for motivation
     • Media types and content issues to attract attention
     • Choice or design of media for aesthetic effects
     • Use of human image and personality in multimedia (Reeve and Nass)
   • Layout and sequencing- synchronous v. asynchronous presentations
   • Design constraints- networked multimedia, context and user considerations (accessibility)
   • Storyboards, scripting and prototypes
4. Navigation and Interaction
   • Planning user access paths
   • Navigation link structure
   • Navigation facilities- maps, visit lists, guided tours, views, filters.
   • Navigation controls for different media types- dynamic v static media
   • Interaction with microworlds
   • Interaction with multimedia agents-speech and gesture.
5. Attention and Integration
   • Attention directing design effects
   • Thematic integration- role of contact points
   • Making items and links between items salient
   • Integration by interaction or by thematically directing users' attention.
6. Evaluation
   • Effectiveness of multimedia in information delivery memorisation, and recall
   • Evaluation approaches, integrated with standard usability evaluation
   • Memory recall, learning and performance testing
   • Detailed diagnosis- protocols and eye tracking studies.