Input Technologies and Techniques
Ken Hinckley

1. Introduction
   This chapter gives the practitioner an overview of some important issues relating to input technologies. It is not an exhaustive tour of existing devices as this material is readily available elsewhere, although examples of some common devices are illustrated. The reader should leave the chapter knowing (1) important questions to ask about input technology, (2) practical knowledge of techniques to effectively use input signals in applications, and (3) awareness of models and theories that can be used to evaluate interaction techniques as well as to reason about design options. The emphasis is on continuous input sensors, with a brief section on keyboards/text entry.
2. What's an Input Device? Understanding Input Technologies
   • What is so good about the mouse, anyway?
   • Fundamentals
     • Resolution, Accuracy of transducer
     • Lag, Sampling Rate, Frame rate, monitor refresh rate
     • Problems: Noise, Aliasing, sensor nonlinearities
   • Illustrated tour: Mice, Trackballs, Tablets, Touchpads, Touchscreens, game controllers; http://www.billbuxton.com/InputSources.html
   • Properties and Models of Input Devices
     • Example: Mouse vs. Touchpad vs. Trackpoint vs. Stylus on PDA. All can specify a 2D position. But what are the important differences?
     • Buxton taxonomy and its relevance / application. Device properties, e.g. Absolute vs. Relative, Direct vs. Indirect, isotonic / isometric.
     • 3-state model with continuous device properties; touch and proximity.
3. What's an Input Device For? The Composition of User Tasks
   • Atomic / Elemental tasks
     • Select, position, orient, path, quantify, text entry
   • Compound tasks
     • Chunking and phrasing of tasks
     • Gestural and Marking techniques
     • Multi-Channel Devices; e.g. scrolling.
   • Higher-level primitives: sampling, scanning, and high-dimensional input; e.g. multi-touch tablets, scanners, haptic lens, "shape tape," motion capture, etc.
4. Evaluation & Analysis of Input Devices
   • Of course, need to apply general usability techniques [Chapters 61-65]
   • Canonical evaluation tasks for pointing devices: target acquisition; pursuit tracking; path following; handwriting; drawing on and off primary axes; moving device very rapidly or very slowly; digitization tasks; clicking, double clicking, dragging (do muscle groups interfere with pointing?); number and placement of buttons, accidental button hits.
   • Criteria: Speed, Accuracy (accuracy metrics), ease of learning / power law of practice, fatigue, device persistence and acquisition, does look and feel suggest intended use, etc.
   • Ergonomic Issues: neutral posture. Effects of flexion on the carpal tunnel (Rempel). repetitive strain injury. [Chapter 20, Workstation Ergonomics]
   • Fitts' Law: A design, engineering, and research tool
     • Fitts' Task Paradigm
     • Fitts' Law.
       • derived from information theory, although psychomotor interpretations also possible..
       • MT = a + b ID; ID = log (A/W +1); IP = ID / MT
       • ID is dimensionless, describes moves in task space.
     • What is Fitts' Law good for?
       • Prediction of Movement Time. (MacKenzie 1992)
       • Evaluation, optimization, comparison. Normalizing for accuracy: Using "effective width" to describe both speed and accuracy. Use ratio of IP's to compare studies.
       • Industry practitioners consider a 5% difference in bandwidth to be a big one - notable to users.
       • Example Fitts' Law data set.
     • Application of Fitts' Law to Research and Design Problems
       • ISO standard, Fitts' in 2D / approach angle, bandwidth of joints/limbs, power / precision grasp, tracking vs. dragging states, in presence of lag, area cursors, Multi-scale
     • When not to use Fitts' Law. Other types of models.
       • Keystroke-Level Model. acquisition / release times.
       • Steering Law - extension to trajectory-based tasks.
5. Mappings: How to get the most out of an input signal
   • Control:Display ratio, gain, movement scale, display scale - related but confusing concepts
   • Transfer functions & nonlinear C:D curves. In practice the best "feel" and performance with an input device may not be a simple mapping of the raw data from the transducer. But be careful, it's easy to design a poor function.
     • Gain theoretically does not affect performance by Fitts' Law. But in practice, with "real" devices and physical limitations, it often does matter.
     • Appropriate mappings: position:position, force:velocity
     • For relative (motion) input, e.g. mouse. Sigmoidal response curve used on Apple computers.
     • Absolute position input, e.g. touchscreen. (Hi-res pointing)
     • Force input, e.g. trackpoint: Rate mapping, Rate control in general.
   • Tips & Tricks of the Trade
     • provide real-time response with minimum lag - even when uncertain about what the user is currently doing.
     • preserve as much of the information contained in the human gesture as possible - "never throw data from the user's input away" - yet sometimes also must ignore artifacts of devices.
     • Make the response to the input readily perceptible; avoiding sudden jarring changes that cannot be visually tracked
     • e.g. Calibrating and linearizing device response; Linearizing control to display feedback; Filtering techniques; FIFO queuing; speculative execution; constraints; animated transitions; limiting derivative of change; subpixel calculation.
6. Feedback: What happens in response to an input?
   • Input and Output are tightly coupled. The separation of the two is somewhat artificial and this should always be kept in mind. The graphical feedback on the screen, as well as cues afforded by the device (tactile and kinesthetic feedback, audio associated with clicking) are all part of the same motor-visual feedback loop.
   • Perceptual structure (Jacob, Sibert et al. 1994)
   • Kinesthetic correspondence.
   • Tactile, force, kinesthetic feedback.
     • Haptic Output - Chapter 10. Examples of pointing devices with force/tactile feedback. Interaction between visual / tactile modalities.
     • Passive Feedback: e.g. mode errors (Sellen, Kurtenbach et al. 1992), kinesthetic reference frames, proprioception. .
   • Audio feedback- Chapter 11
7. Keyboards and Text entry techniques
   • Automation of responses - hot keys, function keys
   • key layout, size, shape, activation force, key travel, tactile & auditory feedback
   • Some recent trends in keyboard design
     • adjustable height, slope, ergonomic design. Natural Keyboards.
     • Special-purpose keys, e.g. internet forward/back; Xerox Star had cut/copy/paste on left of keyboard.
     • Integration of pointing devices with keyboard; trackpoint, touchpads.
     • Enhanced visual grouping, labeling of key combinations; elimination of legacy keys, Wireless, USB hub
     • Miniature keyboards for mobile devices.
   • Alternative designs: one-handed qwerty, Chording keyboards
   • Soft keyboards. Design Issues. Very demanding of visual attention.
   • Character entry and recognition: grafitti, natural handwriting recognition
8. Alternative interaction devices. For disabled use, secondary inputs, or tasks where the user's hands may already be occupied.
   • Feet for input. "Mole", foot pedals
   • Head tracking. Popular in VR applications. Low bandwith for cursor control.
   • Eye Tracking: Technology, Limitations, Novel uses
   • Aids for Diverse Users. Chapters 21-27.
   • Speech - Chapter 8, Chapter 37.
9. Research Trends in Input - What's coming in the future?
   • Background Interaction & Sensing. Input includes collecting information about the user and task context. Example: point and shoot cameras. Brief discussion of a few examples from literature. (embodied interfaces, ubiquitous computing)
   • Sensors
     • Biometrics:
       • Identity: Fingerprint scanning, retinal scanning, etc.
       • Emotional state: Galvanic skin response, heart rate, temperature, etc.
     • Unique ID / electronic tags, bar code readers
     • Cameras. Optical position sensing for mice, OCR devices - pens that look up/translate a word. Vision-based / Perceptual UI.
     • Other Sensors - microphones, accelerometers, capacitance / electric field sensing, etc.
   • Multiple users, multiple computers: big and small displays together, interaction on large work surfaces, multi-computer cut & paste. Chapter 30.
   • Bimanual Input: Practical applications, Guiard theory, pragmatic and epistemic actions.
   • Multi-Dimensional Input: Rotation, tilting, 6DOF sensors. Using 2D devices for 3D interaction. (VR and AR- Chapter 32). Coordination Metrics.
   • Application-specific devices (c.f. Chapter 39), Tangible UI
10. Summary and Conclusions