Multimodal Systems
Systems that combine multiple accessibility technologies to provide comprehensive access to information.
Definition
Multimodal systems pair different accessibility technologies together, creating unique combinations that serve specific needs or use cases.
Examples
Screen-Reader + Braille
- Languages: English, French, German, Spanish, and many others
- Description: Combines synthesized speech output with refreshable Braille display
- Use Cases:
- Learning Braille while hearing the text
- Verifying pronunciation
- Multitasking (listening while reading)
- Examples:
- JAWS + Focus 40 Braille display (English)
- NVDA + Orbit Reader (multiple languages)
- VoiceOver + Braille display (Apple ecosystem)
Pro-Tactile + Speech-to-Text (STT)
- Languages: American Sign Language, British Sign Language, and other sign languages
- Description: Combines Pro-Tactile communication with speech-to-text transcription
- Use Cases:
- DeafBlind individuals accessing spoken content
- Real-time communication support
- Status: Emerging technology
Audio Description + Tactile Graphics
- Languages: Multiple
- Description: Combines spoken descriptions with tactile representations
- Use Cases:
- Understanding complex visual information
- Educational content
- Museum exhibits
- Examples:
- Museum tours with audio guides and tactile models
- Educational materials with both audio and tactile components
Braille + Large Print + Audio
- Languages: Multiple
- Description: Triple-modality approach for maximum accessibility
- Use Cases:
- Educational settings
- Public information displays
- Comprehensive accessibility compliance
Screen Reader + Magnification + Braille
- Languages: Multiple
- Description: Combines visual, auditory, and tactile access
- Use Cases:
- Low vision users who also use screen readers
- Transitioning between modalities
- Comprehensive access solutions
Talking-Touch Devices
Portable devices that integrate multiple modalities:
BrailleSense
- Modalities: Braille display + Speech + Touch screen
- Languages: Multiple
- Description: Portable computing with combined Braille display and speech
BrailleNote Touch
- Modalities: Braille display + Speech + Touch screen
- Languages: Multiple
- Description: Integrated TTS + Braille refreshable display
Counting Methodology
1 per unique pairing
Examples:
- “Screen-reader + Braille (English)” = 1 entry
- “Pro-Tactile + STT (American Sign Language)” = 1 entry
- “Audio description + Tactile graphics (French)” = 1 entry
Notes
- Multimodal systems provide redundancy and flexibility
- Different combinations serve different needs
- Some users prefer multiple modalities simultaneously
- Technology integration continues to evolve
- Standards for multimodal systems are still developing
Future Directions
- AI-powered multimodal systems
- More seamless integration between modalities
- Customizable modality combinations
- Cloud-based multimodal services