Explanation
Multimodality is the coordinated interaction of different modes of expression such as language, image, sound, typography, gesture, gaze, and space. Meaning is not produced by simply adding independent channels. Modes distribute tasks, reinforce, specify, or contradict each other and jointly form an ensemble. A pointing gesture may determine the referent of a word, layout establish hierarchy, and music frame a scene affectively. Which modes are available and how they can be read depends on medium, situation, bodies, and cultural convention. The concept presents communication as a designed coupling of heterogeneous resources and requires attention to accessibility, power, and possible conflict between modes.
Movement of thought
1. A communicative situation first makes several material and semiotic resources available at the same time. 2. Design distributes information, stance, and attention across language, image, sound, body, and spatial order. 3. The modes relate to one another, complementing, weighting, or contradicting each other to produce an overall effect. 4. A counter-test removes or alters one mode and examines which meaning, accessibility, or effect is lost. 5. The utterance returns as a coordinated ensemble whose meaning emerges between its expressive resources.
Scientific use case
Multimodal discourse and interaction analysis can annotate temporal coordination, functional division of labour, and modal contradiction. Reception and accessibility studies can test which information survives restricted access to a mode.
Example
In a lecture, the speaker says “this transition”, points to a coloured curve, and slows her voice. Word, gesture, graphic, and prosody jointly determine the reference.
Counterexample
A page happens to contain text and an unrelated decorative image. Mere co-presence of several media does not yet establish analytically relevant multimodal meaning-making.
Distinction
Distinguish multimodality from multimedia as technical combination, intermediality, mere illustration, and synaesthesia. It examines functional coupling of semiotic resources within a situation.
Effect on thought and perception
Multimodality can make complexity vivid, redundant, and affectively accessible; poorly coordinated modes overload attention or exclude particular user groups.
Resonance
Word, image, sound, and body do not send isolated lines; they meet and form a common current of meaning in which each mode changes the others.
Etymology
Technical core term; individual etymological review pending
Provenance
Unresolved relation network → core-concept review → Metzler/HWRh cross-check → ATLAS 238
Cite this entry
Reich, Michael (Hrsg./ed.): „Multimodality“. Atlas der Sprache, AT-CORE-238-049. ATLAS 260, data model 1.6.0. https://atlasoflanguage.com/concept/AT-CORE-238-049The address remains stable; editorial content may evolve. Editorial reading state: 2026-10-15. Include your access date for precise references.
Historical background
Darwin’s treatment of expression across voice and bodily movement is a useful earlier comparison for multimodality, but it does not define a modern theory of modes. A sensory channel, a conventional representational resource and a bodily action need not be the same analytical unit. The Atlas should therefore separate this profile’s general question about kinds of meaning-making resources from the profile concerned with their coordination in a particular communicative event. Darwin helps challenge an exclusively verbal account of expression; he cannot settle how writing, images, diagrams or digital interfaces should be classified. That extension requires later sources and explicit definitions. The historical witness belongs to a neighbouring explanatory practice, with the English theoretical label and its competing scopes still unresolved.
How conceptual history changes the analysis
Several channels alone do not establish functionally described multimodal meaning-making.
References for the historical section
- Charles Darwin, The Expression of the Emotions in Man and Animals, New York: D. Appleton, 1899; Gutenberg transcription · Chapters I–II: multiple means of expression
Selected historical witness, not a complete genealogy or a first English attestation. Atlas comparisons and applications are original editorial interpretations. The inspected transcription identifies the 1899 Appleton edition, not a collated 1872 first edition. Darwin’s causal speculations, clinical descriptions and racial generalisations are not adopted as current findings.