Multimodal Model
also: VLM · vision-language model
One model over several modalities — text, images, audio, video — in a shared token space.
A multimodal model encodes non-text inputs (image patches, audio frames) into the same token stream a language model reads, so one network can describe a photo, read a chart, or transcribe speech. Vision-language models (VLMs) power document AI and computer-use agents; the frontier is any-to-any — consuming and producing every modality natively.
Worked example: a model that takes more than one input type — text plus images, audio, or video — in a shared representation, so it can answer questions about a picture or reason over audio. Gotcha: ‘multimodal’ usually means other modalities are projected into the same embedding space as text; capability varies sharply by modality (strong on images, weaker on fine detail like reading small text or counting), so test the specific modality task rather than assuming uniform skill.