Published Articles >> Table of Contents >> Abstract

Fourth IEEE International Conference on Multimodal Interfaces (ICMI'02)   p. 87
Context-Based Multimodal Input Understanding in Conversational Systems
Joyce Chai, IBM T.J. Watson Research Center
Shimei Pan, IBM T.J. Watson Research Center
Michelle X. Zhou, IBM T.J. Watson Research Center
Keith Houck, IBM T.J. Watson Research Center

Full Article Text:

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/ICMI.2002.1166974
Send link to a friend

Abstract
In a multimodal human-machine conversation, user inputs are often abbreviated or imprecise. Sometimes, only fusing multimodal inputs together cannot derive a complete understanding. To address these inadequacies, we are building a semantics-based multimodal interpretation framework called MIND (Multimodal Interpretation for Natural Dialog). The unique feature of MIND is the use of a variety of contexts (e.g., domain context and conversation context) to enhance multimodal fusion. In this paper, we present a semantic rich modeling scheme and a context-based approach that enable MIND to gain a full understanding of user inputs, including those ambiguous and incomplete ones.
Back to Top Additional Information
Citation:  Joyce Chai, Shimei Pan, Michelle X. Zhou, Keith Houck, "Context-Based Multimodal Input Understanding in Conversational Systems," icmi, p. 87,  Fourth IEEE International Conference on Multimodal Interfaces (ICMI'02),  2002