A Game Theoretic Approach to Multimodal Verb Sense Disambiguation

Abstract

Verb Sense Disambiguation is a well known task in NLP, the aim is to find the correct sense of a verb in a sentence. Recently, this problem has been extended in multimodal scenario, by exploiting both textual and visual features of ambiguous verbs. The sense of a verb is then assigned by the actual content of the image paired with it. In this thesis such task will be performed in a game theoretic scenario in which each multimodal representation of a pair image-verb is a player and the possible strategies correspond to the set of senses that the verb belong to. A Nash Equilibria in this non cooperative game corresponds to a consistent labeling between verbs and their possible senses. Experiments yield to promising results and they have been carried out on the recently published dataset VerSe.