MusiQAl: A dataset for Music Question-Answering through Audio-Video Fusion

Music Question--Answering (MQA) is a machine learning task where a computational system analyzes and answers questions about music-related data. Traditional methods prioritize audio, overlooking visual and embodied aspects crucial to music performance understanding. We introduce MusiQAl, a multimodal dataset of 310 music performance videos and 11,793 human-annotated question--answer pairs, spanning diverse musical traditions and styles. Grounded in musicology and music psychology, MusiQAl emphasizes multimodal reasoning, causal inference, and cross-cultural understanding of performer-music interactions. We benchmark AVST and LAVISH on MusiQAl, revealing strengths and limitations, underscoring the importance of integrating multimodal learning and domain expertise to advance MQA and Music Information Retrieval (MIR).