On Bimodal Guitar-Chord Recognition
Abstract: We discuss the use of visual information to aid the task of recognizing guitar-chords in real time. The video-based method is analogous to the state-of-the-art audio-based counterpart, relying on a supervised Machine Learning algorithm applied to a visual chord descriptor. The visual descriptor consists of the rough position of the fingers in the guitar fretboard. Experiments were conducted regarding classification accuracy comparisons between methods using audio, video and the combination of the two signals. Four different data-fusion techniques were evaluated: feature fusion, sum rule, product rule and an approach in which the visual information is used as prior distribution, which resembles the way humans recognize chords being played by a guitarist. Results favor the use of visual information to improve the accuracy of audio-based methods, as well as for being applied without audio-signal help.