Ecologically valid motion capture of conducting movements with the kinelyze-system

Abstract

In this study, we examined the usability and validity of our software kinelyze, based on the Microsoft Kinect hardware, for ecologically valid, minimal-invasive capturing of musicians‘ movements in real stage situations, using the example of full body conducting movements. The Kinect is a small device, equipped with infrared sensors, that is simply placed on stage without significantly changing the situation and is especially suitable for capturing full body movement patterns. kinelyze transforms the data into a free scalable and rotatable, animated 3D stick-figure. In a pretrial, the basic manageability and reliability of the kinelyze system in a real stage situation were examined. The following main study examined the suitability and validity of the stick-figures generated by kinelyze to reproduce the full body conducting movements. Five conductors were asked to quasi-conduct an audio recording of the exposition of the first movement of Mozart's 'Eine kleine Nachtmusik' (KV 525). The conducting movements were captured and transformed into stick-figures with kinelyze. These stick-figures were presented to 65 subjects acquainted with the five conductors. Subjects were asked to re-allocate the stick-figures, rate the certainty of their judgement, explain their choice in detail and report further details on their individual experiences with the conductors. The results suggest that – despite the lack of hand and finger movements – the stick-figures reproduce the characteristics of full body conducting movements in a form that allows for a clear and reliable recognition of a person. However, this also depends on the particular concision of the stick-figure, the general awareness of a person as a conductor and the individual experiences of the subject. Basically, the findings presented support for a suitability of the kinelyze system and the generated stick-figures as tools for capturing full body movements in corresponding research.