Dipartimento di Ingegneria informatica, automatica e gestionale

The application of machine learning to recognize hand movements from surface electromyography has led to promising academic results. Unfortunately, it has proven difficult to translate these results in better control methods for the end-users of upper-limb prostheses. Recent studies have pointed out that common offline performance metrics, such as classification accuracy, are not correlated with real controllability of the prosthesis. In this paper, we investigate the cause that learned models start to fail when applied outside the constrained laboratory setting. We performed several analyses at the hand of a dedicated data acquisition composed of a typical academic training session in the first phase and a set of activities of daily living in a home setting afterwards. Our analysis confirms that a model trained in the former setting performs poorly when applied in a home environment. The cause for this degradation is that the distribution of myoelectric data changes between both settings, thus violating the typical assumption in statistical learning theory that train and test data come from the same distribution. This problem persists even when adding data acquired in some home activities to classify others. Our result not only confirms the limited importance of offline performance metrics for real prosthesis usability, but also highlights the difficulties machine learning based approaches will need to overcome to become practically relevant.