Motor adaptation during a sound oriented task (NCM 2017, poster publication)

This poster will be presented in Dublin during the 2017 annual meeting of the Society for the Neural Control of Movement (NCM 2017).

Motor adaptation during a sound oriented task

Eric O. Boyer1,2, Frederic Bevilacqua2, Sylvain Hanneton3, Agnes Roby-Brami1,
1 ISIR – CNRS UMR 7222, UPMC, 2: IRCAM – STMS-CNRS, UPMC, 3: LPP – CNRS UMR 8242, University Paris Descartes, Paris, France

Introduction. Movement sonification systems appear promising for sensori-motor learning in providing users with auditory feedback of their own movements [1]. However, research on sonification for sensori-motor learning has been mainly directed toward “movement oriented tasks” where the instruction and the attention is put on the movement itself. In contrast, the aim of the present study was to test a situation were the instruction and attention is given to the sound, that we call a “sound-oriented task”.
The sonification mapping relied on the metaphor of friction sounds produced by drawing movements. We focused on the drawing of ellipses which is characterized by a well known invariant velocity-curvature relationship [2,3]. The replay of friction sounds (registered or synthesized) can evoke the shape of the drawings [4] and induce a sensori-motor perceptive bias on the reproduction of visual motion [5] .
Elaborating further on these bases, we tested the effect of on-line sonification with friction-like sounds on the kinematics and the shape of elliptical drawing movement. The mapping was implemented as a band pass filter whose center frequency varied linearly with the velocity of the movement [3,4]. We analyzed the motor adaptation of the drawing movements when the subject had the instruction to maintain a constant sonification pattern while the frequency of the filter changed without his/her knowledge.
Our hypothesis was that the alteration of the sonification mapping would induce temporal and/or spatial adaptation of the movement.

(see the poster for methods and results)

Discussion. The motor adaptation tended to compensate for the changes in sound feedback induced by the changes in the sound-movement mapping. This demonstrate that the participant could adapt their movement to the “sound oriented” task. The adaptation was manifested by modifications of the kinematics. There was a tendency for increase of frequency and decreased size of the drawing in the control situation. In addition, the movement was faster with larger movements when the gain of the mapping was increased and slower with smaller movements when it decreased. The global shape and orientation of the ellipse was not modified in 2D.
This demonstrates that the participant privileged the stability of the geometrical shape and adapted their velocity in order to satisfy the instruction to keep the sonification pattern constant. The increase in velocity was more due to a change in frequency while the decrease was more due to a shrinking of the shape, suggesting different movement regimen [e.g. alternative versus discrete, 7]. The modification of the angle of the ellipse during the experiment in 3D but not in 2D was probably due to greater inertial constraints as shown by Pfann et al [8]. Participants who were instructed to draw circles with shoulder-elbow movements made ellipses with increasing eccentricity when the velocity increased (order of magnitude: 1m/s). In addition, the elongation of the ellipse was in the direction of least inertia. A similar effect was also observed for handwriting-like movements (similar to our 2D task) by Dounskaia et al. [9] but for a much higher velocity regimen (instruction level “as fast as possible”, 0.34m/s), when the velocity we used corresponds to their “self paced” level).

Conclusion. This study demonstrates that movement sonification can be used i) to induce implicit motor adaptation in both planar and 3D movements and ii) to control the direction and magnitude of this adaptation through mapping parameters modification.

 

Nouvelle publication : Investigating three types of continuous auditory feedback in visuo‑manual tracking

Investigating three types of continuous auditory feedback  in visuo‑manual tracking

Éric O. Boyer, Frédéric Bevilacqua, Patrick Susini et Sylvain Hanneton
Exp Brain Res – DOI 10.1007/s00221-016-4827-x
(electronic publication, paper soon)

Abstract

The use of continuous auditory feedback for motor control and learning is still understudied and deserves more attention regarding fundamental mechanisms and applications. This paper presents the results of three experiments studying the contribution of task-error, and user-related sonification to visuo-manual tracking and assessing its benefits on sensorimotor learning. First results show that sonification can help decreasing the tracking error, as well as increasing the energy in participant’s movement. In the second experiment, when alternating feedback presence, the user-related sonification did not show feedback dependency effects, contrary to the error and task-related feedback. In the third experiment, a reduced exposure of 50% diminished the positive effect of sonification on performance, whereas the increase of the average energy with sound was still significant. In a retention test performed on the next day without auditory feedback, movement energy was still superior for the groups previously trained with the feedback. Although performance was not affected by sound, a learning effect was measurable in both sessions and the user-related group improved its performance also in the retention test. These results confirm that a continuous auditory feedback can be beneficial for movement training and also show an interesting effect of sonification on movement energy. User-related sonification can prevent feedback dependency and increase retention. Consequently, sonification of the user’s own motion appears as a promising solution to support movement learning with interactive feedback.

Keywords: Tracking · Auditory feedback · Sensorimotor learning · Sound · Interaction

Nouvelle publication : Sensori-Motor Learning with Movement Sonification: Perspectives from Recent Interdisciplinary Studies

 Sensori-Motor Learning with Movement Sonification: Perspectives from Recent Interdisciplinary Studies

Access to the full text HERE

This article reports on an interdisciplinary research project on movement sonification for sensori-motor learning. First, we describe different research fields which have contributed to movement sonification, from music technology including gesture-controlled sound synthesis, sonic interaction design, to research on sensori-motor learning with auditory-feedback. In particular, we propose to distinguish between sound-oriented tasks and movement-oriented tasks in experiments involving interactive sound feedback. We describe several research questions and recently published results on movement control, learning and perception. In particular, we studied the effect of the auditory feedback on movements considering several cases: from experiments on pointing and visuo-motor tracking to more complex tasks where interactive sound feedback can guide movements, or cases of sensory substitution where the auditory feedback can inform on object shapes. We also developed specific methodologies and technologies for designing the sonic feedback and movement sonification. We conclude with a discussion on key future research challenges in sensori-motor learning with movement sonification. We also point out toward promising applications such as rehabilitation, sport training or product design.

Keywords: sonification, movement, learning, sensori-motor, sound design, interactive systems

Citation: Bevilacqua F, Boyer EO, Françoise J, Houix O, Susini P, Roby-Brami A and Hanneton S (2016) Sensori-Motor Learning with Movement Sonification: Perspectives from Recent Interdisciplinary Studies. Front. Neurosci. 10:385. doi: 10.3389/fnins.2016.00385

Focus sur la notion de proprioception

Le mot « proprioception » définit un 6ème sens assez méconnu. Pour tout arranger, il existe des confusions et des débats sur sa définition. Cet article est l’occasion de faire un point le plus rigoureux possible.

Etymologie et origine du terme

La proprioception (formé de proprio-, tiré du latin proprius, « propre », et de [ré]ception) ou sensibilité profonde désigne la perception, consciente ou non, de la position des différentes parties du corps. Elle fonctionne grâce à de nombreux récepteurs musculaires et ligamentaires, et aux voies et centres nerveux impliqués (définition wikipédia)

La proprioception s’oppose à l’ « extéroception » car les signaux nerveux sont générés par des événement produits à l’intérieur du corps, contrairement aux signaux extéroceptifs qui ont pour origine des modifications de l’environnement (toucher, vision, audition).

Les récepteurs sensoriels impliqués dans la proprioception

Il s’agit de récepteurs placés dans les os, les muscles (fuseaux neuromusculaires, organes tendineux de Golgi), dans les viscères et dans les articulations (capsules articulaires). Les fibres nerveuses issues de ces récepteurs font synapse avec la moelle épinière et les signaux remontent ainsi vers les centres supérieurs (cortex somatosensoriel primaire, cervelet par exemple). Certains incluent dans la proprioception le sens de l’équilibre, c’est à dire les signaux issus du système vestibulaire (dit vulgairement « oreille interne »). Mais cela peut être discuté car cet organe réagit à l’orientation de la tête par rapport à la gravité terrestre qui est un signal externe. Le système vestibulaire est-il un organe proprioceptif ou extéroceptif ? En tout cas il donne des informations sur les mouvements de la tête dans l’espace.

Confusion avec d’autres termes voisins

Il faut faire attention car d’autres termes définissent des aspects de la perception des mouvements et des déformations du corps. Il en résulte des débats sans fin sur la définition de ces termes, débats qui n’ont pas beaucoup d’intérêt. Essayons quand même d’y voir plus clair.

Le terme le plus proche est celui de kinesthésie (du grec kinesis signifiant ‘mouvement’ et aisthesis : ‘sensibilité’). La kinesthésie est une perception consciente de la position et des mouvements des différentes parties du corps. Certains différencient les sens kinesthésiques de la proprioception donnant à celle-ci un sens plus général et à la kinesthésie un sens plus spécifique, en excluant par exemple le sens de l’équilibre de la kinesthésie. Ainsi l’oreille interne ferait partie des organes proprioceptifs mais pas de la kinesthésie.

Mais attentions, certains distinguent proprioception et intéroception. L’ « intéroception » ne concernerait que les signaux issus des capteurs viscéraux.

Il existe aussi un autre terme qui interfère avec la notion de proprioception : le sens « haptique ». L’haptique, du grec ἅπτομαι (haptomai) qui signifie « je touche », désigne la science du toucher, par analogie avec l’acoustique ou l’optique. Ce terme a été introduit en psychologie par Revesz (1934, 1950). Au sens strict, l’haptique englobe le toucher et les phénomènes kinesthésiques, c’est-à-dire la perception du corps dans l’environnement. En effet, pour saisir la forme d’un objet, son volume ou même son poids, il faut le toucher et le manipuler. La manipulation de l’objet va stimuler à la fois les récepteurs tactiles de la peau et les récepteurs proprioceptifs présents dans les muscles, les tendons et les articulations.

On trouve en effet des définitions du mot kinesthésie qui interfèrent fortement avec la définition du sens haptique. Certaines viennent de Henri Piéron même !

Kinesthésie : Sens du mouvement; forme de sensibilité qui, indépendamment de la vue et du toucher, renseigne d’une manière spécifique sur la position et les déplacements des différentes parties du corps. « La kinesthésie exploratrice avec les modifications synchrones des impressions cutanées, renseignant sur la forme, l’état des surfaces, le volume, le poids, etc. (…) [intervient] pour permettre l’identification » (Piéron, La Sensation,1945, p. 41)

Dans la citation de Piéron ci-dessus, on a l’impression qu’il parle plutôt du sens haptique.

Déafférentiation (interruption du fonctionnement des circuits proprioceptifs)

Différentes affections des nerfs, de la moelle et de l’encéphale peuvent atteindre la proprioception : traumatisme, compression par une tumeur, inflammation, accident vasculaire, trouble métabolique (carence en vitamine B12). Une atteinte de la proprioception entraîne une altération des sensibilités profondes élémentaires : le patient ne peut pas, les yeux fermés, reconnaître la position de ses différents segments de membre. Elle se traduit également par une ataxie (absence de coordination des mouvements), avec une instabilité en position debout, accentuée lorsque les yeux sont clos (signe de Romberg). La marche est également perturbée. L’anesthésie osseuse se traduit, à l’examen clinique, par l’absence de perception de la vibration provoquée par un diapason appliqué sur les os superficiels.

Références

  • P.M. Gagey, B. Weber (2004) Posturologie ; Régulation et dérèglements de la station debout. Troisième édition, préface du professeur Henrique Martins da Cunha, Elsevier Masson, Paris

  • J. Paillard (1976) Tonus, posture et mouvements. In Kayser C. (Ed) Physiologie, tome II, Flammarion (Paris): 521-728.

  • H. Piéron. La Sensation, guide de vie, Paris, Gallimard, 1945.

  • G. Revesz, System der optischen und haptischen Raumtäuschungen, Zeitschrift für Physiologie, 131, 296-375, 1934

  • G. Revesz, Psychology and Art of the Blind, London, Longmans Green, 1950

  • C. S. Sherrington (1906) The integrative action of the nervous system. New Haven, Yale University Press

  • Charles S. Sherrington (1900) the musclar sense. In Edward A. shafer Ed. Textbook of physiology (Edimbourg-London, 1900) t. II, 1006.

Nouvelle publication : From ear to hand: the role of the auditory-motor loop in pointing to an auditory source

Boyer EO, Babayan BM, Bevilacqua F, Noisternig M, Warusfel O, Roby-Brami A, Hanneton S and Viaud-Delmon I (2013) From ear to hand: the role of the auditory-motor loop in pointing to an auditory source. Front. Comput. Neurosci. 7:26. doi: 10.3389/fncom.2013.00026

PDF : http://www.frontiersin.org/Journal/DownloadFile.ashx?pdf=1&FileId=114799&articleId=43100&Version=1&ContentTypeId=21&FileName=fncom-07-00026.pdf

Lien vers l’article en accès libre : http://www.frontiersin.org/Journal/Abstract.aspx?ART_DOI=10.3389/fncom.2013.00026&name=computational_neuroscience

Studies of the nature of the neural mechanisms involved in goal-directed movements tend to concentrate on the role of vision. We present here an attempt to address the mechanisms whereby an auditory input is transformed into a motor command. The spatial and temporal organization of hand movements were studied in normal human subjects as they pointed toward unseen auditory targets located in a horizontal plane in front of them. Positions and movements of the hand were measured by a six infrared camera tracking system. In one condition, we assessed the role of auditory information about target position in correcting the trajectory of the hand. To accomplish this, the duration of the target presentation was varied. In another condition, subjects received continuous auditory feedback of their hand movement while pointing to the auditory targets. Online auditory control of the direction of pointing movements was assessed by evaluating how subjects reacted to shifts in heard hand position. Localization errors were exacerbated by short duration of target presentation but not modified by auditory feedback of hand position. Long duration of target presentation gave rise to a higher level of accuracy and was accompanied by early automatic head orienting movements consistently related to target direction. These results highlight the efficiency of auditory feedback processing in online motor control and suggest that the auditory system takes advantages of dynamic changes of the acoustic cues due to changes in head orientation in order to process online motor control. How to design an informative acoustic feedback needs to be carefully studied to demonstrate that auditory feedback of the hand could assist the monitoring of movements directed at objects in auditory space.

Recordtablet : un utilitaire pour capturer les mouvements effectués sur une tablette graphique

L’utilitaire recordtablet (version linux seulement) :

Cet outil permet de visualiser ou d’enregistrer dans un fichier les données normalement envoyées au serveur graphique (X) dans un fichier. Il peut servir à évaluer la précision de la tablette ou à enregistrer des mouvements (d’écriture par exemple)  pour les analyser.

Lien : http://sourceforge.net/projects/recordtablet

Nouvelle publication : Perceptual weight judgments when viewing one’s own and others’ movements under minimalist conditions of visual presentation

Publication d’une étude absolument passionnante sur la perception du mouvement des autres.

Malika Auvray, Thomas Hoellinger, Sylvain Hanneton, Agnes Roby-Brami

Auvray M, Hoellinger T, Hanneton S, Roby-Brami A, 2011, « Perceptual weight judgments when viewing one’s own and others’ movements under minimalist conditions of visual presentation » Perception 40(9) 1081 – 1103

Abstract. Across five experiments, we investigated the parameters involved in the observation and in the execution of the action of lifting an object. The observers were shown minimal information on movements, consisting of either the working-point displacement only (ie two points representing the hand and object) or additional configural information on the kinematics of the trunk, shoulder, arm, forearm, and hand, joined by a stick diagram. Furthermore, displays showed either a participant’s own movements or those of another person, when different weights were lifted. The participants’ task was to estimate the weight of the lifted objects. The results revealed that, although overall performance was not dependent on the visual conditions (working point versus stick diagram) or ownership conditions (self versus other), the kinematic cues used to perform the task differed as a function of these conditions. In addition, the kinematic parameters relevant for action observation did not match those relevant for action execution. This was confirmed in experiments by using artificially altered movement samples, where the variations in critical kinematic variables were manipulated separately or in combination. We discuss the implications of these results for the roles of motor simulation and visual analysis in action observation

Lien vers l’article : http://www.perceptionweb.com/abstract.cgi?id=p6879

Modélisation et simulation d’un neurone de type « integrate and fire »

Simulation d’un neurone « Integrate and Fire »