Robot motion with physical interactions and social adaptation: imagining the robot in motion, towards sensitive and motor-based collaboration

Philippe Souères, CNRS research director at LAAS-CNRS, presents structuring research action (AS) 2, ‘Robot motion with physical interactions and social adaptation’, from the Organic Robotics (O2R) research programme. He highlights the collaborative design of the robots developed and their multisensory capabilities.

What technical aspects need to be mastered and explored in order to develop robots’ multisensory and sensorimotor integration capabilities?

Philippe Souères: To enable a robot to move and interact with its environment, it must be equipped with appropriate sensory capabilities. On the one hand, it must be able to assess its internal state (joint position and velocity, configuration of its mobile base) in order to adjust its motor control. On the other hand, it must be able to perceive the external world in order to orient itself within it and direct its movements. This requires different types of sensors providing complementary signals. This information must be integrated and refined (multisensory integration) to enable the evaluation of relevant parameters. Furthermore, variations in this data must be linked to movement control (sensorimotor integration). To achieve this, an approach that has long prevailed involves combining probabilistic techniques of multi-sensor fusion (filtering) and automatic control (observation) to reconstruct a representation of the robot’s state. Recent machine learning techniques are opening up a new avenue by enabling the direct integration of sensor data as input to neural networks for the calculation of high-performance control policies.

Could you give us a specific example of a robot you are working on, and explain which sense and/or motor skill it is designed to develop?

P.S.: Two projects launched during the first phase of AS2 can be cited where issues of multisensory and sensorimotor integration are crucial. The first concerns the manipulation of soft objects using a dual-arm robot, led by the University of Montpellier and IRISA (CNRS/University of Rennes). The aim is to control the movement of the robot’s two grippers, each holding one end of the object, so that it takes on the desired shape. This relies on the integration of data from the robot’s proprioception (arm joints) as well as from vision, which detects deformation, and potentially force feedback. The second example concerns the locomotion control of a quadruped robot led by LAAS-CNRS and the Inria Centre in Paris. In this case, the aim is to coordinate the forces exerted on the 12 joints of the robot’s legs to adapt its behaviour when negotiating obstacles. Here again, this requires the simultaneous processing of heterogeneous multisensory information such as joint position and velocity measurements, inertial measurements and vision data provided by an on-board camera.

You are studying movement to contribute to the collaborative design of robots. What do you mean by ‘collaboration with robots’? What insights do disciplines outside robotics – such as cognitive psychology, human-robot interaction psychology, social psychology, anthropology and sociology – offer you?

P.S.: When we talk about collaborative robot design, or ‘co-design’, we are referring to an approach that aims to take into account, right from the machine’s design phase, various aspects relating to its use and operation. By studying the movement of robots in physical interaction from an interdisciplinary perspective, involving robotics engineers and researchers in the humanities and social sciences, our aim is to identify a set of factors that can guide robot design to make them both effective in performing tasks and socially appropriate. This may concern their morphology, the components and materials from which they are made, their actuation and perception systems, as well as their control mechanisms, in order to better meet users’ needs. The various fields of expertise of the researchers in the humanities and social sciences cited are, in this sense, invaluable for assessing, from different angles, how to adapt these robot characteristics whilst taking the human into account.

What types of robots and tasks, and for what purposes, do you intend to use in the motion design?

P.S.: The work we are conducting raises, at a more fundamental level, a range of questions relating to movement for different types of robots and tasks. At this stage, we have not considered any specific types of users or uses for motion control. From a resolutely interdisciplinary perspective, we are interested in issues such as agency and system transparency in human-robot interactions (Inria-Bordeaux and Onera), co-representation and the prediction of robotic agent actions in industrial settings (PPRIME and CERCA), or, more broadly, the issue of coexisting with robots (University of Picardie), seeking to understand how movement influences interactions between robotic entities and humans, and how interactions influence movement. The use cases behind these studies are numerous. They span a broad spectrum ranging from service robotics to industrial robotics via collaborative robotics, in which these systems are required to move within environments shared with humans and interact physically with them.