Topologies in neuro-inspired multimodal perception: learning, merging, decision-making

This thesis focuses on multimodal merging on simulated topologies for use in an active perception context. As an example, humans receive dense information from multiple sensors and use various mechanisms to select and attend to only the relevant signals, for example by moving their eyes towards a target to see it better. Because of irregularities in sensory topologies (cf. fovea), actions can enhance perception, while extracting and merging data also helps choosing the best course of action. Similar needs are faced by artificial systems, e.g. social robots, albeit with their own set of physical constraints. This thesis proposes computational models for use in AI, taking inspiration from neuroscience case studies such as the superior colliculus, a subcortical structure involved in generating saccadic eye movements towards visual, auditory or multisensory stimuli.
When selecting information from multiple signals in a dynamic and multimodal setting, one needs a way to compute robust and reliable decisions. Decision-making in general has been tackled in either psychophysics or robotics using many different algorithms. One contribution of this thesis is to review and compare these algorithms, underlining their spatio-temporal properties, including feature merging, selective attention, etc. Among these models, dynamic neural fields (DNF) display some very interesting characteristics, including selective attention and data fusion depending on stimulus distance and precision. In another contribution, this thesis then makes use of DNF as a signal filtering and merging tool applied to multimodal fusion. This thesis shows how it can apply to model realistically occurences of the ventriloquist effect, a psychophysical effect of audio/visual stimulus localization capture. Then, in order to further study the role of topologies on these cognitive tasks, a final contribution shows that DNF retain their properties in irregular learned topological maps. In this experience, topologies are learnt via growing neural gas in order to extract intrinsic dimensions of the sensory space, but new perspectives, with deeper models, are suggested for application in active perception and embodied cognition.

Directors:

• Prof. Salima HASSAS (Univ. Lyon 1 )
• Dr. Mathieu LEFORT (Univ. Lyon 1 )
• Dr. Jean-Charles QUINTON (Univ. Grenoble Alpes )

Raporteurs:

• Prof. Estela BICHO (Univ. of Minho )
• Dr. Denis SHEYNIKHOVICH (Univ. Sorbonne )

Examinators:

• Prof. Stéphanie JEAN-DAUBIAS (Univ. Lyon 1 )
• Dr. Marieke ROHDE (VDI/VDE Innovation+Technology GmbH )
• Prof. Jochen TRIESCH (Univ. Goethe )