_________________________________

Daniel Teil

INTRODUCTION

For the Non Verbal Communication Group, 1994 has been a transition period concerning the group's structure and research topics. At the end of five years of existence, we feel the necessity to give shape to several months deep thinking about research orientation, following the evolution of our activities with respect to those of the Human-Machine Communication Department.

When the group was created (at the end of 1989), research activities were divided into 6 topics corresponding either to specialities of researchers like Vision (issued from Robotics and from the Computer Vision team), and 3D Modelling, or to the beginning of new activities such as Gestural Communication, Multimodal Communication and Human Factors.

The apparent diversity and richness of research topics has led to define and to set up an experimental project allowing for the validation of various interface models studied in the group. This project has become the SAMMOVAR project which integrates practically all the researchers abilities in the group.

This project makes research activities to evolve, particularly concerning collaborations with the other groups of the Department (topics Multimodal Communication, Gestural Communication and Human factors).

These different points led us to propose a new structure of Research based on a regrouping of related activities. This new structure divides the research domain in two axis themselves subdivided in topics. The first axis regroups all what concerns Image and the second everything concerning new communication modalities and interface design :

Axis 1 : Virtual representations of space (responsible A. Osorio)

topic 1 - Computer Vision

topic 2 - Computer Aided Geometric Design (CAGD), Computer Graphics

topic 3 - Image Analysis / Synthesis Co-operation

Axis 2 : New Interfaces (responsible D. Teil)

topic 4 - Gesture Analysis and Generation

topic 5 - Multimodal Communication

topic 6 - Auditory Scene Analysis

topic 7 - Human Factors

RESEARCH AXIS :

TOPIC 1 : Computer Vision (R. Gherbi)

A. Osorio, J. Rivaillier, D. Roussel

Computer vision is studied according to two aspects. The first one is to consider vision like a Human-machine communication mode allowing the user to interact with the machine. The second one takes into account the fact that vision can be a tool for the same user to perform a particular visual task.

In the first aspect, vision study lies on using various sensors to grab information about the workstation environment. In this context, we focus our work on two problems: on one hand, we study the capture of operator gestures (using cameras and DataGlove) in order to analyse and recognise these gestural interactions (TOPIC 4) and to capture any visual information to help and complete multimodal systems (TOPIC 5).

Goals in the second aspect are to allow the user to capture real objects in order to construct corresponding 3d models. This study, described in TOPIC 3, inscribes itself more generally into an image analysis-synthesis synergy approach. Other examples of this axis are the recognition of characters and fingerprints.

TOPIC 2: CAGD, Computer Graphics (P. Macé)

G.P. Bonneau, S.A. Thobie, L.P. Untersteller

During 1994, the continuation of works on temporal interpolation and synthesis animation have been the main activity. The aim is to generate intermediate images in 2D from two or several 3D synthesised images. Various algorithms have been developed issued from generalised Bresenham method. All of them can be made parallel and integrated in hardware.

In September 1994, the arriving of G.P. Bonneau allows the reinforcement of this research topic by focusing studies around mathematical basis of the representation of free form curves and surfaces. The objective is to create representations giving the user (designer) more freedom without loosing efficiency during algorithm computation.

TOPIC 3: Image Analysis / Synthesis Co-operation (P. Bourdot)

R. Gherbi, A. Osorio, D. Roussel

The goal of this research topic is to design automatic tools and interactive methods for the reconstruction of virtual objects and for the learning and recognition of 3D shapes, from objects coming from the real world. Our approach is based upon the co-operation between Computer Vision and Computer Graphics (3D modelling and lighting models).

At the present time our work is focused on the ROSA project (Reconstruction of Objets from Stereo-Acquisition) developed in the following examples of research activity. Thanks to an experimental material assembled from 1993 (Indigo 2 workstation + Galileo video board), two graduate students have been welcomed within our team. After his master degree, one of these students (D. Roussel) continues to work on this project by preparing a PhD thesis. The direction of his research is driven with the collaboration of Professor J. P. Cocquerez, based on a co-operation agreement between LIMSI-CNRS and ETIS (Computer Vision and Signal team) of ENSEA (Higher National School for Electronic and its Applications) at Cergy-Pontoise (France).

The main interest of this research topic is to give to future CAD-CAM tools a capacity of perception and virtual reconstruction of objects coming from the real world.

TOPIC 4: Gesture Analysis and Generation (S. Gibet)

A. Braffort, T. Lebourque, D. Teil

Gesture is studied with the objective of being used in Man-Machine communication.

Works concerning gestural analysis of human body movements and especially hand and arm movements are going on. The application field for this research is gesture analysis and gesture recognition of the French Sign Language (A. Braffort).

Concerning gesture generation, a sensori-motor model has been developed for movement control and planning (S. Gibet). This model has been evaluated and applied to realist animation of articulated bodies. This study was the work subject for a student in Data Processing of Paris-Sud University. This student, T. Lebourque, is now preparing a PhD cycle thesis on this research with the orientation to conceive a symbolic command language following the macroscopic organisation of movements and including a semantic structure of gestures.

Researchers of this topic participate in reflections of the working group of Cognitive science network animated by F. Forest.

TOPIC 5: Multimodal Communication (D. Teil)

Y. Bellik, M. Krus

In this topic, research is focused around the setting of a very modular multimodal interface architecture, being usable for a maximum "process control" applications and which permits the synergy of input/output interactive technics.

Recent works in that domain (LimsiDraw, Specimen) allow the design of a text reading and correcting application for the blinds. This project has been realised with the collaboration of the CREARE team of INSERM Institute. The result is the Meditor prototype being now in evaluation at the National Institute for young Blinds (INJA) in Paris. Meditor has obtained the 2d price at the Innovation International competition SETAA 94 in Paris (Y. Bellik).

DRET contract ndeg. 92-358 "Communication Multimodale" has been finished in July 1994 and a proposal to go on these studies is actually setting on. M. Krus collaborated at this work during his military service.

A student in Cognitive Sciences has performed a work on memorisation of mental images by using the demonstration prototype LimsiDraw (D. Fass). These studies were done with the collaboration of the Human-Cognition team.

TOPIC 6: Auditory Scene Analysis

"Auditory Scene management" is a new activity, just being structured. It completes a missing link in communication means research.

TOPIC 7: Human Factors (M.F. Castaing)

During the last few years, we have performed research in the field of human-machine dialogue, specifically the ergonomics of interfaces; this has conducted us to address the difficulty for users to express their requests in written language. Then, following a collaboration with the " CREDIF" (ENS St Cloud) we decided to investigate the area of the graphic representation of " key words " actually used in a spoken or a written request. As an example, we retained the principles of graphical construction of the LimsiDraw prototype and tested a number of icons obtained by association of simple geometrical features, aiming at creating a type of iconic language.

This work was supported by a " Cogniscience " contract on the design of icons for language training. A part of the results appears in the report for the " DESS Ergonomie " Paris V University (S. Darjo) on the design of an iconic language for the representation of the co-verbal gesture. A contract work with the TetraSys company is starting on the interface ergonomics in the area of the consultation of data bases about standardised international technical documentation.

MULTI-TOPIC PROJECT : SAMMOVAR

During the last three years, CNV group took care to federate in a common project the main part of axis and topic research (Vision, CAGD, Gesture, Multimodality, etc.). This was the context where SAMMOVAR (Multimodal System for Acquisition and Manipulation for Virtual Objects Associated to the Real world) project was set up. This year, concerning that plate-form, research activity has been oriented in three directions :

- ROSA project (TOPIC 3),

- "Gesture Analysis / Generation" project (TOPIC 4),

- EMUX study (Multimodal Events under UNIX / X11).

EMUX study has been focused to elaborate solutions for non-standard event management under X11 system context. This was a fundamental step to be able to fuse modalities managed by non conventional devices (speech or gesture recognition systems, eyephone,...). Presently, the results of that study permit us to tackle the adaptation of SPECIMEN (tool for the design of multimodal interfaces) under UNIX / X11, the aim being to use this tool to design a first version of the SAMMOVAR multimodal kernel

CONCLUSION :

All activities centered on the group projects bring competency for the development of the multimodal platform of HCM department.

The re-centring of the CP group activities has induced a partial restructuring of the H.C.M.. Department at the end of 1994. One of the consequences is the recent arriving in the NVC group of eight researchers issued from CP group (two permanents, five thesis students and a contract researcher) whose research problems are either shared or complemented by the previous NVC group members. For most of them, these researcher's activities will complete studies in topics 1 and 5 and for the others, the new topic "Auditory scene analysis" has been created which fills an empty space in the studies of the various communication means.

Following this restructuring, the group will be given a new name.