Geometric Modeling, Simulation and Interaction Assessment2016-2021
Objectives and challenges
The work of the "Geometric Modeling, Simulation and Interaction" theme aims to tackle two major issues. The first concerns the development of tools for the geometric and topological modelling and analysis of 3D objects, both generic and robust enough to be adapted to concrete cases. The second relates to human-machine interaction in a virtual environment, taking into account both human perception of this environment and human-to-machine communication.
Participants
- Three full professors: Dominique Bechmann, David Cazier and Franck Hétroy-Wheeler
- One senior researcher: Birgitta Dresp-Langley (2019-2020)
- Gutenberg Chair 2019, Daniel Oberfeld-Twistel, Associate Professor at Johannes Gutenberg - Universität Mainz (Institute of Psychology)
- Three associate professors: Antonio Capobianco, Jérôme Grosjean and Pierre Kraemer
- Three research engineers: Thierry Blandet, Sylvain Thery and Joris Ravaglia (since 10/2020)
- One engineer from the GEOSIRIS company since 01/09/2018, Lionel Untereiner (research engineer associated with the team since 8/06/2020)
- Postdoctoral students: Sabah Boustila (from 15/01/2020 to 31/12/2020), Flavien Lecuyer (teaching associate IUT Haguenau 2020-2021), Joris Ravaglia (Unistra Idex 2018 - Attractivity programme from 10/2019 to 09/2020)
- PhD students: Paul Viville (Unistra doctoral contract from 10/2019 to 09/2022), Quentin Wendling (Unistra doctoral contract from 10/2019 to 09/2022), Julien Casarin (CIFRE grant with Gfi-Labs from 2016 to 2019. Defended the 30th of September 2019), Alexandre Hurstel (Contract through the 3D-Surg project from 2015 to 2019. Defended the 30th of September 2019), Sabah Boustila (Contract through the CIMBEES project from 2012 to 2015. Defended the 25th of May 2016), Mélinda Boukhana (CIFRE grant with Arvalis from 03/2018 to 12/2021. Defended the 15th of December 2021), Katia Mirande (Contract through the ROMI project from 11/2018 to 04/2022).
Results
Hexahedral mesh generation
The construction of a volumetric mesh for a given geometric domain is a complex problem that has been tackled for many years. The generation of purely hexahedral meshes for domains of any shape is still an open problem. We have developed a processing chain for the generation of hexahedral meshes for domains whose shape can be represented by their skeleton. By exploiting this representation, the generated mesh is well aligned with the geometry of the domain and its connectivity is as regular as possible. The main difficulty lies in the processing of the connectivity of the mesh at the level of the branches of the skeleton which can have any number of incident branches. We have proposed a new solution with the construction of connection surfaces that encode the connectivity of the final solid mesh around each of the vertices of the skeleton. Each vertex is processed independently by choosing, according to the local configuration, an appropriate method. Since these methods are mutually compatible, no system with global constraints needs to be solved. In the end, the proposed processing chain makes it possible to manage complex shapes even in the presence of cycles and many steps can process the cells in parallel. Compared to the state of the art, the quality of the meshes obtained is at least similar if not better. The implemented algorithm is robust to the presence of cycles and generates more symmetric and regular results in many cases. [7-VKB20] [4-VKB21]
Analysis of plant 3D point clouds
In fundamental as well as applied biology, the accurate measurement of geometric features of plants and trees (height, angles of insertion of branches, lengths between nodes, areas of leaves, etc.) is necessary to understand the interaction of a plant with its environment (phenotyping) or the mechanisms underlying its growth, as well as to validate structure-function models (FSPM). In recent years, the development of passive (photogrammetry) or active (laser scanners) acquisition systems has made it possible to limit manual measurements which are costly and require cutting and therefore destroying the plants. These systems generate a virtual model of the object under study in the form of a three-dimensional point cloud. Such a cloud of points must then undergo various geometric treatments: denoising, decomposition into botanical organs (stem or trunk, branches, petioles, leaves), reconstruction of a surface model for each organ, temporal tracking, etc.
Our contribution to such a processing chain is threefold. We propose a method for the robust segmentation of a plant point cloud into botanical organs, based on both a spectral analysis of the point cloud and the incorporation of botanical knowledge (PhD thesis of Katia Mirande, 2018-2022, [7-MHG20]). A spectral analysis makes it possible to determine very precisely the boundaries between organs, in particular between the petiole and the blade of a leaf. The use of prior knowledge on the structure of the plant in a probabilistic algorithm globally improves the robustness of the segmentation. Our second contribution concerns the reconstruction of the surface of a limb, with the aim of precisely estimating its area (PhD thesis of Mélinda Boukhana, 2018-2021, [7-BRHL20]). In particular, we conducted a comparative study between seven classical methods for surface reconstruction, according to nine criteria that we defined for the occasion and using an original synthetic model of a 3D point cloud of a leaf blade. We show that none of these methods robustly estimate the area of a limb, and we propose a more efficient alternative approach based on a Bézier surface. Finally, we propose a multi-scale algorithm for point-to-point monitoring of 3D point clouds of growing plants (Haolin Pan's Master thesis, [4- PHCC21]). Our algorithm is based on the computation of a topological skeleton of the plant. This work was carried out in collaboration with plant specialists, respectively C. Godin (Inria Lyon), B. de Solan (Arvalis) and D. Colliaux (Sony Computer Science Laboratory).
Perception of spatial arrangement in virtual environments
Virtual reality has shown many advantages in several areas, in particular the architectural field. However, previous work has proven that distances are poorly perceived whether in indoor or outdoor spaces. Several visual clues can affect and improve our perception of distances such as colours, etc. The Gutenberg project led by Daniel Oberfeld-Twistel aims to examine other indices, little studied previously, in particular the auditory cue. Previous research has shown that acoustic properties have an effect on the perceived spatial arrangement of an interior space and that, to some extent, the dimensions of a room can be estimated by listening to a sound source inside the room. However, our understanding of the interaction between spatial layout and room acoustic properties remains limited. The objective of the project is to identify the principles underlying the perception of interior spaces from auditory or audiovisual information and thus answer several questions. Can humans estimate the height, width, depth, volume and shape of space when only auditory information is available? Is it also possible to make a small room seem more spacious by manipulating its acoustic characteristics? As part of Sabah Boustila's postdoc, a first experiment examines the effect of visual and auditory cues together as well as the auditory cue alone on the perception of dimensions. The experiment takes place in virtual rooms that are empty, windowless and contain a door to provide a familiar visual cue. The shape of the pieces is rectangular, we vary their size with each test; width (3.4m, 6.5m, 9.5m) X height (2.5m, 4.25m, 6m), depth was calculated based on width; ratio(1.2, 2.2). Each participant performs two conditions: auditory + visual and auditory only. The task is to estimate the perceived width, depth, height, and volume of the rooms in units of the metric system (metres or centimetres). This experiment will provide precise information on the dependence of perceived room size on room acoustics, and whether combining audio and visual displays can improve the accuracy of the room size perception.
Intuitive and contactless interaction
During an image-guided surgery, the consultation of medical images, in 2D or 3D, plays a crucial control role for the surgeon but induces a loss of sterility due to contact with the keyboard and mouse. To avoid this problem, as part of the BPI 3D-Surg project, we have proposed a conceptual approach [2-HB19] to design an intuitive gestural vocabulary, effective and well adapted to the specificities of the surgical context, allowing the use of contactless technologies. The key idea is to observe the gestures that the user instinctively produces when confronted with the task of having to produce a given interaction. In doing so, each subject provided us with their own gesture vocabulary through an elicitation process (Wizard of Oz). The gestures collected were then analyzed to identify those that were most frequently used by all and to deduce a general but intuitive gesture vocabulary, specific to the tested application, and best satisfying the contextual, conceptual and technical criteria that we had previously identified. The second step of our approach consists in having the previous generic vocabulary evaluated by the target users (surgeons) using the feedback from the experts to further develop it in order to optimize ease of use, memorization, consistency, performance and posture comfort [8-Hurs19].
Collaborative work in mixed reality
Recent technological advances in virtual reality have given rise to a varied set of media for viewing virtual environments and interacting by manipulating 3D content. Our objective is to allow the creation of virtual experiences, accessible from any medium, in order to offer collaborative work involving actors, with different trades and roles, and therefore different needs in terms of visualization (and especially of interaction with 3D content). This is why a single device or medium is not enough to support collaborative work. To do this, we have designed the UMI3D protocol [4-CLTB19], an interface between content and media, allowing any medium to interact with a remote 3D content, hosted on a server. The operation of the UMI3D exchange protocol is mainly based on the establishment of interaction contracts between the 3D content and the media. The server communicates with the support thanks to a two-way web connection established between the 3D content and the UMI3D browser dedicated to the support. The server then describes to the browser a set of interaction contracts that define the possible interactions at this moment for the user. These contracts are defined by the transmission, standardized in JSON format, of a combination of objects that we call interaction bricks. The notion of interaction brick is based on our study of degrees of freedom published at IEEE VR 2011 [4-VCB11]. These bricks constitute a reduced number of classes making it possible to describe the possible interactions in the virtual environment. Upon receipt of the interaction contract by the browser, the latter performs a so-called projection operation, to generate or update the user interface specific to the medium, and thus make it possible for the user to carry out the interaction contract. As part of Julien Casarin's CIFRE PhD thesis [8-Casa19], we have also implemented the UMI3D model with the Unity3D game engine [4-CPB18 (honourable mention at CSCW2018), .unistra.fr/6-CGPS18 6-CGPS18] and the result is a Unity toolkit that can be used to design collaborative interactions.