Team IGG : Computer Graphics and Geometry

Difference between revisions of "ExRealis"

From Team IGG : Computer Graphics and Geometry
Jump to navigation Jump to search
 
(57 intermediate revisions by 7 users not shown)
Line 1: Line 1:
−
= IGG's digitization platform =
+
__NOTOC__
 +
__NOEDITSECTION__
 +
ExRealis becomes a pole of the GAIA ICube Platform (Graphics, Artificial Intelligence and data Analysis).
  
−
The goal of this platform is to offer a certain number of techniques (hardware and software) allowing to acquire 3D objects with visual appearance or complex materials for applications in computer-generated pictures and virtual reality.
+
= ''ExRealis'' - A digitization framework =
  
−
Instead of having a person who creates a polygonal model of an existing object, we wish to be able to directly acquire this model from reality. Nowadays there are many existing techniques and equipments (e.g. 3D scanners) allowing to digitize. But these techniques often have strong limitations: their use requires a high degree of expertise, several steps require tedious manual interventions, the resulting models are huge and impossible to render in real-time directly on the GPU, etc.
+
In many industrial sectors, an increasing demand for tools intended to the production of digital content can be observed, either for video game or movie industry, simulation applications or rapid prototyping. Nowadays, many off-the-shelf 3D scanners are available, whose metrological accuracy guarantees reliable acquisitions of real objects and the creation of faithful digital surrogates. This partially leads to turn the digital content production task, so far exclusively granted to computer graphics artists, into a technical act of data gathering, often cheaper.
  
−
IGG's digitization platform covers the three following aspects:
+
Despite the improvement of shape measurement technologies, processing measured data in order to create digital content directly usable by computer graphics applications still requires many intermediate tasks. Moreover, for specific fields like cultural heritage and artistic domains, shape only is far from being sufficient: measuring the appearance is of major importance too for the display of realistic-looking digital models.
  
−
* acquisition of models with visual appearance from dedicated devices,
+
The goal of ''ExRealis'' is to provide a unified framework that answers in an efficient manner to all of these needs. It offers several tools, devices and softwares that give the possibility to produce digital content of various nature from real data, with the purpose of supporting research works conducted by the IGG team around themes like realistic rendering and texture synthesis, as well as enabling service provisions in digitization with our partners. ''ExRealis'' proposes:
−
* automatic (or semi-automatic) treatments of models (registration, filtering, fusion, reconstruction, simplification and representation tuned for the application),
+
* a range of devices and software tools covering the whole digitization processing pipeline, from data acquisition to the creation of textured 3D models;
−
* specific rendering tools on the GPU.  
+
* some advanced methods for texture reconstruction and visualization accounting for complex lighting environments or material characteristics.
  
−
= Equipments =
+
<youtube>https://www.youtube.com/watch?v=916EWgXwFNI</youtube>
  
−
'''Optical scanner by projection of fringes and phase shift measuring'''.
+
= Our devices =
  
−
[[Image:scannerlumstruct_lsiit.png|right|thumb|120px]]
+
The ''ExRealis'' framework contains several devices, detailed below, that enable the measurement of geometry and colour from real scenes or real objects.
  
−
This device allows the 3D acquisition of objects of a size between approximately 10 and 50cm. An external video camera allows to perform an automatic registration of several exposures and also the superposition of information concerning the RGB colored directional appearance. A specific software has been developed in order to make acquisitions and restoration on screen.
+
<gallery widths=250px>
 +
Image:ScannerLumStruct.jpg|Short range scanner
 +
Image:laser_leica.png|Medium range scanner
 +
Image:PhotoHardware.jpg|Photographic equipment
 +
</gallery>
  
 +
'''Short range scanner'''
  
−
'''Goniometer'''.
+
This optical device based on structured light allows the 3D scanning of objects whose size ranges from 20cm to 1m. Depth measurement ranges from 80 to 120cm and the size of the produced range images is of 1280x960 pixels, which leads to an average resolution of 600 to 700 points per square cm<!-- précision de la mesure?? -->.<BR>
 +
The controlling software has entirely been developed by our team, which thus offers a high degree of flexibility and the possibility of a complete reconfiguration, so as to adjust, for instance, the device resolution to specific needs.
  
−
[[Image:goniometre_lsiit.png|left|thumb|200px]]
 
  
−
This device allows the positioning of a light source on a hemispheric dome in order to vary the lighting of a 3D object or of a aterial sample (whose size is between 10 and 20cm). This device will be completed at the end of 2009 by a second mobile rotating arch to allow the positioning of a video camera on a hemispheric dome.
+
'''Medium range scanner'''
  
−
This device, though yet incomplete, has been used for the acquisition of paintings using bidirectional texture function.  
+
The time of flight laser scanner ''ScanStation2'' from Leica Geosystems allows depth measurement ranging from 0.2 to 300m, at a speed rate peaking to 50 000 points/sec. Moreover, a set of targets provided by the manufacturer enables the registration of point clouds acquired from different locations.<BR>
 +
Please see the [http://www.leica-geosystems.fr/fr/Scanner-Laser-3D-Leica-ScanStation-2_62189.htm technical documentation] from Leica Geosystems for more details.
  
−
An example of painting digitization:
 
  
−
[[Image:tableaunum_lsiit.png|200px]]
+
'''Photographic equipment'''
  
−
The left picture shows a photo of a painting, the center image, a computer-generated rendering for the same point of view by using a texture mapping of classical colors, and the right image, by using a bidirectional texture function from which the relief has been decoupled from the bidirectional reflectance.
+
Our team also has at its disposal a semi-professional photographic equipment, including two cameras Canon EOS 5D Mark ii as well as lenses ranging from 25mm to 125mm. We use it for the acquisition of real object appearance (colour, texture), or for photogrammetry based 3D scanning.
  
−
'''Mid-range laser scanner'''.
+
= Our software solution =
  
−
[[Image:laser_leica.png|right|thumb|120px]]
+
''ExRealis'' proposes a software for the processing of scanning data. We manage a continuous development process in order to integrate every tool which is useful to the creation of realistic digital models from physical real objects. The algorithms proposed allow the processing of geometry (reconstruction of 3D meshes from scanning device data) as well as appearance (colour or texture synthesis over those meshes from pictures).
  
−
It consists of a laser ScanStation2 from Leica Geosystems. It allows to perform scans on a range of 0.2-300m with a speed of up to 50000 points/sec. A target set also allows to perform registration. See also the [http://www.leica-geosystems.fr/fr/Scanner-Laser-3D-Leica-ScanStation-2_62189.htm data sheet] (in french) of Leica Geosystems. 
+
[[Image:PageTeaser_ExRealis.jpg|800px]]
  
−
This laser has been used to digitize a part of the military fortification Fort de Bois l'Abbé close to the city Epinal.
+
Dedicated to a practical usage, this software has been designed in a modular way, with a constant care for ergonomics. It is intended to manage huge amounts of data, thanks to out-of-core data structures and mechanisms, and most of its algorithms are optimized thanks to parallel or GPU programming.
−
[[Image:Fort.png|200px]]
 
  
−
= Logiciels =
 
  
−
'''Registration, Denoising, Integration, Simplification'''.
+
'''Geometry acquisition and processing'''
  
−
Many manual user interventions are necessary to obtain a complete 3D polygonal model from the brute scanned data (point clouds). We aim at developping automatic tools (or at least tools making the job less tedious).  
+
After a 3D scanning session (namely the data gathering by itself), many processings have to be applied to the raw data produced by the scanner before obtaining a usable 3D model.
  
−
Example of registration, denoising and simplification:
+
[[Image:PipelineGeo.png|900px]]
−
 
−
[[Image:Recalage.png|250px]] [[Image:Debruitage.png|120px]] [[Image:Simplification.png|250px]]
 
  
−
'''Texture, Apparence, Rendering'''.
+
Our software covers every step of this processing pipeline, thanks either to fully automatic algorithmic solutions, or to a suitable user interface whenever user intervention is needed, like in the case of some of the cleaning tasks, for instance. We are then in position to produce, from real objects, 3D models that are correct in terms of geometry and topology, at different level of details, and that can be exported to many file formats compatible with off-the-shelf CAD softwares.
  
−
The acquisition of appearance is a difficult problem. Many photographs must be taken and registred with the geometric model. We aim at developping tools to ease this work and to make models compatible with modern graphics boards for visualization.
 
  
−
Example of rendering:
+
{|cellpadding=10 cellspacing=0 width=100%
 +
|-
 +
|[[Image:PipelineGeo01.png|250px]]
 +
|'''Geometry registration.''' Devices requiring multiple 3D acquisitions from different viewpoints (structured light range scanners, lidars, etc) provide for each acquisition an independent point cloud, which needs to be realigned with the others, similarly to a big 3D puzzle. We developed an algorithm that enabled us to automate this process for the specific case of structured light range scanners, but ''ExRealis'' also provides manual registration thanks to a dedicated user interface, as well as registration refinement by iterative minimization algorithms.
 +
|-
 +
|[[Image:PipelineGeo02.png|250px]]
 +
|'''Cleaning.''' Since no scanning technology is now noise free, it is not uncommon to recover erroneous measured points, which risk to disturb the following processing. Here again, a dedicated user interface enables manual cleaning. However, this is often a tedious task. That’s why heuristics have been implemented in order to automate this process, using confidence values computed from geometric or colorimetric criteria.
 +
|-
 +
|[[Image:PipelineGeo03.png|250px]]
 +
|'''Surface reconstruction.''' Point clouds provided by shape measurement technologies constitute a representation which is not adapted to many usages to which 3D models are intended (realistic rendering, physical simulation, etc). It is thus often necessary to obtain a closed description of the surface, in the form of a 3D mesh. To do so, we integrated to ''ExRealis'' some methods derived from scientific literature to extract triangulated surfaces from sets of measured points.
 +
|-
 +
|[[Image:PipelineGeo04.png|250px]]
 +
|'''Surface simplification.''' To display 3D models on devices with low graphical performances (smartphones, tablets, autonomous virtual reality head mounted devices), it is important to be able to reduce geometry complexity. ''ExRealis'' incorporates different algorithms allowing this reduction while limiting the loss of geometry details, so as to preserve as most as possible the object shape during the simplification step.
 +
|-
 +
|}
 +
''ExRealis'' also provides many other tools for the processing or the improvement of geometric data: re-sampling of point clouds, surface smoothing, hole filling, distance and volume measurement, etc.
  
−
[[Image:aphro_render.png|200px]]
 
  
−
Exemple of texture acquisition and registration:
+
'''Appearance acquisition and processing'''
  
−
[[Image:acqu_texture.png|200px]]
+
[[Image:PipelineAppearance.png|900px]]
 +
 
 +
Acquiring the appearance of a real object starts from a photographic campaign, so as to measure its photometric properties. Reconstructing a texture from these pictures then requires to solve the following key points:
 +
 
 +
{|cellpadding=10 cellspacing=0 width=100%
 +
|-
 +
|[[Image:Param.png|250px]]
 +
|'''Surface parametrisation.''' Giving an appearance to a 3D object is classically made through the use of textures, which are images that may contain various informations (albedo, reflectivity, roughness, details, and so on). To make each portion of the surface corresponding to an image part, the 3D object first need to be unfolded and flattened onto a plane, similarly to the earth globe onto a planisphere. Several algorithms to solve this so-called parametrisation step are available in the software.
 +
|-
 +
|[[Image:PipelineAppearance02.png|250px]]
 +
|'''Photography registration.''' To reconstruct the colour of a model from pictures, a first step consists in recovering the viewpoints from which they have been shot, so as to be able to project their content onto the 3D model, similarly to what a video-projector would do. Camera calibration methods have then been integrated to ''ExRealis'' for this purpose, as a preliminary step to appearance reconstruction.
 +
|-
 +
|[[Image:PipelineAppearance03.png|250px]]
 +
|'''Colour map reconstruction.''' The use of multiple pictures for colour map reconstruction requires to manage adequately the surface areas where several images are projected. Indeed, in these overlapping regions, a naive blending may result in visual artefacts. ''ExRealis'' provides more elaborated blending strategies based on visibility criteria which allow, for instance, to reduce the impact or reprojection errors or to eliminate the focus blur that may appear in some of the source images.
 +
|-
 +
|[[Image:PipelineAppearance04.png|250px]]
 +
|'''Normal map reconstruction.''' Beyond a certain threshold, surface simplification may lead to the loss of geometry details. By storing those details into a normal map, it thus becomes possible to modify illumination during the rendering, in order to simulate geometry complexity over simplified 3D meshes. This approach is much cheaper than directly displaying highly detailed meshes. ''ExRealis'' allows the reconstruction of normal maps for coarse meshes from more detailed versions of those.<BR>
 +
On the image showed here, the model on the left is made of 10K triangles, the one in the middle of 4M triangles, and the one on the right of 10K triangles + a normal map to store the details.
 +
|-
 +
|}
 +
 
 +
The produced textures may consist either in a simple colour information, or in more advanced representation models that give not only a clue on the object's hue, but also on other properties of its material, like shininess, for instance.
 +
 
 +
The IGG team has worked on the fitting and the real-time rendering of such models, considering especially ''light fields''. Light fields capture the appearance of an object within a given lighting environment: the one corresponding to the acquisition moment. This includes, among other things, all illumination effets related to the observer's displacement around the object (specular peaks, inter-reflections, ''etc.'') and allows afterwards the free inspection of the digital copy with the same lighting conditions, while maintaining a high degree or realism with respect to more basic texture models.
 +
[[Image:Elephant2.png|600px|thumb|left|Comparison between colour texture and ''light field''. Reflections improve realism and offer a better understanding on the nature of the materials the object is made of.]]
 +
<br style="clear: both" />
 +
 
 +
= Our expertise =
 +
 
 +
Working on ''3D + appearance'' digitization since 2005, the IGG team has acquired a good expertise of this scientific field and a good knowledge on how to apply it to the practical cases of digitization campaigns. Within the framework of various collaborations, we had many occasions to apply these expertise and knowledge through service provisions, as illustrated below.
 +
 
 +
 
 +
'''The Voodoo Museum of Strasbourg'''
 +
 
 +
The company [https://www.facebook.com/vrjourney.fr Virtual Journey] has launched a project to propose to the visitors of the [http://www.chateau-vodou.com/fr/chateau-musee-vodou-strasbourg/ Voodoo Musem] in Strasbourg to live the peculiar rituals related to this secret religion still unknown in Europe through the use of a virtual reality head mounted device. The experience allows to be immersed at the heart of ceremonies thanks to 360° 3D videos shot in Benin, as well as to bring to life some of the museum artefacts so as to explain their usage by means of digital copies derived from 3D scanning. Our team has been involved in this project for that purpose.
 +
 
 +
[[Image:FaitsMarquants_MuseeVodou.png|900px]]
 +
 
 +
We thus performed a photogrammetry scanning campaign during two and a half days so as to digitize about fifteen artefacts in relation to the presented rituals. After processing in lab, in particular with the ''ExRealis'' software that we develop, digital copies have been delivered to Virtual Journey for integration into its virtual reality application.
 +
 
 +
[[Image:MuseeVodou.png|900px]]
 +
 
 +
Officially deployed in October 2019, this project has been welcomed in a very enthusiastic way by visitors and by the press, which has offered us a fantastic media coverage:
 +
* article and reportage on the website [https://www.alsace20.tv/VOD/Actu/6-minutes-eurometropole/Realite-virtuelle-invite-Chateau-Musee-Vodou-Strasbourg-6QEjZxHPoj.html Alsace20].
 +
* article in [https://www.20minutes.fr/arts-stars/culture/2629351-20191016-strasbourg-musee-propose-ceremonies-vaudou-realite-virtuelle 20 Minutes].
 +
* article in [https://www.dna.fr/edition-de-strasbourg/2019/10/22/video-une-ceremonie-vodou-a-decouvrir-en-realite-virtuelle-a-strasbourg DNA].
 +
* article on the blog [https://pokaa.fr/2019/10/18/nouvelle-experience-a-strasbourg-se-faire-transporter-au-milieu-dun-rituel-vodou/ Pokaa].
 +
* presentation at the digital festival [http://bizzandbuzz.alsace/vivez-un-rituel-vodou Bizz&Buzz].
 +
<!--
 +
{|cellpadding=2 cellspacing=20 width=100px
 +
!
 +
!
 +
|-
 +
|<youtube width=150>https://www.youtube.com/watch?v=YjimeETqwJY</youtube>
 +
|<youtube>https://www.youtube.com/watch?v=qqd4ysc6vvg</youtube>
 +
|<youtube>https://www.youtube.com/watch?v=czzBWkEMkzA</youtube>
 +
|<youtube>https://www.youtube.com/watch?v=hrMlC6E3pe8</youtube>
 +
|-
 +
|}
 +
-->
 +
 
 +
 
 +
''' ''&OElig;uvre Notre Dame'' foundation'''
 +
 
 +
Within the framework of the [http://www.eveil-3d.eu/francais/index.php Eveil3D] project, and in partnership with the technology transfer center Holo3 and the ''&OElig;uvre Notre Dame'' foundation, the IGG team has performed in october 2013 the digitization of two statues of the cathedral of Strasbourg. The goal was to use these statues in an immersive 3D environment for a language learning serious game. About 50cm high, they've been digitized with our short range structured light scanner. A photographic campaign has also been done so as to produce colour textures for both models.
 +
 
 +
[[Image:OND_Ourson.png|600px|thumb|left|''Bear statue'', cathedral of Strasbourg. From left: picture, 3D model reconstructed after geometry acquisition (35M points), textured 3D model reconstructed after appearance acquisition (55 pictures).]]<br style="clear: both" />
 +
[[Image:OND_Taureau.png|600px|thumb|left|''Bull statue'', cathedral of Strasbourg. From left: picture, 3D model reconstructed after geometry acquisition (22M points), textured 3D model reconstructed after appearance acquisition (28 pictures).]]<br style="clear: both" />
 +
 
 +
'''Inter-university House of Human Sciences (''MISHA''), Strasbourg'''
 +
 
 +
''MISHA'' has at its disposal a huge collection of ethnographic artefacts, and has launched some years ago a project based on the OpenSIM game engine that provides a virtual world to make it possible the study and the contextualization of these artefacts. Obviously, this requires to have digital copies of them, and so to have digitized them beforehand.
 +
 
 +
For this purpose, they contacted us in autumn 2013. They needed to produce digital copies of about ten african ethnographic objects from the Dogon tribe. Their geometry has been digitized with the short range structured light scanner, and their colour thanks to a photographic campaign.
 +
 
 +
[[Image:MISHA_Masks.png|600px|thumb|left|Digital copies of three Dogon masks. From left: ''adone'' mask (antelope), ''kanaga'' mask, bird mask with a ''dege'' at the top of it.]]<br style="clear: both" />
 +
[[Image:MISHA_Hogon.png|400px|thumb|left|Digital copy of a ''Hogon'' cup. Left-hand side: the cup fully assembled; right-hand side: each single piece presented separately.]]<br style="clear: both" />
 +
[[Image:MISHA_Dege.png|300px|thumb|left|Digital copies of two ''dege'' (female figurines).]]<br style="clear: both" />
 +
 
 +
''' ''Bois l'Abbé'' fortifications'''
 +
 
 +
Below are presented some renderings of a digitization we made of a part of the ''Bois l'Abbé'' fortifications [http://maps.google.com/?ie=UTF8&t=h&ll=48.204577,6.40107&spn=0.007022,0.016512&z=17 (48°12'16.3"N&nbsp;6°24'00.8"E)], located at Uxegney near Epinal, France. The final model is made of about 63M points acquired with the Leica Scanstation 2 scanner from 20 different locations. Colour assigned to the points comes from the internal scanner camera, dedicated to data handling conveniency but not to a faithful appearance acquisition.
 +
<gallery widths=250px>
 +
Image:FlortBLAScanPositions.png
 +
<!--Image:Fort.png
 +
Image:FortBLAOverview1.png-->
 +
Image:FortBLAOverview2.png
 +
Image:FortBLAOverview3.png
 +
Image:FortBLACloseup1.png
 +
<!--Image:FortBLACloseup2.png-->
 +
</gallery>
 +
<!--
 +
Ci-dessous, un exemple de numérisation de tableau. L'image de gauche montre une photographie du tableau, l'image du milieu un rendu en synthèse d'images pour le même point de vue en utilisant un placage de texture de couleur classique et l'image de droite en utilisant une fonction bidirectionnelle de texture de laquelle le relief a été découplé de la réflectance bidirectionnelle.
 +
 
 +
[[Image:tableaunum_lsiit.png|600px]]
 +
-->
 +
 
 +
''' ''Gypsothèque'', university of Strasbourg'''
 +
 
 +
We had the occasion to get in touch with the ''Gypsothèque'' of the Strasbourg university (plaster copy museum of cultural artefacts) to perform the digitization of one of their statues.
 +
 
 +
[[Image:Aphro.png|600px|thumb|left|Digitized model of a statue of Aphrodite, ''Gypsothèque'' of Strasbourg.]]
 +
<br style="clear: both" />
 +
 
 +
'''Some digitizations made at home'''
 +
 
 +
Here are some models digitized in our lab in order to produce data sets for illustrating the research works of our team. Some of these 3D models are provided with ''light field'' textures, that enable to simulate illumination variations related to the observer's displacement for more realism, as explained before. It has to be noted that basic colour textures can also be exported using standard image file formats.
 +
 
 +
<gallery widths=200px>
 +
Image:NUM_Elephant.png|''Elephant''<BR>Geometry: 4.9M triangles<BR>Texture: light field + colour
 +
Image:NUM_Dragon.png|''Dragon''<BR>Geometry: 18.2M triangles<BR>Texture: light field + colour
 +
Image:NUM_Mask1.png|''Mask1''<BR>Geometry: 7.7M triangles<BR>Texture: light field + colour
 +
Image:NUM_Mask2.png|''Mask2''<BR>Geometry: 8.7M triangles<BR>Texture: colour
 +
Image:NUM_Venus.png|''Venus at the bath''<BR>Geometry: 3.6M triangles<BR>No texture
 +
</gallery>
  
 
= Collaborations =
 
= Collaborations =
  
−
* IGG's digitization platform takes place in the [https://lsiit.u-strasbg.fr/pnm3d-fr/index.php/Accueil plateform of digitization and 3D modeling] (in french) of Université de Strasbourg.
+
Several national and regional projects are supporting this framework:
−
* Several national and regional projects are associated to this plateform: [http://artis.imag.fr/Projets/ATROCO/ ANR ATROCO], Ministry RIAM AMI3D, Region "Pôle Image".
+
* Projet Interreg EVEIL3D,
 +
* [http://artis.imag.fr/Projets/ATROCO/ ANR ATROCO],
 +
* Ministère RIAM AMI3D,
 +
* Région Pôle Image.
 +
 
 +
= Contacts =
 +
 
 +
If you have needs either for service provisions or for technology transfer, or if you are just interested by one of the models presented in this page, please contact '''[https://sites.google.com/view/frederic-larue/ Frédéric Larue]''', research engineer in charge of the ''ExRealis'' framework.
  
−
[[fr:Plateformes:Numérisation]]
+
<BR>
 +
<BR>
 +
[[fr:ExRealis]]

Latest revision as of 15:44, 18 August 2021


ExRealis becomes a pole of the GAIA ICube Platform (Graphics, Artificial Intelligence and data Analysis).

ExRealis - A digitization framework

In many industrial sectors, an increasing demand for tools intended to the production of digital content can be observed, either for video game or movie industry, simulation applications or rapid prototyping. Nowadays, many off-the-shelf 3D scanners are available, whose metrological accuracy guarantees reliable acquisitions of real objects and the creation of faithful digital surrogates. This partially leads to turn the digital content production task, so far exclusively granted to computer graphics artists, into a technical act of data gathering, often cheaper.

Despite the improvement of shape measurement technologies, processing measured data in order to create digital content directly usable by computer graphics applications still requires many intermediate tasks. Moreover, for specific fields like cultural heritage and artistic domains, shape only is far from being sufficient: measuring the appearance is of major importance too for the display of realistic-looking digital models.

The goal of ExRealis is to provide a unified framework that answers in an efficient manner to all of these needs. It offers several tools, devices and softwares that give the possibility to produce digital content of various nature from real data, with the purpose of supporting research works conducted by the IGG team around themes like realistic rendering and texture synthesis, as well as enabling service provisions in digitization with our partners. ExRealis proposes:

  • a range of devices and software tools covering the whole digitization processing pipeline, from data acquisition to the creation of textured 3D models;
  • some advanced methods for texture reconstruction and visualization accounting for complex lighting environments or material characteristics.

Our devices

The ExRealis framework contains several devices, detailed below, that enable the measurement of geometry and colour from real scenes or real objects.

  • Short range scanner

  • Medium range scanner

  • Photographic equipment

Short range scanner

This optical device based on structured light allows the 3D scanning of objects whose size ranges from 20cm to 1m. Depth measurement ranges from 80 to 120cm and the size of the produced range images is of 1280x960 pixels, which leads to an average resolution of 600 to 700 points per square cm.
The controlling software has entirely been developed by our team, which thus offers a high degree of flexibility and the possibility of a complete reconfiguration, so as to adjust, for instance, the device resolution to specific needs.


Medium range scanner

The time of flight laser scanner ScanStation2 from Leica Geosystems allows depth measurement ranging from 0.2 to 300m, at a speed rate peaking to 50 000 points/sec. Moreover, a set of targets provided by the manufacturer enables the registration of point clouds acquired from different locations.
Please see the technical documentation from Leica Geosystems for more details.


Photographic equipment

Our team also has at its disposal a semi-professional photographic equipment, including two cameras Canon EOS 5D Mark ii as well as lenses ranging from 25mm to 125mm. We use it for the acquisition of real object appearance (colour, texture), or for photogrammetry based 3D scanning.

Our software solution

ExRealis proposes a software for the processing of scanning data. We manage a continuous development process in order to integrate every tool which is useful to the creation of realistic digital models from physical real objects. The algorithms proposed allow the processing of geometry (reconstruction of 3D meshes from scanning device data) as well as appearance (colour or texture synthesis over those meshes from pictures).

PageTeaser ExRealis.jpg

Dedicated to a practical usage, this software has been designed in a modular way, with a constant care for ergonomics. It is intended to manage huge amounts of data, thanks to out-of-core data structures and mechanisms, and most of its algorithms are optimized thanks to parallel or GPU programming.


Geometry acquisition and processing

After a 3D scanning session (namely the data gathering by itself), many processings have to be applied to the raw data produced by the scanner before obtaining a usable 3D model.

PipelineGeo.png

Our software covers every step of this processing pipeline, thanks either to fully automatic algorithmic solutions, or to a suitable user interface whenever user intervention is needed, like in the case of some of the cleaning tasks, for instance. We are then in position to produce, from real objects, 3D models that are correct in terms of geometry and topology, at different level of details, and that can be exported to many file formats compatible with off-the-shelf CAD softwares.


PipelineGeo01.png Geometry registration. Devices requiring multiple 3D acquisitions from different viewpoints (structured light range scanners, lidars, etc) provide for each acquisition an independent point cloud, which needs to be realigned with the others, similarly to a big 3D puzzle. We developed an algorithm that enabled us to automate this process for the specific case of structured light range scanners, but ExRealis also provides manual registration thanks to a dedicated user interface, as well as registration refinement by iterative minimization algorithms.
PipelineGeo02.png Cleaning. Since no scanning technology is now noise free, it is not uncommon to recover erroneous measured points, which risk to disturb the following processing. Here again, a dedicated user interface enables manual cleaning. However, this is often a tedious task. That’s why heuristics have been implemented in order to automate this process, using confidence values computed from geometric or colorimetric criteria.
PipelineGeo03.png Surface reconstruction. Point clouds provided by shape measurement technologies constitute a representation which is not adapted to many usages to which 3D models are intended (realistic rendering, physical simulation, etc). It is thus often necessary to obtain a closed description of the surface, in the form of a 3D mesh. To do so, we integrated to ExRealis some methods derived from scientific literature to extract triangulated surfaces from sets of measured points.
PipelineGeo04.png Surface simplification. To display 3D models on devices with low graphical performances (smartphones, tablets, autonomous virtual reality head mounted devices), it is important to be able to reduce geometry complexity. ExRealis incorporates different algorithms allowing this reduction while limiting the loss of geometry details, so as to preserve as most as possible the object shape during the simplification step.

ExRealis also provides many other tools for the processing or the improvement of geometric data: re-sampling of point clouds, surface smoothing, hole filling, distance and volume measurement, etc.


Appearance acquisition and processing

PipelineAppearance.png

Acquiring the appearance of a real object starts from a photographic campaign, so as to measure its photometric properties. Reconstructing a texture from these pictures then requires to solve the following key points:

Param.png Surface parametrisation. Giving an appearance to a 3D object is classically made through the use of textures, which are images that may contain various informations (albedo, reflectivity, roughness, details, and so on). To make each portion of the surface corresponding to an image part, the 3D object first need to be unfolded and flattened onto a plane, similarly to the earth globe onto a planisphere. Several algorithms to solve this so-called parametrisation step are available in the software.
PipelineAppearance02.png Photography registration. To reconstruct the colour of a model from pictures, a first step consists in recovering the viewpoints from which they have been shot, so as to be able to project their content onto the 3D model, similarly to what a video-projector would do. Camera calibration methods have then been integrated to ExRealis for this purpose, as a preliminary step to appearance reconstruction.
PipelineAppearance03.png Colour map reconstruction. The use of multiple pictures for colour map reconstruction requires to manage adequately the surface areas where several images are projected. Indeed, in these overlapping regions, a naive blending may result in visual artefacts. ExRealis provides more elaborated blending strategies based on visibility criteria which allow, for instance, to reduce the impact or reprojection errors or to eliminate the focus blur that may appear in some of the source images.
PipelineAppearance04.png Normal map reconstruction. Beyond a certain threshold, surface simplification may lead to the loss of geometry details. By storing those details into a normal map, it thus becomes possible to modify illumination during the rendering, in order to simulate geometry complexity over simplified 3D meshes. This approach is much cheaper than directly displaying highly detailed meshes. ExRealis allows the reconstruction of normal maps for coarse meshes from more detailed versions of those.

On the image showed here, the model on the left is made of 10K triangles, the one in the middle of 4M triangles, and the one on the right of 10K triangles + a normal map to store the details.

The produced textures may consist either in a simple colour information, or in more advanced representation models that give not only a clue on the object's hue, but also on other properties of its material, like shininess, for instance.

The IGG team has worked on the fitting and the real-time rendering of such models, considering especially light fields. Light fields capture the appearance of an object within a given lighting environment: the one corresponding to the acquisition moment. This includes, among other things, all illumination effets related to the observer's displacement around the object (specular peaks, inter-reflections, etc.) and allows afterwards the free inspection of the digital copy with the same lighting conditions, while maintaining a high degree or realism with respect to more basic texture models.

Comparison between colour texture and light field. Reflections improve realism and offer a better understanding on the nature of the materials the object is made of.


Our expertise

Working on 3D + appearance digitization since 2005, the IGG team has acquired a good expertise of this scientific field and a good knowledge on how to apply it to the practical cases of digitization campaigns. Within the framework of various collaborations, we had many occasions to apply these expertise and knowledge through service provisions, as illustrated below.


The Voodoo Museum of Strasbourg

The company Virtual Journey has launched a project to propose to the visitors of the Voodoo Musem in Strasbourg to live the peculiar rituals related to this secret religion still unknown in Europe through the use of a virtual reality head mounted device. The experience allows to be immersed at the heart of ceremonies thanks to 360° 3D videos shot in Benin, as well as to bring to life some of the museum artefacts so as to explain their usage by means of digital copies derived from 3D scanning. Our team has been involved in this project for that purpose.

FaitsMarquants MuseeVodou.png

We thus performed a photogrammetry scanning campaign during two and a half days so as to digitize about fifteen artefacts in relation to the presented rituals. After processing in lab, in particular with the ExRealis software that we develop, digital copies have been delivered to Virtual Journey for integration into its virtual reality application.

MuseeVodou.png

Officially deployed in October 2019, this project has been welcomed in a very enthusiastic way by visitors and by the press, which has offered us a fantastic media coverage:


Œuvre Notre Dame foundation

Within the framework of the Eveil3D project, and in partnership with the technology transfer center Holo3 and the Œuvre Notre Dame foundation, the IGG team has performed in october 2013 the digitization of two statues of the cathedral of Strasbourg. The goal was to use these statues in an immersive 3D environment for a language learning serious game. About 50cm high, they've been digitized with our short range structured light scanner. A photographic campaign has also been done so as to produce colour textures for both models.

Bear statue, cathedral of Strasbourg. From left: picture, 3D model reconstructed after geometry acquisition (35M points), textured 3D model reconstructed after appearance acquisition (55 pictures).


Bull statue, cathedral of Strasbourg. From left: picture, 3D model reconstructed after geometry acquisition (22M points), textured 3D model reconstructed after appearance acquisition (28 pictures).


Inter-university House of Human Sciences (MISHA), Strasbourg

MISHA has at its disposal a huge collection of ethnographic artefacts, and has launched some years ago a project based on the OpenSIM game engine that provides a virtual world to make it possible the study and the contextualization of these artefacts. Obviously, this requires to have digital copies of them, and so to have digitized them beforehand.

For this purpose, they contacted us in autumn 2013. They needed to produce digital copies of about ten african ethnographic objects from the Dogon tribe. Their geometry has been digitized with the short range structured light scanner, and their colour thanks to a photographic campaign.

Digital copies of three Dogon masks. From left: adone mask (antelope), kanaga mask, bird mask with a dege at the top of it.


Digital copy of a Hogon cup. Left-hand side: the cup fully assembled; right-hand side: each single piece presented separately.


Digital copies of two dege (female figurines).


Bois l'Abbé fortifications

Below are presented some renderings of a digitization we made of a part of the Bois l'Abbé fortifications (48°12'16.3"N 6°24'00.8"E), located at Uxegney near Epinal, France. The final model is made of about 63M points acquired with the Leica Scanstation 2 scanner from 20 different locations. Colour assigned to the points comes from the internal scanner camera, dedicated to data handling conveniency but not to a faithful appearance acquisition.

  • FlortBLAScanPositions.png
  • FortBLAOverview2.png
  • FortBLAOverview3.png
  • FortBLACloseup1.png

Gypsothèque, university of Strasbourg

We had the occasion to get in touch with the Gypsothèque of the Strasbourg university (plaster copy museum of cultural artefacts) to perform the digitization of one of their statues.

Digitized model of a statue of Aphrodite, Gypsothèque of Strasbourg.


Some digitizations made at home

Here are some models digitized in our lab in order to produce data sets for illustrating the research works of our team. Some of these 3D models are provided with light field textures, that enable to simulate illumination variations related to the observer's displacement for more realism, as explained before. It has to be noted that basic colour textures can also be exported using standard image file formats.

  • Elephant
    Geometry: 4.9M triangles
    Texture: light field + colour

  • Dragon
    Geometry: 18.2M triangles
    Texture: light field + colour

  • Mask1
    Geometry: 7.7M triangles
    Texture: light field + colour

  • Mask2
    Geometry: 8.7M triangles
    Texture: colour

  • Venus at the bath
    Geometry: 3.6M triangles
    No texture

Collaborations

Several national and regional projects are supporting this framework:

  • Projet Interreg EVEIL3D,
  • ANR ATROCO,
  • Ministère RIAM AMI3D,
  • Région Pôle Image.

Contacts

If you have needs either for service provisions or for technology transfer, or if you are just interested by one of the models presented in this page, please contact Frédéric Larue, research engineer in charge of the ExRealis framework.



Retrieved from "https://igg.icube.unistra.fr/en/index.php?title=ExRealis&oldid=3725"