Team IGG : Computer Graphics and Geometry

Difference between revisions of "ExRealis"

From Team IGG : Computer Graphics and Geometry
Jump to navigation Jump to search
m
Line 2: Line 2:
 
__NOEDITSECTION__
 
__NOEDITSECTION__
  
−
= IGG's digitization platform =
 
  
−
The goal of this platform is to offer a set of techniques (hardware and software) enabling to acquire 3D objects with visual appearance or complex materials for applications in computer-generated images and virtual reality. A broad spectrum of data can be acquired, including:
 
−
* shape of objects ''i.e.'' their geometry,
 
−
* appearance of objects in a given lighting environment, or the behavior of complex materials,
 
−
* complex movements of objects or humans, including their deformation with respect to time.
 
  
  
 +
= ''ExRealis'' - A digitization framework =
  
−
== Shape and appearance acquisition ==
+
In many sectors, including industry, one can notice an increasing demand of digital content, either for the enrichment of virtual worlds, or for simulations, or for prototyping before deployment, for instance. Nowadays, many off-the-shelf 3D scanners are available, whose metrological accuracy now makes it possible the reliable measurement of real objects and the capture of faithful digital imprints. This partially leads to turn the digital content production task, so far exclusively allotted to computer graphics artists, into a technical act of data gathering, often cheaper.
  
−
Instead of having a person who creates a polygonal model of an existing object, we wish to be able to directly acquire this model from reality. Nowadays there are many existing techniques and equipments (e.g. 3D scanners) allowing to digitize. But these techniques often have strong limitations: their use requires a high degree of expertise, several steps require tedious manual interventions, the resulting models are huge and impossible to render in real-time directly on the GPU, etc.
+
Despite the improvement of shape measurement technologies, the step to go from measured data to digital content directly usable by computer graphics applications still requires many intermediate processings. Moreover, for several years, the IGG team applies itself to implement its skills in the art for the cultural heritage, a field where the sole shape measurement is not enough. Indeed, capturing the appearance is then of major importance too, since only this information gives the possibility to visualize the produced digital surrogates by simulating in a faithful manner the behaviour of materials they are made of with respect to viewing and lighting conditions. But acquiring such an information requires in turn additionnal processings.
  
−
IGG's digitization platform provides resources to address the three following aspects:
+
The goal of ''ExRealis'' is to provide a unified framework that answers in an efficient manner to all of these needs. It offers several tools, devices and softwares that give the possibility to produce digital content of various nature from real data, with the purpose of supporting research works conducted by the IGG team around themes like realistic rendering, texture synthesis, virtual reality and 3D animation, as well as enabling service provisions in digitization with our partners. ''ExRealis'' proposes:
−
* acquisition of models with visual appearance from dedicated devices,
+
* a range of devices and software tools covering the whole digitization processing pipeline, from data acquisition to the creation of textured 3D models;
−
* automatic (or semi-automatic) treatments of models (registration, filtering, fusion, reconstruction, simplification and representation tuned for the application),
+
* some advanced methods for texture reconstruction and visualization accounting for complex lighting environments or material characteristics;
−
* specific rendering tools on the GPU.  
+
* a motion capture system for the animation of avatars in virtual reality or biomecanic applications.
  
−
<div id="SamplePictures">''Examples of digitization performed using the platform''</div>
 
  
−
Below, a few renders derived from the partial digitization of the ''Fort de Bois l'Abbé'' [http://maps.google.com/?ie=UTF8&t=h&ll=48.204577,6.40107&spn=0.007022,0.016512&z=17 (48°12'16.3"N&nbsp;6°24'00.8"E)], located at Uxegney near Epinal. The final dataset contains circa 63 millions points, acquired using a Leica Scanstation 2 laser scanner from 20 different locations on-site. Coloring of data is derived from the scanner's internal camera dedicated to positioning and control, not for precise appearance acquisition.
+
 
−
<gallery widths=300px>
+
 
−
Image:FlortBLAScanPositions.png
+
= Our devices =
−
Image:Fort.png
+
 
−
Image:FortBLAOverview1.png
+
The ''ExRealis'' framework contains several devices, detailed below, that enable the measurement of different kinds of information (geometry, colour, movement) from real scenes or real objects.
−
Image:FortBLAOverview2.png
+
 
−
Image:FortBLAOverview3.png
+
<gallery widths=250px>
−
Image:FortBLACloseup1.png
+
Image:ScannerLumStruct.jpg|Short range scanner
−
Image:FortBLACloseup2.png
+
Image:laser_leica.png|Medium range scanner
 +
Image:PhotoHardware.jpg|Photographic equipment
 +
Image:Cameramontage2.gif|Motion capture system
 
</gallery>
 
</gallery>
  
−
Two examples of Ethnological objects digitized with color texture out of the MISHA Strasbourg Collection.
 
−
The first model was obtained from 63 scans, 12M points, color was obtained from 23 out of 183 pictures. The rendering
 
−
uses 11.6M triangles reduced to 1%.
 
−
The second model was obtained from 56 scans, 11.1M points, 260 pictures, 7M triangles reduced to 1%
 
  
−
[[Image:Dege275.png|x200px]]
+
'''Short range scanner'''
 +
 
 +
This optical device based on structured light allows the 3D scanning of objects whose size ranges from 20cm to 1m. Depth measurement ranges from 80 to 120cm and the size of the produced range images is of 1280x960 pixels, which leads to an average resolution of 600 to 700 points per square cm<!-- précision de la mesure?? -->.<BR>
 +
The controling software has entirely been developped by our team, which thus offers a high degree of flexibility and the possibility of a complete reconfiguration, so as to adjust, for instance, the device resolution to specific needs.
 +
 
 +
 
 +
'''Medium range scanner'''
 +
 
 +
The time of flight laser scanner ''ScanStation2'' from Leica Geosystems allows depth measurement ranging from 0.2 to 300m, at a speed rate peaking to 50 000 points/sec. Moreover, a set of targets provided by the manufacturer enables the registration of point clouds acquired from different locations.<BR>
 +
Please see the [http://www.leica-geosystems.fr/fr/Scanner-Laser-3D-Leica-ScanStation-2_62189.htm technical documentation] from Leica Geosystems for more details.
 +
 
 +
 
 +
'''Photographic equipment'''
 +
 
 +
Our team also has at its disposal a semi-professional photographic equipment made of a pair of ''Canon EOS 5D MkII'' bodies, as well as two sets of 24mm, 50mm et 135mm lenses.<BR>
 +
This equipement is mainly dedicated to the acquisition of object appearance (colour, texture), but the duplication of every device piece also makes it possible, if needed, to capture pairs of stereoscopic images.
 +
 
 +
 
 +
'''Motion capture system'''
 +
 
 +
This optical system, made of 12 ''Vicon T40'' and ''T40S'' infrared cameras (4 Mpixels, 370 Hz), allows the motion capture of objects or subjects by the measurement of infrared markers trajectories in space. These trajectories may then be used, for instance, for skeleton animation of 3D characters.
 +
<!-- Ce matériel a été acquis en grande partie grâce au CPER IRMC. -->
 +
 
 +
 
 +
 
  
−
[[Image:Kanaga.png|x200px]]
+
= Our software tools =
  
−
An example of digitized object with reconstructed surface lightfield, i.e. the color on each surface point depends on the observer's position.
+
The aforementioned devices provide only raw data. In order to produce usable digital content, these data must be processed by our software. With this in mind, we manage a continuous development process in order to integrate every tool which is useful to the creation of realistic digital models from physical real objects. These developments, based on out-of-core data structures and mechanisms for big data management, allows geometry processing (3D meshes reconstruction from digitized point clouds) as well as appearance processing (color / texture synthesis over meshes from sets of pictures).
−
<gallery widths=300px>
+
 
−
Image:gong.png
+
<!-- Le logiciel développé dans le cadre de la plateforme ''ExRealis'' intègre différentes solutions pour répondre aux problématiques liées à la numérisation. Certaines de ces solutions ont été puisées dans la littérature scientifiques, alors que d'autres proviennent directement des résultats de travaux de recherche menés par l'équipe IGG.
−
Image:gongall.png
+
 
−
</gallery>
+
essayant autant que faire se peut d'automatiser
  
−
An example of painting. On the left, a photo of the painting; on the center, a rendering of the same model from the same viewpoint using only classical texture mapping; on the right, a rendering using a bidirectional texture function in which model geometry and bidirectional reflectance have been isolated.
+
Beaucoup d'interventions manuelles sont nécessaires pour passer des données brutes à un modèle géométrique complet de l'objet. Nous développons des outils permettant d'automatiser (ou de rendre plus accessible) ce travail. -->
  
−
[[Image:tableaunum_lsiit.png|500px]]
 
  
 +
'''Geometry acquisition and processing'''
  
−
= Equipments =
+
After a 3D scanning session (namely the data gathering by itself), many processings have to be applied to the raw data produced by the scanner before obtaining a usable 3D model. These processings generally include, among other things, the following steps:
 +
* the ''cleaning'', that aims at removing possible measurement errors (outliers, digitization noise, ''etc.'');
 +
* the ''geometry registration'', that performs the alignment of point clouds recovered by the scanner from different viewpoints with respect to each other;
 +
* the ''integration'', thanks to which a meshed surface is obtained from measured 3D points;
 +
* the ''decimation'', that provides different versions of the recovered mesh, at multiple resolutions, depending on the targeted application.
 +
[[Image:PipelineGeo_EN.png|900px]]
  
−
'''Optical scanner by projection of fringes and phase shift measuring'''.
+
Our software covers every step of this processing pipeline, thanks either to fully automatic algorithmic solutions, or to a suitable user interface whenever user intervention is needed, like in the case of some of the cleaning tasks, for instance. We are then in position to produce, from real objects, 3D models that are correct in terms of geometry and topology, at different level of details, and that can be exported to many file formats compatible with off-the-shelf CAD softwares.
  
−
[[Image:ScannerLumStruct.jpg|right|thumb|200px]]
 
  
−
This device allows the 3D acquisition of objects of a size between approximately 10 and 50cm. An external video camera allows to perform an automatic registration of several exposures and also the registration of information concerning the RGB colored directional appearance. A specific software has been developed in order to perform acquisitions and then on-screen display.
+
'''Appearance acquisition and processing'''
  
 +
Acquiring the appearance of a real object starts from a photographic campaign, so as to measure its photometric properties. Reconstructing a texture from these pictures then requires to solve the following problems:
 +
* the mesh corresponding to the object, recovered thanks to a prior 3D scanning session, must be first unfolded in texture space thanks to a ''parameterization'' step, which defines the layout to be used for the storage of the appearance information to be reconstructed;
 +
* pictures then must be registered with respect to the object geometry, thanks to an estimation of their poses and optical parameters. These parameters allows the reprojection of the image contents onto the 3D mesh;
 +
* finally, chromatic samples coming from the pictures have to be processed and combined in order to produce a texture that is free from the visual artefacts commonly due to chromatic aberrations or to inaccuracies in the picture-to-geometry registration.
 +
[[Image:PipelineAppearance_EN.png|640px]]
 +
<!-- [[Image:Recalage.png|250px]] [[Image:Debruitage.png|120px]] [[Image:Simplification.png|250px]] -->
  
−
'''Goniometer'''.
+
Produced textures may consist either in a simple colour information, or in more advanced representation models that give not only a clue on the object's hue, but also on other properties of its material, like shininess, for instance.
  
−
[[Image:goniometre_lsiit.png|left|thumb|200px]]
+
The IGG team has worked on the fitting and the real-time rendering of such models, considering especially ''light fields''. Light fields capture the appearance of an object within a given lighting environment: the one corresponding to the acquisition moment. This includes, among other things, all illumination effets related to the observer's displacement around the object (specular peaks, inter-reflections, ''etc.'') and allows afterwards the free inspection of the digital copy with the same lighting conditions, while maintaining a high degree or realism with respect to more basic texture models.
  
−
This device allows the positioning of a light source on a hemispheric dome in order to vary the lighting conditions of a 3D object or a material sample (whose size is between 10 and 20cm). This device, has been used for the acquisition of paintings using bidirectional texture function.  
+
All of these software bricks, needed for appearance acquisition, have been integrated to ''ExRealis'', making it a powerful tool for the reconstruction of textured 3D meshes from real data.
 +
[[Image:Elephant2.png|420px|thumb|left|Comparison between colour texture and ''light field''. Reflections improve realism and offer a better understanding on the nature of the materials the object is made of.]]
 
<br style="clear: both" />
 
<br style="clear: both" />
  
  
−
'''Mid-range laser scanner'''.
 
  
−
[[Image:laser_leica.png|right|thumb|120px]]
 
  
−
This device is a ScanStation2 laser scanner from Leica Geosystems. It allows to perform scans on a range of 0.2-300m with a speed of up to 50000 points/sec. A target set also allows to perform registration. See also the [http://www.leica-geosystems.fr/fr/Scanner-Laser-3D-Leica-ScanStation-2_62189.htm data sheet] of Leica Geosystems (french).
+
= Our skills =
 +
 
 +
After 10 years working on ''3D + appearance'' digitization, the IGG team has acquired a good expertise of the field and a savoir-faire on how to apply it to the practical cases of digitization campaigns. Within the framework of various collaborations, we have had many occasions to apply this savoir-faire through service provisions like those presented below.
  
−
This laser scanner has been used to [[#SamplePictures|digitize part of the Fort de Bois l'Abbé military fortification]] close to the city of Epinal.
+
 
 +
''' ''&OElig;uvre Notre Dame'' foundation'''
 +
 
 +
Within the framework of the [http://www.eveil-3d.eu/francais/index.php Eveil3D] project, and in partnership with the technology transfer center Holo3 and the ''&OElig;uvre Notre Dame'' foundation, the IGG team has performed in october 2013 the digitization of two statues of the cathedral of Strasbourg. The goal was to use these statues in an immersive 3D environment for a language learning serious game. About 50cm high, they've been digitized with our short range structured light scanner. A photographic campaign has also been done so as to produce colour textures for both models.
 +
 
 +
[[Image:OND_Ourson.png|600px|thumb|left|''Bear statue'', cathedral of Strasbourg. From left: picture, 3D model reconstructed after geometry acquisition (35M points), textured 3D model reconstructed after appearance acquisition (55 pictures).]]
 +
<br style="clear: both" />
 +
[[Image:OND_Taureau.png|600px|thumb|left|''Bull statue'', cathedral of Strasbourg. From left: picture, 3D model reconstructed after geometry acquisition (22M points), textured 3D model reconstructed after appearance acquisition (28 pictures).]]
 
<br style="clear: both" />
 
<br style="clear: both" />
  
−
'''Photographic hardware'''
+
 
−
[[Image:PhotoHardware.jpg|left|thumb|120px]]
+
'''Inter-university House of Human Sciences (''MISHA''), Strasbourg'''
−
Prosumer photographic hardware mainly used for appearance capture. Part of the hardware is duplicated (Canon EOS 5D MkII and pairs of 24mm, 50mm, 135mm lenses) in order to allow capturing of stereo pairs.
+
 
 +
''MISHA'' has at its disposal a huge collection of ethnographic artefacts, and has launched some years ago a project based on the OpenSIM game engine that provides a virtual world to make it possible the study and the contextualization of these artefacts. Obviously, this requires to have digital copies of them, and so to have digitized them beforehand.
 +
 
 +
For this purpose, they contacted us in autumn 2013. They needed to produce digital copies of about ten african ethnographic objects from the Dogon tribe. Their geometry has been digitized with the short range structured light scanner, and their colour thanks to a photographic campaign.
 +
 
 +
[[Image:MISHA_Masks.png|600px|thumb|left|Digital copies of three Dogon masks. From left: ''adone'' mask (antelope), ''kanaga'' mask, bird mask with a ''dege'' at the top of it.]]
 +
<br style="clear: both" />
 +
[[Image:MISHA_Hogon.png|400px|thumb|left|Digital copy of a ''Hogon'' cup. Left-hand side: the cup fully assembled; right-hand side: each single piece presented separately.]]
 +
<br style="clear: both" />
 +
[[Image:MISHA_Dege.png|300px|thumb|left|Digital copies of two ''dege'' (female figurines).]]
 
<br style="clear: both" />
 
<br style="clear: both" />
  
  
−
'''Motion capture system'''
+
''' ''Bois l'Abbé'' fortifications'''
−
This hardware is designed to capture movements of objects or humans. The system works using hi-precision optical capture of passive / reflective markers affixed to the capture subject. It comprises 12 IR Vicon T40 or T40S cameras, 4Mpixels at 370 Hz. Trajectories are the reconstructed and can be used to animate skeletons (using the Vicon Blade software). Its purchase has largely been supported by the CPER IRMC program.
+
 
 +
Below are presented some renderings of a digitization we made of a part of the ''Bois l'Abbé'' fortifications [http://maps.google.com/?ie=UTF8&t=h&ll=48.204577,6.40107&spn=0.007022,0.016512&z=17 (48°12'16.3"N&nbsp;6°24'00.8"E)], located at Uxegney near Epinal, France. The final model is made of about 63M points acquired with the Leica Scanstation 2 scanner from 20 different locations. Colour assigned to the points comes from the internal scanner camera, dedicated to data handling conveniency but not to a faithful appearance acquisition.
 +
<gallery widths=250px>
 +
Image:FlortBLAScanPositions.png
 +
<!--Image:Fort.png
 +
Image:FortBLAOverview1.png-->
 +
Image:FortBLAOverview2.png
 +
Image:FortBLAOverview3.png
 +
Image:FortBLACloseup1.png
 +
<!--Image:FortBLACloseup2.png-->
 +
</gallery>
 +
 
 +
<!-- Ci-dessous, un exemple de numérisation de tableau. L'image de gauche montre une photographie du tableau, l'image du milieu un rendu en synthèse d'images pour le même point de vue en utilisant un placage de texture de couleur classique et l'image de droite en utilisant une fonction bidirectionnelle de texture de laquelle le relief a été découplé de la réflectance bidirectionnelle.
 +
 
 +
[[Image:tableaunum_lsiit.png|600px]]
 +
-->
 +
 
 +
 
 +
''' ''Gypsothèque'', university of Strasbourg'''
 +
 
 +
We had the occasion to get in touch with the ''Gypsothèque'' of the Strasbourg university (plaster copy museum of cultural artefacts) to perform the digitization of one of their statues.
 +
 
 +
[[Image:aphro_render.png|400px|thumb|left|Digitized model of a statue of Aphrodite, ''Gypsothèque'' of Strasbourg.]]
 
<br style="clear: both" />
 
<br style="clear: both" />
  
  
−
= Software =
+
'''Some digitizations made at home'''
 +
 
 +
Here are some models digitized in our lab in order to produce data sets for illustrating the research works of our team. Some of these 3D models are provided with ''light field'' textures, that enable to simulate illumination variations related to the observer's displacement for more realism, as explained before. It has to be noted that basic colour textures can also be exported using standard image file formats.
 +
 
 +
<gallery widths=200px>
 +
Image:NUM_Elephant.png|''Elephant''<BR>Geometry: 4.9M triangles<BR>Texture: light field + colour
 +
Image:NUM_Dragon.png|''Dragon''<BR>Geometry: 18.2M triangles<BR>Texture: light field + colour
 +
Image:NUM_Mask1.png|''Mask1''<BR>Geometry: 7.7M triangles<BR>Texture: light field + colour
 +
Image:NUM_Mask2.png|''Mask2''<BR>Geometry: 8.7M triangles<BR>Texture: colour
 +
Image:NUM_Venus.png|''Venus at the bath''<BR>Geometry: 3.6M triangles<BR>No texture
 +
</gallery>
 +
 
  
−
Software development has been carried in order to gather and integrate all the required processing useful for creating realistic digital models from real objects. The resulting software includes:
 
−
* the processing of the acquired geometry to produce useful meshes;
 
−
* the use of camera photos to build a lightfield representation of the objects' appearance.
 
−
[[Image:DigitizationPipelineEn.png|650px]]
 
  
−
'''Registration, Denoising, Integration, Simplification'''.
 
  
−
Many manual user interventions are necessary to obtain a complete 3D polygonal model from the brute scanned data (point clouds). We aim at developing automatic tools (or at least tools making the job less tedious).
+
= Collaborations =
  
−
Example of registration, denoising and simplification:
+
Several national and regional projects are supporting this framework:
−
+
* Projet Interreg EVEIL3D,
−
[[Image:Recalage.png|250px]] [[Image:Debruitage.png|120px]] [[Image:Simplification.png|250px]]
+
* [http://artis.imag.fr/Projets/ATROCO/ ANR ATROCO],
 +
* Ministère RIAM AMI3D,
 +
* Région Pôle Image.
  
−
'''Texture and Apparence'''.
 
  
−
The acquisition of appearance is a difficult problem. Many photographs must be taken and registered with the geometric model. We aim at developing tools to ease this work and to make models compatible with modern graphics boards for visualization.
 
  
−
<gallery widths=300px>
 
−
Image:aphro_render.png|Rendering of a scanned object
 
−
Image:acqu_texture.png|Example of acquisition and registration of texture
 
−
</gallery>
 
−
<br style="clear: both" />
 
  
−
A mesh example with appearence reconstruction. [[Image:mesh_texture.png|x200px]]
+
= Contacts =
 +
 
 +
If you have needs either for service provisions or for technology transfer, or if you are just interested by one of the models presented in this page, please send an email to Frédéric Larue, research engineer in charge of the ''ExRealis'' framework, at: <TT>flarue AT unistra.fr</TT>
 +
 
 +
<BR>
 +
<BR>
 +
 
 +
 
 +
 
  
−
[[fr:Numérisation]]
+
[[en:Digitization]]

Revision as of 14:07, 27 August 2015




ExRealis - A digitization framework

In many sectors, including industry, one can notice an increasing demand of digital content, either for the enrichment of virtual worlds, or for simulations, or for prototyping before deployment, for instance. Nowadays, many off-the-shelf 3D scanners are available, whose metrological accuracy now makes it possible the reliable measurement of real objects and the capture of faithful digital imprints. This partially leads to turn the digital content production task, so far exclusively allotted to computer graphics artists, into a technical act of data gathering, often cheaper.

Despite the improvement of shape measurement technologies, the step to go from measured data to digital content directly usable by computer graphics applications still requires many intermediate processings. Moreover, for several years, the IGG team applies itself to implement its skills in the art for the cultural heritage, a field where the sole shape measurement is not enough. Indeed, capturing the appearance is then of major importance too, since only this information gives the possibility to visualize the produced digital surrogates by simulating in a faithful manner the behaviour of materials they are made of with respect to viewing and lighting conditions. But acquiring such an information requires in turn additionnal processings.

The goal of ExRealis is to provide a unified framework that answers in an efficient manner to all of these needs. It offers several tools, devices and softwares that give the possibility to produce digital content of various nature from real data, with the purpose of supporting research works conducted by the IGG team around themes like realistic rendering, texture synthesis, virtual reality and 3D animation, as well as enabling service provisions in digitization with our partners. ExRealis proposes:

  • a range of devices and software tools covering the whole digitization processing pipeline, from data acquisition to the creation of textured 3D models;
  • some advanced methods for texture reconstruction and visualization accounting for complex lighting environments or material characteristics;
  • a motion capture system for the animation of avatars in virtual reality or biomecanic applications.



Our devices

The ExRealis framework contains several devices, detailed below, that enable the measurement of different kinds of information (geometry, colour, movement) from real scenes or real objects.

  • Short range scanner

  • Medium range scanner

  • Photographic equipment

  • Motion capture system


Short range scanner

This optical device based on structured light allows the 3D scanning of objects whose size ranges from 20cm to 1m. Depth measurement ranges from 80 to 120cm and the size of the produced range images is of 1280x960 pixels, which leads to an average resolution of 600 to 700 points per square cm.
The controling software has entirely been developped by our team, which thus offers a high degree of flexibility and the possibility of a complete reconfiguration, so as to adjust, for instance, the device resolution to specific needs.


Medium range scanner

The time of flight laser scanner ScanStation2 from Leica Geosystems allows depth measurement ranging from 0.2 to 300m, at a speed rate peaking to 50 000 points/sec. Moreover, a set of targets provided by the manufacturer enables the registration of point clouds acquired from different locations.
Please see the technical documentation from Leica Geosystems for more details.


Photographic equipment

Our team also has at its disposal a semi-professional photographic equipment made of a pair of Canon EOS 5D MkII bodies, as well as two sets of 24mm, 50mm et 135mm lenses.
This equipement is mainly dedicated to the acquisition of object appearance (colour, texture), but the duplication of every device piece also makes it possible, if needed, to capture pairs of stereoscopic images.


Motion capture system

This optical system, made of 12 Vicon T40 and T40S infrared cameras (4 Mpixels, 370 Hz), allows the motion capture of objects or subjects by the measurement of infrared markers trajectories in space. These trajectories may then be used, for instance, for skeleton animation of 3D characters.



Our software tools

The aforementioned devices provide only raw data. In order to produce usable digital content, these data must be processed by our software. With this in mind, we manage a continuous development process in order to integrate every tool which is useful to the creation of realistic digital models from physical real objects. These developments, based on out-of-core data structures and mechanisms for big data management, allows geometry processing (3D meshes reconstruction from digitized point clouds) as well as appearance processing (color / texture synthesis over meshes from sets of pictures).


Geometry acquisition and processing

After a 3D scanning session (namely the data gathering by itself), many processings have to be applied to the raw data produced by the scanner before obtaining a usable 3D model. These processings generally include, among other things, the following steps:

  • the cleaning, that aims at removing possible measurement errors (outliers, digitization noise, etc.);
  • the geometry registration, that performs the alignment of point clouds recovered by the scanner from different viewpoints with respect to each other;
  • the integration, thanks to which a meshed surface is obtained from measured 3D points;
  • the decimation, that provides different versions of the recovered mesh, at multiple resolutions, depending on the targeted application.

PipelineGeo EN.png

Our software covers every step of this processing pipeline, thanks either to fully automatic algorithmic solutions, or to a suitable user interface whenever user intervention is needed, like in the case of some of the cleaning tasks, for instance. We are then in position to produce, from real objects, 3D models that are correct in terms of geometry and topology, at different level of details, and that can be exported to many file formats compatible with off-the-shelf CAD softwares.


Appearance acquisition and processing

Acquiring the appearance of a real object starts from a photographic campaign, so as to measure its photometric properties. Reconstructing a texture from these pictures then requires to solve the following problems:

  • the mesh corresponding to the object, recovered thanks to a prior 3D scanning session, must be first unfolded in texture space thanks to a parameterization step, which defines the layout to be used for the storage of the appearance information to be reconstructed;
  • pictures then must be registered with respect to the object geometry, thanks to an estimation of their poses and optical parameters. These parameters allows the reprojection of the image contents onto the 3D mesh;
  • finally, chromatic samples coming from the pictures have to be processed and combined in order to produce a texture that is free from the visual artefacts commonly due to chromatic aberrations or to inaccuracies in the picture-to-geometry registration.

PipelineAppearance EN.png

Produced textures may consist either in a simple colour information, or in more advanced representation models that give not only a clue on the object's hue, but also on other properties of its material, like shininess, for instance.

The IGG team has worked on the fitting and the real-time rendering of such models, considering especially light fields. Light fields capture the appearance of an object within a given lighting environment: the one corresponding to the acquisition moment. This includes, among other things, all illumination effets related to the observer's displacement around the object (specular peaks, inter-reflections, etc.) and allows afterwards the free inspection of the digital copy with the same lighting conditions, while maintaining a high degree or realism with respect to more basic texture models.

All of these software bricks, needed for appearance acquisition, have been integrated to ExRealis, making it a powerful tool for the reconstruction of textured 3D meshes from real data.

Comparison between colour texture and light field. Reflections improve realism and offer a better understanding on the nature of the materials the object is made of.




Our skills

After 10 years working on 3D + appearance digitization, the IGG team has acquired a good expertise of the field and a savoir-faire on how to apply it to the practical cases of digitization campaigns. Within the framework of various collaborations, we have had many occasions to apply this savoir-faire through service provisions like those presented below.


Œuvre Notre Dame foundation

Within the framework of the Eveil3D project, and in partnership with the technology transfer center Holo3 and the Œuvre Notre Dame foundation, the IGG team has performed in october 2013 the digitization of two statues of the cathedral of Strasbourg. The goal was to use these statues in an immersive 3D environment for a language learning serious game. About 50cm high, they've been digitized with our short range structured light scanner. A photographic campaign has also been done so as to produce colour textures for both models.

Bear statue, cathedral of Strasbourg. From left: picture, 3D model reconstructed after geometry acquisition (35M points), textured 3D model reconstructed after appearance acquisition (55 pictures).


Bull statue, cathedral of Strasbourg. From left: picture, 3D model reconstructed after geometry acquisition (22M points), textured 3D model reconstructed after appearance acquisition (28 pictures).



Inter-university House of Human Sciences (MISHA), Strasbourg

MISHA has at its disposal a huge collection of ethnographic artefacts, and has launched some years ago a project based on the OpenSIM game engine that provides a virtual world to make it possible the study and the contextualization of these artefacts. Obviously, this requires to have digital copies of them, and so to have digitized them beforehand.

For this purpose, they contacted us in autumn 2013. They needed to produce digital copies of about ten african ethnographic objects from the Dogon tribe. Their geometry has been digitized with the short range structured light scanner, and their colour thanks to a photographic campaign.

Digital copies of three Dogon masks. From left: adone mask (antelope), kanaga mask, bird mask with a dege at the top of it.


Digital copy of a Hogon cup. Left-hand side: the cup fully assembled; right-hand side: each single piece presented separately.


Digital copies of two dege (female figurines).



Bois l'Abbé fortifications

Below are presented some renderings of a digitization we made of a part of the Bois l'Abbé fortifications (48°12'16.3"N 6°24'00.8"E), located at Uxegney near Epinal, France. The final model is made of about 63M points acquired with the Leica Scanstation 2 scanner from 20 different locations. Colour assigned to the points comes from the internal scanner camera, dedicated to data handling conveniency but not to a faithful appearance acquisition.

  • FlortBLAScanPositions.png
  • FortBLAOverview2.png
  • FortBLAOverview3.png
  • FortBLACloseup1.png


Gypsothèque, university of Strasbourg

We had the occasion to get in touch with the Gypsothèque of the Strasbourg university (plaster copy museum of cultural artefacts) to perform the digitization of one of their statues.

Digitized model of a statue of Aphrodite, Gypsothèque of Strasbourg.



Some digitizations made at home

Here are some models digitized in our lab in order to produce data sets for illustrating the research works of our team. Some of these 3D models are provided with light field textures, that enable to simulate illumination variations related to the observer's displacement for more realism, as explained before. It has to be noted that basic colour textures can also be exported using standard image file formats.

  • Elephant
    Geometry: 4.9M triangles
    Texture: light field + colour

  • Dragon
    Geometry: 18.2M triangles
    Texture: light field + colour

  • Mask1
    Geometry: 7.7M triangles
    Texture: light field + colour

  • Mask2
    Geometry: 8.7M triangles
    Texture: colour

  • Venus at the bath
    Geometry: 3.6M triangles
    No texture



Collaborations

Several national and regional projects are supporting this framework:

  • Projet Interreg EVEIL3D,
  • ANR ATROCO,
  • Ministère RIAM AMI3D,
  • Région Pôle Image.



Contacts

If you have needs either for service provisions or for technology transfer, or if you are just interested by one of the models presented in this page, please send an email to Frédéric Larue, research engineer in charge of the ExRealis framework, at: flarue AT unistra.fr



Retrieved from "https://igg.icube.unistra.fr/en/index.php?title=ExRealis&oldid=2590"