ExRealis

IGG's digitization platform

The goal of this platform is to offer a set of techniques (hardware and software) enabling to acquire 3D objects with visual appearance or complex materials for applications in computer-generated images and virtual reality. A broad spectrum of data can be acquired, including:

• shape of objects i.e. their geometry,
• appearance of objects in a given lighting environment, or the behavior of complex materials,
• complex movements of objects or humans, including their deformation with respect to time.

Shape and appearance acquisition

Instead of having a person who creates a polygonal model of an existing object, we wish to be able to directly acquire this model from reality. Nowadays there are many existing techniques and equipments (e.g. 3D scanners) allowing to digitize. But these techniques often have strong limitations: their use requires a high degree of expertise, several steps require tedious manual interventions, the resulting models are huge and impossible to render in real-time directly on the GPU, etc.

IGG's digitization platform provides resources to address the three following aspects:

• acquisition of models with visual appearance from dedicated devices,
• automatic (or semi-automatic) treatments of models (registration, filtering, fusion, reconstruction, simplification and representation tuned for the application),
• specific rendering tools on the GPU.

Examples of digitization performed using the platform

Below, a few renders derived from the partial digitization of the Fort de Bois l'Abbé (48°12'16.3"N 6°24'00.8"E), located at Uxegney near Epinal. The final dataset contains circa 63 millions points, acquired using a Leica Scanstation 2 laser scanner from 20 different locations on-site. Coloring of data is derived from the scanner's internal camera dedicated to positioning and control, not for precise appearance acquisition.

• 
• 
• 
• 
• 
• 
• 

Two examples of Ethnological objects digitized with color texture out of the MISHA Strasbourg Collection. The first model was obtained from 63 scans, 12M points, color was obtained from 23 out of 183 pictures. The rendering uses 11.6M triangles reduced to 1%. The second model was obtained from 56 scans, 11.1M points, 260 pictures, 7M triangles reduced to 1%

x200px

x200px

An example of digitized object with reconstructed surface lightfield, i.e. the color on each surface point depends on the observer's position.

• Gong.png
• Gongall.png

An example of painting. On the left, a photo of the painting; on the center, a rendering of the same model from the same viewpoint using only classical texture mapping; on the right, a rendering using a bidirectional texture function in which model geometry and bidirectional reflectance have been isolated.

Equipments

Optical scanner by projection of fringes and phase shift measuring.

This device allows the 3D acquisition of objects of a size between approximately 10 and 50cm. An external video camera allows to perform an automatic registration of several exposures and also the registration of information concerning the RGB colored directional appearance. A specific software has been developed in order to perform acquisitions and then on-screen display.

Goniometer.

This device allows the positioning of a light source on a hemispheric dome in order to vary the lighting conditions of a 3D object or a material sample (whose size is between 10 and 20cm). This device, has been used for the acquisition of paintings using bidirectional texture function.

Mid-range laser scanner.

This device is a ScanStation2 laser scanner from Leica Geosystems. It allows to perform scans on a range of 0.2-300m with a speed of up to 50000 points/sec. A target set also allows to perform registration. See also the data sheet of Leica Geosystems (french).

This laser scanner has been used to digitize part of the Fort de Bois l'Abbé military fortification close to the city of Epinal.

Photographic hardware

Prosumer photographic hardware mainly used for appearance capture. Part of the hardware is duplicated (Canon EOS 5D MkII and pairs of 24mm, 50mm, 135mm lenses) in order to allow capturing of stereo pairs.

Motion capture system This hardware is designed to capture movements of objects or humans. The system works using hi-precision optical capture of passive / reflective markers affixed to the capture subject. It comprises 12 IR Vicon T40 or T40S cameras, 4Mpixels at 370 Hz. Trajectories are the reconstructed and can be used to animate skeletons (using the Vicon Blade software). Its purchase has largely been supported by the CPER IRMC program.

Software

Software development has been carried in order to gather and integrate all the required processing useful for creating realistic digital models from real objects. The resulting software includes:

• the processing of the acquired geometry to produce useful meshes;
• the use of camera photos to build a lightfield representation of the objects' appearance.

650px

Registration, Denoising, Integration, Simplification.

Many manual user interventions are necessary to obtain a complete 3D polygonal model from the brute scanned data (point clouds). We aim at developing automatic tools (or at least tools making the job less tedious).

Example of registration, denoising and simplification:

Texture and Apparence.

The acquisition of appearance is a difficult problem. Many photographs must be taken and registered with the geometric model. We aim at developing tools to ease this work and to make models compatible with modern graphics boards for visualization.

• 

  Rendering of a scanned object

• 

  Example of acquisition and registration of texture

A mesh example with appearence reconstruction.