Depth sensors and volumetric video meet deep learning

9^th International Conference on Big Data Analysis and Data Mining & 7^th International Conference on 3D Printing Technology and Innovations

August 03, 2022 | JOINT WEBINAR

Rafael Diniz

University of Brasilia, Brazil

Scientific Tracks Abstracts: J Comput Sci Syst Biol

Abstract :

The new visual immersive media formats provide a 3D visual representation of real objects and scenes. In this new visual format, objects can be captured, compressed, transmitted and visualized in real-time not anymore as a flat 2D image, but as 3D content. This new dimension of image capture is easily possible given the introduction of affordable yet powerful depth (or range) sensors. One of the most popular formats for immersive media is Point Cloud (PC), which is composed by points with 3 geometry coordinates plus color information, and sometimes, other information like reflectance and transparency. The proposed work considers the introduction of novel PC feature extractors [Figure 1] which are obtained by using both color and geometry information of a given scene or selected volumetric element of a larger object or scene. Initially, these feature descriptors were design for the purpose of objective quality assessment of volumetric images, but they can also be used to provide valuable information of the volumetric content. The use of PCs as input for machine learning algorithms is showing potential in providing greater accuracy for many types of machine learning algorithms for computer vision. The proposed descriptors are based on local-neighborhood luminance and surface geometry distances among the volumetric elements (voxels). The joint use of features based on color texture and geometry information is also proposed, and presents a better correlation to the Human Vision System (HVS) when such features are used as quality estimation.

Biography :

Rafael Diniz is Computer Scientist (UNICAMP) and holds masters in Informatics from PUC-Rio and PhD from University of Brasilia. Has experience in the area of digital TV and radio (broadcasting), hypermedia and multimedia systems, computer vision, quality assessment of 2D and volumetric video and electromagnetic spectrum management. He is a Member of the Telemidia Lab at PUC-Rio (since 2013) and the GPDS (Digital Signal Processing Group) at University of Brasilia (since 2016).