A system that converts objects in the photo into a 3D model, developed by research teams such as SNAP and others (ITMEDIA NEWS) --Yahoo! News

delivery

(Photo: ITMEDIA NEWS)

　"Neroic: Neroic: Neroal Objecture Capture and Rendering from ONLINE IMAGE COLLECTION □" developed by the U.S. SNAP research team developing the South California University (USC) and the photo sharing app "Snapchat". If you enter an online image of the object taken under conditions, it is an approach that extracts the object in the image and converts it to a 3D model. [Image] (Left) Multiple images input, (right) Output result This method uses an image group that shows the same object published online, and rebuilds the 3D model of the object at a high level of loyalty. Propose a system to do. Usually, online images are photographed under different conditions such as lighting and camera direction, so it is difficult to simply combine them to accurately reproduction. The developed system estimates the geometry of the object and the surrounding lighting, and renders the 3D model of the object under appropriate lighting conditions. Not only can a new view can be synthesized, but in a new environment and lighting conditions, it is possible to re -light the photographed objects and harmonize with the surroundings. The network consists of a major two -step module, a geometry network and a rendering network. In the first stage, input a new viewpoint view from multiple different perspectives input using a sparse image and a primary view mask that defines the object, and Radiante Field is Radiante Field. Calculates and estimates the surface and geometry on the surface of the object. In the second stage, use the estimated normal and geometry to estimate the lighting condition (represented as a spherical harmony coefficient), the surface material characteristics (using a PHONG rendering model), and the high -quality surface. In order to evaluate this system, we created several real environmental object datasets, such as images taken in various environments and images of objects collected from online resources. In such difficult settings, as a result of comparing with state -of -the -art alternative methods, this approach shows that this approach maintains a qualitative and quantitative and quantitative, and maintains the same learning efficiency and estimation efficiency. rice field. Source and Image Credits: Kuang, Zhengfei, Kyle Olszewski, Menglei Chai, Zeng Huang, Panos Achlioptas, and Sergey Tulyakov. "NeROIC: Neural Rendering of Objects from Online Image Collections." arXiv preprint arXiv:2201.02533（2022）.　※テクノロジーのMr. Yuki Yamashita, who presides over the latest research, "SEAMLESS", wrote. Mr. Yamashita picks up and explains a highly new nature paper.

ITMEDIA NEWS