AI Can Render 3D Hair in Real Time

Researchers from the University of Southern California, Pinscreen, and Microsoft developed a deep learning-based method that can generate full 3D hair geometry from single-view images in real time. This is the first deep learning project that can render hair in real time, the team said.
“Realistic hair modeling is one of the most difficult tasks when digitizing virtual humans,” the researchers said. “In contrast to objects that are easily parameterizable, like the human face, hair spans a wide range of shape variations and can be highly complex due to its volumetric structure and level of deformability in each strand.”

Using NVIDIA TITAN Xp GPUs with the cuDNN-accelerated PyTorch deep learning framework, the researchers trained their convolutional neural network on a dataset comprised of over 40,000 different hairstyles and 160,000 corresponding 2D orientation images rendered from random views.
The neural network pipeline contains three steps, pre-processing, hair strand generation, and reconstruction.
“A preprocessing step is first adopted to calculate the 2D orientation field of the hair region based on the automatically estimated hair mask. Then, HairNet takes the 2D orientation fields as input and generates hair strands represented as sequences of 3D points. A reconstruction step is finally performed to efficiently generate a smooth and dense hair model,” the researchers said.
When compared to similar systems that aim to do that same, the method delivers more details and better-looking results. “The hair from our method can preserve better local details and looks more natural, especially for curly hairs.”
The method can handle different hairstyles, including curly, straight, wavy, and very curly. However, the team concedes their method isn’t perfect.
In future work the researchers say they will focus on amplifying their dataset with more hair types.
The work was recently published on ArXiv.
Read more >