Current topology-based modeling software produces 3D objects in a single level of detail. This makes them inoperable with multiple platforms in the metaverse. In addition, due to the topology creation process, 3D modeling is time consuming and has a high entry barrier for content creation.
NVIDIA Inception Program member and New York-based startup Shapeyard is solving the metaverse 3D content interoperability challenge by automating topology generation in multiple levels of detail at exporting or streaming.
The Shapeyard solution
Shapeyard, available on a variety of mobile devices, is proving to be a powerful and user-friendly 3D modeling tool. It enables developers to create, export, and stream one model into multiple platforms simultaneously–and in multiple levels of detail–with the model topology generated automatically.
Overview of 3D modeling market
Traditionally, the gaming and entertainment industries have used topology-based virtual 3D objects consisting of polygons, enabling developers to render on a computer most efficiently. The more polygons that an object has, the more detailed it appears, but high numbers of polygons demand more computing power. The polygon layout, or topology, of these items must be planned efficiently and created in advance to minimize the number of polygons and generate objects of the highest grade.
Moreover, objects that are further animated (such as characters) maintain their own strict set of topology requirements for the deformation to work efficiently. Topology creation is a critical element in 3D modeling, and requires significant time and effort to master.
Until recently, topology-based modeling has been the only method for creating 3D objects, limiting modeler creativity. Digital sculpting software, however, uses an extremely high-resolution polygon mesh (or a voxel grid). This mesh can be modeled like traditional clay with a vast array of tools, giving modelers more freedom and flexibility in their work.
Unfortunately, 3D objects produced with digital sculpting applications are largely unusable outside the creation software because they are too heavy to process, store, and transfer. This is similar to the limitations of physical sculptures. To enable modelers to work on alternative platforms, much less detailed polygon mesh must be drawn on top of this digital clay. This process is called retopology and represents an industry of its own.
Changing the resolution of a finished mesh (for a mobile game, for example) requires an even less detailed model to run efficiently, forcing the modeler to recreate it from scratch. Most game engines also require multiple levels of asset detailing, which requires the use of multiple assets and leads to additional production and running costs.
In summary, existing 3D modeling tools lack detailing scalability, suffer from operability limitations, and can be difficult to work with. Additionally, sculpting tools are generally inoperable outside their own ecosystems.
Shapeyard’s approach to 3D modeling
Shapeyard reimagines modeling, allowing modelers to produce, auto-export, and stream 3D objects with an infinite level of detail. This makes them compatible with multiple metaverse platforms.
Shapeyard also automates the challenging and time-consuming process of topology creation, alleviating demands on the artist. This 3D modeling technology addresses key requirements for creators, enabling mainstream adoption by offering:
- resolution-free digital clay as a medium
- the ability to execute both free-form modeling and precision modeling
- improved transferability due to lower average master model file sizes (equivalent to an average JPEG file)
- human-grade automatic retopology in any resolution
- smooth performance on an average tablet or smartphone device
Shapeyard’s proprietary engine provides 3D modeling based on signed distance fields, which lowers the barrier to entry for 3D creation and enables creators to produce interoperable assets with multiple platforms.
According to Ashot Gabrelyanov, CEO of Shapeyard development company Magic Unicorn, “As the web evolves, its content form does, too. Our team foresees the Internet fundamentally changing into a series of spatial structures, commonly referred to as the metaverse, with virtual 3D objects lying at its core and USD, OBJ, and FBX becoming the new HTML, JPG, and PNG. Shapeyard is working at the forefront of this shift to make 3D content creation more accessible and to allow creators to capture and magnify the value they create in the metaverse.”
The current version of the product is designed for beginner 3D modelers with mobile devices (iPads and iPhones), as well as casual 3D modelers who prefer the flexibility of a tablet.
The Shapeyard modeling workflow is based on placing primitives, such as Sphere, Box, and Torus, and modifying them by blending, deforming, or smoothing. The solution also enables users to fill objects in with color, paint over them with an RBG brush, or cover regions with physically based rendering (PBR) materials. These interactions are intuitive in the solution’s user interface, resulting in a fast learning curve relative to traditional desktop alternatives.
Shapeyard 3D model construction
As shown in Figure 3, Shapeyard models are constructed with deformable procedural primitives (a), such as cubes, spheres, cones, cylinders, and other shapes, which augment or reduce each other’s volume to produce the continuous surface (b) covering them. Each model starts with a primitive that has a parametric analytical surface (a) and a geometric surface visible to the user (b).
Then the model can be modified by adding, removing, or manipulating primitives within their parameter range.
Automatic retopology and asset exporting from Shapeyard
Shapeyard’s 3D models use procedural analytical surfaces as their main medium, providing an infinite level of detail. The compatibility of the models with other ecosystems is enabled by the models’ automatic conversion into polygons at export.
The procedural analytical surfaces store the entire construction history of the models. In addition to intuitive, nondestructive editing capabilities–including the application of textures–Shapeyard enables automatic human-grade retopology.
Traditional manual retopology is a time-intensive process that requires the identification of basic shapes and flows of form inside a model and filling those areas with polygons (Figure 6).
In comparison, Shapeyard’s retopology algorithm fully automates this process by extracting the primitives’ axes, efficiently converging them, and producing guides for the triangles and quad strips to follow (Figure 7).
The result is near-instant human-grade model retopology with the detail level set by the creator (Figure 8).
NVIDIA Omniverse support and current product vision
At this time, the primary use case for assets produced in Shapeyard is 3D printing. Shapeyard models come as manifold meshes, so they can be perfectly imported without the need for slicing preparation using other software.
Based on the official Shapeyard product road map, which estimates the addition of object grouping and sculpting modifiers in the second quarter of 2023, Shapeyard will expand its capabilities to enable users to offload fully in-game-ready assets in FBX and OBJ. The company recently released a feature that enables export to USDZ, making models created in Shapeyard compatible with NVIDIA Omniverse. To learn more, see the Shapeyard Export to Omniverse in USDZ Guide.
To proliferate 3D modeling and achieve broad adoption, Shapeyard aims to go beyond content creation tools and provide an immense infrastructure for content distribution. Over time, seamless 3D asset integration will be introduced for game development engines like NVIDIA Omniverse, Roblox Studio, Unreal Engine, and Unity.
The company joined the NVIDIA Inception Program in October 2022 and started developing Shapeyard’s integration with NVIDIA Omniverse. These new capabilities will contribute to the availability of 3D content for usage in the NVIDIA Omniverse engine through a Connector and an Extension, scale the creator community of assets for NVIDIA Omniverse, and provide Shapeyard asset creators with a direct distribution channel.
Use of machine learning in Shapeyard
Machine learning (ML) plays a significant role in Shapeyard’s solution development. Among the ML-based features planned for production are higher-level classification systems for model optimization and enhancement.
These systems will help users identify parts of a 3D model that will not be visible (the bottom of a car or the inside of a shield, for example) and elements that will be deformed (character elbows, for example). Trained using TensorFlow on NVIDIA TITAN V, the ML-based model will identify these components and automatically reduce the polygon count or alter topology to reduce deformation in the identified areas.
Shapeyard also uses machine learning to recognize and filter inappropriate content, providing a safe user experience for creators and partners. Shapeyard’s deep learning-based 3D model classification techniques filter content by relevant tags with two types of classificators: render-based and geometry-based.
Render-based classificators are relatively small models that are trained on desktop computers with NVIDIA GPUs and run efficiently on devices. Geometry-based classificators rely on voxel or point cloud inputs, which require more GPU computing memory and power but can be deployed to servers with NVIDIA GPUs using pretrained models deployed with AWS Neuron SDK.
Conclusion
Shapeyard’s existing features make it a powerful and accessible tool for 3D modeling beginners, with forthcoming functionality that will appeal to advanced and professional 3D modelers using mobile devices. Follow the official product road map to keep up with the software’s evolution. To experience the software, download the Shapeyard app.
Register for NVIDIA GTC 2023 for free and join us March 20–23 for the following related sessions: