The latest release of NVIDIA Omniverse delivers an exciting collection of new features based on Omniverse Kit 105, making it easier than ever for developers to…
The latest release of NVIDIA Omniverse delivers an exciting collection of new features based on Omniverse Kit 105, making it easier than ever for developers to get started building 3D simulation tools and workflows.
Built on Universal Scene Description, known as OpenUSD, and NVIDIA RTX and AI technologies, Omniverse enables you to create advanced, real-time 3D simulation applications for industrial digitalization and perception AI use cases. The fully composable platform scales from workstation to cloud, so you can build advanced, scalable solutions with minimal coding.
During the NVIDIA keynote at SIGGRAPH 2023, CEO Jensen Huang announced ChatUSD and RunUSD.
ChatUSD is a large language model (LLM) agent for generating Python-USD code scripts from text and answering USD knowledge questions, helping to simplify and accelerate USD development tasks directly in Omniverse.
RunUSD is a cloud API that translates OpenUSD files into fully path-traced rendered images by checking compatibility of the uploaded files against versions of OpenUSD releases, and generating renders with Omniverse Cloud. A demo of the API is currently available for developers in the NVIDIA OpenUSD Developer Program.
These investments in OpenUSD expand on NVIDIA co-founding the Alliance for OpenUSD (AOUSD)—an organization announced last week that will standardize OpenUSD specifications—along with Pixar, Adobe, Apple and Autodesk. To learn more about OpenUSD and how developers across enterprises, startups, and 3D solution providers are using Omniverse to build tools for the metaverse, see Developers Look to OpenUSD in Era of AI and Industrial Digitalization.
Building OpenUSD-based applications with Omniverse Kit
Omniverse Kit is the development toolkit and engine for building OpenUSD-based applications and extensions on Omniverse. This powerful, extensible SDK is the foundation for every application, Connector, and extension built with Omniverse.
Convai recently developed an extension with Omniverse Kit that allows creators to add characters in their digital twin environments that can provide relevant information about the environment and objects, be a tour guide, or a virtual robot. Cesium for Omniverse, an extension built with Kit, enables 3D Tiles, an open standard for streaming massive geospatial datasets in virtual worlds, including those supported by OpenUSD.
echo3D, a cloud platform for 3D asset management that helps developers and companies build and deploy 3D apps, has developed an extension with Kit that enables you to add 3D assets remotely to an Omniverse project and update them through the cloud.
And Alpha3D is a generative AI-powered platform that transforms 2D images and text prompts into 3D models in a matter of minutes. With the Alpha3D extension developed using Omniverse Kit, the 3D models can be automatically imported into the creator’s Omniverse panel once they are rendered.
Better efficiency and user experience
Get started building on Omniverse with the new Kit Extension Registry, which serves as a centralized repository for accessing, sharing, and managing extensions. From here, you can browse over 500 core extensions. Download instantly once and use anywhere. Extensions can be assembled together in many combinations to build workflows and experiences that deliver new possibilities.
Figure 1. Omniverse Kit Extension Registry
Kit 105 introduces the new Welcome Window, which delivers a significantly improved application launch experience. Quickly access recent files, samples, and learning resources from the customizable welcome window, and jump-start your projects with on-demand extension loading.
After launching your desired experience in Omniverse Kit 105, you’ll notice significant improvements in user interface rendering. New rendering optimizations take full advantage of the NVIDIA Ada Lovelace architecture enhancements in NVIDIA RTX GPUs with DLSS 3.0 technology fully integrated into the Omniverse RTX Renderer. Additionally, a new AI denoiser enables real-time 4K path tracing of massive industrial scenes.
The rendering optimizations have been implemented at the framework level, automatically providing performance enhancements without any changes to existing code. With the introduction of Raster mode for widgets, render costs are reduced by up to 20x.
Enhanced modularity and performance
Developers using Omniverse Kit have always appreciated its extreme modularity, where the Kernel provides the foundation, extensions add specific functionalities, and apps and services combine extensions into complete solutions. With Kit 105, this modularity extends to consumption as well.
The Kit Kernel is now available through Omniverse Launcher, making it easy to access the Kit executable, Python, and other essential core components.
Kit Extensions are now hosted in the Omniverse Extension Registry. You can download these modules on demand into a shared location, which significantly reduces package size. When multiple apps use the same version of an extension, only one download is required, both during development and for the end user. This enables NVIDIA and the developer community to update extensions frequently without requiring new app versions, providing a seamless experience.
Build immersive workflows with new spatial framework
Developers of Kit-based applications can now easily integrate extended reality (XR) into 3D workflows with the new Omniverse spatial framework. XR functionalities, such as teleporting, manipulating, and navigating are easy to incorporate into existing pipelines with the framework.
Key features of the spatial framework include:
- New tools for adding immersive experiences and basic XR functionality, streamlining workflows for design reviews and factory planning.
- Connects NVIDIA RTX Renderer and NVIDIA Omniverse to SteamVR, OpenXR, and NVIDIA CloudXR.
- Support for spatial computing platforms and headsets. Build content, experiences, and applications for OpenUSD-based spatial computing platforms.
To learn more about the spatial framework and supported platforms and headsets, see RTX-Powered Spatial Framework Delivers Full Ray Tracing with USD for XR Pipelines.
Figure 2. The new Omniverse spatial framework enables you to easily integrate XR into 3D workflows
Experience Omniverse Kit in action
Experience all the new functionalities and performance improvements of Omniverse Kit 105 in updated Omniverse foundation applications. These are fully customizable reference applications that you can copy, extend, or enhance.
- Omniverse USD Composer enables 3D users to assemble large-scale, OpenUSD-based scenes. CGI.Backgrounds, developer of premium 360° ready HDRi environments, now has several ultra-high definition HDRi maps available to leverage in USD Composer.
- Omniverse Audio2Face provides access to generative AI APIs that create realistic facial animations and gestures from only an audio file. It now includes multi-language support and a new female base model.
These applications can be used as a template for building your own Kit-based app. You can deconstruct and add on functionalities from the sample application to build your own custom application.
Get started building on Omniverse
If you are an independent or enterprise developer, you can easily build and sell your own extensions, apps, connectors, and microservices on the Omniverse platform. Explore the broad variety of tools and code samples. To get started building, download Omniverse for free and navigate to Omniverse Developer Resources.
Join the NVIDIA OpenUSD Developer Program, and attend OpenUSD Day at SIGGRAPH 2023 on August 9.
Get started with NVIDIA Omniverse by downloading the standard license free, or learn how Omniverse Enterprise can connect your team. If you’re a developer, get started with Omniverse resources. Stay up-to-date on the platform by subscribing to the newsletter, and following NVIDIA Omniverse on Instagram, Medium, and Twitter. For resources, check out our forums, Discord server, Twitch, and YouTube channels.
Source:: NVIDIA