Simulate and Localize a Husky Robot with NVIDIA Isaac

The Husky robot, developed by Clearpath Robotics, is a versatile four-wheeled platform made for indoor and outdoor research use. It is simple to modify by…

The Husky robot, developed by Clearpath Robotics, is a versatile four-wheeled platform made for indoor and outdoor research use. It is simple to modify by adding other sensors and changing the high-level board. This post explains how to use the official ROS 2 Husky packages to import the robot into NVIDIA Isaac Sim and create a simulation.

For this demo, the Husky robot is equipped with an NVIDIA Jetson Orin Nano and a ZED 2 camera mounted on top. Driving the Husky uses the latest version of Isaac ROS 2, which includes Isaac ROS packages for robot localization (NVIDIA Isaac ROS VSLAM), map building (NVIDIA Isaac ROS NvBlox), and Apriltag detection (NVIDIA Isaac ROS AprilTag). 

How the Isaac ROS demo works

The Husky unified robotics description format (URDF) is dynamically loaded from a ROS 2 topic. It is used to visualize the robot in ROS visualization (rviz) and load it into NVIDIA Isaac Sim.

When the robot is loaded, the main Isaac Sim script creates a graph to drive the robot, converting the velocity and steering commands into velocity for each wheel, and publishes the wheel status in a tf (Figure 1).

Figure 1. Motor control action graph, automatically generated when Husky is loaded from the ROS 2 topic

The script generates a graph for each camera to publish the output in a ROS 2 topic. Each camera has a ROS 2 image message and camera information.

Figure 2. Camera graph with resolution configuration and publish ROS 2 topic 

After loading all graphs and successfully importing the URDF, a new Husky robot appears in Isaac Sim, as shown in Figure 3.

Figure 3. NVIDIA Isaac Sim 2023.1.0 with the Husky and all cameras output

When the Husky has been successfully loaded, different options for ROS 2 will be available, including camera streams, the motor controller, and other auxiliary outputs. You can quickly check what is running in another terminal using the following command:

ros2 topic list

Figure 4. List of all Husky ROS 2 topics

When the ROS 2 launcher script starts, all required ROS 2 nodes and Isaac ROS nodes run. The robot will build a connected pipeline where the husky URDF coming from the ‘Husky description’ will be used for Isaac Sim and the robot itself. Isaac ROS VSLAM and Isaac ROS NvBlox will also be used to localize and build a real-time map.

With all the necessary packages now running, the robot can be fully localized and is capable of building a 3D map. You can now drive it using Nav2, a keyboard, or a joystick. 

The next step is setting up your workstation and Jetson Orin (optional) to test Husky on Isaac Sim 2023 and Isaac ROS 2.

Figure 5. Husky on the move and generating a 3D map

Set up your demo

There are two ways to run this demo: directly from your workstation or using hardware-in-the-loop (HIL) configuration. This section covers the setup requirements for each of these options. 

For either option, first install the latest version of Isaac Sim. To see detailed instructions, visit Get Started With NVIDIA Omniverse. 

Note: Do not start Isaac Sim when the download is complete.

Husky demo workstation version

To run the Husky demo, you need a workstation with an NVIDIA RTX GPU and the latest versions of both NVIDIA Isaac Sim and ROS 2 Humble installed. Remember to install the desktop versions.

sudo apt install ros-humble-desktop

Install and run

When your system is ready, you can clone the demo and try it on your workstation using the following script:

git clone https://github.com/NVIDIA-AI-IOT/husky_demo.git
cd husky_demo

After running it, use the following command to start Isaac Sim and open a new terminal with Docker built to use Isaac ROS and all packages needed for the demo:

./husky_demo.sh

This command will:

• Clone all repositories required
• Clone the Isaac ROS Docker image
• Build a new image

When the build is complete, the terminal will be waiting for a new command (Figure 6). Figure 7 shows Isaac Sim running and the environment fully loaded.

Figure 6. Terminal with a Docker container with Isaac ROS prompt waiting for a command

Figure 7. Isaac Sim running with the warehouse environment loaded 

Now you can run this script on your second terminal:

bash src/husky_isaac_sim/scripts/run_in_docker.sh

The script run_in_docker.sh will execute some steps before running the Isaac ROS launch file. The first stage will update and build all required ROS 2 and Isaac ROS packages. Then it will execute the launch file:

ros2 launch husky_isaac_sim allinone.launch.py

This ROS 2 script launches all Isaac ROS packages to localize the robot and start mapping and rviz to visualize the husky on map.

The script will also load a Husky on the environment and automatically set up cameras and controllers.

Husky demo HIL version

For the HIL version, you’ll need a workstation with an NVIDIA RTX GPU and the latest version of NVIDIA Isaac Sim 2023.1.0 installed, as well as a router and an NVIDIA Jetson Orin Nano. (Note that other Jetson Orin models will also do the job.) 

It’s important to establish a wired connection between your workstation and your Jetson, and using a good router between them is highly recommended. Keep in mind that there will be a lot of data shared between your workstation and the Jetson, so a Wi-Fi connection won’t be sufficient.

Your NVIDIA Jetson Orin Nano must use the latest Jetpack 5.1.2 and be fully installed. Keep the IP address. If you hostname, you’ll need to connect remotely from your workstation.

The required components to run with this configuration are listed below:

• x86/64 machine with Ubuntu 22.04
• NVIDIA RTX GPU
• NVIDIA Jetson Orin Nano Developer Kit with Jetpack 5.1.2
• Router and cables

Figure 8 shows how to configure your hardware environment.

Figure 8. Hardware setup: desktop within an NVIDIA RTX graphics card (left), a router (center), and an NVIDIA Jetson Orin Nano Developer Kit (right)

Install and run

When your system is ready, clone the demo and try it on your workstation using the following command:

git clone https://github.com/NVIDIA-AI-IOT/husky_demo.git
cd husky_demo

After running it, use the following command:

./husky_demo.sh --HIL

This script will automatically:

• Clone all repositories required for this demo
• Build all ROS 2 packages
• Start Isaac Sim 

When the build is complete, the terminal will be waiting for a new command. Run the following script:

ros2 launch husky_isaac_sim robot_display.launch.py

You will see an image of a Husky on Isaac Sim displayed on your workstation.

Next, open a new terminal to remotely connect to the NVIDIA Jetson Orin series.

When you are logged in to the Jetsterminal, clone the Husky demo repository: 

git clone https://github.com/NVIDIA-AI-IOT/husky_demo.git
cd husky_demo

After running it, use the following command:

./husky_demo.sh

This script will automatically:

• Clone all repositories required for this demo
• Clone the Isaac ROS docker image
• Build a new image

Next, run the following script from the Docker container:

bash src/husky_isaac_sim/scripts/run_in_docker.sh

This script will automatically:

• Build all ROS 2 packages
• Start Isaac ROS 

You will then see all Isaac ROS packages running on the terminal (Figure 9). 

Figure 9. Isaac ROS running on Jetson Orin Nano and transmitting data

Drive your Husky

Assuming that both Isaas Sim and Isaac ROS are up and running—either on your workstation or Jetson, according to your preference—and there are no issues, you should be able to see the robot localized successfully on rviz. Once localized, it should start building a 3D map (Figure 10).

Figure 10. Husky on rviz localized and plotting a first slice of a 3D map with NvBlox

Now, you can drive your Husky using a keyboard, a joystick, or with Nav2. In just a few quick steps, you can control the movement of the Husky within the environment.

Summary

This post has explained how to operate the Husky robot using NVIDIA Isaac ROS, including how to load a URDF derived from a robot description, and how to drive it on rviz. This demo can be run in various ways, either directly from your workstation with an NVIDIA RTX graphics card or in the Hardware In the Loop configuration, where Isaac ROS runs on your NVIDIA Jetson.

To learn more about Isaac Sim and Isaac ROS, check out our new Isaac ROS Office Hours on YouTube. And join the conversation on the NVIDIA Developer Robotics Forum. 

Source:: NVIDIA