# Deep Learning in ROS

**Note: This tutorial is applicable for Kerloud UAV products equipped with Jetson Nano only.**

To develop robotics applications, we can follow the official repository in <https://github.com/dusty-nv/ros_deep_learning> to integrate Nvidia deep learning capabilities with ROS. The code include serveral ros nodes to deploy networks based on installed jetson inference libraries.

## Code Structure

Main directories are listed below for the ros deep learning repo:

(1) launch/: launch files to deploy ROS nodes for deep learning tasks.

(2) src/: source codes for ROS nodes:

node\_detectnet: ROS node to deploy the detectnet network for object localization.

node\_imagenet: ROS node to deploy the imagenet network for visual recognition.

node\_segment: ROS node to deploy the segnet network for semantic segmentation.

node\_video\_source: ROS node to handle the video input and publish image messages.

node\_video\_output: ROS node to create video stream with overlayed images.

image\_converter.cpp: class to convert images to various ros messages.

## How to Install

Users have to install jetson-inference libraries , ROS and build the ros\_deep\_learning workspace. The official guide is <https://github.com/dusty-nv/ros_deep_learning#installation>. For jetson-inference installation, please refer to the previous page.

For ROS melodic, we have to install the prerequisites below:

```
    sudo apt-get install ros-melodic-image-transport ros-melodic-vision-msgs
```

Then run 'catkin\_make' for the workspace under the directory: \~/ros\_workspace.

## How to Run

(1) Before proceeding, if you're using ROS Melodic make sure that `roscore` is running first.

(2) Launch the video viewer to check whether the video stream is OK:

```
    cd ~/ros_workspace
    source devel/setup.bash
    roslaunch ros_deep_learning video_viewer.ros1.launch input:=csi://0 output:=display://0
```

For input and output settings, refer to <https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-streaming.md> for details

(3) Launch the imagenet node for video recognition:

```
    cd ~/ros_workspace
    source devel/setup.bash
    roslaunch ros_deep_learning imagenet.ros1.launch input:=csi://0 output:=display://0
```

(4) Launch the detectnode for object detection:

```
    cd ~/ros_workspace
    source devel/setup.bash
    roslaunch ros_deep_learning detectnet.ros1.launch input:=csi://0 output:=display://0
```

(5) Launch the segnet for semantic segmentation:

```
    cd ~/ros_workspace
    source devel/setup.bash
    roslaunch ros_deep_learning segnet.ros1.launch input:=csi://0 output:=display://0
```

Make sure that you have downloaded necessary networks for jetson-inference. If not, you might consider downloading them manually by following instructions in <https://github.com/dusty-nv/jetson-inference/releases>.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://cloudkerneltech.gitbook.io/kerlouduav/userguide/tutorial/ros_deeplearning.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.