Autonomous Indoor Localization with a Tracking Camera

Note: This tutorial is applicable for Kerloud Flying Rover Scorpion VIO.

Indoor localization is essential to enable autonomous indoor driving and flight for Kerloud Flying Rover. The Scorpion VIO is equipped with Intel Realsense Tracking Camera T265 and a laser rangefinder, and it can deliver 0.05m and 0.01m accuracy for horizontal and vertical positioning respectively. The tracking camera can output real-time pose estimation based on visual odometry (VIO), and the low level state estimation is conducted in the autopilot by fusing the VIO output with IMU data.

1. Environment Setup

The onboard computer is installed with all necessary prerequisites for this use case. The recommended environment for Scorpion VIO is ubuntu 20.04 and ROS noetic, and the ready-to-use workspace is located in the directory ~/src/catkinws_realsense, which contains the following packages:

• realsense-ros: the official ros package for realsense camera

• VIO: the package to bridge the realsense VIO output with the mavros channel

To build the workspace,

    # source customized offboard control workspace first
    # included in ~/.bashrc for convenience
    cd ~/src/flyingrover_workspace/catkinws_offboard
    source devel/setup.bash

    cd ~/src/flyingrover_workspace/catkinws_realsense
    catkin build -j 2 # set j=2 to avoid overburden

Note that the localization workspace is dependent on our customized packages (mavros, mavlink) located in ~/src/catkinws_offboard, hence users have to source the environment settings in the offboard control workspace first before building the workspace. We highlight that the default mavros and mavlink packages with ROS releases cannot be used for our Kerloud Flying Rover, and the source setup action is included in the ~/.bashrc by default.

2. How to Use

Firmware Setting

The autopilot is well configured in factory, hence no action is needed for firmware setting. Users have to calibrate the IMU sensor (accelerometer, gyroscope and magnetometer) for their first use, and the magnetometer has to be calibrated regularly as the autopilot uses its data for heading estimation. Please consult our after-sales for assistance if you modify any onboard parameters by mistake.

Network Setting

The Scorpion VIO should be connected in a local network with wifi connection so that it can be easily operated remotely. With the ssh tool, users can access the onboard computer with its corresponding IP address:

        ssh ubuntu@<drone IP address>

Similarly with the indoor laser slam tutorial, the drone IP address can be found by logging into the administration page of the router. Taking the TP-Link router as an example, users can visit 192.168.0.1 or http://tplogin.cn/ to checkout all computers connected in the local network. The computer name for the Scorpion VIO is ubuntu by default. The IP address is then displayed as shown below:

📌 Users are suggest to wait at least 3 minutes before connecting to the onboard computer remotely. This is to ensure that all system processes are ready after booting. Otherwise, the Realsense T265 camera may not work.

Start Indoor Localization

To start the localization process,

    cd ~/src/flyingrover_workspace/catkinws_realsense
    bash run.sh

Users can then confirm the localization output by viewing ROS topics from the command:

    rostopic echo /mavros/odometry/out # VIO output
    rostopic echo /mavros/local_position/pose  # fused vehicle position from the autopilot

Usually the localization can take 20-30 seconds to stabilize in the autopilot. Users can checkout the local position topic for the flight controller to verify localization states, and this is achieved by typing the following command in the Mavlink console from the QGroundcontrol station:

    listener vehicle_local_position

Sometimes the console output is not complete due to network burden. Users can try alternatively the command:

    listener vehicle_local_position -n 20

Perform Indoor Flight

We advise all users to perform the semi-autonomous indoor flight as the first step, as the flight can help validate the localization accuracy of the onboard software. The procedures are the same as the Quick Start Section. The vehicle is expected to hover quite well and even better than the outdoor case.

To add up, make sure that there are some features in front of the camera, and don’t make the vehicle face a clean white wall or glasses.

3. Demo Video

Semi-autonomous indoor flight:

Video Link