Autonomous Flying using the RealSense T265 Tracking Camera in GPS Denied Environment

jnIcA3G8M0jyU9jvhvkSxOk1acsBJKWox_OyAhGxwRbQWbxAEG2sLuEnOIk_YmKK0Bj1EX38P177JGRC1rJ18Dut9aDmmZ-As_xx1yKWLKgLauTvRpqBjfuv9JVBu6eFZ1P3_avI (1289×720)

As part of a Google Summer Of Code (GSOC). I have the privilege to Mentor a talented PhD Student of the Nanyang Technological University of Singapore, named Thien Nguyen.

Since the beginning of this project , Thien has delivered a series of Labs to serve not only as milestones for the project but also a step-by-step guideline for anyone who wishes to learn about using the power of computer vision for autonomous robot to follow. The labs include:

  1. Lab 1: Indoor non-GPS flight using AprilTags (ROS 2-based)
  2. Lab 2: Getting started with the Intel Realsense T265 on Rasberry Pi using librealsense and ROS
  3. Lab 3: Indoor non-GPS flight using Intel T265 (ROS-based)
  4. Lab 4: Autonomous indoor non-GPS flight using Intel T265 (ROS-based).
  5. Lab 5: MAVLink bridge between Intel T265 and ArduPilot (non-ROS).
  6. Lab 6: Calibration and camera orientation for vision positioning with Intel T265.

I invite you to read about this series of well detailed  experimentations and  instructions on how you can implement the  RealSense T265 tracking camera system :

https://discuss.ardupilot.org/t/gsoc-2019-integration-of-ardupilot-...

Here is a video showing autonomous indoor flight using the system in ROS-Mavros environment  (this is part of Lab 4):

Lab 5 shows how to fly using a Python Scrit sending MavLink Message Vision_Position_Estimate directly to Flight Controller.

You can read the underlying principles of how to incorporate a VIO tracking camera with ArduPilot using Python and without ROS. After installing necessary packages, configuring FCU params, the vehicle can integrate the tracking data and perform precise navigation in GPS-less environment. Pose confidence level is also available for viewing directly on GCS to quickly analyse the performance of the tracking camera.

Thanks to Thien for this amazing project, experiments can now be carried on the T265 with different Flight Controllers and stacks compatible with the Vision_Position_Estimate  MavLink Message.

Views: 1087

Comment by Patrick Poirier on June 27, 2019 at 6:32am

Thanks for the suggestion :-)

As for the D435 , take a look at my code, you can make arrays of differents resolution and extract the minimum value (closest object) using numpy 

 https://github.com/patrickpoirier51/REALSENSE-EXPERIMENTS/blob/mast...

Comment

You need to be a member of DIY Drones to add comments!

Join DIY Drones

Groups

Season Two of the Trust Time Trial (T3) Contest 
A list of all T3 contests is here. The current round, the Vertical Horizontal one, is here

© 2019   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service