This video describes shortly the hardware and software aspects of my self-made tracking drone. The major parts are: a holybro frame, a pixhaxk 6c flight controller, a raspberry pi 4, a oak-d camera and 8 sensors (gps, lidar, compass, ...). The tracking relies on a fine-tuned YOLOv8 object detector and a custom python tracking code. The yaw and the pitch of the drone are derived form the height and center of the detected object. A simple obstacle detection derived from the oak-d stereo camera's, prevents the drone of crashing into obstacles closer than 3m.

My Holybro 500 v2 is equipped with a Luxonis OAK-D-Lite camera (depthAI) and a RPi4. A python artificial intelligence (AI) code on the RPi4 detects objects (f.i. persons) using mobilenet ssd. The area of the enclosing object rectangle estimates the distance of the object, while the center determines the yaw of the drone. 2 PID controllers determine the forward speed and yaw of the drone. Once the track switch is on, the drone is controlled in guided mode by the custom python code. This algorithm yields a satisfactory tracking behavior as shown in the video.

I have upgraded my quadcopter XJ470 with a Skydroid T12 radio, an OAK-D depthAI stereo camera and a Raspberry Pi 4 companion computer. This configuration enables state-of-the art artificial intelligence drone piloting. The skydroid t12 enables long distance telemetry and video. The OAK-D combines depth measurements and artificial object detection. The RPi 4 has an WiFi access point enabling remote desktop communication by means of VNC. Avoidance python scripts are uploaded to the RPi 4, generating mavlink drone messages controlling the quadcopter. A test video demontrates the new features.

The RosettaDrone 2 app creates a Mavlink wrapper for the DJI SDK and enables Mavlink commands to control the DJI drone. A Python script running on a PC is linked by UDP to the RosettaDrone 2 app. As a test case a SSD Mobilenet AI algorithm controls the flight behavior of DJI Mavic 2 drones

This video demonstrates face tracking using a DJI TELLO drone. By means of the Python DJITELLOPY SDK and the mobilenet ssd artificial intelligence neural network the drone is converted into a smart drone. Tensorflow Lite and Google's Coral usb accelerator enables real time inference.

This video demonstrates a modified ArduPilot-Gazebo simulation for artificial intelligence (AI). A ROS subscriber is implemented with a AI python script. A human walking actor is added in Gazebo to simulate a human obstacle. More details can be found at my wiki.

This video illustrates the adaptation of a ArduPilot Rover to the Artificial Intelligence DonkeyCar environment. This includes the adding-editing of 5 files of the DonkeyCar library. A linear AI model trained by behavioral cloning resulted in a satisfactory autonomous test drive.

By using Intels OpenVINO and neural computing stick (NCS1) the inference speed of my neural network is increased from 5 frames per second (fps) to 20 using a RPI3. This enabled a far better obstacle avoidance performance of my drone. More info at https://github.com/avncalst/drone_cnn.

Simple testflight of my E-Flite Convergence equipped with a pixracer flight controller complemented with a "sbus to pwm" home-made circuit to extend the number of servo outputs. Flight stack: Ardupilot Quadplane; Flight mode: loiter.

By changing the CNN of my previous post CNN (1) into a multi-classification CNN (fly, go left, go right, stop) and adding an appropriate multi-thread python dronekit script, my drone bypasses most of the obstacles encountered at our "Model Club de la Côte de Beauté - MCCB". The YouTube video demonstrates the obstacle avoidance of my drone in loiter flight mode.

Recently I started working on convolutional neural networks (cnn) for obstacle avoidance. With the help of "DroNet" from ETH Zurich and the "Deep Learning for Computer Vision" book from Adrian Rosebrock, I managed to build my first cnn algorithm for obstacle avoidance. The cnn module is written in python, using a keras module with tensorflow backend. I included my cnn algorithm in a dronekit script. By sending mavlink distance messages to the flight controller in loiter or altitude hold flight mode, my drone is able to avoid obstacles. A brief description and demonstration of the developed cnn is given in the YouTube video. First results are promising.

best viewed with 720p HD settings

This winter I rebuilt my XJ470: added a PiZero with a Picam. This PiZero functions as a companion computer with a python dronekit and a WiFi access point enabling a mavlinkbridge. Hence an android tablet, running Tower, can be connected to this WiFi access point for monitoring and telemetry. Moreover the PiZero-Picam streams video from the multicopter. This new configuration was successfully tested at the Model Club de la Côte de Beauté (MCCB) Charente Maritime (France).

For bench marking my own developed flight controller, I built a Quanum XJ470 drone with a DroPix (Arducopter/PixHawk) flight controller. Some additional features: esc AFRO 30A; motors: SunnySky 980 KV; props 10 x 4.5. My first testflight shows that the drone with the DroPix flies great.

Complementary filters enable sensor fusion, are easy to implement and eliminate integrating errors from gyro’s or accelerometers. Normally complementary filters are discussed in the frequency domain.  Using simple Python code a 1^st order linear complementary filter for determining the quadcopter ground speed was analyzed in the time domain. The results are depicted in the figure shown above.  voff represents the filter output of the accelerometer offset,  acc(n) the filter output  of the accelerometer, vgps(n) the output  of the GPS velocity, vgps,  and v(n) the sensor fusion output of the filter.

It follows from the figure above that the integrating drift of the accelerometer is converted into a constant steady state error.  Moreover this error can be eliminated by deriving the filter output at the start of the quadcopter with GPS input zero, and subtracting this value from the complementary filter output during flight mode. It also follows that the output v(n)  (steady state error corrected) corresponds nicely with the GPS velocity (noise added at simulation) and equals the algebraic sum of the more slowly varying acc(n) and vgps(n) contribution. Output noise is suppressed due to the low pass filtering of vgps.

Although these results confirm the already known features of complementary filtering, it helped me a lot in a better understanding of the basic mechanisms of complementary filtering, especially how a delayed, filtered signal vgps(n) nevertheless results in a correct complementary filter response.

The simulated results are confirmed by actual recorded data from my quadcopter, see figure below. For more details consult my google site.

I upgraded my 3D printed crossfire with a Boscam transmitter TS351 and a all-in-one monitor RX-LCD5802. The FPV camera is a GoPro Hero 3. Simultaneously a Sparkfun GPS LS23060 was added. This implied adding code for reading, parsing and processing nmea data. I have chosen a complementary filter approach for fusion of accelerometer data and GPS data. I managed to implement the additional code in my DIY flight controller with a 8 bit PSoC 3 from Cypress running at 45 MHz. In order to minimize the magnetic field from the power distribution print, mu-metal shielding was applied. At the moment the code is being debugged and tested for GPS position hold applications. Speed & position detection is ok. Magnetic interference is reasonable. Flight tests are scheduled in the near future. In the mean time test FPV flights were carried out (see video), showing the Boscam-GoPro setup works satisfactory.

After tuning my PID parameters for both my 3D printed quadcopters crossfire and miniflame, I succeeded in having some decent test-flights. The flight controller used is an own built controller inspired by KK2, AeroQuad and ArduCopter. Future development: including a GPS module.

Having a background in electronics I wanted to build my own flightcontroller. At the same time I decided to build a new quadcopter. Having bought a 3d printer from Ultimaker, I printed a quadcopter frame inspired on the original Crossfire design from MickeyB (thingiverse).

printed part

Inspired by the ArduCopter, KK2.0 and AeroQuad firmware I wrote my own code using the PSOC3 (Programmable System on Chip) platform form Cypress. The custom flightcontroller circuit board was fabricated using Elektor printservice.

assembled flightcontroller

The 9DOF sensor stick of Sparkfun is mounted on top of circuit.

topview mounted flightcontroller

A cheap Nokia 5110 LCD is used as a simple flightcontroller UI. A RS232 interface links the flightcontroller to the PC in order to load the calculated/calibrated offsets and PID parameters using a simple python script.

python quadcopter pc ui

3d graph of a calibrated magnetometer sensor

Finally I tweeked a little bit the FrSky Taranis transmitter in order to display the pitch, roll and heading calculated by the flightcontroller on the telemetry page.

telemetry screen Taranis

The pitch is dispalyed as T1 (temperature1), the roll as T2 (temperature2) and the heading as Curr (current).

At present first testflights are carried out.