Hi all,
following my last post on positional accuracy, the discussion about ground planes and shielding as well as the general debate about the M8N, I started a series of tests comparing different M8N modules. This is the first part where I focus on comparing the modules in a bench test to compare positional accuracy.
The setup
I tested the following boards with settings optimized for Ardupilot:
- CSG Shop EMI / M8N / CGGBP.35.6.A.02 / 8cm ground plane
- CSG Shop XL / M8N / CGGP.35.3.A.02.
- DroTek / M8N / T0027
- Virtual Robotix / M8N / GCCP.25.4.A.04
- 3D Robotics / 6H / GP1575.25.4.A.02
Additionally, I tested the DroTek / M8N / T0027 with an "external" 9cm ground plane.
Since all modules produce low HDOP/PDOP values - around 0.7 and 1.3 respectively - the comparison focusses on positional accuracy/stability. Therefore, all GPS boards were plugged in for 10min before recording. Then I recorded their positions for 10-15min using u-center. Scatter plots of the position errors are used to compare the boards.
As a reference I recorded the GSG EMI in parallel when testing the others.
The image above shows the setup on the roof and the image below a closeup of test rig.
DroTek with the additional ground plane:
XY scatter:
Results
- It is obvious that the CSG EMI and the DroTek with external ground plane outperform all other boards.
- The DroTek and the VR show comparable accuracy.
- The CSG XL shows better performance compared to the DroTek and the VR but is not as good as the CSG EMI and the DroTek with external ground.
- The 3DR 6H shows a much more scattered distribution.
Discussion
- The board design (electronics) does not seem to have any influence.
- Larger patch antennas result in higher accuracies.
- A larger ground plane results in higher accuracies.
- The ground plane seems to have a higher influence compared to the antenna.
u-blox provides a diagram (page 19) showing the effect of the size of the ground plane for patch antennas. Unfortunately, the ublox document only lists 18mm and 25mm antennas. For 25mm antennas 7cm for the ground plane seem to be sufficient. For the 35mm it should be larger.
Remark
The results presented are only from one test. So there is for sure uncertainty. However, I made similar test the past days with comparable results. The 3DR 6H performed better in previous tests but not as good as the M8Ns, which performed not as good as in the results presented above (except the CSG EMI which showed similar results - I have not tested the DroTek with additional plane in previous test).
The next step is to compare the CSG EMI, the DroTek with and without the additional ground plane and the 3DR on a copter to compare the influence of the ground plane as well as of the shielding.
Cheers,
Thorsten
Comments
Oh drat, I accidentally deleted my message. It needed correcting anyway.
First, thank you Thorsten for doing this.
1) If/when you do testing on the copter, please make available the modified 3DR script file.
2) I am interested in Zubax not only for the GPS test, but also the external barometer it houses. Please consider testing that if you get a Zubax.
I currently have a CSGShop 5883 model. It seems to be "ok", but no proof yet. It does hold position better than the the 3DR unit which wonders all over the yard most of the time. I also ordered a RTFQ 45mm M8N to replace the 3DR until there is enough data to convince me which GPS unit is the best all around for flying MR, and for which part of the world as the U.S. and Europe don't use the same sats as far as I know.
Anyway, thanks.
Just out of curiosity, why would anyone desire a 24 hr test? To many variables would affect results, namely atmospherics, space weather, satellite movement, etc...
NOTE: GPS satellites are NOT geosynchronous or geostationary.
GPS satellites orbit in MEO -- Middle Earth Orbit (as opposed to LEO -- Low Earth Orbit where the Space Shuttle and Space Station fly, or GEO -- geosynchronous orbit, where telecommunication satellites fly.)
LEO = 90 minutes per orbit
MEO = 12 hours per orbit
GEO = 24 hours per orbit
It takes roughly 12 hours, (two orbits per day) for GPS satellites to orbit the Earth. So in my humble opinion shorter time is satisfactory, say 2 hrs.
What is the average time any GPS would be powered should be the real test.
Fixed wing aircraft have the longest time in the air, so it makes sense to establish tests that would benefit those folks who fly for hours and in turn will also provide data to any other drone hobbyist.
For a true test of all your modules, I would recommend running them all at the same time and for the same period of time. This will ensure you get a fair sampling of satellite data due to all the events that go on overhead.
As for follow on tests... I feel that the best way to sample EMI is to use one drone and test all GPS modules on it under exact conditions. This process will be far harder to due as constellation and atmospherics change and wil require you to collect shorter samples with the drone powered up.
Look forward to seeing more of your test results. You got a jump on an idea I was thinking on doing as soon as more GPS modules arrive.
It is good to see some actual tests, but as others have mentioned you really need more data. Satellite availability and multipath conditions can change over 15 minute periods. For your test I would be especially concerned about changing multipath conditions. A good start would be a 24 hour recording session from each unit or perhaps ten of these shorter sessions so individual variations will tend to average out. It takes time, but there is really no way around that for a good comparison.
You may want to make sure all M8N receivers are set up the same. Perhaps you did this, but I'd start by resetting each to the default parameters and then entering the changes you listed.
Since you are really testing antenna setups a couple wavelengths of separation or more between antennas is a good idea. Tallysman, for example, recommends at least 0.5 meters. Your test rig might be fine, but it's something to consider.
@Lockhart,
You are right the length of time for data collection is too short for a statistically sound comparison. I was thinking about how to test it and decided to try it under normal conditions. With normal I mean: you go somewhere with your copter, setup the GCS and the camera, plug in the battery and wait until EKF is ready and start the mission. So the period between pugging in the battery and takeoff is often relatively short. What I learned from this test is not do so, but wait for at least 5 min to give the GPS some time to "warm up". The data from the first minutes are much more scattered. This is why I decided to give the GPSs 10-15 min to warm up and 10-15 to record the data.
The second scatter plot of the CSG EMI version was from a parallel test running 100 minutes. So there is a difference for sure. But the versions without a proper ground plane show higher positional uncertainties within 10 min only. I would love to see a full comparison as well. But I simply have to time to do so.
Anyway, the major results of this test are: use a GPS with a proper ground plane and give it time to warm up before flying a mission.
JoeBob,
I had a look at the z axis as well, but not in detail. What I have seen so far is that the larger ground plane also has a positive effect here. In one case the standard deviation of the altitude was 57cm for the round CSG one and 230cm for a normal M8N. But I was analyzing the kml files only.
@Jani,
you find the setting on page 1 in the comments.
Many thanks
Excellent test, and very interesting! However, I think the length of time for data collection in insufficient. Some units may work better at different satellite angles, etc. A redo of this test, only with data collected for a period of time to let the satellite constellations turn over several times would be better. A test for 24 hours? Using this longer period would average out any time/constellation bias in the units. Great work, though.
Thorsten, thank you.
Would it be too difficult to plot 3D accuracy? One of the reasons I'm using M8Ns is to get more input from satellites low on the horizon. Baro fluctuations make the range of Pixhawk altitude readings +/- 2 meters. I hoping the 8 can tighten that up.
Thorsten,
Nice tests and intresting results!
Can you send your setting for M8N as u-center onfiguration file?
I have the cheapest M8N board (BeStar BN-880) on the market from BG and haven't got time to make proper settings for it. Like to try it and see if it works good enough to get in the air :)
-
7
-
8
-
9
-
10
-
11
of 11 Next