I wanted to analyze some data from a tlog file and figured that Excel would be the most flexible way so I created a little utility to extract the data I was interested in and save it as a comma separated value (CSV) file. Each selected parameter is in a separate column, and Excel can open the csv directly.
Note that it can't open the tlog file directly. You must use the Mission Planner to convert it to txt format. so:
1. From the APM Mission Planner Flight Data page select "Telemetry Logs", then "Tlog > Kml or Graph", then "Convert to Text".
2. Select the tlog file you are interested in, and the Mission Planner will create a txt file in its log folder.
3. Open the txt file in this program and it will parse the file and list the available parameters in the left tree control. Double click on the parameters you are interested in and they will be selected and displayed in the right tree control. If you want to remove a parameter from the selected list, double click it.
4."Save As" and provide an output file name. This will produce the csv file that you can open in Excel and plot as desired. The file will look like this, where each row was produced from one packet in the input file that contained at least one of the selected parameters.
If you want to re-use the list of parameters you can save and re-load them from the "configure" menu. These are simple text files you can edit directly if you like.
I am not familiar with the deloyment process for C# programs, so I have placed the entire project folder here.
Comments
Agustin,
You are correct. There is currently no way to do that with the data available on the mavlink. All we get is the time since system boot, then the mission planner adds the time/date from the computer running the mission planner. The mavlink protocol definition of the gps_raw packet allows for either time since boot, or an absolute time (in usec) so I have asked in one of the discussion groups if there is any interest in moving to an absolute time for this field.
From looking at some of my log files this packet is typically only transferred at 2Hz, so there will still be limited precision for all of the intermediate packets, but I still think it would be an improvement.
Andrew
Andrew,
This does not solve the problem of linking other sensors because there is no way to link the elapsed usecs to the actual time: the starting time is recorded in secs only.
We cannot know the time that corresponds to the elapsed 409810452 usec in the 4th row
of your example because we do not know if the initial 0 usec corresponds to 04/03/2012 2:54:12.000 or to 04/03/2012 2:54:12.325 etc, just that corresponds to 04/03/2012 2:54:12.xxx
For example, if I have an image acquired at 04/03/2012 2:54:13.254, I cannot select the appropriate row in your file
We cannot assume that 409917040 corresponded necessarily to 04/03/2012 2:54:13.000. Actually, between the first
04/03/2012 2:54:13 row and the last 04/03/2012 2:54:13 row the elapsed time in usecs is
Thanks a lot Andrew, I will check the code.
best regards!
Inti,
I am not aware if any specific documentation on the data in the packets, although the code is pretty reasonable, Download and extract the sources and check the header files for each packet at a folder something like \ardupilot\libraries\GCS_MAVLink\include\mavlink\v1.0, and the GCSMavlink.pde file in the ardupilot or arduplane folders.
Andrew
Andrew
Oh well, you can only see the first of scaled columns, but the principle should be clear enough.
Agustin,
As far as I can see, the only time related data passed via mavlink (and therefore available for analysis) is derived from the arduino millis() function that reports time, in milliseconds, since the system was started. One example is "time_boot_ms" in the packet "mavlink_global_position_int_t". Another is "time_usec" available in the packets mavlink_attitude_t, mavlink_gps_raw_t and mavlink_raw_imu_t. "mavlink_gps_raw_int_t".
Although "time_usec" is scaled in microsconds, it has been calculated by multiplying the millis() function by 1000, so is really no better than a number in milliseconds. These parameters can be selected now in TLogDataExtractor.
The thing to note is that the data is only updated when a new packet is received. So, lets assume that you received two mavlink_raw_imu_t packets with some other packet like mavlink_vfr_hud_t in between, and you want to look at the airspeed value in that packet. TLogDataExtractor doesn't have aany time data available in the mavlink_vfr_hud_t packet, so it will report the last time from the mavlink_raw_imu_t packet. Maybe the parameters in those packets were calculated in the same iteration and so are valid, but there is no guarantee.
So, to summarize, the time data is already available, but is only accurate for the packets that actually contained the time information. A sample extract, with three manually inserted columns to convert from microseconds to seconds is below. There isn't much more I can do as that is the only data available. Improving the situation would require that the mavlink packet definitions be updated to ensure that each has a time value.
Andrew
Andrew, I'm afraid I do not fully understand. Does this mean that you can modify your program to report milliseconds or not? Alternatively, is there another field that I could use in the output table so that I can
calculate time in milliseconds in "postprocessing"?
Thanks
The parameter "time_boot_ms" in the packet "mavlink_global_position_int_t" has the time, in milliseconds. Code from GCS_Mavlink.pde is
if (g_gps->status() == GPS::GPS_OK) {
fix_time = g_gps->last_fix_time;
} else {
fix_time = millis();
}
The parameter "time_usec" in the packet "mavlink_gps_raw_int_t" is the time in usec, but is calculated by multiplying the milliseconds by 1000, so really no more useful than the first one. Code from GCS_Mavlink.pde is
g_gps->last_fix_time*(uint64_t)1000,
g_gps->last_fix_time is set to the system timer "millis()" when a GPS update is received - it is not derived from the GPS time.
However.... the data in the output from TLogDataExtractor is only updated when one of those packets is received, so it is static between updates. It is better than the time in seconds, and you can always try increasing the update rate in the mission planner configuration/planner page to get better data.
Andrew
Dear Andrew,
I share the same interest with Agus for having the definitions of the columns and the time in miliseconds in order to link information from the IMU and other data. Could you help us with it? Thanks a lot for your effort and time.
best regards!
Inti
Thanks for your interest.
Note that according to the table in
http://code.google.com/p/ardupilot-mega/wiki/MPDatalog
the GPS Time is UTC in ms. I understand this is for the log files, not for the tlog files, but if the time is saved in ms
in the log file there should be no reason for having the same units in the tlog files. I cannot check the log files myself though, by some reason all my log files are almost empty.
Also, I do not know for how long a flight the internal memory for log files last. Any idea? In any case, as I use the device with a usb link, I much prefer dealing with the tlog files, hence my interest on your utility.
Is there any place where the exact definitions of the fields (i.e. "mavlink_global_position_int_t.time_boot_ms")
can be consulted?
Agus
-
1
-
2
-
3
-
4
-
5
of 6 Next