3689456308?profile=original

I wanted to analyze some data from a tlog file and figured that Excel would be the most flexible  way so I created a little utility to extract the data I was interested in and save it as a comma separated value (CSV) file. Each selected parameter is in a separate column, and Excel can open  the csv directly.

Note that it can't open the tlog file directly. You must use the Mission Planner to convert it to txt format. so:
1. From the APM Mission Planner Flight Data page select "Telemetry Logs", then "Tlog > Kml or Graph", then "Convert to Text". 

2. Select the tlog file you are interested in, and the Mission Planner will create a txt file in its log folder. 

3. Open the txt file in this program and it will parse the file and list the available parameters in the left tree control. Double click on the parameters you are interested in and they will be selected and displayed in the right tree control. If you want to remove a parameter from the selected list, double click it.

4."Save As" and provide an output file name. This will produce the csv file that you can open in Excel and plot as desired. The file will look like this, where each row was produced from one packet in the input file that contained at least one of the selected parameters.

3689456415?profile=original

If you want to re-use the list of parameters you can save and re-load them from the "configure" menu. These are simple text files you can edit directly if you like.

I am not familiar with the deloyment process for C# programs, so I have placed the entire project folder here

E-mail me when people leave their comments –

You need to be a member of diydrones to add comments!

Join diydrones

Comments

  • Agustin,

    You are correct. There is currently no way to do that with the data available on the mavlink. All we get is the time since system boot, then the mission planner adds the time/date from the computer running the mission planner. The mavlink protocol definition of the gps_raw packet allows for either time since boot, or an absolute time (in usec) so I have asked in one of the discussion groups if there is any interest in moving to an absolute time for this field. 

    From looking at some of my log files this packet is typically only transferred at 2Hz, so there will still be limited precision for all of the intermediate packets, but I still think it would be an improvement.

    Andrew

  • Andrew,

    This does not solve the problem of linking other sensors because there is no way to link the elapsed usecs to the actual time: the starting time is recorded in secs only.

    We cannot know the time that corresponds to the elapsed 409810452 usec in the 4th row

    of your example because we do not know if the initial 0 usec corresponds to  04/03/2012 2:54:12.000 or to 04/03/2012 2:54:12.325 etc, just that corresponds to 04/03/2012 2:54:12.xxx

    For example, if I have an image acquired at 04/03/2012 2:54:13.254, I cannot select the appropriate row in your file

    We cannot assume that 409917040 corresponded necessarily to 04/03/2012 2:54:13.000. Actually, between the first 

    04/03/2012 2:54:13 row and the last 04/03/2012 2:54:13 row the elapsed time in usecs is 

    410970784-409917040 = 1053744
    which is more than 1 sec.

    Agus
  • Thanks a lot Andrew, I will check the code.

    best regards!

  • Inti,

    I am not aware if any specific documentation on the data in the packets, although the code is pretty reasonable, Download and extract the sources and check the header files for each packet at a folder something like \ardupilot\libraries\GCS_MAVLink\include\mavlink\v1.0, and the GCSMavlink.pde file in the ardupilot or arduplane folders.

    Andrew

    Andrew

  • Oh well, you can only see the first of scaled columns, but the principle should be clear enough.

  • Agustin,

    As far as I can see, the only time related data passed via mavlink (and therefore available for analysis) is derived from the arduino millis() function that reports time, in milliseconds, since the system was started. One example is "time_boot_ms" in the packet "mavlink_global_position_int_t". Another is  "time_usec" available in the  packets mavlink_attitude_t, mavlink_gps_raw_t and  mavlink_raw_imu_t. "mavlink_gps_raw_int_t".

    Although  "time_usec" is scaled in microsconds, it has been calculated by multiplying the millis() function by 1000, so is really no better than a number in milliseconds.  These parameters can be selected now in TLogDataExtractor. 

    The thing to note is that the data is only updated when a new packet is received. So, lets assume that you received two mavlink_raw_imu_t packets with some other packet like mavlink_vfr_hud_t in between, and you want to look at the airspeed value in that packet. TLogDataExtractor doesn't have aany time data available in the mavlink_vfr_hud_t packet, so it will report the last time from the mavlink_raw_imu_t packet. Maybe the parameters in those packets were calculated in the same iteration and so are valid, but there is no guarantee.

    So, to summarize, the time data is already available, but is only accurate for the packets that actually contained the time information. A sample extract, with three manually inserted columns to convert from microseconds to seconds is below. There isn't much more I can do as that is the only data available. Improving the situation would require that the mavlink packet definitions be updated to ensure that each has a time value.

    Andrew

    Original Data (in microseconds) Scaled Data (now in seconds)
    Date  Time  mavlink_attitude_t usec  mavlink_gps_raw_t usec  mavlink_raw_imu_t usec  mavlink_attitude_t usec  mavlink_gps_raw_t usec  mavlink_raw_imu_t usec
    04/03/2012  2:54:12  0 0 409622496 0 0 409.622496
    04/03/2012  2:54:12  0 409666940 409622496 0 409.66694 409.622496
    04/03/2012  2:54:12  409703760 409666940 409622496 409.70376 409.66694 409.622496
    04/03/2012  2:54:12  409810452 409666940 409622496 409.810452 409.66694 409.622496
    04/03/2012  2:54:13  409917040 409666940 409622496 409.91704 409.66694 409.622496
    04/03/2012  2:54:13  409917040 409666940 409939264 409.91704 409.66694 409.939264
    04/03/2012  2:54:13  410021596 409666940 409939264 410.021596 409.66694 409.939264
    04/03/2012  2:54:13  410129812 409666940 409939264 410.129812 409.66694 409.939264
    04/03/2012  2:54:13  410234696 409666940 409939264 410.234696 409.66694 409.939264
    04/03/2012  2:54:13  410234696 409666940 410257284 410.234696 409.66694 410.257284
    04/03/2012  2:54:13  410338620 409666940 410257284 410.33862 409.66694 410.257284
    04/03/2012  2:54:13  410444508 409666940 410257284 410.444508 409.66694 410.257284
    04/03/2012  2:54:13  410550684 409666940 410257284 410.550684 409.66694 410.257284
    04/03/2012  2:54:13  410550684 409666940 410573412 410.550684 409.66694 410.573412
    04/03/2012  2:54:13  410656940 409666940 410573412 410.65694 409.66694 410.573412
    04/03/2012  2:54:13  410656940 410694284 410573412 410.65694 410.694284 410.573412
    04/03/2012  2:54:13  410760704 410694284 410573412 410.760704 410.694284 410.573412
    04/03/2012  2:54:13  410865924 410694284 410573412 410.865924 410.694284 410.573412
    04/03/2012  2:54:13  410865924 410694284 410888932 410.865924 410.694284 410.888932
    04/03/2012  2:54:13  410970784 410694284 410888932 410.970784 410.694284 410.888932
    04/03/2012  2:54:17  414946760 410694284 410888932 414.94676 410.694284 410.888932
    04/03/2012  2:54:17  414946760 414983680 410888932 414.94676 414.98368 410.888932
    04/03/2012  2:54:17  415052804 414983680 410888932 415.052804 414.98368 410.888932
  • Andrew, I'm afraid I do not fully understand. Does this mean that you can modify your program to report milliseconds or not? Alternatively, is there another field that I could use in the output table so that I can

    calculate time in milliseconds in "postprocessing"?

    Thanks

  • The parameter "time_boot_ms" in the packet "mavlink_global_position_int_t" has the time, in milliseconds. Code from GCS_Mavlink.pde is 

    if (g_gps->status() == GPS::GPS_OK) {
    fix_time = g_gps->last_fix_time;
    } else {
    fix_time = millis();
    }

    The parameter  "time_usec" in the packet "mavlink_gps_raw_int_t" is the time in usec, but is calculated by multiplying the milliseconds by 1000, so really no more useful than the first one. Code from GCS_Mavlink.pde is 

            g_gps->last_fix_time*(uint64_t)1000,

    g_gps->last_fix_time is set to the system timer "millis()" when a GPS update is received -  it is not derived from the GPS time.  

    However.... the data in the output from TLogDataExtractor is only updated when one of those packets is received, so it is static between updates. It is better than the time in seconds, and you can always try increasing the update rate in the mission planner configuration/planner page to get better data.

    Andrew

  • Dear Andrew, 

    I share the same interest with Agus for having the definitions of the columns and the time in miliseconds in order to link information from the IMU and other data. Could you help us with it? Thanks a lot for your effort and time. 

    best regards!

    Inti 

  • Thanks for your interest.

    Note that according to the table in 

    http://code.google.com/p/ardupilot-mega/wiki/MPDatalog

    the GPS Time is UTC in  ms. I understand this is for the log files, not for the tlog files, but if the time is saved in ms 

    in the log file there should be no reason for having the same units in the tlog files. I cannot check the log files myself though, by some reason all my log files are almost empty.

    Also, I do not know for how long a flight the internal memory for log files last. Any idea? In any case, as I use the device with a usb link, I much prefer dealing with the tlog files, hence my interest on your utility.

    Is there any place where the exact definitions of the fields (i.e. "mavlink_global_position_int_t.time_boot_ms")

    can be consulted?

    Agus

This reply was deleted.