Developer

Lots of changes to APM development!

3689498091?profile=original

Development moved to github

We moved APM development from subversion to git quite a while ago, and it was a big improvement, but we've been noticing quite a few rough edges to the git support on code.google.com. Meanwhile, github.com has been rapidly developing as a great platform for projects like ours, with a really nice workflow and great issue tracking. One of the big advantages of git is that it's easy to move to a new host, so we decided to move APM development to github.com. The new master repository is at https://github.com/diydrones/ardupilot and the old repository on code.google.com has been frozen with no new commits.

One of the big advantages of github is a nicer system for people to submit patches for inclusion in the master tree. On github this is called a "pull request" and it is explained nicely in the documentation. I'd encourage all APM developers to have a look through the github documentation.

The pull request system also has the advantage that it is much easier for everyone to see what patches are pending review and inclusion in the tree.

Another major change is that we've reduced the number of people that have direct access to pushing new code in the master branch. The way we worked previously was that if you submitted a couple of good patches then you would be given direct push access for future work. That sometimes led to patches going in that didn't really have adequate review, but it was the least painful way to do things with the code.google.com system. Given the nice pull request interface on github we're trying a different approach, where nearly all developers, even people who have been contributing for years, will get their code into the tree via a pull request. Only 3 developers (Pat, Randy and myself) currently have access to accept pull requests, which we hope will lead to a bit more code review and higher quality code. We'll have to see how it works out over the coming months!

Issues now tracked on github

As part of the move to github we've adopted the github issue tracking system. This integrates very nicely with git commits, so we can for example add "fixes issue #7" in a commit message and the issue automatically updates to note the fix.

One tricky decision was whether we import over the old issues from code.google.com to the new system. After much discussion we decided not to, as we'd been so slack in keeping the old issue system well maintained that most of the issues were not relevant any more. There are some important ones in there though, so if your favourite issue has been lost then please post it into the new system (after checking if its has already been dealt it!).

New developer mailing lists

Another big change is the creation of two new mailing lists, drones-discuss and drones-devel. In the past we've had a private developer mailing list, which Chris invited people to join when they made a contribution to the code. That private list really originated from the time where there was a much smaller group of APM developers, and there is no good reason why it was still private, so we've created the two new lists as public for anyone to join, and shutdown the old private list. So if you want to get involved in APM development or just want to lurk in the developer discussions then please join the drones-discuss list!

Please don't post to the drones-devel list though - this is just for coordinating releases. It is a public list, so anyone can join and listen in, but we want to keep the noise on it low if we can.

We'd also like to emphasize that these lists are not the place for help requests - they are meant for development discussions only, such as discussing new features as they are being developed. The diydrones forums are still the right place for user discussion.

AP_HAL - the Hardware Abstraction Layer

The biggest change in the APM code itself comes from the new HAL (Hardware Abstraction Layer) pioneered by Pat Hickey. If you have a look at the current git tree then you'll see some new directories under libraries that start with AP_HAL. The AP_HAL directory itself defines the new API that abstracts out the differences between the various autopilot board types we support. Then we have a separate AP_HAL_$board directory for each board. The code for the APM1 and APM2, which are based on AVR CPUs, is in AP_HAL_AVR, but you may notice there are some other board types in there as well.

The main motivation for the AP_HAL work was to give us a clean way to support the new PX4 autopilot, which is based on a 32 bit ARM core, but it does a lot more than that. It gives us a very nice way to make all of the APM vehicle types portable to a lot more types of hardware with very little work.

In our code you will notice that we have a new 'hal' object which contains methods to access the hardware specific functions. For example, to set IO pin 17 as an output pin we would now do this:

  hal.gpio->pinMode(17, GPIO_OUTPUT);

the HAL API replaces the Arduino specific calls which we used previously, and gives us a lot of flexibility to implement all the board specific functions in different ways for each board.

We think that the ArduCopter 2.9 release which is in its final release candidate now will be the last "pre-HAL" release. The master branch is fully converted to AP_HAL, and has all of the features from 2.9 merged in. It is also flying very well!

One thing to watch out for is that if you are building the APM code using the Arduino development IDE then you will need a special patched version to support the new AP_HAL based code. We've released zip files of the patched IDE for Windows and MacOS. If you build on Linux then please use the 'make' command.

PX4 support

The big test of the new AP_HAL system was the PX4 port. We've had the PX4 hardware for months now, but the software port was lagging behind badly. It was really quite tricky to port the code across to the new NuttX RTOS based platform cleanly. With the introduction of the AP_HAL system it was incredibly easy, and just a few days after we got AP_HAL into the tree we had our first successful flights of both ArduPlane and ArduCopter on PX4 boards on the same day!

So if you have a PX4 then it is now possible to use it with any of the 3 vehicle types we support (copter, plane and rover). There are still some interesting issues with how to plug in servos and radios, but if you don't mind a bit of creative wiring then it does work. We will be posting some instructions soon.

I'd also encourage you to have a good look around the PX4 website. Lorenz and the whole PX4 development team have done a fantastic job with this board, creating something that gives us an incredible scope for future development. The work they have done on the sensor drivers and ORB integration is top notch, and I found it really nice to work with the system they have created in doing the APM port.

If you want to build the APM code for PX4 yourself then it's worth noting that one of the things we don't have yet is a good way to build the APM code for PX4 on Windows - it's building fine on MacOS and Linux, but the Windows build system still isn't ready. We'll post an update when we have that working. The core PX4 code does build on Windows with Eclipse, but we just haven't done the extra bits needed to allow the APM code to build within that development environment yet.

Meanwhile, if you want to build on MacOS or Linux then you'll need the PX4 firmware tree somewhere on your system, and you need to use the "make px4", "make px4-clean" and "make px4-upload" commands.

There are still some missing features on PX4, for example we don't yet have analog inputs, so no airspeed sensor on ArduPlane, but we'll get that sort of thing fixed up pretty quickly I think. Even so, I've done around a dozen flights with my PX4 equipped SkyFun so far, and it is flying very nicely! Both Pat and Randy have flown ArduCopter on PX4 as well, and both found it flew very well. Here is a video showing Pat flying the master git branch on PX4

Note that the port to PX4 doesn't mean we are abandoning the APM1 and APM2. All the APM code builds on all 3 platforms (plus the 'SITL' simulator platform), and our plan is to continue to support the APM1 and APM2 for as long as we can. What we expect to happen is that some new CPU and memory intensive features will only be enabled when you build for PX4, where we have so much more CPU and memory available.

Happy Flying!

I hope the above gives everyone a bit better idea of what has been happening. The pace of development has been very high over the last month, and if you've noticed that some of us have been a bit quiet on the forums then this is why. Over a thousand patches have gone into our git tree in the last 2 months to make all this possible, so we've actually been quite busy!

E-mail me when people leave their comments –

You need to be a member of diydrones to add comments!

Join diydrones

Comments

  • Developer

    @Cliff,

    Yes, the VRBrain is a great bit of work. I'd like to get it ported to using the AP_HAL, and I know Roberto is looking at that. I need to make some time to work with him intensely to help him get that done. If he can get it in AP_HAL then there will be far fewer code diffs between master and the VRBrain code, which will make ongoing maintenance easier.

    Cheers, Tridge

  • Developer

    @Pete, there is a bit of work in progress with the manuals and other docs - I think you may see an announcement soon on that!

  • Great update... and thanks... I'm eager to get the new APM code onto my PX4 and test it out... ;)

  • Sounds like great progress.

    Andrew, did you investigated what Roberto has done on the VRBrain (same sensors and MCU), he's included a set of tools (though can be a conflict on automatic version updates) and added eclipse variables to get the thing to awk/diff/echo/compile the APM code to what the stm mcu needs. Works well here (using it on my VRBrain).

    PX4 on Linux works great. and just use git command line. And for the rest:

    https://github.com/blog/1181-eclipse-git-plugin-2-0-released 

    recommended... (i.e. just suck it up and use Eclipse... :) )

    I'm surprised you guys went with github vs bitbucket. ;) I more of an Hg person myself here...

  • These all seem like good changes. One question though. Will the manuals/wiki still remain on code.google? Or is there a plan to migrate them as well?

  • Thanks! I'll read up on that and give it a go.

  • Developer

    @Jiro,

    The Eclipse environment for PX4 on Windows is great for the base firmware, but it will need some tweaks to make it build the APM tree, as it doesn't yet support having some NuttX applications "out of tree" like we have done with APM. We'll try to get it fixed up soon!

    Cheers, Tridge

  • Developer

    @Andrew,

    I use the command line and gitk, but I know a lot of people like the github client. I just don't use it myself as I'm used to using the command line.

    Cheers, Tridge

  • Developer

    @Jeff,

    You are best off using cherry-pick to pull patches from your old google repo to the new one. The reason is that we wanted to get rid of some large old binaries we had in the git tree as they were causing the git repo to be very large and it was taking a long time for users to make an initial clone. Deleting those old binaries changed the base git hash, which means a rebase or merge won't work right. A cherry-pick avoids that.

    I tend to use gitk to do cherry picking, but any git client should do it.

    Cheers, Tridge

  • Hey Tridge, any tips for the best way for the rest of us to follow the move? My first attempt resulted in hundreds of "both added" merge conflicts so I'm worried my method was flawed in some fundamental way.

    1) Create an empty github repo

    2) Push all my code to that which was previously on my google code clone

    3) Add the new github repo as a remote

    4) Attempt to pull from the apm github repo

    Any thoughts or tips are appreciated!

This reply was deleted.