Hi Everyone -- here's a big (and good!) status update on the Solo gimbal.
The gimbal production pilot is finished and mass production has begun! This means units will be leaving the factory this week. A software update will be timed to go out soon with a variety of updates, and more software updates will follow quickly on the heels of gimbals getting into customers’ hands. That should happen in all likelihood the week of 8/24. (Why can’t we predict this in-hand date exactly? Gimbals are released on a rolling basis, and as they leave the kitting facility in China, they enter various modes of transport, including UPS, FedEx, etc., that all have variability in their delivery dates. Customers who have pre-ordered units from both 3DR and our partners should start seeing them next week, but it will take a couple of weeks to get to everyone. Retail shelves will likely see stock in a few weeks, mid-September or so.)
Here's some sample video from the latest software build (there will be one more build before gimbals arrive with customers early next week). This is the baseline -- it's only going to get better from here!
Of course, there is much more to the story than having gimbals shipping out of the factory.
Our software team is cranking and there’s a lot of performance we have gained recently. This also means that we’ll continue to release updates that improve performance over time.
Our life test data continues to look strong. That means the gimbals should last a long, long time.
User testing continues to go well. Regular people are able to set up the gimbal and get it flying quickly and easily.
Our customer support team is trained up and ready. The how-to videos are shot and ready to go into the Flight School portion of the app. We have refreshed the app to incorporate the gimbal and improve overall usability. The app UI has been made more elegant; Solo will automatically turn itself off if it has a wonky landing or crash (which should happen rarely!) - stay tuned for a number of updates.
We're excited for everyone who has preordered to get their hands on their gimbal in the coming weeks! We deeply appreciate your patience throughout continued development, and we can't wait to see the amazing videos you create.
For those of you who have preordered in the initial months with directly with us, Vu, who heads customer support at 3DR, will be sending out a reauthorization email with some final instructions. It’s important you follow them, we won’t be able to ship your gimbal without it! Please be on the look out for it.
For this post, we want to let you inside 3DR by providing a look into the overall management process for a product like the gimbal. In the past we’ve focused mostly on technical matters, so we hope this is an interesting change of pace.
To cut to the chase: we were not happy with the performance of our gimbal over the past month while we’ve been tuning performance. So we decided to make it better, which means starting production this week (first week of August), not in late July as we last promised. This was a tough decision, to be sure - this post explains our logic.
So, first things first. The gimbal performance we were seeing wasn’tbad. In fact, we were right in that gray area where we could have reasonably decided to ship it right now. Our user tests on “naive users” - people that haven’t seen much aerial video - were uniformly positive. But with a trained eye, you could see movement in yaw in windy conditions.
So, we had a decision to make. Ship the gimbal as-is, or make it better. We decided to make some changes to make our gimbal work great, even if it meant taking a little longer to do so.
Decisions like this are always challenging because they aren’t black and white. At 3DR, one of our guiding principles is to prioritize the user experience (UX) over schedule over cost. We write it this way, as system of inequalities:
UX > schedule > cost
If you ever come to 3DR, you’re likely to see this little equation on whiteboards, in notebooks, in our internal docs.
We know perfection is impossible. Nothing we will do is ever going to be perfect. It’s hard to even write that sentence, because at 3DR, we strive for absolute perfection. Sounds pretty hokey, right? But that’s really the way we work.
And while we know we may not achieve perfection, there is some difficult-to-define point where the team can say, “That looks good. I’d put my name on that. Let’s ship it.” Not all companies have enough synchrony across engineering, design, ops, marketing and every other team such that the whole group hits that point at about the same time. At 3DR, we do have that synchrony. And we just weren’t feeling like the system was ready to go.
So specifically, what weren’t we happy with in the gimbal performance?
Over the past month, we’ve been tuning the performance to get the gimbals to stabilize well across the unavoidable variance of the manufacturing process. In so doing, we pushed our software control system really hard.
At some point in that tuning process, we got together to try and understand how much more performance we could get purely out of software.
Taking a step back: in any hardware/software product, whether it’s a thermostat, fitness tracker or drone, there’s an interplay between hardware and software engineering. The hardware sets the bounds of what software can do. Software can move fast to improve things but cannot operate outside the basic design of the hardware. Hardware is slower to develop, but can open up new territory for software performance and features.
In our case, the weight (or lack thereof) of the top part of the gimbal compared to the bottom part taxed our yaw motor’s ability to correct for disturbances in yaw. Basically, when the copter and gimbal get hit by wind or other disturbances, your video might look like it is vibrating or drifting from left to right. There’s a lot of technical detail here that I will omit, but our decision boiled down to: leave performance as-is, or find a way - a way that would add time to the schedule, no doubt - to address the yaw wandering.
We did find a way. Adding mass to the ends of the spider bracket - the piece that holds the dampers on the gimbal at the very top of the assembly - increases the overall mass of the top part of the gimbal. This allows the yaw motor to react to and correct disturbances hitting the bottom part of the gimbal where the camera lives, resulting in more stable video.
This improvement is relatively straightforward to implement in mass production (as much as anything is straightforward to implement in mass production), so we decided to go for it (after, of course, designing and testing the improvement).
It wasn’t an easy call. We know we could work on the gimbal forever and it would get better each day. We set strict requirements and checklists during our product development process to make that process as efficient and objective as possible. But at the end of the day, it boils down to a subjective, qualitative, ineffable judgment call - “That looks good. I’d put my name on that. Let’s ship it.”
We are very eager to get this gimbal out to you! Solo’s not complete without it. And we know not to let perfect be the enemy of the good. Actually, that’s not quite right - good isn’t good enough. I’ll rephrase: we know not to let perfect be the enemy of the great.
So that’s where we stand. We made some changes and started production. We'll check our product thoroughly before we ship the units. Fixing yaw wandering is only part of the story - there is so much more work that has gone into the gimbal (and is going into the gimbal, and will continue to go into the gimbal via software updates) that I don’t have room to describe here.
But the decision to improve the product and take the schedule hit vs. shipping as-is seems to me to be the most interesting thing to share with you all. We hope we’re painting a clear enough picture - perhaps with this information, you could even imagine yourself here at 3DR, doing this work, and making these decisions.
Thanks for your support and patience. We’re continuing to work hard on your behalf.
Finally, most important, here is some video that does not include any of the recent tuning, but has a version of the latest hardware. This was shot by one of the 3DR team, Jon.
Hello from China! Well, I’m writing this from an office in Palo Alto, but the news comes from the team in the factory in Guangzhou.
The gimbal is making good progress, with our production crew delivering some great footage recently. Check it out below.
The team is currently in China working on DVT2 of the gimbal, a pre-production build. If it goes well we move directly into making sellable units. The DVT2 units are essentially identical to PVT units (the ones we sell and deliver), so right now we’re just verifying the entire assembly and test procedure is in place before we say GO.
WHEN WILL THE GIMBAL BE OUT, BY THE WAY?
Right, a most important question! We’re targeting the end of July, about a month from now, and it’s looking good so far. Like we said, we want to get it right, so we always prioritize quality over schedule. Plus, the schedule has extra buffer time built in for any unknowns that may arise, so we’re feeling really good about making that delivery date. Until then, here’s a look at some footage from the gimbal as it is today—not even a touch of stabilization in post.
MORE ON THE STATUS OF THE GIMBAL
We’re happy to report that the HDMI flexes for production are meeting spec based on the latest testing, and we’ll be putting them into the DVT2 units by the end of this week. We’re still root-causing the 2% failures due the to over-currenting mentioned in the last post, and, long story short, we’re working on fixing this with more advanced filtering techniques on our power management. This means we’ve been able to turn this into a software issue, which means it won’t affect our manufacturing schedule—and for you, the delivery date!
The plan right now is to run the units through the actual assembly line with all the fixtures and test procedures in place. These procedures have been iterated on since the last build.
Right now we are finishing SMT—this acronym basically means the printed circuit boards are being cranked out and tested. The next step is assembly, in which all the pieces of the gimbal are put together and tested—this will be done this week, which means we will be test-flying the latest and greatest on copters over the weekend.
Meanwhile, in software land, a lot of supporting pieces are falling into place. These are mostly key pieces that include behavior not directly related to stabilization. These include making sure that the gimbal updates seamlessly when installed on the copter—this means that the user will be able to update the whole system at once via the Solo app, without any wires or uninstalling/reinstalling/screwdrivering of anything.
There is also internal “devops” infrastructure, such as updating our entire software build process to ensure new gimbal firmwares are always in sync with the rest of the code. The Pixhawk firmware, the Linux firmware on both the vehicle and controller, the STM32 on the controller, the app, the ESCs—whew—it’s a lot of stuff to coordinate. When you are early in the test process, the number of people fiddling with the system is small, so you can have a pretty manual build process. But as we test with more and more people, it’s crucial that we ensure all the systems remain in sync.
Finally, we’re adding much more robust datalogging from the gimbal, so we can better track and understand the behavior of the system over time. This will be one smart gimbal!
Stay tuned to this blog as we continue to roll out updates—we’re hoping to be able to add one every week, but honestly, this stuff might get way too technical and dull, so we might package weeks together. At any rate, we’ll maintain transparency every step of the way until the gimbals are on Solos and blowing minds!
(We wrote a post on the 3DR blog about the gimbal development - check it out if you are interested to get some insight into how it is going. I'm also posting the same content from the post over here at DIYD.)
Inside 3DR: A look at the gimbal development proces
At 3DR we are big believers in open—both in terms of software development and in terms of communication with our community. We’ve lately been hearing lots of questions about the Solo Gimbal—when will it be out? why isn’t it out yet? what can it do? how good is it?—so we thought we’d open the doors and share some insight into how this complicated product is progressing.
We will be keeping these doors open until we ship the gimbal—you can expect updates from us on about a weekly basis, posted to the 3DR blog. Our goal is here isn’t to answer every question (we wish we could), but to share more about the product development process of something as complex as a 3-axis, super-advanced gimbal.
We’ll tackle some initial questions now in this post.
HOW GOOD IS OUR GIMBAL?
Here’s two links to sample footage from the gimbal. First, one of our flight test folks, Terrence Williams, shot recently on a prototype Solo and prototype gimbal:
Solo in Oakland Video (With Adobe Premiere Warp Stabilizer set on Position, Scale, Rotation at 25%.)
We are pretty happy with these! We’re working on more improvements in software, but we believe we’re off to a good start.
WHO IS DOING THE WORK?
Designing something this complex at high volume is a big effort.
In our case, the core engineering is being done in Berkeley and San Francisco, partnered with a firm in Austin, and with 3DR engineers flung across the globe, from Brazil to Australia. We’re producing the gimbals at a super nice factory in Guangzhou, China.
WHERE ARE WE NOW? AND HOW DOES PRODUCT ENGINEERING WORK?
We currently have fixes for a variety of issues, which we’re testing now in California and soon in our factory in China. We are also working hard with the GoPro team to ensure the GoPro control from the mobile app works great.
In terms of schedule, the Solo Gimbal is set to enter a “DVT2” phase that will quickly turn into “PVT.”
…let’s next explain what those acronyms mean!
Most hardware product development processes follow a similar pattern. Whether you go to Apple, Fitbit or 3DR, you’ll usually see the same acronyms—DVU, EVT, DVT, PVT. Generally, this process takes between 12 and 24 months, from first concepts to products ready on shelves.
Early on in the product development process, the core engineering and design team will work on a series of prototypes. At some point, the team feels confident that they have a design they’re ready to release to manufacturing—this is usually called the DVU (“Design Validation Unit”).
At this point, you begin to invest heavily in manufacturing tools—plastic injection molds, inventory and so on—so you need to be confident in your design. There is a long waiting period—8 weeks is common—while these tools are built, so during this time the engineering team will focus on work in software and electrical.
Once the factory is able to produce its first units, you see a series of prototypes called EVT, DVT and PVT. In the EVT (Engineering Validation Test) phase, the factory will attempt to reproduce the DVU—generally there are lots of things that don’t quite work about the EVT unit, so this is a big learning experience for the company and the manufacturer. These issues are addressed with the next step, the DVT (Design Validation Test), which corrects many of the issues discovered in the EVT; the DVT also provides the first opportunity to try out the mass production assembly line, including test fixtures and so on. At this point, the goal is for the DVT unit to look like and work like the product spec pretty closely. Finally, in the PVT (Production Validation Test) phase, the company and manufacturer make a limited run of products that are actually sellable units.
For complex products, a fourth or sometimes fifth iteration is introduced (e.g., EVT2 or DVT2) to reduce risks that crop up during the development process. The Solo vehicle, for example, had a DVT2 phase to help us fine tune things like the antenna design.
DVT gimbal being tested
The 3DR team is currently testing some fixes in the gimbal DVT2. These include an issue in which about 2% of the units built to date will occasionally over-current the motors. Again: We have a fix for this issue in place, and we want to test it thoroughly before releasing the gimbal to the public. We understand that 2% doesn’t seem like a lot—many products would launch with a yield this high (98%)—but we are laser focused on increasing the quality of the user experience so we want to eliminate even risks of this size.
Another issue we’ve solved is making sure that the flex cable—the flexible cabling connecting the control signals and HDMI from the GoPro to the Solo—are properly spec’d (impedance matched, in the case) so that we have stronger guarantees on long-term quality. So far, the gimbals we have work well with regard to both controls and video, but we need to be pretty specific about these details so that we can reduce the risk on long-term issues for our customers.
We’ll talk more about our problem solving in future posts, but we hope that this taste of product engineering gives you a sense of the size and complexity of issues that companies making mass scale products must address in order to ship a product to customers with the highest confidence.
3DR is hiring more engineers, in our Berkeley office. APPLY HERE!
We're growing fast and working on some really neat new products. Hope you can join up!
Specific roles include:
EE
ME
Embedded SW
Mobile developer
Flight test engineer
What we are looking for We are looking for candidates who have proven expertise in creating well-designed, robust, and delightful systems in end-user products (did you work on a smartphone? how about the roomba? an electric car? that's the kind of work we hope you've done), an ability to quickly improvise and iterate, and a passion for building innovative technologies - and building flying robots (natch).
I'm pretty dang sure there are a lot of folks in the community here that would really enjoy working full-time at 3DR and have super amazing talent levels ... so I hope you get in touch. The time is now! :)
