The brilliant Alexis Madrigal at the Atlantic has a great and historically fascinating piece on the fuzzy legal question regarding drones and privacy. Short form: the laws are vague, vary from place to place and nobody really knows what the rules are until they're tested in the courts. Precedents go all the way back to the first hot-air balloons!
It's really worth reading the whole thing, but here's one sample quote:
"This idea of a reasonable expectation of privacy has always been accepted as the standard and the interface of that privacy right and emerging UAV technology is fascinating," Ravitch said. "There is not an answer. The best we can do is arrive at laws and practices of the then-existing sensibilities of the population."