Drones on Boats

Sporting a heavy waterproof coat, Duke marine conservation ecologist David Johnston releases a drone into the frigid air above Antartica’s Bellingshausen Sea. With the help of the drone’s airborne camera, Johnston is able to spot fluorescent teal bubbles floating to the sea’s surface in a spiral pattern. Seconds later, a humpback whale lunges through the bubbles one by one, leaving circles of white seafoam in its wake.

These humpback whales are performing what’s known as bubble-net feeding, a sophisticated method of hunting krill. In the Marine Robotics and Remote Sensing (MaRRS) Lab, Johnston and his team of researchers use drones equipped with a variety of sensing technology to research rarely-witnessed aspects of marine life, like bubble-net feeding.

Drones provide a unique set of eyes on the ocean and its inhabitants, allowing researchers to safely and accurately collect data on animal behavior, population sizes, anatomy, and growth rates without disrupting marine animals. Drones are cheaper and more maneuverable than boats and can adapt to a variety of weather conditions. As a result, the National Oceanographic and Atmospheric Association (a MaRRS lab collaborator) launched a new Unmanned Systems Operation Program in 2020 to increase the usage of drone technology across the agency.

Johnston has been using drones in his research since 2015, when he opened the MaRRS Lab. Located in Beaufort, North Carolina on Duke’s Marine Lab campus, members of the MaRRS Lab can conduct research on salt marsh vegetation, oyster reef habitats, and inlet barrier islands in their own sandy backyard. However, researchers in the lab travel far beyond North Carolina, gathering data in locations ranging from Costa Rica to the Antarctic to observe a huge variety of marine phenomena. In the past decade alone, the lab has logged more than 4,000 flights to nearly 200 different locations around the globe. At each of these destinations, MaRRS Lab researchers conduct population assessments, monitor marine habitats, track animal behavior, and take other measurements of wildlife.

“There’s a lot of synergy between Pratt and Nicholas because of the applied nature of their projects. Faculty members from both schools frequently find themselves on teams together, or teach classes that are cross-listed, like Ocean Engineering.”

Martin brooke, associate professor of Electrical and Computer Engineering

“There’s a lot of synergy between Pratt and Nicholas because of the applied nature of their projects,” says Martin Brooke, associate professor of electrical and computer engineering, who also deploys drones in conservation-focused engineering projects. In recent years Brooke has worked with colleagues in the Nicholas School of the Environment to measure biodiversity in rain forests and to remotely map the ocean floor— extreme environments that are much more accessible to drones than to people. “Faculty members from both schools frequently find themselves on teams together, or teach classes that are cross-listed, like Ocean Engineering,” says Brooke.

In fact, says Brooke, the MaRRS lab’s chief engineer, Julian Dale, was instrumental in helping Brooke’s class get its first large drone out on the open ocean in 2016. And Brooke finds that their drone-based engineering classes are chock full of Nicholas students eager to improve their software skills.

Members of the MaRRS Lab frequently employ computational methods in their research. For example, researchers preprogram drone flight paths to facilitate thorough and efficient field studies. The lab is equipped with a variety of software for image analysis, geospatial analysis (the gathering and manipulation of location data), and motion photogrammetry (a method of approximating a 3D structure using 2D images), enhancing MaRRS Lab researchers’ ability to interpret data collected by drones.

In addition, many researchers in the lab are currently working on computer vision and deep learning algorithms to manage the large volumes of data that the drones acquire on flights. “When you have thousands of images of each whale, it is a manually intensive process to go through and actually measure them,” explains PhD student Patrick Gray. The lab recently published one of the first studies to successfully use drones and convolutional neural networks, a deep learning algorithm commonly used to analyze visual imagery, to count seabirds in mixed colonies. All of the data and code generated by the lab are open access and available on the MaRRS lab website, helping advance the marine conservation field as a whole.

The lab recently published one of the first studies to successfully use drones and convolutional neural networks, a deep learning algorithm commonly used to analyze visual imagery, to count seabirds in mixed colonies.

One of the biggest advantages of utilizing drones in marine research is their customizability. Dale says that the team applies different drone types and sensors to meet a variety of research objectives. “We have to use a mixture of different features depending on the environment we’re working in,” Dale says. The lab currently maintains a flight of 56 flight-ready drones, each tailored to a specific purpose. The two main types of drones that the lab uses are multirotor drones and fixed-wing drones. Multirotor drones rely on spinning blades to provide lift. The lab often uses small, portable multirotor drones to operate from confined vessels such as those used in whale photogrammetry. In comparison, fixed-wing drones have just one pair of rigid wings and resemble an airplane. Since the wing produces lift that keeps the drone in the air, this type of UAV needs energy only to move itself forward, making it an energy efficient option for the longer flights required in population analysis.

Most of the drones employ Sony RGB cameras, but many also have sensors to measure distances or gather concurrent data on a variety of wavelengths, a capability called multispectral imaging. According to Dale, the thermal sensors are particularly helpful in the research they do in Canada and the Arctic; it would be a nearly impossible task to differentiate white seal pups from a snowy backdrop without the help of infrared imaging. The researchers also use laser altimeters (instruments that compute distances by measuring the time for reflected light to return to a receiver) to gather more accurate information on animal morphology.

After deciding which sensors and flight mechanisms are necessary for a specific mission, engineers in the lab must integrate all the different components into a UAV. “It’s a lot of sourcing of parts from all over the place to get it to all fit together,” Dale says.

What is the MaRRS lab working on now? MaRRS lab field researchers are currently looking to incorporate telemetry tags in their research. These tags can be dropped (in the case of suction cup tags) or ejected (in the case of dart-like tags) from a drone, and will stick to the subject upon impact, allowing researchers to better analyze animal movement and population sizes and in turn leading to better data-based conservation efforts. Additionally, computer scientists working in the lab are researching further artificial intelligence-based approaches to population analysis and animal identification.

The application of drones in marine science research is a relatively new, yet extraordinarily valuable practice. With the help of these hawk-eyed machines, the MaRRS lab will continue to develop, adapt, and provide meaningful marine data to the world through engineering.

From the 2023 DukEngineer Magazine