How AI is spotting manatees in the wild

The Florida manatee—one of the state’s most beloved creatures—is big (9-10 feet long, 1,000 pounds plus)—and generally slow. The hefty aquatic beasts forage in warm, shallow waters all day in pursuit of an average 100 pounds of seagrass, and have no natural predators. Not even alligators, strangely. You’d think Florida’s marine biologists would know exactly how many manatees are floating around, but you’d be wrong.

It’s an issue: The estimated seven to 11,000 manatees located in Florida are members of a threatened species, so taking a census of how many there are and where they are is crucial to protecting them. Aside from expensive and intermittent counts done by aerial photography, much of our knowledge about manatee numbers still relies on data painstakingly collected manually, on the ground (or on the water).

This happens at places like Blue Spring State Park in Orange City, Florida, on the St. Johns River. As the temperature dips each winter, manatees migrate towards the spring’s constant 72 degree Fahrenheit flow. Employees count the rotund water dwellers from docks while volunteers conduct surveys via canoe five to six days a week, a structured process that can stretch across hours. This manual method of counting extends beyond the park to warm water refuges across the state, including the warm-water outflows of power plants. 

Though numbers are on the upswing—a count at Blue Spring this January had a single-day record of 931 manatees—the species still faces an uncertain future. Among the chief concerns are human-caused misfortunes like boat strikes, algae blooms, and widespread seagrass loss. In 2021, it was estimated that 2,000 manatees starved to death. 

Hill Zhu, a professor of Computer Science at nearby Florida Atlantic University, saw the unfortunate circumstance as an opportunity for innovation. “At FAU we have a Harbor Branch Oceanographic Institute, and we were looking into how many manatees had died during that 2021 event,” he says. “We started asking, ‘Is there a better way to count the live ones? Is there a timelier way to do it?’“ 

Zhu and his team had an idea: develop an AI that could count manatees in a live-streamed video automatically. Using footage from closed-circuit cameras mounted above the water at Blue Spring, the researchers crafted a novel method that “learns” the density of manatees in a particular scene and counts them through the adaptation of deep neural network methods that have been used primarily to count crowds of people. 

The team’s method identifies how many manatees are present in a particular scene within seconds. They use something called the anisotropic Gaussian kernel (AGK), an adaptive algorithm that can change its shape to closely match the long and varied forms of manatees. They detailed the approach in a writeup in Scientific Reports and published the open-source data on Github.

Unlike other methods of counting—teaching AI to guess at a total count in an image, or encasing manatees in bounding boxes, or annotating them with single dots—the AGK method dynamically adjusts the parameters of where it identifies a manatee. This means it’s more accurate at guessing a correct number of animals in a complex scene or when the creatures overlap. (This is common at Blue Spring during the winter, where clusters of the species can resemble a manatee mosh pit.)

Zhu and his team trained the model using 700 images with manatees identified in them, some with challenging conditions: reflections on the water, murkiness, other wildlife, logs. After training the model for weeks, “the estimated number that we’re getting is pretty accurate,” says Zhu. In scenes where humans count 50 manatees, for example, the AI identifies a range of 47 to 53. His team's report also notes that the AGK method's success extends beyond manatees to promising applications in counting livestock and crops.

Among the advantages of an AI manatee count, Zhu says, are reduced cost, ability to monitor numbers more easily during inclement weather, and estimations given down to the minute-by-minute level. 

“More granularity in the data opens a lot of potential,” says Zhu. “We’ll have a better idea of what their behaviors are and which human interventions are working, or not.” 

At this point the method hasn’t been adopted by any government entity monitoring the species. But it’s piqued the interest of Save the Manatee, which runs the cameras Zhu and team used for their footage. Cora Berchem, a researcher and the Director of Multimedia for STM, often takes part in the meticulous daily count via canoe. She says that the tech is promising—yet sees it as augmentation to the data they’re collecting manually. 

“I don't really think you can ever really replace the entire human component,” says Berchem, “but I think it can certainly help.”

The researchers already have another idea in mind: recently they met with Berchem and discussed the possibility of training the AI to identify specific manatees by the propeller scars on their bodies. A statewide photo identification database contains thousands of manatees; it has been used to track their travels, illuminating patterns of movement from the Atlantic coast inward toward the springs.  

“It's been a steep learning curve to recognize all the individuals [that come to Blue Spring],” says Berchem. “If we had a computer to help us with that, it could be great.“