If there is one thing you can guarantee when you’re planning a trip to Antarctica, it’s that you’re going to take a lot of photos of penguins. Why would you not? For many people, seeing these endlessly charismatic little birds are a highlight of an expedition cruise to the White Continent.
But what if your photos contained more than just images of penguins? What if you could go beyond the metadata to use the image as a source for important ecological data. A revolutionary technique called georegistration promises exactly that. We talk to the authors of a new study about how this might revolutionise citizen science – and how even your old photos might soon be helping researchers better understand the health of Antarctica’s penguin colonies.
Studying the ecology of penguin colonies
Dr Heather Lynch is professor of Ecology and Evolution at the Institute for Advanced Computational Science at Stony Brook University in New York. She’s been studying penguins since 2006 when she started working in Antarctica with Oceanites, a publicly supported non-profit research that for more than 30 years has been tracking changing penguin population changes along the Antarctic Peninsula.
Now Dr Lynch monitoring the penguins of Antarctica both in the field and through remote sensing techniques such as camera traps and satellite imagery. Using the high resolution images taken by satellites to count penguins from space has become increasingly widespread in recent years (we’ve previously written about it here on the Swoop Antarctica blog), but georegistration is something entirely new.
‘We’re using photos taken not from space but from the ground,’ Dr Lynch explains. ‘The goal of our project was to take photos collected from the internet or passed along to us, to figure out exactly where the photographer was standing when they took the photo.’ If that information can be unlocked – where someone was standing when they took the photo of a penguin colony – it can be used to reference other features at the same location, such as the shoreline or a nearby mountain peak, and plug that into a 3D model of the site. With that information available, the researchers could then understand exactly the boundaries of the colony on the day the photo was taken: a crucial piece of information for understanding the size and health of the colony.
How georeferencing works
It’s a simple theory underpinned by a lot of complicated maths and robust computational power. ‘It gives us the type of information we otherwise might gather from a drone, except that it allows us to use the many hundreds of photos collected in Antarctica every day by passengers,’ says Dr Lynch.
Tantalisingly, it even allows researchers to go back in time and use historic photos taken long before the advent of digital photography. Imagine being able to pull the data from a photo taken 10, 20 or 50 years ago and being able to compare it with a shot taken in 2025. The potential to unlock important ecological data is enormous
Dr Lynch’s colleague Dr Haoyu Wu explains in more detailed how the process works. ‘‘Our approach begins with a Segment Anything Model (SAM), which automatically segments penguin colony pixels in ground-level photographs taken by tourists,’ she says, explaining how images are broken down into their component data. The next step is to create a 3D textured mesh, using satellite imagery and a digital elevation model to build a topographical model of the penguin colony. ‘Human experts then align the ground photos with the mesh to estimate camera poses,’ she continues, creating an accurate picture of where the photographer was standing when they took the photo. ‘This semi-automated approach enables accurate colony mapping using unstructured tourist photographs.’
The penguins of Brown Bluff
To test how the model worked, the researchers looked to the Brown Bluff, a landing site on the northeastern tip of the Antarctic Peninsula, which is home to a mixed colony of Adelie and gentoo penguins.
Dr Lynch explains why they chose this location as a great example of a colony whose health could be monitored using tourist photos.
‘The colony is one of the larger Adelie colonies, and it’s spread over such a long coastline that surveying it in the time [researchers] have on shore is near impossible, even with a drone,’ she says, adding that the fact that the Adelie penguins at Brown Bluff are also mixed in with a smaller number of gentoos complicates the interpretation of satellite imagery. Here, the fact that Brown Bluff is commonly visited by expedition cruise ship passengers plays to its advantage.
‘Their photos, combined with this method, would allow us to see how the colony has changed over time, both in terms of overall size but also in the expansion of gentoo penguins into areas traditionally used only by Adelies.’
Climate and changing penguin distribution
The Antarctic Peninsula is one of the fastest warming places on Earth. Penguin colonies are changing as a result, which is one reason why this new technique for understanding their historic distribution is so important. Dr Lynch explains why.
‘The northeastern side of the Antarctic Peninsula is strongly influenced by the cold and icy Weddell Sea and, generally speaking, the Adelie penguin colonies on this side of the Peninsula are doing much better than those on the warmer and more rapidly changing western side.’ In recent years however, increasing numbers of gentoo penguins are starting to nest in areas that once only contained Adelies.
‘Gentoo penguins are like the canaries in the coal mine of climate change. When gentoo penguins show up in new places, it’s inevitably linked to climate change – either an area that no longer freezes solid in winter or perhaps an area where a glacier has recently melted and left behind some open territory. The gentoo penguins are highly opportunistic and move right in when conditions allow, so they are a great marker for tracking climate change in this region.’
Next steps
What’s next for this new technique? Dr Lynch suggests that it could be a powerful new tool for researchers. Many visitors to Antarctica are already familiar with citizen science projects like Happywhale. This is a platform where people submit photos of whales they’ve spotted to a large database, where they can be individually identified by algorithms, allowing whale biologists to learn more about their behaviour . It’s an approach that has already led to new whale conservation moves, such as the geofencing of areas around the Antarctic Peninsula and South Shetland Islands to limit vessel cruising speeds to prevent accidental ship strikes.
But where Happywhale depends on actively soliciting photos, the new georeferencing technique greatly expands the possibilities for monitoring environmental conditions or, as we have demonstrated in our application, populations of wildlife through being able to use photos from almost any source – and back through time.
‘Our method for georeferencing photographs isn’t completely human-free so it remains a bit time consuming to process each photo,’ says Dr Lynch. But it paves the way for expanded use of crowd-sourced or historical photography. It’s hoped that before long it can be used to develop new projects working in those parts of Antarctica that are difficult to survey but which have been visited by expedition cruises ships since the industry started to grow in the early 1990s.
Until then, Dr Lynch and her team are looking for images that thar be helpful to build their datasets. ‘Passengers are our eyes and ears and I would encourage them to reach out to us with photos or videos of anything unusual, she says, ‘Like good photos of banded birds where the band number is visible, birds with unusual colour patterns, or pictures of vagrant species that have wandered too far south. All of that can be incredibly useful for scientists.’
So if you’ve been to Antarctica recently, a decade ago or are even travelling next year, take another look at your photos: they could hold the key to unlocking even more secrets about Antarctica’s penguins by playing their part in this rapidly developing area of polar science.
*
