Merlin Sound ID in Aotearoa

By NZ Bird Atlas Team 19 Jul 2021

Identifying birds from their calls and songs is an important part of birding. As many of you know, you’ll often hear more species than you’ll see when out and about, particularly when in the bush throughout Aotearoa. Wouldn’t it be handy to have a tool that helped you ID all of those calls and songs, to ensure you’re detecting as many of them as possible? Automatic song ID has been a pipe-dream for decades, not least because analyzing sound has always been extremely difficult for researchers. We have some positive news though, researchers in the US have made a breakthrough and luckily for us they’re part of the Merlin ID team, so are heavily involved with eBird.

Merlin will listen to the birds you are hearing and identify them.

What is Merlin Sound ID?

Sound ID is a new tool in the Merlin Bird ID app that helps users identify birds by their songs and calls. As you are recording, Merlin will show suggestions in real time, which you can then compare with the built-in field guide of photos, recordings, and ID tips to learn more about each bird you hear. Sound ID has just been launched with coverage for 400+ species in the United States and Canada, and will be expanding to support to more regions.

Merlin lead researcher Grant Van Horn began treating bird sounds collated by citizen scientists as images and applied new and powerful image classification algorithms like the ones that power Merlin’s Photo ID feature.

“Each sound recording a user makes gets converted from a waveform to a spectrogram—a way to visualize the amplitude [volume], frequency [pitch], and duration of the sound,” Van Horn says. “So just like Merlin can identify a picture of a bird, it can now use this picture of a bird’s sound to make an ID,” Van Horn says.

This pioneering sound-identification technology has now been successfully integrated into the existing Merlin Bird ID app for users in North America, meaning Merlin now offers four ways to identify a bird: by a sound, by a photo, by answering five questions about a bird you saw, or by exploring a list of the birds expected where you are.

To train Merlin to identify bird sounds, the team assembled around 500 recordings for each species. Working on computers, volunteers trimmed and classified each recording by hand before it was fed into a machine-learning model that learned each song and its variations. The app also uses eBird observations to know which birds are most likely to be found at a particular place and time. Merlin project leader Jessie Barry says that Merlin sound ID marks a great leap forward in the ability for people to connect with and understand the sounds of the natural world around them. Macaulay Library web designer Matt Schloss, who describes himself as an advanced beginner and beta-tested the app, agrees.

“Merlin has helped me find birds that I might have overlooked before. It actually makes me feel like I have a superpower, or at least enhanced skills,” Schloss says. “I truly think this is going to change the way people bird.”

Is Merlin Sound ID Available in New Zealand?

Not yet. But now for the exciting news, and where we need your help! The Atlas team have begun working with the Merlin team to bring this feature to Aotearoa in the near future!

Sound recording is not as popular as bird photography, but it is increasing in popularity globally, especially with access to new tools and technology such as better microphones. To bring Merlin Sound ID to Aotearoa, we’ve had to start working through species lists with the Merlin team, to highlight high priority species for the first release. The machine learning is highly dependent on recordings from the Macaulay Library, and requires a minimum number of recordings for each species to have a shot at accurately locking on to specific species when they call. For example, even with all of Merlin’s computing power, some species present more of an ID challenge than others. “A bird like Willow Flycatcher where all the individuals have a very similar song across all of North America, is easier for Merlin to identify,” says project coordinator Drew Weber. “Compare that to something like a Baltimore Oriole, where each bird has its own twist on the typical song. For species like that, it can be difficult for Merlin to make the correct ID with very high accuracy. Fortunately, we have thousands of examples of oriole songs and calls to draw on in the Macaulay Library and the model will get better and better over time”. This brings to mind species like bellbird, tui and yellowhammer which are known to have regional dialects (Pipek et al. 2017 on Yellowhammer regional dialects in NZ).

You never know there may well be an audio related Atlas challenge or two coming soon that you can get involved with…

How can you help?

That is a great question, and the quick answer is, by recording more birds as part of your Atlas eBird checklists! There remains a list of high priority species that need more recordings, the best fit from the machine learning is from 100 recordings or more for a species, so the below need more recordings uploaded to the Macaulay Library via eBird (*brackets denote the last count of the number of audio recordings for this species in NZ):

All recordings of NZ species are still immensely valuable, we just need the above to have far more recordings to try get them into the first release of Merlin Sound ID. We’ll be updating this list and sending round updates to target further species in the future. As you can see, we have a lot of work to do, and we need all the help we can get. Sound recording and uploading to eBird in New Zealand has yet to take off, but we’re confident Merlin Sound ID can change that.

How can you contribute bird recordings?

You can upload sounds just like photos to your complete checklists that you submit to the Atlas eBird portal. Using your smartphone can provide reasonably good quality recordings of species, and of course is immensely handy as many of you will already have it recording your species via the eBird app. If you’re wanting to branch a bit further, we’ve found this microphone (doubled with this converter) to be incredibly useful for getting great quality recordings on our smartphones.

If you have historical audio recordings (i.e. before the Atlas) then please do upload those too. These can be either attached to checklists made at the time, or as part of a historical or incidental checklist. If they are prior to the 1st June 2019, then please ensure they are submitted to the NZ eBird portal, rather than the Atlas eBird portal.

Preparing audio before uploading it to an eBird checklist is highly recommended. There are some fantastic resource documents on the Macaulay Library Resources page, including ‘Recording Tips‘, and ‘Preparing and Uploading Media‘ which we recommend reading. You can also download free audio software such as Ocenaudio and Audacity, which can help get the most out of your recordings, and there is a run through article for software here.

Silver Gull (Red-billed) Chroicocephalus novaehollandiae scopulinus | © Christopher Tuffley | Macaulay Library

Merlin’s new sound identification capability is the product of years of work by the Merlin team, and was made possible thanks to the enormous collection of bird observations and sound recordings contributed by tens of thousands of citizen scientists who use eBird and the Macaulay Library. We hope you join us in trying to increase the quantity and quality of recordings going in to the Macaulay Library from Aotearoa, to help bring Merlin Sound ID to Aotearoa and to aid conservation and research. As mentioned before, keep an eye out for August’s Atlas challenge…it may well be audio themed!

Happy Atlasing!

Previous Story Next Story