folkfriend.app - recognise tunes by audio / automatically generate ABC

It’s been two years since I first shared FolkFriend with the community on thesession.org, and so I’m posting again to inform people about it now that I’ve completed a major rewrite of the entire project.

FolkFriend is a free app that listens to instrumental folk music, transcribes the melody to sheet music, and searches a database of tunes for matches. You may either use the microphone on your device, or upload an existing audio file.

If you’re already sold then read no further, and try it out here!
>>> https://folkfriend.app <<<

FolkFriend is blazingly fast - on an average smartphone, after recording 10 seconds of audio it only takes around 2 seconds (!!) of “thinking” for the algorithm to work out which tune is playing, and bring up the sheet music of any potential matches. It also generates the sheet music of what it actually heard, so even if you’ve written a *completely new tune* FolkFriend should still be able to auto-generate corresponding ABC / sheet music - I’ve used this feature myself a lot to get me started when transcribing tunes from recordings.

FolkFriend runs entirely in browser (Chrome, Safari, Firefox, etc.), which means any modern device with a browser can run it. I’ve tested on recent versions of Android and iOS on several different browsers and have found the app generally works well, I even tested it on my GPS running watch (Samsung Galaxy Active) and was mildly astonished to find it could record audio and that FolkFriend worked perfectly on it, correctly recognising each one of 3 tunes at a session with 20 people playing, and bringing up sheet music on the admittedly tiny screen… although I’m sure there’ll be device / browser combinations I’ve not tested so please let me know if you have any issues. The app works without an internet connection once it has loaded for the first time so if you’re at a pub session in the middle of nowhere it’ll still work, super fast.

This has been a personal project of mine for around four and a half years now, and so I’m very keen to see people use it. I think it’s a great help for people who are new to sessions in a particular area, or new to sessions in general, and likewise very handy for experienced session-goers who want to know where a certain tune is from. It’s also useful for anybody who just wishes to transcribe music! The app is completely free and non-commercial, and what’s more with this release I’ve decided to open source the entire project so that technically-minded musicians can see how it works, and contribute if they have ideas for new features: https://github.com/TomWyllie/folkfriend.

You can try out FolkFriend here:
>>> https://folkfriend.app <<<

Please let me know if you have any thoughts / problems / suggestions for this project 🙂

Hi, I can’t see how to upload an mp3 file, any suggestions?

Ok, I’ve sussed how to upload - very impressive, you’re a genius!!!

Thank you Tom

Thanks Tom. I’ve been using the previous version for a while and the hit rate for recognising tunes is very good.

Thanks Tom. I have been using folkfriend since your post here about the previous version. For me, this is the most reliably accurate tool of it’s kind.

Are there some instructions somewhere to help me get this app to work? I’ve loaded it on my iphone but can’t figure out how to record a tune. I tried using Voice Memo but nothing happened. Java is enabled.
Fred V

Brilliant!
I’m going to try it tonight. (iPhone/iPad).
The thing that interests me is that I occasionally play contemporary (copyrighted) songs and tunes by ear on octave mandolin and I like the idea of being able to play the four or five riffs of a song, into your app. and then save them in .abc, especially.
That way I can print them as just a one sheet PDF of listed musical ideas as a memory aid.

Many thanks, and well done!

(I’ve tried MuseScore but I really want to be playing more music and less computer.)

fredvv44 - you should be able to directly record music in the app using the large microphone button on the “search” screen. If you find that’s not working, go to the settings and enable the setting “Upload file instead of using device microphone”. You can then upload a pre-recorded file by selecting an audio file on your device when prompted to upload a file. Some mobile devices automatically choose to bring up the voice recorder app when asked for an audio file, rather than the file explorer of the device - there is usually a way to choose the latter so you can use a pre-recorded file. If you’re still struggling then please let me know!

Thanks Tom. Silly me, I just thought that was an icon. works now!!!

Does anyone know how big the tune data base is or if it can be expanded? I have about 4000 ABC tunes on my computer.
Fred V

fredvv44 - the tune database for FolkFriend is synced to thesession.org (currently ~42k settings of ~21k tunes) so if there’s a trad tune you’d like FolkFriend to recognise then please consider uploading to this website.

I tried folkfriend.app on my computer now, but I can’t get it to work. After I recorded the tune snippet I just see three cogwheels, and then nothing happens.

Hi TomWyll,
Thank you, this is a great concept!
The primary screen-reader, VoiceOver, which I use with my iPhone seems to be having some trouble. When I try to hit the “play” button, nothing happens. I have heard that unmuting the phone can fix this, but that didn’t help. The other issue I experienced was that my screen-reader was showing most of the buttons as unlabeled, so it was difficult to tell what they were for.
I have another screen-reader I use for my PC which I will try tomorrow. I’d be curious to see if any of the other screen-reader users on here were able to get the site to work.

hnorbeck - what browser / operating system are you using? It’s probably an easy-to-fix bug affecting a setup I’ve not been able to test myself.

Whimbrel - I’ve not yet added any accessibility features e.g. screen reader support, but am happy to include that in the future. In the meantime what browser / version of iOS are you using?

I had trouble with the site on Firefox (which I think Tom is fixing!) - but if your eager to give it a go, it worked for me on Chr*me - and worked very nicely indeed!

Yes, on my phone the app works really well in Chrome but not in Firefox.
I tried it on my computer (windows 10) where it works well in Edge but also not in Firfox.
I found there only seems to be problem if you use the microphone, though. No problem with sound files on your device.

Thanks for the feedback jamesa and stringy (and others), I’ve identified a problem in Firefox which I’m fixing currently and will release this afternoon.

Ah, yes, I’m using Firefox. That probably explains why I’m having problems.

I believe I’ve fixed the issue that was affecting Firefox, it was simply not accessing the device microphone properly. If you’ve had issues on Firefox please make sure to reload the page once or twice (until the “app version” displayed at the very bottom of the “about” page is 3.2.2 or greater) and it should all work smoothly now. Thanks for the feedback, and please keep letting me know if there are other issues 🙂

Great I can confirm it’s working now on Firefox, thank you for fixing it!

Now it works in Firefox.
Next problem:
I first tried it using the tin whistle that I always have lying next to my computer. Complete fail when matching tunes. Nowhere near correct.
For the flute it works fine though.
Perhaps you should “fold octaves”?

Same problem as Henrik. (Chrome on Windows 10 desktop.) Works quite well with the flute, nowhere remotely near with the whistle that I too always have lying next to my computer.

Thanks, Tom. I’m on an iPhone 8 with Safari, using iOS 15.1.

I tried several times with a downloaded audio file of solo flute playing (Cate’s Waltz), the results were a bit off.

Gromit - could you please send me that audio file and I’ll have a look at what might be causing problems?

I tune the algorithm on my dataset which does include over 1k 10 second snippets of flute playing (many with rhythm / accompaniment / background noise etc) and have found that for 65% of the flute recordings the app predicts the correct tune.

Is your problem errors in the transcribed sheet music, or just that the tune search can’t find the tune? If the latter, “Cate’s Waltz” doesn’t appear to have been posted on this website and so won’t appear in Folkfriend, perhaps that’s the issue 🙂

Tom - problem was in the transcribed sheet music and ABC, I know the tune isn’t on here or anywhere else. I’ll send you the mp3. Nice horse

Just to say I’m at a 6 instrument live session today and it’s NUTS how accurate this site/app is…
Blows Tunepal out of the water! Thanks for making it.

Glad to hear it’s working well for you belayatron 🙂

hnorbeck and bex - I’ve tested out some whistle recordings and it seems that there was a tendency to transcribe notes an octave higher than they should be on the whistle (due to harmonics), so I’ve added simple extra step to fix this. I’ve added around a hundred whistle recordings to my test dataset - before the change it was predicting the correct tune on <1% of these recordings but with this small change it’s gone up to ~70%, without compromising performance on other instruments.

I’d appreciate it if you could let me know if this has fixed your issues on the whistle 🙂 Be sure you’re using an app version 3.2.3 or greater (scroll to the bottom of the about section, refresh page a couple times to update).

Hi Tom - I tried Cate’s Waltz at a lower speed and got better results.

Tom -- your app is just fantastic. It instantly identified two reels recorded in ‘home’ session circumstances: Big John’s and The Gladstone. I used it without any trouble on Firefox 90.0.2 on an Apple laptop. Thanks! Chet

I find that if I play a tine the app usually identifies it but if I try it at a session It doesn’t identify any tune. I suspect background noise and multiple instruments playing cause this. Not of much use for me for what I want to do with it.
Fred V

fredvv44 - as tip for noisy sessions, you might need to point/place your phone closer towards the melody players. If you’ve got a guitar on one side and a Bodhran on the other your mic will struggle to hear a whistle player on the other side of the pub - I’ve tried to add as much noise filtering to Folkfriend as possible but it’s definitely not quite as good as human hearing is yet!