An experimental album of Irish traditional music and computer-generated tunes

For the past 6 months, the music album “Let’s Have Another Gan Ainm” (https://soundcloud.com/oconaillfamilyandfriends) has been distributed to reviewers and listeners in Europe and the USA as a new release of Irish traditional music. We have now publicly revealed that each track on the album includes computer-generated material, specifically material generated by our deep neural network folk-rnn (https://folkrnn.org).

Reviews of the album, both published and private, have been very positive. More information about our experiment and the music on the album can be found in our technical report here: http://urn.kb.se/resolve?urn=urn%3Anbn%3Ase%3Akth%3Adiva-235167. We show in that report exactly what folk-rnn generated and the changes that were made. More details about the reception of the album will be provided at a later time.

Read more about the research in general here: https://www.kth.se/en/aktuellt/nyheter/ai-created-more-than-100-000-pieces-of-music-after-analyzing-irish-and-english-folk-tunes-1.845897

Also, if you are in London in early October, check out this concert at the O’Reilly AI conference: https://conferences.oreilly.com/artificial-intelligence/ai-eu/public/schedule/detail/72410

The winning piece of the folk-rnn competition will be premiered: https://folkrnn.org/competition.

I don’t know whether to applaud or cry.

According to iTunes, I listened to the album six times. Not once did I even suspect a little the tunes might not be traditional. Well done! (Indeed, I’m inclined to think this software might be better at writing traditional tunes than many modern tunesmiths…)

When you said “each track on the album includes computer-generated material” I was expecting midi quality tunes. I guess the music was written by a computer but performed by actual musicians.

I don’t know enough Irish tunes to recognize if something isn’t “Traditional” but I certainly can’t tell. It sounds great!

Can we have the musicians on the album named please? I think it was the playing I liked rather than the actual tunes.

(Yes I did find them in the the footnote in the paper but it took some searching)

Excellent work. Did no-one twig?

Wonder what the reactions will be in our session when someone asks me ‘where did that tune come from?’ And I say ‘from an artificial intelligence machine’ 🙂 . This process seems similar to natural language processing, which makes sense as language and music share many common characteristics in human (and some non-human) brains. Interesting, and a little scary.

like David50, I liked the playing slightly more than the tunes - there were a few ‘old favorites’ in there, Battle of Aughrim, Toss the Feathers, etc and they blended quite well with the ‘gan ainms’. I found a few of the gan ainms fairly unmemorable but then I find some trad tunes unmemorable too. It will be interesting to see if any of the gan ainms pass into the session repertoire and become ‘trad’ in the coming decades……………

Thanks all for your comments.

@John Kacur: Why would you cry?

@Calum: If by “twig” you mean “catch on”, then no. A few people wondered why the Ó Conaills didn’t have a social media presence, tour dates, etc.

@dfost: It’s no fun to play a tune at a session that no one else knows! 🙂 That being said, come join us over at https://themachinefolksession.org . I’ve organised a few machine folk sessions. Also, try composing with the machine: https://folkrnn.org/

@christy taylor: I agree. The musicianship is what impresses me most about the album.

Heh heh. The next project surely has got to be an automated tune name generator. I foresee….

Orally engage with the AI behind the large container….
The hag with the robot….
Smash the Windows<TM>….
The Interface’s return to Silicon Valley…

Now if Bob were to directly interface his system to some of Michael Eskin’s emulators, we could all spend a lot more of our time at the bar….

Very interesting implementation. I’ve been fascinated in this general field since writing a random text generator based on markov analysis back around ’85. That program would take input of a passage of text in an input language and the output would be a random (garbage) passage of text that would “look like” and somewhat “sound like” the input language to a non-speaker.

In recent years I have been involved in various machine learning projects (esp. in the credit vetting and transaction analysis for fraud patterns) and the inevitability of computer generated music is certain.

I am not at all surprised at the quality of the generated tunes. Indeed - the future might see a lot of material computer generated and then tweaked/perfected by human. As to how much the latter will even be required will depend on the AI/neural network advances.

I suspect that in the current phase - computers can generate a large volume of tunes/scores. But it takes a human to know whether a tune is nice, great or plain awful. It would be interesting if might be possible to program a tune grading algorithm which would assess “beauty”. I remember reading a paper a long time ago which allowed computer to determine quality/catchiness of tunes based on analysis of Top-100 charts over 6 decades and across multiple genres to fingerprint “hits” versus “duds”.

What is surely going to happen (is already?) is that large volumes of computer generated scores will be graded to identify the better ones and perhaps then that material will be fine tuned/perfected by a human. Lots of copyright and derivative work issues to be sorted out!

By the way, and for the purity and sanity of the genre - it should be mandatory to flag tunes submitted to archive which are computer generated as opposed to human originated. This is to prevent generated material being recycled inadvertently into future analysis of tune archives (e.g. for training the generator). Otherwise there are interesting self-affirming (and therefore distorting/corrupting) feedback effects if you iterate over the generated material to generate further.

I’d love to read more about the approach taken - the link to the article above does not work for me. Best of luck with a very interesting project.

Now, for your next trick….. A better ML/AI based MIDI player!

All the ABC2MP3 type apps which take a score and “play” the music produce something which at best may be accurate but is entirely awful in its machine precision and lack of “soul”.

Take 100 pianists, fiddle players, banjo players, flute players etc. and give them 2000 ITM scores to play. Record and analyse how the score is “interpreted” or “translated” from dots to played sound. Use that dataset to generate “midi musicians” with soul who know how to flex, bend, ornament, draw etc. notes and truly play the music.

This might allow you to have an MP3 generator with the ability to “play in the style of…” one of the musicians who trained the algorithm.

@Terry McGee: Our first model learned to generate titles. Take a look at the 20 volumes of v1 here: https://highnoongmt.wordpress.com/2018/01/05/volumes-1-20-of-folk-rnn-v1-transcriptions/ Here are some from volume 8:

T: Sorpike’s Cat
M: 4/4
K: Dmix
Agfg edBA|GABG E2=c2|AGAB c2 Bd|cAGE FDD2|
Agfd ecBA|GFD^C D2 (3EFG|Ad^cd ^cdef|1gece d2 cB:|2ge^cA d2 ef||
g3 e fdde|fdad bdad|bagf gfed|c2 A^G A2 cA|
e3 d cAGE|GABd cAGB|Aed^c dcAF|GECE D4:|

T: The Catfering Maid
M: 6/8
K: Edor
|: F |“Em” EEE B2 A | “D” def edB | “G” ded BBG | BdB “D” AFD |
“Em” E2 G B2 B | “A7” BB^A A2 B | “G” BAG “Am” FAc | “D” BGE E2 :|
|: B | “Em” Bee e2 f | gag fed | “Em” Bee eef | “G” gee “Em” edB |
“Em” Bee d2 e | “D” fed “G” BAG | “D” F2 F DEF | “A7” GFE E e2 :|

T: Cup Of Tea Brush
M: 4/4
K: Gmaj
|: DGBG DGBG | B3 c BAGE | DG ~G2 DGBd | cABG A/B/A Bc |
dBGB DBGB | DBEB D2 A2 | DGBG DGBG | c/B/A Bc dBcA :||
|: G3 c BGGB | AEcE dEcE | A3 G ABce | dBGA BGGF |
G3 B dGB/c/d | eaag ed B/c/d | gded Bdgd | e/f/g fa g3 :|

T: John Joe Cand’s
M: 6/8
K: Amaj
c | BAG Ece | dBG EFG | AEA FEF | ABG Acd |
ecA EAc | BGB G>dB |AFA BAG | AFA A2 :|
f | gfe dcB | GAB AcA | Bef ged | cdB Aef |
gfe dcB | cde aec | EFE DFA | FGG G2 :|

T: The Comb Pop
M: 6/8
K: Gmaj
c|:BAG Bcd|ece gfe|Bdd dBG|FEF ABc|
B3 Bcd|edc Bcd|agf g2A|dcB A2:|
|:A|dfd edc|efg f3|eaa afd|geg a2f|
ddd gfe|dAA ABc|dBG AFD|DFE D3:|

@gbyrne: The period messed up the link. Here’s the report: http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A1248565&dswid=8361 Thanks for your comments! Since music is a human activity, humans will always be involved. 🙂

@Bob Sturm

Yeah, I thought that might be the easy bit!

Jess came home while I had the file playing. “That’s a pretty tune” was her first comment. I went to the machine and sure enough it was a gan ainm….

I suppose the AI will move on to making flutes once it gets bored with writing tunes……

If the computer can’t play tunes in the right way, then I don’t see how it could write tunes and play tunes the right way.

Are you sure you haven’t been hacked and a genius is sending you tunes anonymously. ? For the record I support your endeavor but think the computer needs to play it correctly before it writes tunes. That is the rule for us why would it be different for ai-paddy?

When the abc players start playing tunes correctly I will play ai-paddy’s tunes but until then, he best be getting a pint and a set of good ears. Cheers and best of luck.

Human tunes come from an infinitesimal set of data and experience. If you can write this complicated algorithm, I’d love to hear your written tunes.

My two pesos worth.

@Terry McGee: Thanks! One of the musicians we have worked with says he tends to like the generated tunes that have titles more than the others. I mean, how could someone not want to hear a generated tune titled, The Humours of Time Pigeon (https://themachinefolksession.org/tune/9)

@Beid: A session.org member once remarked that until we teach the computer to dance, its tunes aren’t going to make sense. Our computer model has no idea of what it’s doing. I call it a “parlor trick”: It has statistical machinary fancy enough to model with clear degrees of success (and clear failures as well) many of the patterns found in the 23,000+ transcriptions of real tunes it was trained on. So the computer spits out ABC sequences that its model say are likely given all this data. It takes a trained human to go through that and make it sound right.

If someone records a tune generated by the computer, who gets the royalties ? 🙂

I tried out some of the tunes in the PDF you posted up a while back. (Assuming it was the same project and there aren’t lots of people out there programming tune-generating AIs!) I remember trying out the first 15 or so and finding one or two I liked and actually learned them. But I remember the majority just struck me as a little too random, just going in directions that didn’t feel intuitive. If there was a strong A part then there’d be a B part that just didn’t feel right, for instance. Listening to that album, I get a bit of that too: a lack of memorability, a certain uniformity to the note-lengths.

That said, to be honest, I find most recently composed tunes to be fairly bland and I generally lose interest in an album when I hear it’s a collection of new tunes, so the rate of hits to misses is probably the same with your HAL 2000’s work.

As a matter of interest, did the musicians get to cherry-pick which tunes to play?

@Kenny: All the transcriptions we have created are released Creative Commons Attribution 4.0 (https://creativecommons.org/licenses/by/4.0/) Our album is free for download. Someone has looked at the implications for ownership: https://alj.orangenius.com/the-next-rembrandt-who-holds-the-copyright-in-computer-generated-art/ In my opinion, the output of the computer is raw material, and enough value is added by curation and adaptation by a human to warrant rights to what that person creates.

@Matt Milton: We have also noticed that the B parts are often weak. Be sure to look at how the original output of the computer was changed by Banarsë: http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A1248565&dswid=8361 Sometimes he swapped the A and B sections, or combined the A sections of two different transcriptions. To create the album, we just gave him all the tunes we generated (https://highnoongmt.wordpress.com/2018/01/05/volumes-1-20-of-folk-rnn-v1-transcriptions/) and let him make his own decisions. The task was to create a plausable folk album from the material. We wanted to see how easy that would be. Turns out it isn’t too hard with this material.

Mr. Strum, you seem genuinely concerned about making the most of this project to develop an AI to
generate some good tunes, finding musicians who want to play the best of them and possibly help or inspire
human composers of trad tunes. I appreciate your enthusiasm & hope the best for all who share in the
pursuit of good tuneage. There will be lots of less than ideal results. But you know that & could
probably even give it a percentage.

Out of curiosity, does the computer-generated tunes community have anything like Asimov’s Three Laws of Robotics? I seriously doubt you’d ever need something like that for a project like yours. But as the people
writing tunes after us may look back on these times it might come up ~ “What would Bob Strum’s project do?”

Cheers!

https://www.auburn.edu/~vestmon/robotics.html
In case anyone online was wondering.

@AB: Thank you! We are not on a pursuit to create the best Irish traditional music using a computer, but to create new tools that can augment music creation. In a more technical direction, this computer model provides a case study of how to interrogate machines to discover what it is they have actually learned to do from training data (e.g., http://musicalmetacreation.org/buddydrive/file/sturm). Along the way, we are having a lot of fun working with and learning from practitioners.

It’s an interesting point you raise about Asimov’s three laws. I don’t have such laws, but I have started contributing to a similar discussion (https://transactions.ismir.net/articles/10.5334/tismir.13/). What do you think those laws should be?

Thank you, Bob! I appreciate the article. It’s a good read and I will post a (less tongue in cheek) ’nother response
once I have read it from start to finish. I skimmed it early this morning and began reading in earnest just now.

When I got to the bit about *well-being metrics* I had to respond. The term was not one I expected.
I don’t think I even knew it existed. But I get the concept. In a parallel way I have been asked about
my level of pain after an accident, surgery, etc. It was a daily question at the time. The good news is that,
in the world of pain metrics, a scale of 1 to 10 is wholly inadequate; though emoticons can help
answer an awkward question & make pain seem less so. ~ http://wongbakerfaces.org/wp-content/uploads/2015/06/FACES_English_Blue1.jpg

Later!

I am sorry I didn’t see this discussion when it was originally posted, I guess I was to busy playing real trad Irish music, as opposed to the artificial variety. Please refer to the following discussion: https://thesession.org/discussions/47950?newcomment#comment960495