Musical Turing Test

I currently TA for an undergraduate Issues in Computer Science class, and during one of the lectures, the professor talked about a computer's ability to create art and music—which, one might suppose, are very "human" skills. Can a computer compose music? Sure...but it won't sound quite right. Right? In class, the professor tried this with two composers: Bach and Joplin. For each of these composers, she had a computer program generate ("compose") a musical piece after it had identified patterns ("learned") from a lot of music from that composer. She then recorded a pianist playing the computer-composed piece, so a piece generated by a Bach-learned program would sound a little something like Bach, and a Joplin-learned one would sound quite a bit like your typical Joplin rag.

I won't bore you with any more details (there are a few more at the actual test—you should take it, it only takes 7 minutes tops), but the results were interesting. During class, the teacher juxtaposed each computer-composed piece with an actual human composed piece, and then asked the students to pick the real piece. The performance was terrible. It was clear these students couldn't tell the difference between a great like Bach and a computer imitation of Bach. But then I began to wonder: as I took the test, I was able to pick out each of the real compositions. Was this because of my musical background? I decided to give the same test to a larger audience, this time recording their self-reported musical experience.

If you're not interested in the individual results and discussion, the takeaway is that, even among self-reported musicians, only just over 40% of the individuals were able to correctly identify the human composer. Yes. Only 40%. In other words, more often than not, humans think computers are more "human" composers than Bach or Chopin or Joplin. This is a terrible result! (There's a pretty good description why this is bad in this RadioLab episode, at about 16:26.) I'll try and go into it more in the Analysis section below.


Population Distribution
For those of you interested in the questions and population distribution, I've been updating the values here. For your information, the quiz consisted of 6 questions: three background, and three with the paired audio samples. For the first three questions, here are the number of respondents (total of 247) in each category:

1. What is your major field of work? The responses were free-form and varied from math to computers to culinary arts to music to motorcycle repair. I haven't looked at any of these responses past collecting answers.
2. Do you play any instruments? This was multiple choice, divided into 4 categories of increasing "musicality." Here are the number of respondents that self-selected each category:
1. No: 28
2. Yes, but I've picked it up on my own without lessons: 27
3. Yes, and I've had official lessons for at least one year:  131
4. Yes, I would consider myself a professional musician: 61
3. What is your technical musical background? This was also multiple choice with 4 categories, and had the following number of respondents:
1. None: 38
2. I know some musical theory, but I've picked it up on my own: 42
3. I've taken a class or lessons in musical theory: 124
4. I'm a professional musician or majored in music theory: 43

Categorical Results by Response
And here are the responses, separated into response for each category

Second question, accuracy by response:

1 - 27.4%

2 - 40.7%

3 - 42.0%

4 - 50.3%

Third question, accuracy by response:

1 - 32.5%

2 - 42.9%

3 - 40.9%

4 - 54.3%

Average total accuracy:
42.2%

A graphical representation of the responses, by category. The x-axis has the question number, and the y-axis has the accuracy. The circle size is representative of the number of respondents for that specific question.

Overall Results and Analysis

The overall accuracy for all three composers was 42.2%. In other words, only 42 out of 100 times were people able to correctly identify who the human composer was. There probably aren't quite enough samples to show statistical significance, and the survey process wasn't exactly accurate (I didn't prevent anyone from cheating—they could take the survey as many times as they wanted), but there does seem to be a general trend that with musical experience, the accuracy increases.

This is what I expected.

Almost.

On closer inspection, this all is actually quite disconcerting. With the sole exception of those who are professional musicians, humans think that the computer composer actually sounds more human. How could this be possible? Well, I can think of a couple of possibilities.

First, it's possible that there was a hidden variable we weren't controlling for. I had my roommate take the quiz first, and he got them all right. I asked him how he did it, and his response was that he just picked the better recording each time. Oops. When I designed the quiz, I already had the mp3 files for the computer composers, but I was lazy with the real compositions and just ripped the music off YouTube clips. I went back and actually purchased the music off iTunes so they would sound more similar. When I sent the survey out to a larger audience, there were more issues. Turns out iTunes adds album artwork, which was displayed in some browsers when they played the music. Oops again. This was a little harder, but with some online conversion and VLC magic, I finally cleared them all of their extraneous information. Perhaps there is something more I'm not controlling for—if this were real science, I'd want to have the same performer play each piece and be recorded with the same device. Good thing this isn't real science, right?

But there's another possibility that seems more likely. When the computer "composed" a piece by Bach, she wanted it to sound as much like Bach as she could, so she consciously added bits and pieces of actual songs. But when Bach really wrote his pieces, his goal was to create something new and fresh, something that didn't sound like anything he'd written before. Something that people would clearly identify as Bach--or would at least recognize it when they heard it before. All of his similarities to previous works were subconscious, influences from his schooling, the music he enjoyed listening to, even some of the music he previously composed.

Conclusion

With high statistical confidence, humans think that a computer who is trying to imitate a Bach piece actually sounds more like Bach than the composer himself. Maybe this is bad (computers will eventually replace composers and artists and singers and all other "human" fields), but maybe, instead, this identifies an aspect of talented human creators: the ability to create something truly unique.

Notes:
Note1: As people take this test, I'll update the results on this page. Hopefully I replace them all so it makes sense. Also, if you want some more statistics, I can try and get them to you—or I could probably give you the data myself, it's pretty well-anonymized.
Note 2: The first numbers were published on 23 Nov 2013 with 65 respondents and 42.6% accuracy. The results were updated on 26 Nov 2013 with 99 respondents and 40.7% accuracy. They were updated again on 11 Dec 2013 with 201 respondents and 42.8% accuracy. They were updated on 12 Sep 2014 with 247 respondents and 42.2% accuracy.

Back it up

My hard drive just crashed, completely unexpectedly. I have a high-end MacBook Pro that I only bought in 2011, so it was well under the typical 5-year life of most spinning drives. I learned two lessons from this experience:

1. Get the Apple Care. If you're buying Apple, it's one of the best add-on purchases you'll make. It gives you three years of protection (most companies only give you two), and it's only a fraction of the cost of your entire machine. The amount you spend will definitely be justified by your peace of mind. Apple replaced everything for free--and did it all over the weekend (I went in on Saturday and was out by Monday. The lag was because I had customized hardware). Very fast and efficient.
2. Back up your devices. While waiting in line to pick up my laptop, I stood behind an older gentleman who was obviously distraught after losing data on his phone--probably in the form of non-replaceable pictures of his grandkids. Unfortunately, there isn't much you can do after the fact, so prevention is the only way to go. Get a large external drive, something like the 3 TB Seagate drive I have (on sale, it's just over $100--definitely worth the hours of time you'd spend rewriting all your papers). If it's this big, you can also use it to store any pictures and music and movies that don't fit on your machine. Make sure you set it up to remind you to back it up every 10 days or so, or you'll forget and will have wasted the $100.

Do it. You won't regret it.

And to help you remember, I've included this wonderful song: 

Le Scaphandre et le Papillon

I watched a movie the other night, Le Scaphandre et le Papillon. It's about Jean-Dominique Bauby, journalist and editor-in-chief of Elle magazine. He suffered a massive stroke that left him with "locked-in syndrome:" he could think, hear, and see, but couldn't move anything but his left eye. Ten days after he finished his memoir (after which the movie was named), he passed away.


A few weeks ago, my right arm started to ache. At first I thought it was carpal tunnel, but the symptoms pointed more toward tendonitis. I looked around on Google hoping for an easy cure (something like, "do arm yoga every morning for 15 minutes") but was sorely disappointed when all sources agreed that "the first stage toward recovery" was to "stop the repetitive actions that caused the tendonitis for at least 3 months." How does a computer science PhD student doing an internship at Google stop typing?

Although it's nowhere near the life-altering condition of Jean-Do, I wondered what it would be like if suddenly my life were completely and irreversibly turned around. Like the construction worker who breaks his back in his late 40s, or the medical doctor who goes blind.  What if I could no longer type? Or even worse--what if I could type, but I'd be miserable the rest of my life unless I chose not to?


Maybe I'll pull an Angelina Jolie an preemptively become a genetic counselor.

On qualification exams

A part of me died last night.

It wasn't really a big deal. Really. I'd been planning for a month or so to give a presentation today–a sort of oral qualifying exam, the only of its kind in my PhD program. I'd been over my slides several times this week with my advisor, and then he finally sort of snapped. The conversation went from "I don't understand this slide" to "You need to do it my way," and then finally ended with a decisive "You're not ready for your presentation tomorrow."

Perhaps his decision was closely linked to our "discussion" earlier, the discussion that started out with ridicule, then proceeded quickly to a heated argument and ended with him walking out saying, "Maybe you need to find someone else willing to bring forward your research ideas." I watched him go in a quandary:  Do I apologize, let him know I'll change my slides to match his desires, or do I just ignore him and go forward with the presentation? He's fairly fickle, and if I could catch him in a good mood, I was confident I could get him to pass me. Besides, I would be presenting to a committee of three professors, and I'm guessing the other two wouldn't have any emotional attachment to my slides.

But then I decided to bow out. I'd had such visions of grandeur up to this point. Coming to graduate school was going to be the best thing I'd done. Sure, it might be hard work, but no longer would I be surrounded by incompetent undergraduates, no longer would my nights be filled with busy work just to get the grade.  Instead, I'd be in an environment surrounded by high-energy, deep-thinking scholars. The only problem I would have is finding enough room on my CV for all the publications I'd be pushing. There would be no drama, no posturing for positions or fighting with the "wunderkind" for an advisor's attention. Just research, and making an impact on the world.

And then I let that dream die.

It was slowly replaced with a kind of saddening understanding. Maybe a PhD program is something akin to the hazing process required for a college fraternity. At times the requirements might not make sense, and they might have nothing to do with the end goal, but if you can only get through the hazing, you'll be accepted by the group and given a lovely PhD diploma to hang on your wall.


Now I have only one question for myself:

Is this really what I want?

MR-ADAM

Dear C++ MapReduce:

All your base are belong to us.

EOM

A day in the life

Sometimes I love my life.

For example, today.

It quickly became apparent that my cubicle-mate was eating lunch because 1) the overpowering smell of something spicy and very oriental swelled over the dividers, and then 2) the loud and incredibly aggravating probably-culturally-acceptable-in-some-other-part-of-the-world sound of him eating his soup kept time with the clock.

I took a break to use the bathroom, passed the bike in the walkway that hasn't moved in months but instead is defiantly challenging the passive-agressive "Bikes are not allowed in cubicle areas" sign just feet away, turned left at what used to be the US army's "I want you..." poster with "...to stay quiet" below but now reads "Help Create a Culture of Acoustic Courtesy" (I only wish that phrase were enclosed in quotation marks or asterisks), and finally got to the bathroom.  I took care of my business next to a lot of ruckus in a stall, then washed my hand as the now-finished ruckus-maker slurped water from the sink (how he found this more sanitary than partaking from the drinking fountains just outside is beyond me).

And then I got back to my computer where I should be working in Lisp on Recursion and Induction homework but instead am trying to figure out at what sampling rate I need to save an mp3 file so an incorrectly-sampled video can have the audio synced with the speaker's lips (if I reduce the rate to 45960Hz, by the way, it looks nearly perfect).


Swell.  All we need is some of the cribbage players to start up their rousing game in the corner with the cot and one of my other cubicle-mates to start talking on the phone in a language that sounds like it's using the N-word all too frequently.


If that doesn't make you envious, I don't know what will.

IEEE Journal Publication

If you're like me, it's relatively late at night and you're trying to submit a camera-ready paper for a conference.  You've used LaTeX so it fits the formatting requirements exactly.  Or so you thought.  Upon trying to jump through the PDFExpress hoops, you get the dreaded:

✘ Error   Font ZapfDingbats is not embedded (40x) 

Exasperated, you search the Internet and find a "brilliant" solution.  It's a couple of steps long, and it seems like it does the trick.  You plug along, resubmit the file, and sure enough, there are no embedded font issues.  But this time, it says some mumbo-jumbo about the file being too large.  Right about now, you start pulling your hair out (it took 90 minutes just to conform to IEEE standards, another 2 hours to find the "brilliant" solution, and it's now 15 minutes past the due date for the conference submission).  Another hour of searching, another step closer to the solution:  Look in Adobe under the "File Properties" menu, under "Fonts" to see if all the fonts are embedded.  Nope.  They aren't there.  There's an interesting article here about changing some of TeXShop's internal commands to try and rectify the situation.  But that doesn't do the trick in your case.

So here's the answer, and you can put your hair back in.

Sometimes, when including external figures/tables/whatever, the figure-producing program doesn't embed the fonts, which are then not embedded in the final PDF.  In my case, I use R to create figures, and the captions or something must be causing some severe issues.  Sure enough, looking at these PDF files individually with Adobe confirms my suspicion.  After a little more digging (thanks A Blog for your genius), I found that you can imbed the fonts using a dirty hack involving changing the file to an eps file and then back to pdf.  This doesn't change the picture quality, but does embeds the fonts.  Here's the code from the website, changed a little bit for Mac commands (I just changed pdftops to pdf2ps):

#!/bin/bash
export GS_OPTIONS='-dEmbedAllFonts=true -dPDFSETTINGS=/printer' cp $1 $1.old pdf2ps $1 tmp.ps ps2eps tmp.ps
epstopdf tmp.eps
mv tmp.pdf $1 rm tmp.ps tmp.eps

Run this program on all your external PDF files (it won't increase the file size significantly—in fact, in my case it decreased the file size), and then re-typeset with your favorite pdflatex editor.  And voila, you run the file through the PDFExpress PDF checker and you get:

Status*: PDF Passed PDF Check; PDF is IEEE Xplore-compatible

That's music to my ears.

Parallel Algorithms part XXX

Disclaimer:  If you have not taken a course in complexity theory of computation (or if you have a life), you might not find this post all that entertaining.  Just be warned.

I'm in a Parallel Algorithms Theory class right now, and since I've already taken a few classes in parallel programming and had lots of experience, I thought it might be insightful.  Unfortunately, I learned the truth about the class today and am still trying to find the missing link to reality.

It seems that there are three general steps to creating a "work-time optimal" algorithm.  These are, in order:

1. Assume an extremem number of processors, such that p>>n. This works best for doing something trivial, such as searching a sorted array for a given element. n can be any arbitrary number between one and the size of an integer (around 4 billion).
2. Create a convoluted algorithm such that:
• The number of parallel steps is very small (Try getting close to O(log log n) )
• The number of steps required to orchestrate parallelization and setup is significantly large (preferably close to n²)
3. Hand-wave and use several mathematical approximations with Big-Oh to show that your new algorithm is actually constant time O(1).

Once this has been done, publish your results and teach a Parallel Algorithms course.