Jeremy Kaplan
Mondegreens are not a garden in Europe. The word sounds odd, but it refers to a problem that occurs with music lyrics. When people hear songs, they often hear the words they want to hear versus what the songwriter actually wrote. No surprise, people can come up with some pretty interesting versions of the same song, lyric-wise. With transcription, however, accuracy needs to be high. So we did a little test between the famous IBM Watson, a venerable automated think tank of computation, and two profession transcriptionists.
Challenge #1: “Taylor Swift’s “I Knew You Were Trouble”
IBM Watson’s version:
Once upon a time a few mistakes i go
I was then you say. You got me a loon.
You’ve gotten me
It was an interesting attempt, but the final tally on this short blurb was 4 errors and 6 words missing, not a very good score at all.
Professional transcriptionist’s version:
Once upon a time, a few mistakes ago,
I was in your sights. You got me alone.
You found me. You found me. You found me.
Clearly putting the computer to shame on the first challenge, the transcriptionist picked scored 0 errors and 0 words missing, a perfect 10.
Challenge #2: We pepped things up with a bit of a power song with Van Halen’s “Running With the Devil”
IBM Watson’s version:
I live my life lacked is.
And no I’ve got. i.
The startled me truck bed through barlow.
You have some live n. n.. Please start.
Apparently, IBM Watson doesn’t like hard rock. Maybe it was the guitar playing or the drum beat, but the computer bombed out with 9 errors and 15 words missing. In short, the song was slaughtered.
Professional transcriptionist’s version:
I live my life like there’s no tomorrow.
And all I’ve got, I had to steal.
Least I don’t need to beg or borrow.
Yes, I’m living at pace that kills.
Again, the score was a perfect 10 with 0 errors and 0 words missing. This is getting embarrassing for Watson.
Challenge #3: So to ease things up a bit, we switched to a classic rock song with Elton John’s “Tiny Dancer”. Here’s what the results were:
IBM Watson’s version:
Blue gene bear bear yeah
Lazy lately yeah
Seamstress for the man.
Either Elton was on drugs in this version (maybe maybe), or the computer was listening to another song entirely. A dismal score of 8 errors and 3 words missing was produced by Watson.
Professional transcriptionist version:
Blue-jeaned baby,
LA lady,
Seamstress for the band.
Brutal! 3-0 scores in challenges and 0 errors and 0 words missing. It’s time to call the challenge early, no?
Challenge #4: But wait, we decided to throw in one more song with a country tune. Ideally, the slower song pace should give a better chance of comprehension. Here’s Brad Paisley’s “Ticks”:
IBM Watson version:
Hey every time you take a sip
And this smoky atmosphere
You press at bottom to your live in peace
And I wish I was you will be here.
Ouch, ouch ouch. Watson had too many beers in his circuits with 12 errors and 5 words missing.
Professional transcriptionist version:
Every time you take a sip
In this smoky atmosphere,
You press that bottle to your lips,
And I wish I was your beer.
A perfect run with yet another score of 0 errors and 0 words missing. ‘Nuff said.
