The Formula by Malcolm Gladwell
Submitted by jim on Wed, 11/01/2006 - 12:27
Tags:
There's a fascinating piece by Malcolm Gladwell up at The New Yorker titled The Formula. In it, he reports on companies that are trying to predict hit songs and movies. The theory is, if you can accurately catalog various attributes of the work, those attributes can map to how big a hit something will be. I figured no chance, but we are a predictable lot, it seems. Consider this:
On his screen was a cluster of thousands of white dots, resembling a cloud. This was a "map" of the songs his group had run through its software: each dot represented a single song, and each song was positioned in the cloud according to its particular mathematical signature.
... and then this:
McCready then hit a button on his computer, which had the effect of eliminating all the songs that had not made the Billboard Top 30 in the past five years. The screen went from an undifferentiated cloud to sixty discrete clusters. This is what the universe of hit songs from the past five years looks like structurally; hits come out of a small, predictable, and highly conserved set of mathematical patterns.
Not only that, they know how to nudge songs towards those clusters. So very cool, and yet also depressing. Are we such sheep? Note that "hit" does not equal "good."
Yes.
I read the article as well some weeks back. I took it to be more evidence of a theory I happened across while studying various courses at once at University some years back (Shenkerian Analysis - Music; Astronomy; Genetics; Anthropology - First Nations of BC).
Perhaps some background information on Shenkerian anaylsis would help here. I have a degree in music, and have studied many ways of anyalsing music. For the most part, Western music fits into pre-existing patterns of form, and are usually written with a a combination of following and breaking the norms of that form. Traditional analysis uses those structures and sees how a piece of music fits into the pre-existing form.
Shenkerian analysis has a different approach. It begins by looking at the piece of music, deciding which notes are the most structurally important, reducing the piece to its barest bones while still retaining its overall shape, and seeing what patterns emerge. What was startling is that all of the great pieces of music that we studied each had its own logic, but that what they all had in common was a uniformity of shape through all levels of the work. What that means is the 3-note shape (for example) that forms the skeleton of the motive looks exactly the same as the shape of the section, the same as the shape of the movement, and the same as the shape of the whole piece. It's a little like fractal geometry, in that at any scale you witness the same patterns repeating themselves.
Fractal geometry, as I understand it, is found in all sorts of places in the natural world. Trees, for example, exhibit fractal geometry in elegant reality. A mathematical pattern defines the beauty of the natural world. Look at the perfect spirals of the inside of a shell, or the overlapping cross-spirals on a sunflower.
So my conclusion while studying the myriad of courses, driven mostly by my studies in Shenkerian analysis, is that the world is governed by predictable, natural patterns. Being members of the natural world, those kinds of patterns please us, and we respond to beautiful things because they embody the perfection of the natural laws of the universe.
So while "hit" definitely does not equal "good", many people respond to that hit song, and what they're likely responding to is a pleasing combination of things that can be mathematically identified. I strongly suspect that if the people who wrote the program knew more about music, and if they could analyse the songs through Shenkerian analysis, they would find the same kind of uniformity of shape and structure that we find in great Classical works. Simpler, maybe (definitely), and surrounded by elements we can choose to like or dislike (artist, style, public opinion), but if those patterns are present, people will respond.
My two cents.
Very interesting, thanks! It still impresses me that they can get such analysis to work for music, but the idea of getting it to work for movies still floors me.
Well it sounded to me like they had a lot less success for the movies. There are too many variables, too many people involved, and movies are not "beautiful" in the way the music is. In all they're a much more complex artwork, and I think whatever algorithms they need would have to be much more complex than the one they have for music.
As for whether the main character wears a hat or not, I think they're barking up the wrong tree, and looking at the wrong things. Create an algorithm for music, one for character, one for cinematography, one for acting ability, , one for costume design, one for screenplay (possibly the hardest), and compound them, then analyse how they interact together. Tough stuff. And what about films that are a success despite the presence of any of those things? What about things like the Blair Witch Project, which came out of left field and shook the nature of horror filmmaking? How do you predict that?
Less success for sure, but that fact that they had any success approaching the problem the way they did is what floored me. The easier it is to boil something down to math, the easier it is to solve. How does one boil a movie down to math?
I'm pretty sure somebody could also write an algorithm to predict Pitchfork "Best New Music" albums. Maybe I should get to work on a Scaruffi algorithm.