The Cupertino Shuffle

Listening to a six year old podcast this morning I was thrilled to learn an anecdote that proves the validity of a thought experiment I wrote about in The Art of Search.

What are the chances of that, eh?

What are the chances of me listening to a six year old podcast about the iPod Shuffle®, on the day when the iPod Shuffle is officially axed by Apple®? Well, as it happens, probably not that small at all.

Driving-into work these past few days I've been catching-up on a podcast I'd subscribed-to but haven't yet found the time to listen to. This morning's episode of The Infinite Monkey Cage centered-upon randomness, probability and chance. Amongst the panelists joinging Professor Brian Cox for the BBC Radio 4 show was Alex Bellos, author of Alex's Adventures in Numberland, who discussed how probability makes sense out of randomness. Bellos instanced the tossing of a coin and the random nature of the heads/tails outcome but, as a result of the experimental million tosses of a coin, we now understand that the probability of a coin landing either heads or tails is 50%.

Study something long enough and you can detect patterns

This is the very same point I make in The Art of Search about observing and extrapolating meaning from Google's search engine results. I'm fond of explaining things with metaphors and, for this point, I chose to write about observing algorithmic thinking (such as Google's search engine results) by asking readers to join in the experiment and observe how an iPod shuffles songs. I dubbed the experiment The Cupertino Shuffle. "Continue the experiment by only ever listening to your music on shuffle and soon enough you will crack because the algorithm will have got caught in a mathematical loop that favours recently played music and what will shuffle into your ears will be the same old music by the same old artists; then there will come a time when you wonder why the shuffle function wasn't actually called groundhog instead."

Anecdote proves validity

Imagine my delight this morning as the podcast's panelists discussed the evolution of the iPod's Shuffle algorithm in response to the human reaction to randomness. Here's a transcript from the podcast:

There's the interesting anecdote about the Shuffle feature on the iPod, people were getting the shuffle feature and were saying: 'Hey, it just plays songs from the same album; I have like a 1,000 songs, how come it always plays songs from the same album?' Well, you would expect clusters to happen, you know birthdays happen together, the Shuffle to choose things together, so people complained so Steve Jobs said I'm going to make and he did, he made the iPod Shuffle less random so people thought it was more random. Alex Bellos

Google's just maths

A point that so many people seem to miss, when it comes to SEO, is that Google's just maths. Google crawls your website and calculates the probability of what any given webpage is about a subject based upon the content its robot discovers and the popularity¹ of the page in question. Having crawled a webpage it summarises what Googlebot calculates your website to be about with keywords and stores this summary in a process known as indexing. Once Google has a summary of your web page it can then compare the significance of this web page against others it has found exhibiting similar keywords; the outcome of this comparison we know to be a rank.

Just like the subatomic world of the very small explored by quantum mechanics or the Enigma machines encoding messages between German WWII forces, we can't see the maths that makes Google what it is; but we can see its output and, if we study it long-enough, we're able to recognise patterns from which we can deduce function. With a sense of how the engine works we can develop and test theories. This approach, this scientific method, to securing Google-visibility for your website is what SEO is all about.

I first wrote about The Cupertino Shuffle in The Art of Search; I had no empirical evidence, no solid proof that an iPod Shuffle algorithm existed, just a gut feel born of experience derived from curiosity-based research. The Cupertino Shuffle exists as a thought experiment about algorithmic search engine results because the physical experiment may only be performed for as long as the iPod shuffles. Having published and referenced a thought experiment, I felt it necessary to go check the validity of Bellos' anecdote before publishing this blog post and, although I couldn't find an authentic and reliable source to quote for the Steve Jobs anecdote², what I found was a similar experience and solution from an employee of Spotify.

No. It used to be perfectly random (using the Fisher-Yates shuffle algorithm) but the problem is that to humans, truly random does not feel random, so we got tons of complaints from users about it not being random.Mattias Petter Johansson

The algorithm exists - get over it

Once you accept that Google's algorithms exist, once you cast-off any paranoia that you may be holding-on-to that the big, bad, nasty search engine algorithm has it in for you and is out to get you and once that you accept that, in the grand scheme of things, your website is tiny relative to the size of the web and the scale of Google's self-assigned mission³ - only then will you realise the value and the importance of thinking like a robot and experimenting with your own website.

• Alex Bellos - Alex's Adventures in Numberland
• The Infinite Monkey Cage - BBC Radio 4
• A Pain In The RSS - a Sub@omic blog about the beauty (and wonder) of data feeds