Hearing sights, seeing music

I wish I had synesthesia!

It's a neurological "condition" (minus the negative connotations), affecting about 4% of the population, in which cognitive and sensory pathways "cross over" one another, leading to an unusual "blending" of perceptions.

Sight gets mixed in with hearing, hearing with taste and so on... One might experience blue as not just a color, but also as a taste; or the sound of a cello as inherently "green", not simply as a metaphor, but because the visual experience of "green" accompanies the auditory experience of the sound in a very real and literal way.

It's fascinating stuff and truly makes one wonder about the possible differences among people's perception of the world.

So if this has sparked your interest, here are two interesting articles on synesthesia:

In this interview, neuroscientist David Eagleman talks about the synesthesia gene and the The Synesthesia Battery, a free online test for synesthesia aimed at promoting scientific progress in the field.

Even more fascinating: consider synesthesia from the perspective of art. Artist and synesthete Marcia Smilack creates photographs and videos to recreate some of her visual-auditory sensations for non-synesthetes. You can read about her in this Seed Magazine article, The Most Beautiful Painting You've Ever Heard.

Also, check out Smilack's portfolio here.

Follow-up review: RjDj

So I finally caved and bought an iPhone, taking a big ole swig from the collective jug of kool-aid. Oh but it tastes so good!!

And, it gave me the opportunity for a follow-up review to my original post about that quirky RjDj app. Quick reminder:

"RjDj takes in sounds from your surroundings (voices, street noise, tapping on computer keyboards...) and plays it back to you as music compositions in real-time."

That's how I had described it the first time around.

Having now sampled it first hand, I'll revise that description. It's not so much music as a stream of psychedelic-sounding repetitions, as if a sound, rather than merely being heard and registered by your brain, instead lingered there a bit longer, bouncing off the sides of your skull and creating weird reverberating echos.

So... music, no.

But it is extremely interesting and weird and fascinating. I had it running while I was unloading the dishwasher, and the clanking of dishes (among my top 10 most annoying sounds) somehow turned into a strangely fitting noise-pop soundtrack to that most mundane of activities.

There's certainly more that can be done with the app, including sampling different "scenes," which are essentially different algorithms for manipulating the sound inputs.

I'll do some more exploring and post a follow-up if I discover anything nifty.

Check out the app here, and my original post here!

Visualizing the Patterns of Poetry

While browsing through Lee Byron's website, another interesting visualization project caught my eye. I've linked to his website before, on a post about visualizing personal audio histories. This one instead is about visualizing poetry in terms of repetition patters.

It always helps that Byron's visuals are so aesthetically appealing -- it makes them even more captivating. Check this one out...


It uses visuals to highlight different levels of patterns in a limerick: large yellow arcs capture the higher-level structure of the poem; narrower purple arcs capture patterns among words, like rhymes and alliteration; the bars along the bottom denote the number of syllables in each line.

If you look closely you notice even more patterns that add complexity to the visual.

I would have loved to see something like this when learning the structure of poems in grade school. Lessons on meter and "perfect" vs. "feminine" rhymes always seems so dreary and hard to remember, especially in contrast to the musical nature of poetry.

Speaking of music... I could see really interesting applications of this sort of visual to music, particularly from the Baroque period -- pieces like Bach's fugues that rely so much on patterns and repetition.

Check out more of Byron's poetry visualizations here.

Soundtrack to your life?

This may be the moment when I finally cave and buy an iPhone.

Possibly the most fascinating iPhone app I've heard of yet: RjDj takes in sounds from your surroundings (voices, street noise, tapping on computer keyboards...) and plays it back to you as music compositions in real-time.

Also amusing is the creators' shameless analogy to psychedelics as a selling point:

"The listening experience of RjDj is similar to the effects of drugs. Drugs affect our sensory perception, so does RjDj. RjDj is a digital drug which causes mind twisting hearing sensation."

Check out this video demo:

Visualizing Personal Audio Histories

I'm always a sucker for a clever data visualization project -- anything that turns an unwieldy mass of information into a clean and intuitive graphic. The "aha!" moment is my favorite.

I'm also a huge music fan so here's one that sparked my interest.
Lee Byon, an information designer at Facebook, created this graphic to map his listening history on Last.fm.


Unfortunately all I have is a partial screen shot, so definitely click through to the article to get a better view.

Here's how it works: imagine time as the horizontal axis; each ribbon represents a different artist, color-coded based on how long the creator has been a fan (cool blues represent old favorites, while warmer colors represent newer discoveries); the width of the ribbon represents the volume of music heard.

What I love is how much information is immediately apparent. First of all, I like this guy's taste in music, but he seems a little boring: he mostly listens to old favorites -- though he seems to have gotten more experimental since. His listening habits tend to come in phases: there are times when he listens to a lot of music and times when he listens to fairly little; he seems very into particular artists at certain times, but they seldom make a comeback once he tires of them.

Would love to see my own audio history graphic, and compare it to those of my friends. I'm imagining a whole world of music histories that flow in and out of each other like liquid, intersecting, affecting each others musical currents.