Reverse Chromesthesia

I know that I promised you more Ruby / Python / Shell stuff this week, but this project was too cool!

Chromesthesia is a type of Synesthesia in which certain sounds trigger the brain to perceive colors. In other words, some people can literally see sound. Trippy, right?

But how about the reverse: turning color into sound? That’s exactly what Chelsea Valentine and Julie Pan did at Pearl Hacks.

By combining a motion tracker with several other JS libraries, they created Chroma Keyboard, a webcam controlled piano. It plays notes based on the colors it recognizes. For example, you could play an A just by holding up a red apple.

Chroma Keyboard in action

Here’s how the colors map to piano notes: A C D E F G

Try playing your own tune with Chroma Keyboard, (works best on desktop).

2017 Neal Shyam