Description:
July 2014
A sound toy presented as an art project at the Singapore mini-maker faire 2014, ‘The Synesthete’s Music Machine’ is a collaborative group effort by my LASALLE ex-classmates and I to create an object inspired by the idea of synesthesia. A synesthete is someone who experiences a secondary sensation of colour when he hears sound or a sensation of hearing sounds when he sees colours. Our sound toy is able to translate any image seen by a webcam into a music sequence based on the colours in the image.
The user places physical objects of different colours (e.g. lego) or static A4 sized images on a specially built white surface. The camera acts as an “eye” and passes on the image data to a computer program that scans the image from left to right, triggering sounds that have been assigned to the colours by the user. In this way, the user can experiment with different objects and sound choices to create their own music sequence. The software is coded in Javascript and runs on any non-IE browser. The user is able to see the camera image and scanning process in real-time on the browser, as well as choose the sounds. You can run the program here. The source code is publicly available here.
I wrote a blog entry about the behind-the-scenes process of making it here.
Images:
Video Documentation:
lohjianhui 