SoundMachine is a LEGO drum sequencer that “builds” music using standard LEGO bricks. SoundMachine is an interactive music tool to collaboratively develop music. SoundMachine scans 2×2 colour LEGO bricks arranged in 4 tracks of 8 beats. The colour information is converted to MIDI messages by a Processing sketch, which sends MIDI to Ableton Live to play the instruments.
I created some videos that show SoundMachine in action. You can see how the notes are built on the plate, scanned in and transformed into music.
The idea for SoundMachine came to me when I was browsing the Processing website and came across a project named “SoundMachines” by a Berlin design studio named The Product. I was immediately struck by the simplicity and elegance of their 3-turntable design, and started to replicate it in LEGO. Over time I achieved a simple working prototype, but was never satisfied with the reliability of what I built. It was time for a re-design.
I came across the Beat Bricks project which was a step sequencer that used a web camera to read 2×2 bricks placed on a 32×32 base plate. The brick locations and colours were converted into MIDI messages. The image analysis and MIDI software was written in Python, and no LEGO NXT was involved. This got me thinking… what if I replaced the video camera with LEGO colour sensors and used a NXT to scan the plate with the bricks???
How it works
SoundMachine is composed of a number of components that all work together:
A SoundMachine scanner box. The SoundMachine scanner scans the music from a LEGO plate and transmits it to my laptop via a USB cable. Multiple SoundMachine units can work in parallel, and each unit can contribute notes to an evolving musical score.
The scanner array is mounted above the plate drive, each scanner unit contains a LEGO Mindstorms NXT which drives a NXT motor to move a plate under an array of 4 LEGO NXT 2.0 colour sensors. As the plate moves under the sensors the colour of the bricks are detected.
A plate is used to “build” the music tracks. The plates are regular blue 32×32 LEGO base plates, and the notes are LEGO 2×2 bricks. White tiles are used to separate the notes on the plate, and the sensors use a black reading to detect the start and end edge of the plate. The standard LEGO colour sensor can detect 6 basic colours, leaving Red, Yellow and Green to encode notes. The notes are arranged in 4 parallel tracks aligned with the colour sensors.
A laptop running Ableton Live (a live music composing and performance program) and Processing (an interactive Java-based programming environment). The laptop plays the music (ideally through attached speakers) and attaches to the SoundMachine scanners using USB cables.
Ableton Live is a well-known music composition and performance program. Used by DJs around the world it is ideal for create live sets by building music loops on-the-fly. Ableton contains a library of thousands of instruments, which are triggered using MIDI messages. Ultimately the sound made by SoundMachine comes from Ableton.
A Processing sketch (essentially a Java program) reads the colour data from the SoundMachine scanner and converts the colours into MIDI messages. The Processing sketch is essentially the “switchboard” in the middle, managing data from multiple SoundMachine scanners, forwarding data to Ableton Live and providing basic play/pause functions. The sketch displays an animated visualisation of the notes that have been read and are currently playing. The sketch provides a simple save/load function to save note data locally.
SoundCipher MIDI library. SoundCipher is a Processing library for playing music and sending MIDI messages. SoundCipher is used to create MIDI NoteOn and NoteOff messages to send to Live.
I’m continually experimenting and adding new features to the code, but here is a working snapshot of what I’ve written so far. SoundMachine consists of a program running on the NXT written in Java for leJOS, and a Processing sketch running on my laptop (in my case an old MacBook).
- BlockSequencer.java: This program runs on the NXT and requires the leJOS 0.9.1 firmware installed. It uses the USB port to send data to the laptop.
- BlockSequencerController.java: This is the Processing code that I developed in Eclipse using the Proclipsing plugin. It uses the leJOS pccomms.jar library to communicate with the NXT. This in turn requires the LEGO fantom driver to be installed. Contact me if you have questions on how to set this up.