Hacking a wii sensor bar

So I needed a wii sensor bar that looks nice enough to be deployed in a commercial installation for a big client in the USA. Previously, for a similar installation in Malaysia, it was my first time doing it, so we proposed a design made up of wood that looks like this:

first version
first version

It was big and unwieldy, and blocked the screen partially. After this installation, we were asked by the same client to deploy this in the US. I was not happy with the wood box design and wanted something smaller and sleeker. The next thing I knew, my wonderful colleague saw this at a shopping mall in Singapore:

The Memorex sensor bar
The Memorex sensor bar

It was perfect! It is small and sleek. Just what I wanted. Unfortunately, there were two veto factors:- firstly, it ran on batteries. This meant replacing the batteries everyday for the week long installation. Secondly, and this is worse: it has a compulsory sleep function that would turn it off automatically every hour or so. This is a big no-no in a public installation.

So, the only way out, was to HACK that thing. Override the sleep function and make it run from wall power.

Hacking is never a guaranteed success. Sometimes, there is no way to hack something cos of various reasons, usually because of physical form factors, such as a lack of space to put in your own parts. Or sometimes, the electronics do not allow you to interface with the built in lights or sound outputs.

Luckily, in this case, the electronics was simple enough cos it’s simply a bunch of infrared LEDs. And I managed to cut away some plastic walls to create space for my own circuit components. Oh, I fried a couple of LEDs in the process, but I just replaced them with similar ones I already have in my inventory.

So here are the steps I went through. First of all, I carefully disassembled the thing to investigate its internal structure.

Disassembly
Disassembly

I decided to use the existing PCB, and create my own circuit on the other non-printed side of the board, cos the PCB board fits nicely into the casing, and I didn’t want to cut out my own board. Well, if I had a 3D printer, that would be another story, but that also involves the time-consuming task of drawing it out in a 3D program and hours spent printing. So I just went along with what was already there.

Hacking away literally
Hacking away literally

I hacked away the battery compartment wall to free up space.

More space!
More space!

More space freed up.

Power and ground line
Power and ground line

Power and ground line.

A simple LED circuit
A simple LED circuit

It’s actually a simple LED circuit.

No more battery compartment
No more battery compartment

What used to be the battery compartment.

Hot glued
Hot glued

The wonders of hot glue. I might be a bit too addicted to it.

Finished product
Finished product

Upgraded version. I hooked it to a 5V wall adaptor.

Two weeks later … …

Here are pics of the setup at Texas, USA :-

front view of sensor stand
front view of sensor stand
back view of sensor stand
back view of sensor stand
whole setup
whole setup

Singapore’s makerspace

I dropped by the One Maker Group (OMG) prototyping lab at the National Design Centre this week. I spent a whole month at this space last year making something for the Singapore maker faire. It is a maker’s den with many tools such as laser cutters and 3D printers.

workshop tools
workshop tools

I caught up with my friend who made some sort of casing for a mobile phone out of laser cut cardboard.

Mithru's cardboard structure
Mithru’s cardboard structure

There are still workshops, meet-ups and prototyping going on, but with the new addition of one humongous laser cutter that belched smoke.

huge-ass laser cutter
huge-ass laser cutter

I am glad that there is such a space for people like myself who appreciate the convenience of having all the machines and tools in one space, and a conducive environment with like minded individuals and consultants on site to work with. I do wish however, that the space could be bigger, but I also understand the cost constraints with land being so expensive in Singapore.

The maker map of Singapore
The maker map of Singapore

You can see a picture of me and my teammate in one of the photos on the lower right, below the maker map.

OMG is at the heart of Singapore's maker movement as well as Singapore's city centre
OMG is at the heart of Singapore’s maker movement as well as Singapore’s city centre
laser cutting
laser cutting
3D printers
3D printers
electronics lab
electronics lab

The electronics lab is my dream workspace. I have a smaller version of this at my own work place.

Deploying the wiimote mouse at the Petronas Towers, Kuala Lumpur, Malaysia

So after spending a few weeks researching on the wiimote mouse and creating a prototype, lo and behold, the powers that be decided to deploy it at an actual event in Malaysia for Petronas, in conjunction with the F1 race going on. While I was glad that my research was actually being used, I didn’t have time to stress test my prototype as I was rushing other projects before being sent to Kuala Lumpur. Needless to say, I was worried and stressed. Especially for the first few days I was there. I hardly slept the first three days as we had to work overnight while the Kuala Lumpur Convention Centre (KLCC) was closed at night, and we had to wait for the fabricators to finish the booth structure before we could deploy the PCs. The rush to pack the night before my flight and the unaccounted excess baggage was a bit of a nightmare as well. Not to mention a last minute mis-communication about the screen size which resulted in last minute changes to the wiimote sensor bar design.

Me setting up the sensor bar
Me setting up the sensor bar
Whole setup
Whole setup

After installing “Paint My Car” – paint a virtual car on the huge screen using the wiimote, I had to stay for 12 hours a day on standby at KLCC on technical support standby. Nothing bad happened requiring my attention so I got bored  and did some shopping. I also caught three movies, all the while checking my phone in case of calls for technical help.

I had a nice view of the Petronas Towers and the movies and shopping was a form of therapy. I had excellent rapport with my colleagues and best of all, everything went smoothly despite all the initial stress and anxiety. And now I have a notch in my belt to brag about, documenting this project here.

View of the Petronas Towers near the Petronas tentage where the installation is
View of the Petronas Towers near the Petronas tentage where the installation is

Proof of concept: Arduino Ethernet shield

Arduino Ethernet shield
Arduino Ethernet shield

I was asked to do some research on the Arduino Ethernet shield for potential uses in interactive installations. So I spent a couple of days learning about it and creating a proof of concept working prototype that shows a two way communication of data via the Ethernet between a PC and an Arduino.

Strings and numeric data are used for two way control of a strip of LED lights. You can control it via hardware controls on the Arduino, or use a simple Processing GUI I created on the PC to send commands to it via the Ethernet. You can type a string command to change the colour, e.g. “red” and then press ENTER to send the command to the Arduino. You can also use the arrow keys on the keyboard to adjust the brightness of the lights. The GUI shows the state of the lights – its colour and brightness, being updated via Ethernet from the Arduino.

Here’s a short video showing how it works:

The Ethernet shield can be used in Internet enabled installations that require remote control and sensor feedback of hardware, since it enables internet connectivity to Arduino-controlled hardware via an Ethernet protocol.

Deploying a positioning sensor at the Singapore Science Centre

This was my first deployment for an installation at my new job and it proved to be a good learning experience.

We had a positioning device deployed at the Singapore Science Centre on which a laser pointer was mounted. We needed to know where the laser was pointing at, at a scene in front of it, so that the screen application can conjure up the appropriate information based on the target being pointed to.

The device has a screen and laser pointer mounted as seen here:

Climate change interaction device
Climate change interaction device

 

At the actual site:-

Climate change exhibit
Climate change exhibit

Depending on where you are pointing at, the screen will show the amount of carbon dioxide emitted by a ship, truck, cow, power station etc, on the mounted screen on the device.

This deployment was memorable because I had to crawl underneath the wooden booth full of dust, cobwebs and in pitch darkness to do some wiring and adjustments. So this was when I learnt that a good torchlight is essential. Also, I learnt about the 5 meter limitation of USB cables, beyond which you need a powered extension. I also helped solder the sensors and wire the electronics inside the casing.

All in all, I am glad that my first deployment went well and we were able to troubleshoot and solve problems on the spot as a team, with me as the electronics guy,together with a programmer and mechanical specialist.

Hacking a Hasbro toy lightsaber

As part of work research, I was asked to hack a Hasbro toy lightsaber. It was the first time I tried hacking anything at all and I considered it a minor miracle that I succeeded. By hacking, I mean actually replacing the entire electronics, LEDs and the logic chip, and replacing them with my own micro-controller and programmable LEDs while keeping the form and casing intact. I also added an extra button and battery light indicator.

In the spirit of open source sharing, I am blogging about my experience here.

This is the toy model that I used:-

darth vader lightsaber
darth vader lightsaber

First of all, I de-soldered the original LEDs which were single colour red LEDs and replaced them with two strips of WS2812 LEDs or more commonly known as NeoPixels by Adafruit. The two strips are arranged back to back. Each strip gives out its light in a 120 degree angle, so there is a dark spot of 60 degrees on the left and right but it cannot be helped. Also, the spacing between each LED is quite big since the density is 66 LEDs per metre. Again, this is due to power constraints since I decided to use the original power supply of 3 x 1.5V AA batteries. I could have upgraded the power supply to a Lithium ion battery used by smart phones that have the highest power density among portable power supplies. But I didn’t. Well, maybe this could be a future improvement.

stripped away LEDs
stripped away LEDs

That being said, the replacement NeoPixel LEDs are truely an amazing innovation. You can program them to any colour you want and configure their animation to any pattern you want: blink, fade, ripple effect etc. You can program thousands of these LEDs just using two Arduino pins! The only constraint would be the power supply since each LED would draw 60mA at full white light power.

Close up look at the Neo Pixel strip.
Close up look at the Neo Pixel strip.

This is the original logic chip which I did not remove because I needed it as a structural support since the struts fit nicely into the light diffusion translucent plastic sheath of the blade.

the original logic chip
the original logic chip

So I stuck my own logic chip on the back side of it. I used the SparkFun Pro Micro microcontroller as the brains of my product. I chose it for its small form factor, having to squeeze it into the tiniest of spaces.

The SparkFun Pro Micro top view
The SparkFun Pro Micro top view
The SparkFun Pro Micro
The SparkFun Pro Micro

The microcontroller runs on 5V. However my battery power output is 4.5V max and this voltage output would deteriorate over time as the batteries drain. So I added in a voltage regulator chip – the Pololu 5V step-up/step-down chip. It is very small and would fit into the crevice in the lightsaber handle. How it works is that you feed it any voltage in a range, say 3-12V and it would regulate it to 5V exactly. The efficiency is pretty good as well. I think it is upward of 70%. This means that the lightsaber power supply would perform consistently until the batteries fall below 3V or something, which would mean the lightsaber would suddenly not light up, and you know you have to change the batteries.

The Pololu 5V voltage regulator
The Pololu 5V voltage regulator

Look at how tiny the Pololu chip is!

As with any electronic prototyping process, you should always test your logic circuit first on a breadboard before soldering all the connections. I tested my programming code for the lightsaber in this configuration before soldering all the connections.

Breadboard testing
Breadboard testing

Now I will move on to the buttons. The original button can be seen below. It looks like there are two buttons but they are actually one switch only.

The main switch
The main switch

I needed another switch so I can enable toggling between different light animation effects. The original main switch would be to trigger on or off each light animation effect, thereby “freezing” the animation, if say, you only wanted that particular colour without the light animation. This is because each light animation is associated with a different colour. So for example, yellow is for a pulsing effect, blue is for a ripple effect etc. The following describes the various modes which you can see in the video at the end of this post.

1. Red – “Health bar mode”. Pressing the main switch would switch off a few leds at a time starting from the tip of the lightsaber.

2. Orange – “Pulsing mode”. The lightsaber would throb. Fade in and out.

3. Yellow – “Chameleon mode”. The colour would gradually change across the colour spectrum. Click the main switch to freeze at a colour that you like.

4. Green – “Glitch mode”. The lightsaber would randomly show a flicker effect running up or down its length.

5. Blue – “Running lights mode”. The lights appear to be continually running up the lightsaber.

6. Violet – “Sparkly mode”. The lights randomly fade in and out individually.

A few observant ones of you out there may notice that the order of the colours follow the ROYGBIV mnemonic for the order of colours in the visible colour spectrum. I left out Indigo cos it looked too close visually to Blue and Violet.

Adding an additional switch
Adding an additional switch

I made a hole for the additional switch by using my hot soldering iron to burn a hole into the cheap plastic mold of the lightsaber handle. Not very professional I know and even a little hazardous to one’s health. But hey, hacking is about making the best of whatever tools you have to get the job done.

For good measure, I added a green LED to indicate whether the battery power is switched on or not. When I was young, I would sometimes forget to turn off the battery switch for my toys, only to find that the battery has drained the next day. So this LED will be on if the battery power is running to hopefully remind you to turn the switch off if you are not using the toy.

The LED power indicator
The LED power indicator

This is what it looks like beneath the hood. The LED is grouped with the Pololu voltage regulator chip.

Removing the battery compartment cover
Removing the battery compartment cover

This pic below shows the step after the breadboard testing when I was trying to plan the layout of the components so that they could fit within the limited space inside. The red SparkFun Pro Micro has to eventually fit inside the plastic sheath which it is sitting on top off now.

Planning the layout
Planning the layout

And then the soldering begins! A process taking a few hours of delicate surgery-like precision and concentration. The huge magnifier glass desk lamp I used was very helpful.  It was all in all, a lengthy and taxing process, both mentally and physically exhausting. Here is the soldered part for the Pro Micro.

The soldering begins
The soldering begins

The picture below shows the completed circuit with the components fixed in place. The Pro Micro is hidden to the left of the picture, and is followed by, from left to right, the main switch (for switching on and off the animation effect for each animation mode. You can see the back of the switch.), the secondary switch (camouflaged into the black background beneath the mess of wires. Used to switch between animation modes.), the perf board for the wiring connections (the large capacitor is to protect the first NeoPixel in the strips against the power-up surge), and finally, the battery switch.

The completed circuit
The completed circuit

The final step involves putting the casing back together. I cried out ‘Hallelujah!’ when everything worked.

Putting it back together
Putting it back together

If I were asked how I could improve this project, I would add a motion sensor and a microphone so that the sword would do a “swoosh” sound when it is swung about. Actually the original lightsaber already had this effect and my hacking is a downgrade in this aspect. Oh well. Win some. Lose some.

New job, arcade buttons, first blooper

As of late November 2014, I got a new job in an interactive agency with a job title “experiential media specialist”. I shall henceforth be known as the “electronics guy”, the one wielding a soldering iron. See, I even got my own tool bag.

tool bag
tool bag

And this is my workbench.

workbench
workbench

This role has very much to do with hardware – wires, buttons, sensors, tangible interfaces and so on. It is very different from my previous job which had to do with software, when I was an interface programmer. It could potentially be more exciting I guess. I will be touching physical objects, going on site to setup stuff, and shop for parts, very different from sitting at a desk programming all day. But we’ll see.

So during my first week into the job, as part training, I was asked to create a circuit. What this circuit does is that you light up a bunch of big arcade buttons whenever they are pressed. The arcade button LEDs run on 12V powered by a wall adaptor. You need something like an Arduino to act as a micro-controller, so that whenever the buttons are pressed, a relay circuit will allow the wall adaptor to close the circuit with the LEDs and turn them on.

relay
relay

First things first. I have no idea what a relay circuit was. As a result, in my years of using the Arduino, having never damaged any before. This time, I managed to fry TWO of them. Yes, I damaged two Arduinos before I realised there was a wrong connection somewhere. There was a short because I wired the relay circuit wrongly and sent 12V directly to the Arduino. Not just that, I fried my laptop as well. And I had to replace the motherboard. Luckily, my new boss was very forgiving.

So after realising my mistake, everything went smoothly after that. Here is the soldered circuit. The buttons and LEDs are for testing purposes and simulate the switches for the arcade buttons.

2014-12-12 11.31.27

Of course I tested the circuit on a breadboard before soldering. This was the part where I messed up.

2014-11-27 13.30.28

This is my circuit diagram drawing on paper. My new boss is electronics trained as well and commented that there should not be any over lapping wires and no diagonal wires, which is good feedback.

2014-12-09 16.27.34

The final stage was to put the circuit into one of those black ABS boxes. The connectors outside connect to the relay circuits, to which the LEDs and wall power supply are connected.

black box
black box

The making of ‘Gramophone’

gramophone_banner

I made this gramophone in four weeks. It looks and feels like a gramophone but obviously doesn’t function exactly like one. It has a turntable and (fake) vinyl records, but it is actually a laptop in disguise. There is a webcam on top of the screen, and the electronics inside include a Mac Mini and Arduino as seen below.

gramophone_insides

This is what it looks like when using it:-

Final product
Final product

The final product is pretty faithful to the original design in SketchUp:-

Design in SketchUp
Design in SketchUp

How it works is that you place lego blocks on the rotating turntable and the webcam will pick up the colours and position of the blocks. A custom software will translate this data into sound based on a virtual scan-line on the turntable. My other team mate for this project did the software. I worked on the hardware and as such, this blog post will only cover the hardware portion.

I spent the first week planning and sketching the design in SketchUp and spent the other three weeks full-time at the new maker space at the National Design Centre. I had the opportunity to use most of their facilities, such as the laser cutter, 3D printers, computer numeric controlled (CNC) machine and vinyl cutter. This post will be structured around how I used each of these machines.

 

CNC machine and laser cutter

I used the CNC machine to cut most of the wood. I used thicker 12mm plywood for most of the frame as mechanical support and 6mm for some parts which needed finer cutting and could be laser cut. Both the CNC machine and laser cutter accept .dxf files which could be created in DraftSight based on my SketchUp drawing dimensions.

Here is a component breakdown when disassembled in SketchUp:-

Component breakdown in SketchUp
Component breakdown in SketchUp

This is an example of what a .dxf file looks like in DraftSight:-

12mm_DS

This is what the CNC machine in action looks like:-

gramophone_CNC

The result which has to be cut out and sanded:-

gramophone_CNC_cut

Bunch of cut wood:-

gramophone_bunch_wood

A lot of cutting involves taking accurate measurements. For wood, an error of a few mm is acceptable, and inevitable.

gramophone_measuring

When sanding, I made the mistake of using the sanding belt with too much strength. As a result, my design was off by a few millimeters and I had to make up by paper sanding other parts. Fortunately, everything worked out in the end. This is the belt sanding machine:-

gramophone_sander

I also used the CNC machine to cut out the shape of the turntable. As you can see, the CNC machine is not good at accuracy and at cutting curves. I had to further cut out the parts I don’t need using this cutting machine.

gramophone_turntable_raw

A mistake was made using the CNC machine and a gash is there where it shouldn’t be. I used wood filler to cover it up. After painting over the entire surface, the mistake becomes less obvious.

gramophone_woodfill

While assembling the wood, I learnt how to properly glue wood together using clamps and nails to hold the surfaces tightly together while the glue dries overnight:-

gramophone_clamps

Oh, and I learnt how to properly nail a nail. I was quite proud that after a few tries, I can nail the nail straight down into the wood instead of going in an angle. The trick is to score a hole first with a few light knocks, and then forcefully hammer it down thereafter. Hammering a nail should be a basic lifeskill.

Me hammering a nail:-

gramophone_nailing

Another basic lifeskill I learnt was how to use a power drill. I had to use it to drill some holes when using the CNC machine:-

gramophone_drill

 

This is what the laser cutter looks like:-

gramophone_laser

The laser cutter is an amazingly accurate tool. You don’t have to account for the width of the laser itself when cutting. What you design is what you get, unlike the CNC machine which has a 6mm width drill bit. This 6mm has to be taken into consideration when designing your drawing. The laser cutter can produce extremely intricate drawings on wood. The laser cuts on wood have a beautifully charred and distinctive brown edges that has to be wiped with a damp cloth to remove the soot. I also used the laser machine to cut out the discs for the fake vinyl records from 3mm acrylic, achieving a perfectly round circle.

After the wood was assembled, I used mahogany wood stain to colour the plywood dark brown and then lacquered the exterior to give it a glossy finish and to protect the wood.

gramophone_painting

This is what the original plywood looks like, which is raw and uneven in colour:-

gramophone_raw

3D Printing

I used SketchUp’s 3D printing template to draw the parts that I needed to 3D print. I had to print out hinges for the lid, the stepper motor gear mount, the webcam mount and gramophone stylus. Because the free version of SketchUp can only export in Collada file or .dae file formats, I had to use a 3D printer software (Cura) to slice the file into .stl format. I used another free software called Nettfab Basic to verify the file to check for any missing surfaces or holes in the drawing.

To take measurements, I used electronic vernier callipers because the measurement has to be in the order of 0.1mm. The SketchUp template for 3D printing has measurements in mm with up to 10 decimal places. I had to reprint a few times because some hollow tube was too big or small. The 3D printer also has an error of 0.2mm due to the width of the filament when printing. So you have to take this error into account in your drawing. Hollow parts have to be designed slightly larger by 0.2mm in radius and solid tubes designed slightly smaller.

Here is the Ultimaker printer in action:-

gramophone_printing

Here is a drawing of the stylus and the 3D printed result-

gramophone_stylus_sketch
gramophone_stylus

 

Here is the webcam mount:-

gramophone_webcam_mount

And here is the stepper motor gear mount. The turntable sits on this.

gramophone_stepper_mount

The hinges and bolts. I used epoxy glue (resin + hardener) to glue the bolt and nut together:-

gramophone_hinges

Electronics

The stepper motor’s speed is controlled by the position of the stylus. I used a potentiometer to keep track of the stylus’s position and a A4988 stepper motor driver which is able to give a step resolution of 1/16 a step of 1.8 degrees. The code is very simple. The mapping is because I only want the stepper motor to move for a small range of the potentiometer knob movement, like 45 degrees, out of a possible 270 degrees. :-

gramophone_Arduino_codeThe wiring was not so simple. I had to draw all the connections on paper to keep track of the wires:-

gramophone_circuit_sketch

Here is the prototyping on breadboard:-

gramophone_breadboard

And the final soldered version. I was amazed everything worked on the first attempt, because there were about 50 solder points:-

gramophone_Arduino

This is what the back of the soldered board looks like:-

gramophone_soldered_back
Crimping is very useful when you want to modularise your wire connections:-

gramophone_crimp

Compare the above with the bad example below which was my first version of this circuit. There is no modularity and the use of hot glue which is unreliable and messy. The K179 driver used below also does not allow micro-stepping which the A4988 does.

gramophone_bad_circuit

 

Vinyl cutting

I had to make the “vinyl record” discs matt because the acrylic itself is glossy. The surface has to be read by a webcam and a glossy surface would give distorted colour readings. Compare the surfaces side by side:-

gramophone_glossy_matt

Here is the vinyl cutter in action. You have to stick the vinyl on a sticky mat and feed it to the machine. The software to operate the machine is called Silhouette Studio. Here, the cutting is for the colour coded disc centres.

gramophone_vinyl_centre_colours

Sticking a large piece of vinyl on any surface is not so straightforward because it will stick together and crumple easily and you have to throw away the entire piece. So the trick is to use soapy water to prepare the vinyl sticker so it wouldn’t stick together and you can adjust its position on the surface you are sticking it to:-

gramophone_vinyl_sticking

You then use a squeegee to squeeze out all the soapy water so that the two surfaces will properly stick together:-

gramophone_soapy

The power accuracy of an artknife to trim edges:-

gramophone_artknife

The finished disc:-

(fake) vinyl record disc
(fake) vinyl record disc

I also used the vinyl cutter for the labels:-

gramophone_words

The labels:-

gramophone_closed

 

Credits 

gramophone_OMG

The people in the lab have been incredibly supportive and I would like to thank them for their help and advice.

In particular,

I have to thank the electronics lead, Sim Joo Khai, for helping me troubleshoot the breadboard connections and teaching me how to crimp a wire. He also helped me rush 3D printing jobs and taught me how to use the vinyl cutter.

Sathi, for helping me with the CNC machine and laser cutter.

Poh Hong for tips on how to to use the various tools in the lab.

Robin and Veera for helping me with tips and laser cutting.

Hidaya for her administrative advice and moral support.

The making of ‘The Synesthete’s Music Machine’

My ex-classmates and I have sometimes talked about doing art projects together outside of school while we were still at LASALLE. We even created our own logo and group name called “Art Makes Us” to reflect that we like to make art. A year after we graduated, even though we are all working then, we actually managed to make this a reality. We got together to create a sound toy inspired by the idea of a synesthete, someone who is able to hear sounds in his head in response to visual stimuli. We presented this object at the Singapore mini-maker faire (smmf14) at Senja Cashew community center at the end of July this year. We treated this as an exploratory art project where we could exercise our artistic faculties in thinking, coding and making.

This project took about two months from conception to completion. I was the project leader for this and coordinated the team’s efforts and acted as a liaison to the organisers of the smmf14. First, I wrote a project proposal to capture our vision for this project and then proceeded to work on the lightbox and create a working software prototype in Processing and Puredata.

For the lightbox which was mostly made up of wood, I went to a makerspace that had just started up – the Experience Centre (XPC)  at Home-Fix’s HQ building at 19 Tai Seng Avenue. They had the wood cutting machines and raw wood materials. I used wood glue, hot glue, a staple gun, nails, specially thin velcro strips, and double sided tape to hold the parts together. All in all, the lightbox cost me about $200 to produce, including rental of equipment and service charges. It is rather expensive, but the cost includes excess materials and mistakes I made. If I were to re-do it, I could probably hack it at half the price of $100.

Some images showing the making of the lightbox at XPC:-

This slideshow requires JavaScript.

For the software, I did a prototype of the application in Processing and Pure Data. The final version was programmed in Javascript and was done by my teammate. The prototype has the functionality and processes required for the application, but requires Processing and Pure Data in order to run it. The great thing about doing it in Javascript is that it runs in the web browser which everyone already uses. An explanation of how the software prototype works is best explained in a video :-

The making of Stop Motion – my thoughts

This project took me the better part of six months. There are about 5000 lines of code. It is also my first commercial project when I was still wet-behind-the-ears after graduating from LASALLE. Because of this project, I gained a fair bit of confidence in programming skills, and would say that I have moved on from being a novice programmer to at least a fairly average one. The software was written in Processing / Java for a Windows Win 7 platform. The application allows the user to create a 150 frame stop motion video with sound effects, and then email it to yourself.  For a project of this size, I used the Eclipse IDE which really helped facilitate the programming process. I also used git to back up all the versions even though I was the only one working on the code.

I was the interface programmer for this project. The user concept and UI design came from my company, and so did the png cartoon graphics. Everything was laid out for me, I merely had to code the concept into a working application. This was a challenging project for me. Before starting on this project, I knew I could it, but I had no idea how long I would take, nor could I anticipate what kind of problems I would face. I pulled out all the tricks I knew in the Processing environment, such as using the Minim library for sound, using the Capture library to interface with the webcam and the Serial library for communicating with the hardware button controls. I had to code the transition animations for the graphics movement and screen changes, although I used one simple easing algorithm for all the movements. The interface transitions were rather tedious and painful but not difficult. This is something that Processing/Java is not good at as compared to Flash where there is a graphical UI for you to design your own custom animation movements using frames.

I had some help from the online Processing community for which I am extremely grateful. At one point in time I was puzzled by a persistent a out-of-memory (OOM) error in the heap space. I posted my issue on the Processing forum and got helpful replies saying that the raw image files I was storing in my memory were really big files. The png and jpeg formats that we are familiar have already been highly compressed but take up a lot of space before compression. I also found out that at present, Java on Windows in limited to a working memory of 1 GB which might sound like a lot, but not if you are working with many raw image files.

In fact, this OOM error was the single biggest problem I faced in my code. I didn’t know how to work around it. I could not reduce the number of image files that I had to use, nor was it possible for me to compress them since I needed to have them in memory somehow. After being stuck and getting frustrated for a week or two, I informed my management in my company about this. I had no senior nor any other colleague who could help me. And then my management did a magic trick – they summoned my predecessor who used to fill my role in the company a few years ago. By now, he had several years of coding under his belt and he described to me a simple solution – conceptually at least :- just save the images to hard disk as jpg or png files when you are not using them. Since the user had to be able to scroll through the images, back and forth, there has to be some sort of real-time “moving window”  which retains the images in program memory that are immediately being used, a few frames to the left, and a few frames to the right. The rest of the images which are further away from the current frame being used could be stored in hard disk memory, which is virtually unlimited compared to program memory.

Easy solution in theory but extremely tricky to implement because of all the possible permutations of operating states in my program that would affect the behaviour of the moving window – e.g. whether it is in preview mode, capture mode or rendering mode. Pretty hideous actually. And so the inevitably numerous bugs took me a month or two to iron out.

So that was the single grotesque problem I faced in this project. There were other challenges too but they were pretty straightforward, and could be solved with a few Google clicks and readings. For example, the Java email API was pretty easy to use. I set up a gmail account and was able to send out the videos to any user-defined email address using this account. Minim was useful in triggering the sound effects although I had to increase the sample size to 4096 to get rid of sound distortion at the expense of a barely noticeable delay effect in playback. I used multi-threading to ensure that the user interface remained responsive while the program was doing back-end stuff like rendering and emailing which takes up a lot of time. 

Oh I have to thank the heavens for FFMPEG. FFMPEG was the groundbreaking discovery I made while I was searching for a means to stitch the images into a video and combine sound as well. The FFMPEG algorithms are blazingly fast and efficient. The rendering time of 30 seconds at most was much less than what my management initially thought it would be, like in the order of minutes. Another discovery I made was how to give command line commands to the operating system from within Java, like to move files around or issue FFMPEG command strings. Very useful and rather straightforward. Java has very good documentation indeed.

In summary, it was a long and a little painful at times, but ultimately very satisfying when you see the fruit of your labour installed on site – no crashes and users coming up me and saying things like “My kids loved the stop motion booth”, “my favourite thing in the whole of Kidzstop”, “oh wow I got the email with the video I made”. It is also rewarding when I see kids and even adults enjoy using the station. If I was given a choice to go back in time and choose whether to do this project, I would gladly do so again with no regrets. And even if I had to re-do this project again now, I wouldn’t be able to think of any other better way to do it than what I have already done.