This project took me the better part of six months. There are about 5000 lines of code. It is also my first commercial project when I was still wet-behind-the-ears after graduating from LASALLE. Because of this project, I gained a fair bit of confidence in programming skills, and would say that I have moved on from being a novice programmer to at least a fairly average one. The software was written in Processing / Java for a Windows Win 7 platform. The application allows the user to create a 150 frame stop motion video with sound effects, and then email it to yourself. For a project of this size, I used the Eclipse IDE which really helped facilitate the programming process. I also used git to back up all the versions even though I was the only one working on the code.
I was the interface programmer for this project. The user concept and UI design came from my company, and so did the png cartoon graphics. Everything was laid out for me, I merely had to code the concept into a working application. This was a challenging project for me. Before starting on this project, I knew I could it, but I had no idea how long I would take, nor could I anticipate what kind of problems I would face. I pulled out all the tricks I knew in the Processing environment, such as using the Minim library for sound, using the Capture library to interface with the webcam and the Serial library for communicating with the hardware button controls. I had to code the transition animations for the graphics movement and screen changes, although I used one simple easing algorithm for all the movements. The interface transitions were rather tedious and painful but not difficult. This is something that Processing/Java is not good at as compared to Flash where there is a graphical UI for you to design your own custom animation movements using frames.
I had some help from the online Processing community for which I am extremely grateful. At one point in time I was puzzled by a persistent a out-of-memory (OOM) error in the heap space. I posted my issue on the Processing forum and got helpful replies saying that the raw image files I was storing in my memory were really big files. The png and jpeg formats that we are familiar have already been highly compressed but take up a lot of space before compression. I also found out that at present, Java on Windows in limited to a working memory of 1 GB which might sound like a lot, but not if you are working with many raw image files.
In fact, this OOM error was the single biggest problem I faced in my code. I didn’t know how to work around it. I could not reduce the number of image files that I had to use, nor was it possible for me to compress them since I needed to have them in memory somehow. After being stuck and getting frustrated for a week or two, I informed my management in my company about this. I had no senior nor any other colleague who could help me. And then my management did a magic trick – they summoned my predecessor who used to fill my role in the company a few years ago. By now, he had several years of coding under his belt and he described to me a simple solution – conceptually at least :- just save the images to hard disk as jpg or png files when you are not using them. Since the user had to be able to scroll through the images, back and forth, there has to be some sort of real-time “moving window” which retains the images in program memory that are immediately being used, a few frames to the left, and a few frames to the right. The rest of the images which are further away from the current frame being used could be stored in hard disk memory, which is virtually unlimited compared to program memory.
Easy solution in theory but extremely tricky to implement because of all the possible permutations of operating states in my program that would affect the behaviour of the moving window – e.g. whether it is in preview mode, capture mode or rendering mode. Pretty hideous actually. And so the inevitably numerous bugs took me a month or two to iron out.
So that was the single grotesque problem I faced in this project. There were other challenges too but they were pretty straightforward, and could be solved with a few Google clicks and readings. For example, the Java email API was pretty easy to use. I set up a gmail account and was able to send out the videos to any user-defined email address using this account. Minim was useful in triggering the sound effects although I had to increase the sample size to 4096 to get rid of sound distortion at the expense of a barely noticeable delay effect in playback. I used multi-threading to ensure that the user interface remained responsive while the program was doing back-end stuff like rendering and emailing which takes up a lot of time.
Oh I have to thank the heavens for FFMPEG. FFMPEG was the groundbreaking discovery I made while I was searching for a means to stitch the images into a video and combine sound as well. The FFMPEG algorithms are blazingly fast and efficient. The rendering time of 30 seconds at most was much less than what my management initially thought it would be, like in the order of minutes. Another discovery I made was how to give command line commands to the operating system from within Java, like to move files around or issue FFMPEG command strings. Very useful and rather straightforward. Java has very good documentation indeed.
In summary, it was a long and a little painful at times, but ultimately very satisfying when you see the fruit of your labour installed on site – no crashes and users coming up me and saying things like “My kids loved the stop motion booth”, “my favourite thing in the whole of Kidzstop”, “oh wow I got the email with the video I made”. It is also rewarding when I see kids and even adults enjoy using the station. If I was given a choice to go back in time and choose whether to do this project, I would gladly do so again with no regrets. And even if I had to re-do this project again now, I wouldn’t be able to think of any other better way to do it than what I have already done.
lohjianhui 