This is my buzzword-compliant winning entry from the Autumn Hackathon:
How it works
- Gyroscope and accelerometer is sampled 40 times per second using W3C DeviceOrientation API.
- Data is sent from phones via WebSockets. It was impossible to upload that much data with such frequency using regular POST (I've tried :). Socket.io turned out to be a pain to install, and Pusher's Pipe API saved the day.
- Central node.js server processes incoming data using neural network to recognise positions and "half-moves" (using sliding window over the samples).
- Recognised states from all players are distributed to viewers (it was quicker to use Pusher again, but this hop would be better served by Server-Sent Events).
- States are corrected using a simple Finite State Machine to eliminate few glitchy/nonsensical state sequences and to add a combo attack. All "game" logic is held on client-side (which is a bad idea generally), because that was easier to test.
- Stick figures and cheesy "special" effects (drawn in Seashore) are animated using mix of CSS transforms+transitions and old-skool DHTML.
- Sound effects are played back using HTML5
Audioobjects. Audio playback is totally unusable on iPhone, but worked surprisingly well in desktop browsers.
I've created mini application to record moves and created hundreds of samples. At first I've tried to use 1-second sliding window to recognise whole moves, but that turned out to be inaccurate and introduced a lot of latency. After experimentation I've settled on splitting moves into start and end parts and small fragment of used smoothed out data.
Accelerometer readings are in range 1-infinity (well, speed of light or more likely limit of hardware), so I've created fudge-factor-filled formula for normalising it into 0…1 range required by the neural network. Negative Y reading uses different scale to better recognise difference between gravity when phone is upside down from centrifugal force during attack.
In the beginning I foolishly thought that dead reckoning is possible with iPhone accelerometers. It's a seriously hard problem, so this hack only uses orientation and direction of movement.
O2 UK's censorship & compression proxy breaks WebSockets.
I've assumed that for a quick hack I can loosen some programming best practices, and of course that bit me hard. I've wasted hours debugging mistrained neural network, because I've messed up copy&pasted code.
Video from the event is coming soon. I've demoed the hack to the tune of floppy drives playing Imperial March:
On an event like this probability of somebody having a Jedi robe with them approaches 1, so I've also had a proper costume.
The hack itself
The hack requires gyroscope and W3C Device Orientation API, and the only phone that supports that currently is iPhone 4/4S. Make sure you're connected over Wi-Fi and visit pornel.net:8000.
Open the same page in a desktop browser to see your moves. Background colour of your character is same as colour displayed on the phone. If two or more people connect (I allow up to 20), you can fight.
The demo might not work at all times. If your phone keeps showing "Stand by!", it means I need to restart the server (ping me) — proper error handling didn't seem important at 4am when I wrote the code :)
I'd like to thank ADL Legal for assistance and sponsorship of the hackaton.