It took a lot of pain, but I got color based object tracking working for Wade Brainerd‘s Colors! Activity, which he ported from Colors! for Nintendo DS by Jens Andersson. The tracking for what I have right now is calibrated to use the XO’s AC adapter. It’s a bright, uniform, unique color, and it’s something that every XO owner has. It’s pretty nifty watching the brush follow the camera around, but the real magic happens with you enable “Pressure Sensitivity” in the palette menu of Colors! The brush will actually change size and opacity based on how far you hold the charger from the camera. It works better than I could have ever expected, and this is just the start. The next step I’m planning is to let the user pick any real life object they want and use it as a brush, with the color of the object being the color of the brush on screen.