« »

Augmenting Reality

I know, I know, augmented reality (AR) is so ten minutes ago. But I still don’t think people are using it to its full potential. Saqoosha’s FLARToolKit is now well documented enough to allow even ActionScript noobs to get fabulous 3D images to display on their webcams.

When I get the chance, I’m going to work on making AR more interactive, but in the meantime, try out my Living Room, a prototype for a DIY interior design application that’s in the video above:

  1. Download and print out this image
  2. Make sure you have a webcam hooked up
  3. Launch the demo here
  4. Flash will ask for permission to use your webcam but you won’t be able to click on the button for about fifteen seconds while all the assets load (I need to add a loading bar)
  5. Hold the printed symbol up in front of your webcam and enjoy!

People are constantly finding applications that take better advantage of AR’s unique characteristics rather than using it as eye candy (as I tried not to do but didn’t quite succeed below). AR allows you to insert 3D objects into the viewer’s space which the viewer can manipulate and interact with. That’s pretty cool. You could, for instance:

  • type a message on the back of a 3D shape so that only a person who knew that shaking the marker a certain way would know how to turn it and read the message;
  • link different buildings or city blocks to unique markers and then try various urban layouts, writing traffic algorithms that simulate flows through the space and update in real time according to the layout;
  • use an AR sticker to generate a 3D mask that would follow your head when you turned it;
  • put an AR sticker on your forehead and then track it to produce responsive panning in a 3D environment.

People have made AR tee-shirts, Topps used the technology to enhance its baseball cards.

And because virtually anything works as a marker, you could in theory detect something ubiquitous, say a black square, and use it to project advertising behind a user a la 1-800-54-GIANT behind the batter’s box at Fenway.

The hardest part of getting AR working in Flash is getting all the various pieces to play nicely with each other. FLARToolKit was developed in Japanese and the documentation is scarce. I had the most trouble importing 3D models and getting them to display correctly. What ended up working for me was finding assets I liked in the Google 3D Warehouse, opening them in SketchUp and centering them before exporting them as .kmz (Google Earth) files. I then placed them in the same folder as my ActionScript file and used the following code to import them. Note that this method is memory and time intensive. If you’re loading a bunch of big KMZ files, you’ll have to increase the time Flash will allow a script to load before timing out.

     //This goes at the top with the other import statements

     import org.papervision3d.scenes.Scene3D;
     import org.papervision3d.view.BasicView;
     import org.papervision3d.view.layer.util.ViewportLayerSortMode;
     import org.papervision3d.render.BasicRenderEngine;
     import org.papervision3d.objects.parsers.Sketchup;
     import org.papervision3d.objects.parsers.KMZ;
     import org.papervision3d.events.FileLoadEvent;

     //This goes in the class declaration
     private var _carpet:KMZ;

     //This goes in the onInit() function
     this._carpet = new KMZ();  //Initializes my carpet object
     this._carpet.load("carpet.kmz");  //Loads in the carpet from the KMZ file
     this._carpet.scale = 3;   //Scales the carpet
     this._carpet.rotationY = 90;
     this._carpet.z = 60;
     this._carpet.x = 0;
     this._carpet.y = -60;

     //Add the carpet to the list of objects that follow the marker

Here are some of the most helpful sites for getting started:


Post a comment

You must be logged in to post a comment.