Unity3D for iOS: AR Flyer/Poster Example with Basic Sound Spectrum Analyzer

Here’s a quick Unity3D Augmented Reality example I made this week using Qualcomm’s free Vuforia for iOS plugin.

In addition to Vuforia, I used the handy open source iTween library & Unity’s AudioListener.GetSpectrumData on a dummy model I made using Blender. The 3 robots are acting as the sound spectrum analyzer, each being tweened in response to a float (decimal) that’s put together from an average from a sound frequency range.

I used a Creative Commons graffiti image from Flickr in the flyer/poster background & a Creative Commons mp3 from SoundCloud.

In this case the entire flyer/poster acts as the Image Target. I was surprised to see that it got a 5 star rating in the Target Management System (TMS) on QDevNet (TMS shows up under My Trackables after you create an account).

Unity3D for iOS: Augmented Reality with a Beer Bottle

I used the beer label as the Image Target for Vuforia for iOS. Qualcomm’s Target Management System (TMS) rated the label image with 3 out of 5 starts, which is better than I thought but still leaves room for a little unwanted jitteriness. To create the image, I simply took a photo of the beer label with my phone camera, silo’d it out in Photoshop & exported as a 24-bit (transparent) PNG, uploaded to the TMS.

Next, I added some random 3D models, an empty GameObject w/ Box Collider behind the beer label (ImageTarget prefab) and basic text & button sprites via UIToolkit, as well as some mind-numbingly simply logic.

The UIToolkit UI object (& it’s child UIToolkit objects) seems to work fine when nested inside Vuforia’s ImageTarget prefab:

I set the ARCamera’s frustum to look at the global Y axis in this case (ARCamera is Vuforia’s version of Main Camera):

A Note on Qualcomm’s Vuforia Samples & Documentation

While helpful, QDevNet’s documentation on Samples is not always Unity friendly. For example, the Developer Guide’s section on the “BackgroundTextureAccess Sample App” refers to “\media folder” as the location of the printable PDF that contains the sample’s marker image. If you’re using Unity3D on Mac, the actual path is here:


Unity3D for iOS: Basic AR via the Free VUFORIA for iOS Plugin

Qualcomm’s dev site & docs for Vuforia

QDevNet’s version of Vuforia for iOS for Unity

  • Comes with Sample projects, each as a .unitypackage.
  • Doesn’t show up as a selectable package in the Project Wizard in Unity. You can still import it via Assets > Import Package > Custom Package.

Unity Asset Store version of Vuforia for iOS

  • Contains no sample projects/files.
  • The Vuforia package shows up Project Wizard when you do File > New Project in Unity.
  • After downloading Vuforia for iOS from the Asset Store, I started with a tutorial called “Getting Started – Unity Extension > Compiling a Simple Project” on QDevNet’s Vuforia section
  • The Asset Store version of Vuforia for iOS didn’t seem to have any sample usage files or demos, as of May 21, 2012. This part was a bit confusing but I was able to get it working after poking around on QDevNet.
  • Comes with empty StreamingAssets/QCAR and Editor/QCAR/ImageTargetTextures folders. This means when you try to follow the “Compiling a Simple Project” tutorial under “Getting Started – Unity Extension” here, and try to add the ImageTarget prefab to your Hierarchy as the tutorial shows, the Image Target Behavior script won’t show any public vars like “Data Set” or “Image Target”.
    The Unity Assets Store version doesn’t seem to have any sample Targets – 1. download a sample project from https://ar.qualcomm.at/ and copy/paste or 2. create your own using their online Target Management System as described here.

Random Notes on Creating Trackable Images (Image Targets) w/ Target Management System (TMS)

  • TMS lets you upload a photo & lets you know if it works for Vuforia using a 5 star rating system. Zero starts means the image is totally unusable. Once you figure out a descent 4 or 5 star rated image you can click “download selected trackables” and you get a .unitypackage with the necessary .xml file and visual assets. Double clicking on the downloaded .unitypackage imports these assets into the correct locations (assuming you have Vuforia for iOS already installed in Unity)
  • A trackable image needs to have “Sharp edges & high contrast”:
    “Example of image that provides no detection and no tracking… There are no features in this image because it lacks visual elements with sharp edges and high contrast. Detection and tracking are not possible.” The TMS didn’t like my first notebook image because of the spiral edge pattern at the top. I got this message: “Avoid repetitive patterns or motifs:”

    The system provided a link to a slide show of various images along with explanations on why they work well or not. That was helpful.
    This image got 4 out of 5 stars from TMS and ended up working well:

    An image that was rated 1 out of 5 starts by TMS produced a super jittery 3D model that did appear when I held my phone up to it from just the right angle but didn’t look great. Developer Guide > Trackables > Image Targets and Developer Guide > AR App Design Guide > Image Target Design Properties and Image Target Enhancement Tricks talk more about which images are acceptable.

  • 2MB is the max file size limit for a TMS uploadable image as of May 2012 – aka, a photo taken on an iPhone 4S & email at Actual Size might be too big. 

Licensing, etc