Sunday, January 10, 2010

A good tutorial about using 'cameraOverlayView' feature for Augmented Reality on iPhone

Update: This method is no longer useful because Apple provides camera APIs now. 

Here is a nice example that showing how to use a new feature, cameraOverlayView, for AR applications on iPhone. There is a example code, so that you can easily learn how to do AR on iPhone.

In the post, what the author try to do is grabbing a screenshot through UIGetScreenImage() and analyze it. UIGetScreenImage() just provide a screenshot, so that the overlaid contents remain on the screenshot. The author of the post try to remove the contents drawn on the overlaid view by simple interpolation.

I downloaded the example and tested it. The below image is screenshot obtained by UIGetScreenImage(). You see the overlaid green marks of edges.

And, after interpolation, the code gives the following image. It looks OK for now. You may see seams in the image, which is the interpolated pixels.

Another example:

However, this approach may not work if we draw large objects like a cube since we cannot remove the cube through interpolation. Thus, this approach is definitely not good for tracking either. Another problem is interpolation, which takes some time to do it. It may take too much time on mobile phones.


  1. I've made a method for grabbing whole frames that works with OS3, though it uses secret APIs and lots of tweakery.

  2. Hey, jd. Thanks for your comment.
    I visited your blog and read your tips. But I couldn't understand how you do it because there is no 'IOSurface' class. How did you get it on iPhone SDK ?

  3. This guy ( JD ) every where keeps saying I've done this but never answers how and refers to stuff that don't exists, so I don't believe him!

  4. Hey its really amiging. Keep Sharing.

    Virgin phones

  5. How is this a tutorial if you provide zero code, and a link to a dead blog?

  6. Hey, anonymous. I think you need to search web or read blog posts carefully before you complain.
    The method in the post is useless now, because it was written before Apple provides camera access APIs. The author of the original at the link already mentioned it in his revised post.