Tuesday, November 3, 2009

iPhone Camera Access Module from GamesAlfresco


Many people may know about the pose at GamesAlfresco (Original post is here), which says they made iPhone camera access possible on the firmware 3.x .Last weekend, I received the source code from Ori Inbar and did some tests with the camera access module.

It works well by itself. I could retrieve the raw data coming from the camera. The module uses a trick similar to what I tried on iPhone OS 2.x, creating CoreSurface buffer of preview from the camera.

However, the problem comes when I overlay something.

When I overlay some contents (like an image view or OpenGL drawings) on the preview view of the camera, the overlaid contents appear in the capture video frame. That is, the captured frame is like a screenshot, not a raw video.

I experienced a similar situation with the 'takePhoto' method, which gives a screenshot of the iPhone's screen and the module received from Ori also have the same issue.

Here are two captured images from incoming raw data of the camera module.They are not screenshots. As we can see, when I overlay an OpenGL view or a UIImageView on top of the 'previewView' of PLCameraController, the overlaid contents are also captured.



So, we still have problems with iPhone 3GS' camera.

16 comments:

  1. 저도 그 소스를 받아서 봤는데요. 실망스럽게도...camera에 live로 access하는 module은 private API를 썼더군요. :(
    OS3.1로 넘어오면서 camera view에 overlay를 하는 것은 자유로워졌으니 layar처럼 POI를 이용하는 App들은 늘어나겠지만 아직 live image processing은 힘들 것 같습니다.
    다만 redlaser나 replay cam처럼 초당 10 frame 정도로 빠르게 camera view의 screenshot을 뜨는 것은 public api로도 가능한 것 같으니 half live는 가능하지 않을까 싶습니다. :)

    ReplyDelete
  2. Hi, I take it that PLCameraController is still the best option for video streaming on the 3GS then? This is especially painful considering the new static analysis tool is designed to reject private frameworks:

    http://m.gizmodo.com/site?sid=gizmodoip&pid=JuicerHub&targetUrl=http://gizmodo.com/5405978/iphone-apps-have-to-be-approved-by-robots-now-too%3Fop%3Dpost%26refId%3D5405978

    Or am I wrong? Did you get an OpenGL display working with the 3GS without using PLCameraController or another private framework? If so, could you post the relevant code?

    ReplyDelete
  3. Lake, I couldn't get raw image data coming from the camera on 3GS. I think using PLCameraController is not a good solution until now. Well, if you use overlayView feature on SDK 3.1, you can overlay OpenGL view on top of the video preview. However, still it is not possible to get raw data.

    ReplyDelete
  4. here is a way to get the camera buffer and overlay some drawings on it without affecting the buffer:

    http://modmyi.com/forums/iphone-ipod-touch-sdk-development-discussion/642261-cameracontroller-3gs-3-0-a-4.html#post5033894

    ReplyDelete
  5. Thanks for your informative post. It saved my day!

    ReplyDelete
  6. Hey, Anonymous. I read the post

    http://modmyi.com/forums/iphone-ipod-touch-sdk-development-discussion/642261-cameracontroller-3gs-3-0-a-4.html#post5033894

    and tried to use an additional UIWindow, but I couldn't display a new one on top of the existing one. Do you know how to manage two UIWindow instances ?

    ReplyDelete
  7. hmm ... i did as it was explained in the forum ...

    - created a first UIWindow with a main nib file
    - added another UIView, and UIViewController thanks to a second nib file.
    - derived this UIViewController to a class called VideoViewController, in which I implemented the following methods :

    initWithNibFile :
    basically the same as in the forum


    viewDidAppear :
    [secondWindow makeKeyAndVisible]
    startpreview of the camera
    start timer.

    ReplyDelete
  8. Hi, anonymous.
    I think I got an idea how to do it.
    Thanks !!

    ReplyDelete
  9. How do you use overlayView when overlaying a opengl object on the camera stream?
    I cant get it to work the same way as with a image.

    ReplyDelete
  10. Please refer for one of my previous posts. "A simple and nice way to video background without OpenGL ES texturing on the iPhone" (http://mobile-augmented-reality.blogspot.com/2009/07/simple-and-nice-way-to-video-background.html)

    ReplyDelete
  11. Hi, I can't manage to take a screenshot of both my camera view and its overlay : an OpenGL object...

    By two different ways I only can display one of them on my screenshot...

    How can I do that please ?

    ReplyDelete
  12. Hi, Anonymous. If you want a screenshot you can get it by pressing the home button with the sleep button.

    ReplyDelete
  13. In fact I need to do this screenshot in my application.

    I want the user to move the openGL object to the suitable place on the camera view and then to take the picture with both camera view and openGL object...

    Is it possible ?

    ReplyDelete
  14. You can do it using takePicture method of PLCameraController or you can use UIGetScreenImage that returns a screenshot as UIImage, as I remember.

    ReplyDelete
  15. It's working perfectly with UIGetScreenImage() !
    Thanks!

    ReplyDelete
  16. Hi,

    If possible, can you please send me the source code from GamesAlfresco?
    I have got no answer from them in a long time...

    Thank you!
    nmouth_oreh_yjnn[at]yahoo[dot]com[dot]br

    ReplyDelete