Friday, September 26, 2008

First marker tracking example


I tested my marker tracker on iPod Touch. 
Current frame rate is about 6 fps. Marker detection and decoding is fast enough and it does not reduce performance much. 

The major bottlenecks are (1) converting a RGB image to a binary image and (2) Background texture update in every frame. As I posted previously, the major problem is texture update. Still there may be a way to update the performance. 

glTexSubImage2D is too slow on iPod Touch

I tested the performance of changing texture data in every frame, using glTexSubImage2D. 
The resolution of texture image is 320x240. Since OpenGL ES does not support non-2^n size of texture, I created texture of 512x512, then just changed part of it. 

The texture is created as: 

glGenTextures(1, &bg_texture) ;

glBindTexture(GL_TEXTURE_2D, bg_texture) ;

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 

tex_size, tex_size, 

0, GL_RGBA, GL_UNSIGNED_BYTE, 

nil);

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);


and updated in renderScene() function : 

glBindTexture(GL_TEXTURE_2D, 0) ;

glBindTexture(GL_TEXTURE_2D, bg_texture) ; 

glTexSubImage2D(GL_TEXTURE_2D, 0,0,0

320, 240, 

GL_BGRA, GL_UNSIGNED_BYTE

pixelData); 


However, the performance is very poor. Calling glTexSubImage2D makes the program hardly be able to run in real-time. Some applications on mobile phones show that more than 20 fps while rendering video. Thus, I think the problem may be texture format. Threre are several texture formats, like :

  • GL_UNSIGNED_SHORT_4_4_4_4
  • GL_UNSIGNED_SHORT_5_5_5_1
  • GL_UNSIGNED_SHORT_5_6_5


Maybe there will be a format that iPhone's hardware prefer.  


Monday, September 8, 2008

OpenCV on iPhone OS - 2

I ran the simple application on the iPod touch. 
I added three image processing step to an OpenGL ES sample application provided by Apple. 

What the application does is : 
- Create OpenGL ES surface 
- Load a color image 
- In the rendering loop 
   1. Gray scale conversion from the original color image 
   2. Simple thresholding 
   3. Canny edge detection. 
- Display 4 resulting images with texture mapping. 

Well, after the image processing with OpenCV functions are added, the OpenGL ES application becomes much slower than before. Unfortunately I couldn't measure the exact frame rate of my application, but I feel that it is about 1 frame/sec. 

Too slow huh ? 

Update on 09/26/2009: Well, the frame rate is because of glTexSubImage2D I called in every frame to render the result and the size of image (I used 512x512 image).


Friday, September 5, 2008

OpenCV on iPhone OS - 1


Today, I tried to use OpenCV library on iPhone OS.
OpenCV is famous computer vision library used by many people working on image processing and computer vision. It is written in C language and platform independent (Windows, Linux, and Mac OS). 

I included the two main library OpenCV (cv and cxcore) in my project and wrote small image processing codes (thresholding, edge detection, and blur) using OpenCV functions. The project was successfully built without errors. 

I tried it on the simulator only. The performance on the real device should be checked. I hope OpenCV to work in real-time on iPhone/touch. ;-)



Tuesday, September 2, 2008

Running iPhone/Touch applications on the devices without the developer registration

Well, Apple blocked installing and running applications developed by a person who does not register to Apple's developer program. This may be one of their policies to control the market as they want. In addition, no applications without developer sign can be uploaded to App. store.  

For me, Apple's policy is quite odd, since I don't want to sell my application or let other people use it. I just want to use it for my research and for fun. Thus, I have to find some other way to do it. 

Some smart people who are familiar with MAC OS X and unix programming already found the way to run applications without developer sign. The steps are as follows. 
  1. iPhone/Touch shoud be hacked and something like OpenSSH should be installed. Cydia makes it easy. 
  2. Write your own code and build it for iPhone platform. (Build target architecture is arm6) 
  3. There are three tips to bypassing code sign issues : (www.saurik.com/id/8)
  4. Copy your application to iPhone/Touch's 'Applications' folder. 
  5. If you don't see your application on the spring board, please reboot your device. 
  6. Now, do you see your application on the spring board ? Click it to run. 

Hacking iPod touch

Today, I tried to hack an iPod touch with QuickPwn tool released by iPhone-Dev team. QuickPwn was much more past since it does not build a custom hacked firmware. The hacking took only 5 minutes. One thing I didn't want is that the QuickPwn changes the boot logo from the apple's to their pineapple. 

After hacking iPod touch, I installed Quake4iPhone from Cydia for testing and it works well. 

Thank you iPhone-Dev team !