When rendering the incoming video frames as a background texture, try to turn off linear interpolation property.
On Direct3D : D3DMTSS_MAGFILTER and D3DMTSS_MINFILTER
It will increase texture rendering speed much. In my case, using Direct3D, the performance is almost improved by the factor 1.5 after disabling linear interpolation. What we sacrifice for this performance improvement is the rendered texture of a video frame. Since there is no interpolation, the rendered background image will have aliasing effect but it may not a big problem.