3-D Video

By Amir Majidimehr

The principal behind 3-D video is very simple: perception of depth is created in the brain due to each eye seeing a different perspective of an object. The brain takes that difference and computes the respective position of each object relative to each other.

When we record an image using a single camera, we lose the differing perspective since we have a single image now and as such, the image depth perception collapses (although there are other cues the brain uses to create some limited perception of depth such as background being out of focus).

To gain back the 3-D perspective then, we need to record and playback two images as they would have been received by each eye. In case of live imagery, we can use two cameras positioned equal to the average distance of human eyes. For computer generated animation, the process is simpler as the modeling software simply generates two perspective streams. Camera angle is a simple variable that can easily be varied and the scene regenerated by the computer.

The challenge then becomes how to store and display the two streams. Ideally we would do this in a backward compatible way so that someone without a 3-D display can still watch the video. Let’s examine both the storage and display requirements.

Transmission and Storage

We have two challenges here. One is how to represent a stream in a backward compatible way. The second is how to avoid losing half of our bandwidth for the extra stream, and thereby reducing fidelity.

The solution is a partial one. We record one channel as we do today. We then take this primary stream, and find the difference between it and the second stream for the other eye. This causes anything in common to zero out leaving us with less data to transmit. For example, let’s say we have a person in front of a background of blue sky. The pixels representing the person are different in each perspective (since the angle of camera is changed). However, the blue sky remains the same. Therefore if we subtract the two images from each other, the sky pixels go away and what we have left are the pixels for the person which takes a much smaller portion of the full screen. This “additive” channel is stored alongside of the primary channel and stored on media (e.g. Blu-ray Disc) or transmitted live in the case of satellite and cable.

I mentioned the above was a partial solution to the fidelity issue. Reason is that the secondary stream while taking less space than the primary stream, still takes up space even in the best case scenario of a lot of commonality between the two images. In the worst case, the entire image may be different due to angle of view, causing the system to essentially send two full-bandwidth streams. While formats like Blu-ray have considerable headroom to produce high fidelity video, the impact may nevertheless be there. Live transmissions with their less efficient encoding (due to the need to run real-time), and lower channel bandwidth (less than half of Blu-ray) will likely compromise fidelity to achieve 3-D.

Put another way, people watching 2-D will be at the end of the short stick here, getting lower fidelity without any enjoyment of the 3-D system.

Displaying 3-D

The challenge we have here is having a single display that has to produce two images. If we attempted to output both images as is, we would see ghosting with one image offset from the other. To get a 3-D effect and eliminate ghosting, each eye must only see the image destined for it.

The solution -- at least today -- is to use polarized glasses. What is polarized glass? It is a special type of glass which only lets light through in one direction. Put another one in front of it rotated 90 degrees and you don’t get light as one would filter half the light in one dimension, and the other half gets filtered by the second one in the other dimension. Align them in the same way and the light goes through. In other words, we now have a way to selectively turn light transmission on and off.

There are two types of glasses: “active shutter” glasses and passive. The latter is just polarized glass as described above. We use vertical polarization for one eye and horizontal for the other. Active shutter glasses on the other hand, use a sandwich of two polarization glasses. One is fixed and the other can rotate to either match its direction or be opposed to it. A small electric signal tells the moving layer to switch directions, allowing us to completely block light on demand.

So far we have not accomplished anything. If you look through a polarizer in either direction, you more or less see the same image – albeit with half the light intensity gone. The trick to 3-D is to get the display system to match the eye glasses in a way that allows the image for each eye to get blocked when we want it.

How we accomplish that depends on the technology used. In the theater there is a need to use lower cost glasses which can be easily replaced if broken (or stolen!). Therefore, we use passive polarization glasses there. Each eye sees an image through either vertical or horizontally oriented polarized glass. A special device is then placed in the light path of the projector which in each frame of video either polarizes the light horizontally or vertically.

Can you guess where this is going? The projector shows the two images by alternating between the two streams fed to it. By polarizing the output of the picture one way and then the other, in sync with the projector, we are able to make sure each eye only sees the image destined for it.

Let’s assume the glass in the left eye is vertically polarized. When the projector passes its light through a vertical polarizer, the left eye sees that image just fine. The right eye has a horizontal polarizer and hence, no light goes through it. The reverse happens in the next frame with the left eye not being able to see the frame and the right eye having matching polarization.

What’s that you say? Wouldn’t we see the switch back and forth when the image goes dark every other frame? Well, we would if we did it slowly. But if we switch very quickly, then the eye doesn’t see the transitions. Same principal is in effect whereby you see a moving image in a theater, even though you are shown 24 separate stills. The brain blends the distinct frames together. In the common Real system, the frame rate is actually 144 Hz (frames/sec) so the eye is very much fooled.

At home, the more cost effective solution is to make the glasses more expensive but make the display cheaper since the user presumably will take better care of the glasses. Here, the TV simply shows the interleaved image from each stream as we did in the theater but there is no polarization device in front of it. Instead, an infrared light (much like what is used in your remote control) or radio frequency signal is sent to the receiver in the active shuttered glasses to block light on demand for each eye. When the frame for the left eye is being played, the TV tells the electronics in the active glass to polarize the right frame as to stop it from transmitting light. In the next frame, the reverse happens. Imagine putting your hand on one eye and then the other eye quickly and you would be simulating what the glasses are doing to stay in sync with what is being played.

In the consumer scenario then, the only thing special required of the TV is to display twice as many video frames per second. Technologies such as Plasma and DLP projection are already running much faster than consumer video frame rates so they have no trouble keeping up. LCD technology however struggles to keep up as it is a slower system to start. Tricks such as insertion of black frame are used to help with this situation. In this method, the display shows a black frame in between every other N frames as to force the LCD panel to dissolve to nothing (black) and hence, remove the traces from the view for one eye before the other is displayed. Still, in many products some amount of "crosstalk" remains as the images intended for one eye remain for the other causing ghosting of images.

Good news is that putting aside the cost of 3-D glasses, there is essentially little increased cost for consumer displays. So people who are not interested in 3-D are not impacted cost-wise.

There is also the eye fatigue factor. Every viewer will suffer some and for a small group of individuals, the 3-D effect will simply not be there. These are the tradeoffs we get for 3-D. On the other side of the coin, when 3-D looks good, it can be breathtaking. This is the case with our reference Sim2 Lumis 3-D Solo projector which produces the best 3-D I have seen. The sense of realism it can create at times is just hard to appreciate until you see it. Objects are not just appearing in front of you but appear to literally and physically be in the room with you. With none of the artifacts of lower end projectors and next to no crosstalk due to its high speed frame rate enabled by its exclusive “triple flash” technology, it can make a believer in anyone that 3-D can work in a home setting. That included me!

Note that with 3-D, “field of view” becomes very important. You need to have an immersive experience or the illusion will look fake. Reason is that once the eye scans past the edge of the screen, the 3-D illusion falls apart reminding you that what you are watching is not real. So where possible, you want to have a larger screen and sit closer to it than you would with 2-D video. This is what we have done with our 17 foot wide reference theater and closer than normal seating distance.

As noted, the other drawback is the loss of light due to use of polarization glass and reduced frame rate. This means even bright displays may not be bright enough to provide a vibrant image. This becomes a especially acute problem with low-end projectors that simply do not have enough light output to cut through the losses. Another issue is loss of color balance as the polarized glass imparts a color shift. This is corrected for in commercial theaters (although in my viewing of Avatar, the green shift was noticeable and annoying). And you would need two settings for your display: one for 2-D and one for 3-D. Well designed projectors such as the Sim2 Solo have dual color correction and brightness settings that automatically activate when you play 3-D content, solving these problems.

So as with most things in life, there is no free lunch. Good 3-D requires well designed equipment in order to reduce if not eliminate the trade offs it can bring. If you have been disappointed by how 3-D is presented in current consumer devices, come and see our reference theater. It will give you a glimps of how good it can work.

3-D Video

Transmission and Storage

Displaying 3-D

Further Reading