.Lukas

About a year ago I got asked if I could provide stereographic output suitable for a single person 3D screen (if I remember it correct it was the Sharp LL-151-3D, but I may be wrong) from one of my programs. As I was programming under Linux, and using CUDA with OpenGL for rendering I needed to figure out how to address the screen. I was not able to dig out any direct description how it worked on the Internet, and thus had to figure it out myself. Afterwards I thought that I should write up a simple example program and post here, but time ran away and I got other things to do. Recently I saw a question about this problem posted somewhere else and remembered what I wanted to do.

I do not any longer have access to the screen and thus can not test an example program for correctness, so I will just describe how to combine a stereo pair for the screen, and give the algorithm. Most graphics programmers in the need of the solution should be able to take it from there.

Background

An autostereoscopic monitor gives the viewer a three dimensional viewing experience without the need of any special glasses, just by looking at the screen. It achieve this by making sure that separate images are viewed by the observer’s left and right eyes. One technical solution for this is to use parallax barriers.

The basic principle is that a set of vertical barriers is located in front of the LCD to block view of more than one image from certain directions. An introduction can be found here. Exactly how this works does not matter in the following, but it allows for two different images to be visible at two different positions in space; corresponding to the left and right eye of an observer. If these images consists of a stereo pair a three dimensional image is perceived.

So in other words: the screen make sure that the frame buffer image sent to it is somehow split up to two different image that is mutually exclusively visible from two different positions in space. The problem is to figure out what screen data goes to the left image and what data goes to the right.

The Sharp screen, like (most) other 3D monitors is addressed through a normal VGA/DVI port, just like normal screens. This means that once you connect it to your graphics card you have to figure out a way to combine your two images so that the monitor correctly show them.

On expensive workstation graphics cards, such as the Nvidia Quadra series, the driver can automatically perform this for example using the OpenGL stereo buffer. However, today’s cheaper “gaming” graphics cards should have that functionality although I guess it is not present in the driver. (At least it wasn’t under Linux when I tried.) I was using a GeForce 9800GT as I recall, but you should be able to program this on any card with a pixel/fragment shader.

So, in order to program our card to send the correct image for stereo-viewing we need to figure out how it interprets the data. Combining several series of data (two images in our case) into a single series (one image) is called multiplexing. There are two different basic ways we can perform multiplexing for the Sharp screen: in time and in space (or well frequency, but I will call it space here).

If the Sharp screen worked by time multiplexing it would accept every second image as the right eye image and every second as the left eye image and then combine them into a single image (or show every second somehow, this is how shutter glasses work by the way).

This kind of screen works by spatial multiplexing though. (I figured this out by implementing the method below and testing. However, it was a fairly safe bet as the barriers themselves divide the resolution of the screen.) So the left and right eye images must be combined into a single image in the frame buffer of the graphics card.

Screen image format

There are of course a great number of different ways to take data from two RGB images and combine them into a single one. The simplest is probably to put the left image in the left half and the right image in the right half. However, considering that the parallax barriers are thin vertical strips over the entire LCD this would require the screen hardware to rearrange the data i an extra processing step. Thus this seem unlikely.

Rather, it would make sense for the monitor to require the image in a ready-to-display format. Thus, in the exact striped pattern as the parallax barrier.

Following this, it would make sense to try to take the first pixel column from the left eye image, the second from the right eye image, the third from the left again, and so on. If one does this however the result is not a stereoscopic image, but rather a striped, blurry version with offset colors. Clearly something is wrong with this approach.

This is the stage where I have seen some experiments at before. The screen does seem to display a different image for each eye, but not the correct one. And what is up with the colors?

Figure 1: Pixel geometries by Peter Halasz (CC 3.0 Share alike license). Our LCD layout is the lower right quadrant.

Focusing on that last question will let us solve the problem. Most color LCD screens has a separate sub-pixel, or element, for each red, green, and blue component. See for example Wikipeda and the lower right part of Fig. 1. A layout like this would create a RGB columns. What if the parallax barriers blocks color channels in stead of whole pixels? This would actually benefit the smoothness of the whole image by making the ‘jumps’ between barriers smaller. So, is it possible to have barriers arranged over the individual color channels and still manage to have the right red, green, and blue values to show simultaneously for each image?

The solution

Figure 2: A part of an LCD line with the different red, green, and blue elements covered by left and right polarizing barriers. Dashed vertical lines indicate pixel borders. The two different wavy patterns are Left (L) and Right (R) polarizing barrier respectively.

It turns out the the answer to that question is yes, and this is also the solution. Consider a barrier pattern of the same size as the color elements. Fig. 2 shows a part of a screen line. The color elements repeats: red, green, blue; red, green, blue; with three elements making up a screen pixel. However, the left and right barriers alternate per element (and not per pixel) as indicated by the pattern overlays and the L and R labels in the image. Now, let us have a look at what information are sifted out by the two different barriers, as shown in Fig. 3.

Figure 3: The L and R barrier separate the image displayed on screen into two separate images. The upper part of the image shows the L(eft) part and the lower the (R)ight part. The boxes show perceived pixels in the new image lines. As can be seen both new images still provide a series of red, green, and blue color triplets. The order of the color elements within one triplet has changed, but this does not have any effect on the color of the perceived pixel.

Note that in Fig. 3 that the barrier separate every second color element into the left image and every second into the right image. The resulting images still have periodic red, green, and blue triplets that may be interpreted as pixels. The boxes in Fig. 3 denotes the a perceived left and right image pixel. (It is larger than the screen pixel, but that is only natural as we multiplex two images in the screen image.) As can be seen, the triplets’ color elements are permuted internally so that the left image has a red, blue, green order and the right one a green, red, blue order. This does not matter however. The perceived color of a pixel is the combined contributions from each of the color components. (This is why all the different pixel geometries shown in Fig. 1 works and all gives the same colors.) What matters is that one element of each base color lies next to each other.

This layout explain why we can not simply take every second pixel in the screen image from every second of the two images we want to multiplex. Each screen pixel gets chopped up so that two of its color elements go to one of the perceived images and one to the other. Thus any perceived pixel will have one wrong color component (and thus also intensity) contribution. This should lead to somewhat blurred image with a color offset, which is exactly what the every second pixel experiment showed.

Now, instead what we need to do when combining the images is exactly the inverse process of Fig. 3: for each pixel on screen we take two color components from one image and one from the other. For the next screen pixel we take the single remaining component from the first image pixel and the two remaining components from the second image pixel, and so on. The algorithm below outlines how to combine two images using this method.

Input: fullScreenWidth, fullScreenHeight // Size of the
                                         // full screen framebuffer
Input: LeftImage, RightImage // Image arrays of size
                             // fullScreenWidth /2 , fullScreenHeight
Output: FrameBuffer // Image of size fullScreenWidth, fullScreenHeight

for y = 0 to fullScreenHeight -1
 for x = 0 to fullScreenWidth -1
   if(x is even) then
      FrameBuffer[x,y] = RGB( LeftImage[x/2,y].red,
                              RightImage[x/2,y].green,
                              LeftImage[x/2,y].blue);
    else
      FrameBuffer[x,y] = RGB(RightImage[x/2,y].red,
                             LeftImage[x/2,y].green,
                             RightImage[x/2,y].blue);
    end if;
  end for;
end for;

The above algorithm should multiplex the two images correctly so that if a stereo pair is used as input the resulting perceived image from the Sharp screen (with 3D turned on) is indeed a 3D image.

As I stated in the introduction, I am not able to provide any screen shots or code from my own experiments, but this is anyway how it works. Let me know if you spot any errors in the text, and if anyone with this screen implements a program after reading this it would be fun to know that it worked out for you as well.

.L

Nikos emailed me some time back and told me about an alternative way on Ubuntu/debian systems called update-alternatives. I have not gotten around to try it out myself, because I have not had any need for it. Posting part of his email below if anyone want to give it a shot however:

There is another debian/ubuntu specific fix that is very handy (imho).
There is a debian utility named update-alternatives which does exatly what you
descibed, only it is more sophisticated!
In case you are interested ( even though cuda 2.3 is supposed to work with
ubuntu 9.04):

     sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.1 60
--slave /usr/bin/g++ g++ /usr/bin/g++-4.1

    sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.3 40
--slave /usr/bin/g++ g++ /usr/bin/g++-4.3

then you can use sudo update-alternatives --config gcc to choose which version
maps to "gcc"

Original post:

I have been doing CUDA development under GNU/Linux for over a year now, and it works remarkably well. There is one small issue that may confuse developers however; the Nvidia CUDA SDK currently can not handle gcc versions higher than 4.1 . The both newer Ubuntu (I am running Kubuntu) releases – 8.04 and 8.10 have higher versions (4.2 and 4.3 respectively) installed.

There is probably some good page about out how to make the system compile using the right compiler and libc somewhere or some switch to set, but at least I could not find it when installing 8.04 so I had to figure it out back then. As I had to rewrite my script after performing a clean 8.10 install a couple of weeks back I thought it might be handy to post it here. It is a rather straight forward process.

First one need to install gcc4.1 (and g++ if developing in C++) as well as the proper libc version. This can be done using any of the apt tools avaliable in any ubuntu distro, and is fairly straight forward. (Just install the g++4.1 development package, and the rest will follow from dependencies). Do not remove the already installed version (4.2 or 4.3) however, as they can happily coexist and you may want to compile some kernel module or Nvidia driver at some stage.

Second, we must make sure that the right gcc version is used. In /usr/bin/ there are three symbolic links that needs to be changed. gcc, cpp, and g++. If you perform ls of one of the files, like

$ ls -l /usr/bin/gcc
lrwxrwxrwx 1 root root 16 2008-11-03 14:14 /usr/bin/gcc -> /usr/bin/gcc-4.3

you can see that it is a link to gcc-4.3 (a binary file). Same thing goes for cpp and g++ . There should also be binary files for the corresponding 4.1 version (e.g gcc-4.1). One way of changing the compiler version used by the system is simply a matter of removing this link and making a new one to the proper executable. As it may come in handy to change back (I tend to want to use the newest g++ when developing non CUDA software) and forth, a couple of simple shell scripts will come in handy. To set the compiler version to 4.1 simply put the following in a file, and save it in som convenient location (I save in /usr/bin for simplicity). I call my file set_compiler_4.1.sh but you can of course choose your own names.

#!/bin/bash
rm /usr/bin/cpp
ln -s /usr/bin/cpp-4.1 /usr/bin/cpp
echo ln -s cpp-4.1 cpp
rm /usr/bin/gcc
ln -s /usr/bin/gcc-4.1 /usr/bin/gcc
echo ln -s gcc-4.1 gcc
rm /usr/bin/g++
ln -s /usr/bin/g++-4.1 /usr/bin/g++
echo ln -s g++-4.1 g++

If you are not comfortable with shell scripting (I am but a rookie myself), rm removes the link, ln creates a link (-s makes it a soft link) and echo merely echoes the argument (I do this just to remind myself what is happening).

Put a similar thing for 4.2 and, or 4.3 in its own file, so that you easily can reset, and make sure that the shell scripts are executable

sudo chmod +x set_compiler_4.1.sh

Now to set a a compiler version, simply call the script with superuser privileges

sudo set_compiler_4.1.sh

for instance.

That is about it. Long text for a simple fix…

Ah, and be careful with those rm’s in superuser mode if you are new to GNU/Linux.

.L