For VR, the position and orientation of the eye in the virtual world are given by a tracking system and possibly controller inputs. By contrast, in computer graphics, it is common to start with a description of where the eye is located and which way it is looking. This is called a look-at, and has the following components:
The second direction indicates which way is up. Imagine holding a camera out as if you are about to take a photo and then performing a roll rotation. You can make level ground appear to be slanted or even upside down in the picture. Thus, indicates the up direction for the virtual camera or eye.
We now construct the resulting transform from (3.36). The translation components are already determined by , which was given in the look-at. We need only to determine the rotation , as expressed in (3.35). Recall from Section 3.2 that the matrix columns indicate how the coordinate axes are transformed by the matrix (refer to (3.7) and (3.8)). This simplifies the problem of determining . Each column vector is calculated as
Steven M LaValle 2020-01-06