What Is
Virtual Reality?
A
Web-Based Introduction
Version 4
– Draft 1, September, 1998
Jerry
Isdale,
email: isdale@acm.org
(preferred)
(alternate
email: isdale@compuserve.com)
(with thanks to the *many* people who contributed bits,
bytes and words either directly to me or by posting to various electronic
sources, especially Chris Hand (caution
- moving soon) and Toni Emerson
(Note: there are a several older versions of this document
out on the net, with the subtitle "A Homebrew Introduction and Information
Resource". These versions are many years older than the one you are now
reading. URL keepers, please note and update your pointer lists!) This document freely distributable to various electronic
networks, BBS, etc. It can be used as a handout for non-profit seminars
sponsored by schools and professional associations. I only ask that you keep my
name as the primary author/editor and do not charge for it beyond normal
on-line connect charges. If you have any corrections, comments or additions,
please send them to me at one of the above email addresses. The now defunct US Congressional Office of Technology
Accessment incorporated large parts of one of the earlier versions in their
report to congress on "Virtual Reality and Technologies for Combat
Simulation" Get a Copy This paper was originally divided into two parts. The first
section was the basic text and the second was a collection of information
sources. However, the rapid change of sources, especially on the net, makes it
very difficult to keep up. Therefore I have eliminated the second section and
refer instead to two excellent sources that do keep a bit more up to date: HUMOR: Every emerging technology needs it’s Evil Scientist. Virtual Reality has The Evil Dr. Flaxon and
his Flaxon Alternative Interface
Technology (FAIT) Labs. Caution!
These people are very dangerous and have been known to cause severe
psychological and physiological problems in their "volunteers". This section provides a relatively simple taxonomy
(meaning:) of Virtual Reality. There
are several much more rigorous taxonomies covering VR. An excellent short treatment of the state of the art and a
taxonomy of VR is a report on the US Government's National Science Foundation
invitational workshop on Interactive Systems Program held March 23-24,
1992. It was published in the given in
the ACM Siggraph publication "Computer Graphics", Vol. 26, #3, August
1992. The purpose of the workshop was to identify and recommend future research
directions in the area of virtual environments. A longer exposition of this taxonomy can be found in the MIT
Journal "Presence" Vol. 1 #2 (Synthetic Experience: A Proposed
Taxonomy By Warren Robinett.) The term Virtual
Reality (VR) is used by many different people with many meanings. There are
some people to whom VR is a specific collection of technologies, that is a Head
Mounted Display, Glove Input Device and Audio. Some other people stretch the
term to include conventional books, movies or pure fantasy and imagination. The
NSF taxonomy mentioned in the introduction can cover these as well. However, my
personal preference, and for purposes of this paper, we restrict VR to computer
mediated systems. The best definition of Virtual Reality I have seen to date
comes from the The
Silicon Mirage": "Virtual
Reality is a way for humans to visualize, manipulate and interact with computers
and extremely complex data" The visualization
part refers to the computer generating visual, auditory or other sensual
outputs to the user of a world within the computer. This world may be a CAD
model, a scientific simulation, or a view into a database. The user can
interact with the world and directly manipulate objects within the world. Some
worlds are animated by other processes, perhaps physical simulations, or simple
animation scripts. Interaction with the virtual world, at least with near real
time control of the viewpoint, in my
opinion, is a critical test for a 'virtual reality'. Some people object
to the term "Virtual Reality", saying it is an oxymoron. Other terms
that have been used are Synthetic Environments, Cyberspace, Artificial Reality,
Simulator Technology, etc. VR is the most common and sexiest. It has caught the
attention of the media. The applications
being developed for VR run a wide spectrum, from games to architectural and
business planning. Many applications are worlds that are very similar to our
own, like CAD or architectural modeling. Some applications provide ways of
viewing from an advantageous perspective not possible with the real world, like
scientific simulators and telepresense systems, air traffic control systems.
Other applications are much different from anything we have ever directly
experienced before. These latter applications may be the hardest, and most
interesting systems. Visualizing the ebb and flow of the world's financial
markets. Navigating a large corporate information base, etc. A major distinction
of VR systems is the mode with which they interface to the user. This section
describes some of the common modes used in VR systems. Some systems use a
conventional computer monitor to display the visual world. This sometimes
called Desktop VR or a Window on a World (WoW). This concept traces its lineage back through the entire history
of computer graphics. In 1965, Ivan Sutherland laid out a research program for
computer graphics in a paper called "The Ultimate Display" that has
driven the field for the past nearly thirty years. "One must look
at a display screen," he said, "as a window through which one beholds
a virtual world. The challenge to computer graphics is to make the picture in
the window look real, sound real and the objects act real." [quoted from
Computer Graphics V26#3] A variation of the
WoW approach merges a video input of the user's silhouette with a 2D computer
graphic. The user watches a monitor that shows his body's interaction with the
world. Myron Kruger has been a champion of this form of VR since the late 60's.
He has published two books on the subject: "Artificial Reality" and
"Artificial Reality II". At least one commercial system uses this
approach, the Mandala system. This system is based on a Commodore Amiga with
some added hardware and software. A version of the Mandala is used by the cable
TV channel Nickelodeon for a game show (Nick Arcade) to put the contestants
into what appears to be a large video game. The ultimate VR
systems completely immerse the user's personal viewpoint inside the virtual
world. These "immersive" VR systems are often equipped with a Head
Mounted Display (HMD). This is a helmet or a face mask that holds the visual
and auditory displays. The helmet may be free ranging, tethered, or it might be
attached to some sort of a boom armature. A nice variation of
the immersive systems use multiple large projection displays to create a 'Cave'
or room in which the viewer(s) stand.
An early implementation was called "The Closet Cathedral" for
the ability to create the impression of an immense environment. within a small
physical space. The Holodeck used in the television series "Star Trek: The
Next Generation" is afar term extrapolation of this technology. Telepresence is a
variation on visualizing complete computer generated worlds. This a
technology links remote sensors in the
real world with the senses of a human operator. The remote sensors might be
located on a robot, or they might be on the ends of WALDO like tools. Fire
fighters use remotely operated vehicles to handle some dangerous conditions.
Surgeons are using very small instruments on cables to do surgery without
cutting a major hole in their patients. The instruments have a small video
camera at the business end. Robots
equipped with telepresence systems have already changed the way deep sea and
volcanic exploration is done. NASA plans to use telerobotics for space
exploration. There is currently a joint US/Russian project researching
telepresence for space rover exploration. Merging the
Telepresence and Virtual Reality systems gives the Mixed Reality or Seamless
Simulation systems. Here the computer generated inputs are merged with
telepresence inputs and/or the users view of the real world. A surgeon's view
of a brain surgery is overlaid with images from earlier CAT scans and real-time
ultrasound. A fighter pilot sees computer generated maps and data displays
inside his fancy helmet visor or on cockpit displays. The phrase
"fish tank virtual reality" was used to describe a Canadian VR system
reported in the 1993 InterCHI proceedings. It combines a stereoscopic monitor
display using liquid crystal shutter glasses with a mechanical head tracker.
The resulting system is superior to simple stereo-WoW systems due to the motion
parallax effects introduced by the head tracker. (see INTERCHI '93 Conference
Proceedings, ACM Press/Addison Wesley , ISBN 0-201-58884-6) There are a number
of specialized types of hardware devices that have been developed or used for
Virtual Reality applications. One of the most
time consuming tasks in a VR system is the generation of the images. Fast computer graphics opens a very large
range of applications aside from VR, so there has been a market demand for
hardware acceleration for a long while. There are currently a number of vendors
selling image generator cards for PC level machines, many of these are based on
the Intel i860 processor. These cards range in price from about $2000 up to $6
or $10,000. Silicon Graphics Inc. has
made a very profitable business of producing graphics workstations. SGI boxes are
some of the most common processors found in VR laboratories and high end
systems. SGI boxes range in price from under $10,000 to over $100,000. The
simulator market has produced several companies that build special purpose
computers designed expressly for real time image generation. These computers
often cost several hundreds of thousands of dollars. One key element for
interaction with a virtual world, is a means of tracking the position of a real
world object, such as a head or hand. There are numerous methods for position
tracking and control. Ideally a technology should provide 3 measures for
position(X, Y, Z) and 3 measures of orientation (roll, pitch, yaw). One of the
biggest problem for position tracking is latency, or the time required to make
the measurements and preprocess them before input to the simulation engine. The simplest
control hardware is a conventional mouse, trackball or joystick. While these
are two dimensional devices, creative programming can use them for 6D controls.
There are a number of 3 and 6 dimensional mice/trackball/joystick devices being
introduced to the market at this time. These add some extra buttons and wheels
that are used to control not just the XY translation of a cursor, but its Z
dimension and rotations in all three directions. The Global Devices 6D Controller is one such 6D joystick It looks
like a racket ball mounted on a short stick. You can pull and twist the ball in
addition to the left/right & forward/back of a normal joystick. Other 3D
and 6D mice, joystick and force balls are available from Logitech, Mouse System
Corp. among others. One common VR
device is the instrumented glove. The use of a glove to manipulate objects in a
computer is covered by a basic patent in the USA. Such a glove is outfitted with sensors on the fingers as well as
an overall position/orientation tracker. There are a number of different types
of sensors that can be used. VPL (holders of the patent) made several
DataGloves, mostly using fiber optic sensors for finger bends and magnetic
trackers for overall position. Mattel manufactured the PowerGlove for use with
the Nintendo game system, for a short time.
This device is easily adapted to interface to a personal computer. It
provides some limited hand location and finger position data using strain
gauges for finger bends and ultrasonic position sensors. The gloves are getting
rare, but some can still be found at Toys R' Us and other discount stores. Anthony Clifton recently posted this suggestion
for a" very good resource for PowerGloves etc.: small children. A friend's son had gotten a glove a couple years
ago and almost NEVER used it, so I bought it off the kid. Remember children like money more than toys
they never use." The concept of an
instrumented glove has been extended to other body parts. Full body suits with
position and bend sensors have been used for capturing motion for character
animation, control of music synthesizers, etc. in addition to VR applications. Mechanical armatures
can be used to provide fast and very accurate tracking. Such armatures may look
like a desk lamp (for basic position/orientation) or they may be highly complex
exoskeletons (for more detailed positions). The drawbacks of mechanical sensors
are the encumbrance of the device and its restrictions on motion. Exos Systems
builds one such exoskeleton for hand control. It also provides force feedback.
Shooting Star system makes a low cost armature system for head tracking. Fake
Space Labs and LEEP Systems make much more expensive and elaborate armature
systems for use with their display systems. Ultrasonic sensors
can be used to track position and orientation. A set of emitters and receivers are used with a known
relationship between the emitters and between the receivers. The emitters are
pulsed in sequence and the time lag to each receiver is measured. Triangulation
gives the position. Drawbacks to ultrasonics are low resolution, long lag times
and interference from echoes and other noises in the environment. Logitech and
Transition State are two companies that provide ultrasonic tracking systems. Magnetic trackers
use sets of coils that are pulsed to produce magnetic fields. The magnetic
sensors determine the strength and angles of the fields. Limitations of these
trackers are a high latency for the measurement and processing, range
limitations, and interference from ferrous materials within the fields.
However, magnetic trackers seem to be one of the preferred methods. The two
primary companies selling magnetic trackers are Polhemus and Ascension. Optical position
tracking systems have been developed. One method uses a ceiling grid LEDs and a
head mounted camera. The LEDs are pulsed in sequence and the cameras image is
processed to detect the flashes. Two problems with this method are limited
space (grid size) and lack of full motion (rotations). Another optical method
uses a number of video cameras to capture simultaneous images that are
correlated by high speed computers to track objects. Processing time (and cost
of fast computers) is a major limiting factor here. One company selling an
optical tracker is Origin Instruments. Inertial trackers
have been developed that are small and accurate enough for VR use. However,
these devices generally only provide rotational measurements. They are also not
accurate for slow position changes. Stereo vision is
often included in a VR system. This is accomplished by creating two different
images of the world, one for each eye. The images are computed with the
viewpoints offset by the equivalent distance between the eyes. There are a
large number of technologies for presenting these two images. The images can be
placed side-by-side and the viewer asked (or assisted) to cross their eyes. The images can be projected through
differently polarized filters, with corresponding filters placed in front of
the eyes. Anaglyph images user red/blue glasses to provide a crude (no color)
stereovision. The two images can
be displayed sequentially on a conventional monitor or projection display.
Liquid Crystal shutter glasses are then used to shut off alternate eyes in
synchronization with the display. When the brain receives the images in rapid
enough succession, it fuses the images into a single scene and perceives depth.
A fairly high display swapping rate (min. 60hz) is required to avoid perceived
flicker. A number of companies made low cost LC shutter glasses for use with
TVs (Sega, Nintendo, Toshiba, etc.). There are circuits and code for hooking
these up to a computer available on many of the On-line systems, BBSs and
Internet FTP sites mentioned later. However, locating the glasses themselves is
getting difficult as none are still being made or sold for their original use.
Stereographics sells a very nice commercial LC shutter system called
CrystalEyes. Another alternative
method for creating stereo imagery on a computer is to use one of several split
screen methods. These divide the
monitor into two parts and display left and right images at the same time. One
method places the images side by side and conventionally oriented. It may not use the full screen or may
otherwise alter the normal display aspect ratio. A special hood viewer is
placed against the monitor which helps the position the eyes correctly and may
contain a divider so each eye e sees only its own image. Most of these hoods,
such as the one for the V5 of Rend386, use fresnel lenses to enhance the
viewing. An alternative split screen
method orients the images so the top of each points out the side of the
monitor. A special hood containing
mirrors is used to correctly orient the images. A very nice low cost
(under $200) unit of this type is the Cyberscope available from Simsalabim. There was an article
supplement with CyberEdge Journal issue #17 entitled "What's Wrong with
your Head Mounted Display". It is a summary report on the findings of a
study done by the Edinburgh Virtual Environment Lab, Dept. of Psychology, Univ.
of Edinburgh on the eye strain effects of stereoscopic Head Mounted Displays.
There have been a number of anecdotal reports of stress with HMDs and other
stereoscopic displays, but few, if any, good clinical studies. This study was
done very carefully and the results are a cause for some concern. The basic test was
to put 20 young adults on a stationary bicycle and let them cycle around a
virtual rural road setting using a HMD (VPL LX EyePhone and a second HMD LEEP
optic equipped system). After 10 minutes of light exercise, the subjects were
tested... "The results
were alarming: measures of distance vision , binocular fusion and convergence
displayed clear signs of binocular stress in a significant number of the
subjects. Over half the subjects also reported symptoms of such stress, such as
blurred vision." The article goes on
to describe the primary reason for the stress - the difference between the
image focal depth and the disparity. Normally, the when your eyes look at a
close object they focus (accommodate) close and also rotate inward (converge).
When they accommodate on a far object, the eyes also diverge. However, a
stereoscopic display does not change the either the effective focal plane (set
by the optics) and the disparity depth. The eyes strain to decouple the
signals. The article
discusses some potential solutions, but notes that most of them (dynamic
focal/disparity) are difficult to implement. It mentions monoscopic HMDs only
to say that while they would seem to avoid the problems, they were not tested. The
article does not discuss non-HMD stereoscopic devices at all, but I would
extrapolate that they should show some similar problems. The full article is
available from CyberEdge Journal for a small fee. There has been a
fair bit of discussion ongoing in the sci.virtual-worlds newsgroup (check the
Sept./Oct. 93 archives) about this and some other studies. One contributor,
Dipl.-Ing. Olaf H. Kelle, University of Wuppertal, Germany, reported only 10%
of his users showing eye strain. His system is setup with a focal depth of 3m
which seems to be a better, more comfortable viewing distance. Others have
noted that long duration monitor use often leads to the user staring or not
blinking. It is common for VDT users to
be cautioned to look away from the screen occasionally to adjust their focal
depth and to blink. Another contributor, John Nagle provided the following list
of other potential problems with HMDs: electrical safety, Falling/tripping over
real world objects, simulator sickness (disorientation due to conflicting
motion signals from eyes and inner ear), Eye Strain, Induced post-HMD accidents
("some flight simulators some flight simulators, usually those for
military fighter aircraft, it's been found necessary to forbid simulator users
to fly or drive for a period of time after flying the simulator".). One hardware device
closely associated with VR is the Head
Mounted Device (HMD). These use some sort of helmet or goggles to place small
video displays in front of each eye, with special optics to focus and stretch the perceived field of
view. Most HMDs use two displays and
can provide stereoscopic imaging. Others use a single larger display to provide
higher resolution, but without the stereoscopic vision. Most lower cost HMDs ($3000-10,000 range ) use LCD displays,
while others use small CRTs, such as those found in camcorders. The more
expensive HMDs use special CRTs mounted along side the head or optical fibers
to pipe the images from non-head mounted displays. ($60,000 and up). A HMD
requires a position tracker in addition to the helmet. Alternatively, the
display can be mounted on an armature for support and tracking (a Boom
display). The following
defines a number of levels of VR hardware systems. These are not hard levels,
especially towards the more advanced systems. The 'Entry Level'
VR system takes a stock personal computer or workstation and implements a WoW
system. The system may be based on an IBM clone (MS-DOS/Windows) machine or an
Apple Macintosh, or perhaps a Commodore Amiga. The DOS type machines (IBM PC
clones) are the most prevalent. There are Mac based systems, but few very fast
rendering ones. Whatever the base
computer it includes a graphic display,
a 2D input device like a mouse, trackball or joystick, the keyboard, hard disk & memory. The next step up
from an EVR system adds some basic interaction and display enhancements. Such enhancements would include a
stereographic viewer (LCD Shutter glasses)
and a input/control device such as the Mattel PowerGlove and/or a
multidimensional (3D or 6D) mouse or joystick. The next step up
the VR technology ladder is to add a rendering accelerator and/or frame buffer
and possibly other parallel processors for input handling, etc. The simplest
enhancement in this area is a faster display card. For the PC class machines,
there are a number of new fast VGA and SVGA accelerator cards. These can make a
dramatic improvement in the rendering performance of a desktop VR system. Other
more sophisticated image processors based on the Texas Instruments TI34020 or
Intel i860 processor can make even more dramatic improvements in rendering
capabilities. The i860 in particular is in many of the high end professional
systems. The Silicon Graphics Reality Engine uses a number of i860 processors
in addition to the usual SGI workstation hardware to achieve stunning levels of
realism in real time animation. An AVR system might
also add a sound card to provide mono, stereo or true 3D audio output. Some
sound cards also provide voice recognition. This would be an excellent
additional input device for VR applications. An Immersion VR
system adds some type of immersive display system: a HMD, a Boom, or multiple
large projection type displays (Cave). An IVR system might
also add some form of tactile, haptic and touch feedback interaction
mechanisms. The area of Touch or Force Feedback (known collectively as Haptics)
is a very new research arena. A common variation
on VR is to use a Cockpit or Cab compartment to enclose the user. The virtual
world is viewed through some sort of view screen and is usually either
projected imagery or a conventional monitor. The cockpit simulation is very
well known in aircraft simulators, with a history dating back to the early Link
Flight Trainers (1929?). The cockpit is often mounted on a motion platform that
can give the illusion of a much larger range of motion. Cabs are also used in
driving simulators for ships, trucks, tanks and 'battle mechs'. The latter are
fictional walking robotic devices (i.e. the Star Wars films). The BattleTech
location based entertainment (LBE) centers use this type of system. One of the biggest
VR projects is the Defense Simulation Internet. This project is a
standardization being pushed by the USA Defense Department to enable diverse
simulators to be interconnected into a vast network. It is an outgrowth of
the Defense Advanced Research Projects
Administration (DARPA) SIMNET project of the later 1980s. SIMNET was/is a collection of tank
simulators (Cab type) that are networked together to allow unit tactical
training. Simulators in Germany can operate in the same virtual world as
simulators in the USA, partaking of the same battle exercise. The basic
Distributed Interactive Simulation (DIS) protocol has been defined by the
Orlando Institute for Simulation & Training. It is the basis for the next
generation of SIMNET, the Defense Simulation Internet (DSI). (love those
acronyms!) An accessible, if somewhat dark,
treatment of SIMNET and DSI can be found in the premier issue of WIRED
magazine (January 1993) entitled "War is Virtual Hell" by Bruce
Sterling. The basic DIS
protocol has been adopted as a standard for communication between distributed
simulations by the IEEE. Basic
information on DIS and SIMNET, including a C library to support the
communication protocol is available via FTP from the Internet site
taurus.cs.nps.navy.mil (pub/warbreaker/NPS_DIS...). Other contact points for
DIS include: Danette Haworth
Institute for Simulation & Training 12424 Research Parkway, Suite 300
Orlando, Florida 32826 (407)658-5000 ModSIM (the language) is available via ftp from
max.cecer.army.mil in the isle directory. ModSAF is being developed to create "Semi-Automated
Forces" - both vehicle based and dismounted. There is a body of research
and techniques on the various levels of scripting behaviors. Integrated Simulation (Systems) Language Environment, (ISLE)
based on ModSIM with extensions to support Imperative Behavior programming
(Prolog-like), an
2.
A Taxonomy of Virtual Reality
3.
Types of VR Systems
3.1.
Window on World Systems (WoW)
3.2.
Video Mapping
3.3.
Immersive Systems
3.4.
Telepresence
3.5.
Mixed Reality
4.
VR Hardware
4.1.
Image Generators
4.2.
Manipulation and Control Devices
4.3.
Stereo Vision
1.1.1. Health
Hazards from Stereoscopic Displays
4.4.
Head Mounted Display (HMD)
4.5.
Force and Touch (Haptic) Rendering
4.6. Motion
Rendering
5.
Levels of VR Hardware Systems
5.1.
Entry VR (EVR)
5.2.
Basic VR (BVR)
5.3.
Advanced VR (AVR)
5.4.
Immersion VR (IVR)
5.5.
SIMNET, Defense Simulation Internet