What Is
Virtual Reality?
A
Web-Based Introduction
Version 4
– Draft 1, September, 1998
Jerry
Isdale,
email: isdale@acm.org
(preferred)
(alternate
email: isdale@compuserve.com)
(with thanks to the *many* people who contributed bits,
bytes and words either directly to me or by posting to various electronic
sources, especially Chris Hand (caution
- moving soon) and Toni Emerson
(Note: there are a several older versions of this document
out on the net, with the subtitle "A Homebrew Introduction and Information
Resource". These versions are many years older than the one you are now
reading. URL keepers, please note and update your pointer lists!) This document freely distributable to various electronic
networks, BBS, etc. It can be used as a handout for non-profit seminars
sponsored by schools and professional associations. I only ask that you keep my
name as the primary author/editor and do not charge for it beyond normal
on-line connect charges. If you have any corrections, comments or additions,
please send them to me at one of the above email addresses. The now defunct US Congressional Office of Technology
Accessment incorporated large parts of one of the earlier versions in their
report to congress on "Virtual Reality and Technologies for Combat
Simulation" Get a Copy This paper was originally divided into two parts. The first
section was the basic text and the second was a collection of information
sources. However, the rapid change of sources, especially on the net, makes it
very difficult to keep up. Therefore I have eliminated the second section and
refer instead to two excellent sources that do keep a bit more up to date: HUMOR: Every emerging technology needs it’s Evil Scientist. Virtual Reality has The Evil Dr. Flaxon and
his Flaxon Alternative Interface
Technology (FAIT) Labs. Caution!
These people are very dangerous and have been known to cause severe
psychological and physiological problems in their "volunteers". This section provides a relatively simple taxonomy
(meaning:) of Virtual Reality. There
are several much more rigorous taxonomies covering VR. An excellent short treatment of the state of the art and a
taxonomy of VR is a report on the US Government's National Science Foundation
invitational workshop on Interactive Systems Program held March 23-24,
1992. It was published in the given in
the ACM Siggraph publication "Computer Graphics", Vol. 26, #3, August
1992. The purpose of the workshop was to identify and recommend future research
directions in the area of virtual environments. A longer exposition of this taxonomy can be found in the MIT
Journal "Presence" Vol. 1 #2 (Synthetic Experience: A Proposed
Taxonomy By Warren Robinett.) The term Virtual
Reality (VR) is used by many different people with many meanings. There are
some people to whom VR is a specific collection of technologies, that is a Head
Mounted Display, Glove Input Device and Audio. Some other people stretch the
term to include conventional books, movies or pure fantasy and imagination. The
NSF taxonomy mentioned in the introduction can cover these as well. However, my
personal preference, and for purposes of this paper, we restrict VR to computer
mediated systems. The best definition of Virtual Reality I have seen to date
comes from the The
Silicon Mirage": "Virtual
Reality is a way for humans to visualize, manipulate and interact with computers
and extremely complex data" The visualization
part refers to the computer generating visual, auditory or other sensual
outputs to the user of a world within the computer. This world may be a CAD
model, a scientific simulation, or a view into a database. The user can
interact with the world and directly manipulate objects within the world. Some
worlds are animated by other processes, perhaps physical simulations, or simple
animation scripts. Interaction with the virtual world, at least with near real
time control of the viewpoint, in my
opinion, is a critical test for a 'virtual reality'. Some people object
to the term "Virtual Reality", saying it is an oxymoron. Other terms
that have been used are Synthetic Environments, Cyberspace, Artificial Reality,
Simulator Technology, etc. VR is the most common and sexiest. It has caught the
attention of the media. The applications
being developed for VR run a wide spectrum, from games to architectural and
business planning. Many applications are worlds that are very similar to our
own, like CAD or architectural modeling. Some applications provide ways of
viewing from an advantageous perspective not possible with the real world, like
scientific simulators and telepresense systems, air traffic control systems.
Other applications are much different from anything we have ever directly
experienced before. These latter applications may be the hardest, and most
interesting systems. Visualizing the ebb and flow of the world's financial
markets. Navigating a large corporate information base, etc. A major distinction
of VR systems is the mode with which they interface to the user. This section
describes some of the common modes used in VR systems. Some systems use a
conventional computer monitor to display the visual world. This sometimes
called Desktop VR or a Window on a World (WoW). This concept traces its lineage back through the entire history
of computer graphics. In 1965, Ivan Sutherland laid out a research program for
computer graphics in a paper called "The Ultimate Display" that has
driven the field for the past nearly thirty years. "One must look
at a display screen," he said, "as a window through which one beholds
a virtual world. The challenge to computer graphics is to make the picture in
the window look real, sound real and the objects act real." [quoted from
Computer Graphics V26#3] A variation of the
WoW approach merges a video input of the user's silhouette with a 2D computer
graphic. The user watches a monitor that shows his body's interaction with the
world. Myron Kruger has been a champion of this form of VR since the late 60's.
He has published two books on the subject: "Artificial Reality" and
"Artificial Reality II". At least one commercial system uses this
approach, the Mandala system. This system is based on a Commodore Amiga with
some added hardware and software. A version of the Mandala is used by the cable
TV channel Nickelodeon for a game show (Nick Arcade) to put the contestants
into what appears to be a large video game. The ultimate VR
systems completely immerse the user's personal viewpoint inside the virtual
world. These "immersive" VR systems are often equipped with a Head
Mounted Display (HMD). This is a helmet or a face mask that holds the visual
and auditory displays. The helmet may be free ranging, tethered, or it might be
attached to some sort of a boom armature. A nice variation of
the immersive systems use multiple large projection displays to create a 'Cave'
or room in which the viewer(s) stand.
An early implementation was called "The Closet Cathedral" for
the ability to create the impression of an immense environment. within a small
physical space. The Holodeck used in the television series "Star Trek: The
Next Generation" is afar term extrapolation of this technology. Telepresence is a
variation on visualizing complete computer generated worlds. This a
technology links remote sensors in the
real world with the senses of a human operator. The remote sensors might be
located on a robot, or they might be on the ends of WALDO like tools. Fire
fighters use remotely operated vehicles to handle some dangerous conditions.
Surgeons are using very small instruments on cables to do surgery without
cutting a major hole in their patients. The instruments have a small video
camera at the business end. Robots
equipped with telepresence systems have already changed the way deep sea and
volcanic exploration is done. NASA plans to use telerobotics for space
exploration. There is currently a joint US/Russian project researching
telepresence for space rover exploration. Merging the
Telepresence and Virtual Reality systems gives the Mixed Reality or Seamless
Simulation systems. Here the computer generated inputs are merged with
telepresence inputs and/or the users view of the real world. A surgeon's view
of a brain surgery is overlaid with images from earlier CAT scans and real-time
ultrasound. A fighter pilot sees computer generated maps and data displays
inside his fancy helmet visor or on cockpit displays. The phrase
"fish tank virtual reality" was used to describe a Canadian VR system
reported in the 1993 InterCHI proceedings. It combines a stereoscopic monitor
display using liquid crystal shutter glasses with a mechanical head tracker.
The resulting system is superior to simple stereo-WoW systems due to the motion
parallax effects introduced by the head tracker. (see INTERCHI '93 Conference
Proceedings, ACM Press/Addison Wesley , ISBN 0-201-58884-6) There are a number
of specialized types of hardware devices that have been developed or used for
Virtual Reality applications. One of the most
time consuming tasks in a VR system is the generation of the images. Fast computer graphics opens a very large
range of applications aside from VR, so there has been a market demand for
hardware acceleration for a long while. There are currently a number of vendors
selling image generator cards for PC level machines, many of these are based on
the Intel i860 processor. These cards range in price from about $2000 up to $6
or $10,000. Silicon Graphics Inc. has
made a very profitable business of producing graphics workstations. SGI boxes are
some of the most common processors found in VR laboratories and high end
systems. SGI boxes range in price from under $10,000 to over $100,000. The
simulator market has produced several companies that build special purpose
computers designed expressly for real time image generation. These computers
often cost several hundreds of thousands of dollars. One key element for
interaction with a virtual world, is a means of tracking the position of a real
world object, such as a head or hand. There are numerous methods for position
tracking and control. Ideally a technology should provide 3 measures for
position(X, Y, Z) and 3 measures of orientation (roll, pitch, yaw). One of the
biggest problem for position tracking is latency, or the time required to make
the measurements and preprocess them before input to the simulation engine. The simplest
control hardware is a conventional mouse, trackball or joystick. While these
are two dimensional devices, creative programming can use them for 6D controls.
There are a number of 3 and 6 dimensional mice/trackball/joystick devices being
introduced to the market at this time. These add some extra buttons and wheels
that are used to control not just the XY translation of a cursor, but its Z
dimension and rotations in all three directions. The Global Devices 6D Controller is one such 6D joystick It looks
like a racket ball mounted on a short stick. You can pull and twist the ball in
addition to the left/right & forward/back of a normal joystick. Other 3D
and 6D mice, joystick and force balls are available from Logitech, Mouse System
Corp. among others. One common VR
device is the instrumented glove. The use of a glove to manipulate objects in a
computer is covered by a basic patent in the USA. Such a glove is outfitted with sensors on the fingers as well as
an overall position/orientation tracker. There are a number of different types
of sensors that can be used. VPL (holders of the patent) made several
DataGloves, mostly using fiber optic sensors for finger bends and magnetic
trackers for overall position. Mattel manufactured the PowerGlove for use with
the Nintendo game system, for a short time.
This device is easily adapted to interface to a personal computer. It
provides some limited hand location and finger position data using strain
gauges for finger bends and ultrasonic position sensors. The gloves are getting
rare, but some can still be found at Toys R' Us and other discount stores. Anthony Clifton recently posted this suggestion
for a" very good resource for PowerGloves etc.: small children. A friend's son had gotten a glove a couple years
ago and almost NEVER used it, so I bought it off the kid. Remember children like money more than toys
they never use." The concept of an
instrumented glove has been extended to other body parts. Full body suits with
position and bend sensors have been used for capturing motion for character
animation, control of music synthesizers, etc. in addition to VR applications. Mechanical armatures
can be used to provide fast and very accurate tracking. Such armatures may look
like a desk lamp (for basic position/orientation) or they may be highly complex
exoskeletons (for more detailed positions). The drawbacks of mechanical sensors
are the encumbrance of the device and its restrictions on motion. Exos Systems
builds one such exoskeleton for hand control. It also provides force feedback.
Shooting Star system makes a low cost armature system for head tracking. Fake
Space Labs and LEEP Systems make much more expensive and elaborate armature
systems for use with their display systems. Ultrasonic sensors
can be used to track position and orientation. A set of emitters and receivers are used with a known
relationship between the emitters and between the receivers. The emitters are
pulsed in sequence and the time lag to each receiver is measured. Triangulation
gives the position. Drawbacks to ultrasonics are low resolution, long lag times
and interference from echoes and other noises in the environment. Logitech and
Transition State are two companies that provide ultrasonic tracking systems. Magnetic trackers
use sets of coils that are pulsed to produce magnetic fields. The magnetic
sensors determine the strength and angles of the fields. Limitations of these
trackers are a high latency for the measurement and processing, range
limitations, and interference from ferrous materials within the fields.
However, magnetic trackers seem to be one of the preferred methods. The two
primary companies selling magnetic trackers are Polhemus and Ascension. Optical position
tracking systems have been developed. One method uses a ceiling grid LEDs and a
head mounted camera. The LEDs are pulsed in sequence and the cameras image is
processed to detect the flashes. Two problems with this method are limited
space (grid size) and lack of full motion (rotations). Another optical method
uses a number of video cameras to capture simultaneous images that are
correlated by high speed computers to track objects. Processing time (and cost
of fast computers) is a major limiting factor here. One company selling an
optical tracker is Origin Instruments. Inertial trackers
have been developed that are small and accurate enough for VR use. However,
these devices generally only provide rotational measurements. They are also not
accurate for slow position changes. Stereo vision is
often included in a VR system. This is accomplished by creating two different
images of the world, one for each eye. The images are computed with the
viewpoints offset by the equivalent distance between the eyes. There are a
large number of technologies for presenting these two images. The images can be
placed side-by-side and the viewer asked (or assisted) to cross their eyes. The images can be projected through
differently polarized filters, with corresponding filters placed in front of
the eyes. Anaglyph images user red/blue glasses to provide a crude (no color)
stereovision. The two images can
be displayed sequentially on a conventional monitor or projection display.
Liquid Crystal shutter glasses are then used to shut off alternate eyes in
synchronization with the display. When the brain receives the images in rapid
enough succession, it fuses the images into a single scene and perceives depth.
A fairly high display swapping rate (min. 60hz) is required to avoid perceived
flicker. A number of companies made low cost LC shutter glasses for use with
TVs (Sega, Nintendo, Toshiba, etc.). There are circuits and code for hooking
these up to a computer available on many of the On-line systems, BBSs and
Internet FTP sites mentioned later. However, locating the glasses themselves is
getting difficult as none are still being made or sold for their original use.
Stereographics sells a very nice commercial LC shutter system called
CrystalEyes. Another alternative
method for creating stereo imagery on a computer is to use one of several split
screen methods. These divide the
monitor into two parts and display left and right images at the same time. One
method places the images side by side and conventionally oriented. It may not use the full screen or may
otherwise alter the normal display aspect ratio. A special hood viewer is
placed against the monitor which helps the position the eyes correctly and may
contain a divider so each eye e sees only its own image. Most of these hoods,
such as the one for the V5 of Rend386, use fresnel lenses to enhance the
viewing. An alternative split screen
method orients the images so the top of each points out the side of the
monitor. A special hood containing
mirrors is used to correctly orient the images. A very nice low cost
(under $200) unit of this type is the Cyberscope available from Simsalabim. There was an article
supplement with CyberEdge Journal issue #17 entitled "What's Wrong with
your Head Mounted Display". It is a summary report on the findings of a
study done by the Edinburgh Virtual Environment Lab, Dept. of Psychology, Univ.
of Edinburgh on the eye strain effects of stereoscopic Head Mounted Displays.
There have been a number of anecdotal reports of stress with HMDs and other
stereoscopic displays, but few, if any, good clinical studies. This study was
done very carefully and the results are a cause for some concern. The basic test was
to put 20 young adults on a stationary bicycle and let them cycle around a
virtual rural road setting using a HMD (VPL LX EyePhone and a second HMD LEEP
optic equipped system). After 10 minutes of light exercise, the subjects were
tested... "The results
were alarming: measures of distance vision , binocular fusion and convergence
displayed clear signs of binocular stress in a significant number of the
subjects. Over half the subjects also reported symptoms of such stress, such as
blurred vision." The article goes on
to describe the primary reason for the stress - the difference between the
image focal depth and the disparity. Normally, the when your eyes look at a
close object they focus (accommodate) close and also rotate inward (converge).
When they accommodate on a far object, the eyes also diverge. However, a
stereoscopic display does not change the either the effective focal plane (set
by the optics) and the disparity depth. The eyes strain to decouple the
signals. The article
discusses some potential solutions, but notes that most of them (dynamic
focal/disparity) are difficult to implement. It mentions monoscopic HMDs only
to say that while they would seem to avoid the problems, they were not tested. The
article does not discuss non-HMD stereoscopic devices at all, but I would
extrapolate that they should show some similar problems. The full article is
available from CyberEdge Journal for a small fee. There has been a
fair bit of discussion ongoing in the sci.virtual-worlds newsgroup (check the
Sept./Oct. 93 archives) about this and some other studies. One contributor,
Dipl.-Ing. Olaf H. Kelle, University of Wuppertal, Germany, reported only 10%
of his users showing eye strain. His system is setup with a focal depth of 3m
which seems to be a better, more comfortable viewing distance. Others have
noted that long duration monitor use often leads to the user staring or not
blinking. It is common for VDT users to
be cautioned to look away from the screen occasionally to adjust their focal
depth and to blink. Another contributor, John Nagle provided the following list
of other potential problems with HMDs: electrical safety, Falling/tripping over
real world objects, simulator sickness (disorientation due to conflicting
motion signals from eyes and inner ear), Eye Strain, Induced post-HMD accidents
("some flight simulators some flight simulators, usually those for
military fighter aircraft, it's been found necessary to forbid simulator users
to fly or drive for a period of time after flying the simulator".). One hardware device
closely associated with VR is the Head
Mounted Device (HMD). These use some sort of helmet or goggles to place small
video displays in front of each eye, with special optics to focus and stretch the perceived field of
view. Most HMDs use two displays and
can provide stereoscopic imaging. Others use a single larger display to provide
higher resolution, but without the stereoscopic vision. Most lower cost HMDs ($3000-10,000 range ) use LCD displays,
while others use small CRTs, such as those found in camcorders. The more
expensive HMDs use special CRTs mounted along side the head or optical fibers
to pipe the images from non-head mounted displays. ($60,000 and up). A HMD
requires a position tracker in addition to the helmet. Alternatively, the
display can be mounted on an armature for support and tracking (a Boom
display). The following
defines a number of levels of VR hardware systems. These are not hard levels,
especially towards the more advanced systems. The 'Entry Level'
VR system takes a stock personal computer or workstation and implements a WoW
system. The system may be based on an IBM clone (MS-DOS/Windows) machine or an
Apple Macintosh, or perhaps a Commodore Amiga. The DOS type machines (IBM PC
clones) are the most prevalent. There are Mac based systems, but few very fast
rendering ones. Whatever the base
computer it includes a graphic display,
a 2D input device like a mouse, trackball or joystick, the keyboard, hard disk & memory. The next step up
from an EVR system adds some basic interaction and display enhancements. Such enhancements would include a
stereographic viewer (LCD Shutter glasses)
and a input/control device such as the Mattel PowerGlove and/or a
multidimensional (3D or 6D) mouse or joystick. The next step up
the VR technology ladder is to add a rendering accelerator and/or frame buffer
and possibly other parallel processors for input handling, etc. The simplest
enhancement in this area is a faster display card. For the PC class machines,
there are a number of new fast VGA and SVGA accelerator cards. These can make a
dramatic improvement in the rendering performance of a desktop VR system. Other
more sophisticated image processors based on the Texas Instruments TI34020 or
Intel i860 processor can make even more dramatic improvements in rendering
capabilities. The i860 in particular is in many of the high end professional
systems. The Silicon Graphics Reality Engine uses a number of i860 processors
in addition to the usual SGI workstation hardware to achieve stunning levels of
realism in real time animation. An AVR system might
also add a sound card to provide mono, stereo or true 3D audio output. Some
sound cards also provide voice recognition. This would be an excellent
additional input device for VR applications. An Immersion VR
system adds some type of immersive display system: a HMD, a Boom, or multiple
large projection type displays (Cave). An IVR system might
also add some form of tactile, haptic and touch feedback interaction
mechanisms. The area of Touch or Force Feedback (known collectively as Haptics)
is a very new research arena. A common variation
on VR is to use a Cockpit or Cab compartment to enclose the user. The virtual
world is viewed through some sort of view screen and is usually either
projected imagery or a conventional monitor. The cockpit simulation is very
well known in aircraft simulators, with a history dating back to the early Link
Flight Trainers (1929?). The cockpit is often mounted on a motion platform that
can give the illusion of a much larger range of motion. Cabs are also used in
driving simulators for ships, trucks, tanks and 'battle mechs'. The latter are
fictional walking robotic devices (i.e. the Star Wars films). The BattleTech
location based entertainment (LBE) centers use this type of system. One of the biggest
VR projects is the Defense Simulation Internet. This project is a
standardization being pushed by the USA Defense Department to enable diverse
simulators to be interconnected into a vast network. It is an outgrowth of
the Defense Advanced Research Projects
Administration (DARPA) SIMNET project of the later 1980s. SIMNET was/is a collection of tank
simulators (Cab type) that are networked together to allow unit tactical
training. Simulators in Germany can operate in the same virtual world as
simulators in the USA, partaking of the same battle exercise. The basic
Distributed Interactive Simulation (DIS) protocol has been defined by the
Orlando Institute for Simulation & Training. It is the basis for the next
generation of SIMNET, the Defense Simulation Internet (DSI). (love those
acronyms!) An accessible, if somewhat dark,
treatment of SIMNET and DSI can be found in the premier issue of WIRED
magazine (January 1993) entitled "War is Virtual Hell" by Bruce
Sterling. The basic DIS
protocol has been adopted as a standard for communication between distributed
simulations by the IEEE. Basic
information on DIS and SIMNET, including a C library to support the
communication protocol is available via FTP from the Internet site
taurus.cs.nps.navy.mil (pub/warbreaker/NPS_DIS...). Other contact points for
DIS include: Danette Haworth
Institute for Simulation & Training 12424 Research Parkway, Suite 300
Orlando, Florida 32826 (407)658-5000 ModSIM (the language) is available via ftp from
max.cecer.army.mil in the isle directory. ModSAF is being developed to create "Semi-Automated
Forces" - both vehicle based and dismounted. There is a body of research
and techniques on the various levels of scripting behaviors. Integrated Simulation (Systems) Language Environment, (ISLE)
based on ModSIM with extensions to support Imperative Behavior programming
(Prolog-like), and Persistent Objects (ARPA/TI - Open OODB, Open Base, Object
Store, etc). Developed by Army Construct Laboratory, Champagne IL Information
on ISLE is available via ftp from max.cecer.army.mil in the isle directory. Integrated Model Development Environment (IMDE) is an
iconic, visual programming tool for creating ModSIM programs, uses the Versant
ObjectBase (OODBMS). Developed by Armstrong Laboratory Defense Modeling
and Simulation Office (DMSO) has an Internet site to support Advanced
Distributed Simulation Technology (ADST). The IP address is 137.249.32.17 Administrative Contact: Kevin Mullally 407.382.4580,
Technical Contact: Brad Mohning 408.473.4962 [NOTE:
This section is BADLY out of date. Most of
the information is from the 1993 version of this paper. It does not address VRML, or other new
systems. Search Yahoo or other service
to find new systems.] There are currently
quite a number of different efforts to develop VR technology. Each of these
projects have different goals and approaches to the overall VR technology.
Large and small University labs have projects underway (UNC, Cornell,
U.Rochester, etc.). ARPA , NIST, National Science Foundation and other branches
of the US Government are investing heavily in VR and other simulation
technologies. There are industry supported laboratories too, like the Human
Interface Technologies Laboratory (HITL) in Seattle and the Japanese NTT
project. Many existing and startup companies are also building and selling
world building tools (Autodesk, IBM', Sense8, VREAM). There are two major
categories for the available VR software: toolkits and authoring systems.
Toolkits are programming libraries, generally for C or C++ that provide a set
of functions with which a skilled programmer can create VR applications.
Authoring systems are complete programs with graphical interfaces for creating
worlds without resorting to detailed programming. These usually include some
sort of scripting language in which to describe complex actions, so they are
not really non-programming, just much simpler programming. The programming libraries are generally
more flexible and have faster renders than the authoring systems, but you must
be a very skilled programmer to use them. (Note to developers: if i fail to
mention your system below, please let me know and I will try to remember to
include it when, and if, i update this document again) At the low end of
the VR spectrum are the freeware products and garage or home-brew VR hackers
(like me!). There are currently a few fast rendering programs that have been
released with source code and no charge. These programs are generally
copyrighted freeware, which means that the original creators retain the copyright
and commercial use is restricted. They are not polished commercial programs,
and are often written by students. However, these programs exist to give people
a very low cost entry into the VR world.
Rend386 is one such
freeware library and world player written for 386/486 DOS systems. It was
written by Dave Stampe and Bernie Roehl
at the University of Waterloo, Canada. It creates images at a resolution
of 320x200x256 and supports various extra devices such as the Mattel
PowerGlove, LC shutter glasses, Split Screen stereo viewers etc. Rend386 is provided both as a complete world
player and as a C source code. It does not provide a full authoring environment
for world and object building. Dave and Bernie co-authored the book
"Virtual Reality Creations" with John Eagan. It serves as the primary
user documentation for Rend386 Version 5. There is also an electronic mail list
for Rend386. Rend386 is available on the via ftp (sunee.uwaterloo.ca),
CompuServe's CyberForum, and also from a large number of BBSes. ACK3D is a freeware
C programming library developed by Lary
Meyer that provides a fast 'raycasting' renderer for PC systems. This technique
restricts the user motion somewhat, but allows textures to be drawn at very
impressive rates. The technique gained a fair bit of exposure with the
Wolfenstein 3D series of shareware games.
ACK3D can be found on the CompuServe Gamer's forum and also via ftp from
ftp.u.washington.edu in the pub/virtual-worlds/cheap-vr area. Gossamer, a
freeware VR package for the Apple Macintosh system, written by Jon Blossom.
Source code has not been released yet, but Jon has released a demo and a Think
C library. Jon is currently working on a new version that will support file
compatibility with Rend386 V5 and a more extensive user program. The current
version is available on via ftp from
ftp.apple.com in the directory pub/VR, and also on CompuServe's CyberForum. Multiverse is a
freeware UNIX based client/server system written by Robert Grant. It is a multi-user, non-immersive, X-Windows
based Virtual Reality system, primarily focused on entertainment/research. It
includes capabilities for setting up multi-person worlds and a client/server
type world simulation over a local or long haul network. Multiverse source and binaries for several
flavors of UNIX are available via anonymous ftp from medg.lcs.mit.edu in the
directory pub/multiverse The MRToolkit is a
programming library for UNIX systems that is available at no cost from the
University of Alberta, but the licensing agreement stipulates no commercial
products may be made with it. VEOS is another
programming toolkit that provides a basis for VR development on networked UNIX
machines. Source code is available from the Human Interface Technology Lab
(HITL) at University of Washington. (ftp.u.washington.edu) There are a number
of commercial VR programs that sell for under $200. Many computer games that
can be considered in this category, such as Wolfenstein 3D, but these are often
closed systems that do not allow much customizing or world building by the
user. Virtual Reality
Studio (aka VR Studio, VRS) is a very low cost VR authoring system that does
allow the user to define their own virtual worlds. This program is also known as
"3D Construction Kit" in Europe.
The program has a fairly nice graphical interface and includes a simple
scripting language. It is available for about $100 from Domark for PC and Amiga
systems. Worlds created with the program can be freely distributed with a
player program. There are a quite number of these worlds available from the
BBSes, and other sources. Compuserve's Cyberforum has several in its libraries,
like the company provided demo VRSDMO.ZIP (VRS.TXT gives a solution to the demo
game). Version 2 of VR Studio was released in early 1993. It has many new
features including a much enhanced scripting language and editor, but also an
annoying number of bugs. The developers of VRS (Dimension International) are
working hard to correct these. Another entrant
into the low cost market is the Lepton VR Data Modeling Toolkit. This package
is a collection of C programming libraries for real-time 3d data modeling on
DOS systems. Version 1.0 is scheduled to be released in Fal1993 and will cost
approximately $150. For the Macintosh
market there are the Qd3d, 3dPane, and SmartPane C++ libraries from ViviStar Consulting ($192 for full package).
These provide a full suite of 3D graphics functions for popular Macintosh C++
compilers as well as Think C 6.0. The next level of
VR System is those costing between two hundred and one thousand dollars. There
are some very excellent professional packages appearing in this price range in
the last year. Most of these systems do not require any specialized hardware
beyond the basic computer system. VREAM is a complete
VR authoring package for MS-DOS systems for about $795 from VREAM, Inc.. It provides a nice GUI environment for
creation of objects and worlds, as well as a fairly powerful scripting
language. VREAM supports a very wide variety of input and output devices,
including HMDs. Two versions of the
runtime system are available at a much lower cost to provide only the playback
ability. The lower cost runtime (under $50) will work only with standard VGA
display and mouse/joystick. The advanced runtime system supports more devices. Virtus Walkthrough,
from Virtus Corp., is available for both Mac and Windows systems. It provides a
nice 3D modeling package and the ability to interactively control the viewpoint
within the created worlds. However, it does not allow for interaction with the
objects. The latest version Walkthrough Pro supports texture maps, including
QuickTime movies. Sense 8 has
announced a $795 programming library for Windows called World Tool Kit for
Windows. This will be released late in 1993 as a DLL for Windows systems. It
will work directly with standard SVGA displays and show worlds with texture
mapping either within a window or allow full screen display. The programming
library will support DDE so a virtual world can be controlled from a
spreadsheet, database or other program. The heavy duty
professional VR software packages begin at about $1000 and can go up
dramatically. The hardware required to
run these systems varies. Most support a DOS environment with add in rendering
cards like the i860 based SPEA Fireboard. A few work on SGI and other
workstation system. There are also other packages available that run on vendor
specific hardware configurations. The really
high end packages require extremely expensive hardware "Image
Generators" such as those used in flight simulators. The Sense8 World
Tool Kit (WTK) is probably the most widely used product of this type. It runs
on a wide variety of platforms from i860 assisted PCs to high end SGI boxes. It
has won several awards for excellence. The Autodesk
Cyberspace Development kit is another product in this range. It is a C++
library for MSDOS systems using the
Metaware HighC/C++ compiler and Pharlap DOS 32bit extender. It supports VESA
displays as well as several rendering accelerator boards (SPEA Fireboard, FVS
Sapphire, Division's dView). I used this system for a few months and found it
requires a strong background in C++ and a rendering accelerator card. VESA
speeds were about 4 frames per second. Straylight Corp.
makes a package called PhotoVR that uses special rendering boards (Intel
ActionMedia cards) to provide excellent texture mapped walkthrough
environments. Dimension
International's Superscape VRT3 is a very powerful authoring system for virtual
worlds. It provides both a graphical environment for object and world
creation and a lower level C library. Division Ltd. sells
a programming environment for VR called dVS. This package runs on SGI systems,
IBM RS/6000 workstations and a proprietary Division workstation. They also sell
a complete world authoring and simulation program similar to VREAM and VRT3
called dVise. Lightscape is a
radiosity rendering package for creating realistically shaded walkthroughs from
Lightscape Graphics Software. This product runs on high end workstations
and is aimed primarily at architects
and lighting designers. There have been a
number of other packages introduced recently for professional VR development. I
do not have full information on all of them and suggest the interested reader
follow up by reading either the AI Expert Special Report on Virtual Reality or
perhaps by purchasing Sophistech's VR Sourcebook. Just what is required
of a VR program? The basic parts of the system can be broken down into an Input
Processor, a Simulation Processor, a Rendering Process, and a World Database.
All these parts must consider the time required for processing. Every delay in
response time degrades the feeling of 'presence' and reality of the simulation.
The Input Processes of a VR program control the devices used
to input information to the computer. There are a wide variety of possible
input devices: keyboard, mouse, trackball, joystick, 3D & 6D position
trackers (glove, wand, head tracker, body suit, etc.). A networked VR system
would add inputs received from net. A voice recognition system is also a good
augmentation for VR, especially if the user's hands are being used for other tasks.
Generally, the input processing of a VR
system is kept simple. The object is to get the coordinate data to the rest of
the system with minimal lag time. Some
position sensor systems add some filtering and data smoothing processing. Some glove systems add gesture recognition.
This processing step examines the glove inputs and determines when a specific
gesture has been made. Thus it can provide a higher level of input to the
simulation. The core of a VR
program is the simulation system. This is the process that knows about the
objects and the various inputs. It handles the interactions, the scripted
object actions, simulations of physical laws (real or imaginary) and determines
the world status. This simulation is basically a discrete process that is
iterated once for each time step or frame. A networked VR application may have
multiple simulations running on different machines, each with a different time
step. Coordination of these can be a complex task. It is the
simulation engine that takes the user inputs along with any tasks programmed into the world such as
collision detection, scripts, etc. and determines the actions that will take
place in the virtual world. The Rendering
Processes of a VR program are those that create the sensations that are output
to the user. A network VR program would also output data to other network
processes. There would be separate rendering processes for visual, auditory,
haptic (touch/force), and other sensory systems. Each renderer would take a
description of the world state from the simulation process or derive it
directly from the World Database for each time step. The visual renderer
is the most common process and it has a long history from the world of computer
graphics and animation. The reader is encouraged to become familiar with
various aspects of this technology. The major
consideration of a graphic renderer for VR applications is the frame generation
rate. It is necessary to create a new frame every 1/20 of a second or faster.
20 frames per second (fps) is roughly the minimum rate at which the human brain
will merge a stream of still images and perceive a smooth animation. 24fps is
the standard rate for film, 25fps is PAL TV, 30fps is NTSC TV. 60fps is
Showscan film rate. This requirement eliminates a number of rendering
techniques such as raytracing and radiosity. These techniques can generate very
realistic images but often take hours to generate single frames. Visual renderers
for VR use other methods such as a 'painter's algorithm', a Z-Buffer, or other
Scanline oriented algorithm. There are many areas of visual rendering that have
been augmented with specialized hardware. The Painter's algorithm is favored by
many low end VR systems since it is relatively fast, easy to implement and
light on memory resources. However, it has many visibility problems. For a
discussion of this and other rendering algorithms, see one of the computer
graphics reference books listed in a later section. The visual
rendering process is often referred to as a rendering pipeline. This refers to
the series of sub-processes that are invoked to create each frame. A sample
rendering pipeline starts with a description of the world, the objects,
lighting and camera (eye) location in world space. A first step would be
eliminate all objects that are not visible by the camera. This can be quickly
done by clipping the object bounding box or sphere against the viewing pyramid
of the camera. Then the remaining objects have their geometry's transformed
into the eye coordinate system (eye point at origin). Then the hidden surface
algorithm and actual pixel rendering is done. The pixel rendering
is also known as the 'lighting' or 'shading' algorithm. There are a number of
different methods that are possible depending on the realism and calculation
speed available. The simplest method is called flat shading and simply fills
the entire area with the same color. The next step up provides some variation
in color across a single surface. Beyond that is the possibility of smooth
shading across surface boundaries, adding highlights, reflections, etc. An effective short
cut for visual rendering is the use of "texture" or "image"
maps. These are pictures that are mapped onto objects in the virtual world.
Instead of calculating lighting and shading for the object, the renderer
determines which part of the texture map is visible at each visible point of
the object. The resulting image appears to have significantly more detail than
is otherwise possible. Some VR systems have special 'billboard' objects that
always face towards the user. By mapping a series of different images onto the
billboard, the user can get the appearance of moving around the object. I need to correct
my earlier statement that radiosity cannot be used for VR systems due to the
time requirements. There have recently been at least two radiosity renderers
announced for walkthrough type systems - Lightscape from Lightscape Graphics
Software of Canada and Real Light from Atma Systems of Italy. These packages
compute the radiosity lighting in a long time consuming process before hand.
The user can interactively control the camera view but cannot interact with the
world. An executable demo of the Atma product is available for SGI systems from
ftp.iunet.it (192.106.1.6) in the directory ftp/vendor/Atma. A VR system is
greatly enhanced by the inclusion of an audio component. This may produce mono,
stereo or 3D audio. The latter is a fairly difficult proposition. It is not
enough to do stereo-pan effects as the mind tends to locate these sounds inside
the head. Research into 3D audio has shown that there are many aspects of our
head and ear shape that effect the recognition of 3D sounds. It is possible to
apply a rather complex mathematical function (called a Head Related Transfer
Function or HRTF) to a sound to produce this effect. The HRTF is a very
personal function that depends on the individual's ear shape, etc. However,
there has been significant success in creating generalized HRTFs that work for
most people and most audio placement. There remains a number of problems, such
as the 'cone of confusion' wherein sounds behind the head are perceived to be
in front of the head. Sound has also been
suggested as a means to convey other information, such as surface roughness.
Dragging your virtual hand over sand would sound different than dragging it
through gravel. Haptics is the
generation of touch and force feedback information. This area is a very new
science and there is much to be learned. There have been very few studies done
on the rendering of true touch sense (such as liquid, fur, etc.). Almost all systems to date have focused on
force feedback and kinesthetic senses. These systems can provide good clues to
the body regarding the touch sense, but are considered distinct from it. Many
of the haptic systems thus far have been exo-skeletons that can be used for
position sensing as well as providing resistance to movement or active force
application. The sense of
balance and motion can be served to a fair degree in a VR system by a motion
platform. These are used in flight simulators and some theaters to provide some
motion cues that the mind integrates with other cues to perceive motion. It is
not necessary to recreate the entire motion perfectly to fool the mind into a
willing suspension of disbelief. The sense of
temperature has seen some technology developments. There exist very small
electrical heat pumps that can produce the sensation of heat and cold in a
localized area. These system are fairly expensive. Other senses such
as taste, smell, pheromone, etc. are beyond our ability to render rapidly and
effectively. Sometimes, we just don't know enough about the functioning of
these other senses. The virtual world
itself needs to be defined in a 'world space'. By its nature as a computer
simulation, this world is necessarily limited. The computer must put a numeric
value on the locations of each point of each object within the world. Usually
these 'coordinates' are expressed in Cartesian dimensions of X, Y, and Z
(length, height, depth). It is possible to use alternative coordinate systems
such as spherical but Cartesian coordinates are the norm for almost all
applications. Conversions between coordinate systems are fairly simple (if time
consuming). A major limitation
on the world space is the type of numbers used for the coordinates. Some worlds
use floating point coordinates. This allows a very large range of numbers to be
specified, with some precision lost on large numbers. Other systems used fixed
point coordinates, which provides uniform precision on a more limited range of
values. The choice of fixed versus floating point is often based on speed as
well as the desire for a uniform coordinate field. One method of
dealing with the limitations on the world coordinate space is to divide a
virtual world up into multiple worlds and provide a means of transiting between
the worlds. This allows fewer objects
to be computed both for scripts and for rendering. There should be multiple
stages (aka rooms, areas, zones, worlds, multiverses, etc.) and a way to move
between them (Portals). [NOTE: this needs to be updated
wrt VRML, etc.] The storage of
information on objects and the world is a major part of the design of a VR
system. The primary things that are stored in the World Database (or World
Description Files) are the objects that inhabit the world, scripts that
describe actions of those objects or the user (things that happen to the user),
lighting, program controls, and hardware device support. There are a number
of different ways the world information may be stored: a single file, a
collection of files, or a database. The multiple file method is one of the more
common approaches for VR development packages. Each object has one or more
files (geometry, scripts, etc.) and there is some overall 'world' file that
causes the other files to be loaded. Some systems also include a configuration
file that defines the hardware interface connections. Sometimes the
entire database is loaded during program startup, other systems only read the
currently needed files. A real database system helps tremendously with the
latter approach. An Object Oriented Database would be a great fit for a VR
system, but I am not aware of any projects currently using one. The data files are
most often stored as ASCII (human readable) text files. However, in many
systems these are replaced by binary computer files. Some systems have all the
world information compiled directly into the application. Objects in the
virtual world can have geometry, hierarchy, scripts, and other attributes. The
capabilities of objects has a tremendous impact on the structure and design of
the system. In order to retain flexibility,
a list of named attribute/values pairs is often used. Thus attributes
can be added to the system without requiring changes to the object data
structures. These attribute
lists would be addressable by name (i.e. cube.mass => mass of the cube
object). They may be a scalar, vector, or expression value. They may be
addressable from within the scripts of their object. They might be accessible
from scripts in other objects. An object is
positionable and orientable. That is, it has a location and orientation in
space. Most objects can have these
attributes modified by applying translation and rotation operations. These
operations are often implemented using methods from vector and matrix algebra. An object may be
part of an object part HIERARCHY with a parent, sibling, and child objects.
Such an object would inherit the transformations applied to it's parent object
and pass these on to it's siblings and children. Hierarchies are used to create
jointed figures such as robots and animals. They can also be used to model other things like the sun,
planets and moons in a solar system. Additionally, an
object should include a BOUNDING
VOLUME. The simplest bounding volume is the Bounding Sphere, specified by a
center and radius. Another simple alternative is the Bounding Cube. This data
can be used for rapid object culling during rendering and trigger analysis.
Objects whose bounding volume is completely outside the viewing area need not
be transformed or considered further during rendering. Collision detection with
bounding spheres is very rapid. It could be used alone, or as a method for
culling objects before more rigorous collision detection algorithms are
applied. The modeling of
object shape and geometry is a large and diverse field. Some approaches seek to
very carefully model the exact geometry of real world objects. Other methods
seek to create simplified representations.
Most VR systems sacrifice detail and exactness for simplicity for the
sake of rendering speed. The simplest
objects are single dimensional points. Next come the two dimensional vectors.
Many CAD systems create and exchange data as 2D views. This information is not
very useful for VR systems, except for display on a 2D surface within the
virtual world. There are some programs that can reconstruct a 3D model of an
object, given a number of 2D views. The sections below
discuss a number of common geometric modeling methods. The choice of method
used is closely tied to the rendering process used. Some renderers can handle
multiple types of models, but most use only one, especially for VR use. The
modeling complexity is generally inversely proportional to the rendering speed.
As the model gets more complex and detailed, the frame rate drops. The simplest 3D
objects are known as PolyPoints and PolyLines. A PolyPoint is simply a
collection of points in space. A Polyline is a set of vectors that form a
continuous line. The most common
form of objects used in VR systems are based on flat polygons. A polygon is a planar, closed multi-sided figure. They
maybe convex or concave, but some systems require convex polygons. The use of
polygons often gives objects a faceted look. This can be offset by more
advanced rendering techniques such as the use of smooth shading and texture
mapping. Some systems use
simple triangles or quadrilaterals instead of more general polygons. This can
simplify the rendering process, as all surfaces have a known shape. However, it
can also increase the number of surfaces that need to be rendered. Polygon Mesh Format
(aka Vertex Join Set) is a useful form of polygonal object. For each object in
a Mesh, there is a common pool of Points that are referenced by the polygons
for that object. Transforming these shared points reduces the calculations
needed to render the object. A point at the edge of a cube is only processed
once, rather once for each of the three edge/polygons that reference it. The
PLG format used by REND386 is an example of a Polygonal Mesh, as is the BYU
format used by the 'ancient' MOVIE.BYU program.) . The geometry
format can support precomputed polygon and vertex normals. Both Polygons and
vertices should be allowed a color attribute. Different renderers may use
or ignore these and possibly more
advanced surface characteristics. Precomputed polygon normals are very helpful
for backface polygon removal. Vertices
may also have texture coordinates assigned to support texture or other image
mapping techniques. Some systems
provide only Primitive Objects, such as cubes, cones, and spheres. Sometimes,
these objects can be slightly deformed by the modeling package to provide more
interesting objects. Solid Modeling (aka
Computer Solid Geometry, CSG) is one form of geometric modeling that uses
primitive objects. It extends the concept by allowing various addition, subtraction,
Boolean and other operations between these primitives. This can be very useful
in modeling objects when you are concerned with doing physical calculations,
such as center of mass, etc. However, this method does incur some significant
calculations and is not very useful for VR applications. It is possible to
convert a CSG model into polygons. Various complexity polygonal models (#
polygons) could be made from a single high resolution ''metaobject" of a
CSG type. Another advanced
form of geometric modeling is the use of curves and curved surfaces (aka
patches). These can be very effective in representing complex shapes, like the
curved surface of an automobile, ship or beer bottle. However, there is
significant calculation involved in determining the surface location at each
pixel, thus curve based modeling is not used directly in VR systems. It is
possible, however, to design an object using curves and then compute a
polygonal representation of those curved patches. Various complexity polygonal models could be made from a single
high resolution 'metaobject'. It is sometimes
desirable to have an object that can change shape. The shape might simply be
deformed, such a bouncing ball or the squash/stretch used in classical
animation ('toons'), or it might actually undergo metamorphosis into a
completely different geometry. The latter effect is commonly known as
'morphing' and has been extensively used in films, commercials and television
shows. Morphing can be done in the image domain (2D morph) or in the geometry
domain (3D morph). The latter is applicable to VR systems. The simplest method
of doing a 3D morph is to precompute the various geometry's and step through
them as needed. A system with significant processing power can handle real time
object morphing. A common method for
creating objects is known as Sweeping and Surfaces of Revolution. These methods
use an outline or template curve and a backbone. The template is swept along
the backbone creating the object surface (or rotated about a single axis to
create a surface of revolution). This method may be used to create either curve
surfaces or polygonal objects. For VR
applications, the sweeping would most likely be performed during the object
modeling (creation) phase, and the resulting polygonal object stored for real
time use. As mentioned in the
section on rendering, texture maps (images) can be used to provide the
appearance of more geometric complexity without the geometric calculations.
Using flat polygonal objects that maintain an orientation towards the
eye/camera (billboards) and multiple texture maps can extend this trick even
further. Texture maps, even without
billboard objects, are an excellent way to increase apparent scene complexity.
Variations on the image mapping concept are also used to simulate reflections,
etc. Lighting is a very
important part of a virtual world (if it is visually rendered). Lights can be
ambient (everywhere), or located. Located lights have position and may have
orientation, color, intensity and a cone of illumination. The more complex the
light source, the more computation is required to simulate its effect on objects. Cameras or
viewpoints may be described in the World Database. Generally, each user has
only one viewpoint at a time (ok, two closely spaced viewpoints for
stereoscopic systems). However, it may be useful to define alternative cameras
that can be used as needed. An example might be an overhead camera that shows a
schematic map of the virtual world and the user's location within it (You Are
Here.) A virtual world
consisting only of static objects is only of mild interest. Many researchers
and enthusiasts of VR have remarked that interaction is the key to a successful
and interesting virtual world. This requires some means of defining the actions
that objects take on their own and when the user (or other objects) interact
with them. This i refer to generically as the World Scripting. I divide the
scripts into three basic types: Motion Scripts, Trigger Scripts and Connection
Scripts Scripts may be
textual or they might be actually compiled into the program structure. The use
of visual programming languages for world design was pioneered by VPL Research
with their Body Electric system. This Macintosh based language used 2d blocks
on the screen to represent inputs, objects and functions. The programmer would
connect the boxes to indicate data flow. There is no common
scripting language used in today's VR products. The commercial authoring packages, such as VR Studio, VREAM and
Superscape all contain some form of scripting language. Autodesk's CDK has the "Cyberspace Description
Format" (CDF) and the Distributed Shared Cyberspace Virtual Representation
(DSCVR) database. These are only partially implemented in the current release.
They are derived from the Linda distributed programming language/database
system. ("Coordiantation Languages
and their Significance", David Gelernter and Nicholas Carriero,
Communications of the ACM, Feb 1992 V35N2).
On the homebrew/freeware side, some people are experimenting with
several Object Oriented interpretive languages such as BOB ("Your own tiny
Object-Oriented Language", David Betz, DrDobbs Journal Sept 1991). Object Orientation, although perhaps not in
the conventional class-inheritance mechanism, is very nicely suited to world
scripting. Interpretive langauges are faster for development, and often more
accessible to 'non-programmers'. Motion scripts
modify the position, orientation or other attributes of an object, light or
camera based on the current system tick.
A 'tick' is one advancement of the simulation clock. Generally, this is
equivalent to a single frame of visual animation. (VR generally uses Discrete
Simulation methods) For simplicity and
speed, only one motion script should be active for an object at any one
instant. Motion scripting is a potentially
powerful feature, depending on how complex we allow these scripts to
become. Care must be exercised since
the interpretation of these scripts will require time, which impacts the frame
and delay rates. Additionally, a
script might be used to attach or detach an object from a hierarchy. For
example, a script might attach the user to a CAR object when he wishes to drive
around the virtual world. Alternatively, the user might 'pick up' or attach an
object to himself. A complex
simulation could be used that models the interactions of the real physical
world. This is sometimes referred to as Procedural Modeling. It can be a very
complex and time consuming application. The mathematics required to solve the
physical interaction equations can also be fairly complex. However, this method
can provide a very realistic interaction mechanism. (for more on Physical
Simulation, see the book by Ronen Barzel listed in the Computer Graphics Books
section) A simpler method of
animation is to use simple formulas for the motion of objects. A very simple example would be "Rotate
about Z axis once every 4 seconds". This might also be represented as
"Rotate about Z 10 radians each frame". A slightly more
advanced method of animation is to provide a 'path' for the object with
controls on its speed at various points. These controls are sometimes referred
to as "slow in-out". They provide a much more realistic motion than
simple linear motion. If the motion is
fixed, some systems can precompute the motion and provide a 'channel' of data
that is evaluated at each time instance. This may be a simple lookup table with
exact values for each frame, or it may require some sort of simple
interpolation. Trigger Scripts are
invoked when some trigger event occurs, such as collision, proximity or
selection. The VR system needs to
evaluate the trigger parameters at each TICK. For proximity detectors, this may
be a simple distance check from the object to the 3D eye or effector object
(aka virtual human) Collision detection is a more involved process. It is
desirable but may not be practical without off loading the rendering and some
UI tasks from the main processor. Connection scripts
control the connection of input and output devices to various objects. For
example a connection script may be used to connect a glove device to a virtual
hand object. The glove movements and position information is used to control
the position and actions of the hand object in the virtual world. Some systems
build this function directly into the program. Other systems are designed such
that the VR program is almost entirely a connection script. The user must be
given some indication of interaction feedback when the virtual cursor selects
or touches an object. Crude systems have only the visual feedback of seeing the
cursor (virtual hand) penetrate an object. The user can then grasp or otherwise
select the object. The selected object is then highlighted in some manner.
Alternatively, an audio signal could be generated to indicate a collision. Some
systems use simple touch feedback, such as a vibration in the joystick, to
indicate collision, etc. A VR system often
needs to have some sort of control panels available to the user. The world
database may contain information on these panels and how they are integrated
into the application. Alternatively, they may be a part of the program code. There are several
ways to create these panels. There could be 2D menus that surround a WoW
display, or are overlaid onto the image. An alternative is to place control
devices inside the virtual world. The simulation system must then note user
interaction with these devices as providing control over the world. One primary area of
user control is control of the viewpoint (moving around within the virtual
world). Some systems use the joystick or similar device to move. Others use
gestures from a glove, such as pointing, to indicate a motion command. The user interface
to the VW might be restricted to direct interaction in the 3D world. However, this is extremely
limiting and requires lots of 3D calculations. Thus it is desirable to have
some form of 2D Graphical user interface to assist in controlling the virtual
world. These 'control panels' of the would appear to occlude portions of the 3D
world, or perhaps the 3D world would appear as a window or viewport set in a 2D
screen interface. The 2D interactions could also be represented as a flat panel
floating in 3D space, with a 3D effector controlling them. There are four
primary types of 2D controls and displays. (controls cause changes in the
virtual world, displays show some measurement on the VW.) Buttons, Sliders,
Gauges and Text. Buttons may be menu items with either icons or text
identifiers. Sliders are used for more analog control over various attributes.
A variation of a slider is the dial, but these are harder to implement as 2D
controls. Gauges are graphical depiction's of the value of some attribute(s) of
the world. Text may be used for both control and display. The user might enter
text commands to some command parser. The system may use text displays to show
the various attributes of the virtual world. An additional type
of 2D display might be a map or locator display. This would provide a point of
reference for navigating the virtual world. The VR system needs
a definition for how the 2D cursor effects these areas. It may be desirable to
have a notion of a 'current control' that is the focus of the activity (button
pressed, etc.) for the 2D effector. Perhaps the arrow keys on the keyboard
could be used to change the current control, instead of using the mouse (which
might be part of the 3D effector at present). Some systems place
the controls inside the virtual world. These are often implemented as a
floating control panel object. This panel contains the usual 2D buttons,
gauges, menu items, etc. perhaps with a 3D representation and interaction
style. There have also
been some published articles on 3D control Widgets. These are interaction
methods for directly controlling the 3D objects. One method implemented at
Brown University attaches control handles to the objects. These handles can be
grasped, moved, twisted, etc. to cause various effects on an object. For
example, twisting one handle might rotate the object, while a 'rack' widget
would provide a number of handles that can be used to deform the object by
twisting its geometry. The world database
may contain information on the hardware controls and how they are integrated
into the application. Alternatively, they may be a part of the program code.
Some VR systems put this information into a configuration file. I consider this
extra file simply another part of the world database. The hardware
mapping section would define the input/output ports, data speeds, and other
parameters for each device. It would also provide for the logical connection of
that device to some part of the virtual world. For example a position tracker
might be associated with the viewer's head or hand. If the system
supports the division of the virtual world into different areas, the world
database would need multiple scene
descriptions. Each area description would give the names of objects in scene,
stage description (i.e. size, backgrounds, lighting, etc.). There would also be
some method of moving between the worlds, such as entering a doorway, etc.,
that would most likely be expressed in object scripts. A virtual world can
be created, modified and experienced. Some VR systems may not distinguish
between the creation and experiencing aspects. However, there is currently a
much larger body of experience to draw upon for designing the world from the
outside. This method may use techniques borrowed from architectural and other
forms of Computer Aided Design (CAD) systems. Also the current technologies for
immersive VR systems are fairly limiting in resolution, latency, etc. They are
not nearly as well developed as those for more conventional computer graphics
and interfaces. For many VR
systems, it makes a great deal of sense to have a Authoring mode and a Playback
mode. The authoring mode may be a standard text editor and compiler system, or
it may include 3D graphic and other tools. Such a split mode system makes it
easier to create a stand alone application that can be delivered as a product. An immersive
authoring ability may be desirable for some applications and some users. For
example, an architect might have the ability to move walls, etc. when immersed,
while the clients with him, who are not as familiar with the system, are
limited to player status. That way they can't accidentally rearrange the house
by leaning on a wall. This section will cover or discuss some of the asthetics and
perhaps other aspects of world design. [NOTE: see Mike Heim “Virtual Realism. http://www.mheim.com ] The following list of early Cyberpunk and other VR related
books was collected from posts to Sci.virtual-worlds and CompuServe's CyberForum.
I cannot vouch that they are all truely VR related. The list was collected
circa 1993. There was an explosion of VR related stories after this. [NOTE: add
ISBN or links to Amazon or equiv. A title search of Amzon.com on “Virtual
Reality” turned up approx 200 books!] Piers Anthony's "Killobyte." Stephen Barnes- Street-Lethal, Gorgan's Child Greg Bear - Blood Music David Brin - Earth John Brunner - Shockwave Rider, The Sheep Look Up, Jagged
Orbit William S. Burroughs
Naked Lunch, Nova Express, The Ticket that Expoloded Pat Cadigan - Mindplayers, Synners, etc. Orson Scott Card's Ender's Game Samuel Delany Nova Philip K. Dick - Do Androids Dream of Electric Sheep, Flow My Tears, the Policeman Said The
Game-Players of Titan Ubik VALIS The Divine Invasion The Transmigration of
Timothy Archer William Gibson - Neuromancer, Count Zero, Mona Lisa
Overdrive, Burning Chrome, Virtual
Light Stanislav Lem - The Cyberiad : Fables for the Cybernetic
Age, (and others) Mark Leyner - My Cousin, My Gastroenterologist Larua Mixon - Glass Houses Tom Maddox: Halo Larry Niven's Known Space series of books - really enjoyable
stuff. Marge Piercy, "He, She and It" Thomas Pynchon: Gravity's Rainbow, V, The Crying of Lot 49,
Vineland Kim Stanley Robinson The Gold Coast Frank Robinson - The Dark Beyond the Stars Rudy Rucker - Wetware, Software Lucius Shepard
- Life During Wartime John Shirley -The Eclipse triology, Freezone Bruce Sterling - Islands in the Net, The Artificial
Kid, Globalhead, Mirrorshades: The
Cyberpunk Anthology Neal Stephenson - Snow Crash Amy Thompson - Virtual Girl James Tiptree, Jr. - The Girl who was Plugged In Vernor Vinge - True
Names and Other Dangers, A Fire Upon the Deep Walter John Williams - Hard-Wired, Voice of the Whirlwind David Wingrove
- Chung-Kuo (series) Zelaney & Saberhaven - Coils "Simulations - 15 tales of virtual reality",
edited by Karie Jacobson, Citadel Twilight Press. ISBN 0-8065-1406-X includes: The Crying
of Lot 49 1. "The Veldt" (1950) by Ray Bradbury 2. "Overdrawn
at the Memory Bank" (1976) by John Varley 3. "Walking the
Moons" (1990) by Jonathan Letham 4. "Virtual
Reality" (1993) by Michael Kandel 5.
"Dogfight" (1985) by Michael Swanwick and Willian Gibson 6. "The Shining
Dream Road Out" (1993) by M. Shayne Bell 7. "The Total
Perspective Vortex" (1980) by Douglas Adams 8. "Plug-in
Yosemite" (1985) by Marc Laidlaw 9. "This Life
and Later Ones" (1987) by George Zebrowski 10. "Steelcollar Worker" (1992) by Vonda McIntyre 11. "I Hope I shall Arrive Soon" (1980) by Philip
K. Dick 12. "From Here to Eternitape" (1992) by Daniel
Pearlman 13. "Pretty Boy Crossover" (1986) by Pat Cadigan 14. "A Guide to Virtual Death" (1992) by J.G.
Ballard 15. "The Happy Man" (1963) by Gerald Page 16. Bibliography of VR in fiction
2.
A Taxonomy of Virtual Reality
3.
Types of VR Systems
3.1.
Window on World Systems (WoW)
3.2.
Video Mapping
3.3.
Immersive Systems
3.4.
Telepresence
3.5.
Mixed Reality
4.
VR Hardware
4.1.
Image Generators
4.2.
Manipulation and Control Devices
4.3.
Stereo Vision
1.1.1. Health
Hazards from Stereoscopic Displays
4.4.
Head Mounted Display (HMD)
4.5.
Force and Touch (Haptic) Rendering
4.6. Motion
Rendering
5.
Levels of VR Hardware Systems
5.1.
Entry VR (EVR)
5.2.
Basic VR (BVR)
5.3.
Advanced VR (AVR)
5.4.
Immersion VR (IVR)
5.5.
SIMNET, Defense Simulation Internet
6.
VR System Technical Approaches
6.1.
High End Vis Sim
6.2.
VR Programmer Toolkits
6.3.
Behavior Modeling
7.
Available VR Software Systems
7.1.
Freeware VR Programs
7.2.
VR Programs for under $200
7.3.
VR Packages under $1000
8.
Aspects of A VR Program
8.1.
Simulation Process
8.2.
Rendering Processes
8.2.1. Visual
Renderer
8.2.2. Auditory
Rendering
8.2.3. Haptic
Rendering
9.
Other Senses
10.
World Space
10.1.World
Coordinates
11.
World Database
11.1.Objects
11.1.1. Position/Orientable
11.1.2. Hierarchy
11.2.Object
Geometry
11.2.1. 3D
PolyLines & PolyPoints
11.2.2. Polygons
11.2.3. Primitives
11.2.4. Solid
Modeling & Boolean Operations
11.2.5. Curves
& Patches
11.2.6. Dynamic
Geometry (aka morphing)
11.2.7. Swept
Objects & Surface of Revolution
11.2.8. Texture
Maps & Billboard Objects
11.3.Lights
11.4.Scripts
and Object Behavior
11.4.1. Motion
Scripts
11.4.2. Physical
or Procedural Modeling and Simulation
11.4.3. Simple
Animation
11.4.4. Trigger
Scripts
11.4.5. Connection
Scripts
11.5.Graphical
User Interface/Control Panels
11.5.1. Two Dimensional Controls
11.5.2. Three
Dimensional Controls
11.6.Hardware
Control & Connections
11.7.Room/Stage/Area
Descriptions
12.
World Authoring versus Playback
13. World
Design
14. Fiction
Books Related to VR