Input Device’s, User Interfaces, VR Environment’s
TEJASWI CHIRUMALLA
U19091105
Locator Device
A locator class device returns a position on the display surface selected by the user. It lets the user position a prompt such as a ``+'' on the workstation surface to define a point. The user can then trigger a request for input. It provides the following logical input values to the application.
Example:
Mouse, Keyboard (Cursor-Position Keys), Tablet(Digitizer), Trackballs, Light pens.
Applications:
Interactive drawing and editing, Graph digitizing.
Stroke Device:
A stroke-class device returns a sequence of point positions that the user draws on the display surface. Your application must specify a distance vector and a length of time in the initialization data record.
A stroke-class device provides the following logical input values to the application:
The sequence of WC points.
The normalization transformation number.
The view index (three-dimensional devices only).
Choice Device:A choice-class device lists a series of choices on the display surface. The user selects a choice, and DEC GKS returns an integer value representing the choice.
It provides the following logical input values to the application:The choice statusThe integer representing the choice made
Example:
Mouse, Keyboard (Function Keys), Touch Panel.
Applications:
Interactive menu selection, Program control.
Picking Device:A pick-class device returns the identity of a segment selected by the user and a pick identifier . A pick identifier is a name that is attached to an output primitive within a segment. When pick input is requested, DEC GKS provides a cursor on the screen. When the cursor is in contact with a segment to be picked and the device is triggered, the cursor provides the following logical input values:
The pick status.
The segment name.
The pick identifier
Example:
Mouse, Cursor Keys, Tablet.
Applications:
Interactive editing and positioning
Physical Devices
mouse trackball light pen
data tablet joy stick space ball
RM
Digitizer
Joystick and Trackball
Pick Correlation:
Pick correlation is the process of determining whether a user-controlled cursor (such as a mouse cursor or touch-point on a touch-screen interface) intersects a given graphical object (such as a shape, line, or curve) drawn on the screen. This testing may be performed on the movement or activation of a mouse or other pointing device
Example:
Three Approaches:
Hit list: Most general approach but most difficult to implement.
Use back or some other buffer to store object ids as the objects are rendered
Rectangular maps: Easy to implement for many applications
See paint program in text
Rendering Modes OpenGL can render in one of three modes selected by
glRenderMode(mode) GL_RENDER: normal rendering to the frame buffer (default)
GL_FEEDBACK: provides list of primitives rendered but no output to the frame buffer
GL_SELECTION: Each primitive in the view volume generates a hit record that is placed in a name stack which can be examined later
Selection Mode Functions and Picking
glSelectBuffer(): specifies name buffer.
glInitNames(): initializes name buffer
glPushName(id): push id on name buffer.
glPopName(): pop top of name buffer.
glLoadName(id): replace top name on buffer.
id is set by application program to identify objects
As we just described it, selection mode won’t work for picking because every primitive in the view volume will generate a hit
Change the viewing parameters so that only those primitives near the cursor are in the altered view volume
Use gluPickMatrix (see text for details)
Picking Template Algorithm
Picking Ideas and getting it to buffer
Scissor Box & Alpha test
Using Regions of the Screen Many applications use a simple rectangular
arrangement of the screen Example: paint/CAD program
Easier to look at mouse position and determine which area of screen it is in than using selection mode picking
Using another buffer and colors for picking
For a small number of objects, we can assign a unique color (often in color index mode) to each object
We then render the scene to a color buffer other than the front buffer so the results of the rendering are not visible
We then get the mouse position and use glReadPixels() to read the color in the buffer we just wrote at the position of the mouse
The returned color gives the id of the object
Writing Modes
XOR write
Usual (default) mode: source replaces destination (d’ = s) Cannot write temporary lines this way because we cannot recover
what was “under” the line in a fast simple way
Exclusive OR mode (XOR) (d’ = d s) x y x =y
Hence, if we use XOR mode to write a line, we can draw it a second time and line is erased!
XOR in OpenGL
There are 16 possible logical operations between two bits
All are supported by OpenGL Must first enable logical operations
glEnable(GL_COLOR_LOGIC_OP)
Choose logical operation
glLogicOp(GL_XOR)
glLogicOp(GL_COPY) (default)
Input Modes:
Input devices contain a trigger which can be used to send a signal to the operating system Button on mouse
Pressing or releasing a key
When triggered, input devices return information (their measure) to the system Mouse returns position information
Keyboard returns ASCII code.
20
Request Mode
Input provided to program only when user triggers the device
Typical of keyboard input Can erase (backspace), edit, correct until enter (return) key (the
trigger) is depressed
21
Event Mode
Most systems have more than one input device, each of which can be triggered at an arbitrary time by a user
Each trigger generates an event whose measure is put in an event queue which can be examined by the user program
22
Rubberbanding
Switch to XOR write mode Draw object
For line can use first mouse click to fix one endpoint and then use motion callback to continuously update the second endpoint
Each time mouse is moved, redraw line which erases it and then draw line from fixed first position to new second position
At end, switch back to normal drawing mode and draw line Works for other objects: rectangles, circles
Rubberband Lines
initial displaydraw line with mouse in XOR mode
mouse moved to new position
first point
second point
original line redrawn with XOR
new line drawn with XOR
25Introduction
What is Virtual Reality(VR)?
Virtual Reality refers to a high-end user interface that involves real-time simulation
and interactions through multiple sensorial channels.
Recent Innovations
Recent innovation from Samsung , Samsung Gera VR powered by Occulus
Display Methodologies:Simulation based VR:
Driving simulators, for example, give the driver on board the impression that he/she is actually driving an actual vehicle by predicting vehicular motion caused by driver input and feeding back corresponding visual, motion, audio and proprioceptive cues to the driver.
The simulator normally consists of several systems as follows:
A real-time vehicle simulation system performing real-time simulation of vehicle dynamics; motion visual and audio systems reproducing vehicle motion driving environment scenes and noise sensed by a driver during driving a control force roading system acting as an interface between the driver and the simulator;
An operator console for monitoring system operation; and system integration managing information and data transfer among subsystems and synchronization. The driving simulators have been used effectively for vehicle system development safety improvement human factor study
Avtar Image Based VR:With avatar image-based virtual reality, people can join the virtual environment in the form of real video as well as an avatar.
The proposed image VR system can handle two types of users.
One can participate in the 3D distributed virtual environment as form of either a conventional avatar or a real video. Background of the video is effectively eliminated to enhance the sense of reality. A user can select his/her own type of participation based on the system capability.
Users with capture board and camera may select a video avatar while others select a conventional computer graphics-based avatar. Avatar image-based VR now provides pretty good interaction environment between human and computer far beyond the conventional desktop computer systems. High-speed networks become available with the advance of network technologies.
Project Based VR In projector-based virtual reality, modeling of the real environment plays a
vital role in various virtual reality applications, such as robot navigation, construction modeling and airplane simulation.
Image based virtual reality system is gaining popularity in computer graphics as well as computer vision communities. The reason is that is it provides more realism by using photo realistic images and the modeling procedure is rather simple. In generating realistic models, it is essential to accurately register acquired 3D data. Usually, camera is used for modeling small objects at a short distance.
Desktop Based VR: Desktop-based virtual reality involves
displaying a 3-dimensional virtual world on a regular desktop display without use of any specialized movement-tracking equipment.
Many modern computer games can be used as an example, using various triggers, responsive characters, and other such interactive devices to make the user feel as though they are in a virtual world. A common criticism of this form of immersion is that there is no sense of peripheral vision, limiting the user's ability to know what is happening around them.
True Immersive Virtual Reality: Hypothetical virtual reality as immersive as consensus reality. Most likely to be
produced using a Brain-computer interface. An intermediate stage may be produced by "Virtual Space" using a head-mounted display with head tracking and computer control of the image presented to the helmet.
VR Examples:Telepresence VR
VR Examples:
Augmented VR
VR Examples:Distributed VR
Technologies of VR Head-Mounted Display (HMD)
- A Helmet or a face mask providing the visual and auditory displays.
- Use LCD or CRT to display stereo images.- May include built-in head-tracker and stereo headphones
Technologies of VR Binocular Omni-Orientation Monitor (BOOM)
- Head-coupled stereoscopic display device.- Uses CRT to provide high-resolution display.- Convenient to use.- Fast and accurate built-in tracking.
Technologies of VRCave Automatic Virtual Environment (CAVE)
- Provides the illusion of immersion by projecting stereo images on the walls and floor of a room-sized cube.
- A head tracking system continuously adjust the stereo projection to the current position of the leading viewer.
Technologies of VR Data Glove
Outfitted with sensors on the fingers as well as an overall position/orientation tracking equipment.
Enables natural interaction with virtual objects by hand gesture recognition.
Technologies of VR Control Devices
Control virtual objects in 3 dimensions.
Architecture of VR System
Input Processor, Simulation Processor, Rendering Processor and World Database.
InputProcessor
RenderingProcessor
World Database
SimulationProcessor
visual, auditory, haptic, touch…
Position &Orientation
Applications
Entertainment
More vivid
Move exciting
More attractive
Applications Medicine
- Practice performing surgery.
- Perform surgery on a remote patient.
- Teach new skills in a safe, controlled environment.
Applications Manufacturing
Easy to modify
Low cost
High efficient
Applications
Education & Training
Driving simulators.
Flight simulators.
Ship simulators.
Tank simulators.
Current problems & Future work
Cybersickness / simulator sickness Low-fidelity Expensive Lack of integration between application packages
High-fidelity system Cost-saving Collaborative High-level contact between participants in distributed
VR
Graphics User Interface:Graphics have revolutionized user interface design. Properly used it can
effectively utilize a person's information processing abilities and allow for faster interaction with computer systems.
Characteristics of a GUI:
A user interface is a collection of techniques and mechanisms that allow a user to interact with a system.
Graphical: primary interaction mechanism is a pointing device
Objects: The user interacts with a collection of elements called objects which are always visible to the user and are used to perform tasks.
Actions: the user performs actions on objects such as accessing and modifying by pointing, selecting and manipulating
User Interface Styles Direct manipulation
GUI (graphical user interface)
WIMP (windows, icons, mouse, pull-down
menus)
Menus
Forms
Command language
Direct Manipulation Interface Characteristics
Screen objects resemble physical objects
Objects arranged in 2-D space
Trades perceptual motor operations for linguistic operations
Use of recognition in place of recall
Expensive to implement
Direct Manipulation Interfaces Allow
Novices to learn basic fundamentals quickly Experts to carry out new tasks Knowledgeable intermittent users to retain operational concepts Error messages are rarely needed Users can assess progress to goals and make changes instantly Users experience less anxiety because systems is understood
and actions are reversible Users gain confidence and mastery through their sense of
control over the system
Direct Manipulation Concerns
Increased system resources
Cumbersome actions
Weak macro techniques
History tracing is hard to log
Visually impaired users at risk
Problems with Direct Manipulation Interfaces
Visual representations are more spread out than simple text - causing "off page" problems
Users must learn meaning of components (e.g. icons) which are meaningful to designer and not user
Visual representation may be misleading
Touch typists do better with a keyboard than with a mouse
Icons The debate concerning text versus icons is an emotional one.
The usefulness of icons depends on how quickly user can figure out their meanings.
Components for a Successful Virtual Reality Application
Visual display
Head positioning and sensing
Hand positioning and sensing
Force feedback
Sound input/output
Other sensation
Cooperative and competitive VR requires networking
Menu Architectures Single
Linear sequence
Tree
Acyclic network
Cyclic network
Menu Item Design Guidelines
Command set small enough to fit within single menu User always have access to all possible menu items
without having to refer to a manual Logical presentation sequences (time, numeric,
alphabetic, physical properties, function/task organization, frequency of use, most important first)
Icons harder to recognize than words for abstract concepts
Avoid screen clutter Don’t assume user will notice subtle cues (e.g. color or
border changes)
Making Selection Easy
Provide command key bypasses for frequent commands
Ensure consistent selection and navigation methods throughout
Be aware of Fitt's law considerations for pointing devices
GOMS Model for Command Language Interface User Task
Step1. Think of and enter command verb
Step2. Think of and enter next argument
Step3. if more arguments then
go to Step2
Step4. if command is incorrect then
correct the command
Step5. Signal computer to process the command.
Step6. go to Step1
Dialog Boxes Combinations of all four interface styles (menu, form fill-in, direct
manipulation, command line)
User task model would also be a composite model
Dialog Box Design Guidelines
Meaningful title
Top-left to bottom right sequencing
Proper clustering and emphasis
Consistent language
Consistent terminology, fonts, capitalization, justification
Standard buttons
Error prevention by direct manipulation
Dialog Box External Relationship Design Guidelines
Smooth appearance and disappearance
Distinguishable boundaries
Sized to reduce overlap problems
Displayed close to appropriate screen objects
No overlap of required items
Easy to make disappear
Clear directions to cancel or complete operations
Advantages of GUIs Symbols are easy to recognize and fast to learn.
Colour is important for classifying objects.
Symbols can aid in problem solving.
Casual users can remember symbols easier than words.
Visual and spatial cues can be utilized to provide more information in a natural way. This, of course, excludes the visually-impaired.
Advantages of GUIs
Some types of error situations can be avoided because they are not allowed to occur through input.
Reduces the need for typing skills. Pointing skills are needed instead and this is not trivial for all types of
users (the elderly, persons with limited fine motor control).
Immediate feedback allows for a better conceptual model of the system for the user.
Advantages of GUIs More attractive than other interfaces and thus encourages more
interaction and exploration. Symbols have the potential to be much more universal than
natural language text. There are still cultural differences in the interpretation of symbols
and colours and thus the need to consider the consequences of internationalism are not entirely eliminated.
There are many fewer symbols than words so not everything can be expressed as a symbol.
Top Related