CS 638, Fall 2001

Interactive Programs

• Games are interactive systems - they must respond to the user

• Today is all about how interactive programs are designed

• Also, some issues to think about in interactive systems

CS 638, Fall 2001

Interactive Program Structure

• Event driven programming– Everything happens in

response to an event

• Events come from two sources:– The user– The system

• Events are also called messages– An event causes a message

to be sent…

Initialize

User Does Somethingor

Timer Goes Off

System Updates

CS 638, Fall 2001

User Events

• The OS manages user input– Interrupts at the hardware level …

– Get converted into events in queues at the windowing level …

– Are made available to your program

• It is generally up to the application to make use of the event stream

• Windowing systems may abstract the events for you

CS 638, Fall 2001

System Events

• Windowing systems provide timer events– The application requests an event at a future time– The system will provide an event sometime around the

requested time. Semantics vary:• Guaranteed to come before the requested time• As soon as possible after• Almost never right on (real-time OS?)

• Application is not interrupted - it has to look for the event– Exception: Alarm signals in UNIX

CS 638, Fall 2001

Polling for Events

• Most windowing systems provide a non-blocking event function– Does not wait for an event, just returns NULL if one is not ready

• What type of games might use this structure?

• Why wouldn’t you always use it?

while ( true )if ( e = checkEvent() )

switch ( e.type )…

do more work

CS 638, Fall 2001

Waiting for Events

• Most windowing systems provide a blocking event function– Waits (blocks) until an event is available

– Usually used with timer events. Why?

• On what systems is this better than the previous method?

• What types of games is it useful for?

e = nextEvent();switch ( e.type )

…

CS 638, Fall 2001

The Callback Abstraction

• A common event abstraction is the callback mechanism• Applications register functions they wish to have called in

response to particular events– Translation table says which callbacks go with which events

• Generally found in GUI (graphical user interface) toolkits– “When the button is pressed, invoke the callback”– FLTK works this way– Many systems mix methods, or have a catch-all callback for

unclaimed events– In X windows, Xlib defines events, Xtoolkit defines callbacks

• Why are callbacks good? Why are they bad?

CS 638, Fall 2001

LithTech Messaging

• LithTech allows any of the methods we have discussed– Callbacks for the system timer

• the update function called once per frame

– Callbacks bound to multiple events• Same callback gets many events and internally decides what to

do with each one• Bindings done with a configuration file

– Polling for keyboard input. What sort of input?

• Read the Programming Guide chapter on Input to get an overview

CS 638, Fall 2001

Upon Receiving an Event …

• Event responses fall into two classes:– Task events: The event sparks a specific task or results in some

change of state within the current mode• eg Load, Save, Pick up a weapon, turn on the lights, …

• Call a function to do the job

– Mode switches: The event causes the game to shift to some other mode of operation

• eg Start game, quit, go to menu, …

• Switch event loops, because events now have different meanings

• Software structure reflects this - menu system is separate from run-time game system, for example

CS 638, Fall 2001

Real-Time Loop

• At the core of games with animation is a real-time loop:

• What else might you need to do?

• The number of times this loop executes per second is the frame rate– # frames per second (fps)

while ( true )process eventsupdate animationrender

CS 638, Fall 2001

Lag

• Lag is the time between when a user does something and when they see the result - also called latency– Too much lag and causality is distorted– With tight visual/motion coupling, too much lag makes people

motion sick• Big problem with head-mounted displays for virtual reality

– Too much lag makes it hard to target objects (and track them, and do all sorts of other perceptual tasks)

• High variance in lag also makes interaction difficult– Users can adjust to constant lag, but not variable lag

• From a psychological perspective, lag is the important variable

CS 638, Fall 2001

Computing Lag

• Lag is NOT the time it takes to compute 1 frame!

• What is the formula for maximum lag as a function of frame rate, fr?

• What is the formula for average lag?

Process input

Update state

Render

Process input

Update state

Render

Process input

Frame time

Lag

frlag

frlag

average

5.1

2max

Event time

CS 638, Fall 2001

Frame Rate Questions

• What is an acceptable frame rate for twitch games? Why?

• What is the maximum useful frame rate? Why?• What is the frame rate for NTSC television?• What is the minimum frame rate required for a

sense of presence? How do we know?• How can we manipulate the frame rate?

CS 638, Fall 2001

Frame Rate Answers (I)

• Twitch games demand at least 30fs, but the higher the better (lower lag)– Users see enemy’s motions sooner– Higher frame rates make targeting easier

• The maximum useful frame rate is the monitor refresh rate– Time taken for the monitor to draw one screen– Synchronization issues

• Buffer swap in graphics is timed with vertical sweep, so ideal frame rate is monitor refresh rate

• Can turn of synchronization, but get nasty artifacts on screen

CS 638, Fall 2001

Frame Rate Answers (II)

• NTSC television draws all the odd lines of the screen, then all the even ones (interlace format)– Full screen takes 1/30th of a second– Use 60fps to improve visuals, but only half of each frame actually

gets drawn by the screen– Do consoles only render 1/2 screen each time?

• It was once argued that 10fps was required for a sense of presence (being there)– Head mounted displays require 20fps or higher to avoid illness– Many factors influence the sense of presence– Perceptual studies indicate what frame rates are acceptable

CS 638, Fall 2001

Reducing Lag

• Faster algorithms and hardware is the obvious answer

• Designers choose a frame rate and put as much into the game as they can without going below the threshold– Part of design documents presented to the publisher

– Threshold assumes fastest hardware and all game features turned on

– Options given to players to reduce game features and improve their frame rate

• There is a resource budget: How much of the loop is dedicated to each aspect of the game (graphics, AI, sound, …)

• Some other techniques allow for more features and less lag

CS 638, Fall 2001

Decoupling Computation

• It is most important to minimize lag between the user actions and their direct consequences– So the input/rendering loop must have low latency

• Lag between actions and other consequences may be less severe– Time between input and the reaction of enemy can be greater– Time to switch animations can be greater

• Technique: Update different parts of the game at different rates, which requires decoupling them– For example, run graphics at 60fps, AI at 10fps– Done in Unreal engine, for instance

CS 638, Fall 2001

Animation and Sound

• Animation and sound need not be changed at high frequency, but they must be updated at high frequency– For example, switching from walk to run can happen at low

frequency, but joint angles for walking must be updated at every frame

• Solution is to package multiple frames of animation and submit them all at once to the renderer– Good idea anyway, makes animation independent of frame rate

• Sound is offloaded to the sound card

CS 638, Fall 2001

Graphics Review

• Recall the standard graphics pipeline:

CS 638, Fall 2001

Normal Vectors

• The intensity of a surface depends on its orientation with respect to the light and the viewer– CDs are an extreme example

• The surface normal vector describes the orientation of the surface at a point– Mathematically: Vector that is perpendicular to the

tangent plane of the surface• What’s the problem with this definition?

– Just “the normal vector” or “the normal”– Will use N to denote

CS 638, Fall 2001

Local Shading Models

• Local shading models provide a way to determine the intensity and color of a point on a surface– The models are local because they don’t consider other

objects at all

– We use them because they are fast and simple to compute

– They do not require knowledge of the entire scene, only the current piece of surface

CS 638, Fall 2001

Local Shading Models (Watt 6.2)

• What they capture:– Direct illumination from light sources

– Diffuse and Specular components

– (Very) Approximate effects of global lighting

• What they don’t do:– Shadows

– Mirrors

– Refraction

– Lots of other stuff …

CS 638, Fall 2001

“Standard” Lighting Model

• Consists of three terms linearly combined:– Diffuse component for the amount

of incoming light reflected equally in all directions

– Specular component for the amount of light reflected in a mirror-like fashion

– Ambient term to approximate light arriving via other surfaces

CS 638, Fall 2001

Diffuse Illumination

• Incoming light, Ii, from direction L, is reflected equally in all directions– No dependence on viewing direction

• Amount of light reflected depends on:– Angle of surface with respect to light source

• Actually, determines how much light is collected by the surface, to then be reflected

– Diffuse reflectance coefficient of the surface, kd

• Don’t want to illuminate back side. Use

)( NL id Ik

)0,max( NL id Ik

CS 638, Fall 2001

Diffuse Example

Where is the light?

CS 638, Fall 2001

Specular Reflection (Phong Model)

• Incoming light is reflected primarily in the mirror direction, R– Perceived intensity depends on the relationship between the

viewing direction, V, and the mirror direction– Bright spot is called a specularity

• Intensity controlled by:– The specular reflectance coefficient, ks

– The parameter, n, controls the apparent size of the specularity• Higher n, smaller highlight

nisIk )( VR

LR

V

CS 638, Fall 2001

Specular Example

CS 638, Fall 2001

Putting It Together

• Global ambient intensity, Ia:– Gross approximation to light bouncing around of all other surfaces

– Modulated by ambient reflectance ka

• Just sum all the terms

• If there are multiple lights, sum contributions from each light

• Several variations, and approximations …

nsdiaa kkIIkI )()( NHNL

CS 638, Fall 2001

Flat shading

• Compute shading at a representative point and apply to whole polygon– OpenGL uses one of the vertices

• Advantages: – Fast - one shading value per

polygon

• Disadvantages:– Inaccurate– Discontinuities at polygon

boundaries

CS 638, Fall 2001

Gourand Shading

• Shade each vertex with it’s own location and normal

• Linearly interpolate across the face

• Advantages:– Fast - incremental calculations

when rasterizing– Much smoother - use one normal

per shared vertex to get continuity between faces

• Disadvantages:– Specularities get lost

CS 638, Fall 2001

Phong Interpolation

• Interpolate normals across faces

• Shade each pixel

• Advantages:– High quality, narrow specularities

• Disadvantages:– Expensive

– Still an approximation for most surfaces

• Not to be confused with Phong’s specularity model

CS 638, Fall 2001