Two Dozen Ways to Screw Up a Two Dozen Ways to Screw Up a Perfectly Good ProjectPerfectly Good Project

Dave RohrlDave Rohrl

Senior Producer, EA/PogoSenior Producer, EA/Pogo

drohrl@pogo.comdrohrl@pogo.com

Presented at GDC 2002Presented at GDC 2002

Mistakes Producers Make

• Producers establish standards and practices• They often search for the comfortable and

intuitive– Bad idea!

• Many common mistakes are covered in the literature:– Project management, software engineering,

general management

Mistakes in Depth

• Why do we do it? – The Lure.

• What is the likely outcome? – The Consequence.

• What should be doing?– The Best Practice

Project Timeline

• Planning/Pre-Production

• Production

• QA/Playtest

• Whenever

1. Don’t Contain Your Ambitions

• Lure: – Life is too short to make B games

• Consequence: – Given infinite time and money, you still

couldn’t finish

• Best Practice:– Understand your team– Grow your team in measured ways– Do risky pieces first

2. Don’t Delineate Clear Roles• Lure:

– Multitalented, energetic teams• Consequence:

– When decisions have to be made, people will get bummed

– Problem-solving gets messy• Best Practice:

– Decide who gets to decide what– Make sure everyone knows/agrees– Input authority

3. Don’t Worry About Tool Sets

• Lure:– Tools are dull (relative to games)

• Consequence:– Assets need to be re-created

• Best Practice:– No game assets until tool set is agreed on– Build tools first with scanty harness

4. Be Afraid to Close Decisions• Lure:

– Great games require iteration– Decisions are hard

• Consequence:– No closure = no reliable progress– Later decisions come undone

The Virtuous Course of Game Development

Time

InfiniteUniverse ofPossibilities Individual

Product

Start of brainstorming

ProductRelease

The Shameful Course of Game Development

Time

InfiniteUniverse ofPossibilities

PossibleProducts

?

4. Be Afraid to Close Decisions (cont.)

• Best Practice:– Set criteria– Define exploration/decision paths– Identify and manage risks

5. Don’t Budget Time for Rework

• Lure:– Tight schedules, solid design planning

• Consequence:– Change will happen. Period.– Playtest will reveal key fun factor plays

• Best Practice:– Add rework time to initial schedule– Less experience = more time– 20% to start

6. Surrender Control of Schedule, Budget, AND Scope

• Lure:– What the suits want

• Consequence:– Hard to keep balanced in best of circumstances– Doomed to failure in worst

• Best Practice:– Keep one wild card– Use the Project Manager’s Triangle

Project Manager’s Triangle

Schedule

Budget Content

Quick & Dirty Project

Schedule

Budget Content

‘A’ Title, Hopefully in Quarter

Schedule

Budget Content

7. Make the task durations fit the schedule constraints

• Lure:– The task estimates add up to a late project.

• Consequence:– Tasks take as long as you originally thought, or– Your staff can kiss their families goodbye

• Best Practice:– Change the project constraints

• Use the Project Manager’s Triangle to decide how.

8. Don’t Prototype

• Lure:– It’s all throwaway– Prototypes create unrealistic expectations

• Consequence:– Your production game becomes the throwaway

• Best Practice:– Build it, circulate it, leverage it– Keep good change notes

9. Don’t Write It Down• Lure:

– Too much time, especially to document processes

– It’s all going to change any way

• Consequence:– What was that game you were building again?– Mass confusion, wasted effort

• Best Practice:– Create design, tech, and process documentation– Budget time to maintain it– Make the level of detail fit your team and game

10. Roll Into Production as Quickly as Possible

• Lure:– It feels great; everyone’s doing what they love

• Consequence:– Inaccurate/incomplete design– Throwaway code and assets

• Best Practice:– Close out design issues– Review design in detail

11. Tackle the Easy/Fun Stuff First

• Lure:– It’s fun!– Duh!

• Consequence:– Spend 65% of your time on 80% of your tasks– Then 70% of your time on the rest

• Best Practice:– Put risky stuff first

• Makes schedule issues evident up front

12. Don’t Practice Change Control

• Lure:– Can’t have a great game without iteration– Some change is inevitable

• Consequence:– Project is completely out of control– You get called out on Fatbabies

• Best Practices:– Create a change control db (or similar)– Enforce the practice

Change Control DB ScreenProposed Change

Modules/Systems Impacted

Department AssessmentsDept People Effort Risk Rec? NotesEngine Jack 3d High No Not enough benefitLevel Des. Jill, Lois 4w Low Yes We have enough

slack & is very cool

Benefit/Justification

Approved? Yes No Pending

Requestor

13. Move Alpha Back, Hold Ship Date

• Lure:– So much QA time on schedule– Maybe less QA time will reduce the bug count?

• Consequence:– Your QA cycle isn’t shrinking– Your ship date isn’t holding

• Best Practice:– Try to keep the total test schedule constant– Submit risky modules to QA first

14. Assume You Can Make Up the Time Later

• Lure:– Comforting fiction

• Consequence:– Slippage is steady or continually increases

• Exemption for one-timers

– Goodbye nights and weekends

• Best Practices:– Increase schedule monitoring, create scaling

factor– Reset expectations if necessary– Create a detailed make-up plan

15. Give 80% Credit For Tasks That Are “80% Done”

• Lure:– Seems logical

• Consequence:– The other 20% takes the other 80% of the time

• Best Practice:– Use fine-grained subtasks and binary evaluation– Put risky subtasks first– Cross-check task estimates

16. Deliver to QA All At Once

• Lure:– Complete game needed for system testing

• Consequence:– Ballooning QA schedule/poor predictability

• Best Practice:– Deliver modules as early as possible– Use early results to model later schedule– Build test harnesses as necessary

17. Be Too Busy To Playtest• Lure:

– All the other tasks need a piece of you– Can’t stand to look at the game one more time– Will find bad stuff

• Consequence:– No one is parsing the overall experience– Bad stuff that hurts the global view ships

• Best Practice:– Budget >= 15% of time for playtesting– Take the game home if necessary

18. Don’t Get Fresh Eyes on the Game

• Lure:– Feedback is redundant or untimely– QA’s all over it– Takes too long to get people up to speed

• Consequence:– Takes too long to get new players up to speed– Potentially useful feedback lost

• Best Practice:– Internal and external fresh eyes throughout

19. Don’t Use Project Management Software

• Lure:– MS Project is overkill– Team members don’t read Gantt/PERT charts

• Consequence:– Hard to track dependencies– Can’t understand schedule impact of slippages

• Best Practice:– Become a power user of Project (or similar)– Learn to use resource/budget features– Keep a stripped format for team

20. Do It All Yourself• Lure:

– Damn, you’re good!– Getting people up to speed takes too long

• Consequence:– Chief bottleneck– No time/energy to make good decisions

• Best Practice– Learn effective delegation– Bring in contractors as needed

21. Don’t Conduct Risk Analysis

• Lure:– Not enough time, not well understood

• Consequence:– Sudden, surprising, and huge disruptions

• Best Practice:– Maintain a top 10 risks list

Top 10 Risks Chart

# Risk Last Week

Weeks on Chart

Abatement Plan

Done Notes

1 OTM dev $ not greenlit

3 6 Muscle on to Mar exec review

No Can’t make Q3 without approval

2 OTG design spotty

N/A 1 Dave rewrites sketchy sections

Yes Still 4 sections for dev to redo

22. Close Lines of Communication

• Lure:– That’s what leads are for

• Consequence:– Low morale, high turnover, flat game

• Best Practice:– Decisions go down through leads and team

meetings– Ideas come up and around from everywhere– Don’t let people confuse input and authority

23. Don’t Schedule Regular Team Meetings

• Lure:– Meetings don’t get tasks done– Lots of developers hate meetings

• Consequence:– Team gets out of synch– Cross discipline problems fester– No clear vision of how overall game is

progressing

• Best Practice– Meet regularly throughout– Get more (not less) frequent as GM gets closer

24. Don’t Use a Review Process

• Lure:– Reviews take time– Good people are putting out quality work

• They get touchy when people critique them.

• Consequence:– Lots and lots of rework– Far more expensive to fix later

• Best Practice:– Use a consistent review process throughout– Budget 5-10% of staff time for reviews

Parting Thoughts

• The lures of these mistakes make them common and recurring.

• Learn from every project– Be honest with yourself and your team

• Study software engineering, project management, and risk management

• Make new mistakes every time out– Send them to mistakes@universeman.com

• Thanks!

Bibliography

• Peopleware: Productive Projects and Teams• Quality Software Management Series (Volumes 1-

2) • Handbook of Walkthroughs, Inspections, and

Technical Reviews• The Mythical Man-Month• Software Project Survival Guide• Rapid Software Development

•Don't contain your ambitions •Don't delineate clear roles.•Don't worry about tool sets.

•Be afraid to close out decisions.•Don't budget time for rework.

•Surrender control of schedule, budget, AND scope.•Make the task durations fit the schedule constraints.

•Don't prototype.•Don't write it down.

•Roll into production as quickly as possible.

Planning/Pre-Production

•Deliver to QA all at once.•Be too busy to play the game.

•Don’t get fresh eyes on the game.

QA/Playtest

•Tackle the easy/fun stuff first.•Don't practice change control.

•Move your Alpha date back, but hold your ship date.•Assume you can make up the time later

•Give 80% credit for tasks that are 80% done.

Production

•Don't use project management software.•Do it all yourself.

•Don't conduct risk analysis •Close lines of communication.

•Don't schedule regular meetings.24. Don't bother with a review process.

Whenever