Hints on System Designand Research Methodology

CS290F

Winter 2005

System Research Change happens

problems have changed in last 20-30 years but research methodology stays “constant”

Success in graduate research choosing the right research problem doing the “work,” addressing the problem presenting the results

writing/publishing papers giving talks

writing / giving the perfect job talk

Wisdom of the Ages “Nuggets” of wisdom available

perspectives from veterans of the trade

We’ll try to summarize a couple of them “3 guidelines on choosing a research problem”

??? “Hints for computer system design”

Butler Lampson (Microsoft Research) “How to have a bad career in research…”

David Patterson (U. C. Berkeley) Various hints on giving talks, writing papers

Fox, Levin, Redell, etc…

Choosing a research problem Work on the edge of what you know

doing so expands your horizon of knowledge

Choose a problem that has impact either applicative or scientific if in doubt, choose scientific over application

Choose a problem you’re uniquely qualified to solve stay away from common problems well within

reach of competitors

Hints for Computer System DesignButler Lampson

PhD Berkeley 1967 Faculty: Berkeley, MIT Researcher: Xerox PARC, Digital

Research, Microsoft Research Turing Award, 1992 Hints… circa 1982 (SOSP-IX)

General Hints

Hints from building and studying real systems

For each hint, two key points why it helps in building a good system where in the system building it helps

Organized along the Why axis functionality speed fault-tolerance

I. Functionality Most basic motivation for a system Design of correct interfaces is key

it is a programming language

Keep it simple capture the minimal essence of an abstraction (do

not generalize, easy to err) simplify minimize implementation/debugging complexity critical for heavily used abstractions (files, V/M …)

less so for rarely used components (passwd gen)

Keep It Simple cont. Make it fast

programmers can build on top of a complete, fast API

can use profiling tools to gauge frequency of calls

Don’t hide power if the system can do something fast, open that

functionality to programmer

Leave it to the client if it’s simple to switch back & forth, solve one

problem and let the client do the rest

Continuity Keep basic interface stable

changing code to follow is expensive

Keep a place to stand design APIs with upgradability in mind support backward compatibility when possible

e.g. method deprecation in Java

Making Implementations Work Implement a strawman

the first try never works out but you learn what you really want can solidify APIs, system design

Keep secrets of the implementation hide things that can change tension with making assumptions for performance

Divide and conquer attack a hard problem by dividing into smaller

problems

Handling All the Cases

Handle normal & worse cases separately normal: should be fast/optimized worst: must make progress optimize for the common case (e.g.

caching)

II. Speed Split resources between consumers

tradeoff utilization for performance dedicated management is simpler/faster

Use static analysis if you can but fall back on dynamic if not possible

Cache answers to expensive computations only if f(x) is functional (same result w/ same args) must choose cache size correctly to avoid thrashing

Use hints to speed up execution speculate optimistically on results of calculations must check for correctness (cheaply)

More Speed …

When in doubt, use brute force choose b.f. rather than optimizations

based on weak/incorrect assumptions easy analysis over complexity leverage Moore’s Law

Compute in background if possibleUse batch processing if possible

III. Fault-tolerance End-to-end

error recovery always necessary e2e all others are optional, for performance

Log updates cheap, append-only allow regeneration of current state “functional” updates using constant args

Make actions atomic or restartable simplify recovery

Break…

Then we’ll talk about Patterson’s advice “how to have a bad career in research”

Hints on giving talks (Fox)Hints on writing papers (Fox)

Hints for Giving a Talk (Fox)

Know the audience and watch the jargon

Keep the big picture in mindTell me a story, don’t read an articlePace yourselfWhat did you just say again?There will be questions

Know the Audience & Watch Jargon

Many talks have varied audiences the most critical one: your job talk

Use the hourglass model start broad (high level), hook everyone in

convince audience the problem is useful and hard dive down into details, impress experts

just sufficient detail to show convincing depth wrap up, recapture entire audience

Avoid jargon / technical terms define terminology before usage

Keep the Big Picture in Mind What is the big picture?

motivate the problem well can take up to 1/3 of your talk

What is the context? what are the subproblems? where does your work fit in? how does it relate to existing work?

What is your high level contribution? come back to high level throughout talk

Tell Me a Story Slides

avoid sentences, use short phrases slides are hints for you to speak from

Presentation style use minimal notes maintain eye contact, move across audience use personal experience to elaborate bullets

Humor can animate talk, grab attention of audience can be annoying if overdone

Pace Yourself Time management is critical

going over is very bad form (esp. job talk)

Keys practice, practice, practice use key timepoints to gauge progress rule of thumb: 2 min / content slide

Be adaptive “speed up” if behind schedule know when and what to skip over be prepared for extra details if time left

What Did You Just Say Again? Use recaps to highlight high-level ideas

especially important if middle of talk focused on technical details

easy to lose track of “the point” remind audience of significance of work

Future work an implications use slides near end to discuss consequences of

your work (new things to be done, new issues discovered by your work)

There will be Questions Questions are unpredictable

know material one level deeper than slides think about issues ahead (prepare backup slides)

Time management answer questions with composure handle nonstop persistent questions with “let’s

take this offline”

Key practice talks you care about get ruthless (and varied) audience members

How to Write a Good xxxx Paper Three classes of papers

presents a real system (implementation) what did you learn from the experience what was unexpected? if measurement paper, determine which graphs

convey your message, then wrap paper around them.

presents an unimplemented system (vision)with useful ideas/techniques

you must be convinced of your vision! make concrete statements that can be backed up

by citations, real data, or prelim. results others (theory papers, survey papers…)

Before You Write Give 5-10 min talk to colleague

forces you to collect thoughts others will help identify strength/weaknesses and

contributions

Do not cram last minute papers do *not* work out corollary: multitasking on papers is generally bad

Know when to take a break can get too close to the work take break to reestablish focus and perspective

Starting Checks Write from an outline

figure out main message (from 10000 feet) everything else follows use bulleted list

Do not write abstract/title until end the paper will change as you write intro will also morph

Make small # of points usually 2-4 appear in: intro, body, conclusion

Writing start from outline make outline reflect subsections

write out succinct points of each section what *must* be said

write out topic sentences for each section read through for logical flow give to colleague in diff. area for validation fill in paragraphs

Editing Organization/structure can change

this is ok, 3-4 major revisions are normal

Pause for top-down reading do this occasionally to maintain perspective

Get early feedback external reviews on drafts are invaluable

Cite cite cite be complete in related work

Writing Style

See Strunk and WhiteUse active sentencesOmit needless wordsBe brief and concise, not flowery or

formalCheck for spelling and grammar

Final Checks Does paper make contributions clear? Does outline of paper flow at each logical level? Before telling me about X, convince me I should

care Have you made each important point 3 times? Have you gotten reviews from each relevant area? Have you searched for and acknowledged related

work? don’t put down previous work, be positive about your

differences

Do the abstract/conclusion capture the top-level message?