COMS/CPRE 425 Spring 2005 Lecture 1cs425/lectures/L05_01.pdf · 1/10/2005 ComS 425 Spring 2005 7 of...

1/10/2005 ComS 425 Spring 2005 1 of 46: Lecture 1

COMS/CPRE 425Spring 2005

Lecture 1

Ricky A. [email protected]


Computer Science 425High Performance Computing for

Scientific and Engineering Applications• Instructor: Ricky A. Kendall• Teaching Assistant: Unknown L• Lectures (Gilman Hall 0312)

ØMWF 11:00-11:50• Recitation (Atanasoff B29)

ØOnly IFF needed!!!!ØW 12:10-13:00

• Reserved Laboratory Time (Atanasoff 116A)ØW 12:10-16:00 ØThe middle LabØYou have priority. Non CS425 students can be “bumped”


Logistics• Pop Quiz TodayØThe only unannounced “exam” in the course.


Student Contacts• Make sure you have a valid email address in

the ISU system. That address will be used to set up a class mailing list.

• If you have to travel or be away make sure I know about this ahead of time.ØEmail is fine!ØThe Midterm.§ 3/2/2005 Wednesday§ You know it avoid travel on this date!

• Class and Lab/Recitation attendance is not mandatory, but advised. ØAttendance is REQUIRED during Student

Presentations!!!!!!!!!!!!!!!!!!


Ricky’s Contact Information• Email: [email protected]ØPreferred normal contact method.Ø I read email daily (even when on vacation). If you don’t get

a response within a few hours re-send it. ØAIM ID : RickyAKendallØYahoo Messenger ID: RickyAKendallØMSN ID: [email protected]§ If I am busy I’ll tell you and make an appointment.

• Office Hours are by appointment.ØDon’t hesitate to make an appointment.Ø I will attend the lab time, when I can.ØThursday @ 2pm if not “overbooked”

• Office 336A Wilhelm HallØOffice Phone: 294-9294 (okay; don’t be offended if I’m busy)ØCell: 450-4254 (emergency)ØHome: 292-1304 (emergency)


Teaching Assistant• UnknownØGraduate Student in Computer Science


Instructor’s Introduction(1)• General InformationØSoon to be 44 year old that is still trying to figure

out what he wants to do when he grows up.ØRaised in farm country in Indiana.ØWife and two sons (9 and 6)§ Very cute (take after their Mother).

ØActivities§ Chasing Children

♦ Soccer, wrestling, baseball, basketball, gymnastics♦ Webmaster for the Ames Youth Wrestling Club

§ Linux Hacking, Abusing Windows§ Camping, Fishing, and Fly Fishing§ Bowling

♦ ~180 Averages in 2 different leagues.♦ High Game 283; High Series 748♦ 1998 Mixed Doubles Champion, Tricities, WA


Instructor’s Introduction(2)

• EducationØB.S. Indiana State University, 1983 §Chemistry/Mathematics

ØPh.D. University of Utah, 1988 §Theoretical and Computational

ChemistryØPostdoctoral Fellowship @ Argonne

National Laboratory, 1988-1989§Computational Chemistry and

Computational Science.


Instructor’s Introduction(3)• Work ExperienceØStaff Scientist, Pacific Northwest National

Laboratory (1989-1999)§ NWChem (4,000,000 lines of code C and F77)

ØInstructor Washington State University (1996-1999)§ Systems Programming

ØComputational Scientist @ Ames Laboratory (1999-?)

ØAdjunct Associate Professor Department of Computer Science, Iowa State University (1999-?)


Instructor’s Introduction(4)• Research InterestsØApplication Enabling Technology§ Communication Systems and Optimization§ Programming Models

♦ Augmenting existing systems♦ Developing new ways of using current technology♦ Application to Computational Chemistry software

§ Integrating Computer Science Technology into legacy applications

♦ Most are written in Fortran even though it’s dead!

ØComputational Chemistry Algorithm Development§ New parallel algorithms of existing technology.§ Applications of Theoretical Methods to Chemistry.


Instructor’s Travel• Gone March 5th and 6th. ØState Bowling Tournament.

• I will inform you of additional travel details if necessary and as soon as possible.

• We may have to make up some lectures during the recitation period if more travel is required and I cannot produce a guest lecturer.ØNote: this has not happened in the past.ØNeeds to be kept as an option.


Catalog Course Description• Introduction to high performance computing

using different computing platforms including parallel computers and workstation clusters.

• Discussion of performance, visualization, and software development issues.

• Sample applications from science and engineering.

• Practical issues in high performance computing will be emphasized via a number of programming projects and case studies.

• Oral and written reports.


CS425 Course Goals• What I hope you will LearnØProper Programming Practices.ØAbility to understand and design high performance

and parallel algorithms.§ Basically to develop a tool kit to use in your next “phase”

of life. ♦ Understand which tool to use!

ØAbility to implement algorithms using a variety of high performance and parallel programming models.ØRespect for the “Art of Programming.”§ An important part of Computer Science is the application

of the theoretical to the practical.


CS425 What I expect you to know!• How to learn!• How to speak up!• C, C++, or Fortran.ØOr all of them ☺

• How to generate simple makefiles• How to use multiple compilersØMy code won’t compile LØYou should know the syntax of you language of choice!ØHow to read documentation about the compilers§ -O is not the highest level of optimization from any compiler!

• How to read man pages.• How to use a debugger!ØMy code doesn’t run LØYou should have some idea why! E.g., where it dies.


CS425 Student Comments• Practical but Hard course• Instructor is arrogant and closed mindedØYep!!!

• Learned a lot but it was a lot of work.ØYep!!!

• Please put the lecture notes up before the lecture.

• Exams should be harder!• Instructor should never teach a class again.ØI messed that one up L

• I yell too much.


Course Expectations• Lectures should be interactiveØNo questions for me à you understand everything ☺Ø If I say something that does not make sense then ask a

question!ØNotes will be put on the course page before the lecture is

given. § But I cannot guarantee they will be there in time for you to print

them out.ØPost questions for discussion on the class news group or

email list!• FeedbackØPlease let me know what you think about the course lecture

style, material, etc.Ø I cannot read your mind, so speak up!Ø I welcome the feedback, positive or negative!!


Course Expectations (2)• Course Material you are responsible for

includes:ØLecture NotesØReading assignments from the TextØHandoutsØMaterial and Specified Links from the Class Home

Page.ØAssigned papers to read.ØStudent and Guest Lecturer presentations.


Your Grade• 60 % Homework AssignmentsØYes this is a programming course!

• 15% Final Exam• 15% Written and Oral ProjectØYou must do this! If you do not complete

this project it is an automatic F.• 10% Midterm Exam• In other words this is a programming

course! • Course ResourcesØURL: http://www.cs.iastate.edu/~cs425


Textbooks• The required Textbook

for this course is “Parallel Programming in C with MPI and OpenMP” Øby Michael J. Quinn

• USB, CBS, onlineØBuy it wherever.


Textbooks• Optional TextbookØ“Patterns for

Parallel Programming” (Software Patterns Series) by Timothy G. Mattson, Beverly A. Sanders, Berna L. Massingill


Textbooks• Optional TextbookØParallel Programming in

OpenMP§ by Rohit Chandra

(Editor), Leo Dagum, Dave Kohr, Dror Maydan, Jeff McDonald, RameshMenon


Textbooks• Optional TextbookØPthreads Programming:

A POSIX Standard for Better Multiprocessing (O'Reilly Nutshell)§ by Bradford Nichols, Dick

Buttlar, Jacqueline ProulxFarrell, Jackie Farrell

• Quinn does not cover Pthreads at all!!!ØBut we will ☺


Textbooks

• Optional TextbookØParallel

Programming With MPI§by Peter Pacheco


Textbooks• Books you might want to look at:ØParallel and Distributed Computing, by

Claudia LeopoldØExpert C Programming,

by Peter van der LindenØA Practical Guide to Linux,

by Mark G. SobelØDesigning and Building Parallel

Programs, by Ian Foster§ Referenced by Quinn frequently

ØHigh Performance Computing, § by Dowd and Severance


Basic Project Information• Research any area of High Performance Computing.• Write a written report that is a critical analysis of the

work (HTML/PDF/WORD).ØNo more than 20 pages.

• Make a presentation to the class regarding this work.ØShow the work and analysisØExamine possible enhancements or where further work

should done.ØDone in powerpoint or similar product.§ Should also provide a PDF file of presentation to post on class web

page for other students to access.• Review other student oral presentations. • Review content of other students presentations before

the final.ØHINT, HINT, HINT


Homework Assignments• We will have 5 to 7 assignments over the semester.• You will get accounts on both local and remote

systems.ØCS systems, SCL systems, NERSC systems.§ SCL and NERSC systems use a BATCH system so you cannot run

your jobs interactively.§ You can debug interactively!!

• You will have to use SSH version 2.ØThere are free Unix and Windows clients.§ See the class web page.

• You cannot abuse these systems, this is just for work on this course.

• Do not wait until the last minute to start the homework assignments. ØRe-read the bullet above!ØRead it again!!!!!!!!!!!!!!!!!!!


Homework Expectations• Programming Assignments:Ø Turned in on time via email; due as specified in

assignment.Ø Late Assignments are penalized 50% per day.Ø DO NOT WAIT UNTIL the LAST MINUTE to START!§ Some work will be run using a batch system (not interactively).

Ø Class discussion of issues regarding each homework among classmates is acceptable, and encouraged.

Ø You may work in teams of 2, or individually for all homework assignments after the first one.§ The first assignment will be done individually. ☺§ Teams should “SHARE” the work!!§ Groups determined “randomly”


Homework Expectations(2)• Cheating, copying others work is not allowed.Ø If caught the words “academic fraud” will mean

something!§ Plagiarism is cheating!

Ø Others means any other person, including the instructor.Ø Code examples given in lecture notes are literally that;

examples§ You may not use code examples as your starting base§ You may not use makefile examples as your starting base§ In other words write your own stuff

Ø 7 students in the past have been caught.§ Highest grade delivered for “cheaters” was a D+

♦ Only on one assignment.Ø Note that I have all homework turned in since I started

teaching this course and I run comparisons over these and work turned in.§ GNU diff is a wonderful tool ☺


Homework Expectations (3)• Each turned in homework should include:

1. Design document based on assignment specification.

2. ANSI C is the official language of the course!– You may use ANSI C, C++, Fortran, etc. but makefiles

and codes must work.3. Codes and makefiles must run on the assigned

system!– YES YOU HAVE TO PROVIDE A MAKEFILE.

4. Output from your test runs.5. List of known problems or issues with your

implementation.6. Appropriate analysis of your implementation etc.


Homework Expectations (4)• Each homework is graded as:Ø Design Document 10%Ø Makefile 10%Ø Code and Performance 70%Ø Results and Analysis Document 10%


Pseudo-Syllabus• Introductions and Expectations• Overview• Parallel Computing/Computers• Matrix Multiply• Parallel Matrix Multiply• Shared Memory/Pthreads/OpenMP• Message Passing/MPI/Others?• Distributed Shared Memory ØGlobal Arrays and SHMEM

• Applications, Applications, Applications• Calendar Link from the Class Home Page shows

timing.


The Programming ProcessRequirementsSpecificationCode Design

CodingTesting

DebuggingRelease

MaintenanceRevisions and Updates

This is a cyclic process


TPP: Requirements• States what the program should do in very

general terms (the customer, teacher, or boss’ perspective).

• Will be vague and incomplete.• Language is a barrier to understanding the

requirements.• The programming assignments are the

requirements.


TPP: Specifications• A detailed description of what the program does (the

programmer’s perspective).• Restatement of the requirements with specifics about

the assumptions for undefined aspects of the recommendations.

• The initial specification should always be:Ø reviewed with the boss/customer, discussed with your peers.

• A list of:Øoptions for input and runtime; description of the output.Øknown error conditions and return codes.

• Changes with the development of software: A “living” document


TPP: Code Design• Where the rubber meets the road!• Always the hardest part for large projects.• Includes the programmers artistic talents.• Determine overall structure of the application.Øscope and function of modules.ØDetermine file formats data structures.ØDetermine algorithms to be used. § Requires algorithmic analysis and may require some

code prototyping. § Portability with different hardware may require multiple

algorithms.• Write pseudo-code for each component.


TPP: Coding• Use the so-called fast-prototyping method.• Develop components of the applicationØThe code design should help determine which

components to work on first.• Test and debug components after each is

developed.• Just like taking an exam; get the trivial stuff

out of the way first.• Use make and makefiles to build the

application.Ømake determines which components need to be

recompiled at each phase (useful!)


TPP: Testing• Develop a test plan for each component.• A good idea to automate this Ø(e.g., using a shell script).

• The test plan and any automation will grow as the code is developed.

• Try to “break” the code by using incorrect input. You should see one of your error handling messages!

• The more QA you do the lower the maintenance burden will be.


TPP: Debugging• If you are only a novice C programmer the use of a

debugger is essential.• It will help you find:Ømismatches between pointers and addressesØwalk through you logic to determine where it is in errorØ is much better that overloading the code with printfØA graphical debugger makes this much more amenable.

• gdb and xxgdb are the line based and graphical debuggers available on the Linux systems.

• Unfortunately for parallel codes the debuggers are limited. ØTotalview is one tool.


TPP: ReleaseThis is when your software is:• packaged (shrink wrap, WWW, FTP)• documented• sent to the “user” community• As in Homework turned in ☺


TPP: Maintenance• Bugs come from users and developers• When a bug is found:Ørecreate the bug (reliability)Ølocate the problemØdesign the patch/fixØmake the changes

• As in the next Homework ØYou get to fix what’s busted!ØEach homework will build upon

previous experience!


TPP: Revisions and Updates• Changed or Revisions come from:Øusers and developers (e.g., new

requirements)Ønew algorithmic developmentØrequests for additional functionalityØstarts the process over from the

specification stepØRe-interpreting the Assingments!!


The Programming Process

Requirements

Specification

Code Design Code Development

Testing

Debugging

ReleaseRevision

Maintenance


TPP: Other Useful Tips• Revision controlØAllows you to log changes to the code.ØGives you a mechanism to backtrack to previous

versions of the software.ØAffords a graceful way for multiple developers to

work on the same system.• Revision control toolsØcp with version numbers,SCCS, ØRCSØCVS (A good open source solution)


Proper Programming Practices• Good programmers didn’t start that way! ØUse a process (like the one just described).ØIt is a Learned Art.

• Writing good software takes practice and this course is a practical “instance” of practice (redundant :)

• Good programming practices sometimes seem like just dumb rules but over time you appreciate the “good” code you have written and detest the rest!

• The detailed definition of good programming practices is analogous to religious arguments.


Dumb Rules (1)• Include the appropriate header files ØSystem < >ØUser “ “

• Always use function prototypes (This is ANSI)• Use obvious variable names:ØMyCpuTime, argument, counter, user_count

• If a variable is simply reused (e.g., it’s context changes)Øuse comment to note this in the source code.

• Use a consistent format for braces, if statements, etc.


Dumb Rules (2)• Use comments where the code is

ambiguous. Your comments will help you figure out what you did when you have to find and fix bugs in your software.

• Break all of these rules if “clarity” wins out!

• If you (or someone grading an assignment) cannot “understand” your code then there is a problem!

COMS/CPRE 425 Spring 2005 Lecture 1cs425/lectures/L05_01.pdf · 1/10/2005 ComS 425 Spring 2005 7 of...

Documents

Transcript of COMS/CPRE 425 Spring 2005 Lecture 1cs425/lectures/L05_01.pdf · 1/10/2005 ComS 425 Spring 2005 7 of...