Complementary Capability Computing on HPCx Dr Alan Gray
description
Transcript of Complementary Capability Computing on HPCx Dr Alan Gray
Complementary Capability Computing on HPCx
Dr Alan Gray
212th November 2008 HPCx User Group Meeting
HPCx and HECToR: complementary services
312th November 2008 HPCx User Group Meeting
Complementary capability computing
• Complementary services provide a unique opportunity to maximise the benefits for UK research– HECToR is our leading HPC facility – HPCx is our "National Supercomputer", trading
overall utilisation in favour of a more flexible servic
• The main principles behind providing complementary services are:– to maximise the combined research benefits of
HECToR and HPCx for the UK HPC user community; – to ensure that the most appropriate service is
chosen for the scientific research to be conducted.
412th November 2008 HPCx User Group Meeting
HPCx: a more flexible service
• HPCx’s operation has been modified to provide a more flexible service– We have adapted policies and enabled new
features and functionalities• HPCx can handle more unusual jobs
– e.g. jobs that cannot easily or readily be accommodated on HECToR
• Certain job classes have adverse impact on overall utilisation on prime national service– but can be accommodated on a complementary
service• HPCx users have helped to guide these changes
– We visited all consortia with >= 1M AUs remaining
512th November 2008 HPCx User Group Meeting
HPCx flexibility
612th November 2008 HPCx User Group Meeting
Flexible Policies
• Very long jobs– HPCx now has 48 hour queues, requests for
longer jobs will also be considered
• Interactive and responsive computing– HPCx now has short high priority 20 minute
debug queues– HPCx can now accommodate more flexible
access patterns, for example to exploit grid technologies
712th November 2008 HPCx User Group Meeting
New Features and Functionality
• Advanced reservation– HPCx now supports the Highly Available Robust
Co-scheduler (HARC) – Allows users to reserve processors for a
particular time in the future – Relevant for meta-computing, computational
steering and on-line visualisation – You may just wish to reserve a series of
processors one afternoon
812th November 2008 HPCx User Group Meeting
New Features and Functionality
• Ensembles of jobs – We have developed functionality which allows
multiple independent jobs to be easily submitted from the same job script to run simultaneously
• Visualisation– Paraview parallel visualisation tool now installed
• Computational steering– We are working directly with consortia to enable
this for their specific cases
912th November 2008 HPCx User Group Meeting
Additional flexibility
• Shared memory– A cluster of fat shared-memory nodes
• each with several processors under the control of a single operating system
– Advantageous for applications that exploit shared memory parallelism and for users with large memory jobs
– Accessing the full memory • Under populating the node • Shared memory segments and shared memory
parallelism
1012th November 2008 HPCx User Group Meeting
Additional flexibility
• Large memory jobs– Phase 4 upgrade of HPCx – Two of the IBM 575+ servers will have 128 GB
of memory
• Data intensive jobs– HPCx has a significant tape store– Ideally suited for users with a need to archive
large amounts of data to tape for local processing
1112th November 2008 HPCx User Group Meeting
How to exploit HPCx’s new flexibility
• For the remainder of HPCx, we will (in addition to running the helpdesk) focus our additional support on Complementarity– In particular we will support the new CCC projects
in any way we can• Please get in touch with the helpdesk with
– questions– suggestions or requests– requirements that cannot easily be met within the
current operational policies of HECToR and HPCx – feedback
• Guidelines on Complementarity are available at:– http://www.hpcx.ac.uk/support/complementarity/