Parallel Iterative Solvers with the Selective Blocking Preconditioning for Simulations of Fault-Zone...
-
Upload
edward-bailey -
Category
Documents
-
view
219 -
download
0
Transcript of Parallel Iterative Solvers with the Selective Blocking Preconditioning for Simulations of Fault-Zone...
Parallel Iterative Solvers with the Parallel Iterative Solvers with the Selective Blocking Selective Blocking
Preconditioning for Simulations Preconditioning for Simulations of Fault-Zone Contactof Fault-Zone Contact
Kengo NakajimaKengo NakajimaGeoFEM/RIST, Japan.GeoFEM/RIST, Japan.
3rd ACES Workshop, May 5-10, 2002.3rd ACES Workshop, May 5-10, 2002.Maui, Hawaii, USA.Maui, Hawaii, USA.
22
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Solving large-scale linear equations Solving large-scale linear equations Ax=bAx=b is the most important and is the most important and expensiveexpensive part of various types of scientific computing. part of various types of scientific computing. for both linear and nonlinear applicationsfor both linear and nonlinear applications
Various types of methods have been proposed and developed. Various types of methods have been proposed and developed. for dense and sparse matricesfor dense and sparse matrices classified into classified into directdirect and and iterativeiterative methods methods
Dense Matrices : Globally Coupled ProblemsDense Matrices : Globally Coupled Problems BEM, Spectral Methods, MO/MD (gas, liquid)BEM, Spectral Methods, MO/MD (gas, liquid)
Sparse Matrices : Locally Defined ProblemsSparse Matrices : Locally Defined Problems FEMFEM, FDM, DEM, MD (solid), BEM w/FMP, FDM, DEM, MD (solid), BEM w/FMP
I am usually working onI am usually working onsolving Ax=b !!!solving Ax=b !!!
33
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Gaussian Elimination/LU Factorization.Gaussian Elimination/LU Factorization. compute compute AA-1-1 directly. directly.
Robust for wide range of applications.Robust for wide range of applications. Good for both dense and sparse matricesGood for both dense and sparse matrices
More expensive than iterative methods (memory, CPU)More expensive than iterative methods (memory, CPU) Not suitable for parallel and vector computation due to its global operations.Not suitable for parallel and vector computation due to its global operations.
Direct MethodsDirect Methods
44
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Stationary Methods (SOR, Gauss-Seidel etc.) and Nonstationary Methods (CG, GMRES, BiCGSTAB etc.)Stationary Methods (SOR, Gauss-Seidel etc.) and Nonstationary Methods (CG, GMRES, BiCGSTAB etc.)
Less expensive than direct methods, especially in memory.Less expensive than direct methods, especially in memory. Suitable for parallel and vector computing.Suitable for parallel and vector computing.
Convergence strongly depends on problems, boundary conditions (condition number etc.)Convergence strongly depends on problems, boundary conditions (condition number etc.) Preconditioning is required.Preconditioning is required.
Iterative MethodsIterative Methods
55
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Convergence rate of iterative solvers strongly depends on the spectral properties (eigenvalue distribution) of the coefficient matrix Convergence rate of iterative solvers strongly depends on the spectral properties (eigenvalue distribution) of the coefficient matrix AA. .
A preconditioner A preconditioner MM transforms the linear system into one with more favorable spectral properties transforms the linear system into one with more favorable spectral properties In "In "ill-conditionedill-conditioned" problems, "" problems, "condition numbercondition number" (ratio of max/min eigenvalue if " (ratio of max/min eigenvalue if AA is symmetric) is large. is symmetric) is large. MM transforms original equation transforms original equation Ax=bAx=b into into A'x=b'A'x=b' where where A'=MA'=M-1-1AA, , b'=Mb'=M-1-1bb
ILU (Incomplete LU Factorization) or IC (Incomplete Cholesky Factorization) are well-known preconditioners. ILU (Incomplete LU Factorization) or IC (Incomplete Cholesky Factorization) are well-known preconditioners.
Preconditioing for Iterative MethodsPreconditioing for Iterative Methods
66
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
1.0
10.0
100.0
1000.0
1 10 100 1000
SMP-Node #
GF
LO
PS
3D linear elastic problem for simple cubic geometry on Hitachi SR8000/MPP with 128 SMP nodes (1024 PEs) (not ES40, unfortunately).Block ICCG Solver. The largest problem size so far is 805,306,368 DOF.
Iterative method is the Iterative method is the ONLYONLY choice for large-scale parallel computing. choice for large-scale parallel computing.Problem specific preconditioning method is the most important issue Problem specific preconditioning method is the most important issue although traditional ILU(0)/IC(0) cover wide range of applications. although traditional ILU(0)/IC(0) cover wide range of applications.
Strategy in GeoFEMStrategy in GeoFEM
128 SMP nodes805,306,368 DOF335.2 GFLOPS
16 SMP nodes100,663,296 DOF42.4 GFLOPS
77
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Contact Problems in Simulations for Earthquake Generation Cycle by GeoFEM.Contact Problems in Simulations for Earthquake Generation Cycle by GeoFEM. Non-linearNon-linear Ill-conditioned problem due to penalty constraint by ALM (Augmented Lagrangean)Ill-conditioned problem due to penalty constraint by ALM (Augmented Lagrangean) AssumptionsAssumptions
Infinitesimal deformation, static contact relationship.Infinitesimal deformation, static contact relationship. Location of nodes is in each "contact pair" is identical.Location of nodes is in each "contact pair" is identical.
No friction : Symmetric coefficient matrixNo friction : Symmetric coefficient matrix
Topics in this PresentationTopics in this Presentation
Special preconditioning : Special preconditioning : Selective Blocking.Selective Blocking. provides robust and smooth convergence in 3D solid mechanics simulations for provides robust and smooth convergence in 3D solid mechanics simulations for
geophysics with contact.geophysics with contact.
Examples on Hitachi SR2201 parallel computer with 128 processing elements. Examples on Hitachi SR2201 parallel computer with 128 processing elements.
88
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
OVERVIEWOVERVIEWBackgroundBackgroundGeneral Remedy for Ill-Conditioned ProblemsGeneral Remedy for Ill-Conditioned Problems
Deep Fill-inDeep Fill-in BlockingBlocking
Special Method for Fault-Contact ProblemsSpecial Method for Fault-Contact Problems Selective BlockingSelective Blocking Special Repartitioning Special Repartitioning
ExamplesExamples Large Scale Computation on Hitachi SR2201 w/128 PEsLarge Scale Computation on Hitachi SR2201 w/128 PEs
SummarySummary
99
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Geophysics Application w/ContactGeophysics Application w/ContactAugmented Lagrangean Method with Penalty CoAugmented Lagrangean Method with Penalty Co
nstraint Condition for Contactnstraint Condition for Contact
Eurasia
Philippine
PacificEurasia
Philippine
Pacific
6,156 elements, 7,220 nodes, 21,660 DOF840km1020km600km region
1010
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
1
10
100
1000
10000
1.E+08 1.E+10 1.E+12 1.E+14
Penalty Number
Itera
tion
s
●● Newton Raphson iteration Newton Raphson iteration
▲▲ Solver iteration for entireSolver iteration for entire Newton Raphson iterationNewton Raphson iteration
■■ Solver iteration for Solver iteration for ONEONE Newton Raphson iterationNewton Raphson iteration
Large Penalty providesLarge Penalty provides ・・ Good N-R convergenceGood N-R convergence ・・ Large Condition NumberLarge Condition Number
Optimum ChoiceOptimum Choice
Augmented Lagrangean MethodAugmented Lagrangean MethodPenalty~Iteration Relation for Contact ProblemsPenalty~Iteration Relation for Contact Problems
Newton-Raphson / Iterative Solver Newton-Raphson / Iterative Solver
1111
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Block-type Preconditioning seems to work well for ill-conditioned casesBlock-type Preconditioning seems to work well for ill-conditioned cases
Results in the BenchmarkResults in the Benchmark7,220 nodes, 21,660 DOFs, 7,220 nodes, 21,660 DOFs, =10=10-8-8
GeoFEM's CG solver (scalar version)GeoFEM's CG solver (scalar version)Single PE caseSingle PE case
=1010
IC(0) : 89 iters, 8.9 sec. DIAG : 340 iters, 19.1 sec. Block LU scaling : 165 iters, 11.9 sec.
=1016
IC(0) : >10,000 iters, >1,300.0 sec. DIAG : No Convergence Block LU scaling : 3,727 iters, 268.9 sec.
1212
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
BackgroundGeneral Remedy for Ill-Conditioned ProblemsGeneral Remedy for Ill-Conditioned Problems
Deep Fill-inDeep Fill-in BlockingBlocking
Special Method for Fault-Contact Problems Selective Blocking Special Repartitioning
Examples Large Scale Computation on Hitachi SR2201 w/128 PEs
Summary
1313
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
The world where direct solvers have governed.The world where direct solvers have governed.But iterative methods are the only choice for large-scale massively parallel computation.But iterative methods are the only choice for large-scale massively parallel computation.We need robust preconditioning !!We need robust preconditioning !!
Remedy : Basically Preconditioning like Direct Solver Remedy : Basically Preconditioning like Direct Solver Deep Fill-inDeep Fill-in Blocking and OrderingBlocking and Ordering
Ill-Conditioned ProblemsIll-Conditioned Problems
1414
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Deep Fill-in : LU and ILU(0)/IC(0)Deep Fill-in : LU and ILU(0)/IC(0)Even if A is sparse, AEven if A is sparse, A-1-1 is not necessarily is not necessarily
sparse due to fill-in.sparse due to fill-in.
Gaussian Elimination do i= 2, n do k= 1, i-1 aik := aik/akk do j= k+1, n aij := aij - aik*akj enddo enddo enddo
ILU(0) : keep non-zero pattern of the original coefficient matrix do i= 2, n do k= 1, i-1 if ((i,k)∈ NonZero(A)) thenif ((i,k)∈ NonZero(A)) then aaik ik := a:= aikik/a/akkkk
endifendif do j= k+1, n if ((i,j)∈ NonZero(A)) thenif ((i,j)∈ NonZero(A)) then aaij ij := a:= aijij - a - aikik*a*akjkj
endifendif enddo enddo enddo
DEEP Fill-in
1515
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Deep Fill-in : ILU(p)/IC(p)Deep Fill-in : ILU(p)/IC(p)
LEVLEVijij=0 if ((i,j)∈ NonZero(A)) otherwise LEV=0 if ((i,j)∈ NonZero(A)) otherwise LEVijij= p+1= p+1 do i= 2, ndo i= 2, n do k= 1, i-1do k= 1, i-1 if (LEVif (LEVikik≦p) then≦p) then aaik ik := a:= aikik/a/akkkk
endifendif do j= k+1, ndo j= k+1, n if (LEVif (LEVij ij = min(LEV= min(LEVijij,1+LEV,1+LEVikik+ LEV+ LEVkjkj)≦p) then)≦p) then aaij ij := a:= aijij - a - aikik*a*akjkj
endifendif enddoenddo enddoenddo enddoenddo
DEEP Fill-in
1616
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Close to direct solver if you have DEEPER fill-in.Close to direct solver if you have DEEPER fill-in.
requires additional memory and computation.requires additional memory and computation. x2 for ILU(0) -> ILU(1)x2 for ILU(0) -> ILU(1)
Deep Fill-in : General IssuesDeep Fill-in : General Issues
DEEP Fill-in
1717
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Apply complete/full LU factorization in the certain size block for process DApply complete/full LU factorization in the certain size block for process D-1-1.. Just divided by diagonal component for scalar cases.Just divided by diagonal component for scalar cases.
3x3 block for 3D solid mechanics.3x3 block for 3D solid mechanics. tightly coupled 3-components (u-v-w) on 1-node.tightly coupled 3-components (u-v-w) on 1-node.
Blocking : Forward/Backward Blocking : Forward/Backward Substitution for ILU/IC ProcessSubstitution for ILU/IC Process
M= (L+D)D-1(D+U)
Forward Substitution (L+D)p= q : p= D-1(q-Lp)
Backward Substitution (I+ D-1 U)pnew= pold : p= p - D-1Up
BLOCKING
1818
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
333 Block ILU(0) Preconditioning3 Block ILU(0) PreconditioningForward SubstitutionForward Substitution do i= 1, N SW1= WW(3*i-2,ZP) SW2= WW(3*i-1,ZP) SW3= WW(3*i ,ZP) isL= INL(i-1)+1 ieL= INL(i) do j= isL, ieL k= IAL(j) X1= WW(3*k-2,ZP) X2= WW(3*k-1,ZP) X3= WW(3*k ,ZP) SW1= SW1 - AL(1,1,j)*X1 - AL(1,2,j)*X2 - AL(1,3,j)*X3 SW2= SW2 - AL(2,1,j)*X1 - AL(2,2,j)*X2 - AL(2,3,j)*X3 SW3= SW3 - AL(3,1,j)*X1 - AL(3,2,j)*X2 - AL(3,3,j)*X3 enddo X1= SW1 X2= SW2 X3= SW3 X2= X2 - ALU(2,1,i)*X1 X3= X3 - ALU(3,1,i)*X1 - ALU(3,2,i)*X2 X3= ALU(3,3,i)* X3 X2= ALU(2,2,i)*( X2 - ALU(2,3,i)*X3 ) X1= ALU(1,1,i)*( X1 - ALU(1,3,i)*X3 - ALU(1,2,i)*X2) WW(3*i-2,ZP)= X1 WW(3*i-1,ZP)= X2 WW(3*i ,ZP)= X3 enddo
Full LU FactorizationFull LU Factorizationfor 3x3 Blockfor 3x3 Block
DD-1-1
BLOCKING
1919
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Iteration number and computation time dramatically decreases by fill-in and blocking.Iteration number and computation time dramatically decreases by fill-in and blocking.
Benchmark : Benchmark : Effect of Fill-in/BlockingEffect of Fill-in/Blocking7,220 nodes, 21,660 DOFs, 7,220 nodes, 21,660 DOFs, =10=10-8-8
CG solver, Single PE caseCG solver, Single PE case
=1016
IC(0) : >10,000 iters, >1,300.0 sec. Block LU scaling : 3,727 iters, 268.9 sec. Block IC(0) : 1,102 iters, 144.3 sec. Block IC(1) : 94 iters, 21.1 sec. Block IC(2) : 33 iters, 15.4 sec.
DEEP Fill-in BLOCKING
2020
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
BackgroundGeneral Remedy for Ill-Conditioned Problems
Deep Fill-in Blocking
Special Method for Fault-Contact ProblemsSpecial Method for Fault-Contact Problems Selective BlockingSelective Blocking Special Repartitioning
Examples Large Scale Computation on Hitachi SR2201 w/128 PEs
Summary
2121
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.Selective BlockingSelective Blocking
Special Method for Contact ProblemSpecial Method for Contact ProblemStrongly coupled nodes are put into the same Strongly coupled nodes are put into the same
diagonal block.diagonal block.
1 2
5 6
9 10
13 14
17 18
25
29 30
26
21 22
1 2 3
10 11 12
19 20 21
28 29 30
37 38 39
46 47 48
55 56 57
64 65 66
73 74 75
82 83 84
91 92 93
ContactContactGroupsGroups
1 2
5 6
9 10
13 14
17 18
25
29 30
26
21 22
1 2 3
10 11 12
19 20 21
28 29 30
37 38 39
46 47 48
55 56 57
64 65 66
73 74 75
82 83 84
91 92 93
1 2
5 6
9 10
13 14
17 18
25
29 30
26
21 22
1 2 3
10 11 12
19 20 21
28 29 30
37 38 39
46 47 48
55 56 57
64 65 66
73 74 75
82 83 84
91 92 93
ContactContactGroupsGroups
2222
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.Selective BlockingSelective Blocking
Special Method for Contact ProblemSpecial Method for Contact ProblemStrongly coupled nodes are put into the same Strongly coupled nodes are put into the same
diagonal block.diagonal block.
Initial Coef. Matrixfind strongly coupled contact groups (each small square:3x3)
Reordered/Blocked Matrixnodes/block
Each block corresponds toa contact group
2323
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Block ILU/IC
Selective Blocking or SupernodeSelective Blocking or Supernode Procedure : Forward Substitution in Lower Tri. PartProcedure : Forward Substitution in Lower Tri. Part
Selective Blocking/Supernode
size of each diagonal block depends on contact group size
Apply full LU factorization for computation of D-1
2424
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Benchmark : Benchmark : SB-BIC(0)SB-BIC(0)Selective Blocking + Block IC(0)Selective Blocking + Block IC(0)
7,220 nodes, 21,660 DOFs, 7,220 nodes, 21,660 DOFs, =10=10-8-8
CG solver, Single PE caseCG solver, Single PE case
=1016
IC(0) : >10,000 iters, >1,300.0 sec. Block LU scaling : 3,727 iters, 268.9 sec. Block IC(0) : 1,102 iters, 144.3 sec. Block IC(1) : 94 iters, 21.1 sec. Block IC(2) : 33 iters, 15.4 sec. SB-Block IC(0) : 82 iters, 11.2 sec.
2525
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Benchmark : Selective BlockingBenchmark : Selective Blocking Selective Blocking converges even if Selective Blocking converges even if =10=102020
1.E-09
1.E-08
1.E-07
1.E-06
1.E-05
1.E-04
1.E-03
1.E-02
1.E-01
1.E+00
1.E+01
0 500 1000 1500 2000Iterations
BIC(1)
BIC(2)SB-BIC(0)
2626
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Benchmark : Benchmark : 4PE cases4PE cases 7,220 nodes, 21,660 DOFs, 7,220 nodes, 21,660 DOFs, =10=10-8-8
=10=101616 ,, CG solverCG solver
Single PE Block IC(0) : 1,102 iters, 144.3 sec. Block IC(1) : 94 iters, 21.1 sec. Block IC(2) : 33 iters, 15.4 sec. SB-BIC(0) : 82 iters, 11.2 sec.
4 PEs Block IC(0) : 2,104 iters, 68.4 sec. Block IC(1) : 1,724 iters, 85.8 sec. Block IC(2) : 962 iters, 69.9 sec. SB-BIC(0) : 1,740 iters, 70.0 sec.
In 4PE case, nodes in tightly connected groups are on different partition and decoupled.In 4PE case, nodes in tightly connected groups are on different partition and decoupled.
2727
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Deep fill-in, blocking and selective-blocking dramatically improve the convergence rate for ill-conditioned problems such as solid mechanics with contact.Deep fill-in, blocking and selective-blocking dramatically improve the convergence rate for ill-conditioned problems such as solid mechanics with contact.
But performance is bad in parallel cases with localized preconditioning when nodes in tightly connected pairs are on different partition and decoupled.But performance is bad in parallel cases with localized preconditioning when nodes in tightly connected pairs are on different partition and decoupled.
Special repartitioning method needed !!Special repartitioning method needed !!
SummarySummary
2828
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
BackgroundGeneral Remedy for Ill-Conditioned Problems
Deep Fill-in Blocking
Special Method for Fault-Contact Problems Selective Blocking Special RepartitioningSpecial Repartitioning
Examples Large Scale Computation on Hitachi SR2201 w/128 PEs
Summary
2929
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Outline of the RepartitioningOutline of the Repartitioning
BEFOREBEFORErepartitioningrepartitioning
Nodes in contact pairs Nodes in contact pairs are on separated are on separated partition.partition.
AFTER AFTER repartitioningrepartitioning
Nodes in contact pairs Nodes in contact pairs are on same partition, are on same partition, but no load-balancing.but no load-balancing.
AFTERAFTERload-balancingload-balancing
Nodes in contact pairs Nodes in contact pairs are on same partition, are on same partition, and load-balanced.and load-balanced.
Convergence is slow if nodes in each contact group locate on different partition.Repartitioning so that nodes in contact pairs would be in samepartition as INTERIOR nodes will be effective.
3030
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Special RepartitioningSpecial RepartitioningBenchmark: 4PE casesBenchmark: 4PE cases
Precond. Iter # sec. BIC(1) 1010 80 3.8
1016 167 7.4 BIC(2) 1010 71 5.8
1016 74 5.9 1020 No Conv. N/A
SB-BIC(0) 1010 126 2.9 1016 124 2.8 1020 231 5.7
Precond. Iter # sec.BIC(1) 1010 90 4.1
1016 1,724 70.7BIC(2) 1010 86 6.6
1016 962 59.81020 No Conv. N/A
SB-BIC(0) 1010 156 3.51016 1,598 33.91020 2,345 55.5
BEFORE Repartitioning AFTER Repartitioning
3131
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
BackgroundGeneral Remedy for Ill-Conditioned Problems
Deep Fill-in Blocking
Special Method for Fault-Contact Problems Selective Blocking Special Repartitioning
ExamplesExamples Large Scale Computation on Hitachi SR2201 w/128 PEsLarge Scale Computation on Hitachi SR2201 w/128 PEs
SummarySummary
3232
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Large-Scale ComputationLarge-Scale Computation DescriptionDescription
NX1 NX2N
Z1
NZ
2
NZ
1+N
Z2
x= 0
x= N
X1
x= N
X1+
1
x= N
X1+
NX
2+1
z= 0
z= NZ1
z= NZ1+1
z= NZ1+NZ2+1
x
y
z
3333
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Problem Setting & B.C.'sProblem Setting & B.C.'s
MPC at inter-zone boundariesMPC at inter-zone boundaries
Symmetric condition at Symmetric condition at x=0x=0 and and y=0y=0 surfaces surfaces
Dirichlet fixed condition at Dirichlet fixed condition at z=0z=0 surface surface
Uniform distributed load at Uniform distributed load at z= Zmaxz= Zmax surface surface
1 2
5 6
9 10
13 14
17 18
25
29 30
26
21 22
1 2 3
10 11 12
19 20 21
28 29 30
37 38 39
46 47 48
55 56 57
64 65 66
73 74 75
82 83 84
91 92 93
ContactContactGroupsGroups
1 2
5 6
9 10
13 14
17 18
25
29 30
26
21 22
1 2 3
10 11 12
19 20 21
28 29 30
37 38 39
46 47 48
55 56 57
64 65 66
73 74 75
82 83 84
91 92 93
1 2
5 6
9 10
13 14
17 18
25
29 30
26
21 22
1 2 3
10 11 12
19 20 21
28 29 30
37 38 39
46 47 48
55 56 57
64 65 66
73 74 75
82 83 84
91 92 93
ContactContactGroupsGroupsx
y
z
x
y
z
3434
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Sample Mesh Sample Mesh 99 nodes, 80 elements.99 nodes, 80 elements.
1 2
5 6
9 10
13 14
17 18
25
29 30
26
21 22
1 2 3
10 11 12
19 20 21
28 29 30
37 38 39
46 47 48
55 56 57
64 65 66
73 74 75
82 83 84
91 92 93
ContactContactGroupsGroups
3535
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Results on Hitachi SR2201 (128PEs)Results on Hitachi SR2201 (128PEs) NX1=NX2=70, NY=40, NZ1=NZ2=70, NX1=NX2=70, NY=40, NZ1=NZ2=70, Repartitioned.Repartitioned.
2,471,439 DOF, 784,000 Elements2,471,439 DOF, 784,000 ElementsIterations/CPU time until convergence (Iterations/CPU time until convergence (=10=10-8-8))
BIC(0)
/E 102
905 iters 194.5 sec.
104 106
> 8,300 > 1,800.0
108 1010
BIC(1)225 92.5
297 115.2
460165.6
BIC(2)183
139.3201
146.3296
187.7
SB-BIC(0)54269.5
54269.5
54269.5
54369.7
54469.8
3636
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Required MemoryRequired MemoryNX1=NX2=20, NY=20, NZ1=15, NZ2=16NX1=NX2=20, NY=20, NZ1=15, NZ2=16
83,649 DOF, 24,000 Elements83,649 DOF, 24,000 Elements
BIC(0) 105 BIC(0) 105 MBMB
BIC(1) 284 BIC(1) 284 MBMB
BIC(2) 484 BIC(2) 484 MBMB
SB-BIC(0) 128 SB-BIC(0) 128 MBMB
3737
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Concluding RemarksConcluding RemarksRobust Preconditioning Methods for Contact Problem.Robust Preconditioning Methods for Contact Problem. General: Deep Fill-in, Blocking.General: Deep Fill-in, Blocking. Problem Specific: Selective-Blocking using Supernodes.Problem Specific: Selective-Blocking using Supernodes. Large-Scale Problems using 128 PEs of Hitachi SR2201.Large-Scale Problems using 128 PEs of Hitachi SR2201.
Selective-Blocking (SB-BIC(0)) provides robust convergence.Selective-Blocking (SB-BIC(0)) provides robust convergence. More efficient and robust than BIC(0), BIC(1) and BIC(2). More efficient and robust than BIC(0), BIC(1) and BIC(2). Iteration number for convergence remains constant while Iteration number for convergence remains constant while increases. increases.
3838
3rd ACES Workshop, Maui, May5-10, 3rd ACES Workshop, Maui, May5-10, 2002.2002.
Further StudyFurther StudyOptimization for Earth Simulator.Optimization for Earth Simulator.Dynamic Update of Contact Information.Dynamic Update of Contact Information. Large Slip / Large Deformation. Large Slip / Large Deformation. More flexible and robust preconditioner under development such as SPAI (SpaMore flexible and robust preconditioner under development such as SPAI (Spa
rse Approximate Inverse). rse Approximate Inverse).