Principles of the Bayesian automatic adaptive...

Principles of the Principles of the BayesianBayesian automatic automatic adaptive adaptive quadraturequadrature

Gh. Adam, S. AdamLaboratory of Information Technologies

JINR-Dubna

Mathematical Modeling and Computational PhysicsDubna, July 7 - 11, 2009

OverviewOverviewStandard automatic adaptive quadrature (AAQ)When standard AAQ may failBayesian AAQA priori well-conditioning criteriaSubrange conditioning classesSubrange binary treePriority queue associated to binary treeEnhancing diagnostic quality through inheritance

Standard approach to automatic adaptive quadrature

Standard approach to automatic adaptive quadrature

systematically developed in QUADPACK, the de factostandard of one-dimensional numerical integration.See:• R.Piessens, E. deDoncker-Kapenga, C.W. Überhuber,

D.K. Kahaner, QUADPACK, A Subroutine Package for Automatic Integration, Springer, Berlin, 1983

• P.J. Davis, P. Rabinowitz, Methods of Numerical Integration, Academic, NY, 1984

• A.R. Krommer, C.W. Ueberhuber, Computational Integration, SIAM, Philadelphia, 1998

Automatic domain discretization into subrangesAutomatic domain discretization into subranges

We ask for the numerical solution of a given the proper (or improper) Riemann integral within input accuracy specifications {εεrr , , εεaa},

i.e., we seek a globally adaptive numerical solution {QQN N , EENN >0}.The computation generates an automatic discretizationautomatic discretization of the integration domain [a, ba, b] into subrangessubranges.At a given subrange discretization the termination criterion is checked,

and the discretization procedure is iterated until either this criterion is satisfied, or an abnormal termination diagnostic is issued.

The automatic discretization procedure is adaptiveadaptive.By means of a subrangesubrange subdivision subdivision strategystrategy, it results into smaller width subranges around the difficult spots of the integrand, while into larger width subranges over the regions of slowly varying integrand.

Over each subrange, local quadrature ruleslocal quadrature rules, yielding local pairs {qqi i , e, eii}, provide numerical approximations of the integral.

Adaptive subrange subdivisionAdaptive subrange subdivision

Local quadra-

turesum

Localerroresti-mate

The local quadrature rule {qi , ei}The local quadrature rule {qqi i ,, eeii}

Global quantities {QN , EN}Global quantities {QQN N ,, EENN}

OverviewOverviewStandard automatic adaptive quadrature (AAQ)

When standard AAQ may failBayesian AAQA priori well-conditioning criteriaSubrange conditioning classesSubrange binary treePriority queue associated to binary treeEnhancing diagnostic quality through inheritance

When standard AAQ may failWhen standard AAQ may failJ.N.Lyness, When not to use an automatic quadrature

routine, SIAM Review, 2525, 63-87 (1983),considered case study integrals with oscillatory integrands with jumps, which were shown to sometimes converge to wrong values with false convergence diagnostics.

Gh. Adam, Case studies in the numerical solution of oscillatory integrals, Romanian J. Phys., 3838, 527-538 (1993),showed that QUADPACK case studies wrongly decide to activate extrapolation procedure and fail with diagnostics of safe output !?!

Computation of parametric integrals, arising in numerical simulations of properties of physical systems with variable parameters – temperature, pressure, e.m. external fields, composition (doping), deformation, etc., were found to yield wrong output, which cannot be detected within the AAQ code.

The computational side of the problem:

Library codes are tailored such as to solve effectivelyspecific classesspecific classes of integrands.Each code has inner limits of validity. It is user’s task to choose the suitable library code from a given menu.There are problems for which it is impossible to choose in advancein advance the right code.The trial and errortrial and error approach resulted in a high rate of spurious outputs carrying the diagnostic of being correct.Therefore, a study aimed at identifying automatically thefailure originsfailure origins is highly desirable.

Bayesian automatic globally adaptive quadratureBayesian automatic globally adaptive quadrature

PurposePurposeTo avoid, as much as possible, the use of unchecked input in the local quadraturerules.Each step towards the final solution is accepted only under fulfilment of specific consistency (well-conditioning) conditions.We abandon the universal belief that integrandvalues are to be computed exclusively atquadrature knots.Reliability of the analysis heavily depends onproper input of two machine numbers withinthe safe machine range:ε0 - the largest relative spacingu - the smallest positive magnitude

ApproachApproach

Unusualfeature

Unusualfeature

Machinedependentconstants

Machinedependentconstants

Definition: INTEGRAND PROFILEDefinition: INTEGRAND PROFILE

o The integrand profile consists of the set of all currently computed and inherited integrand values over the current subrange, at the current quadratureknots, respectively the inherited abscissas from theancestor subranges.

o The conditioning criteria are intended to result in sufficiently resolved integrand profiles.Then all the essential details of the integrand variation are present in the sampling derived over the current set of quadrature knots, with the consequence that thereliability of the local quadrature rules is significantly enhanced.

Global consistency criterion for interpolatory quadrature rules

Global consistency criterion for interpolatory quadrature rules

o Local interpolatory quadrature rules using 2n+1 quadrature knots:Gauss-Kronrod 10-21 rule (GK 10-21) and

Clenshaw-Curtis 16-32 rule (CC 16-32).o Each interpolatory polynomial results in a specific algebraic degree

of precision: GK 10-21: d = 3n+1; CC 16-32: d = 2n+2.o Given the inherited integrand values at subrange ends and the newly

computed value at subrange center, the integrand variation following from these data should be upper bounded by smooth function threshold inferences following from the interpolatory polynomial.

o Characteristic subrange lengths:~1, the allowed integrand variation should not exceed that of xd

≤ ε01/2 , the linear Taylor series expansion is valid for smooth f(x)

>> 1, the allowed reliable variation should be close to the linear oneintermediate, linear interpolations in log-log scale provides bounds.

Features of quadrature knot distributionFeatures of quadrature knot distribution

o GK 10-21 is spanned by orthogonal Legendre polynomials CC 16-32 is spanned by orthogonal Chebyshev polynomials

o Both reduce the integration range to [-1, 1].o Both are symmetric rules, i.e., reduced knots satisfy y-n = - yn

o Consider, e.g., the negative wing of GK 10-21: → within [-1,- 0.9) there are four knots→ within [- 0.9,- 0.5) there are four knots→ within [- 0.5, 0] there are four knots═> the knots are denser to the interval ends

and they are sparser around the centre.Inter-knot distances towards -1 are 0.004 then 0.020Near centre, ν = 0.149 defines the knot distribution norm.

We therefore split [-1, 0] = [-1,- 0.9) U [- 0.9,- 0.5) U [- 0.5, 0]and make the analysis separately over each subrange.

Nyquist-Shannon sampling theorem Nyquist-Shannon sampling theorem

o The reconstruction of the original from a discrete sampling cannot confidently reproduce details of the original over distances smaller than the norm ν of the sampling.

o For Bayesian AAQ, two kinds of well-conditioning criteria result:→ GLOBAL – upper bounds to the maximum admissible numbers

of extrema of f(x) over finite ranges (e.g., [2/ν] over [-1, 1]).→ LOCAL – lowest bounds for the allowed distance between two

successive extrema within any sampling of f(x) (e.g., it cannot be smaller than (2/3) ν over [-1, 1]).

o Illustration for GK 10-21:Over ranges [-1,- 0.9) and [- 0.9,- 0.5) local criteria are checkedOver ranges [-1,- 0.5) and [- 0.5, 0] global criteria are checked

Behaviour near subrange endsBehaviour near subrange ends

o Since the subrange ends and the nearest and next-nearest quadrature knots are very close to each other, two-point and three-point approximations of the first order derivativeof f(x) at the subrange ends should agree with each otherwithin a few percent provided f(x) is smooth.

o Moreover, the curvature of f(x) over the range [-1,- 0.9)should keep constant sign.

o The diagnostic might point to the need of investigating theoccurrence of a boundary layer problem, i.e., to the needto precisely define the integrand behaviour within an infinitesimal/mesoscopic neighbourhood of a subrange end.

Computing System Conformity to IEEE 754 Standard

Computing System Conformity to IEEE 754 Standard

The IEEE 754 Standard governs binary floating point arithmetic (number formats, basic operations, conversions, and exceptional conditions).

Hardware and Software conformity to the standard allows doing a reliable and portable analysis.

Critical points where we have found deviations from the standard:

i. Length of the significand;ii. Floating point comparison operations;iii. Unpredictable effects following from compiler code

optimization;iv. Underflow threshold.

Length of SignificandLength of

Significand

ε0 = 2-52

is right !ε0 = 2-52

is right !

ε0 = 2-53

is wrong !ε0 = 2-53

is wrong !

The boundary layer problem asks for accurate and reproduciblediagnostics of integrand f(x) behaviour at the endpoints a or bof a finite integration domain [a,b].A few critical issues dramatically influence code robustnessand reliability.(i) Definition of smooth behaviour from an unrestricted least squares analysis over mesoscopicmesoscopic neighbourhoodsneighbourhoods of the endpoints a and b. This secures the derivation of continuity criteria which are valid everywherevalid everywhere over the mesoscopic range where the analysis is done.(ii) Formulation of qualitative diagnostic criteria which reliablyreliablysingle out the various kinds of integrand behaviour even under severely damaged accuracy of the computed data.(iii) Derivation of accurate tests for reliable definition of the machine epsilon with respect to the addition.(iv) Identification of critical hardware and software featureswhich could spoil correctness of the diagnostics by deviation from the IEEE 754 standard and code reformulationcode reformulation such as to become insensitive to such drawback of the computing environment.

Boundary layer problem diagnostics Boundary layer problem diagnostics

Long monotonicity subrangesLong monotonicity subranges

o A monotonicity subrange is said to be long if it extends overat least five successive abscissas of integrand sampling.

o The ends of the monotonicity subrange correspond to isolated lateral integrand extrema (ILIE).

o At each ILIE, a smooth function satisfies two consistencycriteria:→ The fine scale approximation of the lateral 1-st order

derivative is closer to zero than its coarse scaleapproximation.

→ Around extremum the curvature keeps constant sign.o Over inner intervals of a long monotonicity subrange, the

ratios of successive divided differences of f(x) cannot exceed a certain threshold that depends on the presenceor absence of an inflection point.

Class: UNDEFINED STATEClass: UNDEFINED STATE

o Definition: Undefined state occurs in two cases:(1) No information on the integrand values is available.(2) Conditioning check has met a criterion pointing to an

insufficiently resolved integrand profile.o Actions:

→ Define descendent subranges by symmetric bisection. → Define descendent inheritances from the parent

subrange.→ Update subrange lists.→ Update the priority queue the root of which defines

the next analyzed subrange.

Class: ILL-CONDITIONED STATEClass: ILL-CONDITIONED STATE

o The scaling law principle: The conditioning diagnostic of agenuinely resolved integrand profile remains invariant undera change of scale of the integrand sampling.

o Definition: Ill-conditioned state: Conditioning check has met a criterion pointing to an integrand ill-behaviour at aninner point xs inside the current subrange.

o Actions:→ Define xs to machine accuracy and lateral limits of f(xs)

and f ′(xs) (Boundary layer problem).→ Move singularity at the endpoints of descendent

subranges, defining them by asymmetric bisection.→ Define descendent inheritances from parent subrange.→ Update subrange lists.→ Update the priority queue the root of which defines


Class: WELL-CONDITIONED STATEClass: WELL-CONDITIONED STATE

o Definition: Well-conditioned state: (1) All conditioning check criteria ended successfully for undefined state inputOR (2) the root of priority queue points to a subrange in

well-conditioned state.o Actions:

→ IfIf (2) thenthen= subdivide current subrange by symmetric bisection.

→ Activate local quadrature rules over current subrange(s)→ Update QN , EN .→ Check for end of computations (eoc).→ IfIf (.NOT. eoc) thenthen

= Update subrange lists.= Update the priority queue the root of which defines


Class: STATE WITH ENDPOINT SINGULARITYClass: STATE WITH ENDPOINT SINGULARITY

o Definition: Endpoint singularity: (1) May arise from solution of a boundary layer problem for an ill-conditioned interval

OR (2) the root of priority queue pointed to a subrange in a state with endpoint singularity.

o Actions:→ IfIf (2) thenthen

= Activate extrapolation procedure= Check for end of computations (eoc).= IfIf (.NOT. eoc) thenthen

¤ subdivide current subrange by symmetric bisection.¤ activate local quadrature rules over current subranges¤ update QN , EN .

→ Update subrange lists.→ Update the priority queue the root of which defines


Binary tree ordering of subrangesBinary tree ordering of subranges

o The binary tree of the subranges is built recursively:→ Root initialization: the initial integration domain [a, b],

assumed to be in undefined state.→ Descendents, no leaves.

Only depths of terminal nodes are stored in a list.Left descendent depth is stored at parent’s siteRight descendent depth is stored at end of the list.

o Special lists provide random storage of:¤ subrange specifications,¤ local quadrature rule outputs,¤ pointer list to sequential subrange ordering.

Priority queue associated to binary treePriority queue associated to binary tree

o The priority queue (PQ) of standard AAQ uses a simple key

o Bayesian AAQ asks for a composite PQ key: → Magnitude of local quadrature error is the primary PQ.

It secures storage of largest local error range at root.For a subrange in undefined state, ei = oflow.

→ Depth of the terminal nodes is the secondary PQ key.If a smaller depth is detected in the depth list, then thecorresponding subrange is moved at the root, hence:

(1) systematic elimination of spurious well-conditioning diagnostics over large subranges;

(2) consistent extrapolation procedure activation.o Subranges the local quadrature errors of which fall below

the significance threshold are ruled out from the PQ.This keeps the PQ length to a minimal value.

Enhancing diagnostic quality through inheritanceEnhancing diagnostic quality through inheritance

o Within standard AAQ all information on integrand values computed over ancestor subranges is lost.

o Bayesian AAQ preserves parent’s inheritance. This addssupplemenary information to the current integrand profile,which increases resolution of local discretization.

o All conditioning criteria are implemented using bothcurrently computed and inherited information.

o Efficient storage, merging & use of inherited information are secured by two-side open ordered lists for abscissas and integrand values.

o Global storage of inherited data is secured in dynamic one-dimensional vectors.

Thank you for your attention !

Principles of the Bayesian automatic adaptive...

Documents

Transcript of Principles of the Bayesian automatic adaptive...