2018 NDIA GROUND VEHICLE SYSTEMS ENGINEERING AND TECHNOLOGY SYMPOSIUMModeling & Simulation, Testing and Validation (MSTV) Technical Session

August 7-9, 2018 – Novi, Michigan

QUANTUM ANNEALING FOR MOBILITY STUDIES:GO/NO-GO MAPS VIA QUANTUM-ASSISTED MACHINE LEARNING

Radu SerbanUniversity of Wisconsin – Madison

Madison, WI

Max Wilson, Marcello Benedetti, John Realpe-Gomez,Alejandro Perdomo-Ortiz, Andre Petukhov

NASA Ames Research CenterMoffett Field, CA

Paramsothy JayakumarU.S. Army RDECOM-TARDEC

Warren, MI

ABSTRACT

We present the results of an exploratory investigation of applying a hybrid quantum-classical archi-tecture to an off-road vehicle mobility problem, namely the generation of go/no-go maps posed as amachine learning problem.

The premise of this work rests on two observations. First, quantum computing allows in principlefor algorithms that provide a speedup over the best known classical counterparts. However, as it is tobe expected of such novel and complex tools (both hardware and algorithmic) at this early develop-mental stage, current quantum algorithms do not always perform well on real-world problems. Second,complex physics-based vehicle and terramechanics models and simulations, currently advocated forhigh-fidelity high-accuracy ground vehicle–terrain interaction analyses, pose significant computationalburden, especially when applied to mobility studies which may require numerous simulation runs.

We describe the Quantum-Assisted Helmholtz Machine formulation, suitable to be implementedon a quantum annealer such as the D-Wave 2000Q machine, discuss the high-performance classicalcomputing framework used to generate through simulation the training and test sets, and provide theresults of our investigations and analysis into the performance of the machine learning model and itspredictive capabilities for generating go/no-go mobility maps.

This work represents a contribution to an ongoing effort of exploring the applicability of the emergingfield of quantum computing to challenging engineering and scientific problems.

DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited. OPSEC#864

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

1 INTRODUCTION

The work presented in this paper is anchored intwo main observations. First, quantum comput-ing allows for algorithms that provide a speedupover the best known classical algorithms for thesame task. However, current implementations ofquantum algorithms do not always perform wellon real-world-scale problems, as is to be expectedof novel and complex tools at such an early stage ofdevelopment. Demonstrating the implementationand applicability of quantum-assisted algorithmson problems of engineering and scientific interestand scale serves two purposes: (i) guiding the de-velopment of the hardware; and (ii) establishingareas of application for future devices. While pre-dictions from various experts vary, it is expectedthat over the next decade quantum computers willbe at a stage where hybrid architectures (classi-cal algorithms with quantum-accelerated subrou-tines) will be useful in a variety of real-world prob-lems. Machine Learning (ML) is a particular areain computational science that is likely to benefitfrom these developments.

The second remark relates to high-fidelity mod-eling and simulation of ground vehicle – terraininteraction. Multibody system dynamics is nowa-days a mature and well-established field, widelyadopted for predictive simulations of mechanicalsystems in general and ground vehicles on hardsurface in particular. Simultaneously, terrain andsoil mechanics is at a stage where complex vehicle–terrain interactions can be resolved at high-fidelityand increasing accuracy. This resulted in a re-newed push to adopt these simulation techniquesand include them in operational tools for assessingmobility of off-road ground vehicles. As an exam-ple, we mention the efforts for the developmentof an updated NATO Reference Mobility Model(NRMM) [1]; namely the Next Generation NRMM(NG-NRMM), which will rely on full 3-D multi-body vehicle models and physics-based terrame-

chanics models, while drawing on more powerfuland varied computer architectures [2].

The goal of this work was therefore an ex-ploratory investigation on the applicability ofquantum-assisted computing to the generation ofgo/no-go maps, cast as a machine learning prob-lem. Concretely, we investigate the applicabilityof Quantum Machine Learning (QML), using analgorithm possible to be implemented on a par-ticular instance of quantum computing hardware(quantum annealing as implemented on the D-Wave 2000Q platform [3]), to a problem of sig-nificant engineering and operational interest: off-road ground vehicle mobility. The definition of theparticular problem considered here and the overallsolution approach is delineated next. The ML ap-proach, the simulation infrastructure, and resultsare described in the subsequent sections, with fur-ther details available in [4].

While there is no general agreement on the exactdefinition of ground vehicle mobility, the consen-sus is that the maximal speed-made-good is a goodmeasure for mobility [1]. Herein, speed-made-goodis defined as the ratio of the straight-line distancebetween two points and the time required to travelbetween them, regardless of the actual path. Thismeasure of mobility depends in a highly-nonlinearfashion on many parameters, among which wemention: terrain topology, profile, and roughness;soil type, properties, and moisture content; me-teorological conditions; and, finally, vehicle type,configuration, and characteristics.

Under the definition adopted here, a go/no-gomap can thus be obtained by color-coding a par-ticular geographical area as function of the cor-responding mobility measure (either a continuousvariable such as the speed-made-good or a binaryoutput indicating whether or not the destinationpoint can be reached). Since terrain variabilityand heterogeneity can be very large even for rel-atively confined geographical regions, a pragmaticapproach consists of discretizing the area in a grid,

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 2 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

not necessarily uniform, such that input parame-ters (e.g., terrain characteristics and soil proper-ties) can be considered constant within a grid cell.Then, the go/no-go map is obtained by estimat-ing a mobility measure for each cell and appropri-ately color-coding the grid.

Generation of such a go/no-go map can bebased on empirical data or on simulation resultsof varying fidelity. On the one hand, empiricaldata alone will lack accuracy and predictive power.On the other hand, a simulation-based approachcan be prohibitively intensive in terms of com-putational cost depending on the level of fidelityadopted for modeling soil and/or vehicles, render-ing this approach unfeasible for operational pur-poses.

The machine learning aspect. A solution tothe potentially prohibitive computational cost ofmobility analysis based on simulations involvinghigh-fidelity vehicle multibody systems and com-plex terramechanics is framing the generation ofgo/no-go maps as a machine learning problem.This can take the form of either a classificationproblem; i.e., the problem of deciding to which la-bel – go or no-go – a given set of observations(input data; e.g., terrain profile, soil properties,vehicle type) belongs, or even more generally, asan unsupervised learning problem, where the goalis to infer/learn correlations among all variablesinvolved, including both input data and labels.

The quantum computing aspect. Quantumcomputers are designed to manipulate vectors andtensor products in high-dimensional spaces. Asa consequence, algorithms for QML can poten-tially provide exponential speed-ups for a widerange of machine learning tasks [5]. Much of therecord-breaking performance of classical machine-learning algorithms regularly reported in the lit-erature pertains to task-specific supervised learn-ing algorithms. On the other hand, unsupervisedlearning algorithms are more general and more

‘human-like’, but their development has been lag-ging behind due to their intractability. This in-tractability may come from the combinatorial na-ture of the problem (e.g. in clustering), or it maycome from the need to sample complex probabil-ity distributions (e.g. the Boltzmann distribution).Quantum annealing holds the potential to sam-ple more efficiently than classical Markov ChainMonte Carlo methods, and this could in turn sig-nificantly advance the field of unsupervised learn-ing.

The hybrid computing aspect. The overallmethodology envisioned and adopted for explo-ration herein was therefore a hybrid approach inwhich quantum computing was used to train amachine learning model using classical computing-based simulation to generate the training and testdata.

This paper is organized as follows. In Section 2 weprovide an overview of the machine learning modelused in this study, namely the Quantum-AssistedHelmholtz Machine. Section 3 provides a brief de-scription of the vehicle, terrain, and vehicle-groundinteraction models and simulation setup. Variousalgorithmic investigations, focusing on the charac-teristics of the problem at hand and their impacton model performance and predictive capabilitiesare discussed in section 4. We conclude the paperwith final remarks and suggestions for future workin section 5.

2 QUANTUM-ASSISTED MACHINELEARNING

The Quantum-Assisted Helmholtz Machine(QAHM) [6] is a concrete proposal that canexploit the sampling power of quantum annealingto learn a complex probability distribution overcontinuous variables.

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 3 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

2.1 Model and learning algorithm

Consider a dataset S = {v1, . . . ,vd} withempirical distribution QS(v). We seek agenerative model P (v) =

∑u P (u,v), where

P (u,v) = P (v|u)PQC(u). The prior distributionPQC(u) = 〈u|ρ|u〉 describes samples obtainedfrom a quantum device, and we assume it canbe described by a quantum Gibbs distributionρ = e−H/Z, where H is the Hamiltonian im-plemented in quantum hardware and Z is thepartition function. In the particular case ofquantum annealing hardware, we have

H =∑i<j

JijZiZj +∑i

hiZi + Γ∑i

Xi, (1)

where Zi and Xi denote Pauli matrices in the z andx direction, respectively, while Jij, hi, and Γ arecontrollable parameters. The conditional distribu-tion P (v|u) stochastically translates samples fromthe quantum computer into samples on the domainof the data. Because v is sampled indirectly fromhardware, it can be any type of data. In our case,it corresponds to a vector of continuous variables.

Ideally, an unsupervised learning algorithm forthis generative model would maximize the averagelog-likelihood of the data L =

∑vQS(v) lnP (v),

with respect to the controllable parameters. Un-fortunately this quantity is intractable because itrequires computation of the posterior P (u|v) forall possible u. The Helmholtz machine overcomesthis by introducing a lower bound∑

v

QS(v) lnP (v) ≥

∑v,u

QS(v)Q(u|v) lnP (v|u)PQC(u)

Q(u|v),

(2)

where Q(u|v) is an auxiliary model that approx-imates the intractable true posterior P (u|v). Inpractice, the newly introducedQ(u|v) and the con-ditional P (v|u) are implemented by classical arti-

ficial neural networks. From now on, we call prob-ability distribution P the generator network, andprobability distribution Q the recognition network.

Note that PQC(u) = ln〈u|ρ|u〉 is intractable dueto the projection of the Gibbs distribution on thestates |u〉. However, this term can be furtherbounded as

ln〈u|ρ|u〉 ≥ 〈u| ln ρ|u.〉 (3)

Combining Eqs. (2) and (3), we get a tractablelower bound for the generator to maximize

G(θG, θQC) =∑v

QS(v)∑u

Q(u|v) [lnP (v|u) + 〈u| ln ρ|u〉] ,

(4)

where θG and θQC denote the parameters of gener-ator network P and quantum state ρ, respectively,and where. In Eq. (4) we neglected terms that donot depend on either θG or θQC , as they vanishwhen computing the gradient of G.

Now we consider the recognition networkQ(u|v), which has to closely track the true poste-rior during learning. The maximization of Eq. (2)with respect to the parameters of the recognitionnetwork is also intractable. Reference [7] intro-duced the wake-sleep algorithm which minimizesthe more tractable Kullback-Leibler divergence

DKL [P (u|v)||Q(u|v)] =∑u

P (u|v) lnP (u|v)

Q(u|v),

averaged over the marginal P (v) to take into ac-count the relevance of each configuration v. Inother words, the reconstruction network maxi-mizes function

R(θR) =∑u

PQC(u)∑v

P (v|u) lnQ(u|v), (5)

where θR denotes, collectively, the parameters ofthe recognition network Q. In Eq. (5) we neglected

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 4 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

terms that do not depend on θR, as they vanishwhen computing the gradient of R.

The gradient ascent equations have structureθ(t+1) = (1− ηλ)θ(t) + η∇θF , where θ stands forthe parameters being updated, η is the learningrate, λ is a regularization factor, and F stands foreither G or R, accordingly. The type of regular-ization used here is called weight decay and helpsin preventing overfitting by keeping the weightssmall.

Since ln ρ = −H− lnZ, for the parameters ofthe quantum distribution θQC = (Jij, hi) we have

− ∂G∂Jij

= 〈uiuj〉Q − 〈uiuj〉ρ, (6)

− ∂G∂hi

= 〈ui〉Q − 〈ui〉ρ, (7)

where 〈 〉Q and 〈 〉ρ denote expectation values withrespect to Q(u|v)QS(v) and PQC(u) = 〈u|ρ|u〉,respectively. Here we have used the propertyZi|ui〉 = ui|ui〉.

The generation and recognition networks can bewritten as deep learning architectures

P (v|u) =∑

u1,...,uL

P0(v|u1)P1(u1|u2) · · ·PL(uL|u)

Q(u|v) =∑

u1,...,uL

QL(u|uL) · · ·Q1(u2|u1)Q0(u1|v),

in terms of L additional sets of hidden variablesu1, . . . ,uL that connect the variables v ≡ u0 in thevisible layer with u ≡ uL+1 in the last hidden layer.More specifically, when using Bernoulli variablesu`i ∈ {−1,+1}, we have

P`(u`|u`+1) =

∏i

π(u`i |u`+1;A`, a`), (8)

Q`(u`|u`−1) =

∏i

π(u`i |u`−1;B`, b`), (9)

where π(ui|u′;C, c) =[1 + e−2ui(

∑j Ciju

′j+ci)

]−1

.

The gradients for the generative network are

∂G∂A`ij

= 〈u`iu`+1j 〉Q − 〈u`i〉P 〈u`+1

j 〉Q,

∂G∂a`i

= 〈u`i〉Q − 〈u`i〉P ,

and similarly for the recognition network

∂R∂B`

ij

= 〈u`iu`−1j 〉P − 〈u`i〉Q〈u`−1

j 〉P ,

∂R∂b`i

= 〈u`i〉P − 〈u`i〉Q.

2.2 Implementation

We now discuss our implementation of the QAHMfor the D-Wave 2000Q quantum annealer. Theannealer implements a noisy version of the pro-grammed Hamiltonian in Eq. (1) defined on asparse graph of qubit interactions. The devicesamples low energy states in the limit Γ→ 0, non-trivial non-equilibrium effects may make samplesdeviate from the corresponding classical Gibbs dis-tribution. Several features of the algorithm are en-gineered to cope with these effects.

To overcome this we follow the work in [8]. Bytreating the annealer as grey-box, this approach al-lows updating the parameters without the need toestimate deviations from the Gibbs distribution.It also allows us to implement a fully connectedprior distribution PQC(u), despite the actual phys-ical connectivity be sparse.

For implementing continuous variables, when us-ing the recognition network, each variable vi inthe dataset is rescaled to lie in [−1,+1] and inter-preted as the expected value of a binary variable in{−1,+1}, i.e. the average value of that variable ifevaluated many times. Conversely, the probabili-ties in output from the generator network are usedto compute the expected value of a binary variable,rather than to actually sample the binary variable.

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 5 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Such a value can then be scaled back to the originalrange of the continuous variable. These techniquesallow to use the model without further modifica-tions and without the need of additional parame-ters.

Further details of the physical implementationcan be found in [4].

3 SIMULATION FOR MOBILITY AS-SESSMENT

Generation of the training and test sets wereperformed using the Chrono software suite [9].Chrono’s strength lies in its ability to simulate thedynamics of large multibody systems [10], includ-ing discipline-specific support for vehicle modelingand simulation and soil/terrain modeling and sim-ulation. In particular, deformable terrains can besimulated using a fully-resolved Discrete ElementMethod (DEM) approach, by modeling the soil asa large system of bodies interacting through con-tact, friction, and cohesion (albeit, not necessarilyat the physical soil particle size). Alternatively,Chrono provides support for more expeditious de-formable soil representation, such as the Soil Con-tact Model (SCM) [11].

The Chrono::Vehicle module provides supportfor template-based modeling and simulation ofground vehicles, both wheeled and tracked, thatcan be fully and implicitly coupled with theterrain/soil models mentioned above. As such,Chrono can be used to generate the input datasets required for the learning algorithms consid-ered here: for a given set of parameters, a coupledvehicle-terrain interaction simulation is conductedand results processed to obtain the desired mobil-ity measure (speed-made-good or simply a binarygo/no-go decision variable).

3.1 Vehicle modeling

Chrono::Vehicle [12] is a module of the open-

source multi-physics simulation package Chrono,aimed at modeling, simulation, and visualizationof wheeled and tracked ground vehicle multi-bodysystems. Its software architecture and design wasdictated by the desire to provide an expeditiousand user-friendly mechanism for assembling com-plex vehicle models, while leveraging the under-lying Chrono modeling and simulation capabili-ties, allowing seamless interfacing to other optionalChrono modules (e.g., its granular dynamics andfluid-solid interaction capabilities), and providinga modular and expressive API to facilitate its usein third-party applications. Vehicle models arespecified as a hierarchy of subsystems, each ofwhich is an instantiation of a predefined subsys-tem template. Written in C++, Chrono::Vehicleis offered as a middleware library.

Chrono::Vehicle provides a comprehensive setof vehicle subsystem templates (tire, suspension,steering mechanism, driveline, sprocket, trackshoe, etc.), templates for external systems (power-train, driver, terrain), and additional utility classesand functions for vehicle visualization, monitor-ing, and collection of simulation results. Threedifferent classes of tire models are supported:rigid (modeled as cylindrical shapes or else asnon-deformable triangular meshes), semi-empirical(Pacejka, Fiala, Lugre, and TMeasy), and finiteelement (based on ANCF or Reissner shell ele-ments). For additional flexibility and to allow inte-gration of third-party software, Chrono::Vehicle isdesigned to permit either monolithic simulations orco-simulation where the vehicle, powertrain, tires,driver, and terrain/soil can be simulated indepen-dently and simultaneously.

For the studies conducted herein, we used amodel of a four-wheel drive off-road vehicle withindependent double-wishbone suspension and Pit-man arm steering (see Fig. 1).

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 6 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Figure 1: Wheeled vehicle with double wishbonesuspensions and Pitman arm steering.

3.2 Granular terrain modeling

Chrono::Vehicle provides several classes of terrainand soil models, of different fidelity and compu-tational complexity, ranging from rigid, to semi-empirical Bekker-Wong type models, to complexphysics-based models based on either a granular orfinite-element based soil representation. For sim-ple terramechanics simulations, Chrono::Vehicleprovides a customized implementation of the SoilContact Model, based on Bekker theory, with ex-tensions to allow non-structured triangular gridsand adaptive mesh refinement. Second, Chronoprovides an FEA continuum soil model basedon multiplicative plasticity theory with Drucker-Prager failure criterion and a specialized 9-nodebrick element which alleviates locking issues withstandard brick elements. Finally, leveraging theChrono::Granular module and support for multi-core and distributed parallel computing in Chrono,off-road vehicle simulations can be conducted usingfully-resolved, granular dynamics-based complexterramechanics, using a Discrete Element Method(DEM) approach. Such simulations can use eitherof the two methods supported in Chrono, namelya penalty-based, compliant-body approach, or a

complementarity-based, rigid-body approach.

Two alternative approaches have emerged as vi-able solutions for large frictional contact problemsin granular flow dynamics and quasi-static geome-chanics applications, collectively termed hereinDEM. The so-called complementarity method(DEM-C) is generally favored within the multi-body dynamics community. In this approach, in-dividual particles in a bulk granular material aremodeled as rigid bodies, and non-penetration con-ditions are written as complementarity equationswhich, in conjunction with a Coulomb friction law,lead to a Differential Variational Inequality (DVI)form of the Newton-Euler equations of motion.Not limited by stability considerations, DEM-C al-lows for much larger time integration steps thanthe alternative penalty-based (DEM-P) solutions,since the latter involve large contact stiffnessesthat impose strict stability conditions on all ex-plicit time integration algorithms. However, DEM-C involves a relatively complex and computation-ally costly solution sequence per time step, sinceit leads to a mathematical program with comple-mentarity and equality constraints, which must berelaxed to obtain tractable linear complementar-ity or cone complementarity problems. More ma-ture and widely adopted within the geomechanicscommunity [13], DEM-P can be viewed either as aregularization (or smoothing) approach, which re-lies on a relaxation of the rigid-body assumption,or as a deformable-body approach localized to thepoints of contact between individual particles in abulk granular material [14]. In this approach, nor-mal and tangential contact forces are calculatedusing various laws [15], which are based on the lo-cal body deformation at the point of contact.

A more in depth comparison between the rigid-body (DEM-C) and soft-body (DEM-P) formu-lations is provided in [16]. All granular terrainsimulations used herein relied on the DEM-C for-mulation. For details on the DEM-C formulation,derivation of the DVI problem, and Chrono imple-

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 7 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

mentation, the reader is directed to [17, 18, 10, 4].Here, we only provide the resulting mixed differ-

ential algebraic–differential variational inequalityproblem in Eq. (10).

q = L(q)v (10a)

M(q)v = f (t,q,v)− gTq (q, t)λ +

∑i∈A(q,δ)

(γi,n Di,n + γi,u Di,u + γi,w Di,w)︸ ︷︷ ︸ithfrictional contact force

(10b)

0 = g(q, t) (10c)

i ∈ A(q(t)) :

0 ≤ Φi(q) ⊥ γi,n ≥ 0(γi,u, γi,w) = argmin√

γ2i,u+γ2i,w≤µiγi,n

vT · (γi,u Di,u + γi,w Di,w) . (10d)

These represent the equations of motion for amultibody system involving both bilateral (equal-ity) constraints and unilateral (frictional contact)constraints. The differential equations in Eq. (10a)relate the time derivative of the generalized posi-tions q and velocities v through a linear trans-formation defined by L(q). The force balance inEq. (10b) ties the inertial forces to the applied andconstraint forces, f (t,q,v) and−gT

q (q, t)λ, respec-tively. The latter are impressed by the bilateralconstraints of Eq. (10c) that restrict the relativemotion of the rigid or flexible bodies present inthe system. The non-penetration conditions inEq. (10d) express the complementarity betweenthe separation function Φi for contact i and theassociated normal contact force. The last equa-tion poses an optimization problem whose first or-der Karush-Kuhn-Tucker optimality conditions areequivalent to the Coulomb dry friction model. Thefrictional contact force associated with contact ileads to a set of generalized forces, shown in red inEq. (10b), which are obtained using the projectorsDi,n, Di,u, and Di,w [18].

The problem in Eq. (10) is discretized to yield amathematical program with complementarity andequality constraints. A relaxation of the comple-

mentarity conditions and further algebraic manip-ulations [17] yield a cone complementarity problem(CCP) whose solution provides the Lagrange mul-tipliers λ and γ.

3.3 Computational and simulation setup

All simulations were conducted on a Cray XC30system with 12-core IntelR© XeonR© E5-2697 v2 pro-cessors and a dedicated Cray Aries high-speed net-work. Independent runs were launched in batchescorresponding to the subsets of training pointsrequested by the sequential approach describedpreviously. Each separate run (corresponding toa particular set of the five independent designparameters considered, namely longitudinal ter-rain slope, particle radius, particle density, inter-particle coefficient of friction, and cohesion pres-sure) was queued on a single node and used 24OpenMP threads.

To accelerate time to solution, simulations con-ducted for this study employed the moving patchfeature provided within Chrono::Vehicle. In thisapproach, granular material is simulated onlywithin a sliding window (of user-defined dimen-sions) centered around and moving with the ve-

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 8 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

hicle. The number of granular material particlesremain constant (and relatively low). Particlesfalling outside the moving patch behind the vehicleare reused by relocating them in front of the ve-hicle in chunks of spatial dimensions controlled bythe user. With this approach, depending on thecurrent particle size, the simulations used hereinemployed between 48,000 and 480,000 particles.

Each simulation run represented a straight-lineacceleration maneuver on longitudinally inclinedterrain. The granular material was consideredhomogeneous and consisting of identical spheres.Particles were initialized in layers (with a numberof layers determined dynamically, function of theparticle radius) and using a uniform random dis-tribution within each layer obtained with a Pois-son disk sampling technique. Following a shortgranular material settling phase, the vehicle is cre-ated above the terrain and allowed to settle. Sub-sequently, throttle is increased from 0% to 100%over the span of 0.5 s, while the gravitational ac-celeration vector is rotated by the appropriate an-gle to model the incline plane. To maintain astraight-line, a path-follower steering controller isused which makes minute adjustments to the ve-hicle steering input. Several heuristics are imple-mented to decide completion of the simulated ma-neuver; tracking running averages of the vehicleforward velocity and acceleration, simulations arestopped when a steady-state maximum velocity isachieved or when the case of the vehicle slidingbackward is identified.

During simulation, we record relevant vehiclestates for each individual run. In a post-processingstage, information from these output files is col-lated to generate the incremental training set forthe sequential QML algorithm, as well as statisticsfor estimating computational performance. Ad-ditional information pertaining to computationalperformance is provided in [4].

4 ALGORITHMIC INVESTIGATIONS

The hybrid quantum-classical computational ar-chitecture, described in Sections 2 and 3 and illus-trated in Fig. 2, was used for generating and learn-ing the distribution of the simulation data. We de-scribe here the implementation of the QAHM ondatasets provided through simulations and exam-ine relevant characteristics of both the dataset andthe model. We highlight and discuss two salientfeatures of the model and draw a strong link be-tween these features and the model effectiveness,using as a performance metric the mean squarederror between the predictions of the model and anevaluation (test) set.

The training and test sets contain 320 and 40points, respectively, sampled from the 6-D param-eter space (5 inputs and 1 output). The majorityof the networks used for these demonstrations con-sist of 2 hidden layers comprising 12 nodes each,and a visible 6-node layer. Several experimentswere needed in order to choose the hyperparame-ters. A learning rate η = 0.01 and a regularizationfactor λ = 0.0001 were found to produce best re-sults. The ranges used for the normalization of theinput parameters were slope = [0.0, 20.0], radius= [8.0, 18.0], density = [1000.0, 2000.0], friction =[0.6, 1.0], cohesion = [500.0, 1500.0] and velocity= [-5.0, 40.0].

4.1 Training and test data

There are three key features of the data which areimportant to these investigations. First, the size ofthe initial training set is 320 points; although neu-ral network machine learning algorithms are noto-riously data intensive, expanding the dataset avail-able was limited by the computational resource re-quired. Second, the points in the training set andthe test set were both distributed regularly, us-ing Latin Hypercube Sampling (LHS), over the 5-D input parameter space. Lastly, the distributionof the velocities is roughly bimodal. As shown in

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 9 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Figure 2: Pipeline for learning simulation datasets with quantum assisted models.

Fig. 3, the final velocity of the simulation tendsto fall into one of two modes, high (20 . . . 34 m/s)or low (−5 . . . 5 m/s), and is skewed to higher ve-locities; this can be explained in part by the LHSsampling used to generate the training set.

Figure 3: Distribution of final velocities in thetraining set (simulation results), showing its skewand bimodal nature.

In classical machine learning these characteris-tics of the training data can be addressed andaccommodated by selecting tailored algorithms.However, at this early stage of quantum machine

learning development, only few viable algorithmicoptions are available. As such, the dataset size lim-its the ability of the algorithm to effectively trainmodels with larger number of nodes (and hencethe number of parameters to be updated duringtraining); this situation can result in underfitting.

Furthermore, sampling the test and trainingsets using the same method resulted in undesiredcorrelations between the predictions on the twosets. However, LHS was a necessary compromisebetween the need for random sampling and theability to only provide a relatively small datasetthrough simulation; indeed, random sampling witha small number of points could have resulted insmall sample size bias.

4.2 Model fitting

Initial investigation into optimizing the model per-formance, primarily by varying the number ofnodes in an attempt to correctly fit the model, in-dicated that the network complexity (number ofhidden nodes) was not the main source of the ob-served systematic errors. As shown in Fig. 4, themodel performance – as measured by the meansquared error between predictions and data – im-proves with increased network complexity up to

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 10 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

about 10 nodes per hidden layer, but then reachesa noise floor around 60 (m/s)2. This is observed forboth the training and test sets, whereas typicallyone expects a monotonic evolution as the modeltends to overfitting.

Figure 4: Prediction accuracy as function of net-work complexity (number of nodes per hiddenlayer).

The source of these prediction errors can beidentified by looking at their distribution acrossgiven dataset. Figure 5 shows the mean squarederrors at the different velocities in the training setand highlights a squeezing phenomenon, whereasthe extremities of the velocity range in the datasetare outside the predictive bounds of the model; inother words, the model tends to generate a rangeof predictions that is squeezed relative to the datain the training set.

A second consequential effect of the larger errorsat the extremities of velocity range, coupled withthe mostly bimodal distribution of velocities in thetraining set, is a tendency of predictions to switchbetween these two modes. This effect, termed heremode-switching, negatively impacts the accuracyof samples from the model and contributed domi-nant anomalous errors to the resulting predictions.

Figure 5: Distribution of the errors in predictionsacross the training set, showing clear dominant er-rors at lower velocities and upticks in error at boththe lowest and highest velocities.

4.3 Model sampling

The model is sampled in order to make predictionsabout the velocity at a given point in the param-eter space. In order to mitigate systematic errorsdiscussed above, due to the squeezing and mode-switching phenomena, several sampling methodswere investigated. these are succinctly listed hereand described in more detail in [4].

Sampling 1: Markov chain sampling from thegenerative network. In this process, one step in theMarkov chain evolution consists of a bottom–uppass of the recognition network (with given inputparameters and randomized velocity), followed bya top–down pass of the generation network. Whilethe system does reach an equilibrium, the varianceof the produced distribution is so large that theresults resemble noise; see Fig. 6.

Sampling 2: Markov chain sampling with av-eraging. Different from the previous case, at eachstep of the Markov chain, the model is sampledmultiple times (e.g., 10 samples) and the resultsaveraged; see Fig. 7. While mitigating the largevariance observed with the pure Markov chain

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 11 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Figure 6: Markov chain evolution of the QAHM:samples from the device are used as measures ofthe predicted velocity.

sampling above, this averaging has the disadvan-tage of further squeezing the results away from theextreme values.

Sampling 3: Peak frequency sampling. In thisprocess, a number of samples (e.g.; 500) are takenat random velocities in the range given by thedataset, without evolving the system like in theprevious sampling methods. A histogram is thengenerated (using 40–50 bins) and the predictionis set to be the velocity corresponding to peakfrequency. This approach provides better resultsfor predicting velocities from the lower and highermodes, but has difficulties in predicting intermedi-ate values. Indeed, in the case illustrated in Fig. 8the predicted normalized velocity is around −0.5,far from the training data, indicating bias towardthe low mode.

4.4 Results

We implemented both a quantum-assisted and aclassical versions of the QAHM model and usethem to predict model output (final velocities)on a regular discretization of the 5-D input pa-rameter space, generating a total of 105 predic-

Figure 7: Markov chain sampling with averaging:at each step of the Markov chain several samplesare taken and the average of these values used tocalculate the value of the next step.

tions. In both cases a 6 x 12 x 12 deep neuralnetwork is trained on a 320 point dataset fromthe 6-D parameter space for 5000 epochs withlearning rate η = 0.01 and regularization factorλ = 0.0001. We present here a comparison of thelearning performance of the two versions of the al-gorithm and then discuss the predictions of thequantum-assisted generative model and their re-spective variations.

As demonstrated in Fig. 9, the quantum andclassical models generate predictions of compa-rable accuracy (as measured against the train-ing data). Furthermore, both sets of predictionshave the same characteristics, namely non-uniformmean squared error distribution with a large rangeand squeezing, as explored in the algorithm inves-tigations above.

The first conclusion drawn from the plots inFig. 9 is that the quantum algorithm is success-fully training a model. The second promising re-sult is that the classical and quantum algorithmsare comparable. Any slight difference can beattributed to the numerical differences incurredwhen translating between the two algorithms (e.g,

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 12 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Figure 8: Peak frequency sampling: for a giveninput vector the QAHM is sampled repeatedly andthe corresponding distribution generated.

a learning rate of 1 in the classical algorithm maynot equate to a learning rate of 1 for the quantumalgorithm). These ideas are complementary and afirst step in establishing the quantum-assisted al-gorithm and laying the groundwork for extensions,variations, and improvements.

Figure 10 confirms intuitive understanding ofthe data, thus providing evidence that thequantum-assisted model is learning the underlyingdistribution. The plot on the left represents a slicethrough the parameter space (for constant particleradius, material density, and inter-particle coeffi-cient of friction) with final velocities obtained asmodel predictions at various combinations of slopeand cohesion. The shape of the predicted surfaceacross the considered range generally follows theexpected trend. In particular, velocity is nega-tively correlated with slope and positively corre-lated with cohesion. The standard deviation of thepredictions, provided on the right of Fig. 10, showsthat the regions of high variance of the predictionstend to be in regions of steep change, namely be-tween the two modes (large and small velocity),characterized by large gradient of the velocity.

Figure 9: Comparison of the performance of thequantum and classical algorithms.

4.5 Algorithm Investigation Discussion

The issues outlined in these investigations aremode-switching and squeezing. These were identi-fied and described above, in conjunction with var-ious attempts at mitigating their effect on predic-tion accuracy. Mode-switching results in dominantanomalous predictions, while squeezing leads to asmall systematic error for the predictions at theextremities of the model.

Throughout the investigations, several sourcesof prediction squeezing have been highlighted. Al-though other sources may contribute to this ef-fect, the evidence suggests that fundamental as-pects of the training algorithm result in a modelwhich squeezes predictions to some range smallerthan that of the input data. It is possible to saywith some confidence that mode-switching is thecombined effect of the particular structure of thedata considered here and some learning feature ofthe algorithm which results in samples with a highvariance and with a bias toward regions trainedwith more data. Potential directions for devisingalgorithm adjustments to reduce variance and in-crease range of the resulting model are outlined inSection 5.

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 13 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Figure 10: Slice of the predictions and corresponding variance.

5 CONCLUSIONS. FUTURE WORK

A main result of this interdisciplinary project wasthe identification of some of the constraints thatcome from integrating the different fields. Mo-bility simulations are computationally expensiveand generate high-dimensional results. This meansthat a dataset of mobility simulations will contain(i) a relatively small number of simulations, and(ii) a large amount of information for each of thesimulations. This regime is the exact opposite ofthe desired one for two reasons. First, some of themost successful machine learning algorithms areknown to perform well when large amount of datais available, in contrast to (i). Second, currentlyavailable quantum computers have a small num-ber of qubits and therefore can handle a limitednumber of variables, in contrast to (ii).

Among the available quantum machine learn-ing proposals, we chose an algorithm that can inprinciple deal with some of the above constraintsand, at the same time, be implemented on ex-isting quantum hardware. In particular, the al-gorithm was designed for quantum annealing asimplemented by the D-Wave 2000Q hardware. A

simplified classical version was used for comparisonpurposes and to establish a baseline; the quantum-assisted version was used for the final results. Al-though the quantum algorithm did not excel, it didnot perform worse than the classical version. Thisis promising in the sense that the algorithm wasable to cope with noise and errors from the quan-tum annealer. As more general gate-model quan-tum computers become available and quantum al-gorithms evolve, we expect much better outcomes.

The machine learning model used here has alarge number of hyperparameters: number of lay-ers, number of units, learning rate, regularizationfactor, number of samples, and data encoding, toname a few. A thorough exploration of these wouldrequire a large number of executions and was be-yond the scope and timeline of this work. Althoughwe successfully proposed and implemented a D-wave-assisted alternative for the go/no-go prob-lem, more work is needed to determine whetherthere could be any advantage in this approach orif it would be better to consider quantum machinelearning implementations of other alternative solu-tions to this problem; e.g., via Bayesian optimiza-tion. However, it is likely that implementation of

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 14 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

these alternative machine learning techniques willrequire quantum hardware resources beyond thestate-of-the-art of currently available devices.

Future work. There are many routes open toexpanding this work. As an early stage imple-mentation of sampling from a quantum annealerto assist classical machine learning, it stands asone of the first efforts to address the problems fac-ing near-term implementation of quantum devices.Lessons learned here inform the development ofquantum algorithms that will eventually surpasstheir classical counterparts. We take it as a giventhat quantum devices will be used in some wayfor computation, with quantum-accelerated sub-routines within hybrid classical-quantum architec-tures, like the one presented here, a likely scenario.

However, much remains to be done beforequantum-assisted algorithms are mature enoughto be robustly applied to complex engineering andscientific problems. We highlight here three keypotential avenues for further exploration.

(i) First, other classical machine learning archi-tectures can be explored for the opportunityof replacing core subroutines with quantum-assisted algorithms. As there are promisingadvantages in the implementation describedin this paper, the lessons learned here canbe applied to the development of novel andpotentially improved hybrid algorithms. Al-ternative quantum computing architectures(notably gate-based) have passed small-scale,proof of principle, tests and are now comingto the second major hurdle: scalability. Algo-rithms such as the one described here must betailored, implemented, and analyzed to maptheir associated range of applicability, advan-tages, and limitations.

(ii) A second route is further research into appli-cations of the generative model learned by theQAHM. The generative model can potentially

be applied to a wide range of scenarios, such astime-series analysis, compression, fast searchof data through example generation, and as-sessing the effect of dataset size and complex-ity on the model.

(iii) Finally, we see great benefit for further re-search into potential QAHM variants, includ-ing placing the annealer at the top of therecognition network. This particular QAHMalgorithm has several advantages: the classi-cal recognition network can map continuousvariables and large data vectors to a smallquantum device, the quantum device does notneed to be queried for each data point, andthe temperature of the system does not needto be calculated, see Section 2. With these ad-vantages in mind, there are significant incen-tives to look at QAHM variants, both to see ifthere can be improvements made to the modellearned and to determine if alternative imple-mentations can be applied to a wider class ofapplications.

While the above directions are focused on theQML aspects of further exploration, there are in-teresting adjustments that can be considered onthe simulation side, to make it more amenable to aQML treatment and to fully tap into the potentialadvantages of using QML over classical machinelearning. In particular, the type of mobility prob-lems considered here invite a higher-dimensionalinput space; in other words, more input param-eters, which would allow an increased number ofnodes in the visible and hidden layers. However,more parameters to be learned will require signif-icantly larger training datasets. This suggests theuse of using lower-fidelity simulations, assumed tobe more expeditious. The disadvantages of loweraccuracy can potentially be offset by a deeperlearning of the parameter space and the genera-tive capabilities of the model. Furthermore, sucha model may be able to capture the uncertainty in

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 15 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

these lower fidelity simulations.Finally, it is worth noting that the nascent field

of quantum computing requires the identificationand exploration of new areas of scientific and engi-neering applications as candidates for quantum ac-celeration. It is the challenges posed by real-worldproblems that inform and direct the developmentof new algorithms and hardware.

Acknowledgments

This work was supported by a U.S. ArmyTARDEC project. Chrono development was sup-ported in part by U.S. Army TARDEC RapidInnovation Fund grant No. W911NF-13-R-0011,Topic No. 6a, “Maneuverability Prediction”. Sup-port for the development of Chrono::Vehicle wasprovided by U.S. Army TARDEC grant W56HZV-08-C-0236.

REFERENCES

[1] R. B. Ahlvin and P. W. Haley. “NATO Ref-erence Mobility Model: Edition II. NRMMUser’s Guide”. U.S. Army Engineer Water-ways Experiment Station, 1992.

[2] M. McCullough, P. Jayakumar, J. Dasch, andD. Gorsich. “The Next Generation NATOReference Mobility Model Development”. InProc. 8th Americas Regional Conference ofthe International Society for Terrain-VehicleSystems, 2016.

[3] D-Wave Systems Inc. “D-Wave 2000Q Tech-nology Overview”. https://www.dwavesys.

com/resources/publications. Accessed:2018-05-17.

[4] R. Serban, M. Wilson, M. Benedetti,J. Realpe-Gomez, and A. Perdomo-Ortiz.“Quantum Annealing for Mobility Stud-ies: Generation of GO/NO-GO Maps

via Quantum-Assisted Machine Learning”.Technical Report TR-2018-03, Simulation-Based Engineering Laboratory, University ofWisconsin-Madison, 2018.

[5] S. Lloyd, M. Mohseni, and P. Reben-trost. “Quantum Algorithms for Super-vised and Unsupervised Machine Learning”.arXiv:1307.0411, 2013.

[6] M. Benedetti, J. Realpe Gomez, andA. Perdomo-Ortiz. “Quantum-AssistedHelmholtz Machines: A Quantum-ClassicalDeep Learning Framework for IndustrialDatasets in Near-term Devices”. QuantumScience and Technology, 3(3):034007, 2018.

[7] G.E. Hinton, P. Dayan, B.J. Frey, andR.M. Neal. “The Wake-Sleep Algorithm forUnsupervised Neural Networks”. Science,268(5214):1158, 1995.

[8] M. Benedetti, J. Realpe-Gomez, R. Biswas,and A. Perdomo-Ortiz. “Quantum-AssistedLearning of Hardware-Embedded Probabilis-tic Graphical Models”. Phys. Rev. X,7:041052, Nov 2017.

[9] Project Chrono. “Chrono: An Open SourceFramework for the Physics-Based Simula-tion of Dynamic Systems”. http://www.

projectchrono.org. Accessed: 2018-02-07.

[10] A. Tasora, R. Serban, H. Mazhar, A. Pa-zouki, D. Melanz, J. Fleischmann, M. Tay-lor, H. Sugiyama, and D. Negrut. “Chrono:An Open Source Multi-physics Dynamics En-gine”. In T. Kozubek, editor, Lecture Notes inComputer Science, in print, pages –. Springer,2016.

[11] A. Tasora, D. Mangoni, D. Negrut, R. Ser-ban, and P. Jayakumar. “Deformable Soilwith Adaptive Level of Detail for Tracked and

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 16 of 17

Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Wheeled Vehicles”. Intl. J. Veh. Performance,in press, 2018.

[12] R. Serban, M. Taylor, D. Negrut, andA. Tasora. “Chrono::Vehicle Template-BasedGround Vehicle Modeling and Simulation”.Intl. J. Veh. Performance, in press, 2018.

[13] C. O’Sullivan. “Particle-Based Discrete Ele-ment Modeling: Geomechanics Perspective”.Int. J. Geomech., 11(6):449–464, 2011.

[14] P. Cundall and O. Strack. “A DiscreteElement Model for Granular Assemblies”.Geotechnique, 29:47–65, 1979.

[15] H. Kruggel-Emden, S. Wirtz, and V. Scherer.“A Study of Tangential Force Laws Applica-ble to the Discrete Element Method (DEM)for Materials with Viscoelastic or Plastic Be-

havior”. Chem. Eng. Sci., 63:1523–1541,2008.

[16] A. Pazouki, M. Kwarta, K. Williams,W. Likos, R. Serban, J. Jayakumar, andD. Negrut. “Compliant Versus Rigid Contact:A Comparison in the Context of Granular Dy-namics”. Phys. Rev. E, 96, 2017.

[17] M. Anitescu. “Optimization-Based Simula-tion of Nonsmooth Rigid Multibody Dynam-ics”. Mathematical Programming, 105(1):113–143, 2006.

[18] M. Anitescu and A. Tasora. “An Iterative Ap-proach for Cone Complementarity Problemsfor Nonsmooth Dynamics”. ComputationalOptimization and Applications, 47(2):207–235, 2010.

Quantum Annealing for Mobility Studies: Go/No-Go Maps via Quantum Machine Learning, Serban, et al.Page 17 of 17