Improved representation of situational awareness within a dismounted small combat unit constructive...

10
Improved representation of situational awareness within a dismounted small combat unit constructive simulation K. David Lee, Mike Colony Decisive Analytics Corp., 1235 S. Clark St. Suite 400, Arlington, VA, USA 22202 ABSTRACT Modeling and simulation has been established as a cost-effective means of supporting the development of requirements, exploring doctrinal alternatives, assessing system performance, and performing design trade-off analysis. The Army’s constructive simulation for the evaluation of equipment effectiveness in small combat unit operations is currently limited to representation of situation awareness without inclusion of the many uncertainties associated with real world combat environments. The goal of this research is to provide an ability to model situation awareness and decision process uncertainties in order to improve evaluation of the impact of battlefield equipment on ground soldier and small combat unit decision processes. Our Army Probabilistic Inference and Decision Engine (Army-PRIDE) system provides this required uncertainty modeling through the application of two critical techniques that allow Bayesian network technology to be applied to real-time applications. (Object-Oriented Bayesian Network methodology and Object-Oriented Inference technique). In this research, we implement decision process and situation awareness models for a reference scenario using Army-PRIDE and demonstrate its ability to model a variety of uncertainty elements, including: confidence of source, information completeness, and information loss. We also demonstrate that Army-PRIDE improves the realism of the current constructive simulation’s decision processes through Monte Carlo simulation. Keywords: Modeling and simulation, Constructive simulation, Small combat unit, Bayesian Inference, Decision making, Object-Oriented Bayesian Network, Object-Oriented Inference 1. INTRODUCTION Modeling and simulation has been established as a cost effective means of supporting the development of requirements, exploring doctrinal alternatives, assessing system performance, and performing trade-off analysis. The level of fidelity built into a simulation is largely a design decision that balances the desire for fidelity against the realities of cost, schedule and quality of the system. The U.S. Army Natick Soldier RD&E Center (NSRDEC) and Army Materiel Systems Analysis Activity (AMSAA) are leading an effort to develop a robust constructive force-on-force simulation that accurately represents individual Ground Soldiers (GS) and dismounted Small Combat Units (SCU). The purpose of this GS/SCU simulation is to do operational effectiveness assessments. One of the several represented factors contributing to these assessments is the impact and value of information. Currently, the simulation uses ground truth data in its GS/SCU Situational Awareness (SA) and decision process models. This effort’s goal is to more realistically represent the realities of the battlefield by incorporating uncertainty elements into the modeling process to provide more realistic SA models. This enhancement will enable more accurate simulation of the impact and value of information on the decision processes and the battlefield systems that are employed within a SCU. To achieve the research goal, two research objectives are crucial. Objective 1 is the implementation of techniques that will allow constructing Bayesian- based decision networks in real-time that represent small combat unit decision problems. This generic engine will provide a stand-alone modeling capability that utilizes a robust mathematical framework, based on advanced real-time Bayesian network techniques, for modeling situational awareness and dynamic decision-making processes that are characterized by uncertainty elements. The engine will perform probabilistic modeling of GS decision processes for constructive simulations, such as Infantry Warrior Simulation (IWARS) [1]. Objective 2 is the implementation of factor analysis mechanisms that enable evaluation of the performance of the improved SA and decision models, which includes uncertainty modeling, against the baseline IWARS system, which does not. Objective 2 is intended to provide analysis mechanisms within the framework of the decision engine to measure and evaluate the outcomes produced by a probability-based simulation. [email protected]; phone 1 703 414 5087; fax 1 703 414 5066; www.dac.us Modeling and Simulation for Defense Systems and Applications VI, edited by Eric J. Kelmelis, Proc. of SPIE Vol. 8060, 80600G · © 2011 SPIE · CCC code: 0277-786X/11/$18 · doi: 10.1117/12.882749 Proc. of SPIE Vol. 8060 80600G-1

description

Modeling and simulation has been established as a cost-effective means of supporting the development of requirements, exploring doctrinal alternatives, assessing system performance, and performing design trade-off analysis. The Army’s constructive simulation for the evaluation of equipment effectiveness in small combat unit operations is currently limited to representation of situation awareness without inclusion of the many uncertainties associated with real world combat environments. The goal of this research is to provide an ability to model situation awareness and decision process uncertainties in order to improve evaluation of the impact of battlefield equipment on ground soldier and small combat unit decision processes. Our Army Probabilistic Inference and Decision Engine (Army-PRIDE) system provides this required uncertainty modeling through the application of two critical techniques that allow Bayesian network technology to be applied to real-time applications. (Object-Oriented Bayesian Network methodology and Object-Oriented Inference technique). In this research, we implement decision process and situation awareness models for a reference scenario using Army-PRIDE and demonstrate its ability to model a variety of uncertainty elements, including: confidence of source, information completeness, and information loss. We also demonstrate that Army-PRIDE improves the realism of the current constructive simulation’s decision processes through Monte Carlo simulation.

Transcript of Improved representation of situational awareness within a dismounted small combat unit constructive...

  • Improved representation of situational awareness within a dismounted small combat unit constructive simulation

    K. David Lee, Mike Colony

    Decisive Analytics Corp., 1235 S. Clark St. Suite 400, Arlington, VA, USA 22202

    ABSTRACT

    Modeling and simulation has been established as a cost-effective means of supporting the development of requirements, exploring doctrinal alternatives, assessing system performance, and performing design trade-off analysis. The Armys constructive simulation for the evaluation of equipment effectiveness in small combat unit operations is currently limited to representation of situation awareness without inclusion of the many uncertainties associated with real world combat environments. The goal of this research is to provide an ability to model situation awareness and decision process uncertainties in order to improve evaluation of the impact of battlefield equipment on ground soldier and small combat unit decision processes. Our Army Probabilistic Inference and Decision Engine (Army-PRIDE) system provides this required uncertainty modeling through the application of two critical techniques that allow Bayesian network technology to be applied to real-time applications. (Object-Oriented Bayesian Network methodology and Object-Oriented Inference technique). In this research, we implement decision process and situation awareness models for a reference scenario using Army-PRIDE and demonstrate its ability to model a variety of uncertainty elements, including: confidence of source, information completeness, and information loss. We also demonstrate that Army-PRIDE improves the realism of the current constructive simulations decision processes through Monte Carlo simulation.

    Keywords: Modeling and simulation, Constructive simulation, Small combat unit, Bayesian Inference, Decision making, Object-Oriented Bayesian Network, Object-Oriented Inference

    1. INTRODUCTION Modeling and simulation has been established as a cost effective means of supporting the development of requirements, exploring doctrinal alternatives, assessing system performance, and performing trade-off analysis. The level of fidelity built into a simulation is largely a design decision that balances the desire for fidelity against the realities of cost, schedule and quality of the system. The U.S. Army Natick Soldier RD&E Center (NSRDEC) and Army Materiel Systems Analysis Activity (AMSAA) are leading an effort to develop a robust constructive force-on-force simulation that accurately represents individual Ground Soldiers (GS) and dismounted Small Combat Units (SCU). The purpose of this GS/SCU simulation is to do operational effectiveness assessments. One of the several represented factors contributing to these assessments is the impact and value of information. Currently, the simulation uses ground truth data in its GS/SCU Situational Awareness (SA) and decision process models. This efforts goal is to more realistically represent the realities of the battlefield by incorporating uncertainty elements into the modeling process to provide more realistic SA models. This enhancement will enable more accurate simulation of the impact and value of information on the decision processes and the battlefield systems that are employed within a SCU. To achieve the research goal, two research objectives are crucial. Objective 1 is the implementation of techniques that will allow constructing Bayesian-based decision networks in real-time that represent small combat unit decision problems. This generic engine will provide a stand-alone modeling capability that utilizes a robust mathematical framework, based on advanced real-time Bayesian network techniques, for modeling situational awareness and dynamic decision-making processes that are characterized by uncertainty elements. The engine will perform probabilistic modeling of GS decision processes for constructive simulations, such as Infantry Warrior Simulation (IWARS) [1]. Objective 2 is the implementation of factor analysis mechanisms that enable evaluation of the performance of the improved SA and decision models, which includes uncertainty modeling, against the baseline IWARS system, which does not. Objective 2 is intended to provide analysis mechanisms within the framework of the decision engine to measure and evaluate the outcomes produced by a probability-based simulation.

    [email protected]; phone 1 703 414 5087; fax 1 703 414 5066; www.dac.us

    Modeling and Simulation for Defense Systems and Applications VI, edited by Eric J. Kelmelis, Proc. of SPIE Vol. 8060, 80600G 2011 SPIE CCC code: 0277-786X/11/$18 doi: 10.1117/12.882749

    Proc. of SPIE Vol. 8060 80600G-1

  • This paper is composed of three primary sections: Method (Army-PRIDE); SA and decision models; and Test & Evaluation of Army-PRIDE. Section 2, Method (Army-PRIDE) covers the first research objective. In it the technical innovations within Army-PRIDE system are addressed. The core technologies within Army-PRIDE are the result of many years of R&D performed under other DOD research contracts to develop advanced probabilistic inference capabilities. The technical advancement made under this research effort is focused on transforming the inference capabilities into an automated Decision-making under Uncertainty engine suitable for use in constructive simulation systems. In Section 3, Models, we introduce a test scenario with simplified GS decision processes, and the corresponding SA and decision models that are implemented within Army-PRIDE. Section 3 covers the ability of Army-PRIDE to model information uncertainty elements, such as confidence of source, information completeness, and information loss. Section 4, Test and Evaluation of Army-PRIDE, covers the second research objective. This chapter demonstrates 1) Improved realism of decision processes using Bayesian decision networks for decision modeling, and 2) The ability of the engine to scale up to sufficiently handle large scale scenario runs. Section 5, Conclusion, provides a summary of the research work and a brief outline of further studies.

    2. METHOD (ARMY PRIDE) 2.1. Army PRIDE Concept

    Our approach is built around a powerful inference engine which is extended into a real-time decision engine for use with constructive simulations. This inference engine combines a number of novel Bayesian network capabilities to provide a robust information fusion and inference capability. Bayesian networks are widely accepted as a superior mathematical framework for the representation of data uncertainties, including the causal influences between variables. The use of Bayesian network technology, however, has historically been infeasible for use in real-time and/or dynamically changing environments for two primary reasons:

    i. Techniques did not exist to enable dynamic construction of networks that could represent a complex fusion or decision problems as they changed and evolved, in real-time.

    ii. The size and complexity of the networks were limited due to the exponential growth of computing resource requirements.

    Through a number of research and development efforts over the past eight years our research team (DECISIVE ANALYTICS Corporation DAC) has developed a suite of tools that overcome these limitations. DAC has pioneered the development of technologies that enable autonomous construction, updating, and management of situation-specific Bayesian networks that accurately represent dynamically changing environments, as well as breakthrough techniques that overcome many of the computational limitations of Bayesian networks. The DAC teams approach to meeting this research objective is to extend our mature inference engine into a Decision Engine. In addition to the ability to automatically form Bayesian Network models for fusion and inference for SA, DACs new Army Probabilistic Inference and Decision Engine (Army-PRIDE) will be able to represent and solve decision problems. This enhancement includes modifying the system to include decision nodes for representing decision alternatives, and value nodes for evaluating the expected value of the decision alternatives with respect to the uncertainties that are represented by a traditional Bayesian network [2][4]. However, due to the initial requirements of our Army customer, we restricted Army PRIDEs role in this initial research phase to modeling only SA situations. The decision engine of the Armys constructive simulation utilizes the output of these SA models to model the actual decision process. A conceptual diagram of the Army-PRIDE system, shown in Figure 1, depicts how the system will interface with a constructive simulation, such as Infantry Warrior Simulation (IWARS). As shown in Figure 1, the constructive simulation passes ground truth conditions (decision elements) such as force size or location to Army PRIDE. Army PRIDE then constructs an appropriate SA model, which includes the observed conditions and uncertainty elements (e.g. confidence of source, information completeness, and information loss). Perceived knowledge (e.g. what do I believe my force size to be?) is derived from inferences computed within the Bayesian SA models. These inferences are returned to the constructive simulation where the decision engine simulates the decision process based on uncertain, inferred information. The graphical user interface (GUI) of Army PRIDE assists the users with the development of the Bayesian network SA models. (This component is not addressed in this paper). SA models are stored within the Decision Element library.

    Proc. of SPIE Vol. 8060 80600G-2

  • Figure 1: Conceptual diagram of Army-PRIDE approach

    2.2. Technical background of Army-PRIDE

    There are two novel techniques within Army PRIDE that require summarization: Object-Oriented Bayesian Networks (OOBN) and Object-Oriented Inference (OOI). OOBN is a foundational technique that enables real-time construction of Bayesian networks [3]. Based upon standard object-oriented design principles, OOBN enables autonomous, data-driven Bayesian network construction which can be leveraged to provide realistic situational awareness that reflects the current information state of a dynamically changing environment. With OOBN, instead of building a belief network ahead of time to represent all aspects of a problem domain, each of the smallest elements of the domain is modeled separately. These element models, known as Frames, can represent any object (e.g. soldier, terrain, equipment), concept (e.g. assessment of enemy threat), event (e.g. enemy fire), or observation data (including the reliability of the data and the confidence in the source). The inference engine then uses these Frames as building blocks to compose an overall belief network that reflects these elements and the relationships between them, in the context of the current situation. Using a data-driven process, the engine uses a modeling schema that describes the Frames and their relationships to data types and other Frames to automatically generate an appropriate belief model in real-time. To put this in the context of a GS simulation, consider a platoon leader who is collecting information that will aid him in assessing an enemy that his unit has just made contact with. To develop a belief or decision model for this GS situation a-priori would be fruitless because the form of the model must reflect the information that is available at each time-step in the simulation. As the situation evolves, so does the information that is available, and so may the relationships between objects. So, since the exact information set that will be available at any given time will not be known in advance, the models also cannot be constructed in advance. Every situation is different and demands an ability to dynamically construct a model that accurately represents the situation. DACs OOBN implementation provides this capability. Most importantly, by utilizing Bayesian networks this approach allows representation of the uncertainties of the data and the causal relationships between them. OOI is a revolutionary structured inference technique that overcomes the most restrictive obstacle associated with the use of Bayesian networks computational tractability. Bayesian networks, while mathematically powerful in their ability to accurately represent uncertainty, carry the burden of exponential growth in the computational resources required to solve them. The OOI technique takes advantage of the fact that many large networks (or large numbers of small networks) contain a very significant amount of repeated structure. The OOBN methodology allows users to capture repeated structures once, within a single well-defined model (i.e. a Frame). OOI is then able to exploit the object-oriented nature of the model by representing each repeated Frame in the model just a single time. Then, inference is performed within the Frame and only when necessary. This dramatically reduces the computational requirements of the network. To illustrate this with an example, consider again the platoon leaders threat assessment situation. In a situation where 10 enemy combatants are observed, users may construct an overall assessment model that includes 10 combatant Frames that represent the perceived strength and maneuverability of each combatant. Now extend the example and consider a

    Proc. of SPIE Vol. 8060 80600G-3

  • large scale constructive simulation of GS operations where this kind of assessment model is being constructed for dozens of GS agents. With traditional Bayesian network techniques, the engine would end up with potentially hundreds or thousands of combatant Frames spread across a large number of threat assessment models. With DACs OOI technique, the combatant Frame is constructed a single time and queried to perform inference about each enemy combatant. As shown in Figure 2, the reduction in computational requirements is dramatic and allows the utilization of Bayesian networks for complex problems where the use of traditional techniques will fail.

    Figure 2: OOI implementation compared to traditional Bayesian network inference algorithm

    3. MODELS 3.1. Test scenario and decision process

    We developed a simple test scenario concept that would allow the testing and evaluation of the uncertainty modeling concept (Army-PRIDE). The goal was to represent various uncertainty types, such as confidence of source, information completeness, and information loss so that the effect of each could be demonstrated, both individually, and in combinations. In order to isolate the effects of interest, it was decided to utilize a relatively small, stripped down scenario. The units in the test scenario are summarized in Table 1. Table 1: Units in test scenario

    Blue Force (BF) Red Force (RF) BF1 (consists of two 4 soldier teams)

    BF1a is the lead team for BF1 BF1b is subordinate to BF1a

    BF2 (consists of two 4 soldier teams) BF2a is the lead team for BF2 BF2b is subordinate to BF2a

    BF3 (consists of one 4 soldier team) Support team located remotely

    RF1 (consists of 8 soldiers) RF2 (consists of 8 soldiers)

    In this simple scenario, Blue Force 1 (BF1) will engage Red Force 1 (RF1), while BF2 will engage RF2. BF3 is a support team that is available to support either battle. For simplicity, we assume BF3 has one decision criteria in making a decision on which engagement to support. The BF3 leader makes his decision based on the unit (BF1 or BF2) that is perceived to be at the biggest engagement disadvantage. In other words, the team that has the smallest advantage, in terms of the difference between his beliefs on the numbers of his soldiers still alive and engaged and the numbers the enemy has, will receive the support. However, BF3 is remotely located and must travel to the vicinity while receiving information about the status of each engagement from the Lead Team of each unit (i.e. BF1a reports for BF1; BF2a reports for BF2). These status report messages include various types of uncertainty, including information loss due to dropped/missed messages. The test scenario is illustrated in Figure 3. The information that the BF3 leader will receive includes uncertainties. It will also be subject to the following elements/constraints factor.

    i. Lead Teams BF1a and BF2a have an error rate in the observations of their engagement (detection rate, false alarm), and their observations are also affected by the confidence of information sources (their own sensors and reports from their subordinates BF1b and BF2b).

    Proc. of SPIE Vol. 8060 80600G-4

  • ii. BF3s decision process is affected by the confidence of the information provided by BF1a, BF2a, which affects his estimation of the number of blue and red soldiers still active in each engagement.

    iii. BF3s estimation of the number of soldiers still active in each engagement is also affected by missed messages, resulting in information loss and in information incompleteness.

    Figure 3: Scenario Overview

    3.2. The exemplary model

    As addressed in the previous section, in order for the BF3 leader to make a decision on which engagement to support, he makes his decision based on his belief of which team (BF1 or BF2) is at the biggest disadvantage. Unlike a decision maker in a constructive simulation, such as IWARS, who has perfect information on the status of his soldiers and the enemy, a decision maker (BF3) in this test scenario must make the decision under conditions of uncertainty regarding the status of each engagement (i.e. the number of blue and red soldiers still active in each engagement). Figure 4 represents the inference and decision model for the test scenario. The model consists of three primary Bayesian network frames (Frame 1, 2, 3) that represent the information fusion and inference processes of BF1, BF2 and BF3 respectively, and one Bayesian decision network frame (Frame 4), which models BF3s decision process. This decision network frame includes a decision node that captures the decision alternatives, and a utility node that captures the evaluation criteria. As previously addressed in the Army PRIDE concept section, the role of Army PRIDE is restricted to modeling the soldiers situational awareness. The actual decisions are made within the constructive simulation, based on the perceived knowledge provided by Army-PRIDE. Therefore, we do not address Frame 4 here in detail.

    Figure 4: Inference and decision models in test scenario

    3.2.1. Frame 1 and Frame 2

    Proc. of SPIE Vol. 8060 80600G-5

  • Referencing Figure 4, Frame 1 and Frame 2 are implemented to provide inference of the number of RF1 soldiers in Battle 1 and the number of RF2 soldiers in Battle 2. Frame 1 and Frame 2 are both composed of two sub-frames which represent each units (BF1a, BF1b, BF2a, BF2b) inference model on the number of soldiers still active in each engagement. The actual Bayesian network model for Frame 1 (also the same for Frame 2) is illustrated in Figure 5. Two examples of the Bayesian network model used to represent information fusion and inference performed by BF1a and BF1b are shown in Figure 5. The example on the left shows a High confidence observation of the number of RF still engaged, which results in a relatively strong belief (i.e. Estimation) in the actual number of RF still engaged. The example on the right is identical, except that the confidence in the observation is Low, resulting in a highly uncertain belief in the number of RF still engaged.

    Figure 5: Frame 1 model captures the uncertain estimations by BF1a and BF1b of the number of red forces still engaged. The model further fuses both estimations into an overall belief for the entire BF1 unit

    This model shows a realistic estimation of the number of RF, which has been affected by Sensor Performance and Information Source uncertainties. The conditional probability table between nodes (#RF Observation from BF1a and #RF Estimation A) is implemented as a normal distribution, which considers Sensor Performance uncertainties such as detection rate and false alarm rate of the sensor. In this case the sensor is the vision of the soldiers. The Confidence node in this figure represents the uncertainty of the information source. Source confidence can be related to a number of factors, such as the distance of observation, time of day (day or night) of the observation, weather conditions, or a number of other factors. As shown in Figure 5, when the BF1a leader observes six RFs under High confidence level, his actual estimation (i.e. belief) of #RF from this model has a relatively tight probability distribution (node - #RF Estimation A). When the BF1b leader observes six RFs under a Low confidence level, his actual estimation of #RF results in a relatively wide probability distribution (node - #RF Estimation B). The observations from both units (BF1a and BF1b) are fused in the node - #RF Fusion. This provides the inferred belief for the entire BF1 unit on the number of RF soldiers still alive. Frame 2 is modeled similarly.

    3.2.2. Frame 3

    Frame 3 (shown in Figure 6) represents the complete SA model for the support unit, BF3, which models his perception of the two ongoing battles. In this Frame, an estimate of which unit (BF1 or BF2) has a stronger advantage in his battle is generated based on the reported information from BF1 and BF2. For simplicity we assume the leaders of each unit know with relative certainty the number of their own forces still engaged. Uncertainty only exists for the belief of the red force status. Frame 3 consists of two parts: the upper part represents the situation awareness of the battle between BF1 and RF1 at time t, and lower part represents the situation awareness of battle between BF2 and RF2 at time t. As previously discussed, the estimates of the number of RF1 and RF2 are provided in the reports from BF1a and BF2a. In this mode these estimations are further affected by the confidence of each information source. The current situation awareness of battle 1 at t = 1 shows the following belief for BF1: [leading: 89.3%, even: 10.6%, losing: 0.03%]. This likelihood distribution is computed in the probability model from the two nodes that influence it: the number of BF (7), and the estimated number of RF (node #RF1 Estimation A). Similarly, the current situation awareness of battle 2 at t = 1 shows the following belief for BF2 [leading: 5.52%, even: 13.8%, losing: 80.8%]. These two situation awareness nodes are utilized for estimating which blue force unit is at a higher degree of disadvantage. As shown in Figure 6, the current degree of disadvantage is computed to be [6.9:93.1]. That is, BF2 is at more of a disadvantage than BF1.

    Proc. of SPIE Vol. 8060 80600G-6

  • Figure 6: Frame 3 captures the uncertainties associated with the support unit, BF3, in estimating which of the two ongoing engagements is at a higher level of disadvantage

    This model also handles other uncertainty elements, such as information incompleteness and information loss. First, we represent information incompleteness by not applying the evidence in the model as demonstrated in Figure 7.

    Figure 7: Information incompleteness for full assessment on #RF

    Figure 7 shows that the information of BF1bs observation on #RF has not been completed, thus the full assessment on #RF (node - #RF Fusion) results in a relatively wide probability distribution being compared to that of Figure 5. Second, we incorporate the information loss by adopting a Markov chain concept as shown in Figure 8. For example, when the incoming information on the #RF and #BF in Battle 1 is missing at t, while the incoming information on the #RF and #BF in Battle 2 are not missing at t, the BF3 leader must estimate the state of Battle 1 at t using only his estimate of the battle from the previous time step (i.e. at t-1). We model this uncertainty element information loss using two assumptions. First, the BF3 leader will have some a priori knowledge of similar situations which provides a belief on how the battle will evolve. The second assumption is that his uncertainty about the situation will grow, which is logical. In this example the Battle 1 situation at time=t has a wider distribution than that of the previous time step (t-1). The example of a transition probability distribution of this matrix is demonstrated in left-bottom of Figure 8.

    Proc. of SPIE Vol. 8060 80600G-7

  • Figure 8: Battle situation model with missing information

    4. TEST AND EVALUATION PLAN AND TEST RESULTS We have discussed how various uncertainties and decision processes are modeled through the Army-PRIDE system. We expect the decisions made in these models to be different from those made within deterministic simulations because Army-PRIDEs model incorporates many of the uncertainties a GS is faced with when making decisions. In this section we present the results of a series of tests that were run to analyze the differences between the two systems in order to demonstrate improved realism of the constructive simulation (e.g. IWARS) decision processes.

    Figure 9: Simulation Process

    For this demonstration, we executed the decision model introduced in the previous section and compared the results to the results provided by the current constructive simulations implementation. The general process for our test procedure is illustrated in Figure 9. To do this, a baseline that closely models the current constructive simulations implementation (i.e. no uncertainty) was established. This was implemented as a Bayesian decision network with all uncertainties removed in Army PRIDE, simply by applying evidence directly to the variables that represent ground truth, not to observation models that include uncertainty. With the test baseline established, we then iteratively exercised each

    Proc. of SPIE Vol. 8060 80600G-8

  • uncertainty model in Monte Carlo fashion (500 times) to statistically compare the decision results between IWARS and Army-PRIDE for each case. The test in this paper has been focused on two specific uncertainty elements, namely: confidence of information source; and information loss. We did not include the test of information incompleteness in this paper since the test results were similar to those of information loss. For simplicity, normal distributions for representing detection errors and false alarm are assumed to be correctly estimated.

    4.1. The effect of information source confidence in the decision process

    In this test we simplified the process by first considering uncertainty associated only with one of the engagements the BF1 engagement. The engagement of BF2 was held constant, as if the battle had just begun, while the BF1 engagement was allowed to run well into the engagement. Therefore, among the four frames defined back in Figure 4, only three confidence of information source nodes are varied. That is, two confidence of information source nodes in Frame 1 and one confidence of information source node for BF1 in Frame 3. The following graph (Figure 10) demonstrates the improved representation of decisions made under uncertainty using Army-PRIDE, as compared against the deterministic decisions represented and made in IWARS. In this graph, GT represents Ground Truth. Among 500 simulations, the Army-PRIDE and IWARS decisions are essentially the same with high confidence in all information sources. In other words, all Highs (HHH) on the three confidence nodes are essentially equivalent to the certain conditions for a decision made by an IWARS GS entity. Each other bar in the graph represents the confidence level of the three information source node representations in the model (Confidence of BF1a, BF1b and BF3 respectively). For example, the fourth bar from the left is labeled HLH, which represents: [BF1a Information Confidence=High; BF1b Information Confidence=Low; BF3 Information Confidence=High].

    Over Uncertainties (Confidence level)

    100.0% 100.0%89.4% 94.2% 86.0% 82.6% 79.6% 82.8% 78.6%

    0.0%

    20.0%

    40.0%

    60.0%

    80.0%

    100.0%

    120.0%

    GT HHH LHH HLH HHL LLH LHL HLL LLL

    Degree of confidence level of information

    Cor

    rect

    ness

    aga

    inst

    GT

    Uncertainty on Confidence of information

    Figure 10: Decision comparison of confidence of information uncertainty type.

    As illustrated in Figure 10, as more uncertainty was introduced into the overall decision model, the number of decisions which are made differently from those of the ground truth (IWARS) model grows correspondingly. As seen in the graph, the first (BF1a leaders confidence level) and the third nodes (BF3 leaders confidence level) appear to be more sensitive than the second one (BF1b leader). That is, the correctness against ground truth of the LHH (89.4%) and HHL (86.0%) case is lower than that of HLH (94.2%). Similarly, the correctness against ground truth of the LHL (79.6%) case is lower than those of LLH (82.6%) and HLL (82.8%). There is a reason for this. In the test scenario, the leader of BF1a is also the overall leader for the entire BF1 unit. As such, he was modeled to be more confident of his own information than the information reported by his subordinate, BF1b. This then also gets reflected up the chain with BF3. 4.2. The effect of information loss in the decision process

    To simulate loss of information during an engagement, we simulated three cases of information loss (one time step loss, two time step loss, and three time step loss). As discussed in Figure 8, the initial model was extended slightly by incorporating a Markov transition matrix that captures a Ground Soldiers a priori belief of how a battle would evolve over time. The following graph demonstrates the realism of the decisions made using Army-PRIDE, as compared against the deterministic decisions made in IWARS. Among the 500 simulation runs, in runs with a single time step of

    Proc. of SPIE Vol. 8060 80600G-9

  • missing information, the Army-PRIDE system made the same decision as the baseline IWARS system 72.0% of the time. 28% of the time the uncertainties caused by the information loss produced different decisions. As the duration of the time with missing information is increased, the effect was, understandably, more uncertainty and hence, more variance in the decisions made, as compared to IWARS. This can be seen in the graph of Figure 11.

    Uncertainty: Information Loss

    100.0%

    72.0%62.4% 61.0%

    0.0%

    20.0%

    40.0%

    60.0%

    80.0%

    100.0%

    120.0%

    GT Loss one time step Loss two time step Loss three time step

    # of information loss time

    Corr

    ectn

    ess

    agai

    nst G

    T

    Figure 11: Decision comparison of Information Loss uncertainty type

    We also tested the scalability of the Army-PRIDE system. To do this we extended the decision model in Figure 4 gradually to ten replicas under the assumption that there are 10 decision makers in the battle field similar to the BF3 leader. The ten decision networks were further extended 10 time steps each. These 100 different networks were executed in the Army-PRIDE decision engine, computing and recording performance metrics for each. The computational time required in most cases is under 0.02 seconds and the longest computing time is 0.1 seconds, which is also very minor.

    5. CONCLUSION The Army-PRIDE decision engine, which is an adaptation of a mature real-time Bayesian network inference engine, provides a means to represent situation awareness under uncertainties, such as confidence of source, and information loss to accurately evaluate the impact and value of information on the decision processes and the battlefield systems that are employed within a small combat unit. Army-PRIDE demonstrates the feasibility of the tool as a long-term strategy that provides a generalized solution that can meet the constructive simulations needs, including the ability of the engine to scale up to large scenario sizes and large numbers of scenario runs.

    ACKNOWLEDGEMENT This research was supported by Army Natick Soldier RD&E Center under an SBIR Phase I contract, number W911QY-09-C-0032. We would like to thank Dave Tucker and Dean Sutherland for precious guidance.

    REFERENCES 1. IWARS Overview http://nsrdec.natick.army.mil/m&a/IVMS200501_IWARS_Overview.pdf 2. Jensen, F. An Introduction to Bayesian Networks. Springer 1996 3. Koller, D.; Pfeffer, A. SPOOK A System For Probabilistic Object-Oriented Knowledge Representation. 1999 4. Kirkwood, C. W. 1997. Strategic Decision Making. Duxbury

    Proc. of SPIE Vol. 8060 80600G-10

    SPIE ProceedingsMAIN MENUContentsSearchClose