Experiences inDeveloping An Intelligent Ground Vehicle (IGV) Ontology In Protégé

Craig Schlenoff, NISTRandy Washington, DCS Corporation

Tony Barbera, NIST

July 8, 2004

Agenda• Background:

– What is an Intelligent Ground Vehicle (IGV)?– NIST 4D/RCS Methodology and Architecture

• Ontology Development:– 4D/RCS to Ontology Mapping– Interchange Formats and Upper Ontologies– IGV Military Equipment– IGV Behaviors– IGV Conditions

• Current Status• Issues and Lessons Learned

What is an Intelligent Ground Vehicle?

4D/RCS Mapping to IGV Ontology Commander &

Vehicle Agent Instances

ServiceInstances

EnvironmentInstances

ProcessModel

Instances

ConditionInstances

DOT Driving Manualsand

ARMY Field Manuals

+

Domain Experts

Task Decomposition Tree(Route Reconnaissance Example)

Hierarchical Organization ofAgent Control Modules

BEHAVIORGENERATION

WORLD MODELKNOWLEDGE

SENSORYPROCESSING

SituationsWorld StatesObjects & Attributes

ColorCameras LADARRadar Stereo FLIR Nav

Segmented Groupings

Features and AttributesObjects and Maps

Object Groupings andClassifications

Cattails

ConductRoadMarchToAnAssemblyArea(AA)

ConductRouteRecon

PrepareForRoadMarch

FollowPlatoonToAssemblyArea

SecureAssemblyArea

OrganizeAssemblyArea

ConductMainRouteRecon

DeployToStartPoint

ConductLeftFlankRouteRecon

Locate&SecureObstacleBypass

ConductObstacleRecon

ConductRightFlankRouteRecon

MoveIntoMarchFormation

PrepareDetailedMovementPlans

MoveToControlPoint

ConductDominateTerrainRecon

MoveToCover/ConcealedPosition

FordWaterObstacle

SecureArea

OverwatchSection

PerformFordRecon

LocateWaterBypass

MoveToControlPoint

AssessFordTerrain

MoveToPosition

ScanPath

CrossFord

ScanAreaForEnemy

MoveToWater

ShiftTo4WhLo

MoveToOppositeBank

ShiftTo4WhHi

DryBrakes

SENSORYPROCESSING

KNOWLEDGEDATABASE

BEHAVIORGENERATION

WORLDMODEL

VALUEJUDGMENT

(Executing)

(ReconToRoute)(Executing)SENSORY

INPUT

(Platoon “B” Section Agent Control

Module)

Select“ConductWaterObsRecon”

Plan State-Table

(ConductRightFlankRecon)

ScoutPlatoon

1st

LightCavalryTroop

CommandGroup(FSO)

QuarteringParty

ScoutPlatoon

3rd

ConductRoadMarchToAssemblyArea

ConductRouteReconSecureAssemblyAreaConductAreaRecon

OrganizeAssemblyArea

FollowPlatoonToAssemblyAreaPrepareForRoadMarchMoveIntoFormationExecuteRoadMarchExecuteHaltOccupyAssemblyArea

PrepareForRoadMarchMoveIntoFormationExecuteRoadMarchExecuteHaltOccupyAssemblyArea

C Section

HMMWV#4 PSG

HMMWV#10

HMMWV#9

Comms

Primitive/Trajectory

Surveillanc(RSTA)

GazeControl

Lethality(Gunner)

Mobility

WeaponControl

TargetingSensors

Pan/TiltControl

FLIR, LaserRadar,Camera

CrossFordMoveToPosition

ScanPath

MoveToWaterShiftTo4WheelLoMoveToOppositeBankShiftTo4WheelHiDryBrakes

AssessFordTerrain ScanAreaForEnemy

A Section

SquadHMMWV #7

SquadHMMWV #3

SquadHMMWV #2

DeployToStartPointConductLeftFlankReconLocate&SecureObstacleBypassMoveToControlPoint

LocateWaterBypassPerformFordReconFordWaterObstacleSecureAreaOverwatchSection

B Section

HMMWV#8

HMMWV#6

HMMWV#5

DeployToStartPointConductRightFlankReconConductDominantTerrainReconConductAmbushSiteReconMoveToControlPoint

SendReport

NormalRouteReconSituation ConductRightFlankRecon

ConductWaterObstacleRecon WaterObstacleDetected

AssessMinefieldRecon MinefieldDetected

DefileDetected ConductDefileRecon

LateralRouteDetected ConductLateralRouteRecon

Selection Conditions Selected Plan

PLAN SELECTION TABLE

PLAN STATE-TABLEInput Conditions Output Commands

S1 sq5_NoBypassOnRouteSide

S2 SetupReport&NewControlMeasures sq5_ReconToFarFlank sq6_ConductTravelingOverwatch sq8_VisuallyClearObstacle-FarSide

NewPlan S1 SetupControlMeasures sq5_ReconToRouteFlank sq6_ConductTravelingOverwatch sq8_VisuallyClearObstacle-FarSide

ConductWaterObstacleRecon

S2 sq5_NoBypassTowardBoundary

S0 SetupReport&NewControlMeasures sq5_MoveTacticallyToControlPoint sq6_ConductTravelingOverwatch sq8_MoveTacticallyToControlPoint

S1 sq5_PossibleFordDetected S4 SetupReport&NewControlMeasures sq5_ConductFordRecon sq6_ConductNearSideOverwatch sq8_VisuallyClearObstacle-FarSide

S1 sq5_LateralBypassFound

S7 SetupReport&ControlMeasures sq5_ConductObstacleFarSideRecon sq6_ConductTravelingOverwatch sq8_VisuallyClearToRoute

S4 sq5_FordNotPassable

S1 SetupControlMeasures sq5_ReconToRouteFlank sq6_ConductTravelingOverwatch sq8_VisuallyClearObstacle-FarSide

S4 sq5_FordLooksPassable S5 SetupReport&NewControlMeasures sq5_MoveToCover/ConcealPosition sq6_ConductNearSideOverwatch sq8_VisuallyClearObstacle-FarSide

ConductRightFlankRecon

BEHAVIORGENERATION

STATUS

STATUS

NEXTSUBGOAL

.COMMANDEDTASK (GOAL)

DriveOnTwoLaneRd

PassVehInFront

PassVehInFront

DriveOnTwoLaneRd

NegotiateLaneConstrict

PassVehInFront . STATE-TABLES

WaterDepthToSixFeet

WaterCoveredLand

MajorGroundDeformation

TractionSlip

LegalToPassMarshDetectedSomeWaterVisible

Mosses,Evergreens,andShrubs

StagnantWater

IndeterminantGroundLevel

SignificantTractionSlip

OrganicMaterialOnWaterSurface

BogDetected

ErodedEarthEmbankments

FlowingWater

NonVegetatedWaterInMiddleNarrowWidth,IndeterminantLength

StreamDetected

ErodedEarthEmbankments

FlowingWater

NonVegetatedWaterInMiddleSignificantWidth,IndeterminantLength

RiverDetected

WaterSheenOnGroundSurface

LittleToNoVegetationSignificantTractionSlip

RuttedWithStandingWater

MudDetected

SignificantGroundDeformation

ExtensiveGrassyVegetation

LongLeafGrasses - very flat, long green leaves-purple/rose/yellow flowers.

Reeds - tall, woody, thin, round, hollow jointed (tan-to-green) stem plants, long narrow green blade leaves, large feathery panicles (elongated clusters of tan/white/purple flowers along main stem).

Sedges - triangular tan/green stem plants, papyrus, narrow green to tan grass- like leaves, spikelets of inconspicuous tan-to-yellow-to-white flowers.

Bulrushes - tall tan-to-green stems, with dark brown cylindrical seed heads that explode into white down, long flat green sword shaped leaves, cattails.

Water Lilies - very large floating green leaves with white flowers.

SaturatedGround

Six Feetof Water

PlantHeight

6-18"

6-48"

1-6'

3-9'

Float

LargeSurfaceAreaOfStillWater

LargeAreaWithoutGrasses,Trees,ShrubsOrganicMaterialMayBeOnWaterSurface Pond/LakeDetectedBoundedBySwamp,Marsh

MostlyTrees,SomeBushes

SlowMovingWaterCoveredLandSignificantTractionSlip

ExtensiveWaterSurfaceVisibleSwampDetected

MajorGroundDeformation

WaterObstacleDetected

15 cm

2.4 cm .9 to 2.7m

3 cm

DeployToStartPointConductMainRouteReconEvaluate&ClassifyObstructionMoveToControlPoint

Interchange Formats andUpper Ontologies

• OWL– Neutral (W3C) interchange format– XML base enables use XSLT transforms– Provides access to emerging semantic web technologies

• OWL-S– Rich semantics for describing complex processes (without

being too complicated)– Well suited to agent architectures

• Pieces of SUMO (Suggested Upper Merged Ontology)– Class structure and properties provide a good starting point for

developing domain specific ontology– Native KIF format too complex for target community and not

necessary for requirements capture• Namespaces

– Used quite a bit to make ontology more manageable

IGV Conceptual ModelService

TroopCommander

Agent

ServicePlatoonLeaderAgent

ServiceSectionLeadAgent

ServiceVehicle

CommanderAgent

ServiceMobilitySystemAgent

ServicePropulsionSubsystem

Agent

ServicePlatoonLeaderAgent

ServicePlatoonLeaderAgent

ServiceSectionLeadAgent

ServiceSectionLeadAgent

ServiceVehicle

CommanderAgent

ServiceVehicle

CommanderAgent

ServiceSurvivability

SystemAgent

ServiceSurveillance

SystemAgent

ServiceLocalizationSubsystem

AgentService

AuxiliarySubsystem

Agent

EngineComponent

TransmissionComponent

External Service Request by a process

ServiceLethalitySystemAgent

ServiceAutomotiveSubsystem

Agent

ServiceSupportSystemAgent

ServiceNavigationSubsystem

Agent

Track-DriveComponent

BrakeComponent

Representing an IGV (cont.)

Tactical BehaviorsPlan State-Table Selection

ScoutPlatoon

3rd

B Section

HMMWV#5

HMMWV#8

HMMWV#6

A Section

Squad/VehHMMWV #7

Squad/VehHMMWV #3

Squad/VehHMMWV #2

C Section

HMMWV#10

HMMWV#9

HMMWV#4 PSG

ConductRightFlankReconConductRouteReconToBridgeConductLeftFlankRecon

Recon toRightFlank

Conduct Traveling Overwatch

Visually ClearObs Far-Side

Visually Clear Route

Conduct Traveling Overwatch

Conduct Recon ToCP

Conduct Traveling Overwatch

Conduct Route Recon To CP

Support Route Recon

NoBypassOnLeft

ConductRouteReconBridgeDetected

ConductWaterBridgeRecon

ConductWaterObsRecon

WaterObsDetectedBridge

Detected

SENSORYINPUT

GENERIC 4D/RCSAGENT

CONTROL MODULE

SENSORYPROCESSING

KNOWLEDGEDATABASE

BEHAVIORGENERATION

WORLDMODEL

VALUEJUDGMENT

STATUS

STATUS COMMANDEDTASK (GOAL)

COMMANDEDSUBGOALS

BEHAVIORGENERATION

STATUS

STATUS

NEXTSUBGOAL

.COMMANDEDTASK (GOAL)

DriveOnTwoLaneRd

PassVehInFront

PassVehInFront

DriveOnTwoLaneRd

NegotiateLaneConstrict

PassVehInFront . STATE-TABLES

Gear ChangeRequired

Commanded VelocityPositive But Gear Is In

Reverse

Commanded VelocityNegative But Gear Is

In Forward

(or)

Commanded VelocityPositive(AND)

Gears In Reverse

Commanded Velocity Negative(AND)

Gears In Forward

New StartupAndOperateCommand S1 proc_StartEngine

S2 S1 EngineStarted

S2 GearChangeRequired S3 proc_ChangeGear

S3 GearChanged S2

S2 NewCommandedVelocity S4 proc_AdjustEngineThrottle

S2 S4 EngineThrottleAdjusted

S2 ShutDownRequested S5 proc_SetGearToPark

S5 GearInPark S6 ShutDownEngine

StartUpAndOperate

Input Conditions Output Commands

S6 EngineShutDown S0 Done

New StartupAndOperateCommand S1 proc_StartEngine

S2 S1 EngineStarted

S2 GearChangeRequired S3 proc_ChangeGear

S3 GearChanged S2

S2 NewCommandedVelocity S4 proc_AdjustEngineThrottle

S2 S4 EngineThrottleAdjusted

S2 ShutDownRequested S5 proc_SetGearToPark

S5 GearInPark S6 ShutDownEngine

StartUpAndOperate

Input Conditions Output Commands

S6 EngineShutDown S0 Done

Representing a Propulsion Service

Propulsion Service Graph

More Visualization Features

Conditions

ScoutPlatoon

3rd

B Section

HMMWV#5

HMMWV#8

HMMWV#6

A Section

Squad/VehHMMWV #7

Squad/VehHMMWV #3

Squad/VehHMMWV #2

C Section

HMMWV#10

HMMWV#9

HMMWV#4 PSG

ConductRightFlankReconConductRouteReconToBridgeConductLeftFlankRecon

Recon toRightFlank

Conduct Traveling Overwatch

Visually ClearObs Far-Side

Visually Clear Route

Conduct Traveling Overwatch

Conduct Recon ToCP

Conduct Traveling Overwatch

Conduct Route Recon To CP

Support Route Recon

NoBypassOnLeft

ConductRouteReconBridgeDetected

ConductWaterBridgeRecon

ConductWaterObsRecon

WaterObsDetectedBridge

Detected

SENSORYINPUT

GENERIC 4D/RCSAGENT

CONTROL MODULE

SENSORYPROCESSING

KNOWLEDGEDATABASE

BEHAVIORGENERATION

WORLDMODEL

VALUEJUDGMENT

STATUS

STATUS COMMANDEDTASK (GOAL)

COMMANDEDSUBGOALS

BEHAVIORGENERATION

STATUS

STATUS

NEXTSUBGOAL

.COMMANDEDTASK (GOAL)

DriveOnTwoLaneRd

PassVehInFront

PassVehInFront

DriveOnTwoLaneRd

NegotiateLaneConstrict

PassVehInFront . STATE-TABLES

New StartupAndOperateCommand S1 proc_StartEngine

S2 S1 EngineStarted

S2 GearChangeRequired S3 proc_ChangeGear

S3 GearChanged S2

S2 NewCommandedVelocity S4 proc_AdjustEngineThrottle

S2 S4 EngineThrottleAdjusted

S2 ShutDownRequested S5 proc_SetGearToPark

S5 GearInPark S6 ShutDownEngine

StartUpAndOperate

Input Conditions Output Commands

S6 EngineShutDown S0 Done

Gear ChangeRequired

Commanded VelocityPositive But Gear Is In

Reverse

Commanded VelocityNegative But Gear Is

In Forward

(or)

Commanded VelocityPositive(AND)

Gears In Reverse

Commanded Velocity Negative(AND)

Gears In Forward

Gear ChangeRequired

Commanded VelocityPositive But Gear Is In

Reverse

Commanded VelocityNegative But Gear Is

In Forward

(or)

Commanded VelocityPositive(AND)

Gears In Reverse

Commanded Velocity Negative(AND)

Gears In Forward

IGV Condition Example

Model Development Status

• OWL entities defined– Classes 175– Properties 130– Instances 700

Issues and Lessons Learned• Developing an ontology is a slow iterative process

– It difficult to evaluate a model construct without inputting detail. – It is very difficult to change the model once you have entered

any level of detail.

• Difficult to develop consistent rules for when to use a Classes vs. an Instance in a large domain– Is knowledge in class restrictions or instances?

• Difficult to present large models to domain experts• Experiences with OWL-S shows that it has applications

outside of the semantic web.– Would like to get involved in its development