Frontiers in Computer Vision for Drilling CoVar · Applied Technologies © 2015 CoVar Applied...

Applied Technologies © 2015 CoVar Applied Technologies, Inc. All Rights Reserved.

Frontiers in Computer Vision for Drilling

CoVarKenneth D. Morton PhD


CoVar• Our Goal:

– Provide cutting-edge machine learning solutions to help industries improve safety and efficiency, provide new insights and capabilities, while saving money

• This talk:– Video processing technology

demonstrations to improve rig safety and efficiency

VideoStructured

Information

2


Why Video?• Video cameras:

– Inexpensive, widely deployed– Commercial-off-the-shelf technology

• Powerful tool for many problems where classical instrumentation is difficult or costly

• Video provides massive amount of unstructured information

• Computer Vision Algorithmsautomatically extract information from video data for safety or efficiency improvements on the rig– Note: Video is a complementary

source of information in addition to existing sensors; not a panacea

3


Video Processing Technology Application Areas• Personnel Video Monitoring

• Muster Point Counting

• Shaker Table Fluid Front and Particle Tracking

• Rig Microstate Identification and Tracking

4


Personnel Video Monitoring - Motivation• The drilling rig is a dynamic, rapidly

changing environment• Increased automation is key to

improving safety and efficiency– But automation comes with its own risks

• Many pieces of information required for safe automation are difficult to obtain

– Difficult or expensive to instrument, require user cooperation

– E.g., transponders – require user action• Lots of information from visual

interpretation of a scene– How to automate?

• Safety Alert 58:

Ensuring safety controls enable workers and drillers to confirm that the path of the iron roughneck is clear of personnel

5


Personnel Video Monitoring (PVM)• Goal

– Prevent machine-human collisions

• Using pre-existing sensors

• No personnel actions required

• Technical challenges solved– Infer locations of

multiple people in a scene from a set of monocular cameras

6

Inferred Personnel &Equip MapPerson 1

Person 2

Person 3


PVM Under the Hood• Person detection

– Poor off-the-shelf person detection performance for some camera angles

– Developed classification techniques and rapid training procedures

• Person location– Via triangulation– Person detections in

images (image space)– Camera transformation

information (image world transformation)

– Multiple cameras

7

Person LocationPerson Detection


PVM Video

8


• Goal: Automate muster point roll-call with automated personnel counting and identification

Muster-Point Personnel ID

Example Video of Automatic Person MatchingExample video of non-duplicative personnel counting

9


Shaker Table Analysis• Leverage video to track fluid front,

losses off table, fluid volume, solid/fluid ratio

• Perform discrete object detection & tracking with probabilistic filtering methods– Enables automatic tracking of hole

cleaning and cuttings volume (with calibration)

– Enables automatic localization of the fluid front, for feedback into automatic control mechanisms

10


• Must related pixel measurements to real-world measurements

• Estimate camera transform, map to world coordinates

• Note: example video is hand-held, shaky

11

Estimating Pixels vs. Estimating Distance

Applied Technologies © 2015 CoVar Applied Technologies, Inc. All Rights Reserved. 12

Fluid Front Tracking


Solids Analysis

13


Rig Microstate and Pipe Tally• Data collected on board an

active rig with 1hr of tripping-out-of-hole

• Camera relatively stable, has clear view of area around well-bore

• Goals:– Can we automate pipe-tallying

– automatically count number of pipes to enter/exit the hole?

– Can we measure other variables of interest? E.g., pipe velocity, pipe length (integral of velocity over time)

14


Requires: Rigid Body Detection• Goal:

– Detect three types of rigid bodies in scene

– Pipe over hole, roughneck engaged, grabber

• Only grabber has significant motion within scene

• Three regions shown:– Blue: pipe detection region (tall, narrow)– Orange: grabber detection region– Yellow: Roughneck engaged detection

region

15


Rigid Body Detection & Classification

• CoVar proprietary processing for automatic rigid body detection in videos

• Enables rapid re-training of new rigid body classification algorithms

16


• To make reliable inferences from observed data, need to use a logical formalism

• What enables a pipe to be removed from hole?

– 1) Grabber– 2) Pipe– 3) Roughneck

• As roughneck disengages, the pipe is still in scene – but this is the same pipe; we do not want to double-count the pipe

• FSM enables logical system:– After roughneck engaged and

disengaged, and pipe not detected, increment tally, do not increment tally further until grabber detected

Pipe Counting System: Finite State Machine

Initial State /

No State

Grabber Detected

Pipe In Scene

Pipe Disconnecting

Grabber detection flag On

Grabber detection flag Off, & Pipe detection flag OnTrack pipe velocity

Roughneck detection flag OnRoughneck flag

OffPipe detect flag OffIncrement pipe tally

17


Example Video; Pipe Tally

18


Automatic Aggregate Action Summary• Automatically• export information, with

pictures/clips, to arbitrary database format

• (E.g., SQL, EXCEL)

• Intelligent data down-sampling enables massive reduction in data storagerequirements

• Very fast, simple access to short video clips around interesting events

19


• Incorporate tool-joint detection

– Count # of pipes comprising pipe stand

• Various nomenclature fixes – Connector Grabber

• Detect other, rare events – e.g., Mud bucket in scene

• Automatic data compression – only store “interesting” events

– Can reduce data storage requirements by orders of magnitude

Moving Forward

Tool Joint Detected: Pipe Stand Component #2

19


Conclusions• Video is an inexpensive sensing modality that can be leveraged

when classical instrumentation is difficult or costly but…• Video provides a massive amount of unstructured information• CoVar’s computer vision algorithms automatically extract

information from video data for safety or efficiency improvements on the rig

21

VideoStructured

Information


Backup


Simulated Rig PVM

23


Leveraging Previous Experience• Previous work in

several DoD funded application areas– Real-time trip-wire

detection in first generation digital night-vision goggles

– Algorithms for road-cataloguing for potential IED detection

Tripwire Detection

24

Urban Change Detection


Pipe Handling and Estimation• Goal: Automate pipe handling and

pipe tallies• CoVar has previously explored

various approaches to video processing for pipe management

– E.g., pipe matching, size/angle estimation; left

• Potential Applications:1. Measure pipe diameters and

lengths as they enter hole and automate drill string pipe tally

2. Count pipe joints during tripping into or out of hole

3. Measure tripping velocity into or out of hole

4. Automate pipe stabbing localization and rotation

Example Video of Automatic Pipe Localization

Example Video of Automatic Pipe Tracking

25


Heave Measurement Real rig example measuring and tracking objects Example: Estimating relative location and angle of pipe, slip joint


Leveraging Commercial Technology

• Can we leverage monocular video data to automatically drive a car through a simulated environment?

• Technology: Map and speedometer information extraction, automated driving feedback control loops

27

Automated Driving

Frontiers in Computer Vision for Drilling CoVar · Applied Technologies © 2015 CoVar Applied...

Documents

Transcript of Frontiers in Computer Vision for Drilling CoVar · Applied Technologies © 2015 CoVar Applied...