Sharings from some RnD projects

Automation in Construction Safety and Health :

City University of Hong Kong 2018

Dr. Ivan Fung Assistant Professor, Department of Architecture and Civil Engineering

Why need Automation ?

Why need Automation ?TimeCost QualityQuantity………

> OSH: Construction Safety & Health?

New things on Construction Safety…

Automation =Evolution +

Integration +Optimization +

Ubiquitous

RnD Ideas/ Projects

RnD #1 : AI x BIM (CIM) BIM will be extended to Level 4 and to be more Smarter!

RnD #1 : AI x BIM (CIM)

RnD #1 : AI x BIM (CIM)

RnD #1 : AI x BIM (CIM)

RnD #1 : AI x BIM (CIM)

RnD #1 : AI x BIM (CIM)

RnD #1 : AI x BIM (CIM)

RnD #1 : AI x BIM (CIM)

RnD #1 : AI x BIM (CIM)

RnD #2 : Real-time Monitoring for Construction Workers by Far-field Automated Surveillance Network Camera & UAV

WWW.PPTSTORE.NET

YOUR COMPANY

Findings on Literature Review 

Principles• Background modeling, track management, object recognition and light trackingDirectly track human and vehicles, or track any moving objects and focus onto itApplications• Not mature in construction industry• Junction Electronic Eyes to monitor the traffic conditions 

Far‐field Automated Surveillance Network Camera 

Unmanned Aerial Vehicles (UAV)

• Inspection and monitoring• Surveying and mapping• Condition Survey & civil engineering

• Aerial imaging (HR photos & stills)• 3D scanning

WWW.PPTSTORE.NET

YOUR COMPANY

01Findings on Literature Review 

• A set of data are collected at first and allow processing and computing

• Allow computer to learn by examples• Allow multiple processing layers to compute classification 

directly from images, text or sound• Deep means the no. of hidden layers in neural network• More than 150 hidden layers

Deep Learning Method

WWW.PPTSTORE.NET

YOUR COMPANY

Findings on Literature Review 

Deep Learning Method

❖ Mainly process 2D data like images

❖ Directly extracts features from the image 

❖ Speed up and simplify R‐CNN 

❖ Joint extractor, classifier and regressor to unified framework and form RoIPool

❖ Regions with CNN features❖ Input an image ❖ Extracts 2000 bounding bo

xes❖ Extract the features for eac

h bounding box ❖ Classify the regions

❖ Deep fully convolutional network and a Fast R‐CNN detector

❖ Add Region Proposal Networks (RPNs)

❖ Reach real‐time

CNN Fast R‐CNN 

Faster R‐CNN 

R‐CNN 

WWW.PPTSTORE.NET

YOUR COMPANY

12Preliminary Design System of TRACE

First trial to use UAV (23/8/2017)

• Identify the limitations of UAV in different environment and construction site

• Select appropriate type of UAV for construction site

Second trial to use UAV, with Dr. Ivan Fung and Mr. Peter Lai (20/1/2018)

WWW.PPTSTORE.NET

YOUR COMPANY

WWW.PPTSTORE.NET

YOUR COMPANY

01Preliminary Design System of TRACE

• Identified the technical requirement and resources for TRACE• Worked on the preliminary design and development of application prog

ramme• Experienced on the usage of existing far‐field automated surveillance ne

twork camera with Mr. Peter Lai and Mr. Sam Huang, Hikvision Oversea Pre‐sales Engineer (22/1/2018)

• Identified the technical problems and limitations of existing surveillance network camera

05

TRACE Prototypes• Allow automated real‐time PPE tracking around const

ruction site and safety monitoringEnsure all workers are well‐equipped with suitable PP

E and permit all‐time safety monitoringIdentify worker who is malpractices and safety warnin

g will be given to the safety officers or supervisors through mobile devicesImprove the traditional visual checkingHighly relief the burden of safety officers

WWW.PPTSTORE.NET

YOUR COMPANY

12Basic setup of TRACE

WWW.PPTSTORE.NET

YOUR COMPANY

Steps for the development of TRACE

❖ Collect photos and videos 

❖ Different posture and movement of workers

❖ With/without PPE

❖ Processing of data

❖ Designed for Deep Learning Method

❖ Carry Faster R‐CNN training 

❖ Collect photos and videos 

❖ Different posture and movement of workers

❖ With/without PPE

❖ Alert the safety officers 

❖ Allow warning to worker in real‐time

❖ Correctness of output and modification of program

Collect relevant Data

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

Design Algorithm for the system

Train the system by Deep Learning Method

Design and program a mobile app for notifying the safety officers 

Testingand modification

WWW.PPTSTORE.NET

YOUR COMPANY

12TRACE ‐ Proposed applications

❖ Address all potential hazards Bamboo scaffolder does not wear 

safety belt when working at height

Site Safety Inspection on PPE

❖ Identify any action or movement of workers prohibited

Worker is holding a cigarette with his finger and place it onto his mouth, the system will be alerted

Site Safety Inspection on Violation of regulations or rules

❖ Construction site condition can be shown on the mobile devices in real‐time

❖ Directly reach the area or worker for improvement

Site Safety Inspection outside the construction site

WWW.PPTSTORE.NET

YOUR COMPANY

12TRACE ‐ Proposed applications

❖ Hikvision DS‐TCG225❖ Identify the model, licenses, color of vehic

les Record the entry of concrete truck Ensure the workability of concrete

Identify and record the vehicles enter to the construction site

❖ Neither recorded workers or visitor❖ minimize unwanted loss of property

❖ Identify the type and application of construction plant using in different zone

1 bulldozing machines in zone 2

Ensure no people or equipment within the safe zone

No people surrounded in a radius of 1m

Ensure the machine operator is authorized

Monitor and record the usage of construction plants or equipment

Prevent unauthorized people from entering the construction site

RnD #3 : Safety Training & Inspection using MR Technology

Reasons why MR Train is impactful

Consultations with Professionals

Selection of Equipment Equipment for projecting training scenario

MR CAVE RoomHead Mounted Displays (HMDs)

Selection of Equipment Senses Controller HMDs can only provide sight and hear sense Functions: add sense of touch to the visual interface Examples: Haptic Gloves Haptic Shoes

Hand Omni (Rice University, 2015) Taclim VR shoes and glov(Cerevo Inc, 2017)

Selection of Safety Training Scenarios

Type of loadshifting Machinery Bulldozer Compactor Loader Excavator Grader Locomotive Dumper Fork Lift - Commonly used in construction sites

- To lift and move material short distance- Cause serious injury and death

Framework of MR Train for Forklift

6. Finish the course

5. Practical Test

4. Safety Procedures (e.g. inspection and operation)

3. Hazards from operating loadshifting machinery

2. Features of the loadshifting machinery

1. Introduction of Relevant Legislation

Development of Scenario Goal: To create a 360 – degree scenario showing all features of forklift

Development of Scenario

Taking photo by Ricoh Theta S

Photo by Ricoh Theta S

Two special functions

Voice Navigation Magnifier

Development of Scenario Prototype of Session 2 – Features of Forklift

Development of Scenario Session 4 – VR Self Learning Purpose: to understand safety inspection before operation Safety operation

Feasibility Study of ImmersiveMixed Reality (MR CAVE) in RemoteConstruction Safety and Risk Assessmentat Distant Work Sites

MR CAVE Prototypes• Communication network system

– MR CAVE system must be to connect with the trolley on site area. The com

munications network can be the Internet or more precisely the WWW.

The End