A Journey to the Center of the Clouds...PROTOTYPE TEST PO and team representatives Research...
Transcript of A Journey to the Center of the Clouds...PROTOTYPE TEST PO and team representatives Research...
1A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 201814th Annual SEI Architecture Technology User Network Conference
MAY 7–10, 2018 | PLANO, TEXAS
A Journey to the Center of the CloudsJørn Ølmheim
2A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Who am I?
• Polyglot programmer• Practicing architect• Open source enthusiast• Amateur guitar player• Overall geeky dude• … and so much more …
3A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Agenda
• What is the cloud?
• Statoil’s cloud gamble
• The Cloud Journey – Architecture
• Pilot project architecture
• Migrating an existing application
• 3rd party cloud integration
• Enterprise Data Modelling
• A new generation of apps
• Learnings
4A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
5A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Cloud basicsJust the short version.
6A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Cloud basics
Cloud Characteristics:Self-service
Network access
Multi-tenancy
Elasticity
Measured
Service Models:Infrastructure as a Service (IaaS)
Platform as a Service (PaaS)
Software as a Service (SaaS)
Read more here >>
7A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Our cloud strategy
Statoil has a Cloud First strategy. All new IT solutions, software or computing capabilities (includingchanges to existing ones) shall be evaluated to be a cloud service.
Not "We must use cloud", but "We always consider cloud and checkif it benefits Statoil"
From“The master of IT Operation & Maintenance”
To“The master of Data & IT Service consumption”
8A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Cloud benefits
Start-up
•No CAPEX•Less deployment time
Operation
•Scalability & Pay per use
•Collaboration with partners
•Multiplatform / mobility included
•Upgrades included
Exit
•Exit by ending subscription
•No hardware involved
SpeedAgility
Improved utilisation of IT personnel and assets
9A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Cloud risksInformation
security
Contracts / Terms and conditions
«Shadow IT»
Exit serviceData:• Personal information• Export control• Geodata
Customisation
10A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
A Journey to the Center of the Clouds
Statoil’s bold cloud gamble
11A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Exp D&W Corporate Vessel Refine-
ries TradingO&MPetec
From Silos of Data…..
12A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Exp D&W Corporate Vessel Refine-
ries TradingO&MPetec
From Silos of Data….. to one common Data Platform for the Statoil Value Chain
Statoil Data Platform Orchestrate all Statoil Data
From Silos of Data…..
13A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
SSDLThe Vision
Enable Statoil subsurface to become a fully data-driven company. A data-driven company is an organisation where every person can use data to make better decisions. All personnel have access to the data they need when they need it.
Find and produce more profitable oil and gas by using the Data Lake to combine knowledge and experience with data and interpretations to unlock potential, improve understanding and create value
• A system to access all data in existing subsurface discipline silos
• Make all data available for the user to run analysis on• Something beyond a Hadoop based storage platform
Geosciences
Supply chain
From TSB2025D – Subsurface Data Lake
14A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Statoil Data Platform Orchestrate all Statoil Data
Cloud Platforms
Petec: Landmark
Drilling: Kongsberg
ERP: SAP
Equipment Monitoring: GE Predix / SiemensExp: Schlumberger
Maj
or P
latf
orm
Par
tner
s
From Silos of Data….. to one common Data Platform for the Statoil Value Chain
15A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
A Journey to the Center of the Clouds
The Cloud Journey - Architecture
16A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
500 GB every day - per turbine
150.000 readings - per secondHistorical data - for 40 years
17A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
1 GB every 10. second32 TB per survey
Every 3. year => 2x per year
18A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SchlumbergerHaliburton
…..Data platform
Statoil Data Platform – Overall platform approach
IIoT,
Pla
nt &
R
ealti
me
Dat
a
External Connection
Advanced Analytics
Business SpecificSystem
Business SpecificSystem
LDWMobileWeb Bots
Business SpecificSystem
Data Streaming
Statoil Current IT - On premise & IaaS
Data Virtualization
Data Platform Storage
Data replicationAPI
Statoil Data Platform
Sources:
• DBR• SAP• Subsurface • datamarts• Endur• ++++++
Automated Batch Loading
Business SpecificSystem Software as a Service, in the cloud
Business SpecificSystem
SAP…..
Data Platform
E,P,C,IProjects
Data Plaform
Portals
APIAPIAPIAPI
FileSystem
Parse / replicate
20A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
A Journey to the Center of the Clouds
Pilot project architecture
21A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
OPENWORKS EDM LANDMARK WellDB
DSIS
Data lake
/raw/corporate/WellDBEDM LANDMARKOPENWORKS
/transform/corporate/
/data/subsurface/
WellDB
dataplatform-data
Subsurface data API
CSD DBR
CSD DBR
CompositeLog
FormationPressure
CPILog
DepthFormationPicks
CompletionIntervals
CasingDepths
Wellpaths
CompletionData
Dashboards and web applications
RECALL
FMU
UNIX file share
Fault lines
Outline
Maps structure
Master data API
FieldWellWellboreStratigraphy
OPENWORKS EDM LANDMARK
ASC
LASDLIS
PerforationData
CompletionString
Perforationdata
WellOperations
CasingDepths
PerforationDepths
22A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
A Journey to the Center of the Clouds
Migrating existing applications
23A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
Cloud migration short term
24A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
External SaaS integration
25A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Cloud migration long term
26A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
A Journey to the Center of the Clouds
3rd party cloud integration
27A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Cloud future vendor landscape
28A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
3rd party SaaS Integration
29A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
A Journey to the Center of the Clouds
Enterprise Data Modelling
30A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
Some data challenges
• No common overview of available data assets.
• Data in multiple sources (< 2500 internal systems), but no common insight into which data sits where.
• Data needed across domains, but difficult to know who to contact and to combine data across different data ecosystems.
• Lack of common terms and definitions of cross-domain key data objects.
• Users spend more time looking for data, than they do analyzing it.
• Users are busy re-producing data that already exists.
31A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
Share and integrate
Enterprise Data Management (EDM) toolboxTo find, understand, share, use and govern data assets
Describe and organize Govern and implement
• Business Information Inventory (BII)
• Data definitions in Business Glossary
• Data and information models
• Statoil Data Catalog
• Enterprise Data Model
• Metadata model
• Roles and responsibilities
• Organization and resources
• Processes and policies
Data vocabulary
• International standards
32A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
Business glossary
Business object
=========
=========
=========
=========
=========
Enterprise Data Model
33A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
Enterprise Data Model v1
34A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Domain Driven Design – Strategic Design
Context Map
Continous Integration
UbiquitousLanguage
SharedKernel
PublishedLanguage
AnticorruptionLayer
Customer/SupplierTeams
Conformist
Open HostService
Bounded Context
SeparateWays
CoreDomain
Domain VisionStatement
GenericSubdomains
SegregatedCore
HighlightedCore
AbstractCore
35A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
Strategic design: Context maps
In large systems (or set of systems), we need a map to give us a picture of the models that are inside.
Bounded context
Bounded context
Context map
Relations
36A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
A Journey to the Center of the Clouds
A new generation of apps
37A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
TESTPROTOTYPEIDEATEEMPATHIZE DEFINE
PO and team representatives
Research (preferablyonsite)
Define the coreissue to be solved
PO and team representatives
Iterate over MVP prototypewith actual business user
UX Lead - Team
Actual business user
Team input and experiments
Alignment on way forward
Interactive prototype
Solves core issue?
DESIGN THINKING
38A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
SATURN 2018
Learnings
• Getting data out of the legacy systems is not as difficult as we initially expected• Some legacy formats can provide a challenge
• The real challenge is to agree on the apis and data models• Existing models mostly dictated by vendors• Application specific vs Discipline specific vs Cross-discipline models
• Bottom-up vs Top-down design of apis and storage
• Introduce DDD concepts to enterprise data modellers
• Building apps to solve specific real user needs using Design Thinking
39A Journey to the Center of the Clouds© 2018 Carnegie Mellon University
A Journey to the Center of the Clouds
Thank you!