RESTful CIM using the Open Data Protocolcimug.ucaiug.org/Meetings/eu2018/2018 Ljubljana...OGOpen...

Open Grid SystemsOGO

RESTful CIM using the Open Data Protocol

!1

Alan McMorran B.Eng Ph.D


CIM Exchanges!2

Open Grid SystemsOGO www.opengrid.com

CIM Data Exchange• The exchange of data using CIM has traditionally been

used in two primary ways:

• Full Model exchange as CIM RDF XML

• Message based exchange as CIM XSD

• CIM RDF XML and CIM XSD cover two common use-cases:

• CIM RDF XML supports Network Model Management (NMM) exchanges both within and between companies and organisations

• CIM XSD exchanges support intra-system exchanges using an Enterprise Service Bus and/or web services

!3


Hierarchical vs Graph

!4


Hierarchical vs Graph• RDF is very well suited to

non-directed graph data

• XSD is better suited for hierarchical data

• Electrical network data rarely fits neatly into a hierarchy

!5


CIM XSD Standards• The IEC 61968 series of standards is primarily

focussed on defining CIM XSD based messages

• These messages are explicitly defined hierarchies that can be used by Web Services

• Examples include:

• IEC 61968-3 Network Operations

• IEC 61968-6 Maintenance & Construction

• IEC 61968-9 Meter Reading and Control

!6


Message Payloads• These standards define

fixed message payloads of hierarchical CIM data

• These payloads are defined within the standard and implemented by vendors

• They are combined with standard verbs (CREATE, UPDATE, DELETE etc.)

!7


Services• This means that every payload must be defined and

agreed between all parties

• This approach has scalability issues

• As more parties connect; more services are interconnected; and more CIM data is being shared a more flexible approach is needed

• Systems should be capable of sharing CIM data without requiring a full system export or all parties to agree a fixed subset and hierarchy for every possible service

!8


RESTful Services!9


What is RESTful?

• “Representational State Transfer (REST) is an architectural style that defines a set of constraints and properties based on HyperText Transfer Protocol (HTTP)"

• “REST-compliant web services allow the requesting systems to access and manipulate textual representations of web resources by using a uniform and predefined set of stateless operations”— Wikipedia

!10


RESTful Principles• All resources are uniquely addressable, usually

through Uniform Resource Identifiers (URI)

• All resources can be manipulated through a constrained set of well-known actions, usually CRUD (Create, Read, Update, Delete)

• The data is transferred through any of a constrained number of well-known representations (e.g. HTML, XML, JSON)

• The communications between client and application are stateless allowing for multiple layers of intermediaries, caching etc.

!11


RESTFUL Example• Read (HTTP GET)

!12

HTTP GET https://opengrid.com/myservice/001

{ "type":"example", "name":"get", "id": "001", "company":"Open Grid Systems", "web":"https://www.opengrid.com"}

Result

<No Content> Payload


RESTful Example• Create (HTTP POST)

!13

{ "type":"example", "name":"post", "company":"Statnett", “web":"http://www.statnett.no"}

HTTP POST https://opengrid.com/myservice

Payload

{ "id": "002",}

Result


RESTful Example

!14

{ “web”:”http://www.statnett.no/en/"}

HTTP POST https://opengrid.com/myservice/002

Payload

Status: HTTP 204 No Content Result

• Update (HTTP POST)


RESTful Example

!15

<No Content>

HTTP DELETE https://opengrid.com/myservice/002

Payload

Status: HTTP 204 No Content Result

• Delete (HTTP DELETE)


RESTful Benefits• All communications go through a single uniform

interface

• With HTTP protocols and JSON/XML payloads the services can be accessed by any modern language

• Simple, easy to use architecture

• Web Browsers use RESTful services over HTTP

• Standard verbs, response codes, data types etc.

• Flexible service endpoint structures

!16


Challenges• This flexibility allows for different but valid URI patterns:

https://opengrid.com/myservice/002 https://opengrid.com/myservice?id=002 https://opengrid.com/myservice(id='002')

• All valid URI structures

• The payload formats are commonly JSON or XML but there is no restrictions

• The contents of the payloads can follow any structure

• For interoperability, these must be restricted and standardised

!17


OData!18


OData Background• The Open Data Protocol (OData) is a protocol standard

that defines a set of best practices for building and consuming RESTful APIs

• OData was initiated by Microsoft in 2007 and since version 4.0 has been under the OASIS OData Technical Committee

• This was submitted to the ISO/IEC as an international standard

• Allows the creation and consumption of REST APIs using meta-data driven URLs and and standard HTTP protocols and messages

!19


OData Enhancements• Standard URI structure for RESTful service endpoints

• Standard Query format

• Supports client-selectable JSON or XML formats for payloads/results

• Supported by existing BI and office applications (including Excel and Sharepoint)

• OData libraries for .NET, Java, Javascript, PHP, C++ (but can be access as plain HTTP RESTful service without a library in any language that supports HTTP)

• Fully meta-data driven with an Entity Data Model (EDM)

!20


Entity Data Model• Standard UML components map one-to-one to the Entity

Data Model

!21

Package Schema

Class EntityType

SuperClass BaseClass

Attribute Property

Association NavigationProperty

UML EDM

• Full support for data typing (e.g. String, Boolean, Floating Point, Integer) that map to XSD types


RESTful with MetaData• This provides a fully meta-model driven RESTful

endpoint for managing data

• The CIM can be fully mapped into the EDM including all inheritance, associations, attributes and data types

!22


Extended Payloads• Synchronous Machine as plain JSON

!23

{ "UUID":"_f4852cc6-ca12-11e7-b499-4ee79879b30e", "name":"Turner V1 SM", "p":0.0, "q":0.8, "maxQ":50.0, "minQ":-17.0, "operatingMode":1, "type":1, }

{ "@odata.context":"http://localhost:8080/cimphony/concert/db/resources.svc$metadata#Cimphony.Concert.CIM16A.IEC61970.Base.Wires.SynchronousMachine/$entity", "@odata.type":"#Cimphony.Concert.CIM16A.IEC61970.Base.Wires.SynchronousMachine", "@odata.id":"ConcertResource(ID='6f4c8710-6995-11e8-b5cf-96b18064619b')/Contents/Cimphony.Concert.Resource.CIM16A/SynchronousMachine('_f4852cc6-ca12-11e7-b499-4ee79879b30e')", "@odata.editLink":"ConcertResource(ID='6f4c8710-6995-11e8-b5cf-96b18064619b')/Contents/Cimphony.Concert.Resource.CIM16A/SynchronousMachine('_f4852cc6-ca12-11e7-b499-4ee79879b30e')", "UUID":"_f4852cc6-ca12-11e7-b499-4ee79879b30e", "name":"Turner V1 SM", "p":0.0, "q":0.8, "maxQ":50.0, "minQ":-17.0, "[email protected]":"#Int32", "operatingMode":1, "[email protected]":"#Int32", "type":1, }

• Synchronous Machine as JSON with extended meta-data


Meta-Data Defined Services• The EDM is used by OData aware clients and servers to

validate communications

• Clients can request the EDM from the endpoint in a standard format to perform auto-discovery and client-side validation

• Servers will validate all endpoint URIs and service payloads against the EDM

• This allows for meta-data driven validation of data at the interface before it reaches the application itself

!24


OData Example• OData abstracts the interface from a particular format

so can provide XML or JSON representations

• Endpoints can also be configured to returning other formats e.g. SVG, PNG or RDF XML through an OData interface

• The standard RESTful URL structure is all based on the meta-model:

!25

http(s)://<host>/cimphony/concert/db/management.svc/ ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/ Contents/Cimphony.Concert.Resource.CIM15/ SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')

!26

{ "@odata.context":"http://localhost:8080/cimphony/concert/db/management.svc$metadata#ConcertResource", "value":[ { "@odata.mediaContentType":"application/rdf+xml", "@odata.type":"#Cimphony.Concert.VersionControl.Management.ConcertResource", "@odata.id":"ConcertResource(ID='de6931d0-4212-11e7-b565-96b6cd723f39')", "ID":"de6931d0-4212-11e7-b565-96b6cd723f39", "head":"0c1dc960-4213-11e7-b565-96b6cd723f39", "name":"Alstom 60", "description":null, "[email protected]":"#DateTimeOffset", "updated":"2017-05-26T12:58:43.318Z", "[email protected]":"#DateTimeOffset", "created":"2017-05-26T12:57:26.637Z", "[email protected]":"#Collection(String)", "path":[ "EMS Examples" ], "frameID":null, "checkedOut":null, "checkedOutBy":null, "owner":"mcmorran", "version":null, "[email protected]":"#Collection(String)", "schemas":[ "http://iec.ch/TC57/2010/CIM-schema-cim15" ], "[email protected]":"ConcertResource(ID='de6931d0-4212-11e7-b565-96b6cd723f39')/Branches", "[email protected]":"ConcertResource(ID='de6931d0-4212-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15" }, { "@odata.mediaContentType":"application/rdf+xml", "@odata.type":"#Cimphony.Concert.VersionControl.Management.ConcertResource", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')", "ID":"d8815f90-41db-11e7-b565-96b6cd723f39", "head":"d9d2bec0-41db-11e7-b565-96b6cd723f39", "name":"ABB 40", "description":null, "[email protected]":"#DateTimeOffset", "updated":"2017-05-26T06:23:36.62Z",


!27

{ "@odata.context":"$metadata#Cimphony.Concert.Resource.CIM15/$entity", "@odata.type":"#Cimphony.Concert.Resource.CIM15", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15", "ID":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/GeographicalRegion", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/DayType", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/BaseVoltage", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/RegularTimePoint", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/PowerTransformer", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Analog", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/ConformLoadSchedule", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/ApparentPowerLimit", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/ShuntCompensator", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Unit", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/RatioTapChanger", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/NonConformLoad", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/PowerTransformerEnd", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/VoltageLimit", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/OperationalLimitSet", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/VoltageLevel", "[email protected]":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/RegulationSchedule",


!28

{ "@odata.context":"http://localhost:8080/cimphony/concert/db/management.svc$metadata#ConcertResource", "value":[ { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", "UUID":"_9497C00D7BFE44D59A57C65D8B76B725", "mRID":null, "aliasName":null, "name":"ABB40APC" }, { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_B7CD48062F2A49358F35F54AEB2D4008')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_B7CD48062F2A49358F35F54AEB2D4008')", "UUID":"_B7CD48062F2A49358F35F54AEB2D4008", "mRID":null, "aliasName":null, "name":"ABB40North" }, { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_04D6363BC30E4663B7AAF08AC930A9A9')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_04D6363BC30E4663B7AAF08AC930A9A9')", "UUID":"_04D6363BC30E4663B7AAF08AC930A9A9", "mRID":null, "aliasName":null, "name":"ABB40South" }, { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/


!29

{ "@odata.context":"$metadata#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion/$entity", "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.SubGeographicalRegion", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/SubGeographicalRegion('_9497C00D7BFE44D59A57C65D8B76B725')", "UUID":"_9497C00D7BFE44D59A57C65D8B76B725", "mRID":null, "aliasName":null, "name":"ABB40APC", "Names":[

], "DiagramObjects":[

], "ModelingAuthoritySet":null, "Substations":[ { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.Substation", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Substation('_58F637D8B03A4B12A67DF2E5797F9B6A')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Substation('_58F637D8B03A4B12A67DF2E5797F9B6A')", "UUID":"_58F637D8B03A4B12A67DF2E5797F9B6A", "mRID":null, "aliasName":null, "name":"CROSSPLAINS" }, { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.Substation", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Substation('_7324D6723635494784A4D8A9578FCE8A')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/Cimphony.Concert.Resource.CIM15/Substation('_7324D6723635494784A4D8A9578FCE8A')", "UUID":"_7324D6723635494784A4D8A9578FCE8A", "mRID":null, "aliasName":null,



OData in Action for DERMS!30


DERMS Project• Open Grid Systems is providing the CIM data management

repository for an ongoing DERMS innovation project at a large UK distribution company

• Multiple internal and external systems all provide and consume CIM data (including real-time data)

!31

Open Grid SystemsOGO www.opengrid.com!31

• The distribution network model from the DMS • Geographical data integrated with the DMS model • DER market data including availability/pricing • TSO market queries and requests • Outputs from market/system simulation engines • Real-time SCADA data

CIM Data


Expanding Interfaces• The number of requested interfaces is constantly

growing

• Requests for access to different streams of data based on application requirements

• The CIM repository provides an OData interface to the static as-built network data and the real-time data

• Role Based Access Control to allow different users to view/update different sets of data

• OData allows most data access requests to be met with the generic service

!32


Example Queries• “We need a list of the current real and reactive power

set points for all the DER on the network”

• The CIM class is PowerElectronicsConnection

• Only need the p and q attributes values for each

!33

http(s)://<host>/cimphony/concert/db/resources.svc/ ConcertResource(ID=‘6f4c8710-6995-11e8-b5cf-96b18064619b’)/ Contents/Cimphony.Concert.Resource.CIM16A/ PowerElectronicsConnection? $select=p,q


Example Queries• “Oh but can we only get DER that are solar?”

• We need to check the PowerElectronicsUnit association of the device is of type PhotoVoltaicUnit

!34

http(s)://<host>/cimphony/concert/db/resources.svc/ ConcertResource(ID=‘6f4c8710-6995-11e8-b5cf-96b18064619b’)/ Contents/Cimphony.Concert.Resource.CIM16A/ PowerElectronicsConnection? $select=p,q& $filter=isof(‘PowerElectronicsUnit’,Cimphony.Concert.CIM16A.Extension.Generation.Production.PhotoVoltaicUnit)


Example Queries• “We need an endpoint to return all the SCADA

measurement points attached to circuit breakers that are monitoring the switch status”

• The CIM class is Discrete (subclass of Measurement)

• Need all instances that associate with a Breaker via the PowerSystemResource association and have a measurementType of SwitchStatus

!35

http(s)://<host>/cimphony/concert/db/resources.svc/ ConcertResource(ID=‘6f4c8710-6995-11e8-b5cf-96b18064619b’)/ Contents/Cimphony.Concert.Resource.CIM16A/Discrete? $filter=isof(‘PowerSystemResource’,Cimphony.Concert.CIM16A.IEC61970.Base.Wires.Breaker) and measurementType eq ‘SwitchStatus’


Services as Queries• The same approach is used for updates

• Applications can run a CIM query for specific CIM object(s) then use the returned data to run subsequent queries to update the data (or run more queries)

• e.g. Updating a SCADA measurement involves querying for the CIM Analog with the desire ICCP name

• The query includes an instruction to also include the associated AnalogValue ($expand=AnalogValue)

• A subsequent update changes the value of the AnalogValue with the returned ID

!36


Conclusion!37


Summary• Increasing numbers of systems, applications, and services

want to access and update CIM data

• A generic, platform-independent, metadata-driven service provides flexibility for users to use CIM data

• OData solves many of the problems by defining standard best practices and libraries for RESTful HTTP services

• Existing support in common applications, programming languages and platforms

• Can still be used as a light-weight plain RESTful service for clients that are not OData-aware

• A single endpoint to support multiple services

!38


Questions

[email protected]

www.opengrid.com

!39

mailto:[email protected]

mailto:[email protected]

RESTful CIM using the Open Data Protocolcimug.ucaiug.org/Meetings/eu2018/2018 Ljubljana...OGOpen...

Documents

Transcript of RESTful CIM using the Open Data Protocolcimug.ucaiug.org/Meetings/eu2018/2018 Ljubljana...OGOpen...