Scalable Storage Management with NVMe-oF · 2019 Storage Developer Conference. © Intel. All Rights...
Transcript of Scalable Storage Management with NVMe-oF · 2019 Storage Developer Conference. © Intel. All Rights...
2019 Storage Developer Conference. © Intel. All Rights Reserved. 1
Scalable Storage
Management with NVMe-oF
Piotr Wysocki, Rafał Bogdanowicz
Intel
2019 Storage Developer Conference. © Intel. All Rights Reserved. 2
Agenda
Intel® RSD Overview
NVMoF management in Intel® RSD
Intel® RSD storage service evolution
Intel® RSD 2.5 Redfish/Swordfish and SPDK mappings
2019 Storage Developer Conference. © Intel. All Rights Reserved. 3
Code availability
All discussed code available on GitHub:
https://github.com/intel/intelRSD
It will be presented on SDC Workshop
2019 Storage Developer Conference. © Intel. All Rights Reserved. 4
1. Source: Quantifying Datacenter Inefficiency: Making the Case for Composable Infrastructure, IDC, Document #US42318917, 2017. 2. Source: Disaggregated Server Architecture Drives Data Center Efficiency and Innovation, Shesha Krishnapura, Intel Fellow and Intel IT CTO, 2017
Today’s data center Challenges Intel®Rack Scale Design
Current Infrastructure
• Fixed ratio of compute, storage, and accelerator resources
• Expensive refresh & scale out
• Outdated software interface
• Cumbersome hardware provisioning process
Composable
Disaggregated
Interoperable
Data Center Agility, Built on Open Standards
“an industry-aligned architecture for composable, disaggregated infrastructure built on modern, open standards.”
DecreaseCosts
IncreaseAgility
50 %Efficiency
IT Operations1
35 People Hours Per
Rack Update2
45 %Utilization
Of Equipment1
2019 Storage Developer Conference. © Intel. All Rights Reserved. 5
Storage Sled
Compute
Compute
Compute
Compute
Network
Accelerator Sled
Intel Pod Manager
Composed Node 2
Orchestration
App 1 App 2 App 3
Composed Node 1
Intel® RSD Key Attributes
ComposableDisaggregated Interoperable
Single-Pane-of-Glass Management
Vendor A Hardware
Vendor B Hardware
Vendor C Hardware
Vendor D Hardware
Open Standard API
Compose hardware resources “on the fly”
Buy less up front andSave money over time
Choose the best now without vendor lock-in
OEMs with solutions based on RSD
Intel®Rack Scale Design
2019 Storage Developer Conference. © Intel. All Rights Reserved. 6
Storage Sled
Compute
Compute
Compute
Compute
Network
Accelerator Sled
Intel Pod Manager
Composed Node 2
Orchestration
App 1 App 2 App 3
Composed Node 1
Benefits of Disaggregation and Composability
ComposableDisaggregated
Compose hardware resources “on the fly”
Buy less up front andSave money over time
Resource poolingMaximize utilization ofhigh-value assets andimprove agility withdynamic composability
Modular RefreshIndependently scale
and upgrade resources
with better lifecycle
Management
Operational CostsImprove Power UsageEffectiveness (PUE) andstreamline operationsand HW management
2019 Storage Developer Conference. © Intel. All Rights Reserved. 7
Intel® RSD – Composability
Orchestration Software
App1 App1 App1Compose hardwareresources “on the fly”for specific workloads
Intel® RSD softwarefunctions include:
Resoureces DiscoveryAutomatically discover and storehardware characteristics andlocation for all your resources
Node Composition Dynamically compose compute,storage, and other resources tomeet workload specific demands
Telemetry Data Monitor data center efficiencyand detect, diagnose, and helppredict resource failures
2019 Storage Developer Conference. © Intel. All Rights Reserved. 8
Intel® RSD – Storage Disaggregation
Storage Sled
Compute
Compute
Compute
Compute
Network
Accelerator Sled
Disaggregation
Save money over timewith modular refresh Great scalability
2019 Storage Developer Conference. © Intel. All Rights Reserved. 9
Intel® Rack Scale Software stack
Rack Scale POD Manager
POD Wide AssetManagement
POD Wide Composed Nodes Management
POD Wide StorageManagement
POD Wide FabricManagement
Cloud or Data Center Manager
OpenStack3rd Party
Datacenter MgrDIY
Microsoft DIY
POD ManagerFunctionality
POD ManagerFunctionality
VMware
Rack Scale POD Management API
Pooled Systems
Compute Module
Network Module
Fabric Module
Storage Module
Storage Services
Storage Pools
Logical Volumes
Storage Endpoints
Rack Manager
Power Thermal
Chassis
Rack Scale Pooled Systems API Chassis Management APIStorage Services API
Memory Module
FPGA Module
Accelerator Module
Network Services
Network Infrastructure
Network Protocols
Networking Services
Network Services API
2019 Storage Developer Conference. © Intel. All Rights Reserved. 10
Intel® Rack Scale Storage Services
Core ManagementApplication
Op
era
tin
g S
yst
em
(Lin
ux* )
Asset Manager CMDB
Redfish/Swordfish API
Storage Management
Storage Assets (Physical and Logical)
Generic Assets Management Interface (JSON-RPC)
iSCSI CEPH
Targets Logical Devices Physical Devices
LVMNativeLinux
NativeLinuxDisk
Mgmt
NVMeover
Fabrics
2019 Storage Developer Conference. © Intel. All Rights Reserved. 11
POD Manager
Discovery
Service
Storage
Server
Client host
(initiator)
Ethernet
Management
network
Data
network
Intel® RSD components in NVMoF
1 2
2019 Storage Developer Conference. © Intel. All Rights Reserved. 12
Intel® RSD 2.3+
Partitions over NVMe
PSME REST Server
PSME SPDK Agent
SPDK (nvmf_tgt)
JSON-RPC
GAMI
Redfish Swordfish
Intel® RSD 2.4+
Storage Performance
Development Kit
Intel® RSD Storage Service evolution
Intel® RSD 1.2+
iSCSI Storage Service
2019 Storage Developer Conference. © Intel. All Rights Reserved. 13
Why SPDK?
SPDK BDEVSubsystem
NVMe-oFTarget
SPDK NVMe PCI Driver
NVMe PCIe Controller(s)
SPDK NVMe Module (NVMe BDEV)
SPDK Logical Volume Store (LVS)
SPDK Logical Volumes (LVOL BDEV)
NVMf Subsystems
Namespaces HostsThe Storage Performance Development Kit – a set of libraries providing:
• High performance
• Scalability
• Low latency
• Efficient use of CPU and memory resources
• Modularity
2019 Storage Developer Conference. © Intel. All Rights Reserved. 14
Service Root
Chassis Systems Fabrics
Ch1
Storage Services
CS1 NVMeoF SS1
D1
Zones
Z1
Ethernet Interfaces
NIC1
NIC2
Endpoints
Initiator1 Target1
DrivesVolumes
Storage Pools
V1 V2 SP1
resource
navigation link
Storage Service and Common Fabric Model
(Redfish + Swordfish)
2019 Storage Developer Conference. © Intel. All Rights Reserved. 15
/Chassis/{id}/Drives/{d-id}NVMe Block Device (BDEV)
Logical Volume Store (LVS)
Logical Volume (LVOL)
NVMf Subsystem
/StorageServices/{id}/StoragePools/{sp-id}
/StorageServices/{id}/Volumes/{v-id}
/Fabrics/{id}/Endpoints/{target-id}
/Fabrics/{id}/Endpoints/{target-id}
/Fabrics/{id}/Endpoints/{initator-id}NVMf Subsystem + Host
/Fabrics/{id}/Zones/{zone-id}
Intel® RSD 2.5 NVM over Fabrics
SPDK to Redfish/Swordfish mapping
2019 Storage Developer Conference. © Intel. All Rights Reserved. 16
Initial discovery for all resources (GET) get_bdevs
get_lvol_stores
get_nvmf_subsystems
Intel® RSD 2.5 NVM over Fabrics
Redfish/Swordfish actions to SPDK mapping
2019 Storage Developer Conference. © Intel. All Rights Reserved. 17
Volume create (POST) construct_lvol_bdev
snapshot_lovl_bdev
clone_lvol_bdev
destroy_lvol_bdev
Volume snapshot (POST)
Volume clone (POST)
Volume delete (DELETE)
Volume extend (PATCH) resize_lvol_bdev
Intel® RSD 2.5 NVM over Fabrics
Redfish/Swordfish actions to SPDK mapping
snapshot_lvol_bdev
2019 Storage Developer Conference. © Intel. All Rights Reserved. 18
Endpoint create (POST)
Endpoint delete (DELETE)
Zone create (POST)
Zone s endpoints modification (PATCH)
Zone delete (DELETE)
nvmf_subsystem_create
nvmf_subsystem_add_listener
nvmf_subsystem_add_ns
delete_nvmf_subsystem
nvmf_subsystem_add_host
nvmf_subsystem_add_host
nvmf_subsystem_remove_host
nvmf_subsystem_remove_host
Intel® RSD 2.5 NVM over Fabrics
Redfish/Swordfish actions to SPDK mapping
2019 Storage Developer Conference. © Intel. All Rights Reserved. 19
Questions?
Thank you