Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless...
Transcript of Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless...
![Page 1: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/1.jpg)
Beyond NVDIMM: Future Interfaces for Persistent Memory
Stephen Bates, Microsemi
![Page 2: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/2.jpg)
Power Matters.TM 2 © 2016 Microsemi Corporation.
Focused Markets and Applications Solving the Difficult Problems
Based on FQ4 2016 revenue
Wireless Backhaul Base Station Routing and Switching Networking Access and CPE Wireless Terminal Fiber Optic Backhaul
Communications 38% of Revenue
Data Center 21% of Revenue
Industrial 15% of Revenue
Aerospace and Defense 26% of Revenue
Military Communication Information Assurance Engine Control Avionics Electronic Actuation Bus and Payload Electronics Launch Systems
Industrial Control Medical Devices Energy Automotive SEMI Cap Equipment Machine to Machine
Hyperscale and Enterprise Servers Storage Systems Rack Disaggregation NVM Solutions Security
![Page 3: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/3.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
Persistent Memory (PM)
3
Low Latency Memory Semantics Storage Features
![Page 4: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/4.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved. 4
Throughput is easy Latency is hard
Throughput easy; latency hard
Throughput is an engineering problem; latency is a physics problem!
![Page 5: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/5.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
Where Are We?
5
![Page 6: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/6.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
What is Needed?
6
Media and Form-
Factors
Protocols and Inter- connect
OS Support
Libraries and
Toolchain
Apps
![Page 7: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/7.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
What is Needed?
7
Media and Form-
Factors
Protocols and Inter- connect
O Support
![Page 8: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/8.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
Where does PM sit? (Answer – anywhere it wants to)
8
CPU
DRAM
PM
NAND
NAND
PM
Fabric PM
NAND
PCIe
DDR
Slide adapted from Paul Grun
![Page 9: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/9.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
Where does PM sit? (Answer – anywhere it wants to)
9
CPU
DRAM
PM
NAND
NAND
PM
Fabric (Today)
PM
NAND
PCIe
DDR
Slide adapted from Paul Grun
Fabric (Tomorrow)
PM
NAND
![Page 10: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/10.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
PM Form Factors
10
NVDIMM-N NVDIMM-P
Not-NAND NVMe NAND NVMe
![Page 11: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/11.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
NVMe Latency
11
Latency, us
7 7.5 8 8.5 9 9.5 10 10.5 11
Pro
babi
lity
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1SQDbl to MSI-X Service Time PDF NVMe is fast but
not PM fast (nor byte addressable, nor coherent).
NVMe QoS is pretty good in the system we tested.
Average <9 us!
Device Average P99 /dev/nullb0 3.9us 5.3us
/dev/pmem0 3.31us 6.2us
/dev/nvme0n1 12us 18.5us
![Page 12: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/12.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
PM Form Factors
Form-Factor Media Latency Memory Semantics
Storage Features
NVDIMM-N DRAM/ MRAM
NVDIMM-P NAND/ PM
Non-NAND NVMe DRAM/ PM
NAND NVMe NAND
12
Form factors impact features (No DMA engines on a DIMM!)
![Page 13: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/13.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
NVDIMM-PCIe
13
Blucky1
or Awesome
1 Blucky = Blah+Yucky – coined by Oisin Bates
![Page 14: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/14.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
NVDIMM-PCIe aka IOPMEM and P2P
14
• RDMA NIC can push data direct to one NVMe w/CMBs. This SSD works as a write-back cache.
• Data is then lazily copied out of the NVMe SSD w/CMB onto standard NVMe SSDs.
• Avoids the need for all SSDs to be CMB enabled (cost reduction).
CPU
DR
AM
PCIe Switch
NV
Me
SSD
NV
Me
SSD
NV
Me
SSD w
/CM
B
RD
MA
N
IC
See SDC2016 Paper for details!
e.g. PM over Fabrics or NVMe-oF write cache!
![Page 15: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/15.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
Coherency
15
Coherency 1
lots1.
War what’s it good for? Absolutely nothing.
Say it again. ~ Borrowed from Edwin Starr ~
1. Virtual addressing, simple (no) driver, shared memory across CPUs and IO devices,
no DMA setup, no get_user_pages()mappings etc…
Loads1.
Coherency
![Page 16: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/16.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
Coherency
16
Coherent! Not Coherent!
OpenCAPI DDR
OmniPath QPI
CCIX PCIe
NVMe
RDMA
Ethernet
GenZ? NVLink
![Page 17: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/17.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
Call to Arms
17
Lots to do – Sisyphean? Very Excited!
![Page 18: Beyond NVDIMM: Future Interfaces for Persistent Memory · Based on FQ4 2016 revenue Wireless Backhaul Base Station Medical Devices Routing and Switching Networking Access and CPE](https://reader031.fdocuments.net/reader031/viewer/2022011913/5fb1ed3863e2f75e69217a4b/html5/thumbnails/18.jpg)
© 2017 SNIA Persistent Memory Summit. All Rights Reserved.
A Final Thought
64 bits ~ 18 EB
180ZB1 ~ 73 bits
18
1 IDC estimate of new data in 2025