Low-Cost ICS Network Performance Testing
-
Upload
jim-gilsinn -
Category
Technology
-
view
311 -
download
7
description
Transcript of Low-Cost ICS Network Performance Testing
![Page 1: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/1.jpg)
SCADASides 1
Low-Cost ICS Network Performance Testing
Jim GilsinnKenexis Consulting
June 6, 2014
![Page 2: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/2.jpg)
SCADASides 2
How This Got Started
• In 2001 while I worked @ NIST my boss said:• Industrial Ethernet is the next big wave for manufacturing, so say our
customers (auto manufacturers)• There are still a lot of questions about how well it performs• Is it deterministic enough for the factory floor? Yes, but…• Are there standardized metrics to show performance? Yes, but…• Are there test tools available? Yes, but…• Can companies put performance requirements into their procurements
yet? Yes, but…
June 6, 2014
![Page 3: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/3.jpg)
SCADASides 3
Determinism
• Vendors were building industrial Ethernet products that claimed certain performance
• End-users were finding quirky performance
• End-users would complain
• Vendors would say, it works in our lab, there must be a problem in your system
• End-users learned not to trust performance claims from vendors
• Some build labs to approve devices before implementing them
June 6, 2014
![Page 4: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/4.jpg)
SCADASides 4
Standardized Metrics
• Vendors would describe their performance in many different ways and with varying definitions
• With ODVA, I helped to create a standard set of metrics for end-point devices based upon IETF definitions
• Throughput• Jitter/Variability• Latency (action latency, response latency)
June 6, 2014
![Page 5: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/5.jpg)
SCADASides 5
Test Tools
• After creating the metrics, NIST helped ODVA develop a set of performance tests
• We build the ODVA Performance Testing Laboratory that ODVA charges companies money to certify their stated performance
• No one has run the test since no one wants to fail• ODVA charges for every time a company tests and retests
• NIST went on to develop a free capture file analysis tool• Available on SourceForge (1st gen is IENetP, 2nd gen is FENT)• Both of these are dormant
• NIST also worked with the ODVA Interoperability Workshop to develop a series of 5 tests that could be conducted quickly
June 6, 2014
![Page 6: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/6.jpg)
SCADASides 6
Procurement Language
• Big auto manufacturers have tried to get their vendors to use ODVA performance lab
• Hasn’t worked out well• Have convinced vendors to go through PlugFest testing
• Vendors and end-users have started using a common language
• I guess that’s as good as it gets for now
June 6, 2014
![Page 7: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/7.jpg)
SCADASides 7
Low-Cost Performance Testing
• Uses low-cost/readily-available equipment• Low-cost is relative, $15 – $3k• Readily-available, like laptops, switches, etc.
• Uses open-source/low-cost/readily-available software• Open-source, like Linux, Wireshark, background traffic, and analysis
tool• Low-cost analysis tool (Kenexis, in development)• Readily-available, like Windows, Office, browsers
• Additional useful tools• Protocol-dependent master/scanner (software will get you ~2ms)
June 6, 2014
![Page 8: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/8.jpg)
SCADASides 8
Testing Equipment
• Laptops x2• Alienware M14x-R2• Ubuntu 14.04 native• Windows VM• Backtrack 5r3 USB
• DreamPlug
• Raspberry PI• Model B, rev 1
• Netgear GS108E Switch
• Throwing Star LAN Tap
• Hilscher netANALYZERJune 6, 2014
![Page 9: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/9.jpg)
SCADASides 9
Testing Software
• Linux (Ubuntu 14.04, Backtrack 5r3, Kali)
• Wireshark (apt-get and compiled)
• PlugFest background traffic captures and scripts
• NIST Analysis Tool• 1st Generation = IENetP – http://www.sourceforge.net/projects/ienetp• 2nd Generation = FENT – http://www.sourceforge.net/projects/fent
• Kenexis Analysis Tool• Follow-on, in development
June 6, 2014
![Page 10: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/10.jpg)
SCADASides 10
PlugFest Background Traffic
• Traffic Captures• Generated by Ixia network analyzer and packet generator• Assembled into different sets (editcap & mergecap)
• tcpreplay Scripts• Generated Linux scripts to replay capture files
• Conducted Analysis of Results• Packet generator transmitting• Laptop transmitting• Laptop receiving
June 6, 2014
![Page 11: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/11.jpg)
PlugFest Background Traffic
Traffic Type Rate (pps)
Baseline
Steady-State Managed
Steady-State Unmanaged
Burst Managed
Burst Unmanaged
ARP Request Broadcasts 180
Gratuitous ARP Broadcasts 180
DHCP Request Broadcasts 100
ICMP (ping) Request Broadcasts 100
NTP Multicasts 10
EtherNet/IP ListIdentity Req. 10
EtherNet/IP Class 1 1800
ARP Burst Requests 240 pkts @ 4k Hz
![Page 12: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/12.jpg)
SCADASides 12
PlugFest Testing Architecture
June 6, 2014
![Page 13: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/13.jpg)
SCADASides 13
Eye Chart Slides Ahead
June 6, 2014
![Page 14: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/14.jpg)
SCADASides 14June 6, 2014
Example PlugFest Testing (Hilscher)
![Page 15: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/15.jpg)
SCADASides 15June 6, 2014
Example PlugFest Testing (Switch Mirror)
![Page 16: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/16.jpg)
SCADASides 16
Low-Cost Testing Architecture
June 6, 2014
![Page 17: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/17.jpg)
SCADASides 17
Low-Cost Testing
• Laptop Laptop
• Laptop DreamPlug
• DreamPlug Laptop
• Laptop Raspberry PI
• Raspberry PI Laptop
June 6, 2014
![Page 18: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/18.jpg)
SCADASides 18June 6, 2014
![Page 19: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/19.jpg)
SCADASides 19
What The Data Shows
• Hilscher Capture Card• 10ns resolution time stamping• Hardware assisted• Good enough for hard real-time performance testing (1s µs)
• High-End Laptop• Backtrack/Kali better than Ubuntu• Running from USB stick works• Good enough for soft real-time performance testing (~100 µs)
June 6, 2014
![Page 20: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/20.jpg)
SCADASides 20
What The Data Shows
• DreamPlug• Good enough for mostprocess control• Offset of mean (~5-10 ms)• Random delays occur (~5-20 ms, sometimes 100+ ms)• On-par with Windows performance
• Raspberry PI• Good enough for slow process control• Offset of mean (~5-25 ms)• Random delays occur (100-1000 ms)
June 6, 2014
![Page 21: Low-Cost ICS Network Performance Testing](https://reader035.fdocuments.net/reader035/viewer/2022062511/54bc2dfa4a7959336b8b4781/html5/thumbnails/21.jpg)
SCADASides 21
More Information
• Jim Gilsinn, Kenexis Consulting• Email: [email protected]• Phone: 614-323-2254• Twitter: @JimGilsinn• SlideShare: http://www.slideshare.net/gilsinnj
• Kenexis GitHub• https://github.com/kenexis/LowCostPerformance
June 6, 2014