LCE13: LNG Testing, benchmarking, etc
-
Upload
linaro -
Category
Technology
-
view
948 -
download
0
description
Transcript of LCE13: LNG Testing, benchmarking, etc
![Page 1: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/1.jpg)
Linaro Connect,Hong KongMarch 2013
July 2013
LCE13LNG Testing and Benchmarking
![Page 2: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/2.jpg)
connect.linaro.org
Agenda
● Discussion on how to enhance LAVA framework for networking applications.
● Discussion on LNG testing strategy and goals.
https://lce-13.zerista.com/event/member/79674
http://pad.linaro.org/p/LCE13_LNG-Benchmarking
![Page 3: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/3.jpg)
connect.linaro.org
● Why: Make sure we don't have regression in functionality and performance. Regression could be introduced by upstream or by us.
● What: Baselines and regression testing.
● How: LAVA.
For example: compare LNG kernel with and without RT patchset.
LNG Testing/Benchmarking
![Page 4: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/4.jpg)
connect.linaro.org
LNG Focus Areas
● Big Endian
● Foundation○ LNG Kernel (LSK + Hugepage + NOHZ_FULL + RT...)
● Data Plane○ RTE + SOC APIs
What are the Key Performance Indicators?
![Page 5: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/5.jpg)
connect.linaro.org
LNG Test Cases
LNG Tests/Benchmarks Tracking Sheet
● Functional Testing○ Additional patches carried in LNG kernel trees. Some have wider impact (e.g.
RT), some may be backported features (e.g. hugepage, nohz_full).○ Need coverage for additional features, as well as kernel in general post
integration.
● Performance Benchmarking○ Some features improve certain performance characteristics (e.g. hugepage,
nohz_full).○ Need to characterize those and make sure they continue to work well across
releases.
![Page 6: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/6.jpg)
connect.linaro.org
LNG Test Cases
Measure:● Throughput● Number of dropped packets● Latency● CPU utilization● Power
Other Variables:● Number of cores● Core frequency● Number of ports
Test Variations:● Accelerated using Dataplane
APIs.● Inside KVM guest.
○ Guest-network.○ Guest-guest.
● Kernel with different patchsets (nohz_full, rt, etc.).
Example: network packet forwarding.
Compare against x86.
![Page 7: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/7.jpg)
connect.linaro.org
LNG Test Bench (Network Traffic)
Tests with network traffic, especially for data plane applications.
DUT: member platforms, multi-10G, multi-1G ports.
Using x86 machines as traffic generator. Could also be DUT for comparative benchmarking.
![Page 8: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/8.jpg)
connect.linaro.org
LNG Test Bench (Network Traffic)
Some ports directly connected between Traffic Generator and DUT.
Some ports connected through switch.
Can even repurpose some DUT as Traffic Generator.
![Page 9: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/9.jpg)
connect.linaro.org
LNG Test Bench (Network Traffic)● Resource Management
○ Isolated data path between traffic generator and DUT.
○ Automated / programmatic way to access traffic generator, switch, etc.
○ Efficient resource scheduling.
● Test & Resource Coordination○ Start app on DUT, start
traffic, stop traffic, stop app on DUT.
● Test Report & Database○ Performance trends,
passing criteria, statistics○ Visualization, alerts.
![Page 10: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/10.jpg)
connect.linaro.org
LNG Test Bench (w/o Network Traffic)
Tests without network traffic, primarily for kernel regression testing.
● Resource Management○ Automated / programmatic
way to access traffic generator, switch, etc.
○ Efficient resource scheduling.
● Test Report & Database○ Performance trends,
passing criteria, statistics○ Visualization, alerts.
![Page 11: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/11.jpg)
connect.linaro.org
● What are the target applications? Example code?● Recommendations of how we should categorize
benchmarks?○ By verticals? e.g. wireless, security?○ L1-L7?
● Any additional hardware platforms to consider?● Wrt comparative benchmarking, which platforms do we care
about?● How much do we care about industry benchmarks such as
SPEC, EEMBC?● How do we characterize success of our data plane APIs?● In case of data plane, there are different solutions: bare
metal, Linux userspace RTE, etc. How do we showcase LNG solution?
●
Questions
![Page 12: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/12.jpg)
Linaro Connect,Hong KongMarch 2013
Questions?
![Page 13: LCE13: LNG Testing, benchmarking, etc](https://reader034.fdocuments.net/reader034/viewer/2022051208/546d8bb0af795976298b53fa/html5/thumbnails/13.jpg)
More about Linaro: http://www.linaro.org/about/More about Linaro engineering: http://www.linaro.org/engineering/
How to join: http://www.linaro.org/about/how-to-joinLinaro members: www.linaro.org/members