June 20–23, 2016 | Berlin, Germany
Multi-site OPNFV Testing ChallengesTrevor Cooper (Intel)
Fatih Degirmenci (Ericsson)
Morgan Richomme (Orange)
Agenda
Multi-site Testing Today - Trevor- Pharos Infrastructure
- CI Today
Multi-site Testing Needs - Morgan- Advanced Test-cases
- Interoperability
Multi-site Testing Plans - Fatih- Challenges and Requirements
- Future Plans and Vision for Infra and Testing
Summary and Call to Action - Fatih
Multi-site Testing Today
Multi-site Testing Introduction
• Motivations for Multi-site Testing• Geographically and technically diverse test environments
• Test resources independent of installers and scenarios promotes robust platform
• Inter-deployment testing for interoperability between data-centers
• Future real-world use-cases and deployment scenarios
• Pharos Infrastructure• Infra Working Group: Pharos, Octopus, Releng, Genesis
• Infrastructure labs, process, tools and features
• CI / CD / CT• Environments increasing in number and complexity
• Process to improve platform robustness
• Challenges of a Distributed Infrastructure • Managing a distributed resources
• Automating testing
• Distributing tests
Pharos Infrastructure
• Current priorities: • Allocation and management of Infra resources for Colorado
• Pharos change process to manage special requirements
• Lab resource booking and usage tool/dashboard
• Pharos compliance verification/validation
• Dealing with today’s multi-site challenges• Challenges and how we solved them
Production Today
CI Yesterday CI Today
POD1Installer XScenario Y
POD2
PODi
Deploy, run tests
POD1Installer XScenario Y
POD2
PODi
Deploy, run tests
Lab 1
Lab 2
Lab 1
Lab 2
Production Evolution - Tomorrow
POD1Installer XScenario Y
POD2
PODi
Deploy, run tests
Lab 1
Lab 2
POD1Installer X’Scenario Y’
POD2
Lab 3
CI Tomorrow
Infra Improvements
• Lab as a Service (LaaS)• Accessible virtual sandboxes for development and smoke tests
• More homogeneous environments for development work
• Off-load shortage of bare-metal resources
• Efficient use of Lab Resources• Strive for CD (provide weekly stable versions)
• Provide dynamic use of lab resources (Dev and CI)
• Verify/validate POD setup for automatic configuration
• All labs/pods/installer configurations for automatic POD configuration
• Jenkins / CI improvements - permissions, scheduling, sandboxes, etc.
Multisite Testing Needs
Test cases
• Advanced test cases• OPNFV Multisite Project - evolving
• Brahmaputra -> Use case definition and bug fixes
• Colorado -> Single machine/Multiple OpenStack Instances, Functional Testing, Kingbird
• Future -> Go after true Multisite
• Discussion on multi tenant vIMS for D river
• Interoperability testing
An advanced test case: vIMS
Today for Functest test: Metaswitch vIMS + Cloudify orchestrator installed in a single tenant on one OPNFV POD.
Not acceptable in “real life” (SPOF)
Geographical redundancy on current production IMS system
Architecture Cloud ready (Cassandra ring)
Possible evolutions: 1) test in multi-tenants (local redundancy)2) Test in multi-sites (geographical redundancy)
An advanced test case: vIMS
Need of Multi-sites to evaluate • the connectivity issues (VPN/Flat)• the consequence on performance• The accuracy of placement algorithm:
• Performance (DPDK/ODP area)• Location (legal constraint) …
• Disaster/recovery scenarios/constraints
Similar considerations on vCDN for next releases
An interoperability test
• OPNFV is a great place to automate interoperability testing…
• Several PODs … with several scenarios
• And a centralized CI
An interoperability test: bgpvpn
• SDNVPN project deals with the implementation of the OpenStack bgpvpn API
• Several possible backends• ODL
• Neutron Bagpipe
• Nuage
• …
• Use OPNFV to perform interoperability testing would be valuable
An interoperability test: bgpvpn
POD1os-nofeature-bgpvpn-ha
POD2os-odl_l2-bgpvpn-ha
Deploy, run tests
BGPVN
Multi-site Testing Plans
Lessons learned
• Distributed infrastructure is not trivial
• Automating tests/ state synchro towards multi-sites
• We learn by doing…first requests on multi-sites needed raised in Brahmaputra => Multi-site project
Next steps
• Apply learnings• Bringing up, managing and using distributed infrastructure
• Evolve continuously• No dramatic changes
• E2E View and holistic approach• Features will come together one day, creating more complete
Multisite solutions
Next steps
• Infrastructure• Provision, manage and use resources cleverly
• Increase the utilization rather than burning CPU uselessly
• Get ready to enter uncharted territories - security and access policies
• CI• Evaluate CI to see if it can support Multisite use cases
• Provide feedback that fits to purpose rather than bunch of log files
• Continue applying best practices
• Testing • Test what needs to be tested when it is needed to be tested
Summary
• We already are challenged by what we had in the past and what we currently have
• The complexity will just increase, requiring us as a community to work together like a community
Call to Action
Top Related