Tools and software process for the FLP prototype B. von Haller 9. June 2015 CERN.
-
Upload
marjory-skinner -
Category
Documents
-
view
214 -
download
0
Transcript of Tools and software process for the FLP prototype B. von Haller 9. June 2015 CERN.
Tools and software process for the FLP prototype
B. von Haller
9. June 2015
CERN
Overview
▶Scope ▶FLP prototype▶O2
▶Software development ▶Software quality ▶Software deployment ▶Continuous Integration
▶Specific to FLP proto :
B. von Haller | CWG11 | 09.06.2015 2
▶Operating Systems▶SLC6, CC7, Ubuntu, Mac
▶Compilers▶Gcc, llvm, icc
▶ Languages▶Low level : C▶Middle level : C++11, stl, boost libraries, ROOT6▶Open to higher level languages (e.g. python)
▶Expose interfaces using generators such as SWIG
▶Web : PHP or Python with Django. To be decided
Ecosystem
B. von Haller | CWG11 | 09.06.2015 3
OnlyLimited to these 2
Software development
▶Tools▶Version control system: Git with GitLab/GitHub▶Software build system: CMake ▶Issue tracking system: JIRA▶Code formatter: clang-format
▶Procedures▶Workflows A la GitHub (pull request, see next
slides)
B. von Haller | CWG11 | 09.06.2015 4
GitLab to be gradually
introduced
Software development GitHub development workflow
B. von Haller | CWG11 | 09.06.2015 5
▶ Collaborators have a full remote clone of the main repo, called a fork▶ Full rights on their fork, none on the main repo▶ Anything in the master branch is always deployable.▶ Owners of the main repo (librarians) comment and accept/reject the
pull requests▶ Open source clone of GitHub: GitLabs
Software developmentGitHub/Gitlab
B. von Haller | CWG11 | 09.06.2015 6
▶Advantages of GitHub/GitLab and the related development workflow▶Collaborative and intuitive web interface▶Code always reviewed and commented before
being accepted▶Code can be checked automatically by tools before
human reviews▶Feature-rich api
Software development
▶ Procedures (continued)▶ Workflows A la GitHub (pull request, see next slides)▶ Coding conventions, naming, formatting -> CWG2▶ Code access policy Open (license still under discussion) -> CWG2
▶ Repositories ▶ Neither monolithic nor excessively fragmented▶ Group components that share same dependencies and/or similar functionalities
▶ Packages▶ Does not always match with repositories (also more flexible)▶ Release pace or common dependent packages to take into account
▶ Dependency handling for developers▶ External : original + patch -> recompile (tool to be decided) ; packages▶ Internal : packages (avoid recompiling what does not need to) ▶ CernVM-FS
B. von Haller | CWG11 | 09.06.2015 7
Software quality
▶Tools▶Static analysis -> Clang Static Analyzer▶Dynamic analysis -> Valgrind▶Tool for code reviews -> GitLab▶Code conventions and formatting enforcer ->
clang-format
B. von Haller | CWG11 | 09.06.2015 8
Not initially
Software quality
▶Testing▶Unit tests, usually at the function level
▶CTest and boost.test
▶Functional tests ▶Integration tests, software modules are combined
and tested as a group. Full chain. ▶Requires the code to be designed for it and to provide
mock systems at the interfaces▶Environment must be controlled
B. von Haller | CWG11 | 09.06.2015 9
Software deployment
▶Packaging▶CPack or fpm (to be decided, evaluation ongoing)
▶Dependency manager for users▶Packages using native system offered by OS▶CernVM-FS
B. von Haller | CWG11 | 09.06.2015 10
Continuous Integration (CI)
▶Actually wider than pure CI▶ Inter connects development, quality and deployment
▶ Jenkins▶Extensible, open source CI server▶Hundreds plugins, wide industry support▶Can do many type of tasks (e.g. build, test, release)▶On scheduled time or following a trigger (e.g. pull
request)▶API to build our own tools in front or make other tools
collaborate (e.g. JIRA)
B. von Haller | CWG11 | 09.06.2015 11
Continuous Integration
▶Master hosted in IT for ALICE on a VM▶Use VMs with Docker on Openstack▶Authentication using CERN SSO and egroups▶Mesos deployment containers to deal with
non-homogeneous tasks and load▶Integration with JIRA and GitHub/GitLab▶Time scale : infrastructure ready this summer
Jenkins for O2
B. von Haller | CWG11 | 09.06.2015 12
Conclusion
▶Tools have been chosen and procedures defined to ensure ▶Code quality, readability and maintainability▶Products delivery, stability and performance
▶The FLP prototype development can start in July directly within this frame
B. von Haller | CWG11 | 09.06.2015 13