Build, Scale, and Deploy Deep Learning Pipelines Using Apache Spark
Apache Continuum Build, Test, and Release
-
Upload
elliando-dias -
Category
Technology
-
view
2.691 -
download
0
description
Transcript of Apache Continuum Build, Test, and Release
![Page 1: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/1.jpg)
Apache ContinuumBuild, Test, and Release
Wendy Smoak
ApacheCon EuropeMarch 27, 2009
![Page 2: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/2.jpg)
• Member, The Apache Software Foundation
• Committer, Apache Continuum, Archiva, Struts, Tiles, Maven, ...
• Solutions Architect, G2iX
http://wsmoak.net
Wendy Smoak
2
![Page 3: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/3.jpg)
How many...
3
![Page 4: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/4.jpg)
Continuous Integration
4
![Page 5: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/5.jpg)
Continuous Integration
... is a software development practice in which members of a team integrate their work frequently.
Usually each person integrates at least daily - leading to multiple integrations per day.
Each integration is verified by an automated build (including tests).
http://www.martinfowler.com/articles/continuousIntegration.html5
![Page 6: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/6.jpg)
Continuous Integration
6
build
check in
test
notify
![Page 7: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/7.jpg)
Why CI?
7
• Save time
• Increase visibility
• Improve coverage
![Page 8: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/8.jpg)
Apache Continuum
8
![Page 9: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/9.jpg)
• Apache top-level project
• More than CI...
• Role-based Access Control
• Release Management
• Parallel and Distributed Builds
• Configurable JVM Wrapper
Apache Continuum
9
![Page 10: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/10.jpg)
Prerequisites
• Java SE 5
• Maven, Ant, Script
• Subversion, CVS, Git
• Tests!
10
![Page 11: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/11.jpg)
• Download
• Install
• Start and Stop
• Configure
• Build Definitions
• Build Environments
• Build Schedules
• Build Queues
• Release
• Project Groups
• Users and Roles
• Notifiers
• JVM Wrapper
• XML-RPC
Demo
11
![Page 12: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/12.jpg)
Download
http://continuum.apache.org12
![Page 13: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/13.jpg)
Install
• Unpack the distribution
• Continuum + Jetty + Derby
13
$ lsLICENSE apps conf derby.log logsNOTICE bin data lib tmp
![Page 14: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/14.jpg)
Pre-Configure
• conf/jetty.xml
• Port number
• SMTP config
• Data sources
14
![Page 15: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/15.jpg)
Start
• bin/continuum start
• Automatically detects platform
• Windows
• bin/continuum.bat console --or--
• bin/continuum.bat install (as a service)
15
![Page 16: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/16.jpg)
Demo Setup
16
svn.example.comhttpd + mod_dav_svn
running locally
build.example.comContinuum + Jetty
running locally
command linedeveloper workstation
repo.example.comArchiva + Jettyrunning locally
![Page 17: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/17.jpg)
Demo!
17
![Page 18: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/18.jpg)
Admin User
18
![Page 19: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/19.jpg)
Configure
19
![Page 20: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/20.jpg)
Add Projects
• Maven 2
• Ant
• Maven 1
• Shell Script
20
![Page 21: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/21.jpg)
Project Groups
• Logical group of projects
• Multi-module Maven project
21
![Page 22: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/22.jpg)
Build
• Build all projects
• Order is determined by dependencies
• Build one or more projects
22
![Page 23: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/23.jpg)
View Results
• Build status
• Build output
• Test results
• Working copy
23
![Page 24: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/24.jpg)
Release
• Web interface
• Prepare, then Perform
• Consistent environment
24
![Page 25: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/25.jpg)
End Demo
25
![Page 26: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/26.jpg)
Installations
• Environment Variable
• Tool (Ant, Maven, JDK)
26
JDK Maven
var=value
![Page 27: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/27.jpg)
Build Environments
27
Build Environment
JDK Maven var=value
![Page 28: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/28.jpg)
Build Schedules
• Hourly by default
• crontab format
28
Schedule
When to run
Max time
Build Queue(s)
![Page 29: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/29.jpg)
Build Definitions
• Build file such as pom.xml
• goals and arguments
• Schedule
• Build Environment
29
![Page 30: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/30.jpg)
Build Queues
• For parallel builds
• Associated with Schedules
30
Build Queue 1
Build Queue 2
![Page 31: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/31.jpg)
Notifiers
• Email, IRC, MSN, Jabber, Wagon
• Configure in Maven pom.xml
• Add directly to Continuum
• Can notify the latest committers
31
![Page 32: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/32.jpg)
32
Parallel Builds
![Page 33: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/33.jpg)
Distributed Builds
33
![Page 34: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/34.jpg)
Users
• Self registration
• Per-project roles
• Web based
• Configurable security rules
34
![Page 35: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/35.jpg)
XML-RPC
• Control Continuum from external scripts or programs
• Java examples available
35
![Page 36: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/36.jpg)
JVM Wrapper
• conf/wrapper.conf
• http://wrapper.tanukisoftware.org
36
![Page 37: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/37.jpg)
SCM Support
http://docs.codehaus.org/display/SCM/SCM+Matrix37
![Page 38: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/38.jpg)
Alternatives
• Bamboo (Atlassian)
• Cruise Control (Thoughtworks)
• Hudson
• ... and many others
http://damagecontrol.codehaus.org/Continuous+Integration+Server+Feature+Matrix38
![Page 39: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/39.jpg)
What’s coming?
• Plugin architecture
• Hosting project websites
• ... what do you need?
39
![Page 40: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/40.jpg)
Where we need help
40
• Discussing and coding features and fixes
• UI Design and Usability
• Writing documentation
• Testing - JUnit, TestNG, Selenium
![Page 41: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/41.jpg)
Where you can find us
41
• http://continuum.apache.org
• #continuum on irc.codehaus.org
• Here at ApacheCon!
![Page 42: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/42.jpg)
Questions?
42
![Page 43: Apache Continuum Build, Test, and Release](https://reader033.fdocuments.net/reader033/viewer/2022051514/5481cecab4af9f6e0d8b460d/html5/thumbnails/43.jpg)
Apache ContinuumBuild, Test, and Release
Wendy Smoak
ApacheCon EuropeMarch 27, 2009