Manoj Kolhe - Setup GitHub with Jenkins on Amazon Cloud - End-to-end Automation

This document highlights the steps wise

procedure for setting up Continuous

Integration with Jenkins and GitHub

repository

Jenkins

Integration

with GitHub Continuous Integration Test

Environment

Manoj Kolhe


Introduction .................................................................................................................................................................................................. 3

Approach ....................................................................................................................................................................................................... 3

Hardware Requirements............................................................................................................................................................................... 5

Software Requirements ................................................................................................................................................................................ 5

Configure Git Plugin with Eclipse IDE............................................................................................................................................................ 6

Create Reposi tory .................................................................................................................................................................................... 7

Track Changes ........................................................................................................................................................................................ 10

Inspect History ....................................................................................................................................................................................... 12

GitHub Tutorial....................................................................................................................................................................................... 15

Create Local Repository..................................................................................................................................................................... 15

Create Reposi tory at GitHub ............................................................................................................................................................. 15

Eclipse SSH Configuration.................................................................................................................................................................. 16

Push Upstream .................................................................................................................................................................................. 17

Download and Install Java........................................................................................................................................................................... 21

Configure Apache Tomcat........................................................................................................................................................................... 22

Changing the Tomcat Default 8080 port ............................................................................................................................................... 24

Configure Apache Maven............................................................................................................................................................................ 25

Configure Jenkins ........................................................................................................................................................................................ 26

Download GitHub related plugins.......................................................................................................................................................... 28

Configure System ................................................................................................................................................................................... 29

Create a new job in Jenkins.................................................................................................................................................................... 30

Adding GitHub Credentials..................................................................................................................................................................... 31

To generate SSH key for GitHub ............................................................................................................................................................ 31

Step 1: Check Exis ting keys................................................................................................................................................................ 31

Step 2: Generate a new SSH key ....................................................................................................................................................... 31

Step 3: Add your SSH key to GitHub.................................................................................................................................................. 32

Step 4: Test everything out ............................................................................................................................................................... 35

Test GitHub Jenkins Integration............................................................................................................................................................. 36

Create User in Jenkins for Remote Job Builds ....................................................................................................................................... 38

Ini tial Steps : ....................................................................................................................................................................................... 38

Push GitHub Changes to Jenkins................................................................................................................................................................. 41

Showing Jenkins Build Status on GitHub..................................................................................................................................................... 43

Queries ........................................................................................................................................................................................................ 44


Jenkins Integration with GitHub

A Step by step guide for the setup and configuration

Introduction

enkins is used for Continuous Integration Testing of builds. Jenkins provides continuous integration

services for software development. It is a server-based system running in a servlet container such as

Apache Tomcat. It supports SCM tools including AccuRev, CVS, Subversion, Git, Mercurial, Perforce,

Clearcase and RTC, and can execute Apache Ant and Apache Maven based projects as well as arbitrary

shell scripts and Windows batch commands. The primary developer of Jenkins is Kohsuke Kawaguchi.

Released under the MIT License, Jenkins is free software.

Builds can be started by various means, including being triggered by commit in a version control system,

scheduling via a cron-like mechanism, building when other builds have completed, and by requesting a

specific build URL.

Approach

1. Configure Jenkins to communicate with GitHub by means of available plugins

2. Jenkins would contain the target repository URL to pull the repo contents

3. Build would happen locally referring to Maven pom.xml

4. Build status would be published and embedded on GitHub readme.md

J


Development Code Check-in at Local repository

Eclipse Git plugin pushes local changes to Git

repository

Git Repository uses WebHook API to push

notification to Jenkins for any changes observed

Jenkins triggers build for the changes notification and

pulls the changes from Git repository

Jenkins build status is published back to Git

repository 'readme.md' for instant notification of build

success / failure


Hardware Requirements 4 GB RAM recommended

3 GHz CPU recommended

Software Requirements

No Software URL

1 Apache

Tomcat v7.0

http://tomcat.apache.org/download-70.cgi

2 Java Runtime

Environment

6.0

http://www.oracle.com/technetwork/java/javase/downloads/jre6downloads-

1902815.html

3 Apache

Maven 3.0

http://maven.apache.org/download.cgi

4 Jenkins

Distributable

WAR

http://mirrors.jenkins-ci.org/war/latest/jenkins.war

5 GitHub

Repository

http://www.github.com

6 Git

Installation

http://git-scm.com/download/win

7 Eclipse Git

Plugin

www.eclipse.org/egit/download


Configure Git Plugin with Eclipse IDE

Whenever the history of the repository is changed (technically, whenever a commit is created), Git keeps

track of the user who created that commit. The identification consists of a name (typically a person's

name) and an e-mail address. This information is stored in file ~/.gitconfig under dedicated keys.

EGit will ask you for this information when you create your first commit. By default, this dialog is shown

only once until you create a new workspace or tick the checkbox "Show initial configuration dialog" on

the Git Preference page:

You can also un-tick "Don't show this dialog again" if you want to see it again later.

Instead of using this dialog, you can always change this information using the Git configuration:

Click Preferences > Team > Git > Configuration

Click New Entry and enter the key value pairs user.email and user.name


Create Repository

Create a new Java project HelloWorld. (In this case, the project was built outside of your Eclipse

Workspace.)


Select the project, click File > Team > Share Project.

Select repository type Git and click Next.

To configure the Git repository select the new project HelloWorld.


Click Create Repository to initialize a new Git repository for the HelloWorld project. If your project

already resides in the working tree of an existing Git repository the repository is chosen

automatically.

Click Finish to close the wizard.

The decorator text "[master]" behind the project shows that this project is tracked in a repository on

themaster branch and the question mark decorators show that the .classpath and .project and

the .settingsfiles are not yet under version control.


Track Changes

Click Team > Add on the project node. (This menu item may read Add to Index on recent versions

of Egit.)

The + decorators show that now the project's files have been added to version control.

Mark the "bin" folder as "ignored by Git", either by right-clicking on it and selecting Team >

Ignore or by creating a file .gitignore in the project folder with the following content:

/bin

This excludes the bin folder from Git's list of tracked files.

Add .gitignore to version control (Team > Add):

You may have to set your Package Explorer filters in order to see .gitignore displayed in the Package

Explorer. To access filters, select the down arrow on the right edge of the Package Explorer tab to

display View Menu.

Select Filters... from the View Menu and you will be presented with the Java Element Filters dialog.

Unselect the top entry to display files that begin with . (period) such as .gitignore.


Click Team > Commit in the project context menu.

Enter a commit message explaining your change, the first line (followed by an empty line) will

become the short log for this commit. By default the author and committer are taken from

the .gitconfig file in your home directory.

You may click Add Signed-off-by to add a Signed-off-by: tag.

If you are committing the change of another author you may alter the author field to give the name

and email address of the author.

Click Commit to commit your first change.

Note that the decorators of the committed files have changed as a result of your commit.


Inspect History

Click Team > Show in History from the context menu to inspect the history of a resource:

Create a new Java class Hello.java and implement it.

Add it to version control and commit your change.

Improve your implementation and commit the improved class.

The resource history should now show 2 commits for this class.


Click the Compare Mode toggle button in the History View.

Double click src/Hello.java in the Resource list of the History View to open your last committed

change in the Compare View.


Congratulations, you just have mastered your first project using Git!!!


GitHub Tutorial

Create Local Repository

Follow EGit/User Guide/Getting Started to create a new local repository (with your content instead

of the demo project).

Create Repository at GitHub

create a new repository at GitHub:

On the next screen you can see the URLs you may use to access your fresh new repository:

click SSH to choose the SSH protocol. It can be used for read and write access.

click HTTP to choose the HTTP protocol. It can also be used for read and write access.

click Git Read-Only to choose the anonymous git protocol for cloning. It's the most efficient protocol

git supports. Since the git protocol doesn't support authentication it's usually used to provide

efficient read-only access to public repositories.


Eclipse SSH Configuration

Open the Eclipse Preferences and ensure that your SSH2 home is configured correctly (usually this

is~/.ssh) and contains your SSH2 keys:


if you don't have SSH keys yet you may generate them on the second tab of this dialog (Key

Management). Use a good pass phrase to protect your private key, for more details see "working

with key passphrases".

upload your public SSH key to your GitHub account settings.

Push Upstream

Click Team > Remote > Push... and copy and paste the SSH URL of your new GitHub repository.

If you are behind a firewall which doesn't allow SSH traffic, use the GitHub HTTPS URL instead and

provide your GitHub user and password instead of using the uploaded public SSH key. To store your

credentials into the Eclipse secure store click Store in Secure Store.

Note: many HTTP proxies are configured to block HTTP URLs containing a user name, since

disclosing a user name in an HTTP URL is considered a security risk. In that case remove the user

name from the HTTP URL and only provide it in the user field. It will be sent as an HTTP header.

http://help.github.com/working-with-key-passphrases/

http://help.github.com/working-with-key-passphrases/

https://github.com/account


Click Next and on first connection accept GitHub's host key.

Enter your SSH key's passphrase and click OK.

On the next wizard page click Add all branches spec to map your local branch names to the same

branch names in the destination repository (on a one-to-one basis).

Click Next. The push confirmation dialog will show a preview of the changes that will be pushed to

the destination repository.


Click Finish to confirm that you want to push these changes.

The next dialog reports the result of the push operation.


Point your browser at your GitHub repository to see that your new repository content has arrived.


Download and Install Java

1. Download and install java using the setup

2. Confirm Java installation by running command ‘java -version’ (without quotes )on the command

prompt


Configure Apache Tomcat

Installing Tomcat on Windows can be done easily using the Windows installer. Its interface and

functionality is similar to other wizard based installers, with only a few items of interest.

Installation as a service: Tomcat will be installed as a Windows service no matter what setting

is selected. Using the checkbox on the component page sets the service as "auto" startup, so that

Tomcat is automatically started when Windows starts. For optimal security, the service should

be run as a separate user, with reduced permissions (see the Windows Services administration

tool and its documentation).

Java location: The installer will provide a default JRE to use to run the service. The installer uses

the registry to determine the base path of a Java 6 or later JRE, including the JRE installed as part

of the full JDK. When running on a 64-bit operating system, the installer will first look for a 64-

bit JRE and only look for a 32-bit JRE if a 64-bit JRE is not found. It is not mandatory to use the

default JRE detected by the installer. Any installed Java 6 or later JRE (32-bit or 64-bit) may be

used.

Tray icon: When Tomcat is run as a service, there will not be any tray icon present when Tomcat

is running. Note that when choosing to run Tomcat at the end of installation, the tray icon will be

used even if Tomcat was installed as a service.

Refer to the Windows Service HOW-TO for information on how to manage Tomcat as a Windows

service.

The installer will create shortcuts allowing starting and configuring Tomcat. It is important to note that

the Tomcat administration web application can only be used when Tomcat is running.

Further help: https://tomcat.apache.org/tomcat-7.0-doc/setup.html

https://tomcat.apache.org/tomcat-7.0-doc/windows-service-howto.html

https://tomcat.apache.org/tomcat-7.0-doc/setup.html


For manual startup of Tomcat, browse to <TOMCAT_DIRECTORY>\bin\startup.bat

On successful configuration of Tomcat, open browser and type http://localhost:8080 in address bar,

enter.

http://localhost:8080/


Changing the Tomcat Default 8080 port

In order to change default 8080 port of tomcat, please navigate to

<TOMCAT_DIRECTORY>\conf\server.xml

Modify default 8080 port to other port.

NOTE:

Ensure no other changes are done in Server.xml.

Tomcat restart would be required for reflecting port changes

Ensure CATALINA_HOME is present in Environment Properties My Computer -> Properties -> Advanced -> Environment Variables -> User Variables


Configure Apache Maven Maven is not installable. Unzip the download in some directory and update M3_HOME environment

variable in System.

Confirm Maven Installation by running ‘mvn -version’ at command prompt:


Configure Jenkins Place downloaded archive ‘Jenkins.war’ in <TOMCAT>\webapps folder and start the Tomcat server.

Browser to URL: http://localhost:8080/jenkins

http://localhost:8080/jenkins


Go to Manage Jenkins -> Manage Plugins -> Available Plugins and in filter box type ‘git’ and download

below plugins:


Download GitHub related plugins

Restart Jenkins to install the plugins and see further configuration.


Configure System

Go to Manage Jenkins -> Configure System

Specify JDK, Maven Installation and Git installation


Create a new job in Jenkins

Go to URL: http://localhost:8080/jenkins and click on create a job

Select Build a maven2/3 project and specify Item name as below:

Once project created specify the GitHub project location:

Specify the Source Code for GitHub and provide credentials

http://localhost:8080/jenkins


Adding GitHub Credentials

To generate SSH key for GitHub

Step 1: Check Existing keys

First, we need to check for existing SSH keys on your computer. Open up your Git Bash and type:

Check the directory listing to see if you have files named either id_rsa.pub or id_dsa.pub. If you don't

have either of those files go to step 2. Otherwise, you can skip to step 3.

Step 2: Generate a new SSH key

To generate a new SSH key, copy and paste the text below, making sure to substitute in your email. The

default settings are preferred, so when you're asked to "enter a file in which to save the key,"" just press

enter to continue.


Next, you'll be asked to enter a passphrase.

Which should give you something like this:

Then add your new key to the ssh-agent:

Step 3: Add your SSH key to GitHub

Run the following code to copy the key to your clipboard.

Alternatively, using your favorite text editor, you can open the ~/.ssh/id_rsa.pub file and copy the

contents of the file manually

Now that you have the key copied, it's time to add it into GitHub:

1. 2. In the user bar in the top-right corner of any page, click Account Settings.

https://github.com/settings/admin


3. Click SSH Keys in the left sidebar.

4. Click Add SSH key.

5. In the Title field, add a descriptive label for the new key. For example, if you're using a personal

Mac, you might call this key "Personal MacBook Air".

6. Paste your key into the "Key" field.

https://github.com/settings/ssh


7. Click Add key.

8. Confirm the action by entering your GitHub password.


Step 4: Test everything out

To make sure everything is working, you'll now try SSHing to GitHub. When you do this, you will be

asked to authenticate this action using your password, which for the passphrase you created earlier.

Open up your Git Bash and type:

You may see this warning:

Type “yes”


Test GitHub Jenkins Integration

Check Console Output for successful execution


Build, if successful will be denoted by BLUE dot whereas failed build would be denoted by RED dot.

You can see the build status in the project home page.


Create User in Jenkins for Remote Job Builds

To automate the process of building, create users in Jenkins

Initial Steps:

1. Go to the Configure Global Security screen (http://server/jenkins/configureSecurity/) and

choose "enable security". An alternate URL to try is http://server:8080/configureSecurity.

2. Select "Jenkins's own user database" as the security realm

3. Place a check mark next to "Allow users to sign up"

4. Select "Matrix-based security" as the authorization

5. Give anonymous user the read access

6. In the text box below the table, type in your user name (you'd be creating this later) and click

"add"

7. Give yourself a full access by checking the entire row for your user name

8. Scroll all the way to the bottom, click "save"

The configuration should look like the picture below:

At this point, you'll be taken back to the top page, and Jenkins is successfully secured.

Restart Jenkins (service jenkins restart)

Now you need to create an user account for yourself.

http://server/jenkins/configureSecurity/

http://server:8080/configureSecurity


1. Click "login" link at the top right portion of the page

2. Choose "create an account"

3. Use the user name you've used in the above step, and fill in the rest.

If everything works smoothly, you are now logged on as yourself with full permissions.

If something goes wrong, follow below steps:

One may accidentally set up security realm / authorization in such a way that you may no longer able to

reconfigure Jenkins.

When this happens, you can fix this by the following steps:

1. Stop Jenkins (the easiest way to do this is to kill the servlet container.)

2. Go to $JENKINS_HOME in the file system and find config.xml file.

3. Open this file in the editor.

4. Look for the <useSecurity>true</useSecurity> element in this file.

5. Replace true with false

6. Remove the elements authorizationStrategy and securityRealm

7. Start Jenkins

When Jenkins comes back, it's in the unsecured mode where everyone gets full access to the system.

If this is still not working, trying renaming or deleting config.xml.

We have created user github/github for GitHub to push Jenkins the push information.

In Project Home click on Project Based Security to enable this feature and sample configuration is as

shown below:


In Build Triggers add below information on Project Homepage in Jenkins


Push GitHub Changes to Jenkins

In order to push any changes such as code commit, branch created those events GitHub can push to

Jenkins.

Go to GitHub project settings:


Add Webhook URL as below format:

http://github:github@<JENKINS_HOST>:<PORT>/jenkins/job/<JOB_NAME>/build?token=<LONG_RAN

DOM_TOKEN_HERE>

Jenkins Host: Host IP where Jenkins is hosted (should be accessible over internet)

Port: Port of Web Server

Job_Name: Jenkins Job Name

LONG_RANDOM_TOKEN_HERE: Authentication token string

Here, when any new comment is pushed in GitHub / new branch created, above URL is used to push

information to remote Jenkins.

LONG_RANDOM_TOKEN_HERE: This is the secret token that we have set in above step

We can simulate the setup by committing single change in GitHub.


Showing Jenkins Build Status on GitHub Download Jenkins plugin Embeddable Build Status and restart Jenkins.

Simply click on the link and screen like below would be shown

You can copy MARKDOWN UNPROTECTED link contents and in GitHub create new file called

‘readme.md’ and paste this content.

To get IP address for Jenkins Hosted machine:

(Make sure localhost is to be replaced with host IP)

Host IP can be obtained by command -> ipconfig/all

Check IPv4 Address in console output.


Queries For any queries or suggestions, please get in touch:

[email protected] / [email protected]

mailto:[email protected]

mailto:[email protected]


THANK YOU

http://www.manojkolhe.com

Manoj Kolhe - Setup GitHub with Jenkins on Amazon Cloud - End-to-end Automation

Software

Transcript of Manoj Kolhe - Setup GitHub with Jenkins on Amazon Cloud - End-to-end Automation