Software Development and DeploymentIn an Oracle EnvironmentIn an Oracle Environment

Discussion OverviewI t d ti• Introduction

• Software Configuration Management• Development and Deployment roles• Development evolution• Ant control• Software deployment process• Software deployment process• Configuration Build details

IntroductionAbout meAbout me

My role as a Systems ConsultantR i t D fi iti• Requirements Definition

• Systems Design and Specifications• Manage overall application development project

• Client interactions• Client interactions• Software development activities• Requirements Clarification• Change ControlChange Control

• QA and Customer Acceptance• Release coordination and control• Ongoing configuration managementg g g g

IntroductionOur spaceOur space

Our typical application environment• Sun™ONE Web Server on Sun Solaris server• Sun™ONE Web Server on Sun Solaris server• Oracle 10g on Sun Solaris server • CVS or Subversion code repository• Eclipse IDEEclipse IDE• PL/SQL Developer IDE

Environment independencep• Development / QA / Production

IntroductionOur space another dimensionOur space – another dimension

Our typical application architectureW b b d UI JSP/J /JDBC• Web-based UI – JSP/Java/JDBC

• Middleware framework – Spring or Struts[2]• Oracle back end - PL/SQL packages containing procedures and functions• Unix Shell utilities Perl ksh embedded SQL• Unix Shell utilities - Perl, ksh, embedded SQL

Application glue• Web Services• Web Services• XML

Software Configuration ManagementAreas of influenceAreas of influence

When we say SCM we meanWhen we say SCM we mean• Control over source modules• Coordination of development activities• QA checkpoint controlQA checkpoint control• Versioning• Release control• Production instance control

Software Development and DeploymentWho does what

Software Developer• Write application code

Unit test code modules

Who does what

• Unit test code modules• Check-In tested code

Release Coordinator• Check-Out, Build, and Deploy

• Dev image• QA image

• Tag configuration for releaseTag configuration for release• Production Release

QAV if f ti lit• Verify functionality

• Open tickets to track defects

Project Managerj g• Manage Customer Acceptance process• Obtain customer sign-off prior to Production deployment

Evolution of PL/SQL Development Process

Unix shell editor• Save in shell• Run SQL/Plus and compile• Save in source control (SCCS)• Save in source control (SCCS)• Deploy manually

PL/SQL DeveloperC / P t t t d d t h ll fil f SCCS h k i• Copy / Paste tested code to shell file for SCCS check-in

• Build in PL/SQL Developer connected to Prod environment

Eclipse and Ant• Import into Eclipse via local PL/SQL Developer file• Versioning and Tagging• Scripted deployment of tagged release

Ant!

Consistent approach to building/deploying different languages• Java

PL/SQL• PL/SQL• Script (Perl, ksh)• SQL tasks

Ant build targets• Checkout• Build and Deploy• Build and Deploy• Tag Release• Deploy Release

Release IdentifiersVersioning 1 01

Production release identifier• Major

Versioning 1.01

• Major• Minor• Patch

Example: R_02-01-12

QA level release• Lower level to track sub-releases

Example: R_02-01-12-02

• Release script prompts for best guess at version

Releasing SQL scriptsOne time tasks

Tied to Ticket Tracking identifier• Option to handle Dev QA and Prod versions independently

One-time tasks

• Option to handle Dev, QA, and Prod versions independently• Executes appropriate script for target environment

Script typesScript types•SQL command (Insert, Update, etc) •PL/SQL Procedure •ant build scriptant build script

Release Process ControlSample worksheet

Build dir Env Activity Tag / Argument Completed Notessql Dev ant checkout 12/21/2009 3:47 PM

Sample worksheet

q / /2/5/2010 10:36 AM

Dev ant deploy 2/5/2010 10:36 AM

Dev ant tagrelease R_01‐03‐14‐00R_01‐03‐14‐01

12/21/2009 3:49 PM2/5/2010 10:36 AM

R_01‐03‐14‐02 3/7/2010 4:38 PM

QA ant deployrelease R_01‐03‐14‐00R_01‐03‐14‐01R_01‐03‐14‐02

12/21/2009 3:50 PM2/5/2010 10:36 AM3/7/2010 8:08 PM

QA ant release‐scripts 488000,489867 12/21/2009 3:59 PM2/5/2010 10:38 AM

Prod ant deployrelease R_01‐03‐14 3/7/2010 9:18 PM

Prod ant release‐scripts 488000,489867 3/7/2010 9:28 PM

Application Code Organization of ModulesOrganization of Modules

• PL/SQL code stored under one project in application comprised of java, perl, and shell scripts.– enotify‐build:  master build file and release.properties– enotify‐ui:  java web application

tif d t t t j b i– enotify‐orderstatuseventws:  java web service app– enotify‐sendmailws:  java web service app– enotify‐common:  common java code and third party libraries– enotify‐sql:  database code:  PL/SQL packages and procedures, SQL scripts– enotify‐data‐server‐tasks:  misc perl and shell scripts

SQL Projecty p p

– enotify‐support:  standalone java app

• All release steps controlled through standard targets in master build.xml and release.properties:  – checkout:  checkout project from HEAD or release tag

b ild b ild th t ( j t b ild l)– build:  build the component (per project build.xml)– deploy:  deploy the component (per project build.xml)– tagrelease:  tag the built component with release name, tracks release numbers– deployrelease:  checkout the tagged released component and deploy– release‐scripts:  execute any release scripts (e.g., pl/sql procedures, SQL scripts, and/or add’l build 

scripts) for the  release (driven by ticket number)

• Project build.xml scripts and build.properties files are unique for each type of project

Development Process

• Use PL/SQL Developer to edit and test PL/SQL code. Create and test SQL scripts• Save packages and/or scripts into local copy of CVS repository

Ch k i h i t it• Check in changes into repository 

• Perform release on development server

– ant checkout• Checks out latest code into build directory on Dev server

– ant tagreleaseant tagrelease• Prompts for release name defaulting to next QA release saved in properties. Tags files as 

QA release (e.g., R_01‐13‐04‐02) and production release candidate (e.g, R_01‐13‐04).  Updates properties storing latest release names and next release number.

d l l– ant deployrelease• Prompts for release name defaulting to last QA release name, checks out files from 

release tag and deploys (compiles new PL/SQL packages and recompiles any invalidated packages).

QA Deployment

• Deploy QA tagged release on QA server

– ant deployrelease• Prompts for release name defaulting to last QA release name, checks out files from 

release tag and deploysrelease tag and deploys

• Only builds items that have changed

– ant release‐scripts (optional)ant release scripts (optional)• Prompts for ticket numbers and executes any procedures, scripts and/or build 

scripts found for the ticket

Production Release

• Deploy tagged release on Production server

– ant deployrelease• prompts for release name defaulting to last production release name, checks out 

files from release tag and deploys• Only builds items that have changed

– ant release‐scripts (optional)• prompts for ticket numbers and executes any procedures scripts and/or build scripts• prompts for ticket numbers and executes any procedures, scripts and/or build scripts 

found for the ticket

Properties files

release.properties (shared by all projects):release.prefix=R_release.name.qa.last=R_01‐03‐14‐05release.patch.number.next=14release.minor.number.next=03l \ \

Release versioning parameters

release.create.timestamp=02‐18‐2010 09\:33\:16release.major.number.next=01release.name.last=R_01‐03‐14release.qa.number.next=06

build properties (unique for each project):build.properties (unique for each project):cvs.project.list=enotify‐sqlproject.name=ENotify Database

dir.development=/uv1157/u311/home/enotifydir.production=/uv1233/u311/home/enotifydir.qa=/uv1510/u01/home/enotify

jdbc.connection.development=host.apps.mc.xerox.com\:1571\:OSCPjdbc.user.development=userjdbc.password.development=password

jdbc.connection.qa=host.apps.mc.xerox.com\:1552\:OSCQjdbc.user.qa=userjdbc.password.qa=password

jdbc connection production=host apps mc xerox com\:1571\:OSCPjdbc.connection.production=host.apps.mc.xerox.com\:1571\:OSCPjdbc.user.production=userjdbc.password.production=password

! T th t t f l

Master build.xmltagrelease Target

<!‐‐ Tag the current component for release ‐‐><target name="tagrelease" depends="checkproperties">

<!‐‐ define cvsroot ‐‐><input message="Enter your cvs userid:" addproperty="cvs.userid"/><property name="cvsroot" value=":${cvs.protocol}:${cvs.userid}@${cvs.server}:${cvs.path}"/>

<!‐‐ generate the default qa release tag ‐‐><property name="default.qa.release.name" value="${release.prefix}${release.major.number.next}‐${release.minor.number.next}‐

${release.patch.number.next}‐${release.qa.number.next}"/>

<!‐‐ prompt for qa release name ‐‐><input message="Enter the qa release tag or press enter for default:" addproperty="qa.release.name" defaultvalue="${default.qa.release.name}"/><! V lid t l d d t b ild ti fil t t t l b ><!‐‐ Validate release name and update build.properties file to store next release numbers‐‐><if><equals arg1="${qa.release.name}" arg2="${default.qa.release.name}"/><then><!‐‐ if qa release name equals default qa release name then just increment qa number ‐‐><echo>Creating incremental release ${qa.release.name}.</echo>

</then>  /then<else><!‐‐ extract the numbers from the new qa release, save in properties ‐‐><propertyregex property="release.major.number.new"

input="${qa.release.name}"regexp="${release.prefix}(\d\d)‐(\d\d)‐(\d\d)‐(\d\d)"select="\1" />

$ /<fail unless="release.major.number.new" message="Invalid release name format.  Expect ${release.prefix}nn‐nn‐nn‐nn"/>….  Check minor, patch and qa number in same way

<echo>Creating new major release ${qa.release.name}.</echo><propertyfile file="release.properties"><entry key="release.major.number.next" type="int" operation="=" value="${release.major.number.new}" pattern="00"/><entry key="release.minor.number.next" type="int" operation="=" value="${release.minor.number.new}" pattern="00"/><entry key="release patch number next" type="int" operation="=" value="${release patch number new}" pattern="00"/><entry key= release.patch.number.next  type= int  operation= =  value= ${release.patch.number.new}  pattern= 00 /><entry key="release.qa.number.next" type="int" operation="=" value="${release.qa.number.new}" pattern="00"/></propertyfile></else>  </if>

tagrelease Target (cont.)

<!‐‐ generate the production release tag based on the qa release tag ‐‐><propertyregex property="release.name"

input="${qa.release.name}"regexp="${release.prefix}(\d\d)‐(\d\d)‐(\d\d)‐(\d\d)"select="${release.prefix}\1‐\2‐\3" />

h h d l ${ l } / h<echo>The production release name tag is:  ${release.name}</echo>

<!‐‐ update release properties file to include this release name as last release and increment next qa number‐‐><tstamp><format property="timestamp" pattern="MM‐dd‐yyyy hh:mm:ss"/></tstamp>    <propertyfile file="release properties"><propertyfile file release.properties ><entry key="release.qa.number.next" type="int" operation="+" default="00" pattern="00"/><entry key="release.name.last" value="${release.name}"/><entry key="release.name.qa.last" value="${qa.release.name}"/><entry key="release.create.timestamp" value="${timestamp}"/>

</propertyfile>

<!‐‐ commit the new release.properties ‐‐><copy file="release.properties" tofile="${cvs.build‐project}/release.properties"  failonerror="false"/><cvs cvsRoot="${cvsroot}" command="commit ‐m'${qa.release.name}' ${cvs.build‐project}/release.properties" failonerror="true"/>

<!‐‐ tag each project with release name and qa release name ‐‐><for list="${cvs.project.list}" param="cvs.project"><sequential>

Attach Tag to all files in hi j<sequential>

<ant target="createrelease" dir="@{cvs.project}"/> <cvs cvsRoot="${cvsroot}" command="tag ‐F ${release.name}" package="@{cvs.project}" failonerror="true"/><cvs cvsRoot="${cvsroot}" command="tag ‐F ${qa.release.name}" package="@{cvs.project}" failonerror="true"/></sequential></for>

this project

<echo>The component ${project.name} has been tagged for release:  ${release.name}.  qa release:  ${qa.release.name}</echo>

</target>

Master build.xmldeployrelease Target

<!‐‐ Checkout the release files from cvs and deploy ‐‐><target name="deployrelease" depends="checkproperties"><echo>Deploy a release for component ${ant.project.name} on ${env.HOSTNAME} [${env.ENVIRONMENT}]</echo>

<condition property="defaultReleaseValue" value="${release.name.qa.last}"><isset property="isDevelopment"/>

</condition><condition property="defaultReleaseValue" value="${release.name.qa.last}"><isset property="isQA"/><isset property= isQA />

</condition><condition property="defaultReleaseValue" value="${release.name.last}"><isset property="isProduction"/>

</condition><fail unless="defaultReleaseValue"/>/<input message="Enter release name :" addproperty="tag" defaultvalue="${defaultReleaseValue}"/>

<!‐‐ checkout from release tag ‐‐><ant target="checkout"/>  Get code for release<!‐‐ deploy release ‐‐><for list="${cvs.project.list}" param="cvs.project"><sequential><ant target="deployrelease" dir="@{cvs.project}"/> </sequential></for>

Call individual build targets

</for></target>

Individual build.xml SQL project p j

• deployrelease target<!‐‐ Required target:  deployrelease ‐‐><target name="deployrelease" ><!‐‐ same as regular deploy ‐‐><antcall target="deploy"/></target>/ g

• deploy target<target name="deploy"><!‐‐ Compile any package file from the cvs packages directory which is different than the package

in the deployed packages directory.  ‐‐><for param="package" ><for param= package  ><path><fileset dir="${cvs.dir.packages}"><different targetdir="${dir.deployed.packages}" ignoreFileTimes="true"/></fileset></path><sequential><sequential><echo>Deploy package @{package}</echo><antcall target="compilepackage"><param name="package.file" value="@{package}"></param>

</antcall></sequential>/f

Call for  package compile

</for>    <!‐‐ Recompile any invalidated packages caused by compilation ‐‐><antcall target="recompile‐invalid‐packages"/><echo>Check for compilation errors </echo><antcall target="check‐for‐compilation‐errors"/></target>

Recompile any invalid packages

Report compile errors

SQL project compilepackage Targetcompilepackage Target

<!‐‐ Compile the package: ${package.file} ‐‐>" l k "<target name="compilepackage">

<fail unless="package.file"/><basename file="${package.file}" property="package.filename"/><basename file="${package.file}" suffix="pck" property="package.name"/>

<echo>Copy the package ${package.file} to deploy directory:  ${dir.deployed.packages}</echo><copy file="${package.file}" tofile="${dir.deployed.packages}/${package.filename}" preservelastmodified="true" overwrite="true"/>

<echo>Compile the package ${dir.deployed.packages}/${package.filename}</echo><antcall target="execute‐plsql‐code"> Call package<param name="file" value="${dir.deployed.packages}/${package.filename}"/>

</antcall>

</target>

Call package compilation

SQL project execute‐plsql‐code targetexecute plsql code target

<!‐‐ Execute the pl/sql code in file: ${file} ‐‐><target name="execute‐plsql‐code" depends="jdbcsetup">  g p q p j p

<fail unless="file"/><fail unless="jdbc.user"/><fail unless="jdbc.password"/><fail unless="jdbc.url"/>

<echo>Execute the PL/SQL code from file ${file}</echo><sql driver="oracle.jdbc.OracleDriver"

url="${jdbc.url}"userid="${jdbc.user}"password="${jdbc password}" JDBC call to Oracle for password= ${jdbc.password}classpathref="sql.classpath"delimitertype="row"delimiter="/"keepformat="yes"escapeprocessing="false"

fpackage compilation

onerror="abort"><transaction src="${file}"/>

</sql>

</target>

SQL project jdbcsetup targetjdbcsetup target

<target name="jdbcsetup">

<condition property="jdbc.user" value="${jdbc.user.development}"><equals arg1="${isDevelopment}" arg2="true"/>

</condition>…<condition property="dir.deployed.packages" value="${dir.development}/packages"><equals arg1="${isDevelopment}" arg2="true"/>

</condition>

<condition property="jdbc.user" value="${jdbc.user.qa}"><equals arg1="${isQA}" arg2="true"/><equals arg1 ${isQA}  arg2 true />

</condition>…<condition property="dir.deployed.packages" value="${dir.qa}/packages"><equals arg1="${isQA}" arg2="true"/>

</condition>

<condition property="jdbc.user" value="${jdbc.user.production}"><equals arg1="${isProduction}" arg2="true"/>

</condition>…<condition property="dir.deployed.packages" value="${dir.production}/packages"><equals arg1="${isProduction}" arg2="true"/>

</condition></condition>

<fail unless="jdbc.user">Cannot determine the jdbc user property.</fail>…<fail unless="dir.deployed.packages">Cannot determine the dir.deployed.packages property.</fail>

</target>

SQL project recompile‐invalid‐packages targetrecompile invalid packages target

<target name="recompile‐invalid‐packages">

<echo>Recompile invalidated packages using procedure in file:  ${file.proc.recompile.invalid.packages}</echo><antcall target="execute‐plsql‐code">

<param name="file" value="${file.proc.recompile.invalid.packages}“</antcall>

</target>

file.proc.recompile.invalid.packagesdeclare

cursor myCursor is select object_type, object_namefrom SYS.USER_OBJECTS where  status = 'INVALID'order by 1, 2;

sqlString varchar2(100);

Cursor of invalidated objects

beginfor r1 in myCursor loopif r1.object_type = 'PACKAGE BODY' thensqlString := 'alter PACKAGE '||r1.object_name||' compile BODY';elsif r1.object_type = 'TYPE BODY' thensqlString := 'alter TYPE '||r1.object name||' compile BODY';

Compile packages and other objects

q g || j _ || p ;elsesqlString := 'alter '||r1.object_type||' '||r1.object_name||' compile';end if;execute immediate sqlString;end loop;end;end;

SQL project check‐for‐compilation‐errorscheck for compilation errors

<target name="check‐for‐compilation‐errors" depends="jdbcsetup">

<echo>Check for compilation errors in package :  ${package}</echo><sql driver="oracle.jdbc.OracleDriver"

url="${jdbc.url}"url ${jdbc.url}userid="${jdbc.user}"password="${jdbc.password}"classpathref="sql.classpath"delimitertype="row"delimiter="/"keepformat="yes"escapeprocessing="false"onerror="abort"><transaction><![CDATA[

BEGIN

DECLARESQL to check for 

DECLARENUM INTEGER;ERROR_TEXT USER_ERRORS.TEXT%TYPE;

SELECT Count(*) INTO NUM FROM USER_ERRORS WHERE Type = 'PACKAGE' OR Type = 'PACKAGE BODY' AND NAME = '${package}';

If num > 0 Then

compile errors

SELECT TEXT INTO ERROR_TEXT FROM USER_ERRORS WHERE Type = 'PACKAGE' OR Type = 'PACKAGE BODY' AND NAME = '${package}' AND SEQUENCE = 1;RAISE_APPLICATION_ERROR(‐20000, ERROR_TEXT, True);End IF;

END;]]>

</transaction></transaction></sql>

</target>

SQL project release‐scripts targetrelease scripts target

<target name="release‐scripts" depends="jdbcsetup">

<!‐‐ prompt for ITT ticket number(s) ‐‐><input message="Enter ITT Ticket number(s) (without prefix ITT) separated by commas:" addproperty="itt.numbers"/>

<!‐‐ For each ticket attempt to find and execute an sql script, a procedure, and/or build script based on ITT number and environment. ‐‐>

<for list="${itt numbers}" param="itt number"><for list= ${itt.numbers}  param= itt.number ><sequential><antcall target="execute‐script" ><param name="file" value="${cvs.dir.scripts}/ITT@{itt.number}.sql"></param><param name="ITT" value="ITT@{itt.number}"/>

</antcall>

Call to run SQL command

/<antcall target="execute‐procedure" ><param name="file" value="${cvs.dir.scripts}/ITT@{itt.number}.prc"></param><param name="ITT" value="ITT@{itt.number}"/>

</antcall><antcall target="execute‐build‐script">

Call to run SQL procedure

Call to run ant<param name="build.script.file" value="${cvs.dir.scripts}/ITT@{itt.number}.xml"></param><param name="ITT" value="ITT@{itt.number}"/>

</antcall></sequential>

</for>

Call to run ant build script

</target>

SQL project execute‐script targetexecute script target

<target name="execute‐script" depends="check‐for‐environment‐file">  

<available file="${environment.file}" property="file.exists"/><if><equals arg1="${file.exists}" arg2="true"/><then>

h f l ${ f l } / h<echo>Execute script in file ${environment.file}.</echo><antcall target="execute‐sql‐command"><param name="file" value="${environment.file}"/>

</antcall></then>

Call to run SQL command

/</if>            

</target>

SQL project execute‐sql‐command targetexecute sql command target

<target name="execute‐sql‐command"  depends="jdbcsetup">  

<fail unless="jdbc.url"/>

<echo>Execute sql in file ${file}</echo><sql driver="oracle.jdbc.OracleDriver"

l "${ db l}" JDBC call to Oracle forurl="${jdbc.url}"userid="${jdbc.user}"password="${jdbc.password}"classpathref="sql.classpath"keepformat="yes">

JDBC call to Oracle for SQL command execution

p y<transaction src="${file}"/>

</sql>

</target>

SQL project execute‐procedure targetexecute procedure target

<target name="execute‐procedure" depends="check‐for‐environment‐file">  

<available file="${environment.file}" property="file.exists"/><if><equals arg1="${file.exists}" arg2="true"/><then><echo>Execute script in file ${environment file} </echo><echo>Execute script in file ${environment.file}.</echo><antcall target="execute‐plsql‐code"><param name="file" value="${environment.file}"/>

</antcall></then>

Call SQL script execution

</if>            

</target>

SQL project execute‐build‐script targetexecute build script target

<target name="execute‐build‐script">  

<available file="${build.script.file}" property="file.exists"/><if><equals arg1="${file.exists}" arg2="true"/><then><then><echo>Execute script in file ${build.script.file}.</echo><ant antfile="${build.script.file}" target="execute" inheritRefs="true"/>

</then></if>            

Execute ant build script

</target>

Software Development and DeploymentIn an Oracle EnvironmentIn an Oracle Environment

Build and Deploy steps

• Development server• Checkout• Build / verify • Tag Release (assign QA build ID)

• QA server• Deploy release and execute scriptsDeploy release and execute scripts• Validate • Approve for Production

• Production server• Production server• Deploy release and execute scripts• Verify

Software Development and DeploymentIn an Oracle EnvironmentIn an Oracle Environment

Ant Build Targets

• checkout• tagrelease• deployrelease

• checkout• Call individual project deployrelease targets

• compilepackage• execute-plsql-codep q

• recompile-invalid-packages• execute-plsql-code

• check-for-compilation-errors

• release-scripts• execute-script

• execute-sql-commandt d• execute-procedure

• execute-plsql-code• execute-build-script