1.

2.

3.

Using the TFS MSBuild scriptsThis example describes how to set up a database deployment pipeline using TFS and Redgate tools. 

You'll use the DLM Automation TFS MSBuild scripts to build, test, sync and publish a NuGet package to your release management tool.

 

1. Install software and download demo files 

Make sure you have the following: 

Access to Team Foundation Server (TFS) 2010, 2012, 2013 or 2015. This tutorial series uses TFS 2012.Visual Studio  2010, 2012, 2013 or 2015. This tutorial series uses Visual Studio 2012.For details of which TFS and Visual Studio versions can work together, see Compatibility between Team Foundation clients and Team

 (MSDN article). Foundation Server

 

SQL Server Management Studio (SSMS) 2008 (RTM, SP1, SP2, R2, R2 SP1, and R2 SP2), 2012, or 2014 (CTP2)Access to a SQL Server 2012 or 2014 instance with common language runtime (CLR) integration enabled. For more information, see Enabling

 (MSDN article). This tutorial uses SQL Server 2012.CLR integrationRedgate SQL Source Control installed on the same machine as SSMS. See SQL Source ControlThe WidgetShop demo files. Download the   file, and extract the contents.WidgetDevelopmentDatabaseCIDemo.zip.NET 4.5 installed on your local machine. This program is installed automatically with Visual Studio 2012 and 2013. If you're using Visual Studio 2010, you'll need to install it manually. See   (Microsoft Download Center)Microsoft .NET Framework 4.5DLM Automation, installed on the machine running the build agent. See Installing (DLM Automation documentation). Redgate's DLM Automation TFS MSBuild scripts copied to the machine running the build agent. See  .Setting up the TFS MSBuild scripts

2. Create a team project in TFS

In Visual Studio, click  :Connect to Team Foundation Server

Select or add your TFS server and then click  . ConnectFor details of how to connect to an instance of TFS for the first time,   (MSDN article)Connect to Team Projects in TFSFrom the   menu, select      .File New > Team Project  The Specify Team Project Settings page is displayed.

Most of this tutorial describes how to set up TFS and SQL Source Control. If you're only interested in how the DLM Automation TFS Build plugin works, start from step 7, .Configure the properties files

If you want to publish test results in MSTest format, you'll need to have Visual Studio Premium, Ultimate or Test Pro installed on the machine running your build agent.

4.

5. 6. 7. 8.

9.

1. 2.

3.

Enter the project name  :WidgetShop

Click  .NextOn the  page, leave the default process template selected and click  .Select a Process Template NextOn the  page, select  and click  .Specify Source Control Settings Create an empty source control folder  NextOn the  page, click   to start creating the project. Confirm Team Project Settings FinishOnce creation is complete, the   page is displayed. Team Project CreatedClick  .Close

3. Set up version control for the project

In Visual Studio, open Source Control Explorer and select  . WidgetShopClick   to create a workspace mapping:Not mapped

The   window is displayed. MapSelect a local folder to map the   root to, for example  :$/WidgetShop D:\tfs\WidgetShop

4.

5. 6.

7.

8.

9. 10.

11. 12.

1. 2. 3. 4.

1. 2.

Click  . The following message is displayed:Map

Click   to check out the contents of the   project to your local folder. Yes WidgetShopOpen the local folder you mapped the version control location to, and copy the downloaded WidgetShop folder to it (extracted from the downloaded   you installed at the start).WidgetDevelopmentDatabaseCIDemo.zipThis folder contains the database scripts folder that's been created for you using SQL Source Control (WidgetShop > Database > ScriptsFolder). You'll create the DLM Automation project file in this folder later.  In Visual Studio, use the Source Control Explorer to add the files you've just copied to version control. To do this, right-click on   and WidgetShopselect  .Add Items to FolderOn the dialog, select every item, and click  .Add to Source Control NextSelect every item on the  and click  . Items to add FinishThe added folders are listed in Source Control Explorer.Right-click on   and select  .WidgetShop Check In Pending ChangesIn Team Explorer, click  : Check In

The files you need are now in TFS version control and using SQL Source Control.

4. Create databases

Create two databases:

the   database that you'll use for shared development in this tutorial.WidgetDevelopmenta blank database called   Once you've configured the sync step,   will be updated automatically every time a change is made to WidgetCI. WidgetCIthe   database.WidgetDevelopment

Create the development database:

Open SQL Server Management Studio (SSMS).From the  menu, select  .File  Open > FileBrowse to the   file (extracted from the   folder) and click  .WidgetDevelopment.sql WidgetDevelopmentDatabaseCIDemo OpenClick  to create the  database.Execute  WidgetDevelopment 

Create the WidgetCI database on a local server:

In SSMS, click  . New QueryExecute the following SQL query to create the   database: WidgetCI

CREATE DATABASE WidgetCIGOUSE WidgetCIGOALTER DATABASE WidgetCI SET TRUSTWORTHY ON

If your TFS build agent is on a different machine, you'll need to install DLM Automation on that machine.

1. 2. 3. 4. 5. 6. 7. 8.

9. 10.

1. 2.

3. a. b.

5. Link the development database to version controlUse SQL Source Control to link the   database to TFS version control. We're using the dedicated developer model, so each developer WidgetDevelopmenthas their own copy of the database. For more information, see  .Database development models

 In SSMS Object Explorer, right-click on the WidgetDevelopment database and select Link database to source control.On the Link to source control page under Source control system, select Team Foundation Server (TFS).Enter the server URL, for example: http://mytfsserver/tfsIn the database folder text box, browse to the scripts folder located at WidgetShop > WidgetShop > Database > ScriptsFolderUnder , select Development model Dedicated database.Click Link. Once the link to source control has been confirmed, click OK.Click the   tab. Commit changes We want to check in every object in the list, so select the check box at the top of the list:

 Enter a comment to describe the changes you're about to check in, for example, new database files added. Click the Commit button. After a few seconds, a message confirms that changes are committed.

6. Set up the build in Visual Studio

In Team Explorer, click     and then click  .Home BuildsClick  :New Build Definition

On the General tab:in the   text box, type  .Build definition name WidgetDevelopmentunder , select  . We'll enable this once configuration is complete.Queue processing Disabled

3.

4.

5. a. b.

6.

7.

Click   and select  :Trigger Continuous Integration

Click  . Workspace Make sure the   source control folder is displayed. If it isn't:$/WidgetShopClick in the   field, and then browse to and select the   folder.Source Control Folder WidgetShopClick  .OK

Click  . Under Build Defaults   Staging , select  and enter the shared location that you  location Copy build output to the following drop folder want to use as your drop folder.

6.

7.

a.

b.

c.

d.

8. 9.

10.

Click  . You need to configure a number of build process parameters on this page.Process

Under 1. Required > Items to Build > Projects to Build, click the browse button. Click   and select Add DLMAutomation.proj, then click on each of the two dialogs.OK   

If you can't see the project file, select MSBuild Project files (*.*proj) from the Items of type drop-down list.Under  , replace the default format with 2. Basic > Build Number Format 1.0$(Rev:.r)This means your builds will be numbered  , and so on. The second '.' is added automatically before the final digit, 1.0.1, 1.0.2, 1.0.3because it's defined in  . (Rev:.r)Under   click the   expand arrow. Click   and select the TFS server name from the drop-down 3.  Agent Settings Advanced > , Name Filterlist, for example, dptc12 - Agent1.Scroll down to the   properties list and click  . Select   from the drop-down list.3. Advanced MSBuild Platform X86

The Process page should look similar to this: 

Press   to save the build definition.Ctrl+S Under   near the top of the page, clickDefault Template Show details. The build process XAML file and version control path are displayed:  

You need to configure the XAML file so that TFS passes the build number and binaries directory as variables. To do this, complete the instructions in  .Passing TFS Build properties to MSBuild using TFS 2010 or later

The build definition is now set up.

7. Configure the properties files

Properties files contain commands that configure your TFS build steps. The properties files are:

DLMAutomation.global.properties.build.properties DLMAutomation.test.propertiesDLMAutomation.sync.propertiesDLMAutomation.publish.propertiesDLMAutomation

1. 2.

3.

4.

1. 2.

3. a.

b. 4.

5.

1.

2.

In this example, we're going to configure the global and build properties files. 

Configure the global properties file

Global commands are required by every TFS build step. You can use most of the default option settings.

In Visual Studio, go to   and open Source Control Explorer $\WidgetShop\dlmAutomation-steps\DLMAutomation.global.properties.Set  the package ID. For example,WidgetPackage as  <packageId> </packageId>WidgetPackageThis is the name of the package you're going to create.At the   option, enter your DLM Automation license key if you have one. For example, licenseSerialKey <licenseSerialKey>123-456-

</licenseSerialKey>789012-ABCDIf the license has been activated on this machine already, the key is displayed. If you don't have a serial number, leave this option blank and a 28 day free trial will start automatically.Press Ctrl+S to save the file.

Configure the build properties file

Build commands create and validate the SQL creation script used to generate the NuGet package. For more information, see . Build step command options

If you're using   (version 1.4.4.313 or later), you can send information to it about the packages DLM Automation builds. Once the Redgate's DLM Dashboardpackages are deployed, they're displayed on the dashboard as recognized schema. For more information, see   (DLM DLM Automation integrationDashboard documentation).

Open $\WidgetShop\dlmAutomation-steps\ .build.properties.DLMAutomationAt the scriptsFolder option enter the path to the database scripts folder in source control relative to the project file. For example:<scriptsFolder>. </scriptsFolder>\WidgetShop\Database\ScriptsFolder  If you're using DLM Dashboard:

at the dlmDashboardHost option enter the name or IP address of the machine hosting DLM Dashboard. For example:<dlmDashboardHost> </dlmDashboard>localhostat the dlmDashboardPort option, enter the port number. Ignore this if you're running the DLM Dashboard on default port 19528.

If you want to include SQL Doc database documentation in the NuGet package, at the includeDocs option, change the value from False to True so that it looks like this:<includeDocs> </includeDocs>True

The documentation is stored in     in the NuGet package. db > docs > main.html If you're using DLM Dashboard version 1.6.3 or later and have set up DLM Dashboard integration, you can also view the documentation directly from DLM Dashboard. See DLM Automation integration (DLM Dashboard documentation). For help understanding SQL Doc documentation, see What's in the documentation? Press   to save the file.Ctrl+S

In Source Control Explorer, select the edited property files, right-click and select Check in Pending Changes. 

Configure the test properties file 

Test commands generate test data using SQL Data Generator and run tSQLt tests (such as static analysis, unit or integration tests) against a temporary version of your database. You'll use the   file extracted from the downloaded   file. For WidgetTestData.sqlgen WidgetDevelopmentDatabaseCIDemo.zipmore information, see Test step command line options.

The database schema for WidgetShopDevelopment includes four basic SQL Cop tests that will check for:

procedures named SP_procedures using dynamic SQL without sp_executesqlprocedures with @@Identityprocedures with SET ROWCOUNT

To configure the file:

Open $\WidgetShop\dlmAutomation-steps\ .test.properties.DLMAutomation

Including database documentation will increase the size of the NuGet package and the time it takes to build, particularly for large databases.

In this example, the  option will use LocalDB by default. If you didn't install LocalDB on the build agent or you want to temporaryDatabaseServeruse a different temporary server, you'd need to specify it here. If you want to run full-text queries against your tables, you'll need to use a different temporary server because LocalDB doesn't support Full-Text Search. For example:

<temporaryDatabaseServer> </temporaryDatabaseServer>.\SqlServer2012

You'll also need to enter a username and password for the database.

If you want to publish test results in MSTest format, you'll need to have Visual Studio Premium, Ultimate or Test Pro installed on the machine running your build agent.

2. 3. 4.

5.

1. 2. 3.

4.

5. 6. 7. 8.

1. 2. 3. 4.

5.

Set the disabled option to  . For example, False <disabled> </disabled>FalseSet the generateTestData option to  . For example, True <generateTestData> </generateTestData>TrueAt the dataGeneratorProjectFile option, enter the path to the SQL Data Generator file in source control relative to the project file. For example, <dataGeneratorProjectFile>. </dataGeneratorProjectFile>\WidgetShop\Database\WidgetTestData.sqlgenThis will populate the temporary database with test data from the file. For more information about SQL Data Generator, see SQL Data Generator

. 3 documentationPress   to save the file.Ctrl+S

Test results are output in JUnit XML format and saved in the drop folder you specified when setting up your build definition.

In Source Control Explorer, select the edited property files, right-click and select Check in Pending Changes.

 Configure the sync properties file

Properties files contain commands that configure your TFS build steps. You've already configured the global and build properties files earlier (see Set up ).the build in Visual Studio

Sync commands update another database (in this example, the   database) with the latest version in source control. For more information, WidgetShopCI

see .Using the sync command

In Visual Studio, go to   and open Source Control Explorer $\WidgetShop\ -stepsdlmAutomation \ .sync.properties.DLMAutomationSet the disabled option to  . For example, False <disabled> </disabled>FalseAt the databaseServer option, enter the name of the target database server you want to update. This is the server on which you created the   Widge

 database earlier. For example, tShopCI <databaseServer> </databaseServer>StagingServerNameAt the databaseName option, enter the name of the target database you want to update. For example,   <databaseName> <WidgetShopCI/databaseName>At the databaseUserName option, enter your username for SQL authentication. At the databasePassword option, enter your password for SQL authentication. Press   to save the file.Ctrl+SIn Source Control Explorer, select the edited property files, right-click and select Check in Pending Changes. 

Configure the publish properties file

Properties files contain commands that configure your TFS build steps. You've already configured the global and build properties files in the earlier (see Set).up the build in Visual Studio

 In Visual Studio, go to Source Control Explorer and open $\WidgetShop\ -stepsdlmAutomation \ .publish.properties.DLMAutomationSet the disabled option to  . For example, False <disabled> </disabled>FalseAt the nugetFeedUrl option, enter the fully-qualified URL for your NuGet feed.  If you're using a private NuGet feed, you'll need to specify the nugetFeedApiKey option. For example, <nugetFeedApiKey>DO8XW4CI7UIROVC

</nugetFeedApiKey>FG4TYTC2DXM8If you're using a public NuGet feed, the API key isn't required. In Source Control Explorer, select the edited property file, right-click and select Check in Pending Changes. 

Trigger a build

Make a change to the   database in SSMS, and use SQL Source Control to commit the change. WidgetDevelopment

A build will start automatically: 

Again, the  option will use LocalDB by default. If you didn't install LocalDB on the build agent or you want to use a temporaryDatabaseServerdifferent temporary server, you need to specify it here.

 

Once the test step is complete, click Open Drop Folder. The NuGet package, test reports file and logs are displayed:

Once the publish step is complete, the package is available on your NuGet feed.

Release management

1. 2.

3.

1. 2. 3. 4. 5. 6.

1. 2. 3. 4.

1.

2. 3. 4.

5.

6.

 Now you can deploy your database from the NuGet package to Octopus Deploy using SQL Release.

The easiest way to do this is using DLM Automation step templates. 

1. Copy the DLM Automation step templates to your Octopus Deploy library

Once you've copied the DLM Automation step templates to your Octopus Deploy library, they're available whenever you add a process step in an Octopus Deploy project:

Go to the   on the Octopus Deploy library website.DLM Automation step templatesIn this example, we'll use the "Redgate - Create Database Release" and "Redgate - Deploy from Database Release" step templates.Click on the "Redgate - Create Database Release" template.Click  :Copy to clipboard

You're now ready to paste the script from your clipboard into your Octopus Deploy library:

In Octopus Deploy, at the top of the page, click  .LibraryOn the  tab, click  .Step templates  ImportIn the Import window, paste the copied template into the empty field.Click  .ImportClick  .SaveRepeat steps 1 to 5 to copy the "Redgate - Deploy from Database Release" step template in the same way.

2. Create an Octopus Deploy project

In Octopus Deploy, click   and  .Projects AllClick  .Add projectIn the  field, enter  .Name  Widget DeploymentClick  .Save

You'll now add a series of deployment process steps to your Octopus Deploy project.

3. Add the "Download and extract database package" step

This step picks up the NuGet package of the database schema you're going to deploy.

Set up your NuGet package feed by doing one of the following:register your existing external NuGet package feed with Octopus. For more details, see  .Adding external package feedsconfigure your build server to push packages to the Octopus built-in repository. For more details, see  .Using the built-in repository

In the   project, on the   tab, click   and select  .Widget Deployment Process Add step Deploy a NuGet packageIn the  field, enter  .Step name  Download and extract database package  In the Machine roles field, enter db-server and press Enter.This must match the role you assigned to the Tentacle.In the NuGet feed field, select either the name of the external feed you registered when you set up your NuGet feed, or the Octopus Server (built-

 repository.in)In the NuGet package ID field, enter the name of the package without the version number. For example, if the package was called WidgetShopLa

, you'd only enter test.0.1.nupkg Widget.When the package is generated, NuGet package manager automatically adds a number. If we included it here, Octopus would only deploy the

6.

7.

8.

1. 2.

3.

4. 5. 6. 7.

8.

9.

1. 2. 3.

4.

5.

1. 2.

3.

4. 5. 6. 7.

8.

9.

package that matched that name and version number. By removing the number, we're telling Octopus to always look for the latest package with that name.In the Environments field, select  .ProductionIf you leave this blank, the step will be accessible to all environments. Click  .Save

4. Add the "Create database release" step

This step creates the database deployment resources, including the Update.sql script.

On the project   tab,Process  click Add step and  .Redgate - Create Database ReleaseIn the   field, enter   and press  .Machine roles db-server EnterThis must match the role you assigned to the Tentacle.In the   field, enterExport path  the path the database deployment resources will be exported to.This path will later be used in the "Deploy from Database Release" step. It must be accessible to all tentacles used in database deployment steps.In the  fieldDatabase package step  , select Download and extract database package.In the  fieldTarget SQL Server instance  , enter the fully qualified SQL Server instance for the database you're deploying to.In the Target database name field, enter the name of the database you're deploying to.In the Username (optional) and  fields, enter the SQL Server username and password Password (optional)    used to connect to the target database.If you leave these blank, Windows authentication will be used to connect to the target database.In the   field, select  .Environments ProductionIf you leave this blank, the step will be accessible to all environments. Click Save.

5.  Add the "Review database deployment resources" step

This step pauses deployment to let you review the database deployment resources, including the Changes.html report, before allowing deployment to go ahead.

On the project   tab, click   and select  .Process Add step Manual intervention requiredIn the   field, enter Step name Review database deployment resources.In the  field, copy and paste this text: Instructions

Please review the schema and static data changes, warnings and SQL change script in 'Changes.html'.

In the   field, select  .Environments ProductionIf you leave this blank, the step will be accessible to all environments. Click Save.

6. Add the "Deploy from database release" step

This step uses the database deployment resources to deploy the database changes.

On the project   tab, click   and select  .Process Add step Redgate - Deploy from Database ReleaseIn the   field, enter   and press  .Machine roles db-server EnterThis must match the role you assigned to the Tentacle.In the   field, enter the path the database deployment resources will be exported to.Export path

 This must match the export path you entered in .4. Add the 'Create database release' stepIn the   field, select Database package step Download and extract database package.In the Target SQL Server instance field, enter the fully qualified SQL Server instance for the database you're deploying to.In the   field, enter the name of the database you're deploying to.Target database nameIn the  and   fields, enter the SQL Server username and password used to connect to the target Username (optional)  Password (optional)  database.If you leave these blank, Windows authentication will be used to connect to the target database.In the   field, select  .Environments ProductionIf you leave this blank, the step will be accessible to all environments. Click  .Save

1. a. b. c.

2.

3. 4. 5.

6.

7.

You've finished setting up the project steps. The Process tab should look like this:

 

7. Create a release

Now all the steps are set up, you can run your deployment process to create a release:

Create a blank database called :WidgetProductionOpen SQL Server Management Studio (SSMS).Click New Query.Execute the following SQL query to create a database:

CREATE DATABASE WidgetProductionGOUSE WidgetProductionGO

In the project, on the tab, click .Widget Deployment Process Create releaseThis page lets you add an optional release note. Click  . SaveClick   (or if there's more than one environment, click   and select  ).Deploy to Production Deploy ProductionClick  .Deploy NowAs the deployment process runs, Octopus Deploy shows the task progress list. The deployment pauses so you can review the database deployment resources:

Click  to download the Change report.Changes.html Use the report to review the update script, warnings, and details of what'll be added, removed or modified if you go ahead with deployment. 

7. 8.

In Octopus Deploy, click  and, in  , enter a comment to say you've reviewed the database deployment resources.assign to me  Notes  If you're happy with the report, click  .ProceedWhen the deployment is complete, the Task progress page looks like this:

You've now completed the deployment of the database package.

What now?

Other DLM Automation step templates are available on the  :Octopus Deploy Library website

"Redgate - Deploy from Database" deploys the schema of a source database to a target database without a review step."Redgate - Deploy from Package" deploys a NuGet package containing a database schema to a target database without a review step.

For more information, see  .Octopus Deploy step templates reference