1

Microsoft SQL Server RecoveryOperations with Datrium DVX

TechnicalReport

385 Moffett Park Dr. Sunnyvale, CA 94089 844-478-8349 www.Datrium.com

2

Table of Contents

Introduction 3

Pre-Requisites 3

Datrium Overview 4

Datrium per-VM and Application-Based Protection Policies 5

Setup Your Microsoft SQL Server Protection Group 5

Summary 18

Appendix A 19Installation of the Datrium VSS Agent

Appendix B 25Automated Installation of the Datrium VSS Agent using PowerShell

Contacting Datrium 27

Online Support 27

Datrium Solutions 27

Document Revisions 27

About the Author 27

3

Virtualizing Microsoft SQL Server has been a long-time discussion and debate between Virtualization Administrators and Database Administrators. For many good reasons, SQL Server is likely to have been one of the last applications within the environment to be a good candidate to virtualize. There have been many enhancements within the hypervisor environments (VMware vSphere and SQL Server on Linux) which now make virtualizing SQL Server a viable solution. Performance is no longer a concern for DBAs and end-user experiences are able to essentially match that of its physical counterpart. In fact, at Datrium we have conducted extensive testing and continue to perform additional testing of SQL Server on DVX. Many Datrium® customers have been able to instantly see a dramatic impact of their SQL environments once migrated over to DVX and off of their legacy storage devices.

Not all environments and organizations have a black-belt ‘certified’ DBA on staff. Even if you do have an expert DBA who performs native SQL backup procedures – Virtualization Administrators are looking to augment this process with zero impact to the running databases. This has become a growing challenge – how can I backup frequently and meet application SLA’s without any impact to the end users that are using the environment? Traditionally, snapshot-based VM backup technologies can adversely affect performance of highly transactional services, like Microsoft SQL Server. These effects limit an administrator’s ability to protect workloads frequently throughout the day. This Datrium Tech Note will focus on the capabilities which Datrium DVX can provide to environments which utilize Datrium’s robust per-VM or application-based Protection Group policies for VSS consistent Microsoft SQL Server recovery options.

This Tech Note is intended as an instructional document which will walk a Virtualization Administrator or DBA through the process of setting up Datrium Protection Group policies for Microsoft SQL Server. Also, you will be provided with all of the necessary information required to use Datrium’s restore capabilities alongside Kroll Ontrack PowerControls for SQL to perform SQL Server database / table recoveries. Whether you are running Datrium today or an alternative solution where you’re lacking the ability to frequently protect and meet the SLAs of your organization, this document is written to guide you through the process.

The examples, illustrations and practices described within this document are based upon Datrium DVX 4.0 or higher, Microsoft SQL Server 2012R2 and higher as well as Kroll PowerControls for SQL version 9.0.2.

After reading this Tech Note you will have all of the necessary tools to successfully utilize Datrium DVX per-VM or application-based Protection Group policies to create VSS consistent backups of Microsoft SQL Server and perform recovery using Kroll PowerControls for SQL.

Introduction

Pre-Requisites

4

For further information on the solutions discussed in this Tech Note as well as additional documentation please visit:

Datrium @ https://datrium.com

Microsoft SQL Server @ https://www.microsoft.com/en-us/sql-server

Kroll OnTrack @ https://ontrack.com

The Datrium DVX Platform is an Enterprise-focused converged infrastructure for compute, primary and secondary storage which provides a simplified approach to scalability and overall operation. These fundamental capabilities alongside the high-performance and integrated cloud data management make Datrium the ideal solution for the Enterprise.

The Datrium DVX Platform is comprised of x86-based compute resources (CPU and Memory), the Datrium Hyperdriver software and local flash (SSD Flash or NVMe) on the virtualization host called a Compute Node. With flash as close to the application as possible, applications are given the high performance, low latency I/O processing they require. From an end-user perspective the application response times are significantly better as well – this is due to the fact that all writes and a huge majority of reads occur from the local flash drives. With DVX, Compute Nodes are stateless. To improve performance, the DVX Platform compiles random writes from a VM and creates larger sequential writes into a scale-out Data Pool (Data Node). Writes are synchronously replicated to mirrored NVRAM on the Data Node(s) for low application latency (< 1 ms) and data durability. The DVX Platform ensures that failure and maintenance events on the Compute Nodes do not affect the data stored in the Data Pool and do not require data to be rebuilt or rebalanced.

Within the DVX Platform users may scale both speed / performance independent of the environmental capacity – this is referred to as Split Provisioning. To cluster computing you may add additional Compute Nodes to the cluster (up to 128 per cluster) or you may improve I/O performance by adding additional flash or larger capacity flash drives. When capacity expansion is required simply add additional Data Nodes to the scale-out Data Pool. This scale-out architecture of the Data Nodes also increases aggregate write bandwidth, increasing overall networking throughput as the environment grows.

You may scale Compute Nodes up to 128 and 10 Data Nodes within an individual DVX cluster. Datrium has conducted independent benchmarks with IOMark to push the upper limits of DVX. Read about this testing here.

For more information on the DVX Platform Architecture

see “DVX Architecture | Datrium Technical Report”

Datrium Overview

5

A core capability within the DVX Platform is the ability for Administrators to define Protection Groups (i.e., backup policies) which defines when the backup and protection of the VMs within the Protection Group occurs. This section is intended to provide you with a quick introduction to the Datrium DVX built-in backup capability and its protection policies.

Datrium DVX offers a built-in backup capability which stores VM-level snapshots in an immutable state within the Data Node. These Datrium-based and initiated snapshots are stored within a separate area of the filesystem known as the Snapstore. Flexible backup policies are created by dynamically binding sets of VMs based on supplied grouping criteria expressed via name pattern matching rules to enable data protection at scale. Different policies are applied to different sets of VMs providing a mechanism to tailor RPO to specific application needs. Snapshots are either crash-consistent or, when configured, application level consistent using the Datrium VSS Provider which in leverages the Microsoft Volume Shadow Copy Services (VSS) framework.

Datrium-based snapshots are fast and efficient and do not carry any type of performance impact to the running VM, unlike traditional hypervisor-based snapshots. Each VM-level snapshot is a logical and self-sufficient copy of a set of VMs (and other artifacts) and serves as a full backup. Taking a snapshot is a fast metadata manipulation operation that involves no physical data copy. The file system maintains no complex snapshot delta chains that need to be replayed, degrade performance or lead to large data losses if corrupted. Restore points (aka snapshots) may also be deleted in any order as there is no dependency upon each. The system accommodates tens of thousands of snapshots per-VM without any performance impact with RPO measured in minutes and RTO in seconds reducing backup windows. Low RPO combined with large numbers of snapshots makes DVX very effective for virus and ransomware protection. A built-in Search Catalog facilitates backup restore via a simple UI and scalable automation. As well as another key benefit, Datrium’s Global Deduplication engine - this means that only unique data consumes capacity - no duplicates!

From a restore perspective users have many different options. For the focus of this paper we will highlight recovering individual VM hard disk(s) which contain the SQL Server .MDF and .LDF database files.

To begin setting up Datrium Protection Groups, you may access the user interface via the vSphere Web Client or the standalone UI. Upon installation of Datrium users are asked to provide their vCenter IP or FQDN along with an administrator user account and password. This will in turn install the HTML5 based plug-in to the vSphere Web Client [Figure 1].

Datrium per-VM and Application-Based Protection Policies

Setup Your Microsoft SQL Server Protection Group

6

Figure 1: Access the HTML5 Datrium DVX from within the vSphere Web Client

To access the standalone UI, you may navigate to the IP or FQDN of the Data Node cluster management IP address. This is useful to access and download the VSS agent to the target guest VM as shown below [Figure 2].

Figure 2: Download the Datrium VSS Agent Installer directly from the DVX Data Node

Once you’re accessing the Datrium platform from one of the methods described above navigate along the top to ‘Protection.’ It is within here that the creation of policies as well as management of replication sites and Protection Group definition occurs [Figure 3].

7

Figure 3: Manage Datrium Protection groups and policies via the Datrium Plugin from within the vSphere Web Client

For this Tech Note we will create a Protection Group with our SQL Server 2016 Always On High Availability servers contained within. In my Always On cluster I have enabled synchronous replication with automatic failover; this ensures that regardless of which node in the cluster is active, my data is always protected. A key benefit is Datrium’s Global Deduplication engine – this means that only unique data consumes capacity – no duplicates!

To enter the Create Protection Group click ‘Create.’ [Figure 4]

Figure 4: Populating Protection Groups is achieved via dynamic VM name pattern matching or via static entry.

8

Within the Protection Group creation wizard administrators may choose from two (2) ways to add VMs to the Protection Group – Note: You can backup individual VM or Datastore files and folders based on individual selection or via dynamic naming pattern search – for the purpose of this Tech Note we will focus on VMs and their resulting SQL install.

1. VM name pattern – VM name pattern provides the user with the ability to dynamically populate the policy with VMs based on their VM naming conventions. All environments have some type of naming convention; this ensures that as VMs (SQL Server specifically) come online they’re automatically included within the policy.

2. Select individual VMs – This method statically assigns VMs to the policy and does not dynami-cally populate after the job is created. This method allows the usage of VSS.

Figure 5: Utilize the Datrium VSS Provider when protecting Microsoft SQL Server to achieve application consistency.

The checkbox next to the VM must be ticked in order for VSS consistency to occur. If the Datrium VSS provider is not present, you will receive a warning prior to moving forward within the wizard. If the ‘Use VSS’ option is enabled but the VSS provider is either unavailable or not installed, DVX will still execute the snapshot but it will be crash consistent and a warning message will be displayed within the UI for the administrator. [Figure 5, above]

Note: “Appendix A” includes step-by-step instructions on how to install the Datrium DVX VSS Agent.

9

With the SQL Servers (or other servers) selected, the schedule and associated retention policy needs to be defined. In the case of database workloads this will likely involve application owners; IT should be meeting with application owners regularly. The main purpose is to bridge the gap between the business and IT so that the organization is able to benefit from additional functionality of the Datrium solution! With Datrium DVX Protection Groups, users can define multiple schedules and multiple replica locations within a given policy. This flexible policy driven approach permits organizations to have multiple retentions and locations, making things super easy to get configured and ensuring applications are protected and backup data is in the correct location. The illustration below [Figure 6] highlights the frequency in which applications will be protected, the required retention as well as whether or not to replicate the snapshots to a secondary Datrium DVX or even an AWS-based Cloud DVX.

Figure 6: Administrators may create protection groups with multiple schedules, separate retention policies as well as replica targets.

Within the Datrium plug-in Administrators can drill into a specific application Protection Group to perform recovery as well as examine the application group details, membership, schedule and locations in which the application group is being protected to. As we examine the illustration below [Figure 7] it becomes apparent that our SQL (VSS) application Protection Group is being protected using VSS several times per day and a set of these restore points are being replicated to a secondary DVX environment. Within this application Protection Group administrators are able to drill into specific restore points and perform recovery operations.

10

Figure 7: The DVX Protection Group summary page provides great detail about the overall status, schedule, replica status as well as all snapshots, on-premises and remote.

Datrium DVX supports many different recovery operations including:

1. Clone VM from snapshot

2. Restore VM from snapshot

3. Replicate VM from snapshot

4. VM files in snapshot

Datrium DVX backup and restore capabilities satisfy the requirements of many different restore use cases including test / development, backup, disaster recovery, cloud archive, ransomware protection, etc., as well as application recovery requirements. As stated previously, this paper focuses on recovering SQL Server database(s) or individual tables; for this we’ll recover the individual VM hard disk(s) which contain the SQL .MDF and .LDF files [Figure 8]. However, if an entire SQL Server node failure occurred, the recovery is as simple as restarting the virtual machine. All restore points are stored within the DVX Snapstore, a separate and immutable area of the Datrium Data Node. This architecture means that there is no mechanical extraction or replacement of data blocks when recovery needs to occur, making the recovery time objective and operation virtually instantaneous. Within the distributed log structured file system all that needs to occur is the updating of where blocks of data are located, making the recovery time objective of even the largest SQL Server(s), near instantaneous.

11

Figure 8: Recovery of a VM virtual disk (.VMDK) from within a Datrium-based VM snapshot

To recover the VM virtual disks within a snapshot, first an administrator would select the appropriate point-in-time in which to recover from. This option restores the entire .VMDKs which house the .MDF (database) as well as the .LDF (log files). It is important to understand that when creating backup policy users should explicitly understand the recovery point objectives for the desired applications – i.e., how frequently a backup occurs along with the tolerance for data loss. For example, if a recovery point objective of 12 hours is defined within the policy and a failure occurs at 11:59 PM--but the last backup taken occurred at 11:59 AM--there would be an 11 hour and 59-minute data loss. From an SQL Server perspective, recovery can include the entire SQL Server, in the case of Always ON, an entire group of SQL Servers as well as individual databases or tables within the database.

Figure 9: Cloning the .VMDK from the Datrium-based snapshot for recovery of the SQL Server .MDF and .LDF database files.

12

The illustration above [Figure 9] displays the virtual hard disks within the restore point which was executed (May 29, 2018 at 4:00 pm). It is very easy to see that virtual disk device SCSI 0:1 holds the SQL data in which we are interested in recovering. For recovery and application extraction to occur it is necessary to clone and attach the VMDK(s) to a server within the environment. In this case again it would be a Windows-based server where Kroll PowerControls for SQL is installed. The URL included provides the User Guide from Kroll to perform the installation of PowerControls for SQL – documenting this process is outside the scope of this paper and assumes that Kroll PowerControls for SQL is already installed as well as licensed for SQL Server recovery operations.

It is worth noting that Kroll PowerControls for SQL is not an absolute requirement to perform recovery. Administrators may use native SQL tools available from within SQL Server Management Studio for instance or other 3rd party tools as well to perform recovery. PowerControls does offer a simplified solution for environments where SQL Server database expertise is not present.

Once the corresponding VMDK is selected within the wizard, administrators are asked for a location to place the VMDK. Again, this is a CLONE of the existing virtual disk file(s) – nothing will change on the production running SQL Server [Figure 10]. It is a best practice to name the cloned virtual disk file accordingly, Datrium recommends appending ‘-clone‘ at the end of the file name to ensures that the recovered VMDK is tracked appropriately through vCenter.

Figure 10: Datrium recommends appending ‘-clone‘ at the end of the file name to ensure that the recovered VMDK is tracked appropriately through vCenter

When executed, the clone operation is instantaneous. As stated previously, there is no mechanical extraction or replacement of data – simply the instantiation of blocks occurs within the Datrium DVX distributed log structured file system. You may now browse the DVX datastore to the location of which the clone occurred to view your cloned virtual hard disk files. If there are multiple virtual disk files which need to be cloned, you would go through the process for each. If this is the case, again, you would want to recover each virtual disk file to the same location.

13

Figure 11: Browse the Datrium DVX Datastore to locate the .VMDK which was cloned from the Datrium-based snapshot

You may now add this virtual hard disk to the VM in which you have Kroll PowerControls installed on [Figure 11] This is done through editing the settings of the VM where Kroll PowerControls is installed and choosing ‘Add New Device’ and then selecting ‘Existing Hard Disk.’ This is illustrated below in [Figure12].

Figure 12: Add the cloned .VMDK to a VM with Kroll PowerControls for SQL present.

14

This particular hard disk was added to a Windows Server 2016 VM, again, Kroll PowerControls for SQL is present and the Windows Server 2016 VM is NOT a SQL Server. On this VM, within Windows Disk Management it will be necessary to Online the disk as well as run a few DiskPart commands to remove ReadOnly, ShadowCopy and Hidden Attributes [Figure 13].

Figure 13: Select the volume which was added to the VM and clear the necessary attributes (pre DVX 4.0.5).

The commands below highlight the exact steps required within DiskPart. It is worth noting that with the upcoming Datrium DVX 4.0.5 and Datrium VSS provider v1.2 these DiskPart commands will no longer be required.

diskpartlist volumeselect volume 3attribute volumeattribute volume clear readonly hidden shadowcopyattribute volume

Example: Diskpart commands required to clear readonly, hidden and shadowcopy attributes.

Within this Tech Note you have successfully learned how to create a Datrium application Protection Group with VSS consistent snapshots as well as how to recover (via clone) individual virtual machine hard disk files. You may now use Windows Explorer to browse the SQL Server DATA directory [Figure 14].

15

Figure 14: Windows Explorer to view the Database files from the cloned .VMDK

To perform recovery, launch Kroll PowerControls for SQL and run as an Administrator. To perform the launch as an Administrator, right click and choose ‘Run as Administrator.’ Upon launch Administrators are prompted to provide the Source as well as the Target [Figure 15]. The source is the virtual hard disk upon which we performed the Datrium clone operation and mounted to our Windows-based VM. The target is the SQL Server 2016 Always On environment in which we will perform recovery against. For demonstration purposes the dbo.customer table has been dropped from the target environment.

Figure 15: Kroll PowerControls for SQL maybe used to perform recovery of SQL Server databases or tables from an SQL Server database.

To recover the dbo.customer table back to our SQL Server 2016 Always On environment first ‘Copy’ the table from the Source and then select ‘Paste.’[Figure 16]. As insinuated, this will copy the dbo.customer table from the backup and paste into the production environment.

16

Figure 16: Recovery of an SQL Server database(s) or table(s) is as simple as a copy and paste action.

Once the paste operation is initiated Kroll PowerControls for SQL will begin to copy over the database information from the source over to the target environment [Figure 17]. Depending on the size of the table or database this may take a few minutes to complete. Upon completion the display will update and will show the completion status. In the example below 6.6 million rows were recovered in 13 minutes.

Figure 17: Kroll PowerControls for SQL will recover the database(s) or table(s) to the target SQL Server instance.

Within SQL Server Management Studio you may verify that not only have your databases or tables been recovered but they have also fully synchronized with nodes within the Always On clustered environment [Figure 18].

17

Figure 18: Utilize SQL Server Management Studio to validate the recovery of the required database(s) or table(s).

To clean up after the restore operations have completed you may reverse the process which was conducted to recover. Remove the virtual hard disk file from the VM in vCenter and choose whether or not the VMDK should be deleted from the datastore [Figure 19]. This will in no way affect the restore point from which the .VMDK was cloned.

Figure 19: When restore operations are completed, remove the cloned .VMDK from within vCenter. Optionally, you may delete the .VMDK from the datastore.

18

Organizations are continuing to virtualize more and more mission critical applications including Microsoft SQL Server. With Datrium, Virtualization Administrators are able to provide the necessary Tier-1 performance and resilience which other platforms struggle to provide along with the ability to ensure that production databases are protected and remain available.

Summary

19

Appendix AInstallation of the Datrium VSS Agent

20

The Datrium VSS Agent is required in order to conduct SQL Server consistent backups. To download the Datrium VSS Agent navigate to the floating IP Address or FQDN of the Data Node, along the bottom of the interface there is a link, choose download [Figure A1].

Figure A1: Navigate to the floating IP Address or FQDN of the Data Node to download the Datrium VSS Agent.

The Datrium VSS Agent will be installed within the guest operating system of the SQL Server in which you would like to conduct Datrium initiated VSS backups from. This VSS Agent is delivered as an MSI. The installation is very straight forward; you will however need to know the admin password for the Datrium DVX [Figure A2].

Figure A2: Navigate to the downloaded .MSI for the Datrium VSS Provider

Browse to the location in which the download was saved and launch Datrium-VSS-Provider*-*.msi

21

Figure A3: Welcome to the Datrium VSS Agent Setup Wizard

Choose Next to select the features you would like to install [Figure A4]. By default, all components will be installed as the recommended approach.

Figure A4: By default, all components are installed as the recommended approach for all users to take.

In the User Authentication window enter the DVX system admin password. This will be used to authenticate with the DVX system to enable VSS snapshots [Figure A5].

22

Figure A5: Enter the Datrium DVX IP or FQDN and admin account password.

By default, the DVX VSS agent uses port 8888 – if there are firewalls in line between the guest and the Datrium DVX you will need to open port TCP port 8888 on the firewall device to allow communication. You may specify an alternate port for the Datrium VSS Agent to utilize as well [Figure A6].

Figure A6: TCP Port 888 is used by default however you may manually enter an alternate TCP port for Datrium VSS Agent to utilize.

23

Lastly, choose install to initiate the installation of the Datrium VSS Agent [Figure A7].

Figure A7: To install the Datrium VSS Agent choose install.

Upon completion, choose Finish [Figure A8].

Figure A8: Choose finish to wrap-up the Datrium VSS Agent installation.

24

Automated Installation of the Datrium VSS Agent using PowerShell

Appendix B

25

Appendix A detailed the installation of the Datrium VSS Agent, a component which is required to conduct SQL Server consistent backups. A. Appendix B will illustrate how to automate the deployment using PowerShell which can be utilized within your software deployment tool(s) of choice. [Figure B1]

Figure B1: Utilize PowerShell to automate the deployment of the Datrium VSS Agent

This script is provided as is with no warranty and each environment will vary and may likely require specific environmental changes. The below PowerShell will install the SQL VSS package on all MS SQL 2008 and newer servers in your environment. You will need VMware PowerCLI installed for this to work. Additionally, you will need to be able to hit all SQL servers and the Management IP of the Datrium Data Node from the system you run this script on. It is assumed that both local and remote hosts are running PowerShell 3.0 or newer.

To obtain DAVSS.ps1 please visit the Datrium GitHub Repository at -

https://github.com/datrium/PowerShell/tree/master/Datrium%20PowerShell%20VSS%20Agent%20

26

Datrium values your feedback on the products and services provided as well as this Reference Framework, thus contacting Datrium is easy.

Online SupportTo receive technical support or to view the Knowledge Base 24x7, you may visit the Datrium Support site at https://support.datrium.com. You may also email Support@Datrium.com.

Datrium SolutionsTo provide comments or feedback regarding this document or any other documentation available you may email Solutions@Datrium.com. Also, please visit the Datrium Resources page for additional documentation, https://datrium.com/resources.

Document RevisionsDate DescriptionJuly 2018 Original publication

About the Author

Clint Wyckoff is a Technology Evangelist and Senior Solutions Engineer at Datrium. He is an avid technologist and virtualization fanatic with more than a decade of enterprise data center architecture experience. Clint is an energetic and engaging speaker and places a large emphasis on solving the real-world challenges IT professionals face. Additionally, he is a 2x Microsoft Most Valuable Professional (MVP) for Cloud and Datacenter Management, VMware vExpert for 2015, 2016 & 2017 and 2018 as well as Co-Founder and Leader of vBrisket. Clint is also an AWS Certified Solutions Architect, Veeam Certified Engineer (VCME) and Microsoft Certified Professional (MCP).

Contacting Datrium