Database Maintenance Optimization - Brad McGehee[2]

8/4/2019 Database Maintenance Optimization - Brad McGehee[2]

1/37

Brad M McGeheeMicrosoft SQL Server MVP

Director of DBA Education

Red Gate Software

[email protected]
http://www.red-gate.com/http://www.red-gate.com/


2/37

You may be a part-time or full-time DBA

You may be a DBA Administrator or DBA Developer

You may have less than one years experience as aSQL Server DBA, and are still adding to your skill set

You may be an experienced DBA:

And, you know you are never too smart to learn something

new. Or, you dont mind being reminded of something you may

have forgotten.


3/37

What is Database Maintenance? Why is Database Maintenance Important?

How Do Database Maintenance Plans Fit In?

Key Database Maintenance Tasks Physical File Defragmentation

Database and Log File Management

tempdb Maintenance

msdb Maintenance

Index Maintenance

Statistics Maintenance

Data Corruption Detection Database and Log File Protection

Database Maintenance Monitoring

Database Maintenance Best Practices


4/37

Database maintenance includes a defined set oftasks that DBAs need to perform on a periodicbasis to ensure that their databases:

Perform Optimally Maintain High Availability

Another way to look at this is: if appropriatedatabase maintenance is not performed asneeded, then you can expect to see: Degraded performance

Unexpected downtime


5/37

While many of the maintenance tasks discussed today mightseem small, the accumulative effectof following eachrecommendation can be huge.

By following these best practices, SQL Serverperformanceand availabilitycan be boosted by 10%, 20%, or even more.

Increased Performance and Availability

Number of

optimal

maintenance

tasks

implemented


6/37

The Database Maintenance Plan Wizard is not an ideal tool forperforming database maintenance.

While it is designed with the novice DBA in mind, unfortunately:

It is harder to use than it appears to be

It is not very flexible and it is hard to modify after the fact

It is hard to troubleshoot if there are problems

It allows DBAs to create maintenance plans that can:

Significantly hurt the performance of SQL Server

Give a false sense of security in regard to availability

Ideally, you should avoid using the Database Maintenance Wizard.But if you cant, be sure to research each of your options and selectthe ones that best meet your needs.


7/37

Physical File Fragmentation

Database and Log File Management

tempdb Maintenance

msdb Maintenance

Index Maintenance Statistics Maintenance

Data Corruption Detection

Database and Log File Protection

Database Maintenance Monitoring

This list is not designed to be all encompassing, but it does include key tasks that allDBAs need to perform regularly.

The assumption is that your hardware, OS, SQL Server, and application are optimallyconfigured.


8/37

When the OS writes a file to disk, if contiguousclusters are not available, they are writtenelsewhere on the disk.

When a file is stored in a non-contiguousmanner, then the file is considered to bephysically fragmented.

Physical file fragmentation can contribute to anadditional load on your I/O subsystem andreduce I/O performance.


9/37

SQL Server does not know about file fragmentation,but is affected indirectly by it because the OS has todo more work to move data to and from SQL Serverto disk.

The amount of file fragmentations negative affecton SQL Servers performance depends on manydifferent factors. For example, random reads are lessaffected by fragmentation than sequential reads.

Since we know fragmentation can affect SQL ServerI/O performance, our goal should be to minimize itas much as possible, even though it might be hard toquantify.


10/37

SQL Server is designed to minimize physical filefragmentation, assuming the DBA is smart aboutthe way he or she manages physical files.

Ways to minimize physical file fragmentation: Ensure there is no physical file fragmentation before

creating new databases and log files.

Pre-size MDF and LDF files instead of letting

autogrowth automatically size files. As needed, defragment physical files, either manually

or with scheduled job.


11/37

In theory, MDF and LDF files manage themselves.

In reality, for optimal performance, DBAs must takefull responsibility of managing them. This includes:

Ensuring instant file initialization is turned on

Locating MDF and LDF files appropriately

Pre-sizing MDF and LDF files appropriately

Setting appropriate autogrowth settings Taking care when shrinking MDF and LDF files

Monitoring MDF and LDF file growth


12/37

When instant file initialization is enabled, it prevents MDF (notLDF) files from being zeroed out, speeding up their creation.

Speeds up CREATE DATABASE, ALTER DATABASE, RESTOREDATABASE, and autogrowth. Also speeds database recovery.

Requires SQL Server 2005/2008, and Windows Server 2003/2008

Instant file initialization is turned on if the SQL Server serviceaccount has been granted the SE_MANAGE_VOLUME_NAMEpermission. Members of the local Windows Administrator group

have this right and can grant it to other accounts by addingthem to the Perform Volume Maintenance Tasks security policy.


13/37

The OS, pagefile.sys, and SQL Server binariesshould be located on their own disk volume.

MDF files should be located on their own disk

volume. LDF files should be located on their own disk

volume.

Benefits: Reduces I/O contention and offers better I/O

performance

Reduces physical file fragmentation


14/37

The DBA needs to estimate the future potential size of the MDF file,and then create the MDF file to that size.

The DBA needs to estimate the future potential size of the LDF file,and then create the LDF file to that size. Harder to estimate. May

have to make educated guess, then resize later once you have datato work with.

As needed, MDF and LDF files should be modified manually.

Benefits:

Reduces physical file fragmentation Reduces unexpected autogrowth events

For log files, prevents LDF file from becoming fragmented into excessive multipleVirtual Log Files, which can hurt performance.


15/37

Autogrowth should not be used to manage filegrowth.

Autogrowth should only be used to coverunexpected file growth.

Autogrowth, by default, is set to grow MDB by 1MBat a time, and LDF is set to grow by 10% at a time.

Set autogrowth to grow by a fixed amount youchoose, and not a percentage amount.

Choose a fixed amount that wont result in a lot ofautogrowth, but is not so large that it will create alot of space that wont be used in the immediatefuture.


16/37

Dont use Auto Shrink (database option). Dont schedule database or file shrinking operations. If you must shrink a database:

Do so manually Rebuild the indexes after the shrink is complete Schedule during slow time of the day

Benefits of not automatically shrinking files: Eliminates grow and shrink syndrome Reduces physical file fragmentation Reduces resources used for these operations, allowing

more important tasks to use them


17/37

If very active, locate on its own disk volume.

Divide tempdb into multiple physical files, so that the number offiles is about 50% to 100% of the number of CPU cores your serverhas.

Each physical file should be the same size, and be set to the sameautogrowth amount.

Pre-size tempdb to typical size so autogrowth doesnt have tohappen. Default starting size is only 8MB.

Set Autogrowth to avoid growth spurts, use fixed amount. Dont shrink tempdb using DBCC SHRINKFILE, use ALTER

DATABASE and then restart server.


18/37

Over time, the msdb database can grow large,storing old data you probably dont need, like:

Backup and restore history (sp_delete_backuphistory)

SQL Server Agent Job history (sp_purge_jobhistory)

Maintenance Plan history (sp_maintplan_delete_log)

You should periodically clear out the older data,

possibly shrink the database, and then rebuild allof its indexes.


19/37

Index Fragmentation Hurts Performance Over time, as indexes are subjected to data modifications,

gaps in data on pages develop, and the logical ordering ofthe data no longer matches the physical ordering of the

data. This is a normal behaviour, but must be regularlyaddressed.

Heavily fragmented indexes can lead to poor queryperformance, especially if scans occur regularly.

Because of this, it is important that DBAs regularly detectand remove index fragmentation from their databases on aregular basis.


20/37

There are three ways to remove fragmentationfrom an index:

Reorganizeonline

Rebuildoffline Rebuildonline (Enterprise Edition Only)


21/37

This is an online task that doesnt block user activity.

Uses an in-place algorithm to reorganize indexes at leaf level.

Removes much fragmentation and empty space, but not all.

Only reduces fragmentation if necessary. Generally takes more time than Rebuild to complete.

Can be stopped and started without losing work.

Less resources are required to reorganize than Rebuild.

Less disk space is required to reorganize than Rebuild. Generally, produces less logging when in full recovery mode.

Index statistics are not updated (must perform this taskseparately).


22/37

Considered to be an off-line activity (unless you have EE).

Index is rebuilt from scratch, and old index is dropped.

Virtually all fragmentation and empty space is removed.

Everything is rebuilt, whether it needs it or not. Rebuild is generally faster than using Reorganize.

Index rebuild is atomic, cant be stopped and restarted.

More physical resources are required then Reorganize.

Additional disk space is required than Reorganize. Will cause log file to grow, as operation is logged.

Index statistics are updated with a FULLSCAN.


23/37

Only Reorganize or Rebuild indexes that need it, dont defragall indexes all the time.

If fragmentation is less 5% and 30%, consider Rebuild.

Use sys.dm_db_index_physical_stats to help you determineif an index should be rebuilt or not.

Reorganize or Rebuild jobs should be scheduled as a SQLServer Agent job using a custom T-SQL script you create tomeet your environments specific needs.


24/37

Indexing Needs Change Over Time Over time, data in databases, and the use of this data,

often changes.

This means that the current indexing scheme may need to

be changed over time.

For examples, indexes may need to be added, modified, orremoved for optimal query performance.

The easiest way to do this is to run the Database Engine

Tuning Advisor against a Profiler Trace on a periodic basis.


25/37

The best way is to identify missing indexes is to use Profiler tocapture a trace file, and then use the Database Engine TuningAdvisor (DTA) to analyze the trace to look for recommendations.

When capturing a Profiler Trace, use the Tuning template and

capture data over a representative time frame. If your trace files become large, you may need to create multiple

traces over the course of a day to get a representative sample ofactivity.

Run the DTA against the trace data, review recommendations,and then add appropriate indexes.


26/37

Most databases have one or more indexes that werecreated because they seemed that they might beuseful, but they have ended up not being used.

Because indexes need to be maintained when datachanges in a table, maintaining indexes that are notused is a waste of resources.

Periodically, identify unused indexes and removethem.

Use the sys.dm_db_index_usage_stats DMV to helpyou identify unused indexes.

Keep in mind that the data in this DMV is cleared outeach time SQL Server is restarted.


27/37

SQL Server maintains statistics on indexes (andsome non-indexed columns) which are used bythe query optimizer to help produce an optimal

query plan.

If these statistics are out of date, or notrepresentative, then the query optimizer may

produce a poorly performing query plan. As DBAs, we need to ensure that appropriate

statistics are created and stay updated.


28/37

Ensure that Auto Create Statistics and Auto Update Statisticsare set to true for all databases.

If Rebuilding indexes, statistics are automatically updated withFULLSCAN, so you dont need to update statistics separately.

If Reorganizing indexes, statistics are not automaticallyupdated, so you should update statistics manually afterwards.

Use sp_updatestats to update statistics. This command runsUPDATE STATISTICS only on those statistics that require

updating, helping to conserve server resources.


29/37

There are two major causes of data corruption:

Physical: Data has been altered in negative way, oftencaused by hardware or hardware drivers.

Logical: A data relationship has been broken

There are two ways to help identify these typesof data corruption:

Turn on the checksum for your databases. Run DBCC CHECKDB as often as you make full

backups.


30/37

This database setting calculates a checksum overthe contents of a page and stores the value in thepage header when the page is written to disk.

When the page is read from disk, the checksum is

recomputed and compared to the checksum valuestored in the page header.

If the values do not match, error message 824 isreported to both the SQL Server error log and the

Windows event log. When you see this error, you need to take action

now. Offers better protection than torn page detection.


31/37

DBCC CHECKDB checks the logical and physical integrity of all objects ina database.

Ideally, the command should be run before a full database backup is madeto identify problems before the backup occurs.

If a problem is detected, you want to identify, and correct it, as soon aspossible.

DBCC CHECKDB has some very limited fixing ability, but it should notbe counted upon, and only used by experts.

Running DBCC CHECKDB is resource-intensive and should be run during

slow times on the server. If you dont have a large enough window to run CHECKDB, restore

database to another server and run CHECKDB there.


32/37

Production databases should use the Full Recovery model.

Create a job to perform full backups daily on all system and userproduction databases, plus log backups hourly (or similar).

Always backup using RESTORE WITH VERIFYONLY to help verify

backup integrity.

Randomly restore backups to verify that you can restore yourdatabases.

Create, and enforce, an appropriate data retention policy.

Store backups securely and off-site. If you have a limited backup window, or have limited disk space,

use backup compression. Can be a big time saver.


33/37

Monitor MDF and LDF file growth

Monitor Free SpaceShould have 15% or more freespace

Monitor SQL Server and OS Logs Monitor Jobs

Monitor Alerts

Monitor Performance

Key Point: Even though it is a boring task, you needto regularly monitor your SQL Server instances.


34/37

Manual checks

SQL Server Alerts

OS Event Log Alerts

Performance Monitor Alerts

Profiler

SQL Server Standard Reports

Performance Data Collector (2008) Write your own monitoring system

Use a third-party tool


35/37

Create maintenance tasks yourself, dont depend on theDatabase Maintenance Wizard.

As much as practical, keep maintenance plans the same fromSQL Server instance to instance.

Script jobs and reuse them when you can. Dont over-maintain your databases. Find the right balance.

Schedule database maintenance tasks during down times orduring the least busy time of the day.

Dont repeat the same maintenance tasks (e.g. Rebuild indexes,then Update Statistics immediately thereafter).

Schedule jobs so that they do not overlap one another.


36/37

Implementing optimal maintenance plans cangreatly affect a SQL Server instances: Availability Performance

You need a good understanding of SQL Serverinternals in order to properly maintain yourdatabases.

Evaluate all of your SQL Server instances to ensure

that all appropriate maintenance tasks are beingperformed, and are being performed optimally. Database maintenance is an on-going task that

never ends. Automate as much as possible.


37/37

Free E-Books: www.sqlservercentral.com/Books

Check these out: www.SQLServerCentral.com

www.Simple-Talk.com www.SQL-Server-Performance.Com

Contact me at:[email protected]

Blogs:www.simple-talk.com/community/blogs/brad_mcgehee/default.aspxhttp://www.sqlservercentral.com/blogs/aloha_dba/default.aspxhttp://twitter.com/bradmcgehee
http://www.sqlservercentral.com/Bookshttp://www.sqlservercentral.com/http://www.simple-talk.com/http://www.sql-server-performance.com/mailto:[email protected]://www.simple-talk.com/community/blogs/brad_mcgehee/default.aspxhttp://www.sqlservercentral.com/blogs/aloha_dba/default.aspxhttp://twitter.com/bradmcgeheehttp://twitter.com/bradmcgeheehttp://www.sqlservercentral.com/blogs/aloha_dba/default.aspxhttp://www.simple-talk.com/community/blogs/brad_mcgehee/default.aspxhttp://www.simple-talk.com/community/blogs/brad_mcgehee/default.aspxhttp://www.simple-talk.com/community/blogs/brad_mcgehee/default.aspxmailto:[email protected]://www.sql-server-performance.com/http://www.sql-server-performance.com/http://www.sql-server-performance.com/http://www.sql-server-performance.com/http://www.sql-server-performance.com/http://www.simple-talk.com/http://www.simple-talk.com/http://www.simple-talk.com/http://www.sqlservercentral.com/http://www.sqlservercentral.com/Books

Database Maintenance Optimization - Brad McGehee[2]

Documents

Transcript of Database Maintenance Optimization - Brad McGehee[2]