Spotlighton SQLServer 8.0Reporting and Trending Guide 2011 Quest Software, Inc.ALL RIGHTS RESERVED.This guide contains proprietary information protected by copyright. The software described inthis guide is furnished under a software license or nondisclosure agreement. This software may beused or copied only in accordance with the terms of the applicable agreement. No part of thisguide may be reproduced or transmitted in any form or by any means, electronic or mechanical,including photocopying and recording for any purpose other than the purchasers personal usewithout the written permission of Quest Software, Inc.The information in this document is provided in connection with Quest products. No license,express or implied, by estoppel or otherwise, to any intellectual property right is granted by thisdocument or in connection with the sale of Quest products. EXCEPT AS SET FORTH INQUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FORTHIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMSANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTSINCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OFMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT,INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES(INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESSINTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE ORINABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THEPOSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respectto the accuracy or completeness of the contents of this document and reserves the right to makechanges to specifications and product descriptions at any time without notice. Quest does notmake any commitment to update the information contained in this document.If you have any questions regarding your potential use of this material, contact:Quest Software World HeadquartersLEGAL Dept5 Polaris WayAliso Viejo, CA 92656email: legal@quest.comRefer to our Web site (www.quest.com) for regional and international office information.PatentsProtected by U.S. Patent #6,901,582. Additional patents pending.TrademarksQuest, Quest Software, the Quest Software logo and Spotlight are trademarks and registeredtrademarks of Quest Software, Inc in the United States of America and other countries. For acomplete list of Quest Softwares trademarks, please seehttp://www.quest.com/legal/trademark-information.aspx. Other trademarks and registeredtrademarks are property of their respective owners.Spotlight on SQL Server 8.0Reporting and Trending GuideMonday, 16 May 2011Table of ContentsAbout Spotlight Reporting and Trending 6Views 7Work with Views 7Reports 9Work with Reports 9Sample Reports 10Database Growth 10Deadlock List 10Server Alarms 10SQL Server Health 11Windows Server Health 11Create Reports 11Web Reports 13Work with Web Reports 13Specify the Location of Your Web Reports Server 13Deploy Web Reports 13Spotlight Statistics Repository 15Schema of the Spotlight Statistics Repository 15How the Spotlight Statistics Repository works 17Store Data in the Spotlight Statistics Repository 20Query the Spotlight Statistics Repository 21Query Custom Counters 26Query Alarm Data 27Table Definitions 29spotlight_datasources 29Spotlight on SQL Server Reporting and Trending GuideTable of Contents4spotlight_domains 29spotlight_monitored_objects 29spotlight_perfdata 30spotlight_stat_classes 31spotlight_stat_keys 31spotlight_stat_names 31spotlight_technologies 32spotlight_timestamps 32Stored Procedures 33Example stored procedures 36Specifications 36spotlight_get_datasources 36spotlight_get_domains 37spotlight_get_monitored_objects 37spotlight_get_table_columns 38spotlight_get_table_data_instance 39spotlight_get_table_keys 39spotlight_get_table_range 40spotlight_get_table_span 40spotlight_get_tables 41spotlight_get_technologies 41spotlight_sqlanalysis_results 42spotlight_ssr_version 42Maintenance Plan for Spotlight Statistics Repository and Playback Database43Database Configuration 43Fragmentation and Index Performance 43Database Backup 44Spotlight on SQL Server Reporting and Trending GuideTable of Contents5Appendix: Contact Quest 45Contact Quest Support 45Spotlight Support 45Contact Quest Software 46About Quest Software 46Index 47About Spotlight Reporting and TrendingIn Spotlight Reporting and Trending, you can access views and reports to assist with long-termtrend analysis and associated reporting. By configuring the Diagnostic Server to store statistics inthe Spotlight Statistics Repository, you can retrieve the data via views and reportsViews display data stored in the Spotlight Statistics Repository in the form of charts and tables.The charts and tables are grouped into logical categories that help you analyze the collecteddata. Views are different to drilldowns in that they show long-term data collected over time, notreal-time data.Reports also display data stored in the Spotlight Statistics Repository and are displayed in theSpotlight Report Viewer. These reports are designed to provide you with a starting point formeeting your reporting requirements and can be modified to suit your needs.To view the data collected by Spotlight in the Reporting and Trending portal, you mustconfigure the Spotlight Statistics Repository. See the Spotlight on SQL Server DeploymentGuide for more information.Note: It is important to implement a good maintenance routine to ensure the Spotlight StatisticsRepository is optimized and free of inconsistencies. See "Maintenance Plan for SpotlightStatistics Repository and Playback Database" (page 43) for more information.1ViewsWork with ViewsUse Views to analyze and investigate long-term SQL Server data for planning purposes, toidentify trends, or to determine potential performance issues. Views display data stored in theSpotlight Statistics Repository in the form of charts and tables. The charts and tables are groupedinto logical categories, called Views, that help you analyze the collected data. Views aredifferent to drilldowns in that they show long-term data collected over time, not real-time data.Note: You have some control over how the data is collected. See the online help for moreinformation.To open a view1. Select Reporting and Trending | Views from the Spotlight Browser.2. Select from the view categories.3. Select the view.4. Click Select Connections. Select the connection(s) to view.5. Click Refresh.Click To do this...SelectRepositorySelect a different Spotlight Statistics Repository.TimeRangeSelect the time frame for the view.FilterDataFilter the data in the view.Export Export the view. Grids are exported as .csv files. Charts are exported as .bmpfiles.2Spotlight on SQL Server Reporting and Trending GuideViews8Click To do this...You cannot change the name of the exported files so if you have previouslyexported a view and you don't want to overwrite the files, either rename theexported files (via Windows Explorer) or export the new files to a differentlocation.Configure|CustomCountersCreate the data for the Custom Counters view.Configure|DiagnosticServerConfigure the diagnostic server. See Configure the Spotlight StatisticsRepository for more information.For descriptions of the views available, see the online help.ReportsWork with ReportsIn Spotlight, you can configure the Diagnostic Server to store the statistics it collects in arepository for reporting purposes. You can then retrieve this data in the form of views andreports. You can use the example reports provided or build on them to create your own reportsby querying the data yourself using stored procedures. See "Stored Procedures" (page 33) formore information.Spotlight reports are displayed in an external viewer. When you launch the Spotlight ReportViewer it connects to the Spotlight Statistics Repository.To view a Spotlight report1. Select Reporting and Trending | Reports from the Spotlight Browser.2. Select a report.3. If SQL Server authentication is required, enter the user and password details.4. In the Spotlight Report Viewer:a. Select the start date.b. Select the end date.c. Select the Diagnostic Server.d. Select the connection.e. Click View Report.Tips:l Change reports by selecting a different report from the Report list at the top of theSpotlight Report Viewer. Complete the Diagnostic Server and Connection Name fieldsand then click View Report.l If you change the report criteria (time frame, Diagnostic Server or Connection Name),click View Report to see an updated version of the report.l Save reports in PDF, or as an .xls (Microsoft Excel) file using the Save icon on the Reporttoolbar.l Print, search, and refresh reports using the Report toolbar.l You can create your own reports for use in Spotlight.3Spotlight on SQL Server Reporting and Trending GuideReports10Sample ReportsSpotlight Reporting and Trending includes a set of example reports that demonstrate how storedprocedures are used to query the Spotlight Statistics Repository.Example reports are located on the Spotlight client in the following folder:l C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\Trending\ReportsExample reports compatible with SQLServer 2008 Reporting Services are located in thefollowing folder:l C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\Trending\Reports\WebReports 2008The input parameters are the same for all example reports. The Input parameters are:l Time startl Time endl Diagnostic Serverl Server name (monitored object)Database GrowthThe Database growth report contains the following charts:Data Growth Trend Shows the following statistics for each database on the specified server:l Data sizel Data allocatedLog Growth Trend Shows the following statistics for the transaction log for each database onthe specified server:l Log sizel Log allocatedDeadlock ListThe Deadlock List report shows all lock requests resulting in a deadlock on the specified serverfor a set time period.Server AlarmsThe Server Alarms report shows all alarms that have occurred on the specified server for a settime period.Spotlight on SQL Server Reporting and Trending GuideReports11SQL Server HealthThe Server Health report shows the following charts for the specified server for a set time period:l Number of Logged on Usersl Batches, Compiles and Recompilesl Buffer Cache Hit Ratiol Procedure Cache Hit Ratiol Page Life Expectancyl Server MemoryWindows Server HealthThe Windows Server Health report shows the following charts for the specified server for a settime period:l Total CPU Utilization chartl Context Switches/Secl Paging Ratel Network Bandwidth Utilizationl Disk Queue LengthSee "Deploy Reports" in this section for information on how to deploy the example reports.Create ReportsThe Spotlight Report Viewer displays reports using SQL Server 2005 or 2008 Report DefinitionLanguage files (.rdl). You can build on the example reports or create your own reports byquerying the data from the Spotlight Statistics Repository yourself.Example reports are located on the Spotlight client in the following folder:l C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\SoSSE\ReportsExample reports compatible with SQLServer 2008 Reporting Services are located in thefollowing folder:l C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\SoSSE\Reports\WebReports 2008To use your own report definition files Set the location of your .rdl files in Spotlight.a. Click Reporting and Trending in the Spotlight browser.Spotlight on SQL Server Reporting and Trending GuideReports12b. Right-click Reports.c. Select Settings.d. Navigate to the directory containing the rdl files you want to use.e. Click OK.or Copy your .rdl files into C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\Trending\Reports.Note: If you do not have write access to the Program Files folders listed above, you can createthe following folder and use it as an alternative:l Windows XP - My Documents\Spotlight Reportsl Vista - \Documents\Spotlight ReportsWeb ReportsWork with Web ReportsView web-based reports within Spotlight by configuring the Web Reports portal to point to yourweb-based reports server.Review the related topics for information on how to use the SQL Server Reporting Services webinterface to deploy the sample reports included with Spotlight Reporting and Trending:To open web reports in Spotlight Select Reporting and Trending | Web Reports from the Spotlight Browser.Specify the Location of Your Web Reports ServerYou can view web-based reports within Spotlight by configuring the Web Reports portal topoint to your web-based reports server.To specify the location of your Web reports server1. Select Configure | Diagnostic Server.2. Click Specify the location of your Web Reports server.3. Enter the URL of your web reports server.Deploy Web ReportsThis topic outlines how to use the SQL Server 2005 and 2008 Reporting Services web interfaceto deploy the example reports included with Spotlight Reporting and Trending. See "SampleReports" (page 10) for more information.To complete the following steps, you need report server administrator privileges.1. In the Reporting Services web interface, create a shared data source by clicking NewData Source.2. Complete the fields on the New Data Source page. You will need the followinginformation:4Spotlight on SQL Server Reporting and Trending GuideWeb Reports14l Data Source nameThe example reports included with Spotlight Reporting and Trending use a shareddata source called Spotlight Statistics Repository.l Connection typeSelect Microsoft SQL Server.l Connection stringEnter the connection string for the SQL Server instance that you installed theSpotlight Statistics Repository on.For example:Data Source=YourServerHostname;Initial Catalog=SpotlightStatistics Repositoryl Connection credentialsIncluding user name and password, if required.3. Upload the report definition (.rdl) files.To do this:a. In the Reporting Services web interface, click Upload File.b. Click Browse to navigate to the .rdl file. Spotlight report definition files arelocated on the Spotlight client in the following folder:C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\SoSSE\ReportsReport definition files compatible with SQLServer 2008 Reporting Services arelocated in:C:\Program Files\Quest Software\Spotlight on SQLServer\Plug-ins\SoSSE\Reports\Web Reports 2008c. Click OK.You can view web-based reports within Spotlight by configuring the Web Reports portal topoint to your web-based reports server.Spotlight Statistics RepositorySchema of the Spotlight Statistics RepositoryThe Spotlight Statistics Repository is based on a star schema because of the simplicity it offersfor data storage and retrieval. Star schemas are a simple schema consisting of a centralized facttable connected to multiple dimension tables.Fact tables hold factual data and the dimension tables hold descriptive data. One of the benefitsof a star schema is the speed of data retrieval. With star schemas, you can use relatively simpleSQL queries to return particular information about, in this case, a SQL Server instance. Later inthis chapter well show you how you can query the Spotlight Statistics Repository usingSpotlight API stored procedures and T-SQL.In the Spotlight Statistics Repository, the fact table (spotlight_perfdata) stores facts related toSQL Server performance, and the dimension tables hold definition data such as SQL Serverinstance information and attributes.5Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository16The schema of the Spotlight Statistics Repository is as follows:The main dimension tables in the Spotlight Statistics Repository are as follows:l Spotlight_datasourcesContains data used to categorize the collections that are performed by Spotlight. There isone row in this table for each category of data collected. Examples of categories are SQLServer, Windows, and Diagnostic Server.l Spotlight_domainsContains data about the Diagnostic Servers feeding the Spotlight Statistics Repository. Inthe case of multiple Diagnostic Servers feeding the Spotlight Statistics Repository, thereis one row per Diagnostic Server. The Domain_description column contains theDiagnostic Server host name.l Spotlight_monitored_objectsContains data about each of the SQL Server or Windows servers that a Diagnostic Serveris monitoring. There is one row in this table for each server being monitored by eachDiagnostic Server.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository17l Spotlight_techonologiesContains data that categorizes the collections that are performed. There is one row in thistable for each category of data collected.The remaining dimension tables contain data about the collections being stored in the SpotlightStatistics Repository. See "Table Definitions" (page 29) for more information on each of thetables in the Spotlight Statistics Repository.How the Spotlight Statistics Repository worksThe Diagnostic Server collects data about the SQL Server instance and then writes it to theSpotlight Statistics Repository. How often this data is written to the Spotlight StatisticsRepository depends on the scheduling options set for each data collection in Spotlight.The data is inserted into the relevant dimension tables and then the fact is written into the facttable (spotlight_perfdata).Lets use the example of a Diagnostic Server writing data about the sqlblockinglistcollection to the Spotlight Statistics Repository. In the following chart, the DiagnosticServer, DS123, is collecting data about SQL Server instance, SQLServer789, running onWindows host, Windows01.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository18The following chart shows how the data flows into the Spotlight Statistics Repository.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository19Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository20Store Data in the Spotlight Statistics RepositoryThere are a number of options that must be configured for Spotlight to store data in the SpotlightStatistics Repository.To configure the Spotlight Statistics Repository1. In Spotlight, select Configure | Diagnostic Server.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository212. Click Configure the Spotlight Statistics Repository.3. Select Enable use of the Spotlight Statistics Repository.4. Complete the remaining options.See Configure the Spotlight Statistics Repository in the Spotlight on SQL ServerDeployment Guide for more information.To store data in the Spotlight Statistics Repository1. Click Configure |Scheduling.2. From here, you can configure how often data is stored in the Spotlight StatisticsRepository for each collection. See Configure Scheduling in the online help for moreinformation.Query the Spotlight Statistics RepositoryBefore querying the Spotlight Statistics Repository we need to know how the repository isstructured and what data it contains. You can use the Spotlight API stored procedures to browsethe dimension tables in the repository. You can then use additional stored procedures or customT-SQL to retrieve data.ScenarioYour manager has asked you to provide statistics on database performance for the lastquarter. You decide to query the Spotlight Statistics Repository to return file I/O statisticsfor the databases your manager is interested in.To retrieve this data, you first browse the dimension tables to become familiar with the datain the Spotlight Statistics Repository. Once familiar, you write queries using the Spotlightstored procedures to get data from the fact table. You then refine the queries using custom T-SQL to retrieve more specific data from the Spotlight Statistics Repository.First up, were going to use some of the Spotlight API stored procedures to browse the dimensiontables and get the range of values available for use in our queries.The Spotlight stored procedures used below are explained in detail in this guide.l See "Stored Procedures" (page 33) for more information.l See "Specifications" (page 36) for more information.In SQL Server Management Studio, create a new query against the Spotlight StatisticsRepository. We want to know the names of the available data sources and the names anddomains of the monitored objects.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository22Lets start exploring the Spotlight Statistics Repository by looking at datasources. Datasourcesare broad categories of data.To get a list of data sources in the repository, run the following query:exec spotlight_get_datasourcesThis returns the following results:We can see here that we have data from Windows, SQL Server, the Diagnostic Server and alarms(availability).Now we want to know the name of the Diagnostic Server storing data in the Spotlight StatisticsRepository. We can determine this by running:exec spotlight_get_domainsTo get the following result:We can see here that there is one Diagnostic Server writing data to this repository. There can bemultiple Diagnostic Servers, each one resulting in a row.Finally, we want to know information about the servers being monitored.We can do this by running the following query:exec spotlight_get_monitored_objectsTo get the following results:A common question is what is the range of timestamps available for a monitored server? Thefollowing query returns the time ranges for all monitored servers:selectSpotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository23so.monitored_object_name,min(st.timecollected) as 'start',max(st.timecollected) as 'end'fromspotlight_timestamps stjoin spotlight_monitored_objects so on st.monitored_object_id = so.monitored_object_idgroup byso.monitored_object_nameNow that we have some information about the Diagnostic Server, monitored servers andcategories of collections in the Spotlight Statistics Repository, we want to know what statisticsare available to us.The spotlight_get_tables stored procedure returns a list of tables in the repository for a givendata source. Since we are looking for SQL Server data, from our first query above, we knowthat the datasource is sqlserver_spotlight, so we will use that as the parameter in thefollowing query:exec spotlight_get_tables 'sqlserver_spotlight'The following results are returned.Were going to look at virtual file stats as this will give us the file IO statistics we needfor our report.To find out what columns are available to us in the virtual file stats table use the followingquery. With this query we need to specify the datasource and table name.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository24exec spotlight_get_table_columns 'sqlserver_spotlight','virtualfilestats'We need to produce the report over a time range, but for curiositys sake, we would like to findout the time range of all stored data for a particular domain name, monitored object, and tablecombination.To do this, use the spotlight_get_table_span stored procedure and specify the domain name,monitored object, and table name.exec spotlight_get_table_span 'DS123:3843','Windows01_SQLServer789_sqlserver','virtualfilestats'Now we can put it all together and query the fact table. Well use the spotlight_get_table_rangestored procedure and specify the following:l start datel end datel domain namel monitored objectl table nameSpotlight_get_table_range returns data for the requested time range for a table, for a particularmonitored object and domain.This gives us the following query:exec spotlight_get_table_range '2009-03-26 11:15:16.153','2009-07-26 17:15:17.113', 'DS123:3843','Windows01_SQLServer789_sqlserver', 'virtualfilestats'Which returns all columns for the table virtualfilestats:Although there is a lot of useful data in the query results, its not exactly what we are lookingfor. We need to refine the data returned from the Spotlight Statistics Repository. Following onSpotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository25from the example above, if we want to retrieve only certain columns from the virtualfilestatstable, we can use custom T-SQL to return specific data.To return only the iorate, disk, readsrate, and writesrate columns from the virtualfilestatstable, we can use the following custom T-SQL:selectsp.timecollected,max(case when sn.statistic_name = 'dbname' thensp.raw_value end) as 'dbname',max(case when sn.statistic_name = 'dbfilename' thensp.raw_value end) as 'dbfilename',max(case when sn.statistic_name = 'disk' then sp.raw_value end) as 'disk',max(case when sn.statistic_name = 'iorate' thensp.raw_value end) as 'iorate',max(case when sn.statistic_name = 'readsrate' thensp.raw_value end) as 'readsrate',max(case when sn.statistic_name = 'writesrate' thensp.raw_value end) as 'writesrate'fromspotlight_perfdata spjoin spotlight_stat_classes sc on sp.statistic_class_id = sc.statistic_class_idjoin spotlight_stat_names sn on sp.statistic_name_id =sn.statistic_name_idjoin spotlight_monitored_objects so on sp.monitored_object_id = so.monitored_object_idwheresc.statistic_class_name = 'virtualfilestats'and so.monitored_object_name = 'Windows01_SQLServer789_sqlserver'and sp.timecollected between '2009-09-01' and'2009-09-30'group bysp.timecollected, sp.statistic_key_idorder bySpotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository26sp.timecollectedThis gives us the following results:Query Custom CountersScenarioYou have created a custom counter in Spotlight on SQL Server and now want to create areport that contains data from that counter.Querying custom counters is exactly the same as querying other data in the Spotlight StatisticsRepository. SQL Server custom counters are stored in the statistic class sqlcustomcounters andWindows custom counters are stored in the class windowscustomcounters. You can use the APIstored procedures to retrieve date ranges and column names with these like any other datacollection.The following example of T-SQL retrieves SQL Server custom counter values:selectsp.timecollected,max(case when sn.statistic_name = 'countername' then sp.raw_valueend) as 'countername',max(case when sn.statistic_name = 'countervalue' then sp.raw_value end) as 'countervalue'fromspotlight_perfdata spjoin spotlight_stat_classes sc on sp.statistic_class_id =sc.statistic_class_idSpotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository27join spotlight_stat_names sn on sp.statistic_name_id =sn.statistic_name_idjoin spotlight_monitored_objects so on sp.monitored_object_id =so.monitored_object_idwheresc.statistic_class_name = 'sqlcustomcounters'and so.monitored_object_name = 'Windows01_SQLServer789_sqlserver'and sp.timecollected between '2009-09-01' and '2009-09-30'group bysp.timecollected, sp.statistic_key_idorder bysp.timecollectedThe following shows the results returned for the custom counter Pages Allocated.Query Alarm DataAlarm data can be queried from the Spotlight Statistics Repository. Querying alarm data is thesame as querying other data stored in the repository. Alarm data is stored in a statistic classcalled alarms.The following is an example of a T-SQL statement that can be used to retrieve alarm data. Youneed to specify the monitored object name and start and end times as parameters.selectsp.timecollected,max(case when sn.statistic_name = 'severity' then sp.raw_valueend) as 'severity',max(case when sn.statistic_name = 'text' then sp.raw_value end)as 'text',max(case when sn.statistic_name = 'rule' then sp.raw_value end)as 'rule',Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository28max(case when sn.statistic_name = 'action' then sp.raw_value end)as 'action',max(case when sn.statistic_name = 'key' then sp.raw_valueend) as 'key',max(case when sn.statistic_name = 'raw text' then sp.raw_valueend) as 'raw text'fromspotlight_perfdata spjoin spotlight_stat_classes sc on sp.statistic_class_id =sc.statistic_class_idjoin spotlight_stat_names sn on sp.statistic_name_id =sn.statistic_name_idjoin spotlight_monitored_objects so on sp.monitored_object_id =so.monitored_object_idwheresc.statistic_class_name = 'alarms'and so.monitored_object_name = 'Windows01_SQLServer789_sqlserver'and sp.timecollected between '2009-09-08' and '2009-09-17'group bysp.timecollected, sp.statistic_key_idorder bysp.timecollectedThe above T-SQL statement returns the following results:Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository29Table Definitionsspotlight_datasourcesspotlight_datasources contains data used to categorize the collections that are performed. There isone row in this table for each category of data collected.Column Descriptiondatasource_id Generated ID.datasource_name Name of the data source.spotlight_domainsspotlight_domains contains data about Diagnostic Servers. If you have multiple DiagnosticServers feeding the Spotlight Statistics Repository, there is one row per Diagnostic Serverin this table.Column Descriptiondomain_id Generated ID.domain_descriptionThe name of the domain (analogous to Diagnostic Server host name meansthat this Spotlight Statistics Repository can hold data from multiplediagnostic servers).spotlight_monitored_objectsspotlight_monitored_objects contains data about the SQL Servers and Windows hosts that aDiagnostic Server is monitoring. There is one row in this table for each SQL Server or Windowshost being monitored by each Diagnostic Server.Column Descriptionmonitored_object_idGenerated ID.monitored_object_nameName of monitored object (Windows host name or SQL Server InstanceName etc).monitored_object_display_nameA user-friendly interpretation of the encoded name that the DiagnosticServer uses to uniquely identify a monitored object. This name is suitablefor display on reports or other user-visible outputs.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository30Column Descriptionparent_monitored_object_idThe monitored_object_id value of another monitored object upon whichthis one depends, or null if no such dependency exists. For example, a SQLServer databases parent will be the Windows host upon which it runs.technology_idThe technology of this monitored object.domain_id The domain (that is, Diagnostic Server) this monitored object belongs to.spotlight_perfdataspotlight_perdata is the fact table into which all of the data is stored. Each fact is effectivelymatrixed by the hierarchy of attributes that denote which Diagnostic Server, monitored object,collection and collection column the datum originally came from.Column Descriptionperfdata_id Generated ID.monitored_object_idThe monitored object that this fact came from.statistic_class_idThe collection that this fact came from.statistic_name_idThe column in the collection that this fact came from.statistic_key_idThe key column of the collection that this fact came from.timecollected The time this fact was collected from the monitored object.raw_value The raw value returned by the collection for that column.num_observationsInternal use only.sum_of_valuesInternal use only.max_value Internal use only.min_value Internal use only.sum_of_squaresInternal use only.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository31spotlight_stat_classesspotlight_stat_classes contains the names of collections being stored in the Spotlight StatisticsRepository. There is one row for each table/data source combination.Column Descriptionstatistic_class_id Generated ID.datasource_id The data source that this collection belongs to.statistic_class_name The name of the collection.spotlight_stat_keysspotlight_stat_keys contains key values for all of the collections stored in the Spotlight StatisticsRepository. Each key value/table/data source combination has a row in this table. The purpose ofthis table is to allow queries to get data from the fact table based on a key value without havingto scan values in the fact table itself.Column Descriptionstatistic_key_idGenerated ID.statistic_class_idThe statistic_class (Collection) this column belongs to.datasource_idThe data source of the collection that this column belongs to.statistic_name_idGenerated ID.statistic_key_nameThe name of the key column.spotlight_stat_namesspotlight_stat_names contains data about the columns belonging to collections. There is one rowin this column for each column/table/data source combination.Column Descriptionstatistic_name_idGenerated ID.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository32Column Descriptionstatistic_class_idThe statistic_class (collection) this column belongs to.datasource_idThe data source of the collection that this column belongs to.statistic_nameThe name of the column.datatype_guideThis column provides a broad guide to the type of data that can be expectedin facts for this named statistic. Values can be string, numeric or date.derived Whether this is a derived column or not.spotlight_technologiesspotlight_technologies contains data used to categorize the collections that are performed. Thereis one row in this table for each category of data collected.Column Descriptiontechnology_id Generated ID.technology_name The name of the technology.spotlight_timestampsspotlight_timestamps contains a summary of times for which data was collected, qualified bymonitored entity. It allows Spotlight to quickly determine the times for which historical datais available for individual monitored entities, without the overhead of scanning thepotentially large fact table spotlight_perfdata. There is one row in this table for eachcategory of data collected.Column Descriptiontimestamp_id Generated ID.timecollected The time this fact was collected from the monitored object.domain_id The domain that this fact belongs to.monitored_object_idGenerated ID.statistic_class_idThe collection that this fact came from.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository33Stored ProceduresThe following stored procedures return either metadata about the Spotlight Statistics Repositoryor data from within the Spotlight Statistics Repository.Stored procedure Descriptionspotlight_get_datasources Returns a list of all data sources in therepository (effectively the contents ofspotlight_datasources).spotlight_get_domains Returns a list of all the diagnostic serversputting data into this repository.spotlight_get_monitored_objects@domain_name, @technology_nameReturns a list of names of all the monitoredobjects that have data in this repository for agiven domain_name. If domain_name is NULLall monitored objects are returned. Also returnsthe monitored object display name, domaindescription, and technology name.spotlight_get_table_columns@datasource_name, @table_nameReturns a list of columns for a table belongingto a data source (effectively the contents ofspotlight_stat_names).spotlight_get_table_data_instance@date_time, @domain_name,@monitored_object_name, @table_nameReturns the rows and columns from a table witha timestamp that most closely matches @date_time.spotlight_get_table_keys @datasource_name, @table_nameReturns a list of keys for a table belonging to adata source (effectively the contents ofspotlight_stat_keys).spotlight_get_table_range @start_date,@end_date, @domain_name,@monitored_object_name, @table_nameReturns the rows and columns from a tablebetween @start_date and @end_date.spotlight_get_table_span @domain_name,@monitored_object_name,@table_nameReturns the time span of all stored data for aparticular domain_name, monitored object, andtable combination.spotlight_get_tables @datasource_name Returns a list of tables in the repository for agiven data source (effectively the contents ofspotlight_stat_classes).spotlight_get_technologies Returns a list of technologies in the repository(effectively the contents of spotlight_technologies).Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository34Stored procedure Descriptionspotlight_rt_get_aggregated_workload_analysis @start_date, @end_date,@domain_name, @monitoredobjectReturns data for the requested time range forthe Workload view.spotlight_rt_get_batch_data @start_date,@end_date, @domain_name,@monitoredobject_list, @table_name,@column_listReturns data for the requested time range for atable and associated columns for multiplemonitored objects in a domain.spotlight_rt_get_fileio_stats @start_date,@end_date, @domain_name,@monitoredobjectReturns data for the Database File IO Statisticsview.spotlight_rt_get_point_data @start_date,@end_date, @domain_name_list,@monitoredobject_list, @table_name,@column_nameReturns the same data as spotlight_rt_get_batch_data but for a particular point in time.spotlight_rt_get_point_data_top @start_date, @end_date, @domain_name_list,@monitoredobject_list, @table_name,@column_list,@sortdesc_column_list,@maxrowsReturns the top n rows of spotlight_rt_get_point_data based on the column sort orderpassed in.spotlight_rt_get_waitstats @start_date,@end_date, @domain_name,@monitoredobjectReturns data for the view Wait Statistics view.spotlight_sqlanalysis_results @domain_name, @monitored_object_name,@date_time_from, @date_time_to,@maxrowsReturns the worst performing SQL recorded inthe Spotlight Statistics Repository over thesupplied time span for a particular SQL Serverinstance.spotlight_ssr_version Returns the version number of the SpotlightStatistics Repository.The following stored procedures are used for the example reports:Stored procedure Descriptionspotlight_rep_alarms Returns a list of alarms for a chosen monitored objectfrom the repository.Used in the Server Alarms report. This is a furtherexample of how to query the Spotlight StatisticsRepository to return tabular information.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository35Stored procedure Descriptionspotlight_rep_cache_hit_ratios Returns hit rates for the buffer and procedure caches forall monitored objects in the repository.Used in the SQL Server Health report. This is an exampleof querying two columns from a table over time from theSpotlight Statistics Repository.spotlight_rep_cpu_utilization Returns the amount of CPU being used by the specifiedserver against the total amount being used by allWindows processes.Used in the Windows Server Health report. This is anexample of querying one column from a table over timefrom the Spotlight Statistics Repository.spotlight_rep_disk_growth Returns information on disk space usage over time.spotlight_rep_disk_statistics Returns disk statistics (percentage busy and disk queuelength) for the specified server.Used in the Windows Server Health report. This is anexample of querying multiple columns with multiplerows from a table over time from the Spotlight StatisticsRepository.spotlight_rep_growth_data_filesReturns the following statistics for the specified server:l Data sizel Data allocatedl Index sizel Table sizel Log sizel Log allocatedUsed in the Database Growth report. This is an exampleof querying multiple columns with multiple rows from atable over time from the repository.spotlight_rep_sql_batches_rate Returns the rate at which batches of SQL statements arebeing submitted to SQL Server for execution.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository36Example stored proceduresThe following stored procedures can be used as examples of how to retrieve data from theSpotlight Statistics Repository.l spotlight_rt_get_allsqlagentjobsl spotlight_rt_get_custom_counterl spotlight_rt_get_failedsqlagentjobsl spotlight_rt_get_mostcommonblockl spotlight_rt_get_sqlblockedmostl spotlight_rt_get_sqlblockingmostSpecificationsThe following section contains specifications for each of the stored procedures.spotlight_get_datasourcesReturns a list of all data sources in the repository (effectively the contents of spotlight_datasources).Syntaxspotlight_get_datasourcesInput parametersThere are no input parameters for this stored procedure.Returned valuesColumnName ColumnType Descriptiondatasource_id int ID of data sourcedatasource_name nvarchar(255) Name of data sourceSpotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository37spotlight_get_domainsReturns a list of all the diagnostic servers putting data into this repository.Syntaxspotlight_get_domainsInput parametersThere are no input parameters for this stored procedure.Returned valuesColumnName ColumnType Descriptiondomain_id int ID of domaindomain_description nvarchar(255) Descriptive name of domainspotlight_get_monitored_objectsReturns a list of names of all the monitored objects that have data in this repository for a givendomain_name. If domain_name is NULL all monitored objects are returned.Syntaxspotlight_get_monitored_objects @domain_name, @technology_nameInput parameters@domain_name@technology_nameSpotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository38Returned valuesColumnName ColumnType Descriptionmonitored_object_name nvarchar(255) Name of monitored objectmonitored_object_display_namenvarchar(255) A user-friendly interpretation of the encodedname that the Diagnostic Server uses touniquely identify a monitored object. Thisname is suitable for display on reports orother user-visible outputs.monitored_object_display_technology_namenvarchar(255) Combines the columns monitored_object_display_name and technology_name todisplay a user-friendly name that includesthe monitored object name and thetechnology.domain_description nvarchar(255) Descriptive name of domaintechnology_name nvarchar(255) Descriptive name of technologyspotlight_get_table_columnsReturns a list of columns for a table belonging to a data source (effectively the contents ofspotlight_stat_names).Syntaxspotlight_get_table_columns @datasource_name, @table_nameInput parameters@datasource_name@table_nameReturned valuesColumnName ColumnType Descriptionstatistic_name nvarchar(255) Name of table columnSpotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository39spotlight_get_table_data_instanceReturns the rows and columns from a table with a timestamp thatmostclosely matches@date_time.Syntaxspotlight_get_table_data_instance @date_time, @domain_name, @monitored_object_name,@table_nameInput parameters@date_time@domain_name@monitored_object_name@table_nameReturned valuesThe returned values are dependent on the columns implicit in the table represented by the@table_name parameter.spotlight_get_table_keysReturns a list of keys for a table belonging to a data source (effectively the contents of spotlight_stat_keys).Syntaxspotlight_get_table_keys @datasource_name, @table_nameInput parameters@datasource_name@table_nameReturned valuesColumnName ColumnType Descriptionstatistic_key_value nvarchar(255) Key value for procedureSpotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository40spotlight_get_table_rangeReturns the rows and columns from a table between @start_date and @end_date.Syntaxspotlight_get_table_range @start_date, @end_date, @domain_name, @monitored_object_name,@table_nameInput parameters@start_date@end_date@domain_name@monitored_object_name@table_nameReturned valuesThe returned values are dependent on the columns implicit in the table represented by the@table_name parameter.spotlight_get_table_spanReturns the time span of all stored data for a particular domain_name, monitored object, andtable combination.Syntaxspotlight_get_table_span @domain_name, @monitored_object_name, @table_nameInput parameters@domain_name@monitored_object_name@table_nameReturned valuesThe returned values are dependent on the columns implicit in the table represented by the@table_name parameter.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository41spotlight_get_tablesReturns a list of tables in the repository for a given data source (effectively the contents ofspotlight_stat_classes).Syntaxspotlight_get_tables @datasource_nameInput parameters@datasource_nameReturned valuesColumnName ColumnType Descriptionstatistic_class_name nvarchar(255) Name of tablespotlight_get_technologiesReturns a list of technologies in the repository (effectively the contents of spotlight_technologies).Syntaxspotlight_get_technologiesInput parametersThere are no input parameters for this stored procedure.Returned valuesColumnName ColumnType Descriptiontechnology_id int ID of technologytechnology_name nvarchar(255) Descriptive name of technologySpotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository42spotlight_sqlanalysis_resultsReturns the worst performing SQL recorded in the Spotlight Statistics Repository over thesupplied time span for a particular SQL Server instance.Syntaxspotlight_sqlanalysis_results @domain_name, @monitored_object_name, @date_time_from,@date_time_to, @maxrowsInput parameters@domain_name@monitored_object_name@date_time_from@date_time_to@maxrowsReturned valuesReturns information about the worst performing SQL recorded in the Spotlight StatisticsRepository.spotlight_ssr_versionReturns the version number of the Spotlight Statistics Repository.Syntaxspotlight_ssr_versionInput parametersThere are no input parameters for this stored procedure.Returned valuesReturns the version number.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository43Maintenance Plan for Spotlight StatisticsRepository and Playback DatabaseIt is recommended that a maintenance plan for both the Spotlight Statistics Repository and theplayback database is implemented.Both the Spotlight Statistics Repository and playback database support a large number ofmonitored servers and automatically maintain the age of data available. Due to the nature of theautomatic purging of old data, the data may become fragmented and this may decrease theefficiency of data retrieval operations over time.This topic provides guidance on how to configure and automate the maintenance of theSpotlight Statistics Repository and the playback database.Database ConfigurationThe Spotlight Statistics Repository and the playback database do not require the database to berun under the Full Recovery model. However, since Spotlight uses the model system database tocreate the repository, your Spotlight repository may inherit the Full Recovery setting. Unless youare required to use the Full Recovery model and are prepared to perform regularly scheduled logfile backups, we recommend you change the repository recovery model to Simple (in SQL Server2000, this option is called Truncate Log on Checkpoint). Doing so will help maintain aconsiderably smaller transaction log file.To change the repository database to Simple recovery, run the following command:SQL Server 2000EXEC [dbo].[sp_dboption] 'SpotlightStatisticsRepository','trunc.log on chkpt.','true'SQL Server 2005 and 2008ALTER DATABASE [SpotlightStatisticsRepository] SET RECOVERYSIMPLE;Fragmentation and Index PerformanceThe Spotlight Statistics Repository implements a star schema because of the data warehousing-style simplicity it offers for data storage and retrieval. The star schema implements a primary facttable that references a number of dimension tables. The fact table holds collections and alarmdata, and the dimension tables hold definition information on everything from instanceinformation to collection names. Spotlight maintains the age of the data in the repository, but thecontinuous INSERT and DELETE operations performed for data insertion and purging can leadto fragmentation. Fragmentation will eventually lead to increased CPU and I/O resourceconsumption.Spotlight on SQL Server Reporting and Trending GuideSpotlight Statistics Repository44Although the playback database uses a simpler schema, its data will also become fragmentedover time as new data is inserted and old data purged.To best address performance concerns, fragmentation should be minimized and index statisticsshould be kept up-to-date. Spotlight's scheduled maintenance feature does this automatically. Bydefault, the Diagnostic Server runs maintenance procedures daily at 3am for the playbackdatabase and Spotlight Statistics Repository. The maintenance schedule can be changed inSpotlight Options. See the online help for more information.Database BackupThe Spotlight Statistics Repository and playback database should be included in the list ofimportant databases which have a disaster recovery plan associated with them. Theimplementation of this task is dependent on the policies and infrastructure of your organization.Appendix: Contact QuestContact Quest SupportQuest Support is available to customers who have a trial version of a Quest product or who havepurchased a Quest product and have a valid maintenance contract. Quest Support providesunlimited 24x7 access to SupportLink, our self-service portal. Visit SupportLink athttp://support.quest.com.From SupportLink, you can do the following:l Retrieve thousands of solutions from our online Knowledgebasel Download the latest releases and service packsl Create, update and review Support casesView the Global Support Guide for a detailed explanation of support programs, online services,contact information, policies and procedures. The guide is available at: http://support.quest.com.Spotlight SupportWhen contacting Quest Support about a Spotlight product, please have the version and buildnumber handy. You can obtain both by choosing the About option from the Help menu in therelevant Spotlight application. If your question is about an error message, write the messagedown and have it available for the technical support representative.Whenever you have a question about Spotlight, click Support Bundle or Contact Support onthe Help menu. This creates a file called SpotlightSupport.zip. This file contains a snapshot ofyour Spotlight installation. Send this file and your request for assistance to Quest Software(support@quest.com).Note: You can find SpotlightSupport.zip in different locations under different operating systems.l Windows XP: C:\Documents and Settings\\Application Data\QuestSoftware\Spotlight\Default\Supportl Vista: C:\Users\\AppData\Roaming\Quest Software\Spotlight\Default\SupportIf you find that information in the Help system or printed user guides is incorrect, unclear, orincomplete, please report the problem to Quest Support.Spotlight on SQL Server Reporting and Trending GuideAppendix: Contact Quest46Contact Quest SoftwareEmail info@quest.comMailQuest Software, Inc.World Headquarters5 Polaris WayAliso Viejo, CA 92656USAWeb site www.quest.comSee our web site for regional and international office information.About Quest SoftwareNow more than ever, organizations need to work smart and improve efficiency. Quest Softwarecreates and supports smart systems management productshelping our customers solve everydayIT challenges faster and easier. Visit www.quest.com more information.IndexAabout Quest Software 46alarm data 27CContact Quest Software 46custom counters 26Customer support 45Ddefinition files 11Eexample reports 10Mmaintenance plan 43Qqueryalarm data 27custom counters 26Spotlight Statistics Repository 21Quest Software 46Quest support 45Rreporting 6reportsabout 9create 11customize 11Database Growth 10definition files 11deploy 13examples 10open 9Server Alarms 10SQLServer Health 10web 13Windows Server Health 10Sschema 15Spotlight Reporting 6Spotlight Statistics Repositoryalarm data 27custom counters 26how it works 17maintenance plan 43query 21schema 15specifications 36store data 20stored procedures 33table definitions 29store data 17, 20Spotlight on SQL Server Reporting and Trending GuideIndex48stored procedures 33Support 45Ttable definitions 29Technical support 45trending 6Vviewscopy 7default 7display 7open 7save 7user 7Wweb reports 13server location 13