05 | SET Operators, Windows Functions, and Grouping Brian Alderman | MCT, CEO / Founder of MicroTechPointTobias Ternstrom | Microsoft SQL Server Program ManagerClick to edit Master subtitle style1Course TopicsQuerying Microsoft SQL Server 2012 Jump Start05 | SET Operators, Windows Functions, and Grouping SET operators, Windows functions, GROUPING sets (PIVOT, UNPIVOT, CUBE, ROLLUP)06 | Modifying Data INSERT, UPDATE, and DELETE statements, use of defaults, constraints, and triggers, OUTPUT07 | Programming with T-SQL Using T-SQL programming elements, implementing error handling, understanding and implementing transactions 08 | Retrieving SQL Server Metadata and Improving Query PerformanceQuerying system catalogs and dynamic management views, creating and executing stored procedures, improving SQL Server query performanceModule 4: Managing SecurityCourse 2786B2SET operators (UNION, INTERSECT, EXCEPT, APPLY)Windows functionsGrouping sets (PIVOT, UNPIVOT, CUBE, ROLLUP)Module Overview3Set Operators Module 4: Managing SecurityCourse 2786B4Interactions between setsThe results of two input queries may be combined, compared, or operated against each otherBoth sets must have the same number of compatible columnsORDER BY not allowed in input queries, but may be used for result of set operationNULLs considered equal when comparing setsSET operators include UNION, INTERSECT, EXCEPT, and APPLY

[ORDER BY ]Module 12: Using Set OperatorsCourse 10774A5Using the UNION operatorUNION returns a result set of distinct rows combined from both sidesDuplicates removed during query processing (affects performance)-- only distinct rows from both queries are returnedSELECT ProductID, OrderQty, UnitPrice FROM Sales.SalesOrderDetailUNIONSELECT ProductID, OrderQty, UnitPrice FROM Purchasing.PurchaseOrderDetailModule 12: Using Set OperatorsCourse 10774A6Using the UNION ALL operatorUNION ALL returns a result set with all rows from both setsTo avoid performance penalty, use UNION ALL even if you know there are no duplicates-- all rows from both queries are returnedSELECT ProductID, OrderQty, UnitPrice FROM Sales.SalesOrderDetailUNION ALLSELECT ProductID, OrderQty, UnitPrice FROM Purchasing.PurchaseOrderDetailModule 12: Using Set OperatorsCourse 10774A7Using the INTERSECT operatorINTERSECT returns only distinct rows that appear in both result setsINTERSECT-- only rows that exist in both queries are returnedSELECT ProductID, OrderQty, UnitPrice FROM Sales.SalesOrderDetailINTERSECTSELECT ProductID, OrderQty, UnitPrice FROM Purchasing.PurchaseOrderDetailModule 12: Using Set OperatorsCourse 10774A8Using the EXCEPT operatorEXCEPT returns only distinct rows that appear in the left set but not the rightOrder in which sets are specified matters-- only rows from Sales are returnedSELECT ProductID, OrderQty, UnitPrice FROM Sales.SalesOrderDetailEXCEPTSELECT ProductID, OrderQty, UnitPrice FROM Purchasing.PurchaseOrderDetailModule 12: Using Set OperatorsCourse 10774A9Using the APPLY operatorAPPLY is a table operator used in the FROM clause and can be either a CROSS APPLY or OUTER APPLYOperates on two input tables, left and rightRight table is often a derived table or a table-valued function

OUTER APPLY is similar to LEFT OUTER JOIN between two tablesOUTER APPLY applies the right table expression to each row in left tableOUTER APPLY adds rows for those with NULL in columns for right table

SELECT FROM AS CROSS/OUTER APPLY AS Module 12: Using Set OperatorsCourse 10774A10Working with Set OperatorsDemoModule 4: Managing SecurityCourse 2786B11Windows FunctionsModule 4: Managing SecurityCourse 2786B12SQL windowingWindows extend T-SQL's set-based approachWindows allow you to specify an order as part of a calculation, without regard to order of input or final output orderWindows allow partitioning and framing of rows to support functionsWindow functions can simplify queries that need to find running totals, moving averages, or gaps in dataModule 13: Using Window Ranking, Offset and Aggregate FunctionsCourse 10774A13Partitioning windowsPartitioning limits a set to rows with same value in the partitioning columnUse PARTITION BY in the OVER() clauseWithout a PARTITION BY clause defined, OVER() creates a single partition of all rowsSELECT CustomerID, OrderDate, TotalDue, SUM(TotalDue) OVER(PARTITION BY CustomerID)AS TotalDueByCustFROM Sales.SalesOrderHeader;CustomerID OrderDate TotalDue TotalDueByCust---------- -------------------------- -------- -------------- 2007-08-01 00:00:00.000 3756.989 9115.134111000 2007-10-01 00:00:00.000 2587.8769 9115.1341 11000 2006-09-01 00:00:00.000 2770.2682 9115.1341 11001 2007-08-01 00:00:00.000 2674.0227 7054.187511001 2006-11-01 00:00:00.000 3729.364 7054.187511001 2007-04-01 00:00:00.000 650.8008 7054.1875Module 13: Using Window Ranking, Offset and Aggregate FunctionsCourse 10774A14Defining window functionsA windows function is a function applied to a window, or set of rowsWindow functions include aggregate, ranking, distribution, and offset functionsWindow functions depend on set created by OVER()

Windows aggregate functions: Similar to grouped aggregate functions such as SUM, MIN, MAX, etc. Applied to windows defined by OVER clause Support partitioning, ordering, and framing

Module 13: Using Window Ranking, Offset and Aggregate FunctionsCourse 10774A15Window ranking functionsRanking functions require a windows order clausePartitioning is optionalTo display results in sorted order still requires ORDER BY!FunctionDescriptionRANK Returns the rank of each row within the partition of a result set. May include ties and gaps.DENSE_RANKReturns the rank of each row within the partition of a result set. May include ties but will not include gaps.ROW_NUMBERReturns a unique sequential row number within partition based on current order.NTILEDistributes the rows in an ordered partition into a specified number of groups. Returns the number of the group to which the current row belongs.Module 13: Using Window Ranking, Offset and Aggregate FunctionsCourse 10774A16Window offset functionsWindow offset functions allow comparisons between rows in a set without the need for a self-joinOffset functions operate on an position relative to the current row, or to the start or end of the window frameFunctionDescriptionLAGReturns an expression from a previous row that is a defined offset from the current row. Returns NULL if no row at specified position.LEADReturns an expression from a later row that is a defined offset from the current row. Returns NULL if no row at specified position.FIRST_VALUEReturns the first value in the current window frame. Requires window ordering to be meaningful.LAST_VALUEReturns the last value in the current window frame. Requires window ordering to be meaningful.Module 13: Using Window Ranking, Offset and Aggregate FunctionsCourse 10774A17Using Windows FunctionsDemoModule 4: Managing SecurityCourse 2786B18Grouping SetsModule 4: Managing SecurityCourse 2786B19What is pivoting?Pivoting data is rotating data from a rows-based orientation to a columns-based orientationDistinct values from a single column are projected across as headings for other columns - may include aggregation

Pivoted dataModule 14: Pivoting and Grouping SetsCourse 10774A20Pivoting includes three phases:Grouping determines which element gets a row in the result set Spreading provides the distinct values to be pivoted acrossAggregation performs an aggregation function (such as SUM)SELECT Category, [2006],[2007],[2008]FROM ( SELECT Category, Qty, Orderyear FROM Sales.CategoryQtyYear) AS D PIVOT(SUM(QTY) FOR orderyear IN([2006],[2007],[2008])) AS pvt;GroupingSpreadingAggregationSELECT VendorID, [250] AS Emp1, [251] AS Emp2, [256] AS Emp3, [257] AS Emp4, [260] AS Emp5FROM (SELECT PurchaseOrderID, EmployeeID, VendorIDFROM Purchasing.PurchaseOrderHeader) pPIVOT(COUNT (PurchaseOrderID)FOR EmployeeID IN( [250], [251], [256], [257], [260] )) AS pvtORDER BY pvt.VendorID;VendorID Emp1 Emp2 Emp3 Emp4 Emp5-------- ----- ----- ----- ----- ----- 1492 2 5 4 4 4 2 5 4 5 41496 2 4 4 5 5Module 14: Pivoting and Grouping SetsCourse 10774A21Writing queries with UNPIVOTCREATE TABLE pvt (VendorID int, Emp1 int, Emp2 int, Emp3 int, Emp4 int, Emp5 int);GOINSERT INTO pvt VALUES (1,4,3,5,4,4);INSERT INTO pvt VALUES (2,4,1,5,5,5);INSERT INTO pvt VALUES (3,4,3,5,4,4);GO

SELECT VendorID, Employee, OrdersFROM (SELECT VendorID, Emp1, Emp2, Emp3, Emp4, Emp5 FROM pvt) pUNPIVOT (Orders FOR Employee IN (Emp1, Emp2, Emp3, Emp4, Emp5))AS unpvt;GO VendorID Employee Orders 1 Emp1 4 1 Emp2 3Module 14: Pivoting and Grouping SetsCourse 10774A22Writing queries with grouping setsGROUPING SETS subclause builds on T-SQL GROUP BY clauseAllows multiple groupings to be defined in same queryAlternative to use of UNION ALL to combine multiple outputs (each with different GROUP BY) into one result setSELECT FROM GROUP BY GROUPING SETS((),--one or more columns(),--one or more columns() -- empty parentheses if aggregating all rows);SELECT TerritoryID, CustomerID, SUM(TotalDue) AS TotalAmountDueFROM Sales.SalesOrderHeaderGROUP BY GROUPING SETS((TerritoryID),(CustomerID),());TerritoryID CustomerID TotalAmountDue--------------- ----------- --------------NULL 30116 211671.2674NULL 30117 919801.8188NULL 30118 313671.5352NULL NULL 123216786.1159 3 NULL 8913299.2473 NULL 18398929.188 NULL 11814376.0952 NULL 18061660.3717 NULL 8119749.346Module 14: Pivoting and Grouping SetsCourse 10774A23CUBE and ROLLUPCUBE provides shortcut for defining grouping sets given a list of columnsAll possible combinations of grouping sets are created

ROLLUP provides shortcut for defining grouping sets, creates combinations assuming input columns form a hierarchySELECT TerritoryID, CustomerID, SUM(TotalDue) AS TotalAmountDueFROM Sales.SalesOrderHeaderGROUP BY CUBE(TerritoryID, CustomerID)ORDER BY TerritoryID, CustomerID;SELECT TerritoryID, CustomerID, SUM(TotalDue) AS TotalAmountDueFROM Sales.SalesOrderHeaderGROUP BY ROLLUP(TerritoryID, CustomerID)ORDER BY TerritoryID, CustomerID;Module 14: Pivoting and Grouping SetsCourse 10774A24Working with PIVOT, CUBE, and ROLLUPDemoModule 4: Managing SecurityCourse 2786B25SummaryWorking with set is he interaction between sets of data results in two input queries that may be combined, compared, or operated against each otherBoth sets must have the same number of compatible columnsORDER BY not allowed in input queries, but may be used for result of the entire set operationSET operators include UNION, INTERSECT, EXCEPT, and APPLY

[ORDER BY ]Module 4: Managing SecurityCourse 2786B26Summary-- only distinct rows from both queries are returnedSELECT ProductID, OrderQty, UnitPrice FROM Sales.SalesOrderDetailUNIONSELECT ProductID, OrderQty, UnitPrice FROM Purchasing.PurchaseOrderDetail-- all rows from both queries are returnedSELECT ProductID, OrderQty, UnitPrice FROM Sales.SalesOrderDetailUNION ALLSELECT ProductID, OrderQty, UnitPrice FROM Purchasing.PurchaseOrderDetail-- only rows that exist in both queries are returnedSELECT ProductID, OrderQty, UnitPrice FROM Sales.SalesOrderDetailINTERSECTSELECT ProductID, OrderQty, UnitPrice FROM Purchasing.PurchaseOrderDetail-- only rows from Sales are returnedSELECT ProductID, OrderQty, UnitPrice FROM Sales.SalesOrderDetailEXCEPTSELECT ProductID, OrderQty, UnitPrice FROM Purchasing.PurchaseOrderDetailModule 4: Managing SecurityCourse 2786B27SummaryA windows function depends on sets created using the OVER() and are applied to a window, or set of rows that can include aggregates, ranking, distribution, and offset functions. Windows Ranking functions includeRANKDENSE_RANKROW_NUMBERNTILE

Windows Offset functions includeLAGLEADFIRST_VALUELAST_VALUEModule 4: Managing SecurityCourse 2786B28SummaryPivoting data is rotating data from a rows-based orientation to a columns-based orientation and DISTINCT values from a single column are displayed across as column headings - may include aggregation

The GROUPING SETS clause builds on the T-SQL GROUP BY clause by allowing multiple groups to be defined in the same query

A CUBE provides a shortcut for defining grouping sets given a list of columns therefore all possible combinations of GROUPING SETS are created

A ROLLUP provides a shortcut for defining grouping sets, by creating combinations with the assumption the input columns form a hierarchy

Module 4: Managing SecurityCourse 2786B292013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Office, Azure, System Center, Dynamics and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.Module 4: Managing SecurityCourse 2786B30