Microsoft ResearchMicrosoft Research

Jim GrayJim GrayResearcherResearcher

Microsoft ResearchMicrosoft ResearchMicrosoft CorporationMicrosoft Corporation

OutlineOutline

Highlights of MS Highlights of MS ResearchResearch

DB-Related ResearchDB-Related Researchin some detailin some detail

Microsoft ResearchMicrosoft Researchhttp://Research.Microsoft.com/http://Research.Microsoft.com/

Goal: pursue strategic technologies for MSGoal: pursue strategic technologies for MS 231 Researchers in 20 areas231 Researchers in 20 areas Plan to grow to 600 in 3 yearsPlan to grow to 600 in 3 years Internationally recognized research teamsInternationally recognized research teams

Hundreds of publications, conference presentationsHundreds of publications, conference presentations Leadership in professional societies, journals, Leadership in professional societies, journals,

conferencesconferences

Virtually all MS products use technology from Virtually all MS products use technology from MS ResearchMS Research

Broad Research AreasBroad Research Areas Systems & architectureSystems & architecture

Distributed & scaleable systems, Distributed & scaleable systems, databases, cryptography, telepresencedatabases, cryptography, telepresence

Make programmers more productiveMake programmers more productive Tools, compilersTools, compilers

Human-oriented interactionHuman-oriented interaction Speech, natural language, vision, intelligenceSpeech, natural language, vision, intelligence 3D Graphics, moving beyond GUI3D Graphics, moving beyond GUI

Fundamental problemsFundamental problems

Development ToolsDevelopment Tools

Analysis of executablesAnalysis of executables Dynamic analysis driven by user scenariosDynamic analysis driven by user scenarios Instrumented codeInstrumented code

Automatic reorganization of executablesAutomatic reorganization of executables Reduce code working set size Reduce code working set size Branch straighteningBranch straightening Boot ordering for boot time reductionBoot ordering for boot time reduction

Reduced code working sets up to 50%Reduced code working sets up to 50% Improved throughput by 10%Improved throughput by 10%

Smaller Working SetsSmaller Working SetsFaster Startup & ExecutionFaster Startup & Execution

0

100

200

300

400

500

600

0 50 100

Seconds

Pag

es R

efer

ence

d

Original

Optimized

2%

7% 8%

35%

39%

10%

33%

0%

10%

20%

30%

40%

50%

TxRd$NoL TxRW$NoL TxWrt$

NT Network Drivers Throughput Speedup

Speech TechnologySpeech Technology Speech recognitionSpeech recognition

Speaker independent, Command and ControlSpeaker independent, Command and Control DictationDictation

Speaker independent, large vocabularySpeaker independent, large vocabulary Discrete and continuous speechDiscrete and continuous speech

Trainable speech synthesisTrainable speech synthesis Prosody and concatenative speech units learned Prosody and concatenative speech units learned

from corpusfrom corpus Demos and SDKs at web siteDemos and SDKs at web site

http://research.microsoft.com/research/srg/http://research.microsoft.com/research/srg/

Natural Language UnderstandingNatural Language Understanding Broad-coverage syntax analyzerBroad-coverage syntax analyzer.. Dictionary-based semantic network Dictionary-based semantic network

provides growing knowledge base.provides growing knowledge base. Underlying system is multi-lingual.Underlying system is multi-lingual. Contributions to MS word-processingContributions to MS word-processing

Levels of writing critiquesWe scheduled teh next meeting for noon.Each of the products are designed to help.I saw the Grand Canyon flying to Arizona.Ladies are requested not to have children in the bar. (from a sign in a Norwegian cocktail lounge)

Advanced summaryAdvanced summaryInteractive gamesInteractive games

Peedy+Peedy+

Improved SRImproved SR

Auto indexingAuto indexingSyntactic critiquesSyntactic critiques

Phrase spacingPhrase spacing

Find and replaceFind and replace

Bob+Bob+

Semantic critiquesSemantic critiques

Enhanced helpEnhanced help

NL queryNL queryImproved IRImproved IR

Interactive moviesInteractive moviesInfo highway cruiserInfo highway cruiser

RoboticsRobotics

VisionVision

LibraryLibrary

Dictionary

Diction

ary

MachineMachinelearninglearning

UIUI

22ndnd F F

ConceptConceptnormalizingnormalizing

Discourse/Discourse/pragmaticspragmatics

UniversityUniversity

SenseSensechoosingchoosing

Logical formLogical form

MorphologMorphologyy

fjeiofjdksleriowe.asmqweiqpo erooei iqpwe iioqwpe ec,l;akl

fjeiofjdksleriowe.asmqweiqpo erooei iqpwe iioqwpe ec,l;akl

Revised syntaxRevised syntax

SRSR

Initial Initial syntaxsyntax

ProbsProbs(DT)(DT)

Natural Language Natural Language Roadmap and Mile StonesRoadmap and Mile Stones

3D Graphics Research3D Graphics Research Bring very high performance, high quality graphics to Bring very high performance, high quality graphics to

PCsPCs InteractiveInteractive Uniform treatment of multimediaUniform treatment of multimedia

ModelingModeling Representation of 3D modelsRepresentation of 3D models Automatic simplificationAutomatic simplification

AnimationAnimation

Simplification ProblemSimplification Problem

70,00070,000 8,7008,700 34,10034,100

Competing goals: accuracy & concisenessCompeting goals: accuracy & conciseness

4,2004,200 2,6002,600 2,3002,300

difficult to author directly!

Simplification Results Simplification Results (creases)(creases)

original

simplified

(~10 minutes)

34,100

2,300

2,300

Vision ProjectsVision Projects

3D Reconstruction from Video and Images3D Reconstruction from Video and Images Motion analysis for video compressionMotion analysis for video compression Model acquisition for renderingModel acquisition for rendering

Visual Human/Computer InteractionVisual Human/Computer Interaction Communication by gestures and expressionsCommunication by gestures and expressions Multimodal speech/vision interfacesMultimodal speech/vision interfaces

Motion AnalysisMotion Analysis

convert masked images into a background convert masked images into a background sprite for content-based coding [Scrunch]sprite for content-based coding [Scrunch]

++ + + + +

==

working on motion trackingworking on motion tracking

OutlineOutline

Highlights of MS Highlights of MS ResearchResearch

DB-Related ResearchDB-Related Research

University of Washington University of Washington & Microsoft Research& Microsoft ResearchSummer Institute 1997Summer Institute 1997

Data Mining & Knowledge Data Mining & Knowledge DiscoveryDiscovery

July 6-11, 1997July 6-11, 1997

A collection of 75 researchers from around the world: –Statistics

–Databases–AI/pattern recognition–Visualization–Systems–HPC

http://research.Microsoft.com/uwmsrdmi

Data Mining and OLAPData Mining and OLAP The Query formulation problemThe Query formulation problem::

difficult to formulate as DB queries difficult to formulate as DB queries which records are fraudulent transactions?which records are fraudulent transactions? which households are likely to prefer a Ford over a Toyota?which households are likely to prefer a Ford over a Toyota? Who’s a good credit risk in my customer DB?Who’s a good credit risk in my customer DB?

Yet database contains the information Yet database contains the information good/bad customer, profitability, response rategood/bad customer, profitability, response rate known objects of interest (stars/galaxies), results in known objects of interest (stars/galaxies), results in

medical studies with higher details, etc...medical studies with higher details, etc...

DM helps DM helps Use data to build predictorsUse data to build predictors

regression, classification, clustering etc.regression, classification, clustering etc. Generate summaries to aid understanding Generate summaries to aid understanding

find “easy to describe” partitions in data find “easy to describe” partitions in data automatically (via clustering/segmentation)automatically (via clustering/segmentation)

find clusters not well-known to analystfind clusters not well-known to analyst Example: Example:

Analyze web server hits to categorize clientsAnalyze web server hits to categorize clients ..

A new mode of interacting with databaseA new mode of interacting with database

DatabasesDatabases RecoveryRecovery

Recover application state after sys fail.Recover application state after sys fail. Data StructuresData Structures

High concurrency data structures High concurrency data structures Key range locking in SS 7 based on Lomet’s work.Key range locking in SS 7 based on Lomet’s work.

Auto DB Design & AdminAuto DB Design & Admin Index DB designer started as research project.Index DB designer started as research project. More comingMore coming

Data Base ResearchData Base Researchhttp://research.Microsoft.com/db/http://research.Microsoft.com/db/

Scaleable ServersScaleable Servers High Availability SQL ServerHigh Availability SQL Server

Virtual Server designVirtual Server design Always Up prototypeAlways Up prototype

Billion Transactions Per Day demoBillion Transactions Per Day demo Combined SQL 6.5 with MTS, DTCCombined SQL 6.5 with MTS, DTC

RAGS: stress test NT DBsRAGS: stress test NT DBs TerraServer: a giant single node SQL7 DBTerraServer: a giant single node SQL7 DB Always Up prototypeAlways Up prototype

Scaleable ServersScaleable ServersBOTH SMP and ClusterBOTH SMP and Cluster

SMPSuper Server

DepartmentalServer

PersonalSystem

Grow Up with SMPGrow Up with SMP4xP6 is now standard4xP6 is now standard

Grow Out with ClusterGrow Out with Cluster

Cluster has inexpensive partsCluster has inexpensive parts

Clusterof PCs

Scale OUTScale OUTClusters Have AdvantagesClusters Have Advantages

Fault tolerance:Fault tolerance: Spare modules mask failuresSpare modules mask failures

Modular growth Modular growth without limitswithout limits Grow by adding small modulesGrow by adding small modules

Parallel data searchParallel data search Use multiple processors and disksUse multiple processors and disks

Clients and servers made from the same stuffClients and servers made from the same stuff Inexpensive: built with commodity CyberBricks Inexpensive: built with commodity CyberBricks

Billion Transactions per Day ProjectBillion Transactions per Day Project Built a 45-node Windows NT Cluster Built a 45-node Windows NT Cluster

(with help from Intel & Compaq)(with help from Intel & Compaq) > 900 disks> 900 disks

All off-the-shelf partsAll off-the-shelf parts Using SQL Server & Using SQL Server &

DTC distributed transactionsDTC distributed transactionsDCOM & ODBC clientsDCOM & ODBC clientson 20 front-end nodeson 20 front-end nodes

DebitCredit TransactionDebitCredit Transaction Each server node has 1/20 th of the DB Each server node has 1/20 th of the DB Each server node does 1/20 th of the workEach server node does 1/20 th of the work 15% of the transactions are “distributed”15% of the transactions are “distributed”

1 B tpd: So What?1 B tpd: So What? Shows what is possible, easy to buildShows what is possible, easy to build

Grows without limitsGrows without limits

Shows scaleup of DTC, MTS, SQL…Shows scaleup of DTC, MTS, SQL… Shows (again) that shared-nothing clusters Shows (again) that shared-nothing clusters

scalescale

Next task: make it easy.Next task: make it easy. auto partition dataauto partition data auto partition applicationauto partition application auto manage & operateauto manage & operate

ParallelismParallelismThe OTHER aspect of clustersThe OTHER aspect of clusters

Clusters of machines Clusters of machines allow two kinds allow two kinds of parallelismof parallelism Many little jobs: online Many little jobs: online

transaction processingtransaction processing TPC-A, B, C…TPC-A, B, C…

A few big jobs: data A few big jobs: data search and analysissearch and analysis TPC-D, DSS, OLAPTPC-D, DSS, OLAP

Both give Both give automatic parallelismautomatic parallelism

Kinds of Parallel ExecutionKinds of Parallel Execution

Pipeline

Partition outputs split N ways inputs merge M ways

Any Sequential Program

Any Sequential Program

Any Sequential

Any Sequential Program Program

Data RiversData RiversSplit + Merge StreamsSplit + Merge Streams

River

M ConsumersN producers

Producers add records to the river, Consumers consume records from the riverPurely sequential programming.River does flow control and buffering

does partition and merge of data records

River = Exchange operator in SQL 7

N X M Data Streams

N x M way ParallelismN x M way Parallelism

A...E F...J K...N O...S T...Z

Merge

Join

Sort

Join

Sort

Join

Sort

Join

Sort

Join

Sort

Merge Merge

N inputs, M outputs, no bottlenecks.

Partitioned DataPartitioned and Pipelined Data Flows

Clustered DBsClustered DBs Pieces are in place with SQL 7.0Pieces are in place with SQL 7.0

Manage arrays of SQL ServersManage arrays of SQL Servers Partition data among filesPartition data among files Disk and CPU Parallel query plansDisk and CPU Parallel query plans Distributed query via OLE DBDistributed query via OLE DB High Availability with MSCS failoverHigh Availability with MSCS failover

Next Parallel Databases on NT clustersNext Parallel Databases on NT clusters Parallel QueryParallel Query Transparent access to partitioned tablesTransparent access to partitioned tables Grow without limits (16 x 2 TB)Grow without limits (16 x 2 TB) Do not add complexityDo not add complexity

Design, Administer, Operate, UseDesign, Administer, Operate, Use

Scaleup - Big DatabaseScaleup - Big Database Build a 1 TB SQL Server databaseBuild a 1 TB SQL Server database

Show off Windows NT and Show off Windows NT and SQL Server scalabilitySQL Server scalability

Stress test the productStress test the product Data must beData must be

1 TB1 TB UnencumberedUnencumbered Interesting to everyone everywhereInteresting to everyone everywhere AndAnd not offensive to anyone anywhere not offensive to anyone anywhere

Loaded Loaded 1.1 M place names from Encarta World Atlas1.1 M place names from Encarta World Atlas 1 M Sq Km from USGS (1 meter resolution)1 M Sq Km from USGS (1 meter resolution) 2 M Sq Km from Russian Space agency (2 m)2 M Sq Km from Russian Space agency (2 m)

Will be on web (world’s largest atlas)Will be on web (world’s largest atlas) Sell images with commerce server.Sell images with commerce server. USGS CRDA: 3 TB more coming.USGS CRDA: 3 TB more coming.

TerraServerTerraServerWorld’s Largest PC!World’s Largest PC!

324 disks (2.9 terabytes) 324 disks (2.9 terabytes)

8 x 440Mhz Alpha CPUs8 x 440Mhz Alpha CPUs

10 GB DRAM10 GB DRAM

NT EE & SQL 7.0NT EE & SQL 7.0

Photo of the planetPhoto of the planetUSGS and RussianUSGS and Russianimagesimages

BackgroundBackground Earth is 500 Tera-meters squareEarth is 500 Tera-meters square

USA is 10 tmUSA is 10 tm22

100 TM100 TM22 land in 70ºN to 70ºS land in 70ºN to 70ºS We have pictures of 6% of itWe have pictures of 6% of it

3 tsm from USGS3 tsm from USGS 2 tsm from Russian Space Agency 2 tsm from Russian Space Agency

Compress 5:1 Compress 5:1 (JPEG)(JPEG) to 1.5 TB. to 1.5 TB. Slice into 10 KB chunksSlice into 10 KB chunks Store chunks in DBStore chunks in DB Navigate withNavigate with

Encarta™ AtlasEncarta™ Atlas globeglobe gazetteergazetteer

StreetsPlus™ in the USAStreetsPlus™ in the USA

40x60 km2 jump image

20x30 km2 browse image

10x15 km2 thumbnail

1.8x1.2 km2 tile

SomedaySomeday multi-spectral multi-spectral

image image of everywhereof everywhere once a day / houronce a day / hour

USGS Digital Ortho Quads USGS Digital Ortho Quads (DOQ)(DOQ)

US Geologic SurveyUS Geologic Survey 3 TeraBytes3 TeraBytes Most data not yet publishedMost data not yet published Based on a CRADABased on a CRADA

TerraServer makes data available.TerraServer makes data available.

USGS “DOQ”

1x1 meter4 TBContinentalUSNew DataComing

Russian Space AgencyRussian Space Agency(SovInfomSputnik)(SovInfomSputnik)

SPIN-2 SPIN-2 (Aerial Images is Worldwide Distributor)(Aerial Images is Worldwide Distributor)

1.5 Meter Geo Rectified imagery of (almost) anywhere1.5 Meter Geo Rectified imagery of (almost) anywhere Almost equal-area projectionAlmost equal-area projection De-classified satellite photos (from 200 KM),De-classified satellite photos (from 200 KM), More data coming (1 m)More data coming (1 m) Want to sell imagery on Internet.Want to sell imagery on Internet. Putting 2 tmPutting 2 tm22 onto TerraServer. onto TerraServer.

SPIN-2

http://www.TerraServer.com

Demo Demo

Microsoft

BackOfficeSPIN-2

1TB Database Server AlphaServer 8400 4x400. 10 GB RAM 324 StorageWorks disks 10 drive tape library (STC Timber Wolf DLT7000 )

SPIN-2

HardwareHardware

S T C9 7 4 0D L TT a p eL i b r a r y

4 89 G BD r i v e s

4 89 G BD r i v e s

4 89 G BD r i v e s

A l p h aS e r v e r8 4 0 0

E n t e r p r i s e S t o r a g e A r r a y

1 0 0 M b p sE t h e r n e t S w i t c h D S 3 I n t e r n e t

M a pS e r v e r

4 89 G BD r i v e s

4 89 G BD r i v e s

4 89 G BD r i v e s

8 x 4 4 0 M H zA l p h a c p u s1 0 G B D R A M

4 89 G BD r i v e s

S i t eS e r v e r s

broswer

HTMLJava

Viewer

The Internet

Web Client

Microsoft AutomapActiveX Server

Internet InfoServer 4.0

Image DeliveryApplication

SQL Server7

MicrosoftSite Server EE

Internet InformationServer 4.0

Image Provider Site(s)

Terra-Server DB Automap Server

Sphinx(SQL Server)

Terra-ServerStored Procedures

InternetInformationServer 4.0

ImageServer

Active Server Pages

MTS

Terra-Server Web Site

SoftwareSoftware

Backup and RecoveryBackup and Recovery STC 9717 Tape robotSTC 9717 Tape robot Legato NetWorker™Legato NetWorker™ Sphinx Backup/Restore UtilitySphinx Backup/Restore Utility Clocked at 80 MBps!!Clocked at 80 MBps!!

SQL Server Enterprise MgrSQL Server Enterprise Mgr DBA MaintenanceDBA Maintenance SQL Performance MonitorSQL Performance Monitor

System System Management & Management & MaintenanceMaintenance

TerraServer File Group LayoutTerraServer File Group Layout Convert 324 disks to 28 RAID5 setsConvert 324 disks to 28 RAID5 sets

plus 28 spare drivesplus 28 spare drives

Make 4 NT volumes (RAID 50)Make 4 NT volumes (RAID 50) 595 GB per volume595 GB per volume

Build 30 20GB files on each volumeBuild 30 20GB files on each volume DB is File Group of 120 filesDB is File Group of 120 files

HSZ70 A

HSZ70 B

HSZ70 A

HSZ70 B

HSZ70 A

HSZ70 B

HSZ70 A

HSZ70 B

HSZ70 A

HSZ70 B

HSZ70 A

HSZ70 B

E: F: G: H:

HSZ70 A

HSZ70 B

AlternateNameName

CountryIDStateIDTypeID

GazSourcIDLatitude

LongitudeUGridIDZGridIDDOQdate

SPIN2date

PlaceID

Place

ImageFlag

FeatureType

TypeIDDescription

13

GazetteerSource

GazSrcIDDescription

11,089,897

Country

CountryIDCountryName

UNcode

264

State

StateIDCountryIDStateName

1083

CountrySearch

AlternateNameCountryIDGazSrcID

1148

StateSerach

AlternateNameCountryID

StateIDFreatureIDGazSrcID

3776

PlaceGrid

ZGridIDBestPlaceName

XDistanceYDistrance

50,000,000

Gazetteer DesignGazetteer Design Classic Snowflake SchemaClassic Snowflake Schema Fast First hint to OptimizerFast First hint to Optimizer

1

ImgSource

SrcIDSrcName

SrcTblNameSrcDescription

GridSysIDImgTypeID

2

Jump

UGridIDZGridID

ZTileGridIDImgDataImgDate

ImgTypeIDImgMetaID

SrcIDEncryptKeyFile Name

.65 M SPIN21.5 M USGS

OriginalMetaData

OrigMetaIDSrcID

ImageSourceAgency

SourcePhotoIDSourcePhotoDateSourceDEMDate

MetaDataDateProductionSystem

ProductionDateDataFileSizeCompressionHeaderBytes

…80 other fields

650 k SPIN22 M USGS

Pick

NameDescription

LinkPickDate

10

ImageMeta

ImgMetaIDOrigMetaIDImgStatusImgDate

ImgTypeIDJumpPixHeightJumpPixWidth

BrowsePixHeightBrowsePixWidthThumbPixWidthThumbPixHeight

CutColCutRowMidLat

MidLongNELat

NELongNWLat

NWLongSELat

SELongSWLat

SWLongUGridID

UTMZoneXUtmIDYUtmIDXGridIDYGridIDZGridID

650 k SPIN22 M USGS

ImgType

ImgTypeIDImgFileDescImgFileExt

MimeStr

4

Browse

UGridIDZGridID

ZTileGridIDImgDataImgDate

ImgTypeIDImgMetaID

SrcIDEncryptKeyFile Name

.65 M SPIN21.5 M USGS

Thumb

UGridIDZGridID

ZTileGridIDImgDataImgDate

ImgTypeIDImgMetaID

SrcIDEncryptKeyFile Name

.65 M SPIN21.5 M USGS

Tile

UGridIDZGridID

ZTileGridIDImgDataImgDate

ImgTypeIDImgMetaID

SrcIDEncryptKeyFile Name

16 M SPIN296 M USGS

xxx

UGridHits

URLUGridID

ZTileGridIDcount

Log

URLTime

<extensivelist of actionparameters

xxx

TileMeta

ImgMetaIDOrigMetaID

SrcIDImgStatusImgDate

ImgTypeIDTilePixHeightTilePixWidth

CutColCutRowMidLat

MidLongNELat

NELongNWLat

NWLongSELat

SELongSWLat

SWLongUGridID

UTMZoneXUtmIDYUtmIDXGridIDYGridIDZGridID

16 M SPIN296 M USGS

Image Data DesignImage Data Design Image pyramid stored in DBMS Image pyramid stored in DBMS (250 M recs)(250 M recs)

Image Delivery and LoadImage Delivery and LoadDLTTape “tar”

\Drop’N’ DoJobWait 4LoadLoadMgr

DB

100mbitEtherSwitch

108 9.1 GBDrives

Enterprise Storage Array

AlphaServer8400

108 9.1 GBDrives

108 9.1 GBDrives

STCDLTTape

Library

604.3 GBDrives

AlphaServer4100

ESAAlphaServer4100

LoadMgr

DLTTape

NTBackup

ImgCutter

\Drop’N’ \Images

10: ImgCutter20: Partition30: ThumbImg40: BrowseImg45: JumpImg50: TileImg55: Meta Data60: Tile Meta70: Img Meta80: Update Place

...LoadMgr

Terra-Server TablesTerra-Server Tables USGS DOQ DataUSGS DOQ Data

48,000 DOQ images are input (45-55mb / image)48,000 DOQ images are input (45-55mb / image) Creates 864,000 Jump, Thumb, & Browse images (3.5 m rows)Creates 864,000 Jump, Thumb, & Browse images (3.5 m rows) Creates 55.3 m Tile images (110.6 m rows)Creates 55.3 m Tile images (110.6 m rows)

SPIN-2 DataSPIN-2 Data 3200 278 MB images are input (approximate size)3200 278 MB images are input (approximate size) Creates 620,800 Jump, Thumb, & Browse images (2.5 m rows)Creates 620,800 Jump, Thumb, & Browse images (2.5 m rows) Creates 15.5 m Tile images (31 m rows)Creates 15.5 m Tile images (31 m rows)

Gazetteer DataGazetteer Data 1.1 m named places (Encarta World Atlas)1.1 m named places (Encarta World Atlas) 45 m cell names45 m cell names

Total Rows = Total Rows = 228 M today (and growing fast) 228 M today (and growing fast)

Other DetailsOther Details Active Server pages Active Server pages

faster and easier than DB stored procedures.faster and easier than DB stored procedures.

Commerce Server is interestingCommerce Server is interesting Images the Inventory Images the Inventory

no SKU, no SKU, millions of themmillions of them

USGS built their ownUSGS built their own they are very smart, but it is easythey are very smart, but it is easy masquerade as a credit-card reader.masquerade as a credit-card reader.

The earth is a geoid, andThe earth is a geoid, and Every Geographer has a coordinate system Every Geographer has a coordinate system (or two).(or two).

Tapes are still a nightmare.Tapes are still a nightmare. Everyone is a UI expert.Everyone is a UI expert.

SQL 7 TestimonialSQL 7 Testimonial We started using it March 4 1997We started using it March 4 1997

SQL 7 Pre-AlphaSQL 7 Pre-Alpha SQL 7 AlphaSQL 7 Alpha SLQ 7 Beta 1SLQ 7 Beta 1 SQL 7 Beta SQL 7 Beta

Loaded the DB twiceLoaded the DB twice (we made application mistakes)(we made application mistakes)

Now doing it “right”Now doing it “right” Reliability: Great! SQL 7 never lost dataReliability: Great! SQL 7 never lost data Ease of use: Great!Ease of use: Great! Functionality: Great!Functionality: Great!

RAGS: RAGS: RAndom SQL test GeneratorRAndom SQL test Generator

Microsoft spends a LOT of money on testing. Microsoft spends a LOT of money on testing. Idea: test SQL by Idea: test SQL by

generating random generating random correctcorrect queries queries executing queries against databaseexecuting queries against database compare results with SQL 6.5, DB2, Oracle, Sybasecompare results with SQL 6.5, DB2, Oracle, Sybase

Being used in SQL 7.0 testing.Being used in SQL 7.0 testing. Very productive test toolVery productive test tool

Sample Sample Rags Rags Generated Generated StatementStatement

SELECT TOP 3 T1.royalty , T0.price , "Apr 15 1996 10:23AM" , T0.notesFROM titles T0, roysched T1WHERE EXISTS ( SELECT DISTINCT TOP 9 $3.11 , "Apr 15 1996 10:23AM" , T0.advance , ( "<v3``VF;" +(( UPPER(((T2.ord_num +"22\}0G3" )+T2.ord_num ))+("{1FL6t15m" + RTRIM( UPPER((T1.title_id +((("MlV=Cf1kA" +"GS?" )+T2.payterms )+T2.payterms ))))))+(T2.ord_num +RTRIM((LTRIM((T2.title_id +T2.stor_id ))+"2" ))))), T0.advance , (((-(T2.qty ))/(1.0 ))+(((-(-(-1 )))+( DEGREES(T2.qty )))-(-(( -4 )-(-(T2.qty ))))))+(-(-1 )) FROM sales T2 WHERE EXISTS ( SELECT "fQDs" , T2.ord_date , AVG ((-(7 ))/(1 )), MAX (DISTINCT -1 ), LTRIM("0I=L601]H" ), ("jQ\" +((( MAX(T3.phone )+ MAX((RTRIM( UPPER( T5.stor_name ))+((("<" +"9n0yN" )+ UPPER("c" ))+T3.zip ))))+T2.payterms )+ MAX("\?" ))) FROM authors T3, roysched T4, stores T5 WHERE EXISTS ( SELECT DISTINCT TOP 5 LTRIM(T6.state ) FROM stores T6 WHERE ( (-(-(5 )))>= T4.royalty ) AND (( ( ( LOWER( UPPER((("9W8W>kOa" + T6.stor_address )+"{P~" ))))!= ANY (

SELECT TOP 2 LOWER(( UPPER("B9{WIX" )+"J" )) FROM roysched T7 WHERE ( EXISTS (

SELECT (T8.city +(T9.pub_id +((">" +T10.country )+ UPPER( LOWER(T10.city))))), T7.lorange , ((T7.lorange )*((T7.lorange )%(-2 )))/((-5 )-(-2.0 )) FROM publishers T8, pub_info T9, publishers T10 WHERE ( (-10 )<= POWER((T7.royalty )/(T7.lorange ),1)) AND (-1.0 BETWEEN (-9.0 ) AND (POWER(-9.0 ,0.0)) ) ) --EOQ ) AND (NOT (EXISTS ( SELECT MIN (T9.i3 ) FROM roysched T8, d2 T9, stores T10 WHERE ( (T10.city + LOWER(T10.stor_id )) BETWEEN (("QNu@WI" +T10.stor_id )) AND ("DT" ) ) AND ("R|J|" BETWEEN ( LOWER(T10.zip )) AND (LTRIM( UPPER(LTRIM( LOWER(("_\tk`d" +T8.title_id )))))) ) GROUP BY T9.i3, T8.royalty, T9.i3 HAVING -1.0 BETWEEN (SUM (-( SIGN(-(T8.royalty ))))) AND (COUNT(*)) ) --EOQ ) ) ) --EOQ ) AND (((("i|Uv=" +T6.stor_id )+T6.state )+T6.city ) BETWEEN ((((T6.zip +( UPPER(("ec4L}rP^<" +((LTRIM(T6.stor_name )+"fax<" )+("5adWhS" +T6.zip )))) +T6.city ))+"" )+"?>_0:Wi" )) AND (T6.zip ) ) ) AND (T4.lorange BETWEEN ( 3 ) AND (-(8 )) ) ) ) --EOQ GROUP BY ( LOWER(((T3.address +T5.stor_address )+REVERSE((T5.stor_id +LTRIM( T5.stor_address )))))+ LOWER((((";z^~tO5I" +"" )+("X3FN=" +(REVERSE((RTRIM( LTRIM((("kwU" +"wyn_S@y" )+(REVERSE(( UPPER(LTRIM("u2C[" ))+T4.title_id ))+( RTRIM(("s" +"1X" ))+ UPPER((REVERSE(T3.address )+T5.stor_name )))))))+ "6CRtdD" ))+"j?]=k" )))+T3.phone ))), T5.city, T5.stor_address ) --EOQ ORDER BY 1, 6, 5 )

This statement yields an error (now fixed)

Always UP: System pairsAlways UP: System pairs Two clusters at two sitesTwo clusters at two sites Changes from one Changes from one

sent to othersent to other When one site failsWhen one site fails

other provides serviceother provides service MasksMasks

Hardware/Software faultsHardware/Software faults Operations tasks (reorganize, upgrade Operations tasks (reorganize, upgrade

movemove Environmental faults (power fail)Environmental faults (power fail)

OutlineOutline

Highlights of MS Highlights of MS ResearchResearch

DB-Related ResearchDB-Related Research