Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos
-
Upload
syawal -
Category
Technology
-
view
1.340 -
download
0
Transcript of Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos
![Page 1: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/1.jpg)
ANALYST SUMMIT 2007
Dr. Greg PapadopoulosExecutive Vice President, R&DChief Technology Officer
Redshift: The Explosion ofMassive Scale Systems
![Page 2: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/2.jpg)
These slides contain forward-looking statements regardingthe future results and performance of Sun Microsystems, Inc., including statements regarding future industry trends and growth, demand for HPC and future trends in computing. These forward-looking statements involve risks and uncertainties, and actual results could differ materially from those contained in these forward-looking statements. Factors that could cause actual results to differ materially from those contained in these forward-looking statements include: risks associated with developing, designing, manufacturing and distributing new products; lack of success in technological advancements; pricing pressures; lack of customer acceptance of new products; the possibility of errors or defects in new products; competition; adverse business conditions; failure to retain key employees; the cancellation or delay of projects; our reliance on single-source suppliers; risks associated with our ability to purchase a sufficient amount of components to meet demand; inventory risks; risks associated with our international customers and operations; delays in product development or customer acceptance and implementation of new products and technologies; our dependence on significant customers and specific industries; and our dependence on channel partners. Please also refer to Sun's periodic reports that are filed with the Securities and Exchange Commission, including Sun's annual report on Form 10-K for the fiscal year ended June 30, 2006 and its quarterly report on Form 10-Q for the fiscal quarter ended October 1, 2006. Sun assumes no obligation to, and currently does not intend to, update these forward-looking statements.
![Page 3: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/3.jpg)
25 Yrs: 100,000 to 1M Times More MIPS/$
1880 19201900 1940 1960 1980 2000 202010-15
10-10
10-5
100
105
1010
Year
MIPS
/1000
USD
0.01 MIPS/$1K
10,000MIPS/$1K
Hermann Brunner, Max-Planck-Institut fuer Extraterrestriche Physik, Germany
![Page 4: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/4.jpg)
Where is the Demand?
1985 1995 2005 20151990 2000 2010.1
1
10
100
1K
10K
1M
World GDP
ProcessorPerformance
$36T$17T
Sun 11 MIPS
$40K (2005$)
T100010,000 MIPS$5K (2005$)
![Page 5: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/5.jpg)
Core per Enterprise Demand
1000x
.1
1
10
100
1K
10K
100K
1M
20101995 2000 2005
Core Enterprise Demand
Moore’s Law
![Page 6: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/6.jpg)
“Last Mile” Bandwidth
1K
10K
100K
1M
10M
100M
1G
10G
1985 1995 2005 20151990 2000 2010
9.6kbps14.4k
28.8k
56k 384k1.55m
4m
22m50–100m
POTS
DSL/Cable
Fiber
![Page 7: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/7.jpg)
ΣBW – Filling the Pipe
NingΣBW
In = Out
![Page 8: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/8.jpg)
ΣBW – Filling the Pipe
NingΣBW
In = Out
# Servers * BW/Server = # Devices * BW/Device
![Page 9: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/9.jpg)
ΣBW – Filling the Pipe
.1
1
10
100
1K
10K
100K
1M
1995 2000 2005 2010
1,0001,000
50
1M ΣBW
Moore’s Law
Core
![Page 10: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/10.jpg)
HPC – Insatiable Demand
![Page 11: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/11.jpg)
HPC – Insatiable Demand
1
10
100
1K
10K
100K
1M
1995 2000 2005 2010.1
HPCHPC
Moore’s Law
Core
ΣBW
![Page 12: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/12.jpg)
*-Prise
Internet
WorkdayNetwork.com
![Page 13: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/13.jpg)
*Prise
Moore’s Law
*-Prise
1
10
100
1K
10K
100K
1M
1995 2000 2005 2010.1
ΣBW
CRM Core
HPC
![Page 14: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/14.jpg)
Redshift – a Move to Massive Scale
1
10
100
1K
10K
100K
1M
1995 2000 2005 2010.1
*Prise
Moore’s Law
ΣBW
Core
HPC
![Page 15: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/15.jpg)
Diverging Concerns
1
10
100
1K
10K
100K
1M
1995 2000 2005 2010.1
*Prise
Moore’s Law
ΣBW
CoreConsolidation
CostServices
HPC
![Page 16: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/16.jpg)
Diverging Concerns
1
10
100
1K
10K
100K
1M
1995 2000 2005 2010.1
ScaleEfficiencyProducts *Prise
HPCHPC
Moore’s Law
ΣBW
CoreConsolidation
CostServices
![Page 17: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/17.jpg)
“The Google Question”
If all you cared about was massive scale, what would you build and how would you operate it?
![Page 18: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/18.jpg)
An Answer
With “Brutal Efficiency”• Hardened metrics (utilization, power, security)• Predictability of service level• Idea-to-deploy time and productivity
![Page 19: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/19.jpg)
An Answer
With “Brutal Efficiency”• Hardened metrics (utilization, power, security)• Predictability of service level• Idea-to-deploy time and productivity
Deep, At-scale Engineering of Systems
![Page 20: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/20.jpg)
How Deep?
Efficiency and predictability at massive scale are as
mission-criticalto Redshift as
RAS has been tothe core enterprise
![Page 21: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/21.jpg)
Don’t Confuse
Commoditization ofComputers
![Page 22: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/22.jpg)
Commoditization ofComputing
Don’t Confuse
Commoditization ofComputers
With
![Page 23: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/23.jpg)
System Includes...
Base HW Plant(Servers, Storage and Switches)
O/S Instances
Network Service
Core Services and Platforms
![Page 24: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/24.jpg)
System Includes...
Base HW Plant(Servers, Storage and Switches)
O/S Instances
Network Service
Core Services and PlatformsMassivelyParallelSystem
![Page 25: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/25.jpg)
SMPs Are Back (With a Vengeance!)
64 Threads in 1997E10K
SizeFull RackPower
(Systems at peak utilization)9620 Watts
Weight1,800 lbs.Performance~150k tpm
![Page 26: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/26.jpg)
SMPs Are Back (With a Vengeance!)
z
64 Threads in 1997E10K
64 Threads in 2007Niagara 2
SizeFull RackPower
(Systems at peak utilization)9620 Watts
Weight1,800 lbs.Performance~150k tpm
![Page 27: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/27.jpg)
SMPs Are Back (With a Vengeance!)
z
64 Threads in 1997E10K
64 Threads in 2007Niagara 2
SizeFull Rack 1UPower
(Systems at peak utilization)9620 Watts 410 Watts
Weight1,800 lbs. 45 lbs.Performance~150k tpm ~300k tpm
24x
40x
30x
2x
![Page 28: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/28.jpg)
SMPs Are Back (With a Vengeance!)
z
64 Threads in 1997E10K
64 Threads in 2007Niagara 2
![Page 29: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/29.jpg)
Neptune and Crossbow
• Solaris virtual networking stacksfor high speed networks, based on protocol, service, or container
• Does the “careful” unleashing of 10G
• A shared, virtualized, non-blocking,multi-homed 10 Gigabit EthernetPCI-e network interface
• Gets the most out of your systemand your I/O bus> Processors get faster faster than I/O buses get
faster – PCI-e is (or soon will be) your bottleneck
Crossbow
Neptune
![Page 30: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/30.jpg)
Neptune and Crossbow
SolarisCrossbow and PEF
MultiCore Microprocessor
PCIe4x10GE
Virtualized Network Streams
1000’s Logical Connections
Neptune
Solaris Zones Processes
![Page 31: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/31.jpg)
Java RTS + Solaris =Open Real Time Operating System
Application
Java RTS 2.0
Solaris
Hardware
![Page 32: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/32.jpg)
Java RTS + Solaris:1000 Times More Predictable
20microseconds
10milliseconds
12milliseconds
10microseconds
5milliseconds
7milliseconds
Maximum Latency Jitter
RTS
![Page 33: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/33.jpg)
Real-Time Application Server ResultsComparison of Conventional Java SE to Java RTS
Distribution of Request/Reply Round-trip Times
0 200,000 400,000 600,000 800,000
10,000
1,000
100
10
1
Round-trip Time (US)
Java RTS Results: ALL priority transactionscomplete in 11 milliseconds or lessStandard Java SE Results: Prioritytransactions range from 11 ms to 3.5 seconds
Numb
er of
Req
uests
![Page 34: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/34.jpg)
Real-Time Javaon Solaris:
Inverted PendulumGreg Bollella
Distinguished Engineer,Sun Microsystems
![Page 35: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/35.jpg)
NetBeans IDE Support
• Cross-develop onthe host
• Deploy over the network
• Execute on the target• ...from the
NetBeans IDE
Download the JavaRTS Plugin, and
![Page 36: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/36.jpg)
Co-Design
Computer+ Storage+ Network+ Power+ Cooling+ Software= BLACKBOX
![Page 37: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/37.jpg)
But Wait, There’s More• The next wave will include leading-edge enterprises• Critical to this wave will be the bridge between
core IT and redshifting services:
Identity and securityProcedural languages and scripting
SOA and Web 2.0New clients
![Page 38: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/38.jpg)
The Bridge
1
10
100
1K
10K
100K
1M
1995 2000 2010.1
*PriseHPCHPC
ΣBW
IdentityRSWSNew
Clients
![Page 39: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/39.jpg)
Enterprise Bridge
Internet
Workday
CoreEnterprise
Network.com
![Page 40: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/40.jpg)
Putting it all Together:The Binary Distribution
Base HW Plant(Servers, Storage and Switches)
O/S Instances
Network Service
Core Services and Platforms“TheDistro”
![Page 41: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/41.jpg)
Source vs. BinaryPlatform Developers
Platform(Open Source)
open Javaopen ...
![Page 42: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/42.jpg)
Source vs. BinaryPlatform Developers
Platform(Open Source)
1.1 1.2
Distributionopen Javaopen ...
1.2.1
...Binary Distribution
![Page 43: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/43.jpg)
Source vs. BinaryPlatform Developers Application Developers
Platform(Open Source)
Applications
1.1 1.2
DistributionApplication Binaries
Binary Distribution
open Javaopen ...
1.2.1
...
![Page 44: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/44.jpg)
Big Computers100Ks – 1Ms NodesGlobally Distributed
Devices
A Likely Scenario
Live.comGCC
![Page 45: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/45.jpg)
Big Computers100Ks – 1Ms NodesGlobally Distributed
Distribution Networks
Devices
A Likely Scenario
Live.comGCC
![Page 46: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/46.jpg)
Big Computers100Ks – 1Ms NodesGlobally Distributed
Distribution Networks
Startups
Devices
A Likely Scenario
Live.comGCC
![Page 47: Redshift: The Explosion of Massive Scale Systems by Dr. Greg Papadopoulos](https://reader033.fdocuments.net/reader033/viewer/2022052619/557022c6d8b42aa8558b4b8f/html5/thumbnails/47.jpg)
Thank You.
ANALYST SUMMIT 2007
The Network is the Computer.TM
Dr. Greg PapadopoulosExecutive Vice President, R&DChief Technology Officer