Virtualisation has set IT workloads free from the constraints of their physical environment, promising a more fluid and elastic future for provision of IT services. But can Australia’s physical assets keep pace with the change? Brett Winterford talks to the 10 executives investing in Australia’s next generation of data centres to find out

After several years of relative stagnation, the brief moment of relief between the Global

Financial Crisis and the 2011-12 credit crunch featured a raft of data centre builds and refurbishment projects across Australia.

NextDC launched on the national stage with a stock market funded network of facilities under construction. US-based giant Digital Realty announced a $350 million investment in Sydney and Melbourne facilities. Sydney-based industry giants Global Switch and Equinix pushed ahead with major extensions, and local managed services outfit Macquarie Telecom announced a $60 million investment in a West Sydney facility.

In Canberra, the first trickle of Government agencies committed to space in the modular Canberra Data Centre and similar builds were announced in Melbourne

Towards an elastic future

iTnews Power PaneliTnews Power Panel

(Metronode) and Adelaide (Tier5). Space became available in the massive Polaris development in Queensland, and Telstra announced it would consolidate into two large facilities in Australia’s two biggest cities. IT service providers HP (Sydney) and Fujitsu (Perth and Sydney) also announced new builds.

A simple explanation for this recent spate of activity is that existing facilities are close to full – Global Switch has been close to capacity for 18 months, while Telstra’s general manager of data centres Jon Curry reports the company has been “running at 400 miles an hour laying data centre space just in time for equipment to arrive.”

Capital to build these assets continues to be constrained, but some business cases got through on the back of plans by the Federal Government and NSW State Government to aggregate their many disparate server rooms into larger, more efficient facilities.

But these factors couldn’t alone account for the explosion in activity. There is plenty of evidence that the builds are not simply a temporary backfill of pent up demand, and nor is it the case that any old data centre space would do. Many of the previous generation of data centres lie dormant while many of the new

builds and extensions are already close to full. There are clearly other forces at play.

Demand driversSome of the increased demand has come from unexpected quarters – from the boardrooms of Australia’s corporate dinosaurs. Even the most staunch bricks and mortar retailers have had online epiphanies.

Glenn Gore, CTO at Melbourne IT, describes it as the ‘Harvey Effect’. If old Gerry Harvey is prepared to

finally bite the bullet, the argument goes, “it’s a very game board that doesn’t look at online initiatives.”

“The digital media folks drank the kool-aid for five to 10 years,” remarks Aidan Tudehope, managing director of Macquarie Hosting. “It’s now traditional businesses finally getting onboard.”

Simultaneous with this uptick in demand for online services has arrived a fundamental shift in the nature of demand.

Much as he tried to avoid saying the ‘cloud’ word, Frost and Sullivan research director Andrew Milroy attempted to summarise the change: “There is a move away from distributed IT systems towards more data centre-centric systems,” he said. “People are interested in accessing IT resources from remote locations using browsers and apps.

“In other words, dare I say it, cloud computing.”

More workloads are shifting onto servers, with a gluttonous amount of data shifting to network storage, and a simply staggering amount of traffic swamping data networks.

Social media – often in combination with mass media – is meanwhile conspiring to create unpredictable patterns in consumer online behaviour.

“Traffic spikes can be an order of magnitude over their daily peaks,” Gore explains. “Two years ago, it would have been rare for an internet facing customer running an online service to be bursting above

100Mbps. Today we have multiple customers that burst well into the gigabits-per-second.

“That’s just due to the number of people accessing these sites in a very short period of time when Twitter, other social media or even traditional media drive people to the sites.”

Organisations have looked to virtualisation as a means of more readily allocating and managing IT resources to cope with this elasticity in demand. And it’s virtualisation – in turn – that is starting to have a marked impact on investments in data centres.

One of the stats that brought it home for Ian Whiting, head of global sales at networking vendor Brocade, was learning that “ from the hours of 5 and 6pm on the East Coast of the United States, [online movie rental company] NetFlix represents 40 percent of internet traffic in the United States”.

“That kind of activity has broken the IT model we grew up with,” he says. “We once had a customer that purchased a set amount of gear via the traditional procurement and build model. As a vendor we’re seeing a fundamental shift from a customer-vendor relationship towards an inexorable move to some form of outsourcing of IT. You can call it cloud, call it co-lo, managed service, hosting service, whatever.”

Damon ReidManaging Director, Sydney

Global Switch

Jon CurryGeneral Manager, managed data centres Telstra

The facility: Global Switch is Australia’s largest data centre, with in excess of 24,000 square metres of operational raised floor space in Ultimo, Sydney powered by 22MW of IT-available UPS capacity. The extention to the existing facility, Sydney East, will add a further14,000 square metres of raised floor space and another 22MW of available UPS capacity. iTnews has reported that Global Switch customers include NBN Co and The Department of Defence, among many others.Key differentiator: Global Switch’s key differentiators include its financial strength, wholesale business model, resilience, carrier choice and neutrality and its low-latency CBD location.

Reid started with Global Switch as operations manager before moving across to the commercial role responsible for Asia Pacific. These roles gave Damon a wealth of knowledge on Australia’s largest users of data centre space as well as the requirements of running the largest data centre in Australia. At the beginning of 2011 Damon took over the role of Managing Director, Sydney and since that time has seen the facility let to near full capacity and been intimately involved in the planning and development of the Sydney east extension.

The facility: Australia’s telco incumbent has over 25,000 square metres of data centre space used for internal purposes (BigPond etc) and the provision of hosting, managed IT services and cloud compute to enterprise and government customers.Jon’s team is currently building a 6MW facility in Melbourne adjoining an existing 5MW facility to help consolidate legacy data centres spread around the nation.iTnews has reported that Telstra cloud customers include industrial giants Visy and Kohmatsu among others.Key differentiator: Telstra operates Australia’s largest terrestrial and mobile networks.

Curry has an impressive legacy of infrastructure project and management roles at the likes of Telecom NZ, NAB, Coca-Cola Amatil and Allianz.

Every tenth rack or so is coming in at 10 to 20 kW. You can’t run those kinds of racks in the older data centres

I think we will move to a stage where the IT load and the facility itself – the power and cooling – will work together

What [a stock exchange ecosystem] needs is a network fabric that is quick because latency means millions and millions of dollars

The facility: Equinix is a global company with over 100 data centres in 38 countries, its three data centres in Sydney grossing over 30,000 square metres with a new phase soon to come online in the Sydney3 facility. iTnews has reported that Equinix customers include the Chi-X stock exchange, cloud services provider OrionVM and Cricial Paradigm, among many others.Key differentiator: The key differentiator for Equinix is to connect ecosystems together – be that a trading platform with the systems of its stockbroking customers, or a cloud computing provider with its enterprise customers.Simonsen takes over at Equinix Australia following a long career in technology sales and services for the likes of EMC and Cisco Systems.

Tony SimonsenManaging Director

Equinix Australia

“The demand is massive at the moment,” agrees Telstra’s Jon Curry. “We’re just keeping ahead of the curve both in terms of laying down enough good quality data centre space with enough power, but also keeping up with demands on the network and demands on the compute and storage fabric.”

Capacity planning has become a full-time concern, he said, to ensure facilities are built ahead of this demand curve. “There is literally not a day that goes by where we aren’t talking about what’s next – how do we provision it? What do we recycle? What can we get back from another room? How do we put more power into a building?”

Pressure pointsClearly the trend puts a lot more strain on networks. Whiting reports that Brocade is doing an “exponential” amount of new business through service providers versus traditional enterprise customers.

Networking suppliers are offering “flatter” networks to customers, cutting out the aggregation layer, in some cases moving to consolidated stacks of server, storage and network infrastructure. Equinix, for example, is eyeing an opportunity to host entire “ecosystems” of users, such as customers taking up space close to a key cloud service

provider, or stockbroking firms co-locating their systems as close as possible to a stock exchange system.

“What that needs is a network fabric that is quick, because any network latency can cost many millions of dollars,” noted Tony Simonsen, the newly- hired managing director at Equinix Australia.

The data centre and network is further impacted by the tendency for cloud applications to create far more storage than traditional enterprise workloads.

“If you look at new modules we are bringing online, there is about three racks of storage for every rack of compute,” said Marty Gauvin, CEO of modular data centre builder Tier5. "Storage requirements were already growing at 60 percent year on year – and that’s helped by our cloud friends."

Malcolm RoeGeneral ManagerMetronode

Leighton-owned Metronode data centres is investing in a national network of data centres – with five existing 10,000 square metre facilities in Brisbane, Sydney, Melbourne, Adelaide and Perth plus plans or construction underway for four new facilities in Melbourne, Sydney, Canberra and regional NSW. Metronode’s latest build in Melbourne is a 3MW modular fit-out on a 12MW site – the first in the world to stack modular BladeRoom data halls and free air cooling units on top of each other. The facility will only need to be cooled by chillers for two percent of the year.Key differentiator: National network, uptime and low power consumption. Metronode claims to have never lost a single minute in downtime in nine years.

Roe has worked in the telco divisions of Leighton Holdings for some 11 years, but also has experience in management and sales with networking vendor Alcatel.

For every megawatt of IT load, a modern data centre could save you a million dollars at today’s prices

iTnews Power PaneliTnews Power PaneliTnews Power Panel

www.crn.com.au April 2012 37

Xxxxxxxx

Xxxx xxxxxx xxxxxxx xxxx xxxxx xxxx xxx

xxxxxxxx xx xxxxx

xxx xxx

“Cloud environments store things three times,” Gauvin says, storing a file in multiple places to improve performance and redundancy. “I’m expecting to end up with only 1000 racks of servers out of 5000 racks when fully expanded.”

Gore agrees that data storage systems are “outstripping any other infrastructure in the data centre, which requires shifts in data centre design.

“A rack full of storage weighs a hell of a lot more than a rack full of servers or networking gear, so the weight limits on the racks is a massive issue,” Gore says. “And the power density of a storage rack is far, far denser than anything else – even if it’s compared to a server with the highest grade CPUs.”

Spinning disks of storage, he notes, are the least tolerant to failure (due to data corruption risk and longer times to recovery).

“And storage is extremely high-density from a power perspective, and runs very, very hot.”

Damon Reid, managing director at Global Switch’s Sydney facility offered some sums to illustrate this trend towards density.

“When Global Switch first went live back in 2002, the design density was about 400 watts per square metre. It went live at 800 watts per square metre, in 2008 we started rolling out floor space at 1000 watts

per square metre, and our final floor – Level 7 – is coming online at 1500 watts per square metre.

“That’s not because the average density has rocketed up; it’s because every tenth rack or so is coming in at 10 to 20 kW, and needs airflow to extract that heat out of the data centre. You can’t run those kinds of racks in the older data centres that don’t have the latest power and the cooling solutions.”

Thus the density of computing is one major factor driving demand for newer facilities. The latest generation data centres from Metronode, Tier5 and Digital Realty, among others, use techniques such as free air cooling and hot aisle/cold aisle containment to cut down on the cost of powering IT loads with chillers (CRAC or Computer Room Air Conditioning Systems).

“Small inefficient facilities are being retired and aggregated into larger, more efficient facilities,” says Malcolm Roe, general manager of Metronode, the data centre business of construction giant Leightons.

“The technology drivers are virtualisation, the provision of cloud services and demand for consumer services driven by the NBN. These are coming together to start to take geographic dependency out of the equation. One we get a ubiquitous high-speed network, the industry will focus on putting data centres in areas where they are most efficient.”

The need for elasticityThe rising cost of power has become a major concern for data centre operators, an issue likely to be exacerbated once a price on carbon is introduced in July 2012.

Traditionally, CIOs have left power consumption as a cost for facilities managers to concern themselves

Aidan TudehopeManaging Director, hosting

Macquarie Telecom

Macquarie Telecom offers managed hosting services from two of its own facilities in Sydney. The first, IC1, is a 3.5MW, DSD-certified facility with clients including the Department of Prime Minister and Cabinet, WebJet and Quickflix. The second is a $60 million 8MW facility due to open within the next few months in Macquarie Park with News Limited named as an anchor tenant.

Key differentiator: Managed services and power efficiency.As an organisation, Macquarie Telecom’s focus has traditionally been around superior customer service. But the new Macquarie Park provides a new opportunity: it is the first design-certified facility in South-East Asia to be certified Tier III by the Uptime Institute, pointing to a new standard of operational and power efficiency.

Tudehope has managed the technology operations of Macquarie Telecom for close to 20 years.

The NBN rollout is effectively a $30 billion shot in the arm for the IT industry. The distinction between WAN and LAN will be removed when the NBN arrives at your office

Greg BoorerManaging Director

Canberra Data Centres

Canberra Data Centres, in operation for the last three years, features two six megawatt data centres – the latest of which was opened in December 2011. iTnews has noted a number of large Federal Government agencies moving workloads to the facility, most notably the Department of Human Services and CrimTrac.

Key differentiator: Local, trusted relationships.CDC is among the few AGIMO-approved enterprise grade data centres serving the Federal Government market in Canberra.

The Carbon Tax will be tremendous for people that invested in good quality, purpose-built data centres, because you’d be an absolute lunatic to keep a server room inside a standard building

with. Mike Andrea, director of data centre consultancy Strategic Directions, says he is consistently surprised at the lack of knowledge within senior IT ranks when it comes to power consumption and other facilities costs.

“They often don’t know how much it costs to run, no idea of their maintenance costs or asset lifecycle, or costs of other utilities like water or diesel,” he says. “They don’t wrap it as a data centre budget, it's split off in IT budgets and facilities budgets. It can take months of investigation to get those numbers.”

At the big end of town, he reports, boards charged with sustainability are “giving IT managers a kick in the rear for not keeping control of it”, but medium-sized organisations don’t have the relevant structures or systems in place to monitor the situation. “Power used to be below sundries and stationary and other items,” noted Tudehope, “but it is creeping up. CFOs are now starting to ask the question.”

Roe reports larger IT providers and end-users are starting to demand “power efficiency commitments” from data centre providers. But power efficiency is inexorably tied to the location of the data centre. At any given hour of the day, a workload is likely to be more expensive to run in one city over another. A data centre running

on peak electricity prices during the day time in a hot summer climate, where air conditioning is required, is far less efficient than a data centre running in a cooler climate using off-peak electricity at night.

Customers and managed service providers are seeking a future in which data centres and telcos offer a more elastic approach to networks and power consumption to meet the unpredictable nature of today’s IT workloads.

“You can’t run a facility for 24 hours, flat out, to keep IT equipment

cool 24 hours a day that doesn’t need it,” Curry said. “At night most of the workloads drop, but you’re still cooling equipment that’s not really doing anything.

“I hope we will move to a stage where the IT load and the facility itself will work together. As the IT load increases, you’ll see the network, storage or compute respond, but also an increase in cooling, be it chilled water or whatever the IT load needs to keep it running. Our facilities will have to become elastic.”

Curry expects networking, storage and cooling technology to mature to the point in which a national network operator like Telstra can move IT workloads to the data centre most appropriate at any given point in time, regardless of what city it is in.

“This new 6MW facility I’m building in Melbourne is 100 percent free-air cooled,” Curry says. ”So we are looking for at least 70-75 percent of the year running on free air. Now if I can move my workloads from Sydney down to Melbourne when the spot price for power in Sydney is high, that’s where we really want to be. But there are so many legacy barriers to doing that.”

Gore – a fellow managed service provider – loves the idea of spot prices. Service providers like Amazon and Google have pioneered infrastructure stacks that – once the enterprise world follows

Glenn GoreChief Technology Officer

Melbourne IT

The facility: Melbourne IT’s domain name and web hosting operations manage over 650KW of IT load from 770 square metres of data centre space leased in three locations around Australia.

Marquee customers include the Queensland Department of Education & Training, Smart Service Queensland and Credit Union Australia.

Gore has managed the company’s IT engineering for some seven years after similar roles at MCI WorldCom and Ozemail. Today he manages the R&D programs and strategic direction of the company.

Key Differentiator: Melbourne IT, after acquiring the WebCentral hosting business, is now moving up the value chain from being a pure-play infrastructure provider to a service provider focused on the marketing outcome of running an online website.

Traffic spikes can be an order of magnitude over their daily peaks. Today we have multiple customers that burst well into the Gbps

The NBN is creating a significant build of data centre capacity outside of Sydney for the first time in Australia’s history

The facility:Tier5 is gradually building out an 8MW facility in Adelaide based on Dell’s third generation of modular data centre, offering wholesale access (over 1MW) to aggregators and hosting companies.

Key differentiator: Modularity and power efficiency. The modular approach allows Tier5 to only add plant as customers come on board to take up compute load.Gauvin founded and eventually exited one of Australia’s largest enterprise hosting companies, Hostworks, before founding Tier5.

Marty GauvinChief Executive Officer

Tier5

iTnews Power PaneliTnews Power Panel

iTnews Power PaneliTnews Power Panel

Xxxxxxx

Xxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

suit – require more robust networks and new usage-based pricing models, he said.

While traditional enterprise workloads still tend to remain tied to a blade server chassis and a SAN (storage area network), Gore notes internet companies are using highly distributed networks of servers connected via clustered file systems – systems in which a SAN lacks performance and proves expensive.

“The new world order is going back to very cheap, commodity pizza boxes in the racks, with as much CPU grunt as you can fit and as much local attached storage as you possibly can – 12 or 14 spindles per 2RU server – and with a clustered file system running over the top of it,” he says.

“If one of those nodes fails, who cares, I’ve got it replicated at least three to five times on other boxes on other racks in other data centres. It’s a completely different way of doing IT and a completely different cost model. The way you look at the data centre has to be totally different.Under that architecture you can consider spot pricing on power. Because if you needed to shut down power to a rack, it’s less of a problem.”

Gore says 90 percent of the workloads he runs for customers

stick to the traditional IT stack, but 70 percent of new business coming online demands this elasticity. “If we look at where we want to be in one or two years, about 50 percent of the content we store will be on that new model,” he said.

It is undoubtedly a huge challenge and opportunity for the vendor community. Virtual machines may have set workloads free within the data centre, but true mobility of data and applications over long distances requires new approaches to networks and storage.

Whiting reports Brocade and other networking vendors are working on ways to move workloads and applications in and out of a hybrid cloud environment. “There is actual real meat behind that concept of a public/private hybrid cloud,” he says. “But we are at stage three of 10 of fixing that problem from a technology point of view.

In the meantime, Brocade is among a small number of “challenger” vendors adjusting models to adapt to this new world. Service providers can, for example, lease switches from the vendor and only pay for ports in use to save on upfront capital investment.

Gore noted that there “hasn’t been good partnerships” to date between the hosting and the data centre

communities to align around the provision of more elastic services.

Reid said Global Switch has been “grappling” with the concept of spot pricing of power “for a year or two,” but doesn’t have any answers just yet.

“It’s not a simple solution,” he said. “If you want spot usage, you need to build more capacity. And how do you contract that capacity so that across dozens of companies you can still guarantee provision of power? People want power and they don’t want to find themselves prioritised below other users.

“Spot pricing is probably something we’ll evolve to over the next four or five years, much like a lot of discussions around cloud standards. It’s something that is so embryonic at the moment; it’s only a discussion point. We haven’t got to the maturity on the infrastructure side to have a solution for it.”

Primarily this is because the model behind financing a new data centre build still relies today on an anchor tenant.

“We are a conservative organisation that has seen how the data centre industry has built out over the years,” Reid said. “We just won’t commit to a new building unless there is a guarantee of

Digital Realty owns and leases 19 million square feet of data centre space across 102 global data centres, turns over U$1.1 billion annually and has a market capitalisation of US$8 billion. The company set up its first Australian office in May 2011 and is spending $350 million building four facilities in Australia on two campuses in Sydney and Melbourne.

The Sydney facility will offer 8000 square metres raised floor across two buildings and 11.2 MW of power from October 2012. The Melbourne facility is 6000 square metres across two buildings and 8.64 MW of capacity. The first has been fully leased to National Australia Bank – the second will offer 5.76 MW of capacity from from Q1 2013.

McLaren has responsibility for Digital Realty’s sales in Australia and has previously worked in the telecommunications sector.

Key differentiator: Digital Realty’s primary differentiator is using its scale and operational expense model to reduce the risk of data centre ownership and deployment for enterprise customers and service providers.

There has been a lack of investment in data centre infrastructure during the period of time the financial environment has been constrained, there hasn’t been the investment in new facilities it could have been

Andrew McLarenSales Director

Digital Realty

Mike AndreaDirector

Strategic Directions

Mike Andrea is the brains behind Strategic Directions, a firm hired by enterprise customers to make decisions on whether to refurbish their facility, consume co-location services or adopt a managed service agreement for their infrastructure needs. The organisation is also involved in large data centre build projects such as the Polaris Data Centre in regional Queensland.

Key differentiator: Industry-wide knowledge.

Andrea was the first non-US citizen appointed a board member for the AFCOM Data Center Institute.

The Australian IT sector still wants to see the lights and touch the buttons, whereas you go to Asia they don’t care if the primary data centre is nine hours away

revenue to match the expense. And it’s not just to build it – it’s to run it over 20 to 30 year life of the facility.”

Indeed, Digital Realty’s entry into the Melbourne market was no doubt spurred on by a deal with National Australia Bank as anchor tenant, as was Bankwest for Fujitsu’s Perth data centre or Westpac for its West Sydney facility.

Roe notes that even with the financing muscle of Leighton Group behind Metronode, the company seeks out Triple-A rated, long-term tenants to build a new facility.

Greg Boorer, managing director of Canberra Data Centres, put it most bluntly.

“Partnering is an interesting term,” he said, responding to Gore’s proposition. “I’ve had discussions on being a great partner with a research organisation in Australia. They said they wanted a facility of the scale of 20 MW of power, they wanted 2000 square metres for a supercomputer, and they wanted to start in the middle of the data centre floor and slowly build out in a circular motion,” he said. “But they said they’ll only use and only want to pay for 150 KW on day one, but still need 20MW available.

“Partnerships are where you commit to me for a period of time, and I’ll commit to you to meet your changing requirements. As long as you can put that in some kind of scope, I will bend over backwards to help you, that’s what a partnering is all about. But remember that a

half a MW customer doesn’t build a data centre. Data centres need a significant anchor tenant.”

Andrea noted that debt financing for new builds is as constricted today as it ever was. Five years ago, data centres could be built with a 70/30 ratio of debt to equity (data centre builders would need only 40-40 percent of the cash upfront). Today banks are seeking 50 percent capital upfront.

Banks see data centres, Boorer noted, as an “incredibly risky form of property development” until the

facility is in operation, after which “they will give you more money than you can poke a stick at.”

“It’s either they won’t lend you $100 million, or they’ll say $100 million isn’t big enough!” he joked.

This means that for now, data centre builders require anchor tenants signing on to multi-year leases. A “managed services agreement” won’t cut it, Andrea noted.

“An MSA has break events that are unacceptable to a bank,” he said.

It again points to the reason the likes of Global Switch and Equinix are building extensions on existing land, or why Tier5, Metronode and CDC are building out in a modular fashion.

“Data centers are hard on left hand of the value chain when it comes to capital allocation,” Metronode’s Roe comments. “Being able to efficiently deploy your capital is going to be key to this business.”

It would thus take a brave entrepreneur to risk changing the status quo and opting for a more elastic ‘spot pricing’ model.

And so a gap appears to be widening between the pace of technology development and the financial model used to finance new data centre developments.

The dream of elastic workloads might be some years off yet.