Cisco Systems new Data Centre 3.0 strategy not only comes with a new series of data centre switches that push switching capacity inside the data centre to the terabit level, but also key “pillars” that address both energy and operational efficiency.

In this feature interview, Green Telecom talks to Cisco vice president and general manager of the data centre business unit, Rajiv Ramaswami, and Bernie Trudel, head of technology, security and ANS for Cisco AP, on the different aspects of next generation data centres as well as the challenges of handling ever-increasing data volumes on networks and data centres.

Green Telecom: So what’s green about Cisco’s Data Centre 3.0 strategy?

Rajiv Ramaswami: If you look at Data Centre 3.0, there’re basically three key pillars to it: One is consolidation; second is virtualization; and third is automation. Each of those actually has a significant energy, or green, component to it. If you look at consolidation – consolidation is about combining multiple units, getting rid of lots of data centres and consolidating them into a few data centres. Within the data centre, consolidation is combining multiple islands into a single network, or single infrastructure. Each of those steps serves, in itself, a significant energy reduction. So for example, we were with a customer – and this is storage consolidation – so what happens was they had separate storage networks for different applications and they had about a hundred storage switches to plug. We have a product called VSAN, or virtual SAN, which allows you to take a storage switch and partition it into multiple islands. Using that, they can actually pull all these islands together into a single island. They replaced 100 switches with 24 switches – significant energy reduction.

Another example – we’ve talked about something called unified I/O, or unified fabric in the Nexus 7000 series, it’s clear that the first thing there is consolidating the I/Os coming out of servers. If you look at power consumption today, roughly 15% of the power is consumed by the network in the data centre – and that includes all the adaptors in the network, whether they are network interface cards or adaptors in the servers. Now if you look at a typical server desk, there’s anywhere between seven to nine I/O interfaces on that server, consuming about 20watts of power each. So, there’s a significant opportunity there to cut those costs as well as energy by taking the seven or nine interfaces and replacing them with two, consolidated, higher speed interfaces. The kind of power efficiency that you can get by doing that is about an 8% power reduction for the entire data centre. So that’s another example of how the network can help reduce energy by consolidating.

Bernie Trudel: Consolidating what we call network services on the server.

RR: Beyond consolidation, let’s talk about virtualization. When people talk about virtualization, they often talk about server virtualization to start with, but there’s server virtualization, there’s storage virtualization, there’s network virtualization – and really, all three go hand in hand. The principal benefit – I mean if you look at the immediate benefit of virtualization – it is to improve the efficiency, or the utilization of your assets, whether its storage, servers or networks. Typically, take an example today, most environments, most enterprises, their server utilization runs at 15%. If you can virtualize your servers, you essentially increase utilization on those servers. In the case of Cisco IT, we went from 15% to 60% after virtualization. What that means effectively is we can actually do with four times fewer servers. That reduces the number of servers by a huge factor, a factor of four.

The same goes for storage as well. As you virtualize your storage, you won’t need as many storage areas.

Again, the network is a key enabler of this. So you can virtualize servers, but you also want your network to be aware that now your infrastructure is virtualized. So the network now needs to be aware of virtual machines. Once you have a virtual machine, you might want to move around, for example, time of day, so during the day you have a heavy work load, so you have all your servers on, but during the night you might want to power down half your servers, so you might want to migrate your virtual machines from one set of servers to another set of servers. When that migration happens, the network actually has to help the migration, so it is seamless and doesn’t impact services. But again, once you have virtualization, once you are able to make use of virtual machine mobility, you can get significant energy reduction. That is the second pillar.

The third piece is automation. In order for things like virtual machine mobility to happen, you need automation in the network. So when you move your virtual machines from here to there, and turn down the servers, tomorrow morning, those servers need to be up again and the machine needs to be migrated back. So at that point, you need automation to make sure that all of this happens. So you cannot really get the benefit of virtualization without having the automation behind it. So it’s really all three of those coming together that help energy efficiency.

Now, at a micro level, in terms of technology, there are design principles that we are looking at. One is: more and more we are looking at designing for average power consumption, not peak power consumption. It is very simple to think about, whenever people designing their data centres, they always size for peak workload. Most of the time, you don’t have the peak workload, you are actually wasting a lot of idle power sitting there.

GT: So what happens when you have peak workload?

RR: You still have to design for that, but the point is that when things are not being used, you can shut them down. So yes, when you have peak power, you have to make sure that you have the power to support it. But when you are not using it, you can shut it down. So during normal operation, the power consumed will be a lot less. The system can be smart enough to detect, to power off ports, power off sites, to run at a lower speed for example. Let’s say you have a 10GB link – 10 Gigabit Ethernet might take a few watts of power, but when that link is not utilized, you might think about running at 1 Gigabit, and automatically rolling it back up to 10 Gigabit if your utilization goes up.

Then there is the matter of efficiency. Five years ago, we use to design power suppliers with 80% efficiency. Today, we design power suppliers with 90% plus efficiency. So at the detail engineering level, we are doing a lot of things there too, to try to reduce average energy use.

BT: That pretty much covers the various angles, where we are trying to save energy, to mitigate the use of power.

GT: Besides all the Cisco equipment inside the data centres, is Cisco working with partners and customers on the soft infrastructure of data centres, such as facilities design for example?

RR: We work with partners. In fact, we work a lot with infrastructure cabling, power supply planning and real estate design for data centres. It is more us partnering with them as opposed to us doing things directly. We do our piece and they do their piece and of course, we work together to try and make things more efficiency.

From a customer’s perspective, when they are designing and planning their data centre, they have to figure out where they can get cheap power. I saw an advertisement in the Wall Street Journal about a few months ago. The country of Iceland is promoting the location of data centre there, because you get natural cooling, cheap power. All those things are big and should not be underestimated. I’ve heard statistics that only half of the power of data centres is used for equipment. The other half is for facilities, like cooling. There is a big chunk of that that needs to be optimized. So it has to be planned properly.

BT: As an example of how we are working with some of these facilities operators, we are working with a partner – in terms of defining a best practice of hot iron and cold iron. Therefore, when we design something like the Nexus 7000, we have front to back air flow exactly for that reason because we know that on one side, it’s going to be hot aisle and on the other side, you get cold aisle. So we designed it so air can flow through the box. It is a more efficiency cooling design, which fits in with the best practices of data centre design.

GT: When you talk about features such as a virtualization, the cooling requirements inside a data centre will change depending on the load, or time of day, are you working with systems that can also match those characteristics, on the cooling side, for example?

RR: I don’t think that exists today, but that is the vision. That is part of the whole data centre automation theme. When we talk about data centre automation, it’s not just about our equipment and infrastructure, but the whole facilities as well. If you think about it, we have home automation. You turn your cooling on during the day and turn it down at night, same thing with heating. You want to do the same thing here too. There’s a lot of efficiency to be gain by doing that.

BT: Just to add to that, our control plane product, which has both northbound and southbound interface. The northbound interface can be controlled by a bigger manager and the southbound interface can push out information about the data centre. From the southbound interface perspective, you now have the ability to push out information on what you are doing in the data centre. As part of the automation, this technology is provisioning parts of your data centre. If you are cutting down on what compute power and storage you want to provision – that has the ability to signal other systems. So we have put the hooks in place for that to happen, but are those systems in place to do that? It’s not quite there yet.

GT: What is Cisco doing in the lifecycle management area?

RR: Beyond that, energy efficiency is also about reducing waste. There’s a whole different angle there. And that’s one factor that we take into account when we actually design our equipment. We design our equipment to have a very long life cycle and investment protection. That is very good news for our customers because once you buy the product, you can, for example in the storage space, every few years, bandwidth goes up – same thing with the Internet. So right now, we have a transition for storage from 4Gbps fibre channel to 8Gbps fibre channel. If we look at the products from some of our competition – two years ago, you have to buy the 4Gbps switch. Today, the expectation is, you take that switch and throw it away and buy a new 8Gbps switch. That does that mean? When you throw away a switch, it goes to the garage dump. In our case, what happens is we are shipping the same platform as 2001. It came initially with 2Gbps LAN cards and you add 4Gbps LAN cards to the platform. Now you can add 8Gbps LAN cards to the same platform going forward. That means you don’t have to throw away all your investment, both in terms of dollars, and in terms of waste. All the investments that you have, you don’t have to throw them away, but you can add newer capabilities to the same platform, leveraging the existing investment.

We think about that, and most customers think about that – not only in terms of waste reduction, or recycling, but they think about it as investment protection. That is true with all our platforms. Any of our platforms tend to be, particularly modular platforms, then to be designed for very long lifecycles. If you look at our Catalyst product, it can out in 1999, and it has a rich road map ahead to 2009/2010/. This equipment has been in customers’ networks for 10 years, and it keeps evolving with minimum changes.

GT: One major area of discussion in the industry right now is the explosive growth of data with reports suggesting that if the amount of data on networks continues to grow, there’s no way to handle the data. What is your perspective on that?

RR: You bring up a very good point. What we have going for us is generation of technology. Whether you go from 130 nanometre technology, to 90 nanometers, to 65 and going forward to 45, each time you do that, we can reduce the power consumed per unit of bandwidth delivered. But at the same time, we are fighting the other battle, in that bandwidth is going up.

So the question is what is the product doing, as power per unit of bandwidth goes down, the bandwidth goes up and you multiple the two and the question is net-net – will power consumption continue to go up? The other issue, it is not just absolute amount of power, but power density. It is getting more and more difficult to cool as they keep cramming more and more bandwidth into the same equipment, therefore it runs hotter, so cooling is going to become a challenge as well.

GT: So what is Cisco working on, or what is its vision to tackle this challenge?

RR: There are lots of little things that we are doing to try to reduce the amount of data on the network. For example, in the storage space, we have compression, which reduces traffic on the network. In our wide area application services area, we have WAAS. The idea behind WAAS is, if you have applications running in the data centre with users sitting in branch offices, for example, your Exchange server, which use to run in the branch office – now you put it back into the data centre as part of consolidation – when you do that, you want to make sure that the user in the branch office don’t see any degradation of performance, but you also want to minimize the bandwidth on the WAN link. WAAS is an appliance that sits on either side and can be integrated with the routers and it basically does intelligent processing of the application data to minimize duplication. For example, an email going from here to there and coming back, the application is smart enough to look at the email and say, I don’t have to send this data back and forth, so reducing the amount of data moved around.

BT: For example attachments. When people forward attachments, the attachment gets transmitted back and forth many times.

RR: WAAS is intelligent enough to look at the attachment and store it locally and then when the email comes back, it then attaches it and sends it off locally. Those are the few areas that Cisco is trying to reduce the amount of data moving area.

GT: What is trend now for consolidation in the Asia Pacific region?

BT: What I’ve seen is that in Asia Pacific, we tend to be behind in terms of consolidation of existing data centres. But where Asia Pacific can leap frog, and to a certain degree some of what is happening in the US, is that the infrastructure that we’ve built out, or the number of data centres per population, or however you want to measure it, is a lot less. So therefore, we can skip all of the huge infrastructure that was built out for smaller data centres that were populated in North America and Europe, and skip directly to the mega data centres, or next generation data centres. I see that in China, Vietnam, India, to a lesser degree Australia as it is more mature from an IT perspective, but we are seeing these big build out of data centres, that are not necessarily the consolidation of existing data centres, but most of it is green field. Therefore, they leapfrog that step that happened in North America and Europe.

GT: I have to ask this question. What do you think of the Iceland idea?

RR: I’ll tell you one thing, bandwidth will not be an issue because that is just submarine cable capacity – that is a solvable problem. The question comes to other areas. Power and cooling is one area. Subsidies will be another: Do the government offer the right incentives for people to relocate there. And the last area is labour. Data centres don’t need a large labour pool, but they do a need a small, skilled labour pool, so is qualified labour available? So it’s a combination of all those things.

I don’t know personally of any company that has decided to locate their data centre in Iceland, but if you look at even what is happening in the US, if you look at the large Web providers, Google and Yahoo, they are absolutely locating their data centre in places where they can get cheap space and cheap power and decent connectivity. Those are the three things – that is where they are building it, in the middle of nowhere, not near any big city or anything. So I don’t see why this trend can’t go international.

Facilities planning, in terms of power and cooling, I don’t think it is that advance. Everybody tends to look at the facility and say, we do it this way and we’ll continue to do it this way. I’d love to hear more innovation on how we plan our facilities, like can you really make use of ambient cooling.

I don’t see why Iceland can’t be an candidate if you take a look at all those things – power, cooling, labour, space, subsidies.