An area for operations growth - Storage Area Nets in MANS
Roxane Googin is usually ahead of the competition in spotting trends in data net use. here is what has has to say about enterprise use of MANs. COOK Report: Where are the metro area networks going? Googin: Probably the killer application for the MAN is grid computing and storage area networks. A lot of people think the driver for broadband access to the home is going to be multi-media. People, when they think about next generation networks, think home use. But this is never where the money has been. And no phone company is ever going to build the infrastructure. Although these new real time applica tions will clearly send more data over the network, the real killer application is going to be remote storage and synchronous storage. Synchronous storage means that you have two large servers doing the exact same thing at the exact same time in two different locations. COOK Report: Like a decentralized disk array? Googin: Yes. The backbone has to be incredibly fast because you cannot complete a transaction until you have acknowledgments from both disk drives. This will happen. Probably this year. What they are already doing is taking fiber channel and putting that on a Cienna Core Director optical switch port. Half of the ports being sold on the Core Director now are fiber channel. They aren't even Ethernet. And this is used for storage area nets (SANs). These are corporate MANs and will have nothing to do with sales to service providers. They are bypass business services where the storage arrays may not be more than a kilometer or two apart. These SANs are backing up continuously terabytes of data. We are talking huge applications that will use every bit of access to every bit of capacity they can get. COOK Report: Is 9/11 a motivation for this? Googin: Partly. Not only that but the whole paradigm of the real time organization will drive it. It used to be that your server had its own storage. It was a "stove pipe" connected to the CPU. Now as soon as you decouple that "stove pipe," you can put it anywhere. What they are finding is that if they have two of them that are mirrored in real time and place remotely that they do not have to do as much management of resources. COOK Report: Then forget the disaster back up and recovery operations? Those are the next to be marked for extinction? Googin: Oh yes. It is so elegant. The new architecture cannot be supported on direct attached storage. It must move off the server and it is doing so. If you have to go through a server to get to the storage, it simply doesn't work. Storage can't become just a utility until you move it off the server. The MAN will be backing up terabytes of data regularly and in real time. Disaster recovery, in real time, comes for "free". For more detail please see Fiber & Wireless as First Mile Technology - Fiber Business Models & Architecture http://cookreport.com/12.04-06.shtml -- ============================================================= The COOK Report on Internet, 431 Greenway Ave, Ewing, NJ 08618 USA 609 882-2572 (PSTN) 703 738-6031 (Vonage) cook@cookreport.com Subscription info & prices at http://cookreport.com/subscriptions.shtml Fiber & Wireless as First Mile Technology - Fiber Business Models & Architecture, July- September 2003, 130 pages available at http://cookreport.com/12.04-06.shtml =============================================================
Although these new real time applica tions will clearly send more data over the network, the real killer application is going to be remote storage and synchronous storage. Synchronous storage means that you have two large servers doing the exact same thing at the exact same time in two different locations.
COOK Report: Like a decentralized disk array?
Googin: Yes. The backbone has to be incredibly fast because you cannot complete a transaction until you have acknowledgments from both disk drives. This will happen. Probably this year. What they are already doing is taking fiber channel and putting that on a Cienna Core Director optical switch port. Half of the ports being sold on the Core Director now are fiber channel. They aren't even Ethernet. And this is used for storage area nets (SANs). These are corporate MANs and will have nothing to do with sales to service providers. They are bypass business services where the storage arrays may not be more than a kilometer or two apart. These SANs are backing up continuously terabytes of data. We are talking huge applications that will use every bit of access to every bit of capacity they can get.
Erm - perhaps I'm misunderstanding what Googin is trying to say here, but if he's talking about synchronous remote replication over fibre channel, this exists today. In fact, it has existed for years, and before it was over fibre channel, it was over ESCON. Today, still, if you want to go father than a certain distance (70km maybe?) its generally recommended to switch over to ESCON rather than fibre channel. In practicality, complete remote replication is often inadvisable for high-performance, heavy-write applications. The rule of thumb is that every KM adds another millisecond worth of I/O transaction time, so a few kilometers distance can add a significant overhead to writes. Not to mention Fibre Channel is very unkind when it comes to recovery from fabric segmentation. A poorly designed (or just very unlucky) SAN can be completely downed on both sides of the split, somewhat ruining the disaster recovery strategy if the production and DR storage networks are both taken down. Ugh - I'll be very happy when fibre channel is dead and buried. Thanks, Matt -- Matthew Zito GridApp Systems Email: mzito@gridapp.com Cell: 646-220-3551 Phone: 212-358-8211 x 359
Sorry for any confusion. Fiber channel is old tech for sure and there was no implied intent to evaluate the tech itself. Roxane has apparently observed that on devices like the core directors the number of fiber channel ports being ordered is going up dramatically.... apparently about the only thing it makes sense to do with such ports is to use them for SANs. The remark was not intended to be definitive or in great depth and was added almost as an after thought at the end of a short (35 minute) interview at the san jose airport. Certainly there is major movement in large fiber based in metropolitan areas that have been built and are being built by banks, and fortune 500 kinds of enterprises as they move a LOT of data and voice traffic off the PSTN and on to new cheap equipment that they can get a much better return for on the dollars they invest. All a part of the commoditization and decentralization of telecom. I am not an expert in the application of the technology being discussed. However at some time in the next six month i probably will want to explore this in much more depth.
Although these new real time applica tions will clearly send more data over the network, the real killer application is going to be remote storage and synchronous storage. Synchronous storage means that you have two large servers doing the exact same thing at the exact same time in two different locations.
COOK Report: Like a decentralized disk array?
Googin: Yes. The backbone has to be incredibly fast because you cannot complete a transaction until you have acknowledgments from both disk drives. This will happen. Probably this year. What they are already doing is taking fiber channel and putting that on a Cienna Core Director optical switch port. Half of the ports being sold on the Core Director now are fiber channel. They aren't even Ethernet. And this is used for storage area nets (SANs). These are corporate MANs and will have nothing to do with sales to service providers. They are bypass business services where the storage arrays may not be more than a kilometer or two apart. These SANs are backing up continuously terabytes of data. We are talking huge applications that will use every bit of access to every bit of capacity they can get.
Erm - perhaps I'm misunderstanding what Googin is trying to say here, but if he's talking about synchronous remote replication over fibre channel, this exists today. In fact, it has existed for years, and before it was over fibre channel, it was over ESCON. Today, still, if you want to go father than a certain distance (70km maybe?) its generally recommended to switch over to ESCON rather than fibre channel.
In practicality, complete remote replication is often inadvisable for high-performance, heavy-write applications. The rule of thumb is that every KM adds another millisecond worth of I/O transaction time, so a few kilometers distance can add a significant overhead to writes.
Not to mention Fibre Channel is very unkind when it comes to recovery from fabric segmentation. A poorly designed (or just very unlucky) SAN can be completely downed on both sides of the split, somewhat ruining the disaster recovery strategy if the production and DR storage networks are both taken down.
Ugh - I'll be very happy when fibre channel is dead and buried.
Thanks, Matt
-- Matthew Zito GridApp Systems Email: mzito@gridapp.com Cell: 646-220-3551 Phone: 212-358-8211 x 359
-- ============================================================= The COOK Report on Internet, 431 Greenway Ave, Ewing, NJ 08618 USA 609 882-2572 (PSTN) 703 738-6031 (Vonage) cook@cookreport.com Subscription info & prices at http://cookreport.com/subscriptions.shtml Fiber & Wireless as First Mile Technology - Fiber Business Models & Architecture, July- September 2003, 130 pages available at http://cookreport.com/12.04-06.shtml =============================================================
Although I agree that storage networking as a backup/tandem resiliency operation makes a lot of sense, it does not come anywhere nearly "free" because the assumptions made there require flawless software. If you always synchronize your systems you'll also blow them both out when you hit a software issue and as most of the audience is painfully aware, they are not infrequent enough to count out of the equation. So you need to keep track how to re-synchornize after an upgrade/failure/etc. With a large stream of updated data on the system keeping the state difference becomes expensive quickly. (unless you are going to re-sync everything after a failure, and then you'll be vulnerable until that's completed) Pointers to perfect SAN solutions appreciated. Pete Gordon Cook wrote:
Roxane Googin is usually ahead of the competition in spotting trends in data net use. here is what has has to say about enterprise use of MANs.
COOK Report: Where are the metro area networks going?
Googin: Probably the killer application for the MAN is grid computing and storage area networks. A lot of people think the driver for broadband access to the home is going to be multi-media. People, when they think about next generation networks, think home use. But this is never where the money has been. And no phone company is ever going to build the infrastructure.
Although these new real time applica tions will clearly send more data over the network, the real killer application is going to be remote storage and synchronous storage. Synchronous storage means that you have two large servers doing the exact same thing at the exact same time in two different locations.
COOK Report: Like a decentralized disk array?
Googin: Yes. The backbone has to be incredibly fast because you cannot complete a transaction until you have acknowledgments from both disk drives. This will happen. Probably this year. What they are already doing is taking fiber channel and putting that on a Cienna Core Director optical switch port. Half of the ports being sold on the Core Director now are fiber channel. They aren't even Ethernet. And this is used for storage area nets (SANs). These are corporate MANs and will have nothing to do with sales to service providers. They are bypass business services where the storage arrays may not be more than a kilometer or two apart. These SANs are backing up continuously terabytes of data. We are talking huge applications that will use every bit of access to every bit of capacity they can get.
COOK Report: Is 9/11 a motivation for this?
Googin: Partly. Not only that but the whole paradigm of the real time organization will drive it. It used to be that your server had its own storage. It was a "stove pipe" connected to the CPU. Now as soon as you decouple that "stove pipe," you can put it anywhere. What they are finding is that if they have two of them that are mirrored in real time and place remotely that they do not have to do as much management of resources.
COOK Report: Then forget the disaster back up and recovery operations? Those are the next to be marked for extinction?
Googin: Oh yes. It is so elegant. The new architecture cannot be supported on direct attached storage. It must move off the server and it is doing so. If you have to go through a server to get to the storage, it simply doesn't work. Storage can't become just a utility until you move it off the server. The MAN will be backing up terabytes of data regularly and in real time. Disaster recovery, in real time, comes for "free".
For more detail please see
Fiber & Wireless as First Mile Technology - Fiber Business Models & Architecture http://cookreport.com/12.04-06.shtml
Non-flawless software (i.e., 100% of it) is usually dealt with storage snapshots; if they are kept local or remote, that's a design choice that a metro SAN can impact. Doing backup/restores thru a metro SAN also has some advantages on facilities with tight-controlled physical access; the bandwagon that moves tapes to off-site storage still has more bandwidth than the fiber, but it can also be used to smuggle <insert-your-favorite-paranoia-here>. Rubens ----- Original Message ----- From: "Petri Helenius" <pete@he.iki.fi> To: "Gordon Cook" <cook@cookreport.com> Cc: <Nanog@merit.edu> Sent: Monday, May 19, 2003 7:33 AM Subject: Re: An area for operations growth - Storage Area Nets in MANS | | | Although I agree that storage networking as a backup/tandem resiliency | operation makes a lot of sense, | it does not come anywhere nearly "free" because the assumptions made | there require flawless software. | If you always synchronize your systems you'll also blow them both out | when you hit a software issue | and as most of the audience is painfully aware, they are not infrequent | enough to count out of the equation. | | So you need to keep track how to re-synchornize after an | upgrade/failure/etc. With a large stream of | updated data on the system keeping the state difference becomes | expensive quickly. (unless you are going | to re-sync everything after a failure, and then you'll be vulnerable | until that's completed) | | Pointers to perfect SAN solutions appreciated. | | Pete | | | Gordon Cook wrote: | | > | > Roxane Googin is usually ahead of the competition in spotting trends | > in data net use. here is what has has to say about enterprise use of | > MANs. | > | > | > COOK Report: Where are the metro area networks going? | > | > Googin: Probably the killer application for the MAN is grid computing | > and storage area networks. A lot of people think the driver for | > broadband access to the home is going to be multi-media. People, when | > they think about next generation networks, think home use. But this | > is never where the money has been. And no phone company is ever going | > to build the infrastructure. | > | > Although these new real time applica tions will clearly send more data | > over the network, the real killer application is going to be remote | > storage and synchronous storage. Synchronous storage means that you | > have two large servers doing the exact same thing at the exact same | > time in two different locations. | > | > COOK Report: Like a decentralized disk array? | > | > Googin: Yes. The backbone has to be incredibly fast because you | > cannot complete a transaction until you have acknowledgments from both | > disk drives. This will happen. Probably this year. What they are | > already doing is taking fiber channel and putting that on a Cienna | > Core Director optical switch port. Half of the ports being sold on | > the Core Director now are fiber channel. They aren't even Ethernet. | > And this is used for storage area nets (SANs). These are corporate | > MANs and will have nothing to do with sales to service providers. | > They are bypass business services where the storage arrays may not be | > more than a kilometer or two apart. These SANs are backing up | > continuously terabytes of data. We are talking huge applications that | > will use every bit of access to every bit of capacity they can get. | > | > COOK Report: Is 9/11 a motivation for this? | > | > Googin: Partly. Not only that but the whole paradigm of the real time | > organization will drive it. It used to be that your server had its | > own storage. It was a "stove pipe" connected to the CPU. Now as soon | > as you decouple that "stove pipe," you can put it anywhere. What they | > are finding is that if they have two of them that are mirrored in real | > time and place remotely that they do not have to do as much management | > of resources. | > | > COOK Report: Then forget the disaster back up and recovery | > operations? Those are the next to be marked for extinction? | > | > Googin: Oh yes. It is so elegant. The new architecture cannot be | > supported on direct attached storage. It must move off the server and | > it is doing so. If you have to go through a server to get to the | > storage, it simply doesn't work. Storage can't become just a utility | > until you move it off the server. The MAN will be backing up | > terabytes of data regularly and in real time. Disaster recovery, in | > real time, comes for "free". | > | > For more detail please see | > | > Fiber & Wireless as First Mile Technology - Fiber Business Models & | > Architecture http://cookreport.com/12.04-06.shtml | | |
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Rubens Kuhl Jr. Sent: Monday, May 19, 2003 6:37 AM To: Nanog@merit.edu Subject: Re: An area for operations growth - Storage Area Nets in MANS
Non-flawless software (i.e., 100% of it) is usually dealt with storage snapshots; if they are kept local or remote, that's a design choice that a metro SAN can impact.
Doing backup/restores thru a metro SAN also has some advantages on facilities with tight-controlled physical access; the bandwagon that moves tapes to off-site storage still has more bandwidth than the fiber, but it can also be used to smuggle <insert-your-favorite-paranoia-here>.
Rubens
Yeah, people with serious business continuity needs use real-time replication for datacenter disasters, plus point-in-time copies to protect against software error that often get replicated remotely as well. That way, the DR servers can serve a purpose doing regression testing, development work, integrity testing, etc, without exposing the possibility of unrecoverability in the event of a disaster by disabling the real-time replication to do testing. The other interesting thing in terms of point-in-time replication is what has come to be known as near-line storage. Big boxes of inexpensive drives that are used to offload point-in-time copies from expensive storage for long-term retrieval. This is someplace where metro networks could come into play - having organizations use this excess bandwidth to offload encrypted copies of various data sets to a centralized point run by some neutral third-party. This would be a good way to enforce data retention policies and offsite storage of data. Like an Iron Mountain, only without all the irritation of trucks, people, and magnetic tapes. Economies of scale, plus much better recovery SLAs than having someone retrieve tapes out of a vault somewhere. Thanks, Matt -- Matthew Zito GridApp Systems Email: mzito@gridapp.com Cell: 646-220-3551 Phone: 212-358-8211 x 359
participants (4)
-
Gordon Cook -
Matthew Zito -
Petri Helenius -
Rubens Kuhl Jr.