Thanks Nick, I only see the following layers... I see no extension headers behind the ipv6 header. I sent you the wireshark sniff directly so you can see what I'm seeing. Ethernet - Type 0x86dd Ipv6 - Next Header IPIP (4) Ipv4 Icmp -Aaron

Oh snap! Hey hey, that's good, thanks Nick. I had to go into the locator service of the remote pe and find a sid that would respond to ping. This is apparently an OAM Endpoint with Punt (End.OP) https://www.cisco.com/c/en/us/td/docs/routers/asr9000/software/asr9k-r7-0/se... Here I'm executing ping/trace from the SRv6 ingress pe...to the egress PE RP/0/RP0/CPU0:r1#ping fc00:0:4:4::1 source lo0 use-srv6-op-sid fc00:0:0:4:40:: Mon Sep 14 20:27:09.727 UTC Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to fc00:0:4:4::1, timeout is 2 seconds: SSSSS Success rate is 100 percent (5/5), round-trip min/avg/max = 329/371/416 ms RP/0/RP0/CPU0:r1#traceroute fc00:0:4:4::1 source lo0 use-srv6-op-sid fc00:0:0:4:40:: Mon Sep 14 20:27:19.068 UTC Type escape sequence to abort. Tracing the route to fc00:0:4:4::1 1 ::ffff:0.0.0.0 [IP tunnel: DA=fc00:0:0:4:40:: SRH Stack 0 =(fc00:0:4:4::1 ,SL=1) ] 29 msec [IP tunnel: DA=fc00:0:0:4:40:: SRH Stack 0 =(fc00:0:4:4::1 ,SL=1) ] 56 msec [IP tunnel: DA=fc00:0:0:4:40:: SRH Stack 0 =(fc00:0:4:4::1 ,SL=1) ] 12 msec 2 ::ffff:0.0.0.0 [IP tunnel: DA=fc00:0:0:4:40:: SRH Stack 0 =(fc00:0:4:4::1 ,SL=1) ] 118 msec [IP tunnel: DA=fc00:0:0:4:40:: SRH Stack 0 =(fc00:0:4:4::1 ,SL=1) ] 101 msec [IP tunnel: DA=fc00:0:0:4:40:: SRH Stack 0 =(fc00:0:4:4::1 ,SL=1) ] 99 msec 3 fc00:0:4:4::1 224 msec 277 msec 254 msec 4 ::ffff:0.0.0.0 237 msec 209 msec 204 msec 5 fc00:0:4:4::1 386 msec 431 msec 403 msec Now I see this on the wireshark capture... Ethernet - 86dd Ipv6 - DA fc00:0:0:4:40:: (cool, this is the active/top SID, and not the ping'ed DA) - routing header for v6 (segment routing) --- segments left: 1 --- address next segment: fc00:0:4:4::1 Icmpv6 -Aaron

Mark Tinka wrote on 15/09/2020 07:04:

My head hurts:-)...

yep, and you're not alone - the complexity level is pretty high, right from the control plane to the hardware. It's not clear that the modest net gain in functionality is worth it. Nick

On Tue, 15 Sep 2020 at 12:15, Nick Hilliard <nick@foobar.org> wrote:

yep, and you're not alone - the complexity level is pretty high, right from the control plane to the hardware.

It's not clear that the modest net gain in functionality is worth it.

Many people are buying hook, line and sinker on the narrative 'it is just IP, nothing scary and complex like MPLS'. I think SRv6 is an abomination, it is complex SW, and very complex HW, because it exists. We pay the premium to add HW support for it. For use cases it has, MPLSoUDP would do the same, fast, and it would actually pass unmanaged IP domains, unlike SRv6, since many people filter EH at edges. -- ++ytti

Sorry guys, I'm not aware of much of what you mention as far as agenda, vendor motive, and hardware support, etc.... I'm still learning, but, It does seem interesting that the IP layer (v6) can now support vpn's without mpls. So one less layer of encapsulation seems cool. Don't get me wrong, I love all that mpls has done for us and offers, but, seems that SRx6 (x=v or m) is able to do it. Seems that the end to end label challenges with unified mpls, and all the csc and vpn type a,b,c might be better done with an IPv6 stack of headers and SIDs. (wow, I'm not a prophet, but am I sensing another death of a labeling protocol ?! this would be interesting if like MPLS killed ATM.... SR kills MPLS !) (namely SRv6/SRm6) And with this v6 SID being smartly divided into Locator:Function(Argument), I'm reading that this will carry with it much more functionality as well, like network programmability, application-to-network interaction or something like that. Oh and I do agree that this SRv6 terminology and architecture does make your head hurt, lol, there is some very different/new stuff going on there. -Aaron

Saku Ytti wrote on 15/09/2020 18:05:

You just move the encapsulation from in-order to inside-ip making everything harder for SW and much harder for HW, the simplicity is a lie.

to quantify this, the tunneling header increased in size from a minimum of 4 octets to a minimum of 40 octets. If you want explicit path routing, you'll need to tack on a SRH which is another 8 octets + 16 octets per SID, so e.g. an mpls frame with 2-node ERO goes from 12 octets to 80 octets. This comes at a cost. What was previously a simple lookup operation on a tightly optimised format is now up to 10x bloated with little extra functionality to show for it. It's true that these devices already do ipv6, but can they do ipv6 + complex decapsulation in a single pass? If you're using an NPU, probably yes. If an ASIC, maybe not. What if the decapsulated packet has a stash of ipv6 extension headers? This gets complicated quickly, and that complication can only be solved by adding complication to silicon, which is what you want not to do when the speed of your underlying forwarding plane is increasing by leaps and bounds. Good, cheap, fast. Choose two - or maybe one. The control plane is byzantine. This also has a cost in terms of design, build and support / maintenance. As Mark points out, many companies have made their fortunes with a full stack product offering, from hardware and software to design, engineering and operations. It's not a bad business model as long as you realise that it's time-limited to the technology of the day. When it draws to a close, it's hard to turn companies around that have been used to a single-product or single-vertical cash trough which no longer exists. Some have done this successfully; others have floundered. Nick

On 15/Sep/20 19:05, Saku Ytti wrote:

Ultimately it is very simple, we need tunneling, then the question is how much does it cost to look up those tunnel headers and how much space they take on the wire (relevant for overspeed), everything else is noise.

As we've said many times before, MPLS is not a bad tunneling service. And while things can always be better, it's a bit hard to find anything else, today, that is reasonably well baked-in and efficient (as it can be). Mark.

GRE, VXLAN or any other tunneling encap of the day. As long as next-hop could be resolved behind remote end Regards, Jeff

On Sep 15, 2020, at 11:14, Randy Bush <randy@psg.com> wrote:



...

I'm still learning, but, It does seem interesting that the IP layer (v6) can now support vpn's without mpls.

as the packet payload is nekkid cleartext, where is the P in vpn?

Randy, Was meant as the reply to Aaron’s comment about VPN’s over non MPLS underlay, not the encryption of it (which is orthogonal). Cheers, Jeff On Sep 15, 2020, 12:59 PM -0700, Randy Bush <randy@psg.com>, wrote:

...

GRE, VXLAN or any other tunneling encap of the day. As long as next-hop could be resolved behind remote end

i was not aware that GRE, VXLAN (without CN103618596A), and other tunnel encaps encrypted the payload. learn something new every day. thanks!

...

...

...

I'm still learning, but, It does seem interesting that the IP layer (v6) can now support vpn's without mpls. as the packet payload is nekkid cleartext, where is the P in vpn?

You might be on to something, but I'm unsure... are you suggesting that it's any less private over SRv6 than it was over MPLS ?

neither srv6, srmpls, mpls, gre, ... provide privacy. they all transport the payload in nekkid cleartext. Dance like no one's watching. Encrypt like everyone is.

On Tue, Sep 15, 2020 at 5:08 PM Randy Bush <randy@psg.com> wrote:

...

You might be on to something, but I'm unsure... are you suggesting that it's any less private over SRv6 than it was over MPLS ?

neither srv6, srmpls, mpls, gre, ... provide privacy. they all transport the payload in nekkid cleartext.

Dance like no one's watching. Encrypt like everyone is.

It depends on the definition of VPN. In terms of services like MPLS-based VPNs, it refers to the extension of a Private network over a shared infrastructure, allowing entities using the shared infrastructure to have their own private address space and routing tables.

On 16/Sep/20 23:22, Anoop Ghanwani wrote:

It depends on the definition of VPN.  In terms of services like MPLS-based VPNs, it refers to the extension of a Private network over a shared infrastructure, allowing entities using the shared infrastructure to have their own private address space and routing tables.

Really, it was just a way to leverage IP networks to make more money. Nothing against that, as long as "buyer be aware" applies. Mark.

On 17/Sep/20 17:56, mark seery wrote:

For operators already offering FR/ATM services, it was a replacement, using the same principles of traffic separation over a common infrastructure, without encryption as part of the service. So from that perspective only, it was not much of a change for *existing* enterprise customers.

Indeed. But the difference with Frame Relay and ATM was that telco's never called it a (V)PN. At worst, it was a leased line.

This community is aware of the responsibility of a network is to ensure that traffic is forwarded to the (originally?) intended destination to prevent confidential information being exposed to a third-party. It is in this respect that the term “privacy” is often used. So seems like there is a taxonomy issue here. Perhaps traffic separation is a better term than privacy, because while traffic is probablistically private with respect to other VPN customers (separated with some high level of probability), it is not private with respect to the operator (who could intercept it).

Or someone else who might "capture" the operator, and thus, be able to intercept it.

Sure, transparency is good.

I remember 20 years ago at a London IETF where the issue arose, and a food fight arose over who would own and manage encryption keys if traffic was encrypted. I don’t recall what the resolution of that debate was.

That said, we live in an era where there is increasing sensitivity to protecting consumer (at least) information. This sensitivity exists at multiple layers of the “stack”. So it is an interesting question / issue, and certainly would not be of any surprise if governments mandated it in the future, as long as they could intercept it for law enforcement purposes of course, and until they could, they probably would not be encouraging operators to encrypt data in any difficult to crack way (a speculation on my part).

Perhaps all the more reason why end-to-end encryption should be part of the buyer beware conversation (not arguing against operator encryption in saying that - privacy is something everyone in I[C]T has to think about today).

If gubbermints mandate that l2vpn's and l3vpn's be encrypted, the cloud bags will simply take over (not that they haven't, already). Mark.

On 17/Sep/20 19:36, mark seery wrote:

Private line was a common term for leased lines. Leased lines were not encrypted by the operator, AFAIK. This terminology morphed to virtual private line, Ethernet Private Line, virtual private LAN service (VPLS), etc.

"In telecommunication, a private line is typically a telephone company service that uses a dedicated, usually unswitched point-to-point circuit, but it may involve private switchingarrangements, or predefined transmission physical or virtual paths.”

https://en.wikipedia.org/wiki/Private_line <https://en.wikipedia.org/wiki/Private_line>

https://www.business.att.com/products/dedicated-internet/#/ <https://www.business.att.com/products/dedicated-internet/#/>

http://etler.com/docs/AT&T%20Pub/TR54077.pdf <http://etler.com/docs/AT&T%20Pub/TR54077.pdf>

https://business.comcast.com/enterprise/products-services/data-networking/et... <https://business.comcast.com/enterprise/products-services/data-networking/ethernet-virtual-private-line>

VPN is a terminology consistent with past practices. It is P in all the ways discussed on this thread, and historically consistent (at least from a marketing perspective). Whether it is P enough is a reasonable discussion, everyone in I(C)T is going to be facing a wave of voter concern about privacy, IMO.

It's six and half-a-dozen. "Private Line" isn't the same thing as "Private Network". A small, but crucial distinction, particularly for folk on a list like this. Probably interchangeable to the ordinary wi-fi user. But then again, operators always choose words carefully, and if the contract said "Private Line" in lieu of "Private Network", or vice versa, that wasn't by mistake.

Torn between two lovers: Growing voter concern about privacy & longheld, and arguably increasing, desire to intercept criminal / terrorist communication. I’d actually be curious if any operators have received public sector pushback when they tried to implement encryption.

Sounds like you're making a/the case for MACSec :-). Anyone know how widely MACSec has been adopted? Or for that matter, large-scale encryption on the backbone side? For me, MACSec is kind of like SyncE... great on paper and in the sales pitch, but anyone that truly wants to use those features is probably going to be architecting, deploying and managing them themselves, and not paying a 3rd party network operator for the priviledge. As always, I could be wrong... Mark.

On 18/Sep/20 11:40, tim@pelican.org wrote:

I've got MACSec deployed for exactly one customer as a point solution. It works once it's in, but the documentation, vendor or otherwise, and choice of suitable equipment were fairly sparse. I certainly wouldn't want to offer it at scale.

Encrypted network conversations with customers, I always try to be very clear about what they're trying to protect against, and make them think properly about trust boundaries. Sure, I can slap a managed CPE on site if I don't already have one and provide overlay encryption - but that doesn't stop a rogue engineer on my side from capturing data before it's encrypted. If what you're concerned about is fibre taps, or security flaws in the MPLS traffic-segregation model or implementation, that helps. If you don't want to trust me as a service provider not to sniff your traffic in the middle, having me encrypt it at the edge really doesn't help - you need to encrypt it yourself, or have a different third-party that you do trust do the encryption.

Some people get it, some people are just trying to fill auditor check-boxes ;)

Agreed. There was a time when the use-case for MACSec was to move banks away from running their own DWDM/FC networks, and letting operators do it. I'm yet to find a bank willing to do this. Maybe I'm not paying enough attention. Mark.

Sounds like you're making a/the case for MACSec :-).

While I get your point, and it is a good one, no. Once lawyers, finance, and other functions get involved, it goes from being just another technology, to a pain for suppliers and customers alike. Export laws complicate implementation, and vendors can only afford and/or have the operational agility, to do an implementation once. Any security tech that is sufficiently interesting, is going to be a pain for router vendors to implement and operationalize given the government’s attitude to such tech. The lower in the stack it is, the bigger the pain. That said, vendors are being asked to put MACSec in and I suspect more platforms supporting it will become available over time.

On Thu, 17 Sep 2020 18:24:36 +0200, Mark Tinka said:

On 17/Sep/20 17:56, mark seery wrote:

...

Perhaps all the more reason why end-to-end encryption should be part of the buyer beware conversation (not arguing against operator encryption in saying that - privacy is something everyone in I[C]T has to think about today).

If gubbermints mandate that l2vpn's and l3vpn's be encrypted, the cloud bags will simply take over (not that they haven't, already).

Are there any actual countries heading that way? Seems like most of them insist they have the ability to snoop unencrypted traffic (where "crypto that has a baked-in back door" counts as unencrypted).

Mark,

On 20 Sep 2020, at 13:02, Mark Tinka <mark.tinka@seacom.com> wrote:

On 19/Sep/20 22:53, Valdis Kl ē tnieks wrote:

...

Are there any actual countries heading that way? Seems like most of them insist they have the ability to snoop unencrypted traffic (where "crypto that has a baked-in back door" counts as unencrypted).

Let's not give them a reason.

The most I've heard (from Africa) is countries making requirements for nominated information to not be stored outside of the country, especially in the U.S, and parts of Europe. To some extent, this has pushed many of the cloud bags to become present in Africa so they can comply, although I'm not sure even sleeping with one eye open counts as being safe in that respect.

I believe right now the only country in the world with enforcing of crypto backdoors is Australia[1], which is kind-of crazy. OTOH, they had their own set of problems with massive Chinese intelligence penetration. And we have couple of countries like Russia, obviously China, Turkey(?) that insist or either having your data locally, dear content provider, or forbid your service to operate at all in given country. Apple, Amazon, Microsoft and Google of this world are on a different level of compliance here. As far as I know, in most of EU countries, inspecting payload of customer traffic is explicitly forbidden by telco laws. Ah, and there’s cooperation between US and EU about exchanging citizen data, which recently was stopped by EU once it become obvious, US was abusing that cooperation[2]. That can help potential malicious SP to cross-check and correlate user to content across continents. We’re living in interesting times. [1]. https://www.cyberscoop.com/australia-encryption-backdoors-law-passes/ [2]. https://www.wsj.com/articles/eus-top-court-restricts-personal-data-transfers... -- Łukasz Bromirski CCIE R&S/SP #15929, CCDE #2012::17, PGP Key ID: 0xFD077F6A

On Tue, 15 Sep 2020 at 19:14, Randy Bush <randy@psg.com> wrote:

...

I'm still learning, but, It does seem interesting that the IP layer (v6) can now support vpn's without mpls.

as the packet payload is nekkid cleartext, where is the P in vpn?

Define "privacy". In the kind of VPN I think you're suggesting (e.g. an IPSEC based VPN) they implement the classic CIA acronym (Confidentiality, Integrity and Authentication, with the "C" essentially meaning "encrypted" in practice however, privacy requires all three of "CIA", encryption alone isn't privacy). "VPN" is not mutually dependent on "CIA", the two things can and do exist separately. WIth MPLS L3 VPNs for example, the traffic isn't encrypted, but by creating a layer of abstraction (at the control plane, by signalling via MP-BGP using RDs and RTs, and at the forwarding plane by using MPLS tunneling) a customer's routing data and forwarding data is kept private (not encrypted!) from my physical infa forwarding- and control-planes, and from each other L3 VPN customer on my infra [1]. In fact, the point that customer (control- and forwarding-plane) data is kept private from MY INFRA, is *the* fundamental aspect of MPLS L3 VPNs; they wouldn't scale at all without it. Privacy != encryption. Cheers, James. [1] This doesn't mean there aren't security flaws in MPLS (there are, but there are in things like IPSEC too), and "how secure" it is, is a separate subject.

My backyard is private. It offers no privacy with its chain link fence against a major street. On 9/16/20 4:38 PM, Randy Bush wrote:

...

Privacy != encryption. cleartext == privacy * 0

cleartext * complexity == privacy * 0

randy

Information can be in plaintext and private

Three can keep a secret, if two of them are dead. -- franklin i know you truely believe the tunnel k00laid. the security community does not. randy

One thing that is true: not all present or historical definitions (or acceptable uses) of the word "private" strictly imply or infer privacy.

newspeak -- 1984

On 19 September 2020 03:23:15 BST, Randy Bush <randy@psg.com> wrote:

...

Information can be in plaintext and private

Three can keep a secret, if two of them are dead. -- franklin

i know you truely believe the tunnel k00laid. the security community does not.

Hi Randy, I'm not sure what you're saying here, I never said MPLS VPNs are secure, only private. I hope others recognise that they are different concepts. Cheers, James.

On 22/Sep/20 00:06, Greg Shepherd wrote:

Call me old, but I miss the days when this thread was still on the SRv6 rails. Can we get back the proper bashing to match this thread title?

Probably not off-topic, since vendors may push SRv6 as a(n) (MPLS) VPN replacement and new money-maker for operators, all being done in IPv6 and what-not... Mark.

james,

I'm not sure what you're saying here, I never said MPLS VPNs are secure, only private. I hope others recognise that they are different concepts.

yes, privacy is one aspect of security. and, as mpls vns are not private sans encryption, they are not secure. randy

On 9/21/20 6:16 PM, Randy Bush wrote:

yes, privacy is one aspect of security. and, as mpls vns are not private sans encryption, they are not secure. randy

As my backyard is not surrounded by a cement enclosure with acoustic baffling and white noise generators inside, it's not really private property.

Nick, does CRH-16/32 and uSID change the overhead concern? I could be wrong, but I thought that's what SRm6 was for, was to shrink the overhead, perhaps amongst other things. Also, with VPN's over SRv6 would this enable automatic vpn capability over the internet? I mean if I can do VPN's over an IPv6 network, seems that I could do that across the Internet as well. Thanks Tom, man, I have read so much the last week or so regarding sr/spring... so if you mean that I borrowed someone else's description of the anatomy of an SRv6 SID.... then, yes, I may have and didn't know it. Hey, was it you? LOL - Aaron

aaron1@gvtc.com писал 2020-09-15 20:31: Hi Aaron,

Also, with VPN's over SRv6 would this enable automatic vpn capability over the internet? I mean if I can do VPN's over an IPv6 network, seems that I could do that across the Internet as well.

I think you already can do it over any kind of tunnel, and there are a lot of SDWAN solutions are available. Or do you expect from a transit provider a capability to respect and process SID stack programmed by another provider? I wouldn't bet on this ;) From administrative PoV it's similar to Inter-AS or CsC, based on trusted relations between parties, but seems not very popular in real life. Otherwise, it will be the same best path routing as for any general tunnel. Doesn't look as a distinctive advantage of SRv6 at least.

- Aaron

-- Kind regards, Andrey

Mark,

On 16 Sep 2020, at 10:32, Mark Tinka <mark.tinka@seacom.com> wrote:

On 15/Sep/20 19:00, aaron1@gvtc.com wrote:

...

Sorry guys, I'm not aware of much of what you mention as far as agenda, vendor motive, and hardware support, etc....

I'm not shy... this would be Cisco.

And that’s fine. The fact that some Intellectual Property[1] was created by one vendor or another (disclaimer - I work for Cisco) shouldn’t be by default something that discredits the idea. And BTW, if You look into the history of how SR started, it was close feedback loop with at least some of the ISPs wanting to have “easier” and SDN-ish control over the network so the blame should be shared :) Having support from other vendors was de facto requirement to even think about deploying it widely and that's better approach IMHO than “lets patent everything out and force everyone into our black-box-architecture that’s best in the world”. I’m observing the discussions over the last couple of months and generally they boil down to “leave us alone, everything sucks, we’ll stay with what we have”. And sure, as you indisputably proven during last 30 years, running modern ISP network can be done over IP, MPLS, and the network can even survive introduction of IPv6. And I get it - vendors have generally failed to address your ideas and problems in timely manner, and when we did, quality was simply not there. But really, is that all what’s interesting in life? I doubt it. Unless the whole point of discussing here would be to start from technical topics (because of ‘rules’) and end up with everybodys favorite part - beating virtual Pinata made to look like representation of most hated salesman/vendor. Then sorry, I somehow missed that :) While I personally find the idea of stacking IPv6 labels gruesome for any non-trivial label depth[2] (and I'm really worried about software guys coming in from the “mobile app” world soon, and finding out that they can create those IPv6 EH stacks easily), going forward we have to think about what will keep networks running in for the next 5, 10 or 20 years. IPv4 with MPLS+LDP+RSVP-TE will work fine of course, so will SRoMPLS, but IPv6 is gaining adoption and need to multiplex/demultiplex more and more services and users will only grow. And BTW, MPLS forwarding between ASes in the internet is still something that works mainly on slides, highly paid consulting “proposals” and of course on the CCIE exam. On the other side, there’s Elon Musk moving us to Mars, wretched IoT world with “build, sell and forget" mentality w/r to firmware and good network stacks. And yet only 59% people around the world today have internet access. At least good portion of that heavily filtered one by the way. IPv6 seems to be good plan forward (and would potentially unify architecture of normal routing and overlay routing with SRv6), even if things like MPLSoUDP or GRE would really solve everything if pushed with enough force[3] ;) It’s worth observing, that from this perspective IPinIP would be as good as SRv6 if everyone would agree 20 years ago that source routing is acceptable. LDP or RSVP-TE would never gain any usage and maybe we wouldn’t learn lesson or two. BTW, we tried to somehow get back to this simplification with LISP[4], and in the long run it seems overloading address semantics is not something that is happily accepted everywhere, and it doesn’t matter if that's IPv4 or IPv6. So much hated middleboxes will also adopt faster to IPv6, as adopting MPLS data plane after those 20 years on firewalls, load balancers and what-you looks kind of dissapointingly. And if we are talking about network functions - I believe it’s more important right now to agree on one way of doing service chaining, than discussing SR or SRv6 as evil seed created to conquer the world. SR takes out state out, and SRv6 has the same address format on the outside as well as inside. You can happily run it with both data planes, and while today maybe you can’t provide migration of ALL services, SR+IGP quite nicely interworks with MPLS+LDP. Will HW evolve? It has to anyway, no previous change was done day one and 128 bits times 5 or 8 or 12 seems horrible only today. Over the years, people got used to bigger horrors ;) — ./ [1]. https://patents.google.com/patent/US20140169370 [2]. Yeah, Binding SIDs of course, but that’s a solution to self-made problem as Ivan Pepelnjak would say. [3]. https://tools.ietf.org/html/rfc1925 point 2.3. [4]. https://tools.ietf.org/html/rfc6830

Hello All , On Tue, 15 Sep 2020, Mark Tinka wrote:

On 15/Sep/20 11:53, Saku Ytti wrote:

...

I think SRv6 is an abomination, it is complex SW, and very complex HW, because it exists. We pay the premium to add HW support for it.

And that is what the vendor(s) pushing this hope operators "realize"... that SRv6 is a complex mess that needs some kind of centralized system to manage. But wait, the centralized system is, itself, quite complex that no operator would dare spend time installing, commissioning or maintaining it themselves.

So what ever shall we do, as operators?

Simple, pay the vendor(s) to take care of all of it for you... planning, design, spec'ing, bill of materials, deployment, operation and refresh programs... lock that vendor top and bottom line in for them for the next 10 years, while they find some other RFC to create in order to keep the cycle going 11 years later.

Mark.

So then here we are back at the older days of hard wired devices configured on site by vendor 'X' to do what buyer 'Y' was told it would do . And Buyer 'Y' is still wondering WHEN it will be that kitchen sink it ordered . Twyl , JimL -- +---------------------------------------------------------------------+ | James W. Laferriere | System Techniques | Give me VMS | | Network & System Engineer | 3237 Holden Road | Give me Linux | | jiml@system-techniques.com | Fairbanks, AK. 99709 | only on AXP | +---------------------------------------------------------------------+