Common operational misconceptions
Hi friends, As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct. For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing. I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with. I'd prefer replies off-list and can summarize back to the list if there is interest. John
Switching VS Bridging Collision Domain VS Broadcast Domain L2 in general is the layer that the new folks often misunderstand. I once had someone ask me what a hub was. That pretty much told me how old I was.... -Hammer- "I was a normal American nerd" -Jack Herer On 2/15/2012 2:47 PM, John Kristoff wrote:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
Keep the discussion on the list. I would like to know as well. Kenneth M. Chipps Ph.D. -----Original Message----- From: John Kristoff [mailto:jtk@cymru.com] Sent: Wednesday, February 15, 2012 2:47 PM To: nanog@nanog.org Subject: Common operational misconceptions Hi friends, As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct. For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing. I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with. I'd prefer replies off-list and can summarize back to the list if there is interest. John
On Wed, Feb 15, 2012 at 1:10 PM, Kenneth M. Chipps Ph.D. <chipps@chipps.com>wrote:
Keep the discussion on the list. I would like to know as well.
Kenneth M. Chipps Ph.D.
-----Original Message----- From: John Kristoff [mailto:jtk@cymru.com] Sent: Wednesday, February 15, 2012 2:47 PM To: nanog@nanog.org Subject: Common operational misconceptions
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
I don't know how many times I have "Network Administrators" ask questions like this... Speaking in the context of configuring an ipsec tunnel.. "I have my side built. Can you lock your side down to a specific protocol? Our sets his device to TCP 104. Makes it nice for me when I set my ACLs." I am pretty sure that he meant protocol TCP and Port 104, but I do grind my teeth when I have to go show them that a specific protocol number means something completely different than what they were asking. -- Mark Grigsby Network Operations Manager PCINW (Preferred Connections Inc., NW) 3555 Gateway St. Ste. 205 Springfield, OR 97477 Office 541-242-0808 ext 408 TF: 800-787-3806 ext 408 DID: 541-762-1171 Fax: 541-684-0283
Mark Grigsby <mark@pcinw.net> writes:
Speaking in the context of configuring an ipsec tunnel..
Once upon a time: Admin: "We need Port 50 and Port 51 for the tunnel!" Me: "You mean IP protocol 50 and 51?" Admin: "It the same! You have no clue!" Jens -- ------------------------------------------------------------------------- | Foelderichstr. 40 | 13595 Berlin, Germany | +49-151-18721264 | | http://blog.quux.de | jabber: jenslink@guug.de | ------------------- | -------------------------------------------------------------------------
On 02/15/12 14:47 -0600, John Kristoff wrote:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
I almost always see someone fill in the 'default gateway' field when they're configuring a temporary address on their computer to communicate with a device on the local network. On the topic of VLANs, it's common to think of 'trunking' and something that happens between switches. It's hard to get someone to ponder the fact that trunking isn't an all or nothing concept, and that a server can be configured to speak vlan. Confusing ftp, sftp, ftps. Or telnet, telnets, ssh. Packet loss at hop X in traceroute/mtr does not necessarily point to a problem at hop X. BGP does not magically load balance your ingress and egress traffic. Just because it's down for you, doesn't mean it's down for everyone. -- Dan White
"Packet loss at hop X in traceroute/mtr does not necessarily point to a problem at hop X." Good one. Also, security as a whole seems to be confusing for folks. They've seen "Firewall" with Harrison Ford and therefore the FW is some secret master voodoo widget that only people from Area 51 can operate. They don't understand "header manipulation" vs "payload". -Hammer- "I was a normal American nerd" -Jack Herer On 2/15/2012 3:52 PM, Dan White wrote:
Packet loss at hop X in traceroute/mtr does not necessarily point to a problem at hop X.
A few for me that come to mind which haven't been covered yet. *) Latency, jitter, etc when pinging a router means packets going through the router suffer the same fate. Never fails that I get a call about the latency changes that occur every 60 seconds, especially on software based routers. uh, huh. *) admin/admin is okay in a private network behind a firewall Oh, look, a console port! *) Assign arbitrary MTUs in a layer 2 transport network based on exactly what customers order. *) MTU/packet/frame/ping size means the same thing on all vendors. *) If Wireshark looks right, it must be right (unless Windows discarded 1 (and only 1) layer of 802.1q tags) *) Upgrades should always be done, even when there's no relevant security or functionality that is needed in newer code. Amazing how many code changes break things which don't necessarily show up in test environments but will show in production networks (Your mpls worked for months with an invalid router-id configured, and then broke when you change codes? DHCP worked fine, but after upgrade quit accepting <300 byte DHCP packets?). Jack
Autoneg. The old timers that don't trust it after a few decades of decent code. Or those that lock one side and expect the other to adjust to that. /Tias 15 feb 2012 kl. 21:47 skrev John Kristoff <jtk@cymru.com>:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
Mathias Wolkert <tias@netnod.se> writes:
Autoneg. The old timers that don't trust it after a few decades of decent code. Or those that lock one side and expect the other to adjust to that.
Autoneg is black magic. Doesn't work. You have manually configure duplex and speed on one side !!!!1! SCNR Jens -- ------------------------------------------------------------------------- | Foelderichstr. 40 | 13595 Berlin, Germany | +49-151-18721264 | | http://blog.quux.de | jabber: jenslink@guug.de | ------------------- | -------------------------------------------------------------------------
On Fri, 17 Feb 2012, Jens Link wrote:
Mathias Wolkert <tias@netnod.se> writes:
Autoneg. The old timers that don't trust it after a few decades of decent code. Or those that lock one side and expect the other to adjust to that.
you are referring to ehh *kuch* certain internet exchanges *kuch* ? :P auto mdi/mii breaks teh internets! oeh noes! (not on any equipment we've owned for the past 15 years... funny how that works ;)
Autoneg is black magic. Doesn't work. You have manually configure duplex and speed on one side !!!!1!
SCNR
Jens -- ------------------------------------------------------------------------- | Foelderichstr. 40 | 13595 Berlin, Germany | +49-151-18721264 | | http://blog.quux.de | jabber: jenslink@guug.de | ------------------- | -------------------------------------------------------------------------
I am grateful you have not used the hardware I have in the past 15 years. I haven't seen anything recently not do it, but when interfacing with a customer who knows what old stuff they may be using. Jared Mauch On Feb 17, 2012, at 12:41 PM, Sven Olaf Kamphuis <sven@cb3rob.net> wrote:
auto mdi/mii breaks teh internets! oeh noes! (not on any equipment we've owned for the past 15 years... funny how that works ;)
Auto-neg, as someone already mentioned. MD5 makes BGP peering sessions more secure. There was a nice recent NANOG rant on that one. One of my favorites from corporate america; if you run one application on a server you can put in that apps port in the firewall and block everything else and the server will be happy. Evidently folks don't know servers need to do things like make DNS queries, have remote access to them, contact domain controllers or software update servers. *sigh* -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
DNS only uses UDP DNS only uses 512 byte UDP packets or maybe just.. DNS is easy -- It's always a long day; 86400 doesn't fit into a short. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/
In message <4F3C2E47.80903@dougbarton.us>, Doug Barton writes:
DNS only uses UDP DNS only uses 512 byte UDP packets
or maybe just..
DNS is easy
Or that it is correct/does no harm to filter fragmented packet / icmp. -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
ICMP is bad, and should be completely blocked for "security". On Wed, Feb 15, 2012 at 02:47:15PM -0600, John Kristoff wrote:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
On Feb 15, 2012, at 23:36, Chuck Anderson wrote:
security
That must be the top of the list: Switches provide security (by traffic isolation) DHCP provides security (by only letting in hosts we know) MAC address filtering provides security (fill in the blanks…) NAC provides security NATs provide security Firewalls provide security Buying Vendor-_ provides security Grüße, Carsten
With security in mind: Use other VLANs other than vlan1. Disable vlan1. Disable ports (physical and logical) that aren't in use. Encrypt your passwords in your config, etc etc etc... On Wed, Feb 15, 2012 at 2:49 PM, Carsten Bormann <cabo@tzi.org> wrote:
On Feb 15, 2012, at 23:36, Chuck Anderson wrote:
security
That must be the top of the list:
Switches provide security (by traffic isolation) DHCP provides security (by only letting in hosts we know) MAC address filtering provides security (fill in the blanks…) NAC provides security NATs provide security Firewalls provide security Buying Vendor-_ provides security
Grüße, Carsten
-- Mike Lyon 408-621-4826 mike.lyon@gmail.com http://www.linkedin.com/in/mlyon
On Wed, Feb 15, 2012 at 04:51:44PM -0600, Anton Kapela wrote:
On Wed, Feb 15, 2012 at 4:36 PM, Chuck Anderson <cra@wpi.edu> wrote:
ICMP is bad, and should be completely blocked for "security".
I can't tell if this reply is to say "this ought to be done" or if "this is often done, and should not be."
Clarify?
This thread is about misconceptions. What I said was a common misconception that "all ICMP should be blocked for security reasons". In reality, some kinds of ICMP are REQUIRED for proper functioning of an internetwork for things like Path MTU Discovery (ICMP Fragmentation Needed/Packet Too Big). Other kinds of ICMP are good to allow for being nice to the users and applications by informing them of an error immediately rather than forcing them to wait for a timeout (ICMP Destination Unreachable).
(1) Block all ICMP (obviously some are required for normal operations, unreachables, pMTU too large/DF set, etc). (2) Block certain ports (blindly, w/o at least "established") taking out legitimate ephemeral port usage. (3) Local uRPF is unnecesary (or source spoofing mitigation in general) (4) Automagical things are necessary (Microsoft proprietary, UPnP, Apple Bonjour, mDNS, etc) (5) WAN routing to multiple providers will automagically load-balance automagically. or for that matter... (6) IGP routing across multiple paths will automagically load-balance automagically. Or for that matter... (7) Port-channel (link aggregation) will load-balance automagically. (8) Connectivity/throughput issues are always local or first-hop. (We have a gig connection, why am I not getting a gig throughput) I'm sure there are more, but those were at the top of my head :) Jeff
ICMP is evil. Firewalls can be configured default-permit. Firewalls can be configured unidirectionally. Firewalls will solve our security issues. Antivirus will solve our security issues. IDS/IPS will solve our security issues. Audits and checklists will solve our security issues. Our network will never emit abuse or attacks. Our users can be trained. We must do something; this is something; let's do this. We can add security later. We're not a target. We don't need to read our logs. What logs? (with apologies to Marcus Ranum, from whom I've shamelessly cribbed several of these) ---rsk
This isn't so much a list of misconceptions that recent students have as a list of misconceptions that security management have… On 15 Feb 2012, at 22:52, Rich Kulawiec wrote:
ICMP is evil. Firewalls can be configured default-permit. Firewalls can be configured unidirectionally. Firewalls will solve our security issues. Antivirus will solve our security issues. IDS/IPS will solve our security issues. Audits and checklists will solve our security issues. Our network will never emit abuse or attacks. Our users can be trained. We must do something; this is something; let's do this. We can add security later. We're not a target. We don't need to read our logs. What logs?
(with apologies to Marcus Ranum, from whom I've shamelessly cribbed several of these)
---rsk
Or a security vendor, or a security publication... the whole "top ten" delivered as ten individual clicks with pay-per-view banner ads on each page and a bazillion tracker cookies.... arrrrrrgh..... Jeff On 2/16/2012 5:26 AM, Chris Campbell wrote:
This isn't so much a list of misconceptions that recent students have as a list of misconceptions that security management have…
On 15 Feb 2012, at 22:52, Rich Kulawiec wrote:
ICMP is evil. Firewalls can be configured default-permit. Firewalls can be configured unidirectionally. Firewalls will solve our security issues. Antivirus will solve our security issues. IDS/IPS will solve our security issues. Audits and checklists will solve our security issues. Our network will never emit abuse or attacks. Our users can be trained. We must do something; this is something; let's do this. We can add security later. We're not a target. We don't need to read our logs. What logs?
(with apologies to Marcus Ranum, from whom I've shamelessly cribbed several of these)
---rsk
Telco provided VPN makes communication between your sites secure. If you can use (virtual) circuits, even better. -- Alg
On Wed, Feb 15, 2012 at 3:47 PM, John Kristoff <jtk@cymru.com> wrote:
I have a handful of common misconceptions that I'd put on a top 10 list,
By your classful addressing example, it sounds like these students are what most nanog posters would consider to be entry-level. RFC1918 is misused a lot by entry-level folks, most seem not to know about 172.16.0.0/12 I think students should be able to learn how "traceroute" actually works, which I have found, is a lot easier to teach as a conceptual lesson than by just telling them "maybe the problem is in the return path" without giving them any understanding of how or why. MTU, Path MTU Detection, and MSS NxGE isn't a serial 4Gbps link, and why this is so important On the other hand, more than half of the CCIEs I have worked with are clueless about all of the above. :-/ -- Jeff S Wheeler <jsw@inconcepts.biz> Sr Network Operator / Innovative Network Concepts
traceroute shows _a_ path. Your packets might have taken a different path. (& the return traffic yet another) labeling something "backup link" on the network diagram doesn't make it one. Lee On 2/15/12, John Kristoff <jtk@cymru.com> wrote:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
Or more to the point, it is a misconception that traffic is symetrical (the path out and the path back are the same) whereas in the present network, symetrical paths are the exception rather than the rule, especially as your radius increases. On Wed, Feb 15, 2012 at 07:17:57PM -0500, Lee wrote:
traceroute shows _a_ path. Your packets might have taken a different path. (& the return traffic yet another)
--- Wayne Bouchard web@typo.org Network Dude http://www.typo.org/~web/
On Thu, 16 Feb 2012, Wayne E Bouchard wrote:
Or more to the point, it is a misconception that traffic is symetrical (the path out and the path back are the same) whereas in the present network, symetrical paths are the exception rather than the rule, especially as your radius increases.
To add to that, I feel pretty sure in stating that many of the people on this list, at one point or another, have either had people tell them that asymmetric routing is "bad", or have had to explain why asymmetric routing across 'the Internet' is not "bad", even if it can make troubleshooting a 'network problem' somewhat more involved. The path from A to B is not necessarily the same as the path from B to A. jms
Right, asymmetric paths are the norm for many inter-provider communications. A related misconception is that asymmetric paths are problematic. Another mis-conception related to path is that more (visible) hops are bad with the corollary that routers introduce (significant) latency. Of course, propagation delay is the most significant contributor to latency in WAN environments (or serialization delay for low-speed links). Tony On Thu, Feb 16, 2012 at 3:35 PM, Wayne E Bouchard <web@typo.org> wrote:
Or more to the point, it is a misconception that traffic is symetrical (the path out and the path back are the same) whereas in the present network, symetrical paths are the exception rather than the rule, especially as your radius increases.
On Wed, Feb 15, 2012 at 07:17:57PM -0500, Lee wrote:
traceroute shows _a_ path. Your packets might have taken a different path. (& the return traffic yet another)
PKI is cryptographically secure. IDN is internationalized. IPv6 reduces router load by not allowing fragmentation. IPv6 is operational. Masataka Ohta
On 2012.02.15 19:55, Nathan Eisenberg wrote:
IPv6 is operational.
How is this a misconception? It works fine for me...
Imagine an operator who is v6 ignorant, with a home provider who implements v6 half-assed, and tries to access a v6 site that has perhaps v6-only accessible nameservers, when their provider who 'offers' v6 has resolvers that operate only over v4. *huge* misconception about the operational status of IPv6 (imho). Steve
In message <4F3C6703.4050607@gmail.com>, Steve Bertrand writes:
On 2012.02.15 19:55, Nathan Eisenberg wrote:
IPv6 is operational.
How is this a misconception? It works fine for me...
Imagine an operator who is v6 ignorant, with a home provider who implements v6 half-assed, and tries to access a v6 site that has perhaps v6-only accessible nameservers, when their provider who 'offers' v6 has resolvers that operate only over v4.
*huge* misconception about the operational status of IPv6 (imho).
This doesn't prove that IPv6 is not operational. All it proves is people can misconfigure things. If you provide the recursive nameservers with IPv6 access they will make queries over IPv6 even if they only accept queries over IPv4. If you want to know if your resolver talks IPv6 to the world and supports 4096 EDNS UDP messages the following query will tell you. dig edns-v6-ok.isc.org txt Similarly for IPv4. dig edns-v4-ok.isc.org txt Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
Mark Andrews wrote:
This doesn't prove that IPv6 is not operational. All it proves is people can misconfigure things.
How do operators configure their equipments to treat ICMP packet too big generated against multicast and unicast? Note that, even if they do not enable inter-subnet multicast in their domains, the ICMP packets may still transit over or implode within their domains. Note also that some network processors can't efficiently distinguish ICMP packets generated against multicast and unicast. Masataka Ohta
Not understanding RFC1918. Actually got read the riot act by someone because I worked for an organization that used 10.0.0.0/8 and that was "their" network and "they" owned it. Chuck 2012/2/15 Masataka Ohta <mohta@necom830.hpcl.titech.ac.jp>
Mark Andrews wrote:
This doesn't prove that IPv6 is not operational. All it proves is people can misconfigure things.
How do operators configure their equipments to treat ICMP packet too big generated against multicast and unicast?
Note that, even if they do not enable inter-subnet multicast in their domains, the ICMP packets may still transit over or implode within their domains.
Note also that some network processors can't efficiently distinguish ICMP packets generated against multicast and unicast.
Masataka Ohta
On Wed, 15 Feb 2012 22:26:11 -0500 Charles Mills <w3yni1@gmail.com> wrote:
Not understanding RFC1918. Actually got read the riot act by someone because I worked for an organization that used 10.0.0.0/8 and that was "their" network and "they" owned it.
Once upon a time, a now deservedly defunct organization called marchFIRST, was brought in to do a security assessment of a university network I was involved in and one issue I recall them "identifying" was that the university was not "RFC 1918 compliant". The following statement was also made: As part of the Forum of Incident Response and Security Teams (FIRST) Coalition, there is no firewall between the DePaul University data network and the Coalition network. *plonk* John
Not understanding RFC1918 also gets my vote. Don't call them "unroutable", ever. I like it when I hear people say "if you type a net 10 address into a router, it will reject it". What do they think people do with those networks anyway? Call them "private" or "RFC1918" networks/address spaces/ranges. I don't know if it's really a common misconception, but worth underlining to people, especially in these days of IPv4 kinkiness, the expectation of global address uniqueness in the IP model. (Yes, NATs are probably here to stay, but they corrupt the model in a number of ways and hacks result.) Encourage people to use technically precise language. When reporting or discussing problems, at the IP layer at least, always get the answers to these questions: - What is the source IP addresses (including external NAT IP if applicable)? - What is the destination IP address - What is the application being used? - What is the error message or behavior if any? - Did this ever work and, if so, what date and time did it stop working? Tony On Wed, Feb 15, 2012 at 10:26 PM, Charles Mills <w3yni1@gmail.com> wrote:
Not understanding RFC1918. Actually got read the riot act by someone because I worked for an organization that used 10.0.0.0/8 and that was "their" network and "they" owned it.
Chuck
In message <4F3C76D5.9040603@necom830.hpcl.titech.ac.jp>, Masataka Ohta writes:
Mark Andrews wrote:
This doesn't prove that IPv6 is not operational. All it proves is people can misconfigure things.
How do operators configure their equipments to treat ICMP packet too big generated against multicast and unicast?
Well you need to go out of your way to get a ICMP PTB for IPv6 multicast as the default is to fragment multicast packets at the source at network minimum mtu (RFC3542 - May 2003). That's not to say it won't happen. As for generation of PTB you rate limit them the way you do for IPv4.
Note that, even if they do not enable inter-subnet multicast in their domains, the ICMP packets may still transit over or implode within their domains.
Note also that some network processors can't efficiently distinguish ICMP packets generated against multicast and unicast.
And why do you need to distingish them? You look at the inner packet not the ICMP source if you want to rate limit return traffic.
Masataka Ohta
-- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
Mark Andrews wrote:
Well you need to go out of your way to get a ICMP PTB for IPv6 multicast as the default is to fragment multicast packets at the source at network minimum mtu (RFC3542 - May 2003). That's not to say it won't happen.
Yes, it will happen, because RFC3542 was, as was discussed in IETF, written not to prohibit multicast PMTUD. So, the problem is real.
As for generation of PTB you rate limit them the way you do for IPv4.
A problem is that a lot of ICMP packet too big against unicast is generated, because PMTUD requires hosts periodically try to send a packet a little larger than the current PMTU. BTW, that's why IPv6, which inhibit fragmentation by routers, is no better than IPv4 with fragmentation enabled, because, periodic generation of ICMP packet too big by routers is as painful as periodic fragmentation by routers.
Note also that some network processors can't efficiently distinguish ICMP packets generated against multicast and unicast.
And why do you need to distingish them?
We don't need to. Instead, we can just give up to use PMTUD entirely and just send packets of 1280B or less. A problem is that a tunnel over 1280B PMTU must always fragment 1280B payload.
You look at the inner packet not the ICMP source if you want to rate limit return traffic.
That is a possible problem. Destination address of inner packet is located far inside of the ICMP (beyond 64B) that it can not be used for intrinsic filtering capability of some network processors. Masataka Ohta
On 2012.02.15 22:12, Mark Andrews wrote:
In message<4F3C6703.4050607@gmail.com>, Steve Bertrand writes:
On 2012.02.15 19:55, Nathan Eisenberg wrote:
IPv6 is operational.
How is this a misconception? It works fine for me...
Imagine an operator who is v6 ignorant, with a home provider who implements v6 half-assed, and tries to access a v6 site that has perhaps v6-only accessible nameservers, when their provider who 'offers' v6 has resolvers that operate only over v4.
*huge* misconception about the operational status of IPv6 (imho).
This doesn't prove that IPv6 is not operational. All it proves is people can misconfigure things. If you provide the recursive nameservers with IPv6 access they will make queries over IPv6 even if they only accept queries over IPv4.
If you want to know if your resolver talks IPv6 to the world and supports 4096 EDNS UDP messages the following query will tell you.
dig edns-v6-ok.isc.org txt
Similarly for IPv4.
dig edns-v4-ok.isc.org txt
Thank you :) Steve
If you want to know if your resolver talks IPv6 to the world and supports 4096 EDNS UDP messages the following query will tell you.
dig edns-v6-ok.isc.org txt
Similarly for IPv4.
dig edns-v4-ok.isc.org txt
Both PowerDNS recursor 3.3 and Nominum CNS 3.0.5 have problems with these queries. They both get the TC answer from 149.20.64.58 / 2001:4f8:0:2::8. Then: - CNS tries with 4000 EDNS UDP size (4000 is the CNS documented max UDP size), gets another TC. - PowerDNS doesn't try to used EDNS at all. Then they both try TCP and get a RST. And then they return SERVFAIL. Steinar Haug, Nethelp consulting, sthaug@nethelp.no
In message <20120216.130143.74691634.sthaug@nethelp.no>, sthaug@nethelp.no writes:
If you want to know if your resolver talks IPv6 to the world and supports 4096 EDNS UDP messages the following query will tell you.
dig edns-v6-ok.isc.org txt
Similarly for IPv4.
dig edns-v4-ok.isc.org txt
Both PowerDNS recursor 3.3 and Nominum CNS 3.0.5 have problems with these queries. They both get the TC answer from 149.20.64.58 / 2001:4f8:0:2::8. Then:
I stated very clearly the conditions under which the queries would resolve.
- CNS tries with 4000 EDNS UDP size (4000 is the CNS documented max UDP size), gets another TC.
- PowerDNS doesn't try to used EDNS at all.
Then they both try TCP and get a RST. And then they return SERVFAIL.
Correct. Those servers are deliberately configured to not answer TCP as they are for testing the EDNS UDP path. They also put out a answer that will exactly fill a 4096 byte EDNS UDP message which is the default and largest EDNS UDP size advertised by named. This allows someone running named to test their firewall configuration to ensure that it will let through any EDNS UDP reply, size wise, that can occur. As IPv4 and IPv6 are often configured independently we provide a way to test each independently.
Steinar Haug, Nethelp consulting, sthaug@nethelp.no -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
On 2012-02-16 14:51 , Mark Andrews wrote: [..]
that can occur. As IPv4 and IPv6 are often configured independently we provide a way to test each independently.
Could you make that label also for both IPv4/IPv6, that way, one could figure out if the query ends up being answered over IPv4 or IPv6... though then maybe the content would need to have a IPv4/IPv6 label in it. Maybe it is even more useful to show the source of the querier? Greets, Jeroen
In message <4F3D0C45.9020100@unfix.org>, Jeroen Massar writes:
On 2012-02-16 14:51 , Mark Andrews wrote: [..]
that can occur. As IPv4 and IPv6 are often configured independently we provide a way to test each independently.
Could you make that label also for both IPv4/IPv6, that way, one could figure out if the query ends up being answered over IPv4 or IPv6... though then maybe the content would need to have a IPv4/IPv6 label in it. Maybe it is even more useful to show the source of the querier?
Greets, Jeroen
I don't really see much benefit in that other than satisfying curiosity. Also this is a stock stand server + firewall to block TCP. It would require a custom server to sent back the source address and that requires getting ops to run it. That's not to say that it can't be done but it becomes more than a simple request. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
Mark Andrews (marka) writes:
If you want to know if your resolver talks IPv6 to the world and supports 4096 EDNS UDP messages the following query will tell you.
dig edns-v6-ok.isc.org txt
Similarly for IPv4.
dig edns-v4-ok.isc.org txt
9.8.1P1 on a dual stacked native v6 host: I'm seeing TC on both answers, the difference is the TCP answer comes through on v4 but v6 gives SERVFAIL. Don't see any v6 fragments (that'd be a problem since PF doesn't handle them on this host). P.
In message <20120216134437.GB65401@macbook.bluepipe.net>, Phil Regnauld writes:
Mark Andrews (marka) writes:
If you want to know if your resolver talks IPv6 to the world and supports 4096 EDNS UDP messages the following query will tell you.
dig edns-v6-ok.isc.org txt
Similarly for IPv4.
dig edns-v4-ok.isc.org txt
9.8.1P1 on a dual stacked native v6 host: I'm seeing TC on both answers, the difference is the TCP answer comes through on v4 but v6 gives SERVFAIL.
You will see TC between dig and the resolver. If you see TC between the resolver and the server it will fail as neither server answers over TCP. If you are seeing TC between the resolver and the server and the TCP query is being answers then something in the path is intercepting the DNS queries.
Don't see any v6 fragments (that'd be a problem since PF doesn't handle them on this host).
You should see something like this on the wire. The second query is to answer dig's query over TCP. 01:19:30.421959 IP6 2001:470:1f00:820:6965:eba7:eff6:1242.64345 > 2001:4f8:0:2::8.53: 44698% [1au] TXT? edns-v6-ok.isc.org. (47) 01:19:30.591828 IP6 2001:4f8:0:2::8 > 2001:470:1f00:820:6965:eba7:eff6:1242: frag (0|1232) 53 > 64345: 44698*- 1/0/1 TXT[|domain] 01:19:30.592851 IP6 2001:4f8:0:2::8 > 2001:470:1f00:820:6965:eba7:eff6:1242: frag (1232|1232) 01:19:30.593889 IP6 2001:4f8:0:2::8 > 2001:470:1f00:820:6965:eba7:eff6:1242: frag (2464|1232) 01:19:30.593963 IP6 2001:4f8:0:2::8 > 2001:470:1f00:820:6965:eba7:eff6:1242: frag (3696|408) 01:19:30.596552 IP6 2001:470:1f00:820:6965:eba7:eff6:1242.61500 > 2001:4f8:0:2::8.53: 60740% [1au] TXT? edns-v6-ok.isc.org. (47) 01:19:30.767351 IP6 2001:4f8:0:2::8 > 2001:470:1f00:820:6965:eba7:eff6:1242: frag (0|1232) 53 > 61500: 60740*- 1/0/1 TXT[|domain] 01:19:30.768362 IP6 2001:4f8:0:2::8 > 2001:470:1f00:820:6965:eba7:eff6:1242: frag (1232|1232) 01:19:30.769399 IP6 2001:4f8:0:2::8 > 2001:470:1f00:820:6965:eba7:eff6:1242: frag (2464|1232) 01:19:30.769473 IP6 2001:4f8:0:2::8 > 2001:470:1f00:820:6965:eba7:eff6:1242: frag (3696|408)
P. -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
Borderline dns-ops, sorry folks! - but this is interesting as we've been talking about ipv6 being operational, and this is part of it... Mark Andrews (marka) writes:
If you are seeing TC between the resolver and the server and the TCP query is being answers then something in the path is intercepting the DNS queries.
TC is on the answer from the remote server to my resolver, so yeah, seems like something is messing with the packets.
Don't see any v6 fragments (that'd be a problem since PF doesn't handle them on this host).
You should see something like this on the wire. The second query is to answer dig's query over TCP.
I'm not seeing fragments as you are. Here's what I see: 14:40:20.955876 IP6 2001:2000:1080:d::2.64561 > 2001:4f8:0:2::8.53: 52841 TXT? edns-v6-ok.isc.org. (36) 14:40:21.141948 IP6 2001:4f8:0:2::8.53 > 2001:2000:1080:d::2.64561: 52841*-| 0/0/0 (36) 14:40:21.142259 IP6 2001:2000:1080:d::2.53262 > 2001:4f8:0:2::8.53: Flags [S], seq 1112939462, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 2571957531 ecr 0], length 0 14:40:21.327895 IP6 2001:4f8:0:2::8.53 > 2001:2000:1080:d::2.53262: Flags [R.], seq 0, ack 1112939463, win 0, length 0 Cheers, Phil
In message <20120216165308.GE65401@macbook.bluepipe.net>, Phil Regnauld writes:
Borderline dns-ops, sorry folks! - but this is interesting as we've been talking about ipv6 being operational, and this is part of it...
Mark Andrews (marka) writes:
If you are seeing TC between the resolver and the server and the TCP query is being answers then something in the path is intercepting the DNS queries.
TC is on the answer from the remote server to my resolver, so yeah, seems like something is messing with the packets.
Don't see any v6 fragments (that'd be a problem since PF doesn't handle them on this host).
You should see something like this on the wire. The second query is to answer dig's query over TCP.
I'm not seeing fragments as you are.
Here's what I see:
14:40:20.955876 IP6 2001:2000:1080:d::2.64561 > 2001:4f8:0:2::8.53: 52841 TXT? edns-v6-ok.isc.org. (36) 14:40:21.141948 IP6 2001:4f8:0:2::8.53 > 2001:2000:1080:d::2.64561: 52841*-| 0/0/0 (36) 14:40:21.142259 IP6 2001:2000:1080:d::2.53262 > 2001:4f8:0:2::8.53: Flags [S], seq 1112939462, win 65535, optio ns [mss 1440,nop,wscale 6,sackOK,TS val 2571957531 ecr 0], length 0 14:40:21.327895 IP6 2001:4f8:0:2::8.53 > 2001:2000:1080:d::2.53262: Flags [R.], seq 0, ack 1112939463, win 0, l ength 0
Which means you are seeing named in fallback mode, or have configured named to not take EDNS to this server. In anycase your firewall is misconfigured/broken if it is blocking fragments.
Cheers, Phil -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
Seems like dig doesn't always advertise a big enough buffer, I was having the same issue as you. If you set the buffer size on the command line it works as directed. Daniels-Mac-mini:~ daniel$ dig edns-v4-ok.isc.org txt @149.20.64.58 ;; Truncated, retrying in TCP mode. ;; Connection to 149.20.64.58#53(149.20.64.58) for edns-v4-ok.isc.orgfailed: connection refused. Daniels-Mac-mini:~ daniel$ dig edns-v4-ok.isc.org txt @149.20.64.58+bufsize=4096 ; <<>> DiG 9.7.3-P3 <<>> edns-v4-ok.isc.org txt @149.20.64.58 +bufsize=4096 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18209 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;edns-v4-ok.isc.org. IN TXT ;; ANSWER SECTION: edns-v4-ok.isc.org. 0 IN TXT "EDNS-4096-OK" "EDNS-4096-OK" "EDNS-4096-OK" "EDNS-4096-OK" "EDNS-4096-OK" "EDNS-4096-OK" <snip> "EDNS-4" ;; Query time: 176 msec ;; SERVER: 149.20.64.58#53(149.20.64.58) ;; WHEN: Fri Feb 17 10:22:08 2012 ;; MSG SIZE rcvd: 4096 On 17 February 2012 05:53, Phil Regnauld <regnauld@nsrc.org> wrote:
Borderline dns-ops, sorry folks! - but this is interesting as we've been talking about ipv6 being operational, and this is part of it...
Mark Andrews (marka) writes:
If you are seeing TC between the resolver and the server and the TCP
query is being answers then
something in the path is intercepting the DNS queries.
TC is on the answer from the remote server to my resolver, so yeah, seems like something is messing with the packets.
Don't see any v6 fragments (that'd be a problem since PF doesn't
handle
them on this host).
You should see something like this on the wire. The second query is to answer dig's query over TCP.
I'm not seeing fragments as you are.
Here's what I see:
14:40:20.955876 IP6 2001:2000:1080:d::2.64561 > 2001:4f8:0:2::8.53: 52841 TXT? edns-v6-ok.isc.org. (36) 14:40:21.141948 IP6 2001:4f8:0:2::8.53 > 2001:2000:1080:d::2.64561: 52841*-| 0/0/0 (36) 14:40:21.142259 IP6 2001:2000:1080:d::2.53262 > 2001:4f8:0:2::8.53: Flags [S], seq 1112939462, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 2571957531 ecr 0], length 0 14:40:21.327895 IP6 2001:4f8:0:2::8.53 > 2001:2000:1080:d::2.53262: Flags [R.], seq 0, ack 1112939463, win 0, length 0
Cheers, Phil
-- Daniel Griggs Network Operations e: daniel@fx.net.nz d: +64 4 4989567
In message <CA+ycCUOoLgwAMUn_aSBf8FFiPczWmt2oo7T45jOnqthJWx+xpg@mail.gmail.com>, Daniel Griggs writes:
--001636c5b8ca93b4eb04b91b7066 Content-Type: text/plain; charset=ISO-8859-1
Seems like dig doesn't always advertise a big enough buffer, I was having the same issue as you. If you set the buffer size on the command line it works as directed.
Well you were supposed to ask your recursive server, not ask the authoritative server directly. We were talking about testing the path from the recursive server (which you may not have log in access to) to the authoritative server. If you want to ask the authoritative server directly then +edns=0 or +dnssec or +bufsize=4096 or you can use dig from BIND 9.9.0 which sets the ad flag and enables edns (version 0) by default. % dig edns-v6-ok.isc.org txt ;; Truncated, retrying in TCP mode. ; <<>> DiG 9.7.3-P3 <<>> edns-v6-ok.isc.org txt ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46198 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1 ;; QUESTION SECTION: ;edns-v6-ok.isc.org. IN TXT ;; ANSWER SECTION: edns-v6-ok.isc.org. 0 IN TXT "EDNS-4096-OK" "EDNS-4096-OK" <snipped> ;; AUTHORITY SECTION: edns-v6-ok.isc.org. 7199 IN NS edns-v6-ok.isc.org. ;; ADDITIONAL SECTION: edns-v6-ok.isc.org. 7199 IN AAAA 2001:4f8:0:2::8 ;; Query time: 174 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Fri Feb 17 09:36:37 2012 ;; MSG SIZE rcvd: 4127 %
Daniels-Mac-mini:~ daniel$ dig edns-v4-ok.isc.org txt @149.20.64.58 ;; Truncated, retrying in TCP mode. ;; Connection to 149.20.64.58#53(149.20.64.58) for edns-v4-ok.isc.orgfailed: connection refused. Daniels-Mac-mini:~ daniel$ dig edns-v4-ok.isc.org txt @149.20.64.58+bufsize=4096
; <<>> DiG 9.7.3-P3 <<>> edns-v4-ok.isc.org txt @149.20.64.58 +bufsize=4096 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18209 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;edns-v4-ok.isc.org. IN TXT
;; ANSWER SECTION: edns-v4-ok.isc.org. 0 IN TXT "EDNS-4096-OK" "EDNS-4096-OK" "EDNS-4096-OK" "EDNS-4096-OK" "EDNS-4096-OK" "EDNS-4096-OK" <snip> "EDNS-4"
;; Query time: 176 msec ;; SERVER: 149.20.64.58#53(149.20.64.58) ;; WHEN: Fri Feb 17 10:22:08 2012 ;; MSG SIZE rcvd: 4096
On 17 February 2012 05:53, Phil Regnauld <regnauld@nsrc.org> wrote:
Borderline dns-ops, sorry folks! - but this is interesting as we've been talking about ipv6 being operational, and this is part of it...
Mark Andrews (marka) writes:
If you are seeing TC between the resolver and the server and the TCP
query is being answers then
something in the path is intercepting the DNS queries.
TC is on the answer from the remote server to my resolver, so yeah, seems like something is messing with the packets.
Don't see any v6 fragments (that'd be a problem since PF doesn't
handle
them on this host).
You should see something like this on the wire. The second query is to answer dig's query over TCP.
I'm not seeing fragments as you are.
Here's what I see:
14:40:20.955876 IP6 2001:2000:1080:d::2.64561 > 2001:4f8:0:2::8.53: 52841 TXT? edns-v6-ok.isc.org. (36) 14:40:21.141948 IP6 2001:4f8:0:2::8.53 > 2001:2000:1080:d::2.64561: 52841*-| 0/0/0 (36) 14:40:21.142259 IP6 2001:2000:1080:d::2.53262 > 2001:4f8:0:2::8.53: Flags [S], seq 1112939462, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 2571957531 ecr 0], length 0 14:40:21.327895 IP6 2001:4f8:0:2::8.53 > 2001:2000:1080:d::2.53262: Flags [R.], seq 0, ack 1112939463, win 0, length 0
Cheers, Phil
-- Daniel Griggs Network Operations e: daniel@fx.net.nz d: +64 4 4989567
--001636c5b8ca93b4eb04b91b7066 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
149.20.64.58</a> +bufsize=3D4096<br>;; global options: +cmd<br>;; Got answ= er:<br> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18209<br>;;= flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1<br>;; WA= RNING: recursion requested but not available<br><br>;; OPT PSEUDOSECTION:<b= r> ; EDNS: version: 0, flags:; udp: 4096<br>;; QUESTION SECTION:<br>;<a href= =3D"http://edns-v4-ok.isc.org">edns-v4-ok.isc.org</a>.=A0=A0=A0 =A0=A0=A0 I= N=A0=A0=A0 TXT<br><br>;; ANSWER SECTION:<br><a href=3D"http://edns-v4-ok.is= c.org">edns-v4-ok.isc.org</a>.=A0=A0=A0 0=A0=A0=A0 IN=A0=A0=A0 TXT=A0=A0=A0= "EDNS-4096-OK" "EDNS-4096-OK" "EDNS-4096-OK"= "EDNS-4096-OK" "EDNS-4096-OK" "EDNS-4096-OK"= <br> <snip><br>"EDNS-4"<br><br>;; Query time: 176 msec<br>;; SER= VER: 149.20.64.58#53(149.20.64.58)<br>;; WHEN: Fri Feb 17 10:22:08 2012<br>= ;; MSG SIZE=A0 rcvd: 4096<br><br><br><br><br><div class=3D"gmail_quote">On = 17 February 2012 05:53, Phil Regnauld <span dir=3D"ltr"><<a href=3D"mail= to:regnauld@nsrc.org">regnauld@nsrc.org</a>></span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> =A0 =A0 =A0 =A0Borderline dns-ops, sorry fo= lks! - but this is interesting<br> =A0 =A0 =A0 =A0as we've been talking about ipv6 being operational, and=
<br>Seems like dig doesn't always advertise a big enough buffer, I was = having the same issue as you. If you set the buffer size on the command lin= e it works as directed.<br><br>Daniels-Mac-mini:~ daniel$ dig <a href=3D"ht= tp://edns-v4-ok.isc.org">edns-v4-ok.isc.org</a> txt @<a href=3D"http://149.= 20.64.58">149.20.64.58</a><br> ;; Truncated, retrying in TCP mode.<br>;; Connection to 149.20.64.58#53(149= .20.64.58) for <a href=3D"http://edns-v4-ok.isc.org">edns-v4-ok.isc.org</a>= failed: connection refused.<br>Daniels-Mac-mini:~ daniel$ dig <a href=3D"h= ttp://edns-v4-ok.isc.org">edns-v4-ok.isc.org</a> txt @<a href=3D"http://149= .20.64.58">149.20.64.58</a> +bufsize=3D4096<br> <br>; <<>> DiG 9.7.3-P3 <<>> <a href=3D"http://edns= -v4-ok.isc.org">edns-v4-ok.isc.org</a> txt @<a href=3D"http://149.20.64.58"= this<br> =A0 =A0 =A0 =A0is part of it...<br> <div class=3D"im"><br> Mark Andrews (marka) writes:<br> ><br> > If you are seeing TC between the resolver and the server and the TCP q= uery is being answers then<br> > something in the path is intercepting the DNS queries.<br> <br> </div> =A0 =A0 =A0 =A0TC is on the answer from the remote server to my reso= lver, so yeah, seems<br> =A0 =A0 =A0 =A0like something is messing with the packets.<br> <div class=3D"im"><br> > > =A0 =A0 Don't see any v6 fragments (that'd be a problem s= ince PF doesn't handle<br> > > =A0 =A0 them on this host).<br> ><br> > You should see something like this on the wire. =A0The second query is= to answer<br> > dig's query over TCP.<br> <br> </div> =A0 =A0 =A0 =A0I'm not seeing fragments as you are.<br> <br> =A0 =A0 =A0 =A0Here's what I see:<br> <br> 14:40:20.955876 IP6 2001:2000:1080:d::2.64561 > 2001:4f8:0:2::8.53: 5284= 1 TXT? <a href=3D"http://edns-v6-ok.isc.org" target=3D"_blank">edns-v6-ok.i= sc.org</a>. (36)<br> 14:40:21.141948 IP6 2001:4f8:0:2::8.53 > 2001:2000:1080:d::2.64561: 5284= 1*-| 0/0/0 (36)<br> 14:40:21.142259 IP6 2001:2000:1080:d::2.53262 > 2001:4f8:0:2::8.53: Flag= s [S], seq 1112939462, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS = val 2571957531 ecr 0], length 0<br> 14:40:21.327895 IP6 2001:4f8:0:2::8.53 > 2001:2000:1080:d::2.53262: Flag= s [R.], seq 0, ack 1112939463, win 0, length 0<br> <br> =A0 =A0 =A0 =A0Cheers,<br> =A0 =A0 =A0 =A0Phil<br> <br> </blockquote></div><br><br clear=3D"all"><br>-- <br>Daniel Griggs<br>Networ= k Operations<br>e: <a href=3D"mailto:daniel@fx.net.nz" target=3D"_blank">da= niel@fx.net.nz</a><br>d: +64 4 4989567<br>
--001636c5b8ca93b4eb04b91b7066-- -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
On Feb 15, 2012, at 6:16 PM, Steve Bertrand wrote:
On 2012.02.15 19:55, Nathan Eisenberg wrote:
IPv6 is operational.
How is this a misconception? It works fine for me...
Imagine an operator who is v6 ignorant, with a home provider who implements v6 half-assed, and tries to access a v6 site that has perhaps v6-only accessible nameservers, when their provider who 'offers' v6 has resolvers that operate only over v4.
*huge* misconception about the operational status of IPv6 (imho).
Steve
By that definition, IPv4 is non-operational. You can break anything if you try hard enough. Owen
On 16/02/2012 07:45, Owen DeLong wrote:
On Feb 15, 2012, at 6:16 PM, Steve Bertrand wrote:
On 2012.02.15 19:55, Nathan Eisenberg wrote:
IPv6 is operational.
How is this a misconception? It works fine for me...
Imagine an operator who is v6 ignorant, with a home provider who implements v6 half-assed, and tries to access a v6 site that has perhaps v6-only accessible nameservers, when their provider who 'offers' v6 has resolvers that operate only over v4.
*huge* misconception about the operational status of IPv6 (imho).
Steve
By that definition, IPv4 is non-operational.
You can break anything if you try hard enough.
This being well demonstrated by most of the "Internet" access provided by hotels, for example. -- Paul
On Feb 15, 2012, at 7:55 PM, Nathan Eisenberg wrote:
IPv6 is operational.
How is this a misconception? It works fine for me...
I think he left off "In Japan". There's been a lot of local politics as it relates to the broken nature of IPv6 in japan. When its there, it's not globally accessible in many cases (at the consumer or last-mile level). Most (all?) major backbones are IPv6 capable these days, but in some cases it's 6PE vs "native". IPv6 is operational and does work, but like any protocol there are issues. If you are unaware, take a look at what people are trying to put into IPv4 still at IETF. The fact that the IPv6 day went so well last year, and the IPv6 launch is coming quickly is a reminder its real. Me? I can't wait to have this behind us. (Oh, and if you're not at least routing your IPv6 space to your lab/NOC LAN, get on it. Even if you have to poke the 'security' guys who think you need an IPv6 NAT in the eye). - Jared
On Feb 16, 2012, at 4:31 AM, Jared Mauch wrote:
On Feb 15, 2012, at 7:55 PM, Nathan Eisenberg wrote:
IPv6 is operational.
How is this a misconception? It works fine for me...
I think he left off "In Japan". There's been a lot of local politics as it relates to the broken nature of IPv6 in japan. When its there, it's not globally accessible in many cases (at the consumer or last-mile level). Most (all?) major backbones are IPv6 capable these days, but in some cases it's 6PE vs "native".
IPv6 is operational and does work, but like any protocol there are issues. If you are unaware, take a look at what people are trying to put into IPv4 still at IETF. The fact that the IPv6 day went so well last year, and the IPv6 launch is coming quickly is a reminder its real. Me? I can't wait to have this behind us. (Oh, and if you're not at least routing your IPv6 space to your lab/NOC LAN, get on it. Even if you have to poke the 'security' guys who think you need an IPv6 NAT in the eye).
- Jared
Yes, I'm well aware of the problems being created by the attempts by NTT to force the government to let them be a residential ISP. Owen
I help with networking curriculum and labs here at the University of Maine, especially for network security. There seems to be (even among faculty) a gross misunderstanding of Layer-2. Nearly every textbook starts with IP, and talks about it as if we were 20 years in the past. I've found starting off with some history on Ethernet (Maine loves Bob Metcalfe) becomes a very solid base for understanding; how "Ethernet" today is very different; starting with hubs, bridges, collisions, and those problems, then introducing modern switching, VLANs, broadcast domain's etc. Then expanding on that by introducing Layer-3 starting with its relationship to L-2 (ARP, how packets are manipulated when a host makes the determination if a packet is on link or needs to be routed, etc). On Thu, Feb 16, 2012 at 8:03 AM, Owen DeLong <owen@delong.com> wrote:
On Feb 16, 2012, at 4:31 AM, Jared Mauch wrote:
On Feb 15, 2012, at 7:55 PM, Nathan Eisenberg wrote:
IPv6 is operational.
How is this a misconception? It works fine for me...
I think he left off "In Japan". There's been a lot of local politics as it relates to the broken nature of IPv6 in japan. When its there, it's not globally accessible in many cases (at the consumer or last-mile level). Most (all?) major backbones are IPv6 capable these days, but in some cases it's 6PE vs "native".
IPv6 is operational and does work, but like any protocol there are issues. If you are unaware, take a look at what people are trying to put into IPv4 still at IETF. The fact that the IPv6 day went so well last year, and the IPv6 launch is coming quickly is a reminder its real. Me? I can't wait to have this behind us. (Oh, and if you're not at least routing your IPv6 space to your lab/NOC LAN, get on it. Even if you have to poke the 'security' guys who think you need an IPv6 NAT in the eye).
- Jared
Yes, I'm well aware of the problems being created by the attempts by NTT to force the government to let them be a residential ISP.
Owen
-- Ray Soucy Epic Communications Specialist Phone: +1 (207) 561-3526 Networkmaine, a Unit of the University of Maine System http://www.networkmaine.net/
On 2/16/2012 8:17 AM, Ray Soucy wrote:
I've found starting off with some history on Ethernet (Maine loves Bob Metcalfe) becomes a very solid base for understanding; how "Ethernet" today is very different; starting with hubs, bridges, collisions, and those problems, then introducing modern switching, VLANs, broadcast domain's etc.
It's a bit dated (1998) but I always thought Rich Siefert covered "the basics" very well... http://www.amazon.com/Gigabit-Ethernet-Technology-Applications-High-Speed/dp... Jeff
On Thu, Feb 16, 2012 at 08:27:14AM -0500, Jeff Kell wrote:
On 2/16/2012 8:17 AM, Ray Soucy wrote:
I've found starting off with some history on Ethernet (Maine loves Bob Metcalfe) becomes a very solid base for understanding; how "Ethernet" today is very different; starting with hubs, bridges, collisions, and those problems, then introducing modern switching, VLANs, broadcast domain's etc.
It's a bit dated (1998) but I always thought Rich Siefert covered "the basics" very well... http://www.amazon.com/Gigabit-Ethernet-Technology-Applications-High-Speed/dp...
I like this free Juniper online training to introduce people to Layer 2 and Layer 3 and how they interact: https://learningportal.juniper.net/juniper/user_fasttrack_home.aspx "Networking Fundamentals" eLearning course.
On 2/16/2012 7:17 AM, Ray Soucy wrote:
There seems to be (even among faculty) a gross misunderstanding of Layer-2. Nearly every textbook starts with IP, and talks about it as if we were 20 years in the past.
Understanding all layers and how they can interact stacked within layers is a big issue. Granted, they aren't coming out of school, but I've seen old Sonet/TDM guys trying to figure out transport of Ethernet and it has been a nightmare on dealing with terminology. It at first started with trying to explain that vlan based switching is not Layer-3. :( Use your imagination when they finally got into MPLS, which kindly takes the OSI model like a flat piece of paper and wads it up. Jack
On Feb 16, 2012, at 18:08, Jack Bates wrote:
It at first started with trying to explain that vlan based switching is not Layer-3. :(
Ah, one of the greatest misconceptions still around in 2012: -- OSI Layer numbers mean something. or -- Somewhere in the sky, there is an exact definition of what is layer 2, layer 3, layer 4, layer 5 (!), layer 7 or -- my definition is righter than yours Grüße, Carsten
On 2/16/2012 8:30 PM, Carsten Bormann wrote:
On Feb 16, 2012, at 18:08, Jack Bates wrote:
It at first started with trying to explain that vlan based switching is not Layer-3. :( Ah, one of the greatest misconceptions still around in 2012:
-- OSI Layer numbers mean something. or -- Somewhere in the sky, there is an exact definition of what is layer 2, layer 3, layer 4, layer 5 (!), layer 7 or -- my definition is righter than yours
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. Paul
On Feb 17, 2012, at 07:50, Paul Graydon wrote:
what OSI means
Yet another common misconception popping up: -- You can talk about the OSI model in the present tense (That said -- yes, it is still useful as a set of simple terms for certain combinations of functions. It is also still useful as a way to calibrate your gut feeling of what is going on in a network. Just never expect OSI terms to have a precise meaning in today's networks. 1978 is now a third of a century ago... If you need precision, you need to spell out what you mean in today's terms.) Grüße, Carsten
This list is awesome. Is anyone consolidating it? I'm still catching up on the thread.... -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 1:05 AM, Carsten Bormann wrote:
On Feb 17, 2012, at 07:50, Paul Graydon wrote:
what OSI means Yet another common misconception popping up:
-- You can talk about the OSI model in the present tense
(That said -- yes, it is still useful as a set of simple terms for certain combinations of functions. It is also still useful as a way to calibrate your gut feeling of what is going on in a network. Just never expect OSI terms to have a precise meaning in today's networks. 1978 is now a third of a century ago... If you need precision, you need to spell out what you mean in today's terms.)
Grüße, Carsten
If you do, please share it. Thank you. -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 9:36 AM, Jared Mauch wrote:
On Feb 17, 2012, at 9:29 AM, -Hammer- wrote:
This list is awesome. Is anyone consolidating it? I'm still catching up on the thread....
I was thinking of making a checklist out of it.
- Jared
Original poster who started thread said he would. On Fri, Feb 17, 2012 at 10:51 AM, -Hammer- <bhmccie@gmail.com> wrote:
If you do, please share it. Thank you.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 9:36 AM, Jared Mauch wrote:
On Feb 17, 2012, at 9:29 AM, -Hammer- wrote:
This list is awesome. Is anyone consolidating it? I'm still catching up
on the thread....
I was thinking of making a checklist out of it.
- Jared
On Fri, 17 Feb 2012 08:29:42 -0600 -Hammer- <bhmccie@gmail.com> wrote:
This list is awesome. Is anyone consolidating it? I'm still catching up on the thread....
I'm collecting all responses, many of which have been sent to me off list. I was waiting for the thread to eventually end before summarizing it. Thanks everyone. John
On 2/17/2012 1:05 AM, Carsten Bormann wrote:
On Feb 17, 2012, at 07:50, Paul Graydon wrote:
what OSI means
Yet another common misconception popping up:
-- You can talk about the OSI model in the present tense
(That said -- yes, it is still useful as a set of simple terms for certain combinations of functions. It is also still useful as a way to calibrate your gut feeling of what is going on in a network. Just never expect OSI terms to have a precise meaning in today's networks. 1978 is now a third of a century ago... If you need precision, you need to spell out what you mean in today's terms.)
Actually, I find it makes a perfect troubleshooting guideline in today's world; at least up to layer 4. I'm not saying it's a perfect match to troubleshooting a variety of MPLS problems, but it is a reminder that there are dependencies which must be checked. In dealing with transport companies, the model is still a good representation of their service levels. It isn't uncommon to find their products defined as layer 2 services (ranging from tdm/sonet services to ethernet switching services), layer 3 services (often handled by their ISP department), and MPLS services (which can range from p2p transport to l3vpn). Which brings up my final point. Until we quit naming things l2vpn or l3vpn, OSI still applies. :P Jack
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc.
I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken. The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times. I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
Actually I taught in a three year tech program for a while and although trouble shooting was not in the curriculum, and as far as I know it isn't anywhere, several of us adjunct faculty did teach it and got reprimanded for it as part of our classes. So much for the education industry understanding the needs of business. I taught basic PC hardware and NT networking at the time. We would actually have the students assemble a PC and then in a subsequent class bring up a network. I got pretty good at nailing then with bugs while they were on breaks. Heck, they had to go out to smoke. They would come back with a network or PC that was no longer working. I would then have them explain what they saw, what they thought was wrong, justify it BEFORE they could take any corrective action. I also used some classroom scenarios - they could ask me anything that they could physically learn if they could tell me how they would check that. I let them run bad rabbit trails if it looked like it would cement the right way. It taught some step by step processes. BTW, the best trouble shooting course I have ever taken was the Kepner Trego Problem Analysis/Decision Analysis class. Caterpillar used it but I have not seen it run anywhere in years. It is hard-nosed and may not be glitzy enough for today. If you have a junior tech who isn't getting there, I suggest - get their book and see if it helps. NO, I do not sell them or have stock in the company and NO, it will not do any good unless he reads it. I still use methodologies I learned in the class. Ralph Brandt Communications Engineer HP Enterprise Services Telephone +1 717.506.0802 FAX +1 717.506.4358 Email Ralph.Brandt@pateam.com 5095 Ritter Rd Mechanicsburg PA 17055 -----Original Message----- From: Leo Bicknell [mailto:bicknell@ufp.org] Sent: Friday, February 17, 2012 9:29 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc.
I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken. The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times. I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
Let me simplify that. If you are over 35 you know how to troubleshoot. Yes, I'm going to get flamed. Yes, there are exceptions in both directions. -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of
Hammer, you are at least 75% right. You will get flamed and in most cases, the 35 year age is close to right. But then in Programming where I spent most of my IT time since Feb 1963, few current programmers have skills that they need to be really successful. Same thing. It is the fault of the educational system like one school district here that teaches Alice, VB and then two days of C++ to High School Kids. Heck, they will fiddle with Alice on their own. They need some exposure to one of the SQL's and how to build some tables, maybe a good script language, some command line on SQL+ and unix or PostgresSQL and linux if the school can't afford the unix licenses. The fun and games is more important than the substance and it goes into the colleges in spades. BTW, I am a school board member who votes 1:8 often on things.... But let me give you a perspective, one of the board members called Golf an "Essential Life Skill." Maybe, but how about balancing a checkbook... Ralph Brandt Communications Engineer HP Enterprise Services Telephone +1 717.506.0802 FAX +1 717.506.4358 Email Ralph.Brandt@pateam.com 5095 Ritter Rd Mechanicsburg PA 17055 -----Original Message----- From: -Hammer- [mailto:bhmccie@gmail.com] Sent: Friday, February 17, 2012 9:52 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions Let me simplify that. If you are over 35 you know how to troubleshoot. Yes, I'm going to get flamed. Yes, there are exceptions in both directions. -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 8:29 AM, Leo Bicknell wrote: that
should be done in a group enviornment.
Well said. An American tragedy. -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 9:01 AM, Brandt, Ralph wrote:
Hammer, you are at least 75% right. You will get flamed and in most cases, the 35 year age is close to right.
But then in Programming where I spent most of my IT time since Feb 1963, few current programmers have skills that they need to be really successful. Same thing.
It is the fault of the educational system like one school district here that teaches Alice, VB and then two days of C++ to High School Kids. Heck, they will fiddle with Alice on their own. They need some exposure to one of the SQL's and how to build some tables, maybe a good script language, some command line on SQL+ and unix or PostgresSQL and linux if the school can't afford the unix licenses.
The fun and games is more important than the substance and it goes into the colleges in spades.
BTW, I am a school board member who votes 1:8 often on things.... But let me give you a perspective, one of the board members called Golf an "Essential Life Skill." Maybe, but how about balancing a checkbook...
Ralph Brandt Communications Engineer HP Enterprise Services Telephone +1 717.506.0802 FAX +1 717.506.4358 Email Ralph.Brandt@pateam.com 5095 Ritter Rd Mechanicsburg PA 17055
-----Original Message----- From: -Hammer- [mailto:bhmccie@gmail.com] Sent: Friday, February 17, 2012 9:52 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of
On 2/17/2012 8:29 AM, Leo Bicknell wrote: that
should be done in a group enviornment.
BTW, I am a school board member who votes 1:8 often on things.... But let me give you a perspective, one of the board members called Golf an "Essential Life Skill." Maybe, but how about balancing a checkbook...
Ralph Brandt Communications Engineer HP Enterprise Services
One of the best courses I ever had was in 9th grade math class when Mr. Martin taught us how to do taxes. And I mean even long forms with all the schedules and stuff for people who had investments and small sole proprietor businesses. It was a great practical math application, though it was mostly just arithmetic, some of the example cases were quite complex with estimated taxes, carrying forward losses from previous years, depreciation, etc. It gave us some context in which we could understand why we might need to learn math in real life and made taxes less daunting when we got older. Thanks, Mr. Martin!
Well, I will argue this. I think the important factor in any troubleshooting is having a real understanding of how the system works. That is, how different things interact with each others to achieve a specific goal. The biggest problem I see is that many people understand understand the individual parts but when it comes to understanding the system as a whole they fall miserably short. A short example, probably not the best but the one that comes to mind right now: Someone replaces a device on the network with a new one. They give it the same IP address as the old device. They don't understand why the router cant communicate with it at first and then starts working. The people "understand" ARP, but cant correlate one event with another. I guess if your 35 you have seen this at least once and can fix it. But what happens if you have never seen this problem or a related one? At this point your going to have to really troubleshoot, not just go on experience. Mario Eirea ________________________________________ From: -Hammer- [bhmccie@gmail.com] Sent: Friday, February 17, 2012 9:52 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions Let me simplify that. If you are over 35 you know how to troubleshoot. Yes, I'm going to get flamed. Yes, there are exceptions in both directions. -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
I agree with this 100%. Having worked with many people over the last 40 years, the good trouble shooters understood how things were suppose to work. This helps immeasurably in determining where to start looking. On 02/17/2012 10:12 AM, Mario Eirea wrote:
Well, I will argue this. I think the important factor in any troubleshooting is having a real understanding of how the system works. That is, how different things interact with each others to achieve a specific goal. The biggest problem I see is that many people understand understand the individual parts but when it comes to understanding the system as a whole they fall miserably short.
A short example, probably not the best but the one that comes to mind right now:
Someone replaces a device on the network with a new one. They give it the same IP address as the old device. They don't understand why the router cant communicate with it at first and then starts working. The people "understand" ARP, but cant correlate one event with another.
I guess if your 35 you have seen this at least once and can fix it. But what happens if you have never seen this problem or a related one? At this point your going to have to really troubleshoot, not just go on experience.
Mario Eirea ________________________________________ From: -Hammer- [bhmccie@gmail.com] Sent: Friday, February 17, 2012 9:52 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
-- Stephen Clark *NetWolves* Director of Technology Phone: 813-579-3200 Fax: 813-882-0209 Email: steve.clark@netwolves.com http://www.netwolves.com
On 2/17/2012 9:18 AM, Steve Clark wrote:
Having worked with many people over the last 40 years, the good trouble shooters understood how things were suppose to work. This helps immeasurably in determining where to start looking.
Ran into this not too long ago with a transport problem. The behavior I was seeing was indicative of the transport not stripping their outer tag. They put wireshark on a windows laptop and sent me the traffic captures. While I didn't know that M$ decided to do something silly like removing a single tag, all indicators were that the M$ stack "fixed" whatever was broken prior to wireshark. We took a capture from another device and proved the problem. Which is a common transport problem I often see, "Our configuration looks right, it must be on your end." Jack
Sent from my iPhone On Feb 17, 2012, at 7:48, Jack Bates <jbates@brightok.net> wrote:
On 2/17/2012 9:18 AM, Steve Clark wrote:
Having worked with many people over the last 40 years, the good trouble shooters understood how things were suppose to work. This helps immeasurably in determining where to start looking.
Ran into this not too long ago with a transport problem. The behavior I was seeing was indicative of the transport not stripping their outer tag. They put wireshark on a windows laptop and sent me the traffic captures. While I didn't know that M$ decided to do something silly like removing a single tag, all indicators were that the M$ stack "fixed" whatever was broken prior to wireshark. We took a capture from another device and proved the problem.
Which is a common transport problem I often see, "Our configuration looks right, it must be on your end."
Jack
If i had a dollar for everytime i've heard that from a telco, i'd be a rich man...
On Fri, Feb 17, 2012 at 08:46:02AM -0800, Mike Lyon wrote:
Sent from my iPhone
On Feb 17, 2012, at 7:48, Jack Bates <jbates@brightok.net> wrote:
On 2/17/2012 9:18 AM, Steve Clark wrote:
Having worked with many people over the last 40 years, the good trouble shooters understood how things were suppose to work. This helps immeasurably in determining where to start looking.
Ran into this not too long ago with a transport problem. The behavior I was seeing was indicative of the transport not stripping their outer tag. They put wireshark on a windows laptop and sent me the traffic captures. While I didn't know that M$ decided to do something silly like removing a single tag, all indicators were that the M$ stack "fixed" whatever was broken prior to wireshark. We took a capture from another device and proved the problem.
Which is a common transport problem I often see, "Our configuration looks right, it must be on your end."
If i had a dollar for everytime i've heard that from a telco, i'd be a rich man...
That and "I'm getting a good ping response here" while I've got the cable at my end unplugged from the equipment. -- Mike Andrews, W5EGO mikea@mikea.ath.cx Tired old sysadmin
----- Original Message -----
From: "Mike Andrews" <mikea@mikea.ath.cx>
Which is a common transport problem I often see, "Our configuration looks right, it must be on your end."
If i had a dollar for everytime i've heard that from a telco, i'd be a rich man...
That and "I'm getting a good ping response here" while I've got the cable at my end unplugged from the equipment.
"The problem's leaving here fine!" Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
On 2/17/2012 10:18 AM, Steve Clark wrote:
I agree with this 100%.
Having worked with many people over the last 40 years, the good trouble shooters understood how things were suppose to work. This helps immeasurably in determining where to start looking.
This is dead on and why I always start classes with a refresher on the OSI model. While the model isn't perfect it lets technicians and engineers construct a reasonable model of how things *ought* to be working. While you certainly will run into devices that bend or even break the rules (sometimes for good reasons) its much easier to understand the exceptions if you know the normal operation for a repeater, bridge, or router. -- Scott Helms Vice President of Technology ISP Alliance, Inc. DBA ZCorum (678) 507-5000 -------------------------------- http://twitter.com/kscotthelms --------------------------------
To find counterfeit they teach you what good money looks like. When you are looking at a sniffer trace you are generally looking for something that is not right. Ralph Brandt Communications Engineer HP Enterprise Services Telephone +1 717.506.0802 FAX +1 717.506.4358 Email Ralph.Brandt@pateam.com 5095 Ritter Rd Mechanicsburg PA 17055 -----Original Message----- From: Scott Helms [mailto:khelms@ispalliance.net] Sent: Friday, February 17, 2012 11:24 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions On 2/17/2012 10:18 AM, Steve Clark wrote:
I agree with this 100%.
Having worked with many people over the last 40 years, the good trouble shooters understood how things were suppose to work. This helps immeasurably in determining where to start looking.
This is dead on and why I always start classes with a refresher on the OSI model. While the model isn't perfect it lets technicians and engineers construct a reasonable model of how things *ought* to be working. While you certainly will run into devices that bend or even break the rules (sometimes for good reasons) its much easier to understand the exceptions if you know the normal operation for a repeater, bridge, or router. -- Scott Helms Vice President of Technology ISP Alliance, Inc. DBA ZCorum (678) 507-5000 -------------------------------- http://twitter.com/kscotthelms --------------------------------
On Fri, Feb 17, 2012 at 10:18:57AM -0500, Steve Clark wrote:
I agree with this 100%.
Having worked with many people over the last 40 years, the good trouble shooters understood how things were suppose to work. This helps immeasurably in determining where to start looking.
Don't forget that a lot of the best figure things out *while* troubleshooting things they don't (yet) understand. Give curious people good tools and interesting problems -- the rest will sort itself out.
On Feb 17, 2012, at 11:33 AM, John Osmon wrote: On Fri, Feb 17, 2012 at 10:18:57AM -0500, Steve Clark wrote:
I agree with this 100%.
Having worked with many people over the last 40 years, the good trouble shooters understood how things were suppose to work. This helps immeasurably in determining where to start looking.
Don't forget that a lot of the best figure things out *while* troubleshooting things they don't (yet) understand. Give curious people good tools and interesting problems -- the rest will sort itself out. Quote I have hear over the years "Problems are opportunities for learning - Just wish I was not doing so much learning some days...."
Mario, I was kinda having fun and kinda not. My point is that the 40-50 year olds that were doing this 30 years ago grew up understanding things in order. Bits. Bytes. KiloBits. KiloBytes. (Some folks still get those confused). Hex. etc. Move on to the OSI model and it's the same thing. Voltage. Amplitude. Binary. etc. I think that this generation that I'm referring to is a great generation because we were at the beginning of the Internet blooming. There are folks on this forum that go back further. Into DARPA. Before DARPA was just chapter 1 one every single Cisco Press book. They have a unique understanding of the layers. I had that understanding in my 20s. The technology is so complicated these days that many folks miss those fundamentals and go right into VSS on the 6500s or MPLS over Juniper. In the end, it all comes in time. -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 9:12 AM, Mario Eirea wrote:
Well, I will argue this. I think the important factor in any troubleshooting is having a real understanding of how the system works. That is, how different things interact with each others to achieve a specific goal. The biggest problem I see is that many people understand understand the individual parts but when it comes to understanding the system as a whole they fall miserably short.
A short example, probably not the best but the one that comes to mind right now:
Someone replaces a device on the network with a new one. They give it the same IP address as the old device. They don't understand why the router cant communicate with it at first and then starts working. The people "understand" ARP, but cant correlate one event with another.
I guess if your 35 you have seen this at least once and can fix it. But what happens if you have never seen this problem or a related one? At this point your going to have to really troubleshoot, not just go on experience.
Mario Eirea ________________________________________ From: -Hammer- [bhmccie@gmail.com] Sent: Friday, February 17, 2012 9:52 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
I definitely understand and agree with what you saying. Actually, most my friends are over 50 years old... I do agree with you on the generational statement. My argument was that many people over 35 have no idea what they are doing, and some under 35 do know what they are doing. On thing is for sure, experience goes a long way. The importance is knowing the fundamentals and putting it all together (a skill that has been lost in recent times) I have a lot to say about the current generation of people growing up in this country, but that's a whole other thread in a whole other list. :-) Mario Eirea ________________________________________ From: -Hammer- [bhmccie@gmail.com] Sent: Friday, February 17, 2012 10:51 AM To: Mario Eirea Cc: nanog@nanog.org Subject: Re: Common operational misconceptions Mario, I was kinda having fun and kinda not. My point is that the 40-50 year olds that were doing this 30 years ago grew up understanding things in order. Bits. Bytes. KiloBits. KiloBytes. (Some folks still get those confused). Hex. etc. Move on to the OSI model and it's the same thing. Voltage. Amplitude. Binary. etc. I think that this generation that I'm referring to is a great generation because we were at the beginning of the Internet blooming. There are folks on this forum that go back further. Into DARPA. Before DARPA was just chapter 1 one every single Cisco Press book. They have a unique understanding of the layers. I had that understanding in my 20s. The technology is so complicated these days that many folks miss those fundamentals and go right into VSS on the 6500s or MPLS over Juniper. In the end, it all comes in time. -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 9:12 AM, Mario Eirea wrote:
Well, I will argue this. I think the important factor in any troubleshooting is having a real understanding of how the system works. That is, how different things interact with each others to achieve a specific goal. The biggest problem I see is that many people understand understand the individual parts but when it comes to understanding the system as a whole they fall miserably short.
A short example, probably not the best but the one that comes to mind right now:
Someone replaces a device on the network with a new one. They give it the same IP address as the old device. They don't understand why the router cant communicate with it at first and then starts working. The people "understand" ARP, but cant correlate one event with another.
I guess if your 35 you have seen this at least once and can fix it. But what happens if you have never seen this problem or a related one? At this point your going to have to really troubleshoot, not just go on experience.
Mario Eirea ________________________________________ From: -Hammer- [bhmccie@gmail.com] Sent: Friday, February 17, 2012 9:52 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
This reminds me of what I think is the biggest root misconception of the 20th and 21st centuries: Rapid step-by-step training can replace conceptual education on the fundamentals. In other words, we have moved from the old-school of teaching people why things work and how they work to a newer school of teaching people how to complete specific tasks. This has had the following negative effects, IMHO: 1. When the only tool you have is a hammer, you try to mold every problem into a nail. 2. When you only know a procedure for doing something and don't understand the fundamentals of why X is supposed to occur at step Y, then when you get result A instead of X, your only options are to either continue to step Z and hope everything turns out OK, or, go back to an earlier step and hope everything works this time. 3. Troubleshooting skills are limited to knowing the number of the vendor's help desk. I once worked with a director of QA that epitomized this. It was a small company, so, as director, he was directly responsible for most of the tasks in the QA lab. He was meticulous in following directions which was a good thing. However, when he reached a step where he did not get the expected result, he was limited to telling the engineers that the test failed at step X and would not make any effort to identify or resolve the problem and would literally block the entire QA process waiting for engineering to resolve the issue before he would continue testing. Worse, he would not test independent pieces of the system in parallel, so, when he blocked on one system failing, he wouldn't test the others, either. Further investigation revealed that this was because he didn't actually know which systems were or were not dependent on each other. He was so completely immersed in the procedural school of thought that he was literally unwilling to accept conceptual knowledge or develop an understanding of the theory and principles of operation of any of the systems. Owen On Feb 17, 2012, at 8:13 AM, Mario Eirea wrote:
I definitely understand and agree with what you saying. Actually, most my friends are over 50 years old... I do agree with you on the generational statement. My argument was that many people over 35 have no idea what they are doing, and some under 35 do know what they are doing. On thing is for sure, experience goes a long way. The importance is knowing the fundamentals and putting it all together (a skill that has been lost in recent times)
I have a lot to say about the current generation of people growing up in this country, but that's a whole other thread in a whole other list. :-)
Mario Eirea
Well put and great example Owen. -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 12:59 PM, Owen DeLong wrote:
This reminds me of what I think is the biggest root misconception of the 20th and 21st centuries:
Rapid step-by-step training can replace conceptual education on the fundamentals.
In other words, we have moved from the old-school of teaching people why things work and how they work to a newer school of teaching people how to complete specific tasks. This has had the following negative effects, IMHO:
1. When the only tool you have is a hammer, you try to mold every problem into a nail. 2. When you only know a procedure for doing something and don't understand the fundamentals of why X is supposed to occur at step Y, then when you get result A instead of X, your only options are to either continue to step Z and hope everything turns out OK, or, go back to an earlier step and hope everything works this time. 3. Troubleshooting skills are limited to knowing the number of the vendor's help desk.
I once worked with a director of QA that epitomized this. It was a small company, so, as director, he was directly responsible for most of the tasks in the QA lab. He was meticulous in following directions which was a good thing. However, when he reached a step where he did not get the expected result, he was limited to telling the engineers that the test failed at step X and would not make any effort to identify or resolve the problem and would literally block the entire QA process waiting for engineering to resolve the issue before he would continue testing. Worse, he would not test independent pieces of the system in parallel, so, when he blocked on one system failing, he wouldn't test the others, either. Further investigation revealed that this was because he didn't actually know which systems were or were not dependent on each other. He was so completely immersed in the procedural school of thought that he was literally unwilling to accept conceptual knowledge or develop an understanding of the theory and principles of operation of any of the systems.
Owen
On Feb 17, 2012, at 8:13 AM, Mario Eirea wrote:
I definitely understand and agree with what you saying. Actually, most my friends are over 50 years old... I do agree with you on the generational statement. My argument was that many people over 35 have no idea what they are doing, and some under 35 do know what they are doing. On thing is for sure, experience goes a long way. The importance is knowing the fundamentals and putting it all together (a skill that has been lost in recent times)
I have a lot to say about the current generation of people growing up in this country, but that's a whole other thread in a whole other list. :-)
Mario Eirea
Owen DeLong <owen@delong.com> writes:
1. When the only tool you have is a hammer, you try to mold every problem into a nail.
Ack.
2. When you only know a procedure for doing something and don't understand the fundamentals of why X is supposed to occur at step Y, then when you get result A instead of X, your only options are to either continue to step Z and hope everything turns out OK, or, go back to an earlier step and hope everything works this time.
But procedures are important. How else can you get enough exper^Widiots working for little money. "Big Macs vs. The Naked Chef" is great: http://www.joelonsoftware.com/articles/fog0000000024.html
3. Troubleshooting skills are limited to knowing the number of the vendor's help desk.
There are no problems! Can't be. And if there are they hire external experts. BTDT. Those are well paid jobs. Jens -- ------------------------------------------------------------------------- | Foelderichstr. 40 | 13595 Berlin, Germany | +49-151-18721264 | | http://blog.quux.de | jabber: jenslink@guug.de | ------------------- | -------------------------------------------------------------------------
3. Troubleshooting skills are limited to knowing the number of the vendor's help desk.
There are no problems! Can't be. And if there are they hire external experts. BTDT. Those are well paid jobs.
I see that a lot and there is often an organizational reason for it. If a tech says "I have a ticket open with the vendor" and provides the ticket and status updates on a regular basis, he's covered as far as the people higher up in the organization are concerned. If the C(X)O wants to know what's going on, the manager can shift the focus to the vendor and say we are waiting for a fix from them. A tech trying to troubleshoot it and fix it themselves is going to be hounded every five minutes for status updates and won't be able to get any work done because every five minutes (I kid you not, I have worked where that is a requirement) he has to pull his head out of what he is doing and answer a bunch of questions from the PHBs. And you always get "how long is it going to be" and you want to say "10 minutes longer than it would have been if you hadn't interrupted me" but you bite your tongue.
A tech trying to troubleshoot it and fix it themselves is going to be hounded every five minutes for status updates and won't be able to get any work done because every five minutes (I kid you not, I have worked where that is a requirement) he has to pull his head out of what he is doing and answer a bunch of questions from the PHBs. And you always get "how long is it going to be" and you want to say "10 minutes longer than it would have been if you hadn't interrupted me" but you bite your tongue.
Though the flip side of that is that if someone has been neck deep in a problem for hours, you should force them to take a break, go get a drink of water, step outside for fresh air or a smoke if they do, or just talk to a colleague for a moment and review the problem. In my case, the stepping away for a few minutes has sometimes allowed the answer to the problem to suddenly snap into focus or in the process of describing it to someone else the forming of the thoughts to describe it often allows a new aspect of the problem to become visible that you hadn't noticed before.
I have found that the best solution to persistent hounding goes about like this: "Sir, I'm doing everything I can to resolve the problem as quickly as possible. However, I can focus on giving you status updates every 5 minutes, or, I can focus on resolving the problem. I cannot do both. which would you prefer?" As to the internal vs. external question, most organizations I've worked for have just wanted the problem solved. They didn't so much care whether I was taking a lot of time to solve it or the vendor was taking a lot of time to respond to me, they wanted the problem fixed and I was the one they could fire. As such, it was in my interest to usually learn most of the systems better than the tech support folk at the other end of the phone. YMMV Owen On Feb 17, 2012, at 11:35 AM, George Bonser wrote:
A tech trying to troubleshoot it and fix it themselves is going to be hounded every five minutes for status updates and won't be able to get any work done because every five minutes (I kid you not, I have worked where that is a requirement) he has to pull his head out of what he is doing and answer a bunch of questions from the PHBs. And you always get "how long is it going to be" and you want to say "10 minutes longer than it would have been if you hadn't interrupted me" but you bite your tongue.
Though the flip side of that is that if someone has been neck deep in a problem for hours, you should force them to take a break, go get a drink of water, step outside for fresh air or a smoke if they do, or just talk to a colleague for a moment and review the problem. In my case, the stepping away for a few minutes has sometimes allowed the answer to the problem to suddenly snap into focus or in the process of describing it to someone else the forming of the thoughts to describe it often allows a new aspect of the problem to become visible that you hadn't noticed before.
Blocking incoming spam is worth spending $ on for software, 3rd party filtering services, or dedicated spam filtering hardare. Blocking outgoing spam? Huh? ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
I often struggle with the concept of teaching someone how to troubleshoot. We have young guys coming in all the time and it is often an area in which they need to hone their skills. I found this article a while back and I keep it bookmarked, its the best article I've seen that lays it all out pretty clearly. http://packetlife.net/blog/2010/mar/10/the-science-of-network-troubleshootin... -Marcel I'm guessing, but I believe the author would fall into the under 35 category ;-) On Fri, Feb 17, 2012 at 10:51 AM, -Hammer- <bhmccie@gmail.com> wrote:
Mario, I was kinda having fun and kinda not. My point is that the 40-50 year olds that were doing this 30 years ago grew up understanding things in order. Bits. Bytes. KiloBits. KiloBytes. (Some folks still get those confused). Hex. etc. Move on to the OSI model and it's the same thing. Voltage. Amplitude. Binary. etc. I think that this generation that I'm referring to is a great generation because we were at the beginning of the Internet blooming. There are folks on this forum that go back further. Into DARPA. Before DARPA was just chapter 1 one every single Cisco Press book. They have a unique understanding of the layers. I had that understanding in my 20s. The technology is so complicated these days that many folks miss those fundamentals and go right into VSS on the 6500s or MPLS over Juniper. In the end, it all comes in time.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 9:12 AM, Mario Eirea wrote:
Well, I will argue this. I think the important factor in any troubleshooting is having a real understanding of how the system works. That is, how different things interact with each others to achieve a specific goal. The biggest problem I see is that many people understand understand the individual parts but when it comes to understanding the system as a whole they fall miserably short.
A short example, probably not the best but the one that comes to mind right now:
Someone replaces a device on the network with a new one. They give it the same IP address as the old device. They don't understand why the router cant communicate with it at first and then starts working. The people "understand" ARP, but cant correlate one event with another.
I guess if your 35 you have seen this at least once and can fix it. But what happens if you have never seen this problem or a related one? At this point your going to have to really troubleshoot, not just go on experience.
Mario Eirea ________________________________________ From: -Hammer- [bhmccie@gmail.com] Sent: Friday, February 17, 2012 9:52 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc.
I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
I just pulled the humorous point about Mary the accountant out, pasted its link into an email and sent it to our staff with the suggestion they read it. I was going to past it here but decided to let someone who wants to read it go to the site, they may learn something by accident if they do. I have been unable to get our group to read anything else, maybe they will read this very well written document. It really is a "comm oriented" Cliff Notes of Kepner Trego Problem Analysis. I would love them to read the text book, but will settle for anything. Ralph Brandt Communications Engineer HP Enterprise Services Telephone +1 717.506.0802 FAX +1 717.506.4358 Email Ralph.Brandt@pateam.com 5095 Ritter Rd Mechanicsburg PA 17055 -----Original Message----- From: Marcel Plug [mailto:marcelplug@gmail.com] Sent: Friday, February 17, 2012 12:01 PM To: -Hammer- Cc: nanog@nanog.org Subject: Re: Common operational misconceptions I often struggle with the concept of teaching someone how to troubleshoot. We have young guys coming in all the time and it is often an area in which they need to hone their skills. I found this article a while back and I keep it bookmarked, its the best article I've seen that lays it all out pretty clearly. http://packetlife.net/blog/2010/mar/10/the-science-of-network-troubleshootin... -Marcel I'm guessing, but I believe the author would fall into the under 35 category ;-) On Fri, Feb 17, 2012 at 10:51 AM, -Hammer- <bhmccie@gmail.com> wrote:
Mario, I was kinda having fun and kinda not. My point is that the 40-50 year olds that were doing this 30 years ago grew up understanding things in order. Bits. Bytes. KiloBits. KiloBytes. (Some folks still get those confused). Hex. etc. Move on to the OSI model and it's the same thing. Voltage. Amplitude. Binary. etc. I think that this generation that I'm referring to is a great generation because we were at the beginning of the Internet blooming. There are folks on this forum that go back further. Into DARPA. Before DARPA was just chapter 1 one every single Cisco Press book. They have a unique understanding of the layers. I had that understanding in my 20s. The technology is so complicated these days that many folks miss those fundamentals and go right into VSS on the 6500s or MPLS over Juniper. In the end, it all comes in time.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 9:12 AM, Mario Eirea wrote:
Well, I will argue this. I think the important factor in any troubleshooting is having a real understanding of how the system works. That is, how different things interact with each others to achieve a specific goal. The biggest problem I see is that many people understand understand the individual parts but when it comes to understanding the system as a whole they fall miserably short.
A short example, probably not the best but the one that comes to mind right now:
Someone replaces a device on the network with a new one. They give it the same IP address as the old device. They don't understand why the router cant communicate with it at first and then starts working. The people "understand" ARP, but cant correlate one event with another.
I guess if your 35 you have seen this at least once and can fix it. But what happens if you have never seen this problem or a related one? At this point your going to have to really troubleshoot, not just go on experience.
Mario Eirea ________________________________________ From: -Hammer- [bhmccie@gmail.com] Sent: Friday, February 17, 2012 9:52 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc.
I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
Now, come on... If you're in the 40-50 range, you should have put octal before hex. :p Owen (Who grew up on a PDP-11 in his high-school and still remembers 1777300 and its significance to anyone who has used a larger PDP system) On Feb 17, 2012, at 7:51 AM, -Hammer- wrote:
Mario, I was kinda having fun and kinda not. My point is that the 40-50 year olds that were doing this 30 years ago grew up understanding things in order. Bits. Bytes. KiloBits. KiloBytes. (Some folks still get those confused). Hex. etc. Move on to the OSI model and it's the same thing. Voltage. Amplitude. Binary. etc. I think that this generation that I'm referring to is a great generation because we were at the beginning of the Internet blooming. There are folks on this forum that go back further. Into DARPA. Before DARPA was just chapter 1 one every single Cisco Press book. They have a unique understanding of the layers. I had that understanding in my 20s. The technology is so complicated these days that many folks miss those fundamentals and go right into VSS on the 6500s or MPLS over Juniper. In the end, it all comes in time.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 9:12 AM, Mario Eirea wrote:
Well, I will argue this. I think the important factor in any troubleshooting is having a real understanding of how the system works. That is, how different things interact with each others to achieve a specific goal. The biggest problem I see is that many people understand understand the individual parts but when it comes to understanding the system as a whole they fall miserably short.
A short example, probably not the best but the one that comes to mind right now:
Someone replaces a device on the network with a new one. They give it the same IP address as the old device. They don't understand why the router cant communicate with it at first and then starts working. The people "understand" ARP, but cant correlate one event with another.
I guess if your 35 you have seen this at least once and can fix it. But what happens if you have never seen this problem or a related one? At this point your going to have to really troubleshoot, not just go on experience.
Mario Eirea ________________________________________ From: -Hammer- [bhmccie@gmail.com] Sent: Friday, February 17, 2012 9:52 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
From nanog-bounces+bonomi=mail.r-bonomi.com@nanog.org Fri Feb 17 13:11:28 2012 To: Owen DeLong <owen@delong.com> Subject: Re: Common operational misconceptions From: Valdis.Kletnieks@vt.edu Date: Fri, 17 Feb 2012 14:04:45 -0500 Cc: "nanog@nanog.org" <nanog@nanog.org>
On Fri, 17 Feb 2012 10:49:13 PST, Owen DeLong said:
Now, come on... If you're in the 40-50 range, you should have put octal before hex. :p
IBM S/360 definitely preferred hex. And EBCDIC.
And the _real_ number crunchers used ones-complement arithmetic. Which led to to mahines that couldn't add -- they did addition by 'complement and subtract'.
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160 On 2/17/12 11:04 AM, Valdis.Kletnieks@vt.edu wrote:
On Fri, 17 Feb 2012 10:49:13 PST, Owen DeLong said:
Now, come on... If you're in the 40-50 range, you should have put octal before hex. :p
IBM S/360 definitely preferred hex. And EBCDIC.
GCOS - 36 bits and Octal and BCD (ASCII added later) DEC 10 and 20 - 36 bits and Octal PDP-8 - Octal - --tep -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREDAAYFAk8+qEEACgkQPu53fhcIEuBr9gCfU46kCDPbmgSVQGAw5nnOsrNO hJ4AnRpr4Ig46x5JZlcK+kL43JGFcbCS =cSWb -----END PGP SIGNATURE-----
On Feb 17, 2012, at 11:04 AM, Valdis.Kletnieks@vt.edu wrote:
On Fri, 17 Feb 2012 10:49:13 PST, Owen DeLong said:
Now, come on... If you're in the 40-50 range, you should have put octal before hex. :p
IBM S/360 definitely preferred hex. And EBCDIC.
Strictly an artifact of it's EBCDIC nature, as a matter of fact... The BCD in EBCDIC stands for Binary Coded Decimal which inherently requires Hex. In fact, if you compare punched cards to EBCDIC, you'll find some remarkable similarities... For example C1 (12 1) is A (Hollerith A is a 12 punch plus a 1 punch). Owen
ICMP is evil. To that I would add under... Security misconceptions
On Wed, Feb 15, 2012 at 4:52 PM, Rich Kulawiec <rsk@gsp.org> wrote: 0. Security is just common sense. a. More draconian/more complicated policies/practices automatically result in a good secure, usable environment. i. For secure results, require users to set a 25-character complex password with 1-day expire. ii. For best results, get a checklist containing every possible "security measure" that can be implemented, and implement them in no particular order. iii. For best results, ask a committee of accomplished bureaucrats.... b. For best results, leave all settings at their default values. i. A security focused analysis is not required to design a secure system/network. ii. If each device is secure, the overall system is automatically secure. c. Just install Product $X, Product $Y. Everything will be fine. d. If that doesn't work, and we still get a security breach, adding Product $Z will definitely make it secure forever, without log checking and security reviews. e. A simple automated scan can detect all possible security issues. 1. Script kiddies don't want access to my router, because they can't run malware i. Routers always encrypt passwords in memory; the *s displayed when you look at the password field in the webui prove it; no worries throwing out old equipment. ii. It's okay to re-use the admin password for a POP3 account, no security risk there. 2. If your organization partitions its internal network from the internet with a firewall.... a. The network will be invincible to attack. or i. Private addressing ensures a LAN secure against outside attack. ii. SSL certificates don't matter, just click Continue. b. Sources of possible abuse/intrusion will always be on the outside. [or] i. The perimeter firewall makes the LAN safe against packet sniffers ii. Use of Ethernet switches instead of hubs make the LAN completely safe against packet sniffers. iii. Managing local LAN devices (such as routers) using telnet or plain HTTP is safe and secure (because of i or ii). iv. Plain e-mail is an excellent file transfer protocol -- it's also a secure way to transfer large files into or out of a Firewall-secured LAN, since e-mail is private. v. External USB drives are a safe, secure, convenient way to bring data into or out of the partitioned network. Antivirus will thwart any attempt to transfer malicious files of any type. vi. FTP is a safe way to bring data into or out of a secure network, and the data is safe against interception because a password is required to connect. c. The one perimeter firewall will protect the network against internal attacks, and even outsiders gaining access to open wifi. i. WEP or open access with MAC address filtering is pretty secure. ii. MAC address filtering on the core router will make sure unauthorized devices plugged in cannot possibly gain access to the LAN. iii. MAC address filtering on the DHCP server will make sure unauthorized devices plugged in cannot possibly gain access to the LAN. d. No need to worry about having a DMZ or separate network, for hosting internet services behind a firewall. i. If traffic is only allowed to port 80, shell access cannot be obtained by exploit, even if the PHP scripts have bugs, because port 23 is required for shell access. ii. If traffic from the internet is alllowed to pass to one host through a firewall, any possible security risk is limited to exclusively that one host.
Firewalls will solve our security issues. Antivirus will solve our security issues. ...
$MAGIC_PRODUCT will solve our security issues. For many different values of $MAGIC_PRODUCT -- -JH
I give I give. I should have. But I left a bunch of stuff out and the folks that I'm referring to know it. One of the fun things I do with network guys is ask them about canonical bit ordering across routers. Always causes the room to go quiet. Them someone of the appropriate age group will speak up after he's done laughing. The best thing I have ever figured out to tell less experienced (I didn't say younger) folks is to start at the bottom of the stack and work your way up. That's the way many of us troubleshoot. Is the cable on the floor? That's bad. If not, is the ARP entry incomplete? That's bad. If not, is the route missing? That's bad. If not, can you reach the route? Try this radical command that was invented by Steve Jobs while working on his first IPhone (They won't know who Vint Cerf or anyone else is and by using Steves name they will trust you)(I run Android): telnet 1.2.3.4 1433 What? It answered? So the SQL service is running? Then it ain't the network dude.... So many times people don't pick up on that. But when they do, it's like a light bulb went off and they see the world differently. Like subnetting.... -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 12:49 PM, Owen DeLong wrote:
Now, come on... If you're in the 40-50 range, you should have put octal before hex. :p
Owen (Who grew up on a PDP-11 in his high-school and still remembers 1777300 and its significance to anyone who has used a larger PDP system)
On Feb 17, 2012, at 7:51 AM, -Hammer- wrote:
Mario, I was kinda having fun and kinda not. My point is that the 40-50 year olds that were doing this 30 years ago grew up understanding things in order. Bits. Bytes. KiloBits. KiloBytes. (Some folks still get those confused). Hex. etc. Move on to the OSI model and it's the same thing. Voltage. Amplitude. Binary. etc. I think that this generation that I'm referring to is a great generation because we were at the beginning of the Internet blooming. There are folks on this forum that go back further. Into DARPA. Before DARPA was just chapter 1 one every single Cisco Press book. They have a unique understanding of the layers. I had that understanding in my 20s. The technology is so complicated these days that many folks miss those fundamentals and go right into VSS on the 6500s or MPLS over Juniper. In the end, it all comes in time.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 9:12 AM, Mario Eirea wrote:
Well, I will argue this. I think the important factor in any troubleshooting is having a real understanding of how the system works. That is, how different things interact with each others to achieve a specific goal. The biggest problem I see is that many people understand understand the individual parts but when it comes to understanding the system as a whole they fall miserably short.
A short example, probably not the best but the one that comes to mind right now:
Someone replaces a device on the network with a new one. They give it the same IP address as the old device. They don't understand why the router cant communicate with it at first and then starts working. The people "understand" ARP, but cant correlate one event with another.
I guess if your 35 you have seen this at least once and can fix it. But what happens if you have never seen this problem or a related one? At this point your going to have to really troubleshoot, not just go on experience.
Mario Eirea ________________________________________ From: -Hammer- [bhmccie@gmail.com] Sent: Friday, February 17, 2012 9:52 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
missing? That's bad. If not, can you reach the route? Try this radical command that was invented by Steve Jobs while working on his first IPhone (They won't know who Vint Cerf or anyone else is and by using Steves name they will trust you)(I run Android): telnet 1.2.3.4 1433 What? It answered? So the SQL service is running? Then it ain't the network dude....
steve jobs knew how to operate a computer? the guy that renamed apple computer to "apple" :P i thought he just knew how to come up with overheating cases and shiney designs to put -around- the computers (well until the chips fell out of their sockets due to the heat or they caught fire :P they had woz for the computer stuff remember :P the guy that first turned a perfectly open computer platform manufacturer capable of keeping the ibm pc out of the market for many decades to come into a gadget for the managers desk, and then came back to turn a unix workstation manufacturer into an electronics toys (iphones) company. the question is: where would apple have been all those years, if steve jobs wasn't around to screw it up every time :P (and where are the BMCs in their mac-mini "servers" ;) 2 video chips.. but no bmc..hmm.
On Fri, Feb 17, 2012 at 06:52, -Hammer- <bhmccie@gmail.com> wrote:
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
"Necessity is the mother of invention" Long before there was a Grainger (and Home Depot) in every city, and you could get parts shipped overnight, one had to "make do", and "making do" meant being able to figure things out to be able to "git r done" with what you had on hand, or could figure out. When working on my Grandfather's farm, I did not look for work to do (actually, I looked for ways not to do any work :-), but if the project required pulling out the oxy-acetylene torch to cut and weld something onto the tractor to get something done, that is what you had to do, so you did it. If the TV went on the blink (they all did then), you opened up the back, looked for fried components, and if one of the resistors was smoking, you soldered in a replacement. Or you took the tubes down to the local drugstore and tested them. Even if you had no idea what you were doing, you were willing (and expected) to give it a shot, and try to fix it. More often than not you learned something along the way, even if it took hours to figure it out (and had to repair your repair a few times :-). For those without the capabilities, you took it to the shop, where someone else did the troubleshooting and repair. Along the line, the costs of technicians to do that type of work started to exceed the cost of simply replacing the entire unit (how many people remember when going to the auto dealer that the cost of the parts far exceeded the cost of the labor? Now it is the other way around). Troubleshooting became a lost art. "Swap 'til you drop" became the mantra. It became the cost effective way to do repairs. There are advantages to the new way of disposable devices, but almost no one knows how they work anymore, and they do not care to know. The members of this list are likely to be sufficiently self selected to be in the minority of actually wanting to know. There is a (small) backlash of people who are trying to get back into the world of actually building things, and understanding how they work (popularized by such things as Make magazine, and Maker Faires). Gary
Long before there was a Grainger (and Home Depot) in every city, and you could get parts shipped overnight, one had to "make do", and "making do" meant being able to figure things out to be able to "git r done" with what you had on hand, or could figure out.
When working on my Grandfather's farm, I did not look for work to do (actually, I looked for ways not to do any work :-), but if the project required pulling out the oxy-acetylene torch to cut and weld something onto the tractor to get something done, that is what you had to do, so you did it.
Yep, when looking for troubleshooters, look for people that grew up/worked on a farm. I absolutely agree. They approach things from a completely different mindset.
On 2/17/2012 12:00 PM, Gary Buhrmaster wrote:
If the TV went on the blink (they all did then), you opened up the back, looked for fried components, and if one of the resistors was smoking, you soldered in a replacement. Or you took the tubes down to the local drugstore and tested them.
Wow... would be handy if Radio Shack stocked router modules and blades, and chassis to test your suspect ones? :) (Yes, remember the tube testers as well...) Jeff
Wow... would be handy if Radio Shack stocked router modules and blades, and chassis to test your suspect ones? :)
(Yes, remember the tube testers as well...)
Jeff
Heh, that's been a notion I have had for a while. Opening an all-night shop somewhere in Silicon Valley that sold patch cords, memory sticks, disk drives, maybe even common router blades, optics modules, fans, etc. Sell it for a bit more margin than the going rate for the "day" shops and ONLY be open at night/early morning, say 7pm to 11am. Maybe do some "remote hands" work, too. Fry's wanted $55 for a 1 meter LC-LC multi-mode patch cord yesterday at the store on Arques in Sunnyvale.
In a message written on Fri, Feb 17, 2012 at 06:06:52PM +0000, George Bonser wrote:
Heh, that's been a notion I have had for a while. Opening an all-night shop somewhere in Silicon Valley that sold patch cords, memory sticks, disk drives, maybe even common router blades, optics modules, fans, etc. Sell it for a bit more margin than the going rate for the "day" shops and ONLY be open at night/early morning, say 7pm to 11am. Maybe do some "remote hands" work, too.
I've repeatedly asked $BIG_COLO_PROVIDERS to offer a vending machine in the lobby next to the one with sodas that sold Cat 5, Fiber, SFP's, USB sticks, and so on. Even at a moderate margin I suspect it would be quite profitable to them, and quite welcomed by customers who show up in the middle of the night when nothing is open and need parts. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
Leo Bicknell <bicknell@ufp.org> writes:
I've repeatedly asked $BIG_COLO_PROVIDERS to offer a vending machine in the lobby next to the one with sodas that sold Cat 5, Fiber, SFP's, USB sticks, and so on.
Hmm..... http://gearomat.com/ Jens -- ------------------------------------------------------------------------- | Foelderichstr. 40 | 13595 Berlin, Germany | +49-151-18721264 | | http://blog.quux.de | jabber: jenslink@guug.de | ------------------- | -------------------------------------------------------------------------
On Fri, Feb 17, 2012 at 18:06, George Bonser <gbonser@seven.com> wrote: ....
Fry's wanted $55 for a 1 meter LC-LC multi-mode patch cord yesterday at the store on Arques in Sunnyvale.
Admittedly high, but in the same store, one set of rows to the left (as you were looking at the fibres) they sell 12-24 rack screws for something like $10/bag of 12. Now *that* is markup.
On Feb 17, 2012, at 6:52 AM, -Hammer- wrote:
Let me simplify that. If you are over 35 you know how to troubleshoot.
Is this a statement or something to be added to the list of misconceptions that are commonplace out there? Not trying to be flip... Honestly asking the question. I can see it going either way, frankly. ;-) Owen
On Feb 17, 2012, at 1:35 PM, Owen DeLong wrote:
On Feb 17, 2012, at 6:52 AM, -Hammer- wrote:
Let me simplify that. If you are over 35 you know how to troubleshoot.
Is this a statement or something to be added to the list of misconceptions that are commonplace out there?
Not trying to be flip... Honestly asking the question. I can see it going either way, frankly. ;-)
Owen
Clearly, it is a misconception. The effect of aging on trouble-shooting skills is simply the opportunity for gaining experience. (Personal anecdotes omitted here. You are welcome.) James R. Cutler james.cutler@consultant.com
As someone who was born in 1984 I respectfully disagree. ;-) On Fri, Feb 17, 2012 at 9:52 AM, -Hammer- <bhmccie@gmail.com> wrote:
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc.
I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
-- Ray Soucy Epic Communications Specialist Phone: +1 (207) 561-3526 Networkmaine, a Unit of the University of Maine System http://www.networkmaine.net/
as a 33 year old, I'm looking forward to hitting 35 so I can finally understand what you guys are talking about! Will I get some sort of glow or achievement? think I'll get a raise when I can add 'troubleshooting' to my resume? :) On Fri, Feb 17, 2012 at 2:42 PM, Ray Soucy <rps@maine.edu> wrote:
As someone who was born in 1984 I respectfully disagree. ;-)
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both
On Fri, Feb 17, 2012 at 9:52 AM, -Hammer- <bhmccie@gmail.com> wrote: directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route
to
it, etc. etc.
I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
-- Ray Soucy
Epic Communications Specialist
Phone: +1 (207) 561-3526
Networkmaine, a Unit of the University of Maine System http://www.networkmaine.net/
Still buzzing over that cheap auto insurance eh? :) Wait till people stop carding you..... -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 1:42 PM, Ray Soucy wrote:
As someone who was born in 1984 I respectfully disagree. ;-)
On Fri, Feb 17, 2012 at 9:52 AM, -Hammer-<bhmccie@gmail.com> wrote:
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
Maybe ;-) I don't think it's an age thing, though. The number of people who have a real interest in technology, and how things work "under the hood" hasn't changed much. I know people 10 years younger than me who can keep up with the best of us, and people 10 years older who are complete failures at technology. People like us have always been a fairly small number. What has changed, though, is that there are a lot more young people who think they have technology skills; perhaps as a side effect of growing up in a world where the Internet has always been there. Naturally, we have a lot of people filling IT spots that aren't qualified and lack the basic knowledge of how complex systems are built. To troubleshoot effectively, you need to be able to break down systems into their components and isolate the problem; and a lot of people just don't have the background to be able to do that because they never cared to do so. It's just a paycheck to them. Those of us in my age group were lucky enough to be around for the transition from dial-up BBS, to dial-up Internet, to broadband. As a networking geek I don't think I could ask for a better year to be born, really. It's always been exciting. These days I'm playing with DWDM and a state wide R&E network in a state that established dark fiber as a public utility; doesn't get much better than that. I'd say the future is pretty bright. ;-) On Fri, Feb 17, 2012 at 3:26 PM, -Hammer- <bhmccie@gmail.com> wrote:
Still buzzing over that cheap auto insurance eh? :) Wait till people stop carding you.....
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 1:42 PM, Ray Soucy wrote:
As someone who was born in 1984 I respectfully disagree. ;-)
On Fri, Feb 17, 2012 at 9:52 AM, -Hammer-<bhmccie@gmail.com> wrote:
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc.
I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
-- Ray Soucy Epic Communications Specialist Phone: +1 (207) 561-3526 Networkmaine, a Unit of the University of Maine System http://www.networkmaine.net/
I couldn't argue with any of that. Again, there are exceptions on either side. -Hammer- "I was a normal American nerd" -Jack Herer On 2/17/2012 2:40 PM, Ray Soucy wrote:
Maybe ;-)
I don't think it's an age thing, though.
The number of people who have a real interest in technology, and how things work "under the hood" hasn't changed much. I know people 10 years younger than me who can keep up with the best of us, and people 10 years older who are complete failures at technology. People like us have always been a fairly small number.
What has changed, though, is that there are a lot more young people who think they have technology skills; perhaps as a side effect of growing up in a world where the Internet has always been there. Naturally, we have a lot of people filling IT spots that aren't qualified and lack the basic knowledge of how complex systems are built. To troubleshoot effectively, you need to be able to break down systems into their components and isolate the problem; and a lot of people just don't have the background to be able to do that because they never cared to do so. It's just a paycheck to them.
Those of us in my age group were lucky enough to be around for the transition from dial-up BBS, to dial-up Internet, to broadband. As a networking geek I don't think I could ask for a better year to be born, really. It's always been exciting.
These days I'm playing with DWDM and a state wide R&E network in a state that established dark fiber as a public utility; doesn't get much better than that.
I'd say the future is pretty bright. ;-)
On Fri, Feb 17, 2012 at 3:26 PM, -Hammer-<bhmccie@gmail.com> wrote:
Still buzzing over that cheap auto insurance eh? :) Wait till people stop carding you.....
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 1:42 PM, Ray Soucy wrote:
As someone who was born in 1984 I respectfully disagree. ;-)
On Fri, Feb 17, 2012 at 9:52 AM, -Hammer-<bhmccie@gmail.com> wrote:
Let me simplify that. If you are over 35 you know how to troubleshoot.
Yes, I'm going to get flamed. Yes, there are exceptions in both directions.
-Hammer-
"I was a normal American nerd" -Jack Herer
On 2/17/2012 8:29 AM, Leo Bicknell wrote:
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
+1 Mario Eirea ________________________________________ From: Leo Bicknell [bicknell@ufp.org] Sent: Friday, February 17, 2012 9:29 AM To: nanog@nanog.org Subject: Re: Common operational misconceptions In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc.
I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken. The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times. I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/
I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken.
Look for people who grew up on a farm. They are used to figuring out how to fix things they haven't seen before and generally attempt to gain knowledge of the fundamental principles of how things work so they can apply those principles in a similar situation. For example, such a person may know enough about troubleshooting both gasoline and diesel engines and might have a better understanding of the underlying fundamentals of internal combustion engines to do a passable job troubleshooting something they have never seen before (air, fuel, timing). There is a certain APPROACH to troubleshooting that transcends various fields. Some naturally have a talent for it, others aren't so good at it. Such people might be better in a multi-vendor network when there is a problem. You can generally spot those people not by what they know, but by the quality of the questions they ask. They generally know what they want to accomplish or what they are looking for, but they might want to know how that is done with this particular vendor's command set or how this particular vendor processes traffic. Some are natural designers, some are natural troubleshooters, some are natural documenters/support staff and they LIKE doing it. It takes all of those skills. One important thing to keep in mind, too, is that by identifying the skills and natural talents of your line staff, you yourself are being a value multiplier to your organization. You are making best use of the resources that you have at your disposal and are improving the efficiency of the organization as an organic entity. So this benefits everyone in the entire organization, including you.
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote:
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build things, not deal with them when they are broken. The Cisco CCNA syllabus used to emphasise the layer 1->7 approach to
On 02/17/2012 04:29 AM, Leo Bicknell wrote: troubleshooting. Not sure if they still do, or if trainers even bother to mention it (mine did back when I did it several years ago)
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times. Never trust what you can't prove yourself, that includes vendors and customers. Every now and then I forget this and find hours later that I've wasted a whole bunch of time because I trusted when someone said something that it actually was the case. It's really often better to test something a third time even if Vendor and Customer tell you something is a particular way.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
Definitely. I've learnt more in my time from breaking things than I've ever learnt setting them up; however the education system is focused on breadth of knowledge, not depth. Students are expected to be able to regurgitate ridiculous amounts of facts and figures, so that they pass standardised tests, not understand how to actually use them. Paul
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote: things, not deal with them when they are broken. The Cisco CCNA syllabus used to emphasise the layer 1->7 approach to
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times. Never trust what you can't prove yourself, that includes vendors and customers. Every now and then I forget this and find hours later that I've wasted a whole bunch of time because I trusted when someone said something
Exactly right. They have some much information floating around in their heads many of them cannot fit it together. But once they get on the job, all of those little synapses rapidly connect, and then the light comes on. Higher education is just like drivers education. You did not learn to drive in drivers education. You learned how to drive by driving. Higher education gives you the foundation on which to learn. -----Original Message----- From: Paul Graydon [mailto:paul@paulgraydon.co.uk] Sent: Friday, February 17, 2012 12:33 PM To: nanog@nanog.org Subject: Re: Common operational misconceptions On 02/17/2012 04:29 AM, Leo Bicknell wrote: troubleshooting. Not sure if they still do, or if trainers even bother to mention it (mine did back when I did it several years ago) that it actually was the case. It's really often better to test something a third time even if Vendor and Customer tell you something is a particular way.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
Definitely. I've learnt more in my time from breaking things than I've ever learnt setting them up; however the education system is focused on breadth of knowledge, not depth. Students are expected to be able to regurgitate ridiculous amounts of facts and figures, so that they pass standardised tests, not understand how to actually use them. Paul
Give me someone who can already think and analyse over someone who 'knows' it all, any day. You can be qualified to the hilt but absolutely useless in the real world (I've watched CCNP and higher struggling to figure out why they can't ping a 10.0.0.0/24 address at a customers remote site, not even realising it's a private range, let alone trying to trace the path of the ping,) If you're capable of symptoms->synthesis->solution you're of much more use to me. You can pick up technical knowledge on the job, or around the job. It's extremely hard to mold someone's thinking patterns by the time they're adults. When we interview we try to spend more time trying to gauge problem solving capabilities than anything else, after first quickly establishing their technical level. Paul On 2/17/2012 8:43 AM, Kenneth M. Chipps Ph.D. wrote:
Exactly right. They have some much information floating around in their heads many of them cannot fit it together. But once they get on the job, all of those little synapses rapidly connect, and then the light comes on.
Higher education is just like drivers education. You did not learn to drive in drivers education. You learned how to drive by driving. Higher education gives you the foundation on which to learn.
-----Original Message----- From: Paul Graydon [mailto:paul@paulgraydon.co.uk] Sent: Friday, February 17, 2012 12:33 PM To: nanog@nanog.org Subject: Re: Common operational misconceptions
At the same time, it's shocking how many network people I come across with no real grasp of even what OSI means by each layer, even if it's only in theory. Just having a grasp of that makes all the world of difference when it comes to troubleshooting. Start at layer 1 and work upwards (unless you're able to make appropriate intuitive leaps.) Is it physically connected? Are the link lights flashing? Can traffic route to it, etc. etc. I wouldn't call it a "misconception", but I want to echo Paul's comment. I would venture over 90% of the engineers I work with have no idea how to troubleshoot properly. Thinking back to my own education, I don't recall anyone in highschool or college attempting to teach troubleshooting skills. Most classes teach you how to build
In a message written on Thu, Feb 16, 2012 at 08:50:11PM -1000, Paul Graydon wrote: things, not deal with them when they are broken. The Cisco CCNA syllabus used to emphasise the layer 1->7 approach to
On 02/17/2012 04:29 AM, Leo Bicknell wrote: troubleshooting. Not sure if they still do, or if trainers even bother to mention it (mine did back when I did it several years ago)
The basic skills are probably obvious to someone who might design course material if they sat down and thought about how to teach troubleshooting. However, there is one area that may not be obvious. There's also a group management problem. Many times troubleshooting is done with multiple folks on the phone (say, customer, ISP and vendor). Not only do you have to know how to troubleshoot, but how to get everyone on the same page so every possible cause isn't tested 3 times. Never trust what you can't prove yourself, that includes vendors and customers. Every now and then I forget this and find hours later that I've wasted a whole bunch of time because I trusted when someone said something that it actually was the case. It's really often better to test something a third time even if Vendor and Customer tell you something is a particular way.
I think all college level courses should include a "break/fix" exercise/module after learning how to build something, and much of that should be done in a group enviornment.
Definitely. I've learnt more in my time from breaking things than I've ever learnt setting them up; however the education system is focused on breadth of knowledge, not depth. Students are expected to be able to regurgitate ridiculous amounts of facts and figures, so that they pass standardised tests, not understand how to actually use them.
Paul
Paul Graydon wrote:
Give me someone who can already think and analyse over someone who 'knows' it all, any day. You can be qualified to the hilt but absolutely useless in the real world (I've watched CCNP and higher struggling to figure out why they can't ping a 10.0.0.0/24 address at a customers remote site, not even realising it's a private range, let alone trying to trace the path of the ping,)
Hard to believe, but you're obviously serious. What are their job titles? What were they hired to accomplish? Also hard for me to understand that someone could study for CCNx and not get exposed to Private space and 1918...what am I missing? --Michael
On 2/17/2012 10:55 PM, Michael Painter wrote:
Paul Graydon wrote:
Give me someone who can already think and analyse over someone who 'knows' it all, any day. You can be qualified to the hilt but absolutely useless in the real world (I've watched CCNP and higher struggling to figure out why they can't ping a 10.0.0.0/24 address at a customers remote site, not even realising it's a private range, let alone trying to trace the path of the ping,)
Hard to believe, but you're obviously serious. What are their job titles? What were they hired to accomplish? Also hard for me to understand that someone could study for CCNx and not get exposed to Private space and 1918...what am I missing?
Yes I'm serious, they were CCNP qualified, hired as a NOC engineer for an ISP & Hosting company. For the company the NOC team was the top tier of customer support (3rd line+), they looked after routers, switches, firewalls, servers, leased lines, and so on. This individual was perfectly capable of regurgitating all the facts, figures and technical details you can imagine, probably pretty much the entire CCNP syllabus. What they didn't seem that capable of was actually applying that to anything. I'd bet good money that if I'd asked him at the time what the 1918 network ranges are he'd have been able to tell me. This is exactly what we're teaching kids to do these days (makes me feel so old that I've already been saying this for several years and I'm only 31) standardised tests aren't marked based on ability to apply knowledge, just the knowledge itself. Hence my view, give me someone who knows how to think over someone who is qualified to the hilt. These exam cram 'do a CCNP in a week' courses only serve to make it worse. Paul
Yes I'm serious, they were CCNP qualified, hired as a NOC engineer for an ISP & Hosting company.
There was a time a new hire with all the right holes punched in his ticket deleted an item in an access-list in a PIX that was running an older version of the software than he was familiar with. The entire access-list disappeared and he was locked out, production stopped, like a train hitting a brick wall.
For the company the NOC team was the top tier of customer support (3rd line+), they looked after routers, switches, firewalls, servers, leased lines, and so on. This individual was perfectly capable of regurgitating all the facts, figures and technical details you can imagine, probably pretty much the entire CCNP syllabus. What they didn't seem that capable of was actually applying that to anything.
You might be surprised at how common that is. If you present them with ALL the diagrams and ALL of the configs on paper, they can figure it out. In other words, if you recreate the same environment they had in their training class, they can do fine. But what some can't seem to be able to do is visualize in their head how things are. It is that layer of abstraction that separates them out. They are fine for maintaining documentation or even for participating in a design review but you don't want them designing some new addition to the network or working on something "live". My first clue often comes from the quality of diagrams they produce. If the diagrams are accurate as far as what connects to what but do not reflect the actual flow of the network, that's a telltale sign. Sort of like an electronic schematic. If they sort of have random components/stages at random locations in the drawing that doesn't really reflect the functional flow through the device, that is my clue that I am likely dealing with a concrete thinker and not an abstract thinker. Ditto if they demand that the symbol representing a particular piece of gear actually be a picture of that piece of gear. If they get lost when gear is represented by a square box then they are probably part of the normal 85% of people who find it more difficult (actually have to try) to translate a square box on a diagram to a router in the rack in their head vs the 15% who do that naturally without any effort. The access-list guy mentioned above would be great for looking at the config and producing a new one with the correct access control, but you wouldn't want him to be the one to apply it in production on a live network. So even in that guy's case, there is a place where their skills can be quite useful and there are other places where their chance of making a costly mistake increase. It is a matter of matching the person's role to their skills.
I'd bet good money that if I'd asked him at the time what the 1918 network ranges are he'd have been able to tell me.
You'll be surprised how many people "forget" that 172.28.0.0 is rfc1918 space. They are so used to seeing 172.16 that they tend to forget 172.17-31. I've had to change null routes and access controls to include the entire /12. They "know" that it is a /12 but seem to forget in practice when they see a second octet that isn't "16".
This is exactly what we're teaching kids to do these days (makes me feel so old that I've already been saying this for several years and I'm only 31) standardised tests aren't marked based on ability to apply knowledge, just the knowledge itself.
Yes. We teach them facts but now how to FIND facts. Part of teaching is in teaching how to teach yourself. It started with me when I was a kid. When I had a question, my father would always say "look it up and tell me" even if he knew the answer perfectly well. He had invested in an encyclopedia and the annual updates and was determined that I would use it. It taught me how to research to find my own answers and it taught me to learn it well enough to explain it to someone else because Pop would always throw in a couple of questions for me after I explained it to him just to see if I actually "got" the underlying concept. Besides, often in the course of researching one thing, I happened across a completely unrelated thing that caught my interest in that volume of the book and learned something I hadn't even been looking for. Forget the Internet, for people with kids at home, I would recommend a hard copy set of World Book with the Year Book and Science Year annual additions. That one in particular because the style in which they are written, they are actually pretty fun to read and have a lot of illustrations. http://www.worldbook.com/browse-all-products/item/953-advanced-research-pack... (no affiliation at all with them, just a satisfied "customer"). Soon, going to the books when a question arose became natural. It is one thing to produce a "teachable" child, something quite different to produce the ability to learn independently and allow their own natural curiosity to "pull" them to that knowledge.
Paul Graydon wrote: >> Yes I'm serious, they were CCNP qualified, hired as a NOC engineer for > an ISP & Hosting company. For the company the NOC team was the top tier > of customer support (3rd line+), they looked after routers, switches, > firewalls, servers, leased lines, and so on. > This individual was perfectly capable of regurgitating all the facts, > figures and technical details you can imagine, probably pretty much the > entire CCNP syllabus. What they didn't seem that capable of was > actually applying that to anything. I'd bet good money that if I'd > asked him at the time what the 1918 network ranges are he'd have been > able to tell me. > This is exactly what we're teaching kids to do these days (makes me feel > so old that I've already been saying this for several years and I'm only > 31) standardised tests aren't marked based on ability to apply > knowledge, just the knowledge itself. Hence my view, give me someone > who knows how to think over someone who is qualified to the hilt. These > exam cram 'do a CCNP in a week' courses only serve to make it worse. > > Paul Ahh, I get you now...thanks. Took me back to '64 and the battery of tests (all day!) I was given before getting hired by IBM for the 360 rollout. I was amazed by the amount of questions of the "if gear a turns ccw, what does lever b do?" variety. Later I was told that -all- the testing results were important, even the psychological ones, but what they really wanted to find was the best analytical *mind*. Best, --Michael
On Sat, 18 Feb 2012, Paul Graydon wrote:
Yes I'm serious, they were CCNP qualified, hired as a NOC engineer for an ISP & Hosting company. For the company the NOC team was the top tier of customer
The CCNP was a success from the point of view of the person. It got the person hired by an ISP & Hosting company to work in the top tier of customer support. Is it the fault of the CCNP that the hiring process at the ISP & Hosting company couldn't tell the difference? If anyone has suggestions for improving the network and information security education in the US, please send your suggestions to the folks at the National Initiative for Cybersecurity Education (NICE). http://csrc.nist.gov/nice/ I've been trying to hire 5 federal civil servants (GS7 to GS13) to work on informations security architecture and program management at small and micro US government agencies. Its amazing how hard it is to find people that both understand the technology and can think beyond the manual. I understand the attraction of using certificates as a screening mechanism by HR departments. But that is why you have the job interview. -- P.S. I have a functional APC Back-UPS XS1500 (4+ year old batteries) if anyone in Northern Virginia wants to pick it up. Otherwise I have to pay the battery recycling/disposal fee.
On Friday, February 17, 2012 01:30:30 AM Carsten Bormann wrote:
Ah, one of the greatest misconceptions still around in 2012:
-- OSI Layer numbers mean something. or -- Somewhere in the sky, there is an exact definition of what is layer 2, layer 3, layer 4, layer 5 (!), layer 7
Misconception: Layers are not recursive. Thanks to tunneling/MPLS/other encapsulation techniques, they are.
On 02/16/12 05:17, Ray Soucy wrote:
I've found starting off with some history on Ethernet (Maine loves Bob Metcalfe) becomes a very solid base for understanding; how "Ethernet" today is very different; starting with hubs, bridges, collisions, and those problems, then introducing modern switching, VLANs, broadcast domain's etc.
There was an old cruddy 1950s building on the UCB campus called Stanley Hall. (Now there's a new, nice, modern building on the UCB campus called Stanley Hall in place of the old one.) The old building had both UCB net and Lawrence Berkeley Lab (LBNL) net running through it. The LBNL net was fed from fiber using a 10base-FL-to-AUI repeater. The AUI connected into the coax spine. The cool thing is that the coax spine was provisioned exactly as you would expect in an old ethernet textbook. The spine ran through the hallway (usually just tied to a pipe, but sometimes with a J-hook) and every 1.5 meters, there was a vampire tap and (usually) a transceiver with an AUI cable connected to it that went into an office and dropped down through the ceiling. Amazingly, the UCB network was somewhat more modern. There were DEC Delni MPTs (or "AUI hubs") coming off the UCB coax. There were even some 10base-T hubs and concentrators that fed offices that had newer cat-3 or even cat-5 wiring. It was great to take students on tours through this operational museum. (Well, the LBNL net was sort of operational. It would just stop working for minutes on end and then come back mysteriously.) You could basically see the first 10-15 years of the evolution of ethernet, and it was installed and working. The new Stanley is plumbed to the gills with cat-5e, gigabit switches, and vlans all over the place. A modern LAN, yes, but no sense of history. michael
On Thu, 16 Feb 2012, Michael Sinatra wrote:
There was an old cruddy 1950s building on the UCB campus called Stanley Hall. (Now there's a new, nice, modern building on the UCB campus called Stanley Hall in place of the old one.)
It was great to take students on tours through this operational museum. (Well, the LBNL net was sort of operational. It would just stop working for minutes on end and then come back mysteriously.) You could basically see the first 10-15 years of the evolution of ethernet, and it was installed and working.
I have a few cruddy old 1950s (or earlier) buildings on campus where I can find thicknet and (dead for many years) vampire taps, along with thinnet, many different vintages of voice and data cabling, and many different vintages of fiber, including lots of OM1 multimode. Makes for some very interesting pictures and "what *is* that?" conversations. jms
Wouldn't know about that part. You would have to ask an ntt employee. Jared Mauch On Feb 16, 2012, at 8:03 AM, Owen DeLong <owen@delong.com> wrote:
Yes, I'm well aware of the problems being created by the attempts by NTT to force the government to let them be a residential ISP.
On 2012.02.15 19:19, Masataka Ohta wrote:
IPv6 is operational.
This is an intriguing statement. Any ops/eng I know who have claimed this, actually know what they are talking about, so it is factual. I've never heard anyone claim this in a way that could be a misconception. I state further in this sub-thread how the opposite could be true though :) Steve
On 2012.02.15 15:47, John Kristoff wrote:
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
It is ok to use non-rfc1918 (allocated/assigned) IP space internally, because this network will NEVER see the Internet. Steve
On 2012.02.15 19:23, Steve Bertrand wrote:
On 2012.02.15 15:47, John Kristoff wrote:
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
It is ok to use non-rfc1918 (allocated/assigned) IP space internally, because this network will NEVER see the Internet.
...referring to space they don't own of course. Did a lot of IP address re-design for companies who suddenly couldn't reach microsoft.com years ago.
ULA is the IPv6 equivalent of RFC1918 RFCs are standards (i.e. all of them, or RFC is synonymous with standard) The words "Internet" and "Web" can be used interchangeably Not only does NAT provide "security," but it's NECESSARY for "security." Alternatively, you can't possibly be as secure without NAT than with. Link capacity is how fast the bits move through the wire Security is the responsibility of the Security Group
Michael Sinatra wrote:
The words "Internet" and "Web" can be used interchangeably
I prefer the term "intergophers" myself. -- Earthquake Magnitude: 4.9 Date: Friday, February 17, 2012 14:28:20 UTC Location: Komandorskiye Ostrova, Russia region Latitude: 54.5969; Longitude: 168.8863 Depth: 34.70 km
On Thu, Feb 16, 2012 at 4:46 AM, Michael Sinatra <michael@rancid.berkeley.edu> wrote:
ULA is the IPv6 equivalent of RFC1918
Michael, could you explain this a bit more? In the sense that : a. Anyone can use ULA pretty much as they wish without having to go to their ISP or RIR - same for RFC1918 b. In order to get to the public Internet, with ULA addressing, some kind of translation is required - same for RFC1918 c. Without centralised registration, two different networks could end up using same ULA space - same for RFC1918 There are certainly not identical but I'd think loosely equivalent. What am I missing?
-- Mukom Akong [Tamon] ______________ “We don't LIVE in order to BREATH. Similarly WORKING in order to make MONEY puts us on a one way street to irrelevance.“ [In Search of Excellence & Perfection] - http://perfexcellence.org [Moments of TechXcellence] - http://techexcellence.net [ICT Business Integration] - http://ibiztech.wordpress.com [About Me] - http://about.me/perfexcellence
On 03/03/12 00:33, Mukom Akong T. wrote:
On Thu, Feb 16, 2012 at 4:46 AM, Michael Sinatra <michael@rancid.berkeley.edu> wrote:
ULA is the IPv6 equivalent of RFC1918
Michael, could you explain this a bit more? In the sense that :
a. Anyone can use ULA pretty much as they wish without having to go to their ISP or RIR - same for RFC1918 b. In order to get to the public Internet, with ULA addressing, some kind of translation is required - same for RFC1918
Actually, (b) isn't quite correct, especially depending on how you define "the public Internet." If you define it as "the DFZ," then we're *probably* okay. If you define it as "any commercial ISP and/or any inter-AS traffic," then it's not so clear. To plagiarize myself in a previous private response to someone else: ULAs allow for native routing across a commercial ISP backbone to support "extranets" (ugh, I hate that word)--essentially to allow an enterprise to use a regular ISP (or ISPs) to carry ULA traffic between sites. RFC 1918 requires that all privately-addressed traffic be encapsulated if it is routed into another AS. The consequence is that any AS can filter all RFC 1918 routes *and* traffic at its border(s) and ISPs can (and SHOULD) refuse to route unencapsulated RFC 1918-addressed traffic between ASes. ULAs may require holes to be poked in any blanket filter. I see that as a significant difference because you can no longer "guarantee" non-routability of the space, which is what people tend to count on. (We hope this won't happen, but it's not explicitly prevented by RFC 4193 as it is by RFC 1918. And see Ted Hardie's "rant" on the subject at NANOG 40 for an argument that it might: www.nanog.org/meetings/nanog40/presentations/ula-nanog.pdf) My own view on this is that it's likely that ULA will stay out of the DFZ (due to the now-widespread availability of IPv6 PI), but that you may not be able to count on security (i.e. *traffic*) filters being magically in place everywhere as one does for RFC 1918. (Again, that's probably a misconception of RFC 1918, but there is still a big difference in the potential for the consistent violation of the "magic filter" assumption for ULA.)
c. Without centralised registration, two different networks could end up using same ULA space - same for RFC1918
Yeah, but there's an orders-of-magnitude difference in the ability to generate a reasonable expectation of uniqueness. Look at common RFC1918 usage--it often falls into 192.168.0.0/23 (e.g. most CPE NAT devices) or 10.0.0.0/23. Larger users tend to be all over net 10, which makes merging networks a lot harder. It's much more likely that such mergers will work better with ULA. The large ULA space had put pressure on the standards community to define something like ULA-C, but PI IPv6 has relieved that pressure. However, the fact that ULA-C-like-things were ever proposed illustrates the differences between ULA and RFC 1918 space.
There are certainly not identical but I'd think loosely equivalent. What am I missing?
There's also a third difference that interacts with the typical misconception that RFC 1918 implies or somehow specifies NAT (which I think I mentioned elsewhere). When people think that RFC 1918 == NAT, they get really surprised that there's no stateful NAT in IPv6. Given the address space of IPv6, stateless prefix translation could go a long way toward giving one the same functionality, but people tend to want NAT to perform the stateful overloading of public IP addresses. So there are really three misconceptions at work here: RFC 1918 implies NAT NAT is defined as stateful overloading of a small pool of public IP addresses to support lots of private IP addresses. (Stateless NAT? Whut??) ULAs are the same as RFC 1918 addresses I realize that last one is cheating a bit because it it requires three misconceptions to create the resultant confusion about there not being stateful NAT66, but it *does* exist in the wild. michael
NANOG don't need no stinkin' glossary, everybody knows what our alphabet soup means. Getting a file by bittorrent will always be faster and stress the network less than downloading it by FTP or HTTP. The best wide-area network topology is exactly the same as that used by the Bell network of decades ago. Corollary of the above, the best back-up route between San Francisco and Los Angeles in the event of a fiber cut in San Jose is Chicago or Virginia, not Fresno or Bakersfield. The only way to provide Metropolitan Optical Ethernet is to install a Cisco router that costs over one million dollars. Distance does not matter. Serve your site from California or Virginia, and it will work in the panhandle of Oklahoma or the Australian outback just as well as a closer location would. Fiber is just too fast, all networking should be wireless. No traffic is ever wasteful, just get bigger pipes and all problems will be solved.
Something that makes me crawl out of my skin is when they refer to an access point as "router". -Mario Eirea On Feb 15, 2012, at 3:47 PM, "John Kristoff" <jtk@cymru.com> wrote:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
I whole-heartedly agree with that last one. -Grant On Wed, Feb 15, 2012 at 8:07 PM, Mario Eirea <meirea@charterschoolit.com>wrote:
Something that makes me crawl out of my skin is when they refer to an access point as "router".
-Mario Eirea
On Feb 15, 2012, at 3:47 PM, "John Kristoff" <jtk@cymru.com> wrote:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
Mario Eirea (meirea) writes:
Something that makes me crawl out of my skin is when they refer to an access point as "router".
I have colleagues that work with radio and wireless, and they crawl out of *their* skin when I call an access point an access point, and they tell me it's a station :)
"IS-IS is a legacy protocol that nobody uses" 15.02.2012 22:47, John Kristoff kirjoitti:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
How widespread would you say the use of IS-IS is? Even more as to which routing protocols are used, not just in ISPs, what percent would you give to the various ones. In other words X percent of organizations use OSPS, Y percent use EIGRP, and so on. -----Original Message----- From: Antti Ristimäki [mailto:antti.ristimaki@gmx.com] Sent: Wednesday, February 15, 2012 10:47 PM To: John Kristoff Cc: nanog@nanog.org Subject: Re: Common operational misconceptions "IS-IS is a legacy protocol that nobody uses" 15.02.2012 22:47, John Kristoff kirjoitti:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
On 2/15/12 21:04 , Kenneth M. Chipps Ph.D. wrote:
How widespread would you say the use of IS-IS is?
Even more as to which routing protocols are used, not just in ISPs, what percent would you give to the various ones. In other words X percent of organizations use OSPS, Y percent use EIGRP, and so on.
Using EIGRP implies your routed IGP dependent infrastructure is a monoculture. That's probably infeasible without compromise even if you are largely a Cisco shop. ISIS is used in organizations other than ISPs.
-----Original Message----- From: Antti Ristimäki [mailto:antti.ristimaki@gmx.com] Sent: Wednesday, February 15, 2012 10:47 PM To: John Kristoff Cc: nanog@nanog.org Subject: Re: Common operational misconceptions
"IS-IS is a legacy protocol that nobody uses"
15.02.2012 22:47, John Kristoff kirjoitti:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
"ISIS is used in organizations other than ISPs" Any examples you can share of some other than ISPs? -----Original Message----- From: Joel jaeggli [mailto:joelja@bogus.com] Sent: Wednesday, February 15, 2012 11:58 PM To: Kenneth M. Chipps Ph.D. Cc: nanog@nanog.org Subject: Re: Common operational misconceptions On 2/15/12 21:04 , Kenneth M. Chipps Ph.D. wrote:
How widespread would you say the use of IS-IS is?
Even more as to which routing protocols are used, not just in ISPs, what percent would you give to the various ones. In other words X percent of organizations use OSPS, Y percent use EIGRP, and so on.
Using EIGRP implies your routed IGP dependent infrastructure is a monoculture. That's probably infeasible without compromise even if you are largely a Cisco shop. ISIS is used in organizations other than ISPs.
-----Original Message----- From: Antti Ristimäki [mailto:antti.ristimaki@gmx.com] Sent: Wednesday, February 15, 2012 10:47 PM To: John Kristoff Cc: nanog@nanog.org Subject: Re: Common operational misconceptions
"IS-IS is a legacy protocol that nobody uses"
15.02.2012 22:47, John Kristoff kirjoitti:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
Some recent questions from interview and lab sessions I took. - I've allowed vlan X on trunk but still its not working? why do I have to create it on every switch? - any-any rules on firewall with AV enabled is better. - ACL inboud/outbout misconcept. Always end up cutting the rope. - BGP is for ISPs only. - MPLS is for security and is fast. Regards, Aftab A. Siddiqui On Thu, Feb 16, 2012 at 11:00 AM, Kenneth M. Chipps Ph.D. <chipps@chipps.com
wrote:
"ISIS is used in organizations other than ISPs" Any examples you can share of some other than ISPs?
-----Original Message----- From: Joel jaeggli [mailto:joelja@bogus.com] Sent: Wednesday, February 15, 2012 11:58 PM To: Kenneth M. Chipps Ph.D. Cc: nanog@nanog.org Subject: Re: Common operational misconceptions
On 2/15/12 21:04 , Kenneth M. Chipps Ph.D. wrote:
How widespread would you say the use of IS-IS is?
Even more as to which routing protocols are used, not just in ISPs, what percent would you give to the various ones. In other words X percent of organizations use OSPS, Y percent use EIGRP, and so on.
Using EIGRP implies your routed IGP dependent infrastructure is a monoculture. That's probably infeasible without compromise even if you are largely a Cisco shop.
ISIS is used in organizations other than ISPs.
-----Original Message----- From: Antti Ristimäki [mailto:antti.ristimaki@gmx.com] Sent: Wednesday, February 15, 2012 10:47 PM To: John Kristoff Cc: nanog@nanog.org Subject: Re: Common operational misconceptions
"IS-IS is a legacy protocol that nobody uses"
15.02.2012 22:47, John Kristoff kirjoitti:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
We run IS-IS at the University of Pennsylvania as the IGP for IPv6. I know of a few other non-ISPs too but I won't speak for them. At the time we initially deployed IPv6, it was pretty much one of the few safe choices (OSPFv3 implementations were very new then). --Shumon. On Thu, Feb 16, 2012 at 12:00:04AM -0600, Kenneth M. Chipps Ph.D. wrote:
"ISIS is used in organizations other than ISPs" Any examples you can share of some other than ISPs?
-----Original Message----- From: Joel jaeggli [mailto:joelja@bogus.com] Sent: Wednesday, February 15, 2012 11:58 PM To: Kenneth M. Chipps Ph.D. Cc: nanog@nanog.org Subject: Re: Common operational misconceptions
On 2/15/12 21:04 , Kenneth M. Chipps Ph.D. wrote:
How widespread would you say the use of IS-IS is?
Even more as to which routing protocols are used, not just in ISPs, what percent would you give to the various ones. In other words X percent of organizations use OSPS, Y percent use EIGRP, and so on.
Using EIGRP implies your routed IGP dependent infrastructure is a monoculture. That's probably infeasible without compromise even if you are largely a Cisco shop.
ISIS is used in organizations other than ISPs.
-- Shumon Huque University of Pennsylvania.
I would say that the average University is more of an unusual ISP than a non-ISP. Almost every University I know of has a networking group that functions like an ISP for the various departments of the college(s) as well as providing essentially residential ISP services to their residence halls and in some cases fraternities, faculty housing, etc. From a networking perspective they tend to operate much more like an ISP than an enterprise. One of the key defining differences (IMHO) is that an enterprise (mostly) trusts the employees connected to its network whereas an ISP and a University cannot. Owen On Feb 16, 2012, at 6:08 AM, Shumon Huque wrote:
We run IS-IS at the University of Pennsylvania as the IGP for IPv6. I know of a few other non-ISPs too but I won't speak for them. At the time we initially deployed IPv6, it was pretty much one of the few safe choices (OSPFv3 implementations were very new then).
--Shumon.
On Thu, Feb 16, 2012 at 12:00:04AM -0600, Kenneth M. Chipps Ph.D. wrote:
"ISIS is used in organizations other than ISPs" Any examples you can share of some other than ISPs?
-----Original Message----- From: Joel jaeggli [mailto:joelja@bogus.com] Sent: Wednesday, February 15, 2012 11:58 PM To: Kenneth M. Chipps Ph.D. Cc: nanog@nanog.org Subject: Re: Common operational misconceptions
On 2/15/12 21:04 , Kenneth M. Chipps Ph.D. wrote:
How widespread would you say the use of IS-IS is?
Even more as to which routing protocols are used, not just in ISPs, what percent would you give to the various ones. In other words X percent of organizations use OSPS, Y percent use EIGRP, and so on.
Using EIGRP implies your routed IGP dependent infrastructure is a monoculture. That's probably infeasible without compromise even if you are largely a Cisco shop.
ISIS is used in organizations other than ISPs.
-- Shumon Huque University of Pennsylvania.
"You can trust your supplier's marketing literature." Heck, maybe just "You can trust your supplier." ==ml
15.02.2012 22:47, John Kristoff kirjoitti:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
-- Michael W. Lucas http://www.MichaelWLucas.com/, http://blather.MichaelWLucas.com/ Latest book: SSH Mastery http://www.michaelwlucas.com/nonfiction/ssh-mastery mwlucas@BlackHelicopters.org, Twitter @mwlauthor
On Feb 15, 2012, at 12:47 PM, John Kristoff wrote:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
I think one of the most damaging fundamental misconceptions which is not only rampant among students, but, also enterprise IT professionals is the idea that NAT is a security tool and the inability to conceive of the separation between NAT (header mutilation) and Stateful Inspection (policy enforcement). Owen
On 02/15/12 23:34, Owen DeLong wrote:
I think one of the most damaging fundamental misconceptions which is not only rampant among students, but, also enterprise IT professionals is the idea that NAT is a security tool and the inability to conceive of the separation between NAT (header mutilation) and Stateful Inspection (policy enforcement).
Another misconception is that RFC 1918 somehow implies/specifies/requires NAT. The idea of using private address without NATing them seems to totally bewilder some people. And they often can't wrap their heads around the possibility of routing RFC 1918 space internally and also not using NAT. (This causes them to be even more confused at the fact that RFC 4193 specifies ULA for IPv6, but there is no stateful NAT currently specified.) Concepts/words that often get confused: Difference between 'allocation' and 'assignment' in IP addressing. Use of the word "IP" alone to mean "IP address," e.g.: Person: "Does that server have an IP assigned?" Me: "Yeah, it's got a whole stack." Then, of course, there's the silly situation where people mean to say "rogue" but they type "rouge" as in "rouge DHCP server," "rouge RA advertiser," etc. michael
On Feb 16, 2012, at 23:41, Michael Sinatra wrote:
Use of the word "IP" alone to mean "IP address," e.g.:
Person: "Does that server have an IP assigned?" Me: "Yeah, it's got a whole stack."
Yeah, and P: "Can you give me your email?" M: "All 20 GB of it?" Grüße, Carsten
On 15 Feb 2012, at 20:50, "John Kristoff" <jtk@cymru.com> wrote:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
When I took an A level computing course in the 90s the course material still talked about primary stor and backing stor, batch jobs and the like... Needless to say I quit in disgust but the point is that the people who write these courses are often woefully out of touch. -- Leigh ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com ______________________________________________________________________
When I took an A level computing course in the 90s the course material still talked about primary stor and backing stor, batch jobs and the like...
I was working with a lot of batch jobs in my first development role in 1993, and still supporting overnight scheduling to make best use of the Cray by 1999. I still leave the occasional big data set crunching overnight now. I'll grant you it's not exactly mainstream computing, but it's not exactly up there with drum memory either... The concept that a computer can do things when a person isn't there, or without the need for clicking things, is probably not a bad one to impart. Regards, Tim.
The idea that the network will always match the documentation. I have walked into many projects where this is not the case, but a Junior Admin working with me can't seem to get around the fact that the Visio we were handed isn't to be trusted and we've got to double-check everything. -Brett Lykins On Feb 15, 2012, at 3:47 PM, John Kristoff <jtk@cymru.com> wrote:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
Teach the TCP/IP model ... On 16/02/2012, John Kristoff <jtk@cymru.com> wrote:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
With telcos increasingly implementing Metro Ethernet Forum (MEF) networks, I have found that telco technicians tasked with maintaining and operating these carrier Ethernet networks appear to disregard common high availability practices. For instance, after diagnosing a routing protocol neighbor flap (consistently 20-30 seconds) and isolating the problem to the carrier MEF network, the carrier technician told me that the problem was a spanning tree convergence that took their primary and back-up links down during convergence, but that "this is no big deal because the network was only down for 20-30 seconds". -----Original Message----- From: John Kristoff [mailto:jtk@cymru.com] Sent: Wednesday, February 15, 2012 12:47 PM To: nanog@nanog.org Subject: Common operational misconceptions Hi friends, As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct. For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing. I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with. I'd prefer replies off-list and can summarize back to the list if there is interest. John This communication, together with any attachments or embedded links, is for the sole use of the intended recipient(s) and may contain information that is confidential or legally protected. If you are not the intended recipient, you are hereby notified that any review, disclosure, copying, dissemination, distribution or use of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by return e-mail message and delete the original and all copies of the communication, along with any attachments or embedded links, from your system.
I'm surprised I haven't seen QoS mentioned! If you're teaching college students, you might want to go over stuff that directly relates to what they're doing at home, or misconceptions they might make in a small WAN/ISP environment. *Why disabling ICMP doesn't increase security and only hurts the web* *(path MTU discovery, diagnostics) *How NAT breaks end-to-end connectivity (fun one..., took me hours to explain to an old boss why doing NAT at the ISP level was horrendously wrong) *Not to be afraid of ACLs on an edge router. Understanding what does/doesn't affect cpu utilization *Layer 3 Switch vs Router. Old concepts like switch vs router need to be clarified... *When vendors and numbers lie ;) aka *oversubscription*! *MAC is not security *Irrelevant security concepts (smurf attacks, ping of death). More focus should be on real modern day security concerns, like layer 7 exploits, router software 0days, VLAN hopping, and UDP floods and BGP spoofing. This might be a good place to explain why downloading IOS firmware from thepiratebay is a bad idea :) This is just coming from a sysadmin who likes to play with network gear and once endured college networking classes. Thanks! Andreas On Wed, Feb 15, 2012 at 1:47 PM, John Kristoff <jtk@cymru.com> wrote:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
Andreas Echavez wrote:
*Why disabling ICMP doesn't increase security and only hurts the web* *(path MTU discovery, diagnostics)
That PMTUD works is a misconception.
*How NAT breaks end-to-end connectivity (fun one..., took me hours to explain to an old boss why doing NAT at the ISP level was horrendously wrong)
That's another misconception. While NAT breaks the end to end connectivity, it can be restored by end systems by reversing translations by NAT, if proper information on the translations are obtained through some protocol such as UPnP. Masataka Ohta
2012/2/16 Masataka Ohta <mohta@necom830.hpcl.titech.ac.jp>:
Andreas Echavez wrote:
*How NAT breaks end-to-end connectivity (fun one..., took me hours to explain to an old boss why doing NAT at the ISP level was horrendously wrong)
That's another misconception.
While NAT breaks the end to end connectivity, it can be restored by end systems by reversing translations by NAT, if proper information on the translations are obtained through some protocol such as UPnP.
Masataka Ohta
UPnP can scale to about the size of an average home use, but it's worth jack squat at the ISP level when NAT44 comes into play. UPnP is not an ISP grade solution, it's a consumer one.
Josh Hoppes wrote:
While NAT breaks the end to end connectivity, it can be restored by end systems by reversing translations by NAT, if proper information on the translations are obtained through some protocol such as UPnP.
UPnP can scale to about the size of an average home use,
It depends on how you use UPnP and version of it. E.g. security to control UPnP gateway is not a problem if the gateway is configured statically.
but it's worth jack squat at the ISP level when NAT44 comes into play.
NAT44 or 444 means you only need small scale UPnP cascaded.
UPnP is not an ISP grade solution, it's a consumer one.
As I said "such as", PCP may be fine. But, recently, I found several fundamental defects in it, some if which are already corrected but the discussion is still continuing in its WG. Masataka Ohta
On Fri, 17 Feb 2012 10:11:22 +0900, Masataka Ohta said:
While NAT breaks the end to end connectivity, it can be restored by end systems by reversing translations by NAT, if proper information on the translations are obtained through some protocol such as UPnP.
You got a front end NAT. You got 3 boxes behind it that all want to listen for inbound connections on port 49734. Let me know how that works out for you.
Valdis.Kletnieks@vt.edu wrote:
While NAT breaks the end to end connectivity, it can be restored by end systems by reversing translations by NAT, if proper information on the translations are obtained through some protocol such as UPnP.
You got a front end NAT. You got 3 boxes behind it that all want to listen for inbound connections on port 49734.
Let me know how that works out for you.
It's just like your box can't listen for inbound connections at address 131.112.32.132 (address of my box). However, if UPnP box is configured properly, your box behind it can listen for inbound connections on some ports at some public address. Issues on stability and advertisements of the port numbers are not very different from those of addresses, which may be assigned by DHCP. Masataka Ohta
On Fri, 17 Feb 2012 11:07:59 +0900, Masataka Ohta said:
Valdis.Kletnieks@vt.edu wrote:
While NAT breaks the end to end connectivity, it can be restored by end systems by reversing translations by NAT, if proper information on the translations are obtained through some protocol such as UPnP.
You got a front end NAT. You got 3 boxes behind it that all want to listen for inbound connections on port 49734.
Let me know how that works out for you.
It's just like your box can't listen for inbound connections at address 131.112.32.132 (address of my box).
However, if UPnP box is configured properly, your box behind it can listen for inbound connections on some ports at some public address.
No, you said specifcially that it can be restored by end system*S* plural. Yes, I can get one box listening. Now tell me how to get the second and third boxes listening on the same port. If you can't do that, then in fact, it is *not* possible to restore *full* end-to-end connectivity.
Valdis.Kletnieks@vt.edu wrote:
No, you said specifcially that it can be restored by end system*S* plural.
Yes, end to end connectivity is restored. However, that end to end connectivity is restored does not mean your boxes can use 131.112.32.132 nor port 49734.
Yes, I can get one box listening. Now tell me how to get the second and third boxes listening on the same port.
Perhaps, you misunderstand how end systems behind NAT must interact with UPnP or something like that to be able to restore the end to end connectivity. End systems behind UPnP boxes are allocated disjoint sets of global port numbers, only among which, end systems can use as their global port numbers. End systems can obtain information on port numbers they can use through UPnP or something like that. Thus, there is no port number collision at the global side of the UPnP box. Similar mechanism is described in draft-ohta-e2e-nat-00.txt Masataka Ohta
I believe he understands just fine. However, his point (and I agree with him) is that if you are behind NAT, it isn't full end-to-end functionality, even if it does allow some degraded form of end-to-end connectivity with significant limitations which are not present in the absence of NAT. "I can't use your address" is inherent in the network. "I can't use whatever port number I want on my side of the connection" is not. Owen On Feb 16, 2012, at 10:24 PM, Masataka Ohta wrote:
Valdis.Kletnieks@vt.edu wrote:
No, you said specifcially that it can be restored by end system*S* plural.
Yes, end to end connectivity is restored.
However, that end to end connectivity is restored does not mean your boxes can use 131.112.32.132 nor port 49734.
Yes, I can get one box listening. Now tell me how to get the second and third boxes listening on the same port.
Perhaps, you misunderstand how end systems behind NAT must interact with UPnP or something like that to be able to restore the end to end connectivity.
End systems behind UPnP boxes are allocated disjoint sets of global port numbers, only among which, end systems can use as their global port numbers.
End systems can obtain information on port numbers they can use through UPnP or something like that.
Thus, there is no port number collision at the global side of the UPnP box.
Similar mechanism is described in draft-ohta-e2e-nat-00.txt
Masataka Ohta
Owen DeLong wrote:
I believe he understands just fine. However, his point (and I agree with him) is that if you are behind NAT, it isn't full end-to-end functionality, even if it does allow some degraded form of end-to-end connectivity with significant limitations which are not present in the absence of NAT.
I'm not interested in your own definitions on end-to-end something. For correct terminologies, you should read Saltzer's original paper. As for "in the absence of NAT", RFC3102 may also be helpful to you. Abstract This document examines the general framework of Realm Specific IP (RSIP). RSIP is intended as a alternative to NAT in which the end- to-end integrity of packets is maintained. Masataka Ohta
On Feb 16, 2012, at 5:11 PM, Masataka Ohta wrote:
Andreas Echavez wrote:
*Why disabling ICMP doesn't increase security and only hurts the web* *(path MTU discovery, diagnostics)
That PMTUD works is a misconception.
It actually works where people have not made active efforts to break it.
*How NAT breaks end-to-end connectivity (fun one..., took me hours to explain to an old boss why doing NAT at the ISP level was horrendously wrong)
That's another misconception.
While NAT breaks the end to end connectivity, it can be restored by end systems by reversing translations by NAT, if proper information on the translations are obtained through some protocol such as UPnP.
Sigh... NAT is a horrible hack that served us all too well in address conservation. Beyond that, it is merely a source of pain. Owen
-----Original Message----- From: Owen DeLong Sent: Thursday, February 16, 2012 8:48 PM To: Masataka Ohta Cc: nanog@nanog.org Subject: Re: Common operational misconceptions
On Feb 16, 2012, at 5:11 PM, Masataka Ohta wrote:
Andreas Echavez wrote:
*Why disabling ICMP doesn't increase security and only hurts the web* *(path MTU discovery, diagnostics)
That PMTUD works is a misconception.
It actually works where people have not made active efforts to break it.
Modern (RFC 4821) PMTUD that is used by default by Solaris and Microsoft does not require ICMP and works well. For Linux you have to enable it: /proc/sys/net/ipv4/tcp_mtu_probing = 1 or 2 (I believe the default is still 0 which means it relies on ICMP for PMTUD by default and you must turn on RFC 4821 PMTUD). If you're relying on ICMP for PMTUD, still, then yeah, you probably run into problems from time to time but fewer stacks use that method of PMTUD these days.
George Bonser wrote:
Modern (RFC 4821) PMTUD that is used by default by Solaris and Microsoft does not require ICMP and works well. For Linux you have to enable it:
It depends on how you define "work well". As the RFC says: indication of some significantly disruptive event in the network, such as a router failure or a routing change to a path with a smaller MTU. it can not react against PMTU changes very well. It is seemingly working well means there is not much PMTU changes, which means we had better assumes some PMTU (1280B, for example) and use it without PMTUD. Masataka Ohta
It depends on how you define "work well".
As the RFC says:
indication of some significantly disruptive event in the network, such as a router failure or a routing change to a path with a smaller MTU.
it can not react against PMTU changes very well.
It is seemingly working well means there is not much PMTU changes, which means we had better assumes some PMTU (1280B, for example) and use it without PMTUD.
Masataka Ohta
It depends on the OS and the method being used. If you set the option to "2" on Linux, it will do MTU probing constantly and react to MTU changes. Also, the MTU for a given path only "lives" for 5 minutes anyway (by default) and is "rediscovered" with Linux. (value in /proc/sys/net/ipv4/route/mtu_expires) but other operating systems may behave in other ways. I agree that if one tries hard enough, they can ensure a broken path and there are always people who seem to devote a lot of energy to that end. There's nothing that can be done about them but there is much the OS can try to do to defeat them at their task.
George Bonser wrote:
It is seemingly working well means there is not much PMTU changes, which means we had better assumes some PMTU (1280B, for example) and use it without PMTUD.
It depends on the OS and the method being used. If you set the option to "2" on Linux, it will do MTU probing constantly and react to MTU changes.
It actually does nothing. Given the following statements in the RFC: An initial eff_pmtu of 1400 bytes might be a good compromise because it would be safe for nearly all tunnels over all common networking gear, and yet close to the optimal MTU for the majority of paths in the Internet today. and Each Packetization Layer MUST determine when probing has converged, that is, when the probe size range is small enough that further probing is no longer worth its cost. When probing has converged, a the hosts are keep assuming PMTU of 1400B and if local MTU is 1500B or less, no discovery is performed because "the probe size range is small enough".
Also, the MTU for a given path only "lives" for 5 minutes anyway (by default) and is "rediscovered" with Linux. (value in /proc/sys/net/ipv4/route/mtu_expires) but other operating systems may behave in other ways.
See above. Rediscovery with initial eff_pmtu of 1400B and search_high of 1500B immediately terminates without any probe packets sent. Masataka Ohta
On Mon, 20 Feb 2012 15:42:56 +0900, Masataka Ohta said:
George Bonser wrote:
It is seemingly working well means there is not much PMTU changes, which means we had better assumes some PMTU (1280B, for example) and use it without PMTUD.
It depends on the OS and the method being used. If you set the option to "2" on Linux, it will do MTU probing constantly and react to MTU changes.
It actually does nothing.
Did you find this by just reading RFCs, or by actual code inspection?
the hosts are keep assuming PMTU of 1400B and if local MTU is 1500B or less, no discovery is performed because "the probe size range is small enough".
And if the local MTU is 9000?
George Bonser wrote:
It is seemingly working well means there is not much PMTU changes, which means we had better assumes some PMTU (1280B, for example) and use it without PMTUD.
It depends on the OS and the method being used. If you set the option to "2" on Linux, it will do MTU probing constantly and react to MTU changes.
It actually does nothing.
Must be magic then, because it works for me. I've got a few dozen servers with MTU 7500 that aren't having a bit of trouble talking to anyone.
George Bonser wrote:
Must be magic then, because it works for me.
Yes, but magicians always use tricks.
I've got a few dozen servers with MTU 7500 that aren't having a bit of trouble talking to anyone.
Your trick is that your routers at the border between MTUs 7500 and 1500 (or maybe, 1400 or so) generate ICMP packet too big packets to your servers and no intermediate entities filter them, isn't it? Masataka Ohta
Your trick is that your routers at the border between MTUs 7500 and 1500 (or maybe, 1400 or so) generate ICMP packet too big packets to your servers and no intermediate entities filter them, isn't it?
Masataka Ohta
I am saying that MTU probing works just fine, even with a link in between that has a shorter MTU and doesn't pass ICMP. I actually have one of those. It actively probes with packets of varying sizes and learns what the path MTU is. It does not rely on ICMP. Again, it actively probes with varying sizes of packets until it believes it has "converged". There are two modes with linux. 1: where it only probes if there is a problem and 2: where it probes all the time. The initial value for search_high SHOULD be the largest possible packet that might be supported by the flow. This may be limited by the local interface MTU, by an explicit protocol mechanism such as the TCP MSS option, or by an intrinsic limit such as the size of a protocol length field. In addition, the initial value for search_high MAY be limited by a configuration option to prevent probing above some maximum size. Search_high is likely to be the same as the initial Path MTU as computed by the classical Path MTU Discovery algorithm. It is RECOMMENDED that search_low be initially set to an MTU size that is likely to work over a very wide range of environments. Given today's technologies, a value of 1024 bytes is probably safe enough. The initial value for search_low SHOULD be configurable. ... The probe may have a size anywhere in the "probe size range" described above. However, a number of factors affect the selection of an appropriate size. A simple strategy might be to do a binary search halving the probe size range with each probe. However, for some protocols, such as TCP, failed probes are more expensive than successful ones, since data in a failed probe will need to be retransmitted. For such protocols, a strategy that raises the probe size in smaller increments might have lower overhead. For many protocols, both at and above the Packetization Layer, the benefit of increasing MTU sizes may follow a step function such that it is not advantageous to probe within certain regions at all. As an optimization, it may be appropriate to probe at certain common or expected MTU sizes, for example, 1500 bytes for standard Ethernet, or 1500 bytes minus header sizes for tunnel protocols. Some protocols may use other mechanisms to choose the probe sizes. For example, protocols that have certain natural data block sizes might simply assemble messages from a number of blocks until the total size is smaller than search_high, and if possible larger than search_low. Each Packetization Layer MUST determine when probing has converged, that is, when the probe size range is small enough that further probing is no longer worth its cost. When probing has converged, a timer SHOULD be set. When the timer expires, search_high should be reset to its initial value (described above) so that probing can resume. Thus, if the path changes, increasing the Path MTU, then the flow will eventually take advantage of it. The value for this timer MUST NOT be less than 5 minutes and is recommended to be 10 minutes, per RFC 1981. The timer for Linux is 5 minute by default but you can change it.
George Bonser wrote:
I am saying that MTU probing works just fine, even with a link in between that has a shorter MTU and doesn't pass ICMP.
And I have been saying your statement is unfounded.
I actually have one of those.
I can't see any.
It actively probes with packets of varying sizes and learns what the path MTU is.
First, it sets eff_pmtu to 1400B. OK?
It does not rely on ICMP. Again, it actively probes with varying sizes of packets until it believes it has "converged".
The initial value for search_high SHOULD be the largest possible packet that might be supported by the flow. This may be limited by the local interface MTU, by an explicit protocol mechanism such as the TCP MSS option,
OK, so, even with local MTU of 7500B and without ICMP PTB, if local MTU of your peer is 1500B, TCP MSS makes search_high 1500B. As eff_pmtu of 1400B is close enough to search_high, you are done. Eff_pmtu of 1400B will be used forever with no probe packets sent.
The timer for Linux is 5 minute by default but you can change it.
Timer timeouts do not affect TCP MSS. Masataka Ohta PS Your lengthy quotation means you don't see the point.
The timer for Linux is 5 minute by default but you can change it.
Timer timeouts do not affect TCP MSS.
RFC 2923: TCP should notice that the connection is timing out. After several timeouts, TCP should attempt to send smaller packets, perhaps turning off the DF flag for each packet. If this succeeds, it should continue to turn off PMTUD for the connection for some reasonable period of time, after which it should probe again to try to determine if the path has changed. It's Informational, not standards track, but the problem -- and the fix -- have been known for a very long time. --Steve Bellovin, https://www.cs.columbia.edu/~smb
Steven Bellovin wrote:
Timer timeouts do not affect TCP MSS.
RFC 2923: TCP should notice that the connection is timing out. After several timeouts, TCP should attempt to send smaller packets, perhaps turning off the DF flag for each packet. If this succeeds, it should continue to turn off PMTUD for the connection for some reasonable period of time, after which it should probe again to try to determine if the path has changed.
So?
It's Informational, not standards track, but the problem -- and the fix -- have been known for a very long time.
I'm not sure what, do you think, is the problem, because the paragraph of RFC2923 you quote has nothing to do with TCP MSS. The relevant section of the RFC (relevant to MSS) should be: The MSS should be determined based on the MTUs of the interfaces on the system, as outlined in [RFC1122] and [RFC1191]. which means MSS is constant. Note also that the next paragraph (next to the paragraph you quote) of the RFC eventually says to use PMTU of 1280B for IPv6 if there are black holes. It is not a very good thing to do especially for IP over IP tunnels, because 1280B packets are always fragmented if they are carried over a tunnel with MTU of 1280B. As implosion cause by multicast PMTUD of IPv6 requires ICMP PTB black holed, you can expect a lot of black holes. Masataka Ohta
On Feb 20, 2012, at 10:27 PM, Masataka Ohta wrote:
Steven Bellovin wrote:
Timer timeouts do not affect TCP MSS.
RFC 2923: TCP should notice that the connection is timing out. After several timeouts, TCP should attempt to send smaller packets, perhaps turning off the DF flag for each packet. If this succeeds, it should continue to turn off PMTUD for the connection for some reasonable period of time, after which it should probe again to try to determine if the path has changed.
So?
It's Informational, not standards track, but the problem -- and the fix -- have been known for a very long time.
I'm not sure what, do you think, is the problem, because the paragraph of RFC2923 you quote has nothing to do with TCP MSS.
Sure it does. That's in 2.1; the start of it discusses PMTUD failing for various reasons including firewalls.
The relevant section of the RFC (relevant to MSS) should be:
The MSS should be determined based on the MTUs of the interfaces on the system, as outlined in [RFC1122] and [RFC1191].
which means MSS is constant.
The text I quoted says, in so many words, "send smaller packets". I don't know how it's possible to be more explicit than that.
Note also that the next paragraph (next to the paragraph you quote) of the RFC eventually says to use PMTU of 1280B for IPv6 if there are black holes. It is not a very good thing to do especially for IP over IP tunnels, because 1280B packets are always fragmented if they are carried over a tunnel with MTU of 1280B.
Please cite in context. The text I quoted says that one option is to try turning off DF; the next paragraph notes that you can't do that on v6. It also doesn't say to to use PMTU of 1280, it says that that's a good fallback, and notes that v6 support requires that. Although it doesn't say so, I'll note that IP in IP makes the outer IP effectively a link layer for the inner IP; as such, it has to preserve all of the relevant properties including a link MTU of 1280. If that doesn't work -- though it most likely will, since the most common hardware MTU is from the ancient 1500 byte Ethernet size -- the outer IP endpoint has to deal with it appropriately, such as by intentional fragmentation. just as is done for IP over ATM with its 53-byte cell size (RFC 2225).
As implosion cause by multicast PMTUD of IPv6 requires ICMP PTB black holed, you can expect a lot of black holes.
Masataka Ohta
--Steve Bellovin, https://www.cs.columbia.edu/~smb
Steven Bellovin wrote:
I'm not sure what, do you think, is the problem, because the paragraph of RFC2923 you quote has nothing to do with TCP MSS.
Sure it does. That's in 2.1; the start of it discusses PMTUD failing for various reasons including firewalls.
Firewalls? Though I have never assumed existence of firewalls, if you are saying IPv6 will be even less operational because of firewalls, I have no counter argument.
The text I quoted says, in so many words, "send smaller packets". I don't know how it's possible to be more explicit than that.
No disagreement. I have been keep saying that IPv6 can't depend on PMTUD and must send packets of 1280B or less. It's George Bonser, not me, who said there were other ways.
Please cite in context. The text I quoted says that one option is to try turning off DF; the next paragraph notes that you can't do that on v6.
I thought the context is whether IPv6 is operational or not. Or, is it whether PMTUDv4 operational or not? Or, is your problem completely different from the above two? Your clarification is helpful Masataka Ohta
-----Original Message----- From: Masataka Ohta
First, it sets eff_pmtu to 1400B. OK?
Where did you get 1400 from? Are you talking specifically with the linux implementation? "As eff_pmtu of 1400B is close enough to search_high, you are done." I suppose that depends on a specific implementation of "close enough" is. I haven't looked at the specific code linux uses to implement this and "close enough" is fairly subjective and can be interpreted in different ways by different people. But even 1400 on, say, a 1420 MSS ICMP black hole is one heck of a lot better than running no PMTUD at all and running at something under 600 bytes.
PS
Your lengthy quotation means you don't see the point.
I am wondering where you got this magic 1400 value from. It should basically zero in on a number pretty close to the real path MSS in a few iterations. Maybe that one specific implementation stops at the first successful value, but that isn't the way the recommendation is written. Did I say it was "perfect"? No, but the notion that PMTUD is "broken" or "doesn't work" isn't exactly true. With the old mechanism, such a connection would simply hang or force people to turn off PMTUD completely. The new mechanism actually seems to perform rather well in the face of an ICMP black hole.
I, in fact, HAVE read the RFC. The initial value for search_high SHOULD be the largest possible packet that might be supported by the flow. This may be limited by the local interface MTU, by an explicit protocol mechanism such as the TCP MSS option, or by an intrinsic limit such as the size of a protocol length field. So the initial probe would be the reported MSS of the remote system in this case 1460 which would fail. I believe you might be getting confused by this paragraph: The general strategy is for the Packetization Layer to find an appropriate Path MTU by probing the path with progressively larger packets. If a probe packet is successfully delivered, then the effective Path MTU is raised to the probe size. And believing that as soon as a value is found that passes, the process stops. That isn't the case. PLPMTUD uses a searching technique to find the Path MTU. Each conclusive probe narrows the MTU search range, either by raising the lower limit on a successful probe or lowering the upper limit on a failed probe, converging toward the true Path MTU. For most transport layers, the search should be stopped once the range is narrow enough that the benefit of a larger effective Path MTU is smaller than the search overhead of finding it. The issue here is that the judgement of "once the range is narrow enough that the benefit of a larger effective Path MTU is smaller than the search overhead of finding it" and one might well argue that someone decided that the difference between 1400 and 1420 MSS (20 bytes) isn't worth the extra overhead of finding it those 20 bytes. But in the case of a 7500 MTU and a 4500 MTU black hole link in between, it certainly does NOT go to 1400 and stay there.
-----Original Message----- From: Masataka Ohta [mailto:mohta@necom830.hpcl.titech.ac.jp] Sent: Monday, February 20, 2012 6:43 PM To: George Bonser Cc: nanog@nanog.org Subject: Re: Common operational misconceptions
George Bonser wrote:
First, it sets eff_pmtu to 1400B. OK?
Where did you get 1400 from?
Read the RFC. PERIOD.
Masataka Ohta
George Bonser wrote:
I, in fact, HAVE read the RFC.
You don't, at all.
The initial value for search_high SHOULD be the largest possible packet that might be supported by the flow. This may be limited by the local interface MTU, by an explicit protocol mechanism such as the TCP MSS option, or by an intrinsic limit such as the size of a protocol length field.
It is a section on "search_high", while your question in your previous mail was on "eff_pmtu":
First, it sets eff_pmtu to 1400B. OK? Where did you get 1400 from?
Note that rest of your mail also contains a lot of misunderstandings on the RFC. But, as you don't read the RFC, collecting them is a waste of time. Read the RFC thoroughly again and again, 10 times a day for 30 days. Then, I may reply your further questions if asked off list. PERIOD. Masataka Ohta
From: Owen DeLong owen@delong.com
Sigh... NAT is a horrible hack that served us all too well in address conservation. Beyond that, it is merely a source of pain.
I understand why you say that - NAT did yeoman's work in address conservation. However, it also enabled (yes, really) lots of topologies and approaches which are *not* designed upon the end-to-end model. Some of these approaches have found their way into business proceses. An argument you and others have made many times boils down to "but if we never had NAT, think how much better it would be!" To this, the response "so what?" is not unreasonable - organizations which have built up processes and products around the non-end-to-end model may or may not see a benefit in changing their ways. Asserting that there is something wrong with existing, succesful business practices is not, by itself, compelling. While you and I may find this type of packet header manipulation distasteful, there's lots of organizations for which it's normal operations. The more NAT for v6 gets fought, the more folks will fight to preserve it. Time could be better spent demonstrating why NAT isn't needed in X or Y use case, and providing configuration snippets / assistance for non-NAT-based solutions to those various groups. David Barak Need Geek Rock? Try The Franchise: http://www.listentothefranchise.com
On Feb 17, 2012, at 7:20 AM, David Barak wrote:
From: Owen DeLong owen@delong.com
Sigh... NAT is a horrible hack that served us all too well in address conservation. Beyond that, it is merely a source of pain.
I understand why you say that - NAT did yeoman's work in address conservation. However, it also enabled (yes, really) lots of topologies and approaches which are *not* designed upon the end-to-end model. Some of these approaches have found their way into business proceses.
Yes... An unfortunate and very damaging side effect to be sure.
An argument you and others have made many times boils down to "but if we never had NAT, think how much better it would be!"
To this, the response "so what?" is not unreasonable - organizations which have built up processes and products around the non-end-to-end model may or may not see a benefit in changing their ways. Asserting that there is something wrong with existing, succesful business practices is not, by itself, compelling.
People who make money selling weapons don't necessarily see a benefit to peace. People who avoid the high costs of toxic waste disposal by putting it into ground water don't necessarily see a benefit to having an EPA or laws that prohibit doing so. If you're not party to the war and you're not one of the people whose water supply is affected by the toxins flowing into it, then, the response "so what?" may seem reasonable from your perspective. NAT is much the same way. It is a form of toxic pollutant that has damaging effects outside of the business that has chosen to deploy NAT. At best, it started out as a necessary evil for address conservation. Dependence on it beyond that seems to me to be akin to a form of drug addiction.
While you and I may find this type of packet header manipulation distasteful, there's lots of organizations for which it's normal operations. The more NAT for v6 gets fought, the more folks will fight to preserve it. Time could be better spent demonstrating why NAT isn't needed in X or Y use case, and providing configuration snippets / assistance for non-NAT-based solutions to those various groups.
And I do exactly that when presented with actual use cases where people believe NAT is needed. You can find several instances of my having done that in previous NANOG threads. Owen
David Barak wrote:
From: Owen DeLong owen@delong.com
Sigh... NAT is a horrible hack that served us all too well in address conservation. Beyond that, it is merely a source of pain.
I understand why you say that - NAT did yeoman's work in address conservation. However, it also enabled (yes, really) lots of topologies and approaches which are *not* designed upon the end-to-end model. Some of these approaches have found their way into business proceses.
I'm afraid both of you don't try to understand why NAT was harmful to destroy the end to end transparency nor the end to end argument presented in the original paper by Saltezer et. al: The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible. While plain NAT, which actively hide itself from end systems, which means there can be no "knowledge and help of the application" expected, is very harmful to the end to end transparency, it is possible to entirely neutralize the harmful effects, by let NAT boxes ask help end systems.
An argument you and others have made many times boils down to "but if we never had NAT, think how much better it would be!"
The reality is much better that NAT is not so harmful if NAT clients and gateways are designed properly to be able to reverse the harmful translations by NAT gateways. I have running code to make the reverse translations, with which protocols such as ftp with PORT commands are working. Masataka Ohta
On Feb 18, 2012, at 3:31 AM, Masataka Ohta wrote:
David Barak wrote:
From: Owen DeLong owen@delong.com
Sigh... NAT is a horrible hack that served us all too well in address conservation. Beyond that, it is merely a source of pain.
I understand why you say that - NAT did yeoman's work in address conservation. However, it also enabled (yes, really) lots of topologies and approaches which are *not* designed upon the end-to-end model. Some of these approaches have found their way into business proceses.
I'm afraid both of you don't try to understand why NAT was harmful to destroy the end to end transparency nor the end to end argument presented in the original paper by Saltezer et. al:
The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible.
While plain NAT, which actively hide itself from end systems, which means there can be no "knowledge and help of the application" expected, is very harmful to the end to end transparency, it is possible to entirely neutralize the harmful effects, by let NAT boxes ask help end systems.
An argument you and others have made many times boils down to "but if we never had NAT, think how much better it would be!"
The reality is much better that NAT is not so harmful if NAT clients and gateways are designed properly to be able to reverse the harmful translations by NAT gateways.
I have running code to make the reverse translations, with which protocols such as ftp with PORT commands are working.
Masataka Ohta
No, I think you do not understand... I have a NAT gateway with a single public address. I have 15 FTP servers and 22 web servers behind it. I want people to be able to go to ftp://<hostname> and/or http://<hostname> for each of them. Please explain to me how your code solves this problem? Yeah, thought so. Owen
I have running code to make the reverse translations, with which protocols such as ftp with PORT commands are working.
No, I think you do not understand...
I have a NAT gateway with a single public address.
I have 15 FTP servers and 22 web servers behind it.
I want people to be able to go to ftp://<hostname> and/or = http://<hostname> for each of them.
Owen, Your suggestion here would set many "security experts" heads on fire. Whatever will they do when NAT doesn't make such things virtually impossible? :-) ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net "We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
In message <201202200107.q1K17W5l000294@aurora.sol.net>, Joe Greco writes:
I have running code to make the reverse translations, with which protocols such as ftp with PORT commands are working.
No, I think you do not understand...
I have a NAT gateway with a single public address.
I have 15 FTP servers and 22 web servers behind it.
I want people to be able to go to ftp://<hostname> and/or = http://<hostname> for each of them.
Owen,
Your suggestion here would set many "security experts" heads on fire.
Whatever will they do when NAT doesn't make such things virtually impossible?
:-)
Time to write "How to use SRV with FTP". CGN is going to push the extension of a whole lot of protocols. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
On Feb 19, 2012, at 5:21 PM, Mark Andrews wrote:
In message <201202200107.q1K17W5l000294@aurora.sol.net>, Joe Greco writes:
I have running code to make the reverse translations, with which protocols such as ftp with PORT commands are working.
No, I think you do not understand...
I have a NAT gateway with a single public address.
I have 15 FTP servers and 22 web servers behind it.
I want people to be able to go to ftp://<hostname> and/or = http://<hostname> for each of them.
Owen,
Your suggestion here would set many "security experts" heads on fire.
Whatever will they do when NAT doesn't make such things virtually impossible?
:-)
Time to write "How to use SRV with FTP". CGN is going to push the extension of a whole lot of protocols.
That would be the worst case scenario, actually. Owen
On Sun, Feb 19, 2012 at 6:24 PM, Owen DeLong <owen@delong.com> wrote:
I have 15 FTP servers and 22 web servers behind it. I want people to be able to go to ftp://<hostname> and/or http://<hostname> for each of them.
For HTTP; You put a device on that one IP that will accept each TCP connection, await the SNI or Host header from the client, and then make/forward the connection to a proper server for that hostname. The public IP address belongs to the FTP/HTTP "service" instead of belonging to a server. For FTP, send to a desired FTP server based on the login username or otherwise make a SRV record for the _ftp service for each hostname, and set aside a TCP port for each FTP service's control connection. The ftp://user@<hostname> approach or the ftp://user@<basehostname>/<hostname>/ is probably more convenient than ftp://<hostname>:<1234>, since many clients do not support SRV records. The problem is with the FTP protocol not supporting virtual hosting, though; this missing FTP feature is not a NAT problem per se. The VHOST problem was solved with HTTP a long time ago. It's just that FTP is a lot less popular / fell into some disuse, so the deficiency (lack of virtual hosting support) was never corrected -- and the protocol hasn't had a single update in a long time. So you'll have to have a workaround to do 15 FTP servers with one global IP, because your circumstance is so unusual, that's just life. -- -JH
On Sun, 2012-02-19 at 19:09 -0600, Jimmy Hess wrote:
For HTTP; You put a device on that one IP that will accept each TCP connection, await the SNI or Host header from the client, and then make/forward the connection to a proper server for that hostname.
So you need an extra device to work around NAT. Or you have to build extra smarts into existing devices to work around NAT. There is a pattern here...
For FTP, send to a desired FTP server based on the login username or otherwise make a SRV record for the _ftp service for each hostname, and set aside a TCP port for each FTP service's control connection.
So NAT does indeed prevent the scenario Owen outlined. It does not make sense to make that the application's fault. If you have to build NAT-awareness (even indirectly, as in SRV-awareness) into every application, then you've lost the game and it might be time to realise that NAT is the problem, not all the applications.
The problem is with the FTP protocol not supporting virtual hosting, though; this missing FTP feature is not a NAT problem per se.
I'm not sure I agree with that, see above. And while virtual hosting may be a Good Thing for various other reasons, it seems to me that if it is required with NAT and is not required without NAT, then it is most certainly "the fault of NAT" that it is required. Regards, K. -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Karl Auer (kauer@biplane.com.au) http://www.biplane.com.au/kauer GPG fingerprint: AE1D 4868 6420 AD9A A698 5251 1699 7B78 4EEE 6017 Old fingerprint: DA41 51B1 1481 16E1 F7E2 B2E9 3007 14ED 5736 F687
Owen DeLong wrote:
I have running code to make the reverse translations, with which protocols such as ftp with PORT commands are working.
No, I think you do not understand...
How can't I understand several minor issues with the running code.
I have 15 FTP servers and 22 web servers behind it. I want people to be able to go to ftp://<hostname> and/or http://<hostname> for each of them. Please explain to me how your code solves this problem?
See draft-ohta-urlsrv-00.txt DNS SRV RRs of a domain implicitly specify servers and port numbers corresponding to the domain. By combining URLs and SRV RRs, no port numbers have to be specified explicitly in URLs, even if non-default port numbers are used, which makes URLs more concise for port based virtual and real hosting, where port based real hosting means that multiple servers sharing an IP address are distinguished by port numbers to give service for different URLs, which is the case for port forwarded servers behind NAT and servers with realm specific IP.
Yeah, thought so.
The draft has been available since September 29, 2011. Masataka Ohta
On Mon, 20 Feb 2012 11:17:32 +0900, Masataka Ohta <mohta@necom830.hpcl.titech.ac.jp> wrote:
draft-ohta-urlsrv-00.txt
DNS SRV RRs of a domain implicitly specify servers and port numbers corresponding to the domain.
By combining URLs and SRV RRs, no port numbers have to be specified explicitly in URLs, even if non-default port numbers are used, which makes URLs more concise for port based virtual and real hosting, where port based real hosting means that multiple servers sharing an IP address are distinguished by port numbers to give service for different URLs, which is the case for port forwarded servers behind NAT and servers with realm specific IP.
It seems to me that this will create all sorts of headaches for firewall ALGs. Rather than just passing port 21/tcp traffic to the FTP ALG for example, the devices would need to inspect traffic on all ports and perform DPI. This is not as much of a problem on the firewall protecting the servers (you know what ports to inspect), but will require a lot more processing power on the client-side NAT firewall. Jonesy
On Sun, Feb 19, 2012 at 10:09 PM, Andrew Jones <aj@jonesy.com.au> wrote:
On Mon, 20 Feb 2012 11:17:32 +0900, Masataka Ohta It seems to me that this will create all sorts of headaches for firewall ALGs. Rather than just passing port 21/tcp traffic to the FTP ALG for example, the devices would need to inspect traffic on all ports and perform [snip]
That doesn't work when the FTP control connection is encrypted using SSL. Layer 4 Firewall devices should not be expecting to intercept FTP traffic and make decisions based on the application layer contents of the traffic. I would suggest a requirement that FTP clients utilizing SRV records to access FTP on an alternate port MUST utilize Firewall-Friendly FTP as described by RFC1579. Each FTP server can then be assigned its own port range, or the FTP server can be configured to notify the Firewall device which ports to forward using UpNP or a NAT traversal protocol such as STUN, and the Firewall device can be configured to forward the appropriate range of ports to the correct server. -- -JH
On Sun, 19 Feb 2012 16:24:49 PST, Owen DeLong said:
No, I think you do not understand...
I have a NAT gateway with a single public address.
I have 15 FTP servers and 22 web servers behind it.
I want people to be able to go to ftp://<hostname> and/or = http://<hostname> for each of them.
Please explain to me how your code solves this problem?
I suspect that Masataka thinks the fact you can say http://hostname1:81, http://hostname2:82, and so on means it's Not A Problem. Except of course if you're trying to deploy this to actual users on actual networks.
End user devices will not benefit from end-to-end connectivity (e.g., globally routeable IPv4 addresses as opposed to being in a RFC1918 space behind NAT). If I have a wildcard DNS record, *.example.edu AAAA 2001:db8::5, then adding in an explicit record, x.example.edu AAAA 2001:db8::5, will make no visible difference. There is no legitimate reason for a user to use BitTorrent (someone will probably disagree with this). Our organization is not running out of IPv4 addresses so we don't need IPv6. (Similarly: Our orginization is running out of IPv4 addresses so that's why we need IPv6.) I can't use IPv6 because I still need to serve IPv4 clients. Any IP that starts with 192 is a private IP and any IP that starts with 169 is a self-assigned. Authentication by client IP address alone is sufficient. Long passwords requiring letters, numbers, and symbols with a no-repeat policy and a 90-day maximum password age are very secure. +1 for "We should drop all ICMP(v6) traffic." (Related: "I can't ping the box so it must be down.") +1 for "NAT is security". Regarding "DNS only uses UDP", I give out a technical test during interviews and one of the questions is basically "Use iptables to block incoming DNS traffic" and all applicants so far have only blocked UDP port 53.
----- Original Message -----
From: "Ridwan Sami" <rms2176@columbia.edu>
There is no legitimate reason for a user to use BitTorrent (someone will probably disagree with this).
Yeah, no. You've clearly never tried to download a Linux installer DVD. Cheers, -- jr 'among dozens of other legitimate uses' a -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
On Fri, Feb 17, 2012 at 10:33:12AM -0500, Jay Ashworth wrote:
----- Original Message -----
From: "Ridwan Sami" <rms2176@columbia.edu>
There is no legitimate reason for a user to use BitTorrent (someone will probably disagree with this).
Yeah, no.
You've clearly never tried to download a Linux installer DVD.
Nevermind that Bram Cohen is preparing to tackle TV with a BitTorrent-related protocol (no further details known yet).
wasn't tv already tackled by dvb-iptv + multicast (oh wait, multicast, that stuff that hardly ever globally works on ipv4 ;) (yes, i'm that old that i even know what a tv was ;) On Fri, 17 Feb 2012, Eugen Leitl wrote:
On Fri, Feb 17, 2012 at 10:33:12AM -0500, Jay Ashworth wrote:
----- Original Message -----
From: "Ridwan Sami" <rms2176@columbia.edu>
There is no legitimate reason for a user to use BitTorrent (someone will probably disagree with this).
Yeah, no.
You've clearly never tried to download a Linux installer DVD.
Nevermind that Bram Cohen is preparing to tackle TV with a BitTorrent-related protocol (no further details known yet).
When I bring up Linux ISOs to the believers of this misconception, they generally argue that Linux ISOs can be obtained without BitTorrent as well so blocking BT is okay. But I believe it is up to the user to decide which protocol to use to obtain the data and if the user wants to use BT but the network prevents this, the network is at fault. Other valid uses of BitTorrent include content intentionally distributed via BT for free by Hollywood studios, television broadcasters, and artists of Creative Commons works. There's also Blizzard patches and other game patches. Some companies like Twitter apparently use BitTorrent internally (https://github.com/lg/murder). Quoting Jay Ashworth <jra@baylink.com>:
----- Original Message -----
From: "Ridwan Sami" <rms2176@columbia.edu>
There is no legitimate reason for a user to use BitTorrent (someone will probably disagree with this).
Yeah, no.
You've clearly never tried to download a Linux installer DVD.
Cheers, -- jr 'among dozens of other legitimate uses' a
Alot of people are unclear on how hard it is for someone to sniff internet traffic if the aren't physically located at or near one of the endpoints IE: connected to the same access point or same switch. 2012/2/15 John Kristoff <jtk@cymru.com>
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
On Wed, 15 Feb 2012 12:47:15 -0800, John Kristoff <jtk@cymru.com> wrote:
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
The idea that "the more access-points, the better our WiFi". Oh, and the use of RJ45 for the unconfigured 8P8C, but this is not that annoying. Cheers.
Op 15-2-2012 21:47, John Kristoff schreef:
Hi friends,
As some of you may know, I occasionally teach networking to college students and I frequently encounter misconceptions about some aspect of networking that can take a fair amount of effort to correct.
For instance, a topic that has come up on this list before is how the inappropriate use of classful terminology is rampant among students, books and often other teachers. Furthermore, the terminology isn't even always used correctly in the original context of classful addressing.
I have a handful of common misconceptions that I'd put on a top 10 list, but I'd like to solicit from this community what it considers to be the most annoying and common operational misconceptions future operators often come at you with.
I'd prefer replies off-list and can summarize back to the list if there is interest.
John
Haven't seen this one yet: "TCP/IP is based on the osi model." Erik van Westen.
participants (89)
-
-Hammer- -
Aftab Siddiqui -
Alexandre Grojsgold -
Andreas Echavez -
Andrew Jones -
Anton Kapela -
Antti Ristimäki -
Brandt, Ralph -
Brett Lykins -
Carsten Bormann -
Charles Mills -
Chris Campbell -
Chuck Anderson -
Cutler James R -
Dale Carstensen -
Dan White -
Daniel Griggs -
David Barak -
David Walker -
Doug Barton -
Eugen Leitl -
Gary Buhrmaster -
George Bonser -
Grant Ridder -
Holmes,David A -
Jack Bates -
Jared Mauch -
Jay Ashworth -
Jeff Kell -
Jeff Wheeler -
Jens Link -
Jeroen Massar -
Jeroen van Aart -
Jerry Jones -
Jimmy Hess -
Joe Greco -
Joel jaeggli -
John Kristoff -
John Osmon -
Jon Lewis -
Josh Hoppes -
Justin M. Streiner -
Karl Auer -
Keegan Holley -
Kenneth M. Chipps Ph.D. -
Lamar Owen -
Lee -
Leigh Porter -
Leo Bicknell -
Marcel Plug -
Mario Eirea -
Mark Andrews -
Mark Grigsby -
Masataka Ohta -
Mathias Wolkert -
Michael Painter -
Michael Sinatra -
Michael W. Lucas -
Mike Andrews -
Mike Lyon -
Mukom Akong T. -
nanog -
Nathan Eisenberg -
Octavio Alvarez -
Owen DeLong -
Paul Graydon -
Paul Thornton -
Phil Dyer -
Phil Regnauld -
R. Sami -
Randy Bush -
Ray Soucy -
Rich Kulawiec -
Ridwan Sami
-
Robert Bonomi -
Scott Helms -
Sean Donelan -
Shumon Huque -
Steve Bertrand -
Steve Clark -
Steven Bellovin -
sthaug＠nethelp.no -
Sven Olaf Kamphuis -
Tim Franklin -
Todd Snyder -
Tom Perrine -
Tony Tauber -
Valdis.Kletnieks＠vt.edu -
Wayne E Bouchard