December 2014 - NANOG - lists.nanog.org

Shapefiles, KMZs, etc.
by Mike Hammett 30 Dec '14

30 Dec '14

I am looking for shapefiles, KMZs, etc. for networks primarily in the Midwest, but really throughout the area that is the scope of this list. I am a small ISP that just happens to know more than your average ISP about where people are and how to use GIS tools. I use them to help other ISPs find transport and they may come in handy for some start-up IX work I'm involved with. They would not go public and I would be willing to sign NDAs to get them. I have gotten several form public sources, but I may not have gotten all of the public ones and I have some (but still only a few) private ones. Thank you. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com

3 4

Re: Charter ARP Leak
by Larry Sheldon 30 Dec '14

30 Dec '14

On 12/29/2014 22:32, Ricky Beam wrote: > On Mon, 29 Dec 2014 17:41:45 -0500, Corey Touchet > <corey.touchet(a)corp.totalserversolutions.com> wrote: > >> We'll I would for one be very interested if the 8 ARP packets a second >> count against the caps. > > Depends on where and what counters they probe. I would assume they look > at "unicast" fields, so it wouldn't counted. (of course, *I* would use > netflow accounting, but I care about my data being accurate.) I probably should know--what kind of packets get counted? All? TCP & UDP? ICMP? BDU's? -- The unique Characteristics of System Administrators: The fact that they are infallible; and, The fact that they learn from their mistakes. Quis custodiet ipsos custodes

1 0

Re: Charter ARP Leak
by Larry Sheldon 30 Dec '14

30 Dec '14

On 12/29/2014 11:35, Brett Frankenberger wrote: > On Mon, Dec 29, 2014 at 12:27:04PM -0500, Jay Ashworth wrote: >>> >>> Valdis, you are correct. What your seeing is caused by multiple IP >>> blocks being assigned to the same CMTS interface. >> >> Am I incorrect, though, in believing that ARP packets should only be visible >> within a broadcast domain, > > broadcast domain != subnet It surprises me that in this day and age, in a forum like this that has an active thread about kids being taught archaic concepts, we see language like "broadcast domain != subnet" and a perceived need to explain it. [no longer germane material deleted to reduce excess baggage charges] > int ethernet 0/0 > ip address 10.0.0.1 255.255.0.0 > ip address 11.0.0.1 255.255.0.0 secondary > ip address 12.0.0.1 255.255.0.0 secondary > > The broadcast domain will have ARP broadcasts for all three subnets. This are not "subnets"! They are IP addresses in three different IP networks. > Doing it over a CMTS doesn't change that. Communication here perceived as hostile is apologized-for. -- The unique Characteristics of System Administrators: The fact that they are infallible; and, The fact that they learn from their mistakes. Quis custodiet ipsos custodes

2 1

RE: The state of TACACS+
by emille＠abccomm.com 29 Dec '14

29 Dec '14

I've long since deleted the OP's message, but figured I would share our experiences having been using TACACS+ with our Cisco hardware for a couple of years. Originally deployed for the need and want of controlling multiple users across several devices, and to safely control 3rd party read, or reverse-telnet access to the very few nodes that may need it, without needing to mess around with parser views on every device. To that end it's worked just fine without complaint. Note: We're using shrubbery.net's tac_plus. The per-command authorization does slow some nodes down slightly, but nothing as severe as a few seconds each It does work out to about 1 command per (1000ms / Node to AAA RTT) as you'd expect. Eg; The worst I've seen on a ~200ms link, copy/paste lump-of-config will work out to about the expected 4-5 commands/second. Devices running v15 seem to speed this up somehow, not sure if they multiplex commands under the hood, or if I'm just crazy. I've never looked into it that closely for lack of interest and time. There is a stupid gotcha when dealing with the command authorization in the TACACS configuration. If you permit 'johndoe' a 'show ip bgp .*', and he is also a member of a group with subsequent show commands, the show commands in the 'group' config block are completely ignored. This makes some scenarios tricky. We utilize a local root, unprivileged user with unique credentials across each device. It's possible to configure Cisco's AAA to prevent the local user login while AAA is up / reachable. Generally, we are of the opinion that if our nodes cannot reach the AAA server, we have bigger problems that would necessitate a senior administrator with access to the local root user credentials anyway. Otherwise, a TACACS server can be setup in literally minutes and the configuration required is minuscule and easy to backup safely. A note on the local root user. By far and away, the worst possible scenario is not AAA going down / becoming entirely unreachable, but instead when experiencing network instability. Having experienced this scenario for a few very frustrating hours, the experience is along the lines of; - Enter a pile of commands. Some fail (wile AAA is briefly up), some succeed (while AAA is down). - Swear at your console, and repeat until the problem(s) are resolved. Our workaround was; Add your backup / root user with full privileges to your TACACS backend, with _no_ password. This denies them access when AAA is running as there is no password to authenticate against, but prevents "Authorization failed!" when the AAA server is briefly available in the middle of your diags / trying to resolve the connectivity problem. For the Unix admins; The TACACS binary itself, is awful - It has no status exit codes. The process cannot be monitored or controlled safely by way of something like DJB's daemontools, even with the fg_helper hack - at least I've not managed to succeed to date and have given up. To that end, we have a hacked together script to assist with safely reloading configs and such that parses stdout and stderr to decide what to do. Eg; trying to gracefully restart TACACS with a broken config will cause the daemon to exit - not awesome. All that said, I have heard a lot of praise from an enterprise in my neck of the woods who shelled out for Cisco's TACACS+ VM Appliance. If you have the money it's supposedly worth it especially for the AD hooks. I hope this provides some insight to those that may need it. ________________________________________ From: NANOG [nanog-bounces(a)nanog.org] On Behalf Of Colton Conor [colton.conor(a)gmail.com] Sent: Monday, December 29, 2014 9:28 AM To: Michael Douglas Cc: NANOG Subject: Re: The state of TACACS+ Glad to know you can make local access only work if TACAS+ isn't available. However, that still doesn't prevent the employee who know the local username and password to unplug the device from the network, and the use the local password to get in. Still better than our current setup of having one default username and password that everyone knows. On Mon, Dec 29, 2014 at 9:38 AM, Michael Douglas <Michael.Douglas(a)ieee.org> wrote: > In the Cisco world the AAA config is typically set up to try tacacs > first, > and local accounts second. The local account is only usable if > tacacs is > unavailable. Knowledge of the local username/password does not > equate to > full time access with that credential. Also, you would usually > filter the > incoming SSH sessions to only permit a particular management IP > range; the > local credential, or tacacs credential, shouldn't be usable from any > arbitrary network. > > On Mon, Dec 29, 2014 at 10:32 AM, Colton Conor > <colton.conor(a)gmail.com> > wrote: > > > Scott, > > > > Thanks for the response. How do you make sure the failsafe and/or > root > > password that is stored in the device incase remote auth fails > can't be > > accessed without having several employees engaged? Are there any > mechanisms > > for doing so? > > > > My fear would be we would hire an outsourced tech. After a certain > amount > > of time we would have to let this part timer go, and would disabled > his > or > > her username and password in TACAS. However, if that tech still > knows the > > root password they could still remotely login to our network and > cause > > havoc. The thought of having to change the root password on > hundreds of > > devices doesn't sound appealing either every time an employee is > let go. > To > > make matters worse we are using an outsourced firm for some network > > management, so the case of hiring and firing is fairly consistent. > > > > On Mon, Dec 29, 2014 at 9:22 AM, Scott Helms <khelms(a)zcorum.com> > wrote: > > > > > Colton, > > > > > > Yes, that's the 'normal' way of setting it up. Basically you > still > have > > > to configure a root user, but that user name and password is kept > locked > > up > > > and only accessed in case of catastrophic failure of the remote > > > authentication system. An important note is to make sure that > the fail > > > safe password can't be accessed without having several people > engaged > so > > it > > > can't be used without many people knowing. > > > > > > > > > Scott Helms > > > Vice President of Technology > > > ZCorum > > > (678) 507-5000 > > > -------------------------------- > > > http://twitter.com/kscotthelms > > > -------------------------------- > > > > > > On Mon, Dec 29, 2014 at 10:15 AM, Colton Conor > <colton.conor(a)gmail.com > > > > > wrote: > > > > > >> We are able to implement TACAS+. It is my understanding this a > fairly > > old > > >> protocol, so are you saying there are numerous bugs that still > need to > > be > > >> fixed? > > >> > > >> A question I have is TACAS+ is usually hosted on a server, and > > networking > > >> devices are configured to reach out to the server for > authentication. > My > > >> question is what happens if the device can't reach the server if > the > > >> devices network connection is offline? Our goal with TACAS+ is > to not > > have > > >> any default/saved passwords. Every employee will have their own > username > > >> and password. That way if an employee gets hired/fired, we can > enable > or > > >> disable their account. We are trying to avoid having any > organization > > wide > > >> or network wide default username or password. Is this possible? > Do the > > >> devices keep of log of the last successful username/password > > combinations > > >> that worked incase the device goes offline? > > >> > > >> On Sun, Dec 28, 2014 at 5:02 PM, Robert Drake > <rdrake(a)direcpath.com> > > >> wrote: > > >> > > >> > Picking back up where this left off last year, because I > apparently > > only > > >> > work on TACACS during the holidays :) > > >> > > > >> > > > >> > On 12/30/2013 7:28 PM, Jimmy Hess wrote: > > >> > > > >> >> Even 5 seconds extra for each command may hinder operators, > to the > > >> extent > > >> >> it would be intolerable; shell commands should run almost > > >> >> instantaneously.... this is not a GUI, with an hourglass. > > Real-time > > >> >> responsiveness in a shell is crucial --- which remote auth > should > not > > >> >> change. Sometimes operators paste a buffer with a fair > number of > > >> >> commands, not expecting a second delay between each command > --- a > > >> >> repeated delay, may also break a pasted sequence. > > >> >> > > >> >> It is very possible for two of three auth servers to be > unreachable, > > >> in > > >> >> case of a network break, but that isn't necessary. The > "response > > >> >> timeout" might be 5 seconds, but in reality, there are > cases > where > > >> you > > >> >> would wait longer, and that is tragic, since there are > some > > obvious > > >> >> alternative approaches that would have had results that > would be > > more > > >> >> 'friendly' to the interactive user. > > >> >> > > >> >> (Like remembering which server is working for a while, or > > remembering > > >> >> that all servers are down -- for a while, and having a 50ms > > timeout, > > >> >> with all servers queried in parallel, instead of a 5 > seconds > > >> timeout) > > >> >> > > >> > I think this needs to be part of the specification. > > >> > > > >> > I'm sure the reason they didn't do parallel queries was > because of > > both > > >> > network and CPU load back when the protocol was drafted. But > it > might > > >> be > > >> > good to have local caching of authentication so that can > happen even > > >> when > > >> > servers are down or slow. Authorization could be updated to > send > the > > >> > permissions to the router for local handling. Then if the > server > dies > > >> while > > >> > a session is open only accounting would be affected. > > >> > > > >> > That does increase the vendors/implementors work but it might > be > > doable > > >> in > > >> > phases and with partial support with the clients and servers > > negotiating > > >> > what is possible. The biggest drawback to making things like > this > > >> better > > >> > is you don't gain much except during outages and if you > increase > > >> complexity > > >> > too much you make it wide open for bugs. > > >> > > > >> > Maybe there is a simpler solution that keeps you happy about > > redundancy > > >> > but doesn't increase complexity that much (possibly anycast > tacacs, > > but > > >> the > > >> > session basis of the protocol has always made that not > feasible). > > It's > > >> > possible that one of the L4 protocols Saku Ytti mentioned, > QUIC or > > >> MinimaLT > > >> > would address these problems too. It's possible that if we > did the > > >> > transport with BEEP it would also provide this, but I'm > reading the > > docs > > >> > and I don't think it goes that far in terms of connection > assurance. > > >> > > > >> >> -- > > >> >> -JH > > >> >> > > >> >> > > >> > So, here is my TACACS RFC christmas list: > > >> > > > >> > 1. underlying crypto > > >> > 2. ssh host key authentication - having the router ask tacacs > for > an > > >> > authorized_keys list for rdrake. I'm willing to let this go > because > > >> many > > >> > vendors are finding ways to do key distribution, but I'd still > like > to > > >> have > > >> > a standard (https://code.google.com/p/openssh-lpk/ for how to > do > this > > >> > over LDAP in UNIX) > > >> > 3. authentication and authorization caching and/or something > else > > >> > > > >> > > > >> > > > > > > > > >

1 0

Twitter contact
by Chaim Rieger 29 Dec '14

29 Dec '14

Seems that your devices are set one year ahead. That is all

2 1

merry xmas
by Ken Chase 28 Dec '14

28 Dec '14

(mtr|lft|traceroute) xmas.futile.net /kc -- Ken Chase - Toronto Canada

12 19

Estonian IPv6 deployment report
by Tarko Tikan 28 Dec '14

28 Dec '14

hey, Some time ago, many people noticed rapid IPv6 deployment growth in Estonia (from 0% to 5% in 4 weeks). We at 3249/Elion/Estonian Telecom were behind this, other operators don't have any serious IPv6 deployments at the moment. We rolled out v6 to everyone (both business and residential customers) with last-gen CPE, there was no hop-in our hop-out program - aim was to do it perfectly and without customers even noticing. I'm happy to say that we achieved this goal :) To satisfy general interest, I promised small (somehow it turned out longer than I expected) technical writeup how we enabled v6 for our subscribers. If you have any other questions, feel free to ask and I do my best to answer them. You can also skip the technical content and there are some statistics below. Our access network is mix of DSL/GPON/wimax/p2p-ETH and broadband service is deployed in shared service vlans. IPv6 traffic shares vlan with IPv4. Service vlans are transported over MPLS metro network using pseudowires and terminated in geo-clustered Alcatel 7750 BNG routers. Each subscriber is allocated up to 4 mixed v4 and v6 IP hosts. For v4 we are using the usual DHCP, for IPv6 we are using DHCPv6 with IA_PD only, no IA_NA is provided. Unfortunately DHCPv6 provides no way to signal IPv6 default-route thus we have to fall back to RA for default-route. RA does not include any on-link prefixes or DNS information. RAs are L2 unicasted to CPE MAC so no other CPE in service vlan picks up those RAs. To ensure rapid switchover between BNG routers, we are signalling virtual link-local address as default-route. We are using ALU internal DHCP/DHCPv6 servers to allocate leases but we also signal IP information from radius (in such case BNG "fakes" DHCP server) for static IP customers. Provided IPv6 prefix is always /56 and we keep the old lease for 24h even if the CPE is turned off (actual lease time is 30min). Unfortunately, IPv6 LDRA is not available on most of our access platforms so we have to rely on IPv4 session information for authentication. This linking is done in the radius server during subscriber authentication (excellent radiator + quite awful SQL queries :) - if subscriber has IPv4 session (that has been authenticated using DHCP opt82), same MAC address is allowed to have IPv6 session on exactly the same virtual BNG port. IPv4 and v6 session are both tied to same subscriber and share shapers, QOS etc. We were able to enable IPv6 only on our last-gen Inteno CPEs. They run modified OpenWrt and because it's linux - everything is possible :) In CPE, /56 is divided up to /64s, first one is currently reserved but we will configure it on loopback interface and use it for CPE management. Second /64 is configured on LAN and third is configured on public wifi SSID (if you choose to enable this option). In the LAN, IPv6 config is provided by RAs, we also support RDNSS and stateless DHCPv6 for DNS. There is also ingress IPv6 firewall in the CPE and configuration is modifiable by user. To make deployment as smooth as possible, we rolled out IPv6 capable CPE software first. Then, during the BNG platform refresh, we deployed L2 ACLs that dropped all IPv6 traffic based on 0x86dd ethertype. We then deployed IPv6 config to all BNGs and could verify everything before single v6 lease was handed out to the subscribers. Then, interface by interface, we replaced L2 ACL with one that only allowed 0x86dd for certain, supported, OUIs. This is the current situation and we are investigating ways to support 3rd party CPEs - main problem is unreliable IPv6 config in CPEs. Many don't enable DHCPv6 (or enable NA but no PD) but still pick up default-route from RA and happily signal it to LAN. Some others hammer our BNGs with NA request every 0.1 seconds etc. As statistics go, there are 30000+ active IPv6 subscribers (almost 15% of our customer base, based on our public numbers), 81% of them have have at least one IPv6 enabled device in the LAN, 70% have more than one. Most IPv6 traffic is generated by Google+Youtube, Facebook and Akamai. Not bad for a country with 1.3M people. Next up: mobile network :) -- tarko

5 8

AS6713 (aka IAM / MOROCCO TELECOMS) peering contact
by Youssef Bengelloun-Zahr 28 Dec '14

28 Dec '14

Hello, If someone from IAM peering team is watching, could he please get in touch OFF-list please ? Best regards. -- Youssef BENGELLOUN-ZAHR

11 15

Anyone from Cloudflare ? (IPv6 issue)
by Brandon Applegate 28 Dec '14

28 Dec '14

Anyone from Cloudflare able/willing to contact me off list to troubleshoot a very frustrating and intermittent IPv6 connectivity issue ? I have plenty of data points, multiple test systems (Testing from 2 working ASes, and the 1 AS in question that’s broken). Otherwise - if anyone could share a way to get to clue @Cloudflare I would greatly appreciate it. I put a request in through the web support front door, but I got back about what I expected. Thanks. -- Brandon Applegate - CCIE 10273 PGP Key fingerprint: 830B 4802 1DD4 F4F9 63FE B966 C0A7 189E 9EC0 3A74 "SH1-0151. This is the serial number, of our orbital gun."

2 2

North Korean internet goes dark (yes, they had one)
by Valdis Kletnieks 27 Dec '14

27 Dec '14

Any of you guys want to fess up? :) http://www.msnbc.com/the-ed-show/watch/north-koreas-internet-goes-dark-3760… (Yes, I know, they're saying it's a DDoS, not a routing hack...)

12 14