- NANOG - lists.nanog.org

NTP problems - (one) root cause
by Jay Ashworth 20 Nov '12

20 Nov '12

A subscriber to Outages who wishes to remain nameless passes along the pointer to this story: Apparently, either tick or tock got rebooted yesterday, and came up with its local clock set to 2000. And lots of people believed it. https://isc.sans.edu/diary.html?n&storyid=14548 Did you believe it? Why? (Yes, those are just rhetorical questions, intended to make people examine their NTP architecture more closely. :-) Cheers, -- jra -- Jay R. Ashworth Baylink jra(a)baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274

1 0

AS 2379
by Eric C. Miller 20 Nov '12

20 Nov '12

Does anybody know of a list of BGP communities for AS2379 (EMBARK-WNPK now CenturyLink)? I haven't reached out to Centurylink yet, but I'm used to just finding them through Google Searches. Eric Miller, CCNP Network Engineering Consultant (407) 257-5115

2 1

Re: [outages] NTP Issues Today
by Colin Johnston 20 Nov '12

20 Nov '12

no idea, re sigterm cause checked firewall system logs and could not see cause from that either times are GMT Colin On 20 Nov 2012, at 17:05, Jeremy Chadwick <jdc(a)koitsu.org> wrote: > Colin, > > Signal 15 = SIGTERM, so something intentionally shut ntpd down on your > side. The logs I'd be interested in would be prior to what you've > provided, i.e. what lead to the SIGTERM. > > Also, no timezone is mentioned anywhere in your timestamps, so please > provide that (UTC offset would be best). > > -- > | Jeremy Chadwick jdc(a)koitsu.org | > | UNIX Systems Administrator http://jdc.koitsu.org/ | > | Mountain View, CA, US | > | Making life hard for others since 1977. PGP 4BD6C0CB | > > On Tue, Nov 20, 2012 at 05:02:06PM +0000, Colin Johnston wrote: >> >> On 20 Nov 2012, at 15:38, Jeremy Chadwick <jdc(a)koitsu.org> wrote: >> >>> I'm still waiting for someone who was affected by this to provide >>> coherent logs from ntpd showing exactly when the time change happened. >>> Getting these, at least on an *IX system, is far from difficult folks. >>> >> >> from firewall ntp logs >> Nov 19 09:58:06 [192.168.0.1.128.176] 2012:11:19-09:58:06 ntpd[21385]: ntpd exiting on signal 15 >> Nov 19 09:58:19 [192.168.0.1.128.176] 2012:11:19-09:58:19 selfmonng[3503]: W check Failed increment ntpd_running counter 3 - 3 >> Nov 19 09:58:22 [192.168.0.1.128.176] 2012:11:19-09:58:22 selfmonng[3503]: W NOTIFYEVENT Name=ntpd_running Level=INFO Id=147 sent >> Nov 19 09:58:22 [192.168.0.1.128.176] 2012:11:19-09:58:22 selfmonng[3503]: W triggerAction: 'cmd' >> Nov 19 09:58:22 [192.168.0.1.128.176] 2012:11:19-09:58:22 selfmonng[3503]: W actionCmd(+): '/var/mdw/scripts/ntp restart' >> Nov 19 09:58:25 [192.168.0.1.128.176] 2012:11:19-09:58:25 ntpd[24120]: ntpd 4.2.4p8(a)1.1612-o Tue Feb 2 21:46:54 UTC 2010 (1) >> Nov 19 09:58:25 [192.168.0.1.128.176] 2012:11:19-09:58:25 selfmonng[3503]: W child returned status: exit='0' signal='0' >> Nov 19 09:58:35 [192.168.0.1.128.176] 2012:11:19-09:58:35 ntpd[24121]: kernel time sync status change 0001 >> >> was sync'd to 84.25.175.98, stratum 2 at the time I believe >> >> Colin

1 0

Re: [outages] NTP Issues Today
by Colin Johnston 20 Nov '12

20 Nov '12

On 20 Nov 2012, at 15:38, Jeremy Chadwick <jdc(a)koitsu.org> wrote: > I'm still waiting for someone who was affected by this to provide > coherent logs from ntpd showing exactly when the time change happened. > Getting these, at least on an *IX system, is far from difficult folks. > from firewall ntp logs Nov 19 09:58:06 [192.168.0.1.128.176] 2012:11:19-09:58:06 ntpd[21385]: ntpd exiting on signal 15 Nov 19 09:58:19 [192.168.0.1.128.176] 2012:11:19-09:58:19 selfmonng[3503]: W check Failed increment ntpd_running counter 3 - 3 Nov 19 09:58:22 [192.168.0.1.128.176] 2012:11:19-09:58:22 selfmonng[3503]: W NOTIFYEVENT Name=ntpd_running Level=INFO Id=147 sent Nov 19 09:58:22 [192.168.0.1.128.176] 2012:11:19-09:58:22 selfmonng[3503]: W triggerAction: 'cmd' Nov 19 09:58:22 [192.168.0.1.128.176] 2012:11:19-09:58:22 selfmonng[3503]: W actionCmd(+): '/var/mdw/scripts/ntp restart' Nov 19 09:58:25 [192.168.0.1.128.176] 2012:11:19-09:58:25 ntpd[24120]: ntpd 4.2.4p8(a)1.1612-o Tue Feb 2 21:46:54 UTC 2010 (1) Nov 19 09:58:25 [192.168.0.1.128.176] 2012:11:19-09:58:25 selfmonng[3503]: W child returned status: exit='0' signal='0' Nov 19 09:58:35 [192.168.0.1.128.176] 2012:11:19-09:58:35 ntpd[24121]: kernel time sync status change 0001 was sync'd to 84.25.175.98, stratum 2 at the time I believe Colin

1 0

Technical help required to check routing or filtering of Gobal IP routing range: 1.128.0.0 - 1.159.255.255 in the Level 3 Network Space.
by Attila Vekas 20 Nov '12

20 Nov '12

Hi All, I need technicial assistance with either a routing or ip restrication problem impacting our mobile ip customers in Australia. Customers using our Mobile Network are allocated an address in the 1.128.0.0 - 1.159.255.255 range to allow communication on the internet. I have a customer's who can't reach websites hosted oversea where the like cause appears to be some were in the Level 3 routing domain (I suspect). 1. Customer A trying to get www.readingeggs.com.au fails when using a Mobile Internet connection. C:\Documents and Settings\c832677>tracert www.readingeggs.com.au Tracing route to readingeggs.com.au [209.251.186.131] over a maximum of 30 hops: 1 458 ms 419 ms 539 ms 10.5.81.50 2 * * * Request timed out. 3 * * * Request timed out. 4 350 ms 659 ms 659 ms bundle-ether12.pie8.perth.telstra.net[120.151.2 55.17] 5 578 ms 629 ms 449 ms bundle-ether1.pie-core1.perth.telstra.net[203.5 0.6.221] 6 448 ms 509 ms 539 ms bundle-ether6.way-core4.adelaide.telstra.net[20 3.50.11.16] 7 457 ms 549 ms 529 ms bundle-ether9.exi-core1.melbourne.telstra.net[2 03.50.11.93] 8 468 ms 509 ms 539 ms bundle-ether12.chw-core2.sydney.telstra.net[203 .50.11.74] 9 458 ms 709 ms 559 ms bundle-ether1.oxf-gw2.sydney.telstra.net[203.50 .6.90] 10 448 ms 519 ms 789 ms 203.50.13.174 11 597 ms 759 ms 709 ms i-0-0-0-0.paix-core01.bx.telstraglobal.net[202. 84.140.145] 12 649 ms 659 ms 730 ms i-0-0-0-0.eqnx-core01.bi.telstraglobal.net[202. 84.140.142] 13 579 ms 719 ms 709 ms i-0-0-0-3.eqnx03.bi.telstraglobal.net[202.84.25 1.126] 14 599 ms 689 ms 689 ms l3-peer.eqnx03.pr.telstraglobal.net[134.159.61. 6] 15 618 ms 689 ms 689 ms ae-22-70.car2.SanJose1.Level3.net[4.69.152.68] 16 * * * Request timed out. 17 * * * Request timed out. 18 * * * Request timed out. 19 * * * Request timed out. 20 * * * Request timed out. 21 * * * Request timed out. 22 * * * Request timed out. 23 * * * Request timed out. 24 * * * Request timed out. 25 * * * Request timed out. 26 * * * Request timed out. 27 * * * Request timed out. 28 * * * Request timed out. 29 * * * Request timed out. 30 * * * Request timed out. Trace complete. A successful traceroute to www.readingeggs.com is shown below and was obtained from a fix-line connection to the internet: C:\Documents and Settings\c832677>tracert www.readingeggs.com.au Tracing route to readingeggs.com.au [209.251.186.131] over a maximum of 30 hops: 1 <1 ms <1 ms <1 ms 10.0.0.1 2 1 ms 1 ms 1 ms gigabitethernet2-16.lon69.melbourne.telstra.net[139.130.43.29] 3 4 ms 2 ms 3 ms bundle-ether3.exi-core1.melbourne.telstra.net [203.50.80.1] 4 23 ms 23 ms 23 ms bundle-ether12.chw-core2.sydney.telstra.net[203 .50.11.74] 5 22 ms 23 ms 23 ms bundle-ether1.oxf-gw2.sydney.telstra.net[203.50 .6.90] 6 16 ms 16 ms 16 ms tengigabitethernet14-0.sydo-core01.sydney.reach.com [203.50.13.42] 7 159 ms 158 ms 158 ms i-0-6-2-0.paix-core01.bx.telstraglobal.net[202. 84.140.90] 8 193 ms 192 ms 192 ms i-0-7-2-0.eqnx-core01.bi.telstraglobal.net[202. 84.140.30] 9 193 ms 193 ms 193 ms i-0-0-0-1.eqnx03.bi.telstraglobal.net[202.84.25 1.50] 10 208 ms 191 ms 208 ms l3-peer.eqnx03.pr.telstraglobal.net[134.159.61. 6] 11 209 ms 209 ms 209 ms ae-42-90.car2.SanJose1.Level3.net[4.69.152.196] 12 193 ms 193 ms 193 ms DATA-RETURN.car2.SanJose1.Level3.net[4.53.18.13 4] 13 234 ms 234 ms 234 ms t0-0-0-1.br1.stc.terremark.net[66.165.161.169] 14 235 ms 234 ms 234 ms po14.br1.mia.terremark.net [66.165.160.225] 15 359 ms 248 ms 250 ms t8-1.gw1.mia.terremark.net [66.165.161.82] 16 234 ms 234 ms 237 ms 66.165.170.14 17 229 ms 229 ms 229 ms 72.46.239.94 18 233 ms 233 ms 232 ms 209.251.186.131 In the above examples the source IP for the failed traceroute from the Australia's Mobile Network was a 1.126.42.153 address (Telstra) and for the working traceroute from the fix-line it was 139.130.43.29. It appears the path is being treated differently between routers: ae-22-70.car2.SanJose1.Level3.net [4.69.152.68] & ae-42-90.car2.SanJose1.Level3.net [4.69.152.196] from the Telstra Global Router. I am looking for network help and confirmation that the source network 1.128.0.0 - 1.159.255.255 is being routed correctly back to Telstra's network in Australia correctly on the SanJose - Level3 routers. Also check if any bogan filtering is in the network that could be blocking the source ranges from reaching the hosting URL. Regards, Attila Vekas Telstra

3 2

Long and unabbreviatable IPv6 addresses with random overloaded bits, vs. tunnelbroker
by Constantine A. Murenin 20 Nov '12

20 Nov '12

Dear NANOG@, I came across an interesting problem in trying to find an affordable KVM provider with IPv6 support. It seems like several rather major and reputable providers in the value sector do claim to have IPv6 support, but once you get your IPv6 addresses or subnets from them, you might be rather disappointed. == Example of edis.at == edis.at gives you an IPv4 address of, for example, 158.255.21x.xxx, and the IPv6 /112 that you get is 2a03:f80:ed15:158:255:21x:xxx:0/112 (really a /48), with 2a03:0f80:ed15::1 as the gateway. I've tried contacting them in an effort to receive any kind of a "proper" IPv6 address without the plaintext IPv4 embedment, but they've given me all sorts of crazy and (IMHO) far-sketched excuses; from not wanting to maintain a separate database of IPv6 addresses/subnets, and from lack of software provisioning support; to supposedly RIPE and/or edis' upstream providers requiring public whois entries for any /64's that edis.at would allocate for their customers, and to not having control of the underlying routers/switches/firewalls in any but one of their 13+ datacentres that thus makes creating 2^16 separate routes problematic. ??? All I'm asking for, are sane and short addresses and/or a saner /112, don't really care if it's still a shared /48 underneath. == Example of buyvm.net == On the other hand, buyvm.net claims to give you 16 IPv6 addresses. This is the kind of addresses that they give out (some consecutive 4 out of the 16 provided addresses listed): 2607:f358:0001:fed5:0022:a124:fe56:xxxx, 2607:f358:0001:fed5:0022:f6a6:dffe:xxxx, 2607:f358:0001:fed5:0022:c6a3:7826:xxxx, 2607:f358:0001:fed5:0022:654f:964d:xxxx. This is the response I've got from buyvm.net when trying to ask for some reasonable (shorter) addresses or a whole subnet instead: << We cannot offer that at this time as we have to store these IPs in MySQL which would slow down the database. >> WTF? Surely offering some 16 random addresses that are 128-bits long doesn't slow down MySQL, but offering one single, short and abbreviatable /64 or /112 subnet would. I'm puzzled! == == (HE's tunnelbroker.net, on the other hand, has no problem in giving out IPv6 addresses that, when abbreviated, can be represented by the same number of ASCII characters as an IPv4 address; for free, might I add.) == == Am _I_ supposed to have a MySQL database with the list of the IPv6 addresses that my virtual private servers have been assigned? Wouldn't it "slow down MySQL", so to speak? :-) What's your take on this? Since IPv6 is still a very low priority for many of the hosting providers (edis.at cited a rather negligible amount of IPv6 traffic, so, they argue that, as a small shop, they simply can't justify spending much R&D on further improving something that they consider already supported and not broken in the first place), would I be crazy to opt-out of such pathetic IPv6 support, and sign back up with a tunnelbroker from HE.net instead? (In fact, my preliminary tests show that IPv6 latency between Austria and Fremont, London and Moscow are either the same or slightly better when using the HE.net tunnelbroker in Prague, instead of edis.at native IPv6 support; bandwidth doesn't seem to be affected, either.) A little extra latency (only potentially) and MTU reduction by 20 bytes are the only negatives, right? Yet on the positive side, I'll get a proper and short /64 or /48 subnet, proper rDNS support, guaranteed proper intermediate hops with proper rDNS entries and no stupid hops impeding traceroute, potentially much better IPv6 routing/peering/latency, and short or custom ...:face:b00c::-style addresses to my liking; and, last but not least, a higher number of potential KVM providers to choose from, since I wouldn't require any native IPv6 support with such an arrangement. Comparing these options, seems like on the KVM hoster side, a blanket "IPv6 support" (with no /48 or /64 promises) is thus basically a useless concept anyways, and, at this time, should not even be sought? Any thoughts? Did I miss anything? In summary, I thought I have to have native IPv6 support in any new VPS that I buy, but looks like using a tunnelbroker might just be a much cleaner and better solution anyways. P.S. Does anyone other than me object to calling /48 subnets with allocations such as 2a03:f80:ed15:158:255:21x:xxx:0/112 as native IPv6 in this context of VPS hosting? How should such addresses/subnets/arrangements be called? Best regards, Constantine.

11 10

fiber termination tools
by Jon Lewis 20 Nov '12

20 Nov '12

We've always done our own fiber termination in-house using Corning Unicam (currently Unicam Pretium) tool kits and ends. Over the years I've dealt with fiber (primarily 62.5um mm FDDI), I've noticed that fiber seems to come in two "styles". Some lets me strip the buffer and coating easily in separate steps using the buffer/coating dual-notch mechanical stripper from our Corning tool kit. Some, it seems the buffer and coating are fused and the coating comes off with the buffer, but because both are being stripped at the same time, much more pulling force is needed, and the buffer/coating has to be removed in small steps or the fiber breaks :( First, I wonder if anyone knows why this is? Second, I wonder if a thermal stripper would help and is preferable to a strictly mechanical stripper? ---------------------------------------------------------------------- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

1 0

Re: NTP Issues Today
by Warren Bailey 20 Nov '12

20 Nov '12

Or you could just concede the fact that the navy is playing with time travel again. From my Galaxy Note II, please excuse any mistakes. -------- Original message -------- From: Scott Weeks <surfer(a)mauigateway.com> Date: 11/19/2012 3:52 PM (GMT-08:00) To: nanog(a)nanog.org Subject: Re: NTP Issues Today On 11/19/12 6:32 PM, "Scott Weeks" <surfer(a)mauigateway.com> wrote: >--- vanwolfe(a)gmail.com wrote: >From: Van Wolfe <vanwolfe(a)gmail.com> > >Did anyone else experience issues with NTP today? We had our server >times update to the year 2000 at around 3:30 MT, then revert back to 2012. >----------------------------------------- >You need to provide more information. For example, what NTP >source are you using? ------------------------------------------ --- chaynes(a)centracomm.net wrote: From: Clay Haynes <chaynes(a)centracomm.net> I can confirm this had happened on one of my test servers - it was pointing to tick.usno.navy.mil and tock.usno.navy.mil at the time. ------------------------------------------- That's not a very diverse set of NTP servers. In the future if you think it might be an outage, you might try on the 'outages' list: http://puck.nether.net/mailman/listinfo/outages For this one, you might ask the server contact if there was a problem: http://support.ntp.org/bin/view/Servers/TockUsnoNavyMil That assumes you've done your homework first and made sure it wasn't something in your network. scott

1 0

Muni Fiber redux: nuts and volts
by Jay Ashworth 20 Nov '12

20 Nov '12

Last May we talked at some length about municipally owned wholesale fiber, and whether it was a commercially feasible idea. For those who have a few minutes (I figure, it's a holiday; the Whacky Weekend starts early :-), I'd like some advice, input, pointers, or the like, on exactly how you *do* this these days. I'm looking at 2.8 square miles of city, about 10k connections or so. I'm inclined to say trunking should be underground; I'm not sure if the drops need to be as well, though this *is* Florida; One Big Storm probably eats up even the 6x differential I've been told trenching the drops cost. What sort of ONT would one use for this? I am assuming it's impractical to do layer 1 and just move patch cords around; I'll have to work at layer 2, and supply to my wholesale customers some termination interface that extends them to the GigE or DOCSIS coax coming out the other end of the box...? Are there installation contractors with packaged proposals for this sort of thing, at this point? Cheers, -- jra -- Jay R. Ashworth Baylink jra(a)baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274

1 0

NTP Issues Today
by Oscar Orosco 19 Nov '12

19 Nov '12

We had the same issue on our NTP server pointing to tick.usno.navy.mil. Set date back to year 2000. Date: Mon, 19 Nov 2012 16:21:55 -0700 From: Van Wolfe <vanwolfe(a)gmail.com<mailto:vanwolfe@gmail.com>> To: nanog(a)nanog.org<mailto:nanog@nanog.org> Subject: NTP Issues Today Message-ID: <CAMeggd4cDQwhxQE_JbvpNR-PKKe9LXqA+KzJ97anHFonjwZhdQ(a)mail.gmail.com<mailto:CAMeggd4cDQwhxQE_JbvpNR-PKKe9LXqA+KzJ97anHFonjwZhdQ@mail.gmail.com>> Content-Type: text/plain; charset=ISO-8859-1 Hello, Did anyone else experience issues with NTP today? We had our server times update to the year 2000 at around 3:30 MT, then revert back to 2012. Thanks, Van

1 0