I solved this issue by making my own ISP. On 04-01-2017 12:10, Dovid Bender wrote:

Whenever I call my local ISP with an issue I just make tier1 and tier2 dizzy so they escalate.

On Wed, Jan 4, 2017 at 3:28 AM, Paul B. Henson <henson@acm.org> wrote:

...

On Tue, Jan 03, 2017 at 06:56:13PM -0800, Paul B. Henson wrote:

...

Hopefully it won't be three days this time. Well, my FIOS mysteriously came back online about 9:45pm, a bit over 18 hours after it mysteriously dropped offline. I happened to be in the wiring closet staring angrily at the ONT about 8:30ish and noticed that it reset itself 2 or 3 times over the course of about 10 minutes, so I get the feeling somebody was fiddling with it remotely. I suppose I'll never really know what was broken or how it ended up being fixed, other than having about 100% certainty it was on the far side of the fiber.

It's understandable that equipment breaks, and people misconfigure things sometimes. What I find insanely frustrating is the complete disconnect between level 1/2 support and the network engineers that actually know what's going on. When I called this morning with a complete outage of my business class FIOS, somebody probably knew it was down, or at least should have been able to tell it was down. But instead I get to waste hours of my time going through meaningless troubleshooting steps because lower level support doesn't have that information. And then after actually getting an escalation to someone who confirms an outage and gives me a ticket #, later follow up once again yields a complete lack of knowledge of what's clearly an outage, whether of just my connection or more widespread.

I'm about at the point where next time it goes down and it appears to be a remote issue I'm not going to bother to call it in; I'll just cross my fingers and hope it fixes itself within a day or so and only report it if it doesn't. I don't think my calls today did anything but waste my time.

Our model is 15k a mile all in, this is for aerial not underground for our HFC/Coax builds. A partner of ours models their underground fiber builds at 30k a mile. This is in south Louisiana so your market may vary as always. Luke Guillory Network Operations Manager Tel: 985.536.1212 Fax: 985.536.0300 Email: lguillory@reservetele.com Reserve Telecommunications 100 RTC Dr Reserve, LA 70084 _________________________________________________________________________________________________ Disclaimer: The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. . -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Jared Mauch Sent: Wednesday, January 04, 2017 7:37 AM To: Baldur Norddahl Cc: nanog@nanog.org Subject: Re: SoCal FIOS outage(?) / static IP readdressing

On Jan 4, 2017, at 7:54 AM, Baldur Norddahl <baldur.norddahl@gmail.com> wrote:

I solved this issue by making my own ISP.

I’ve been thinking of the same in my underserved area. Labor is $5/foot here and despite friends and colleagues telling me to move, it seems I have a sub-60 month ROI (and sub-year for some areas I’ve modeled with modest uptake rates of 15-20% where the other options are fixed wireless, Cellular data or dial). Hope is to do a presentation in the fall or next year with progress. We have areas around here where Comcast, (AT&T or Frontier) don’t even serve. The municipality is off getting bids to build due to market failure by the incumbents to invest. municipal fiber is nigh on illegal here in Michigan but with no incumbent it is feasible and my hope is will lock out people who are unwilling to invest despite their market cap. - Jared

I don't know about the rest of the list, but I find these numbers fascinating. There's probably not that many people who are allowed to share them, but if more could I think that would be educational for a lot of folks. In a message written on Wed, Jan 04, 2017 at 08:37:19AM -0500, Jared Mauch wrote:

I’ve been thinking of the same in my underserved area. Labor is $5/foot here and despite friends and colleagues telling me to move, it seems I have a sub-60 month ROI (and sub-year for some areas I’ve modeled with modest uptake rates of 15-20% where the other options are fixed wireless, Cellular data or dial).

In a message written on Wed, Jan 04, 2017 at 01:50:48PM +0000, Luke Guillory wrote:

Our model is 15k a mile all in, this is for aerial not underground for our HFC/Coax builds. A partner of ours models their underground fiber builds at 30k a mile.

In a message written on Wed, Jan 04, 2017 at 09:08:51AM -0500, Shawn L wrote:

Depending on the area and conditions (rock, etc). We're seeing

$4 /foot Aerial $5-$7 /foot direct bury $10 - $14 /foot directional bore

-- Leo Bicknell - bicknell@ufp.org PGP keys at http://www.ufp.org/~bicknell/

On Jan 10, 2017, at 10:21 AM, Fletcher Kittredge <fkittred@gwi.net> wrote:

Numbers for building fiber optic systems are out there if you do the research. Joining the FTTH Council is a good start. One thing to recognise is that the numbers vary widely based on what is being built and where it is being built. There are large regional, technology, and product variations. Verizon has economies of scale few can match.

Having said that, some of the numbers listed are unrecognizably low.

Labor can vary quite widely based on project, distance and environment. What is $5/foot in a rural location can be $150 or more in an urban environment. It’s not uncommon to see pricing around $12/foot before engineering and other permitting work. If you’re in an environment that has favorable permitting process and can work with an existing insured contractor, $5/foot is attainable. You are still up against 600-800 feet per day in favorable soil, which can easily turn to 200 should there be a rock or complex utility work involved. I’ll say depending on your project, you can start with the big-dreamer communities out there, or you go the other way and talk to folks that are doing it on the ground in your local area. I’ve talked to people about pole attach as well as underground. You can see costs as low as 10k per mile on poles, but that’s the really low end. All numbers I’ve mentioned are for Michigan in the communities around my home as well as outlying areas. If you’re around my area and want to talk costs and the projects, a private e-mail is welcome. If you’re in Maine, that granite rock is really tough, the hemlocks rot and fall more often than the birch, etc. Those risks make the situation tougher, and the population north of Bangor/Orono really thins out, but at least the speed limit is higher on 95 now. :-) - Jared

On Wed, Jan 4, 2017 at 8:37 AM, Jared Mauch <jared@puck.nether.net> wrote:

...

On Jan 4, 2017, at 7:54 AM, Baldur Norddahl <baldur.norddahl@gmail.com> wrote:

I solved this issue by making my own ISP.

I’ve been thinking of the same in my underserved area. Labor is $5/foot here and despite friends and colleagues telling me to move, it seems I have a sub-60 month ROI (and sub-year for some areas I’ve modeled with modest uptake rates of 15-20% where the other options are fixed wireless, Cellular data or dial).

Hope is to do a presentation in the fall or next year with progress. We have areas around here where Comcast, (AT&T or Frontier) don’t even serve. The municipality is off getting bids to build due to market failure by the incumbents to invest. municipal fiber is nigh on illegal here in Michigan but with no incumbent it is feasible and my hope is will lock out people who are unwilling to invest despite their market cap.

and think about it, you could get ipv6 on your network... the OP still doesn't have that native on his fios I bet.

On Wed, Jan 4, 2017 at 4:53 PM, Paul B. Henson <henson@acm.org> wrote:

...

From: Christopher Morrow Sent: Wednesday, January 04, 2017 8:42 AM

and think about it, you could get ipv6 on your network... the OP still doesn't have that native on his fios I bet.

Yeah, sure, pour salt on my still open wound ;).

maybe now would be a good time to ask your vz rep about this 'feature'?

On Wed, 04 Jan 2017 00:28:57 -0800, "Paul B. Henson" said:

I'm about at the point where next time it goes down and it appears to be a remote issue I'm not going to bother to call it in; I'll just cross my fingers and hope it fixes itself within a day or so and only report it if it doesn't. I don't think my calls today did anything but waste my time.

Even if nothing else happens, calling in and reporting the problem *does* (or at least it *should*) set the clock running for any SLA-related compensation.

From: Valdis.Kletnieks@vt.edu Sent: Wednesday, January 04, 2017 6:49 AM

Even if nothing else happens, calling in and reporting the problem *does* (or at least it *should*) set the clock running for any SLA-related compensation.

I'm pretty sure FIOS doesn't have any contractual SLA's. I suppose if you call and whine enough you might get a billing credit, but as another poster pointed out, it's generally not worth it.

On Wed, Jan 04, 2017 at 01:57:10PM -0800, Ray Van Dolson wrote:

Have been evaluating going to more consumerish-grade circuits like this at remote locations, but this scenario is one that has kept me sticking with the more traditional (and more expensive) SLA-bound circuits.

I'd call my business FIOS "prosumer" ;). Honestly, I'm not sure why you'd get business FIOS over residential FIOS if you don't need static IP addresses, at least if you're at an address where both are available. I pay $125/month for 50/50 with 5 statics, which serves my household and my IT consulting home office. I don't think my budget could cover "more expensive" SLA-bound circuits :). I looked into whether I could get same speed lower cost or higher speed same cost after the Frontier cutover, but I'm afraid all I have to look forward to is same speed higher cost when my current two year legacy Verizon contract expires :(. Maybe I should move to Texas - Google business fiber 250/250 for $100/month, with 13 statics for some additional cost I can't find documented. Plus IPv6. Be nice to have some last mile competition around here. LA is on their "some day" list, but who knows what that means geographically plus they've pretty much stopped expanding and switched to wireless 8-/.

On Jan 6, 2017, at 08:21 , Leo Bicknell <bicknell@ufp.org> wrote:

In a message written on Wed, Jan 04, 2017 at 04:51:26PM -0800, Paul B. Henson wrote:

...

I'd call my business FIOS "prosumer" ;). Honestly, I'm not sure why you'd get business FIOS over residential FIOS if you don't need static IP addresses, at least if you're at an address where both are available.

I can't speak to Verizon, but I can speak to Comcast.

At a past address I had Comcast Business (cable modem) service at a residential address, and then later downgraded it to Comcast Residential service.

The similarities: - Both used the exact same cable coming into the house. - Both offered the same speeds. - Both offered static IP's for an additional fee. — Not available in San Jose on residential service. If you want static, must go business here. - Both clearly used the same routers, backbone, peering, etc.

The differences I could see: - Cable Modem - Residential: could rent a consumer grade or BYO (I did, a good one) - Business: Comcast supplied and required their better-than-average, modem. It could be in bridge mode though. - San Jose, I was able to use BYO. Had to escalate several levels and pull several teeth to get bridge mode on the Comcast unit while I had it. - Support - Residential: 0-30 minutes on hold, the one dispatch when I needed a truck roll took ~24 hours. - Business: 0-2 minutes on hold, I had two dispatches one where the truck arrived within 30 minutes, the other in about 2 hours. - Cost (At the time) - Residential: $75/month. - Business Class: $90/month. - San Jose, Residential and business both about $90/month. Difference is that Residential includes Television in that price. - Data Caps: - Residential: 250GB/month. - San Jose, I just received a notice indicating that they were just now instituting a 500GB/month limit on my service. Prior to that, no documented cap. I don’t think I’ve tried to move more than 1/2 a terabyte in any month, so I don’t know if there was an undocumented cap or not. - Business: None (with two paragraphs of disclaimer)

One other visible difference: - Residential: One mac address only, 15 minutes to reset DHCP server if changing MAC address - Business: Multiple mac addresses supported

Differences I could not see/verify: - Cable Modem Channel Selection - I'm told in some cases business class cable modems get different DOCSYS channels which have less congestion than typical residential channels. This of course varies greatly market to market, and is also dependent on the number of both resi and business subs on the segment. - Packet prioritization. - I'm told that business class packets are given somewhat higher priority (QoS) in the network. I could find no way to verify this, and generally had no packet loss issues inside the Comcast network with either service.

Ultimately the reason to buy business class at a residential address (and I think the Prosumer description is correct) is generally faster repair times. On congested segments it may also result in slightly lesser packet loss. Maybe, depending on how caps are done, it could be worth while if you move a lot of data.

In my case, I started residential and it was abysmal. There were so many problems and multiple truck rolls did not resolve anything. Finally, I resorted to business class in desperation (My choices here for any bandwidth >1.5Mbps/384k are Comcast, Comcast, or Comcast). Within a few days of installation, 3-4 truck-rolls later, I had about 3 months of free service in credits and working service that was stable for years. Later I downgraded back to residential and it seems that having gotten the neighborhood equipment up to business class standards has resolved the issues and things continue to be reliable.

Obviously if these differences are worth the delta in price depends on your situation and the exact delta in your location. At the time I had this I was working from home, so the extra $15/month insurance that I could do my job was money well spent.

The delta may be more variable than you describe as well. If you’re only looking at internet, then it’s about $15 or maybe even $0 in some cases (It was actually cheaper at one point to buy a la carte business class internet than residential here). However, if you add TV, then the double-play residential price is almost always such that your internet service price a la carte is roughly equal to double-play price for both on residential. They don’t offer double-play business pricing in my area and, in fact, refused to sell me business class TV service in a residential unit. When I was running business class internet, I was paying about $60 for TV and $90 for business class internet, so it was $150 vs. $90. For me, it was worth even that much larger differential at the time because at least the service worked and at least I could get them to fix things when it didn’t. To me, that was the single largest differentiator for business vs. residential service. OTOH, this was in the years when Comcast was consistently winning the most hated company in America award, so I believe there have been some significant improvements in their residential service since then. (Though I still wouldn’t call myself a “happy” customer so much as one that is quite a bit less angry.) I’d still like to get a real internet provider here, or better yet, more than one that offered real bandwidth over fiber. Owen

It is interesting to see the differences. For instance to put my unit in bridge mode I just logged into it, said bridge mode, and rebooted it. That method was actually documented in their business class FAQ. I'm sure there are great differences in plant and capabilities. There's a lot of M&A history and a lot of historical reasons, good and bad. In a message written on Fri, Jan 06, 2017 at 11:55:56AM -0800, Owen DeLong wrote:

They don’t offer double-play business pricing in my area and, in fact, refused to sell me business class TV service in a residential unit. When

On some of the issues like this I wonder if the reason is regulatory. In the two areas I've checked there is double play business pricing. In one of them they offered business class TV at residential, or at least the rep tried to sell it to me. Its the sort of thing that just feels like a regulator issue to me. -- Leo Bicknell - bicknell@ufp.org PGP keys at http://www.ufp.org/~bicknell/

From: Matthew Black Sent: Wednesday, January 04, 2017 9:41 AM

I'm a Frontier FiOS customer in SoCal and have had trouble loading the Google home page for weeks. Had trouble loading Gmail last night.

When it's up, I rarely have connectivity issues. Of course, I have business class fios and only use their pipe; I run my own DNS and other services, I don't know specifically what is causing your issue.. In the eight months since the cutover, I had some intermittent connectivity issues during the week of the cutover itself, one three-day complete outage pretty much identical to this 18 hour outage, and maybe two or three times I can recall having issues getting to some piece of the Internet which may or may not have been a Frontier problem. I'd say overall I'm pretty happy with it when it's working. My main complaint is that when it stops working I drop into a complete vacuum of helplessness where I am unable to speak to anybody who can actually diagnose and rectify whatever the underlying failure is which is preventing it from working. That and it's ridiculous not to have native IPv6 for business class Internet at this point.