what you're calling a political failure could be what others call a rate war.
I only used the term "political failure" because it was the best match of the two options given. But you are right that it is necessary to let go of those terms and maybe, define your own, if you want to get to a deep understanding of what is going on.
but i am saying that if this long chain of guesses is accurate it likely also represents the ONLY way to drive efficiency in a competitive capital-intensive market.
In other words, the market demands more efficiency and gives their resources (money) to those willing to provide it. This is generally how any given industry makes step changes in efficiency, due to competitive pressures.
It is now part of every nation's and everbody's
critical infrastructure. It needs to be engineered and operated better so that it does not end up partitioning for dumb reasons.
that sounds like justification for government regulation, if true.
Not at all. It is justification for ISP management to stop peddling the same old, same old, and to restructure their businesses to be more like a utility. Granted, there is more margin to be made in vale-added services, but the core network operation needs to be run as an efficient utility, not have its inefficiency propped up by some lucrative voice products. Cogent seems to be operating according to the pure utility model. If the other ISPs don't want to get dragged into that space, they need to partition their businesses so that the high-priced services provide actual added value to customers over and above the generic Internet transit service. Regulation is just a way to prop up inefficient businesses as we should have all learned with the global telco disaster from the 1960's onward. As the sophistication of technology rose exponentially along with drastic drops in prices, telcos just ate up the extra margin by becoming more inefficient, until finally the regulators said enough is enough, and opened the doors to competition. --Michael Dillon