----- Original Message -----
From: "Roland Dobbins" <rdobbins@arbor.net>
On Apr 1, 2013, at 11:18 PM, Patrick W. Gilmore wrote:
Of course, since users shouldn't be using off-net name servers anyway, this isn't really a problem! :)
;>
It's easy enough to construct ACLs to restrict the broadband consumer access networks from doing so. Additional egress filtering would catch any reflected attacks, per your previous comments.
So, how would Patrick's caveat affect me, whose recursive resolver *is on my Linux laptop*? Would not that recursor be making queries he advocates blocking? Or don't I remember DNS well enough? Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
On Mon, 01 Apr 2013 14:19:16 -0400, Jay Ashworth said:
So, how would Patrick's caveat affect me, whose recursive resolver *is on my Linux laptop*? Would not that recursor be making queries he advocates blocking?
You're sending queries, not replies. That's why DPI is needed to do the blocking, rather than just by port.
---- Original Message -----
From: "Valdis Kletnieks" <Valdis.Kletnieks@vt.edu>
On Mon, 01 Apr 2013 14:19:16 -0400, Jay Ashworth said:
So, how would Patrick's caveat affect me, whose recursive resolver *is on my Linux laptop*? Would not that recursor be making queries he advocates blocking?
You're sending queries, not replies. That's why DPI is needed to do the blocking, rather than just by port.
Aha. Right. Got it. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
On Mon, 1 Apr 2013, Valdis.Kletnieks@vt.edu wrote:
You're sending queries, not replies. That's why DPI is needed to do the blocking, rather than just by port.
What queries are sourced from port 53 nowadays? I'd imagine it's pretty safe to block Internet->customer UDP/53 packets. -- Mikael Abrahamsson email: swmike@swm.pp.se
On Mon, 1 Apr 2013 20:33:36 +0200 (CEST) Mikael Abrahamsson <swmike@swm.pp.se> wrote:
You're sending queries, not replies. That's why DPI is needed to do the blocking, rather than just by port.
What queries are sourced from port 53 nowadays?
I would expect from stubs this will be close enough to zero to be effectively zero. At least I would hope so. I don't have a great source of insight for a resolver of this type of source data that I can easily look at the moment, but if someone does I'd be interested to hear otherwise. On the authoritative side, which is easier for me to examine however, when I've looked at this before, and the last time was a year ago it was about 1% of all queries came from resolvers using source port 53. I just now checked another server and the percentage is practically the same. Before anyone dismisses 1% of queries as insignificant, keep in mind that if all remaining queries from all other possible source port values were equally distributed, that 1% (1 out of 100) is easily more common than any other. John
On 2013-04-02, at 18:18, John Kristoff <jtk@cymru.com> wrote:
I would expect from stubs this will be close enough to zero to be effectively zero. At least I would hope so.
This (below) is one of four resolvers, together providing service for two recursive DNS servers used by residential DSL and cable internet users at a medium-sized ISP in Canada. These resolvers are running unbound, which will never choose a source port of 53 on an outbound query; hence anything we see here with src port = dst port = 53 is one of those effective zeros. [dns1-p1:~]% sudo tcpdump -i em0 -n -c 10000 -w sample.pcap udp port 53 tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size 96 bytes 10000 packets captured 10267 packets received by filter 0 packets dropped by kernel [dns1-p1:~]% tcpdump -r sample.pcap -q udp src port 53 and udp dst port 53 | wc -l reading from file sample.pcap, link-type EN10MB (Ethernet) 26 [dns1-p1:~]% 26/1000 is more than zero but still quite small. Subsequent samples with bigger sizes give 332/100000, 3017/1000000. No science here, but 2% - 3% is what it looks like, which is big enough to be a noticeable support cost for a medium-scale provider if the customer damage is not robo-mediated in some way (e.g. whitelist known offenders to avoid the support phone glowing red when you first turn it on). Joe
On Tue, 2 Apr 2013 18:41:17 -0400 Joe Abley <jabley@hopcount.ca> wrote:
26/1000 is more than zero but still quite small. Subsequent samples with bigger sizes give 332/100000, 3017/1000000.
No science here, but 2% - 3% is what it looks like, which is big enough to be a noticeable support cost for a medium-scale provider if the customer damage is not robo-mediated in some way (e.g. whitelist known offenders to avoid the support phone glowing red when you first turn it on).
Thanks Joe. That is interesting. I can only imagine that on the customer side there are queries coming from something other than typical OS stub resolvers on unix and Windows based hosts. I suppose some sort of NAT/PAT box could account for some of it, maybe more likely could be some common CPE forwarder that uses that port by default. If the latter, that might be considered a serious enough risk that the vendor should address it if they haven't already. If no one else does, another side project I'll add to my list of things to do on a rainy day. John
I think that is .2% - .3%, no? On Tue, Apr 2, 2013 at 5:41 PM, Joe Abley <jabley@hopcount.ca> wrote:
On 2013-04-02, at 18:18, John Kristoff <jtk@cymru.com> wrote:
I would expect from stubs this will be close enough to zero to be effectively zero. At least I would hope so.
This (below) is one of four resolvers, together providing service for two recursive DNS servers used by residential DSL and cable internet users at a medium-sized ISP in Canada. These resolvers are running unbound, which will never choose a source port of 53 on an outbound query; hence anything we see here with src port = dst port = 53 is one of those effective zeros.
[dns1-p1:~]% sudo tcpdump -i em0 -n -c 10000 -w sample.pcap udp port 53 tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size 96 bytes 10000 packets captured 10267 packets received by filter 0 packets dropped by kernel [dns1-p1:~]% tcpdump -r sample.pcap -q udp src port 53 and udp dst port 53 | wc -l reading from file sample.pcap, link-type EN10MB (Ethernet) 26 [dns1-p1:~]%
26/1000 is more than zero but still quite small. Subsequent samples with bigger sizes give 332/100000, 3017/1000000.
No science here, but 2% - 3% is what it looks like, which is big enough to be a noticeable support cost for a medium-scale provider if the customer damage is not robo-mediated in some way (e.g. whitelist known offenders to avoid the support phone glowing red when you first turn it on).
Joe
----- Original Message -----
From: "Joe Abley" <jabley@hopcount.ca>
On 2013-04-03, at 11:25, Jerry Dent <effinjdent@gmail.com> wrote:
I think that is .2% - .3%, no?
Oh, you're right -- it does seem substantially closer to zero when you put the decimal point in the right place :-)
Huh? 23 in 1000 is in fact 2.3%. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
On 2013-04-03, at 12:52, Jay Ashworth <jra@baylink.com> wrote:
----- Original Message -----
From: "Joe Abley" <jabley@hopcount.ca>
On 2013-04-03, at 11:25, Jerry Dent <effinjdent@gmail.com> wrote:
I think that is .2% - .3%, no?
Oh, you're right -- it does seem substantially closer to zero when you put the decimal point in the right place :-)
Huh?
23 in 1000 is in fact 2.3%.
That's it, no more e-mail for me today. Joe
His sample was 10K not 1000. Look higher. On Wed, Apr 3, 2013 at 12:15 PM, Joe Abley <jabley@hopcount.ca> wrote:
On 2013-04-03, at 12:52, Jay Ashworth <jra@baylink.com> wrote:
----- Original Message -----
From: "Joe Abley" <jabley@hopcount.ca>
On 2013-04-03, at 11:25, Jerry Dent <effinjdent@gmail.com> wrote:
I think that is .2% - .3%, no?
Oh, you're right -- it does seem substantially closer to zero when you put the decimal point in the right place :-)
Huh?
23 in 1000 is in fact 2.3%.
That's it, no more e-mail for me today.
Joe
( Ok, ok, another bad customer =D ) Starting today at 5h15m EST... There is a bigger than usual DDoS amplification against the IP's listed below. Granted root servers query is barely 1k while the usual isc.org is 3.5k and this is a "possible" 15Mbps from this one source but still :( PS: If you're a Tier and wish to track down the *^%$*#@ source ISP's to explain to them the joy of BCP38... Contact me off list, from your corporate email address, and I'll provide you with the IP of that server. ----- IP are targeted for DDoS amplification. Format: <IP> <query count during 10 seconds> [query] 94.23.42.215 2128 . IN ANY +E 208.98.25.130 3079 . IN ANY +E 188.134.46.102 2639 . IN ANY +E 108.61.239.105 2270 . IN ANY +E 95.129.166.186 2416 . IN ANY +E 176.9.210.53 2839 . IN ANY +E 145.53.65.130 2326 . IN ANY +E 99.198.100.86 1223 . IN ANY +E 37.59.72.74 2508 . IN ANY +E 199.83.133.42 2392 . IN ANY +E 74.63.248.210 1481 . IN ANY +E 173.199.68.62 1178 . IN ANY +E 82.80.17.4 2666 . IN ANY +E 188.162.228.50 1075 . IN ANY +E 79.225.4.183 1014 . IN ANY +E 78.108.79.171 1291 . IN ANY +E 31.53.123.192 1093 . IN ANY +E 90.3.194.151 1245 . IN ANY +E 27.50.70.191 1304 . IN ANY +E 198.7.63.39 1579 . IN ANY +E 81.220.28.129 1103 . IN ANY +E 198.105.218.12 1110 . IN ANY +E 86.160.85.37 1128 . IN ANY +E 184.95.35.194 1237 . IN ANY +E 134.255.237.244 1245 . IN ANY +E 178.32.36.67 1588 . IN ANY +E 204.45.55.8 1419 . IN ANY +E 95.211.209.182 1520 . IN ANY +E 80.192.224.22 1430 . IN ANY +E 24.244.248.8 1414 . IN ANY +E 79.71.69.165 1090 . IN ANY +E 24.244.248.57 1364 . IN ANY +E 82.132.226.216 1079 . IN ANY +E 69.162.97.99 1601 . IN ANY +E ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443
Is anyone in particular being pocketed, or are these random addresses? Sent from my Mobile Device. -------- Original message -------- From: Alain Hebert <ahebert@pubnix.net> Date: 05/09/2013 10:16 AM (GMT-08:00) To: nanog@nanog.org Subject: Open Resolvers pseudo Honey Pot (Was: Open Resolver Problems) ( Ok, ok, another bad customer =D ) Starting today at 5h15m EST... There is a bigger than usual DDoS amplification against the IP's listed below. Granted root servers query is barely 1k while the usual isc.org is 3.5k and this is a "possible" 15Mbps from this one source but still :( PS: If you're a Tier and wish to track down the *^%$*#@ source ISP's to explain to them the joy of BCP38... Contact me off list, from your corporate email address, and I'll provide you with the IP of that server. ----- IP are targeted for DDoS amplification. Format: <IP> <query count during 10 seconds> [query] 94.23.42.215 2128 . IN ANY +E 208.98.25.130 3079 . IN ANY +E 188.134.46.102 2639 . IN ANY +E 108.61.239.105 2270 . IN ANY +E 95.129.166.186 2416 . IN ANY +E 176.9.210.53 2839 . IN ANY +E 145.53.65.130 2326 . IN ANY +E 99.198.100.86 1223 . IN ANY +E 37.59.72.74 2508 . IN ANY +E 199.83.133.42 2392 . IN ANY +E 74.63.248.210 1481 . IN ANY +E 173.199.68.62 1178 . IN ANY +E 82.80.17.4 2666 . IN ANY +E 188.162.228.50 1075 . IN ANY +E 79.225.4.183 1014 . IN ANY +E 78.108.79.171 1291 . IN ANY +E 31.53.123.192 1093 . IN ANY +E 90.3.194.151 1245 . IN ANY +E 27.50.70.191 1304 . IN ANY +E 198.7.63.39 1579 . IN ANY +E 81.220.28.129 1103 . IN ANY +E 198.105.218.12 1110 . IN ANY +E 86.160.85.37 1128 . IN ANY +E 184.95.35.194 1237 . IN ANY +E 134.255.237.244 1245 . IN ANY +E 178.32.36.67 1588 . IN ANY +E 204.45.55.8 1419 . IN ANY +E 95.211.209.182 1520 . IN ANY +E 80.192.224.22 1430 . IN ANY +E 24.244.248.8 1414 . IN ANY +E 79.71.69.165 1090 . IN ANY +E 24.244.248.57 1364 . IN ANY +E 82.132.226.216 1079 . IN ANY +E 69.162.97.99 1601 . IN ANY +E ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443
It looks like to be a service and some of their customers. ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443 On 05/09/13 13:18, Warren Bailey wrote:
94.23.42.215 2128 . IN ANY +E 208.98.25.130 3079 . IN ANY +E 188.134.46.102 2639 . IN ANY +E 108.61.239.105 2270 . IN ANY +E 95.129.166.186 2416 . IN ANY +E 176.9.210.53 2839 . IN ANY +E 145.53.65.130 2326 . IN ANY +E 99.198.100.86 1223 . IN ANY +E 37.59.72.74 2508 . IN ANY +E 199.83.133.42 2392 . IN ANY +E 74.63.248.210 1481 . IN ANY +E 173.199.68.62 1178 . IN ANY +E 82.80.17.4 2666 . IN ANY +E 188.162.228.50 1075 . IN ANY +E 79.225.4.183 1014 . IN ANY +E 78.108.79.171 1291 . IN ANY +E 31.53.123.192 1093 . IN ANY +E 90.3.194.151 1245 . IN ANY +E 27.50.70.191 1304 . IN ANY +E 198.7.63.39 1579 . IN ANY +E 81.220.28.129 1103 . IN ANY +E 198.105.218.12 1110 . IN ANY +E 86.160.85.37 1128 . IN ANY +E 184.95.35.194 1237 . IN ANY +E 134.255.237.244 1245 . IN ANY +E 178.32.36.67 1588 . IN ANY +E 204.45.55.8 1419 . IN ANY +E 95.211.209.182 1520 . IN ANY +E 80.192.224.22 1430 . IN ANY +E 24.244.248.8 1414 . IN ANY +E 79.71.69.165 1090 . IN ANY +E 24.244.248.57 1364 . IN ANY +E 82.132.226.216 1079 . IN ANY +E 69.162.97.99 1601 . IN ANY +E
In message <518BD982.60709@pubnix.net>, Alain Hebert writes:
( Ok, ok, another bad customer =D )
Starting today at 5h15m EST...
There is a bigger than usual DDoS amplification against the IP's listed below.
Granted root servers query is barely 1k while the usual isc.org is 3.5k and this is a "possible" 15Mbps from this one source but still :(
With a validating resolver "dig any . +edns" return a 1872 byte payload. "dig any . +dnssec" return a 2030 byte payload. (difference is NS RRSIG records) Getting the DNSKEY records included isn't hard. Throw a single DNSKEY query into the stream once a day/hour and it will be cached for 48 hours. If you have the SOA cached as well it gets to "dig any . +edns" return a 2087 byte payload. "dig any . +dnssec" return a 2245 byte payload. Mark
PS:
If you're a Tier and wish to track down the *^%$*#@ source ISP's to explain to them the joy of BCP38...
Contact me off list, from your corporate email address, and I'll provide you with the IP of that server.
----- IP are targeted for DDoS amplification.
Format:
<IP> <query count during 10 seconds> [query]
94.23.42.215 2128 . IN ANY +E 208.98.25.130 3079 . IN ANY +E 188.134.46.102 2639 . IN ANY +E 108.61.239.105 2270 . IN ANY +E 95.129.166.186 2416 . IN ANY +E 176.9.210.53 2839 . IN ANY +E 145.53.65.130 2326 . IN ANY +E 99.198.100.86 1223 . IN ANY +E 37.59.72.74 2508 . IN ANY +E 199.83.133.42 2392 . IN ANY +E 74.63.248.210 1481 . IN ANY +E 173.199.68.62 1178 . IN ANY +E 82.80.17.4 2666 . IN ANY +E 188.162.228.50 1075 . IN ANY +E 79.225.4.183 1014 . IN ANY +E 78.108.79.171 1291 . IN ANY +E 31.53.123.192 1093 . IN ANY +E 90.3.194.151 1245 . IN ANY +E 27.50.70.191 1304 . IN ANY +E 198.7.63.39 1579 . IN ANY +E 81.220.28.129 1103 . IN ANY +E 198.105.218.12 1110 . IN ANY +E 86.160.85.37 1128 . IN ANY +E 184.95.35.194 1237 . IN ANY +E 134.255.237.244 1245 . IN ANY +E 178.32.36.67 1588 . IN ANY +E 204.45.55.8 1419 . IN ANY +E 95.211.209.182 1520 . IN ANY +E 80.192.224.22 1430 . IN ANY +E 24.244.248.8 1414 . IN ANY +E 79.71.69.165 1090 . IN ANY +E 24.244.248.57 1364 . IN ANY +E 82.132.226.216 1079 . IN ANY +E 69.162.97.99 1601 . IN ANY +E
----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443
-- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: marka@isc.org
On 05/09/13 19:03, Mark Andrews wrote:
In message <518BD982.60709@pubnix.net>, Alain Hebert writes:
( Ok, ok, another bad customer =D )
Starting today at 5h15m EST...
There is a bigger than usual DDoS amplification against the IP's listed below.
Granted root servers query is barely 1k while the usual isc.org is 3.5k and this is a "possible" 15Mbps from this one source but still :(
With a validating resolver
"dig any . +edns" return a 1872 byte payload. "dig any . +dnssec" return a 2030 byte payload. (difference is NS RRSIG records)
Getting the DNSKEY records included isn't hard. Throw a single DNSKEY query into the stream once a day/hour and it will be cached for 48 hours.
If you have the SOA cached as well it gets to
"dig any . +edns" return a 2087 byte payload. "dig any . +dnssec" return a 2245 byte payload.
Mark
Well during the spamhaus incident I saw some at around 8k. On another note... After 18 hours, that "pot" is still receiving ~200pps (down from 800 and 400pps) and its up to 614 IP now... I still do not see the motive behind this one: Either someone messed up his botnet and he's stuck on it =D Could be a rootkit using this server as a DNS server (lots of targets are hosted Linux in outfit like OVH). ( But again why spamming . IN ANY queries and not cache the results ) And a new query popped up -> doc.gov IN ANY +E, granted I only saw a few of them. And a few of the source IP's are gaming forums mostly Minecraft oriented. PS: Reminder, that this server do not actually amplify anything and the service at that location is not affected.
PS:
If you're a Tier and wish to track down the *^%$*#@ source ISP's to explain to them the joy of BCP38...
Contact me off list, from your corporate email address, and I'll provide you with the IP of that server.
----- IP are targeted for DDoS amplification.
Format:
<IP> <query count during 10 seconds> [query]
94.23.42.215 2128 . IN ANY +E 208.98.25.130 3079 . IN ANY +E 188.134.46.102 2639 . IN ANY +E 108.61.239.105 2270 . IN ANY +E 95.129.166.186 2416 . IN ANY +E 176.9.210.53 2839 . IN ANY +E 145.53.65.130 2326 . IN ANY +E 99.198.100.86 1223 . IN ANY +E 37.59.72.74 2508 . IN ANY +E 199.83.133.42 2392 . IN ANY +E 74.63.248.210 1481 . IN ANY +E 173.199.68.62 1178 . IN ANY +E 82.80.17.4 2666 . IN ANY +E 188.162.228.50 1075 . IN ANY +E 79.225.4.183 1014 . IN ANY +E 78.108.79.171 1291 . IN ANY +E 31.53.123.192 1093 . IN ANY +E 90.3.194.151 1245 . IN ANY +E 27.50.70.191 1304 . IN ANY +E 198.7.63.39 1579 . IN ANY +E 81.220.28.129 1103 . IN ANY +E 198.105.218.12 1110 . IN ANY +E 86.160.85.37 1128 . IN ANY +E 184.95.35.194 1237 . IN ANY +E 134.255.237.244 1245 . IN ANY +E 178.32.36.67 1588 . IN ANY +E 204.45.55.8 1419 . IN ANY +E 95.211.209.182 1520 . IN ANY +E 80.192.224.22 1430 . IN ANY +E 24.244.248.8 1414 . IN ANY +E 79.71.69.165 1090 . IN ANY +E 24.244.248.57 1364 . IN ANY +E 82.132.226.216 1079 . IN ANY +E 69.162.97.99 1601 . IN ANY +E
----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443
Is it me or the bigger a corporation gets the more vindictive (a b-word intended) they are to customers leaving them? ------ One of my *new* customer was caught by the local "monopole" into moving their domain/site/emails/phone/oxygen supply/etc to them. But when the usual grace period stopped and they started receiving invoices that would make a loan shark go: "are you insane?!?", they decided to move. After a somewhat pleasant call to the "monopole" informing them that they are planning to divorce them in 30 days, and that it was clearly stated that since they are paying for those additional 30 days that their services wont be cut off... 15 minutes later. Boom. No domain, no site, no emails... After a rather stern call, they get their domain up, no site... After another rather stern call, they get their site up, no emails... After another rather stern call, they get... no emails. "Oh its 17:01, our support is closed for day. Is there anything else I can help you with?" On top of it that "monopole" has for their procedure to let the 5 days "AUTO-ACK" expire when they are asked to transfer a domain away from them. ( They are a reseller of tucows, and PS: it is not Tucows fault. That 5 days 'AUTO-ACK" process is there because of corporation acting like that "monopole" ) Good thing I was able to get back into their account earlier this morning and recreate their emails... Now lets hope they don't do this again until my customer get his domain back in its hands. ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443
The biggest grievance I have is in regards to carriers with automatic contract renewals. Combined with the fact that these carriers either refused to allow month to month billing or will allow it at double / triple current rates, coordinating disconnection of older services while turning up new services with a different carrier in the same time frame can be a real challenge. Adding insult to injury is the fact that one does not simply resolve carrier billing issues - I've had multiple incidents which took almost a year to resolve. I personally think that the automatic renewal of a three year term should be criminal. The same goes for price increases while I'm under a contract rate - Apparently as long as there is a provision in the small print (which is able to be changed at will, due to the small print referencing a document on the carrier's website), be ready to pay more whenever the carrier dictates, regardless of what your contract says. Typing this was somewhat therapeutic. -----Original Message----- From: Alain Hebert [mailto:ahebert@pubnix.net] Sent: Friday, July 12, 2013 10:45 AM To: nanog@nanog.org Subject: Friday Hosing Is it me or the bigger a corporation gets the more vindictive (a b-word intended) they are to customers leaving them? ------ One of my *new* customer was caught by the local "monopole" into moving their domain/site/emails/phone/oxygen supply/etc to them. But when the usual grace period stopped and they started receiving invoices that would make a loan shark go: "are you insane?!?", they decided to move. After a somewhat pleasant call to the "monopole" informing them that they are planning to divorce them in 30 days, and that it was clearly stated that since they are paying for those additional 30 days that their services wont be cut off... 15 minutes later. Boom. No domain, no site, no emails... After a rather stern call, they get their domain up, no site... After another rather stern call, they get their site up, no emails... After another rather stern call, they get... no emails. "Oh its 17:01, our support is closed for day. Is there anything else I can help you with?" On top of it that "monopole" has for their procedure to let the 5 days "AUTO-ACK" expire when they are asked to transfer a domain away from them. ( They are a reseller of tucows, and PS: it is not Tucows fault. That 5 days 'AUTO-ACK" process is there because of corporation acting like that "monopole" ) Good thing I was able to get back into their account earlier this morning and recreate their emails... Now lets hope they don't do this again until my customer get his domain back in its hands. ----- Alain Hebert ahebert@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.net Fax: 514-990-9443
----- Original Message -----
From: "Alain Hebert" <ahebert@pubnix.net>
Is it me or the bigger a corporation gets the more vindictive (a b-word intended) they are to customers leaving them?
[ long saga elided ] And now you know why it's standard operating procedure: *Never* tell the losing service provider they're losing *until you've actually done the cutover*. If that costs you some money, well, at least you didn't go off the air. Responsibility for you not going off the air rests, finally, with you. Your clients don't care if you later win the lawsuit. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
On Fri, 12 Jul 2013, Alain Hebert wrote:
Is it me or the bigger a corporation gets the more vindictive (a b-word intended) they are to customers leaving them?
"Never attribute to malice that which is adequately explained by stupidity." Hopefully this isn't one of mine, but I've seen this happen in our billing/sales, regardless.
Composed on a virtual keyboard, please forgive typos. On Jul 12, 2013, at 13:25, nanog@namor.ca wrote:
On Fri, 12 Jul 2013, Alain Hebert wrote:
Is it me or the bigger a corporation gets the more vindictive (a b-word intended) they are to customers leaving them?
"Never attribute to malice that which is adequately explained by stupidity."
I prefer Heinlein's version: Never attribute to malice that which can be adequately explained by stupidity, but don't rule out malice. And, of course the corollary that any sufficiently advanced stupidity is indistinguishable from malice. Put another way, whether it was stupid or evil, the results are the same. Turning off a customer in good standing is actionable in court, and should be avoided by legitimate businesses at nearly all costs. Not correcting the error (should it happen) when notified goes from "oops" to evil, whether intentional or not. And yes, I've probably worked for a corporation that has done this at least once over the years. (I did work for a telco for a while. :-) Doesn't mean I can't think it was evil of "us" and work to stop it from ever happening again. -- TTFN, patrick
On 13-07-12 11:44, Alain Hebert wrote:
After a somewhat pleasant call to the "monopole" informing them that they are planning to divorce them in 30 days, and that it was clearly stated that since they are paying for those additional 30 days that their services wont be cut off...
1- You call to make cancellation on date X. Speak to person 1. 2- Wait 20 minutes 3- Call again, speak to person 2, confirm your services will be cancelled on date X and that you have already paid for services until then. (or that an invoice has already been produced or will be produced.) I am not one to defend the big bad incumbents. But consider you are calling on the day before new billing cycle begins. The agent may have flagged your account to to close at end of current billing cycle thinking it would be about a month from now. But it happens to be only a few hours from now. You should also note that big bad incumbents have bad reputation of requiring one extra month payment before they allow you to leave them. So the agent may have been nice in waiving that requirement and allowed you to leave earlier, saving you a month's worth of billing. (and not realising the troubles it will cause a business when service is cut before the date specified by customer)
On 7/12/13, Jean-Francois Mezei <jfmezei_nanog@vaxination.ca> wrote:
On 13-07-12 11:44, Alain Hebert wrote: 1- You call to make cancellation on date X. Speak to person 1. 2- Wait 20 minutes
In a theoretical ideal world, that would probably work fine, but a request for "Shut off service effective the 15th" is just asking for trouble; with many large providers, making a request like that is very likely to be misinterpreted or result in unintended early shutoff. Some providers, even if they understand the request, may choose not to entertain a request more complicated than "Cancel immediately"; they may agree to cancel at date X, then things get terminated immediately, and their terms of service probably contains a rule that the provider may end service at their discretion, without notice with the payment for the rest of the same subscription term following date X still being due. Their account terms also probably specify required binding arbitration, and liability limited to hosting fees, and the subscription cost -- a few dollars to be recovered for a few days extra downtime is not likely to exceed the filing fees that would be required for any sort of court actions. If the domain contains critical resources, you have new hosting setup, before you even think about cancelling old hosting. You make sure the DNS servers point to reliable name service provider(s) who will continue to provide DNS hosting, and you make sure the domain registration is under your full control with full access to all settings, and no provider listed as Admin contact.. Best practice would be Only after you secured all those things and updated A records to point to new hosting provider, should the original provider be contacted with a request to "cancel" the web hosting or particular service cancelled. This is a case of: pay significantly more (multiple providers a short time) in order to greatly reduce risk ---- whereas, asking a provider to cancel at date X, and avoiding overlap --- significantly reduces cost while greatly increasing risk of a 24-48 hour outage.
3- Call again, speak to person 2, confirm your services will be cancelled on date X and that you have already paid for services until then. (or that an invoice has already been produced or will be produced.) -- -JH
I work for a telco and have seen customers double up on circuits. Why do you think even the largest carriers don't send in a disconnect order for a customer circuit until the replacement circuit is in place and working? Because they've learned the hard way, too. Frank -----Original Message----- From: Jimmy Hess [mailto:mysidia@gmail.com] Sent: Saturday, July 13, 2013 4:09 AM To: Jean-Francois Mezei Cc: nanog@nanog.org Subject: Re: Friday Hosing On 7/12/13, Jean-Francois Mezei <jfmezei_nanog@vaxination.ca> wrote:
On 13-07-12 11:44, Alain Hebert wrote: 1- You call to make cancellation on date X. Speak to person 1. 2- Wait 20 minutes
In a theoretical ideal world, that would probably work fine, but a request for "Shut off service effective the 15th" is just asking for trouble; with many large providers, making a request like that is very likely to be misinterpreted or result in unintended early shutoff. Some providers, even if they understand the request, may choose not to entertain a request more complicated than "Cancel immediately"; they may agree to cancel at date X, then things get terminated immediately, and their terms of service probably contains a rule that the provider may end service at their discretion, without notice with the payment for the rest of the same subscription term following date X still being due. Their account terms also probably specify required binding arbitration, and liability limited to hosting fees, and the subscription cost -- a few dollars to be recovered for a few days extra downtime is not likely to exceed the filing fees that would be required for any sort of court actions. If the domain contains critical resources, you have new hosting setup, before you even think about cancelling old hosting. You make sure the DNS servers point to reliable name service provider(s) who will continue to provide DNS hosting, and you make sure the domain registration is under your full control with full access to all settings, and no provider listed as Admin contact.. Best practice would be Only after you secured all those things and updated A records to point to new hosting provider, should the original provider be contacted with a request to "cancel" the web hosting or particular service cancelled. This is a case of: pay significantly more (multiple providers a short time) in order to greatly reduce risk ---- whereas, asking a provider to cancel at date X, and avoiding overlap --- significantly reduces cost while greatly increasing risk of a 24-48 hour outage.
3- Call again, speak to person 2, confirm your services will be cancelled on date X and that you have already paid for services until then. (or that an invoice has already been produced or will be produced.) -- -JH
Just finished migration from a provider that I was no longer happy with to a new provider. Fully expecting them to turn me off the moment I said 'cancel', I prepared everything in advance, moved all the pointers over a few days prior to my planned day to tell them to 'shutoff', retrieved a final backup, then used their web billing interface to tell them I wanted to cancel service. Much to my surprise, they actually had a selection for "what date do you want to cancel service on". I set it to be the next day. I had no issues, but I do attribute much of this to "I migrated services beginning 6 months prior" (when I decided I was not going to renew... it was a 2 year contract). Note: Neither provider is local. As mentioned by others, it is your responsibility to maintain continuity of service when you move between providers. The best way I know to ensure this is to maintain control. Make sure your new system is up and configured before discontinuing your old system. - Eric
On 7/13/13, Eric Adler <eaptech@gmail.com> wrote: Yes. Maintain control. If you want to avoid the cost of doubling, discuss the risk for the new provider, and ask if they can waive their fee for the period until the old service is cancelled, before agreeing to complete the sale.
As mentioned by others, it is your responsibility to maintain continuity of service when you move between providers. The best way I know to ensure this is to maintain control. Make sure your new system is up and configured before discontinuing your old system.
- Eric -- -JH
On 2013-04-01, at 14:19, Jay Ashworth <jra@baylink.com> wrote:
From: "Roland Dobbins" <rdobbins@arbor.net>
On Apr 1, 2013, at 11:18 PM, Patrick W. Gilmore wrote:
Of course, since users shouldn't be using off-net name servers anyway, this isn't really a problem! :)
;>
It's easy enough to construct ACLs to restrict the broadband consumer access networks from doing so. Additional egress filtering would catch any reflected attacks, per your previous comments.
So, how would Patrick's caveat affect me, whose recursive resolver *is on my Linux laptop*? Would not that recursor be making queries he advocates blocking?
The badness that Patrick is talking about blocking are DNS responses being sent from consumer devices to the Internet, answering DNS queries being sent from the Internet towards consumer devices. (I think. This thread is sufficiently circular that I feel a bit dizzy, and could be mistaken.) The DNS traffic outbound from your laptop will be DNS queries (not responses) and the inbound traffic will be DNS responses (not queries). The traffic profiles are different. The case where infected consumer devices originate source-spoofed queries towards open resolvers, feeding a query stream to an amplifier for delivery to a victim, is mitigated by preventing those consumer devices from spoofing their source address, so BCP38. The case where infected consumer devices originate non-source-spoofed queries towards DNS servers in order to overwhelm the servers themselves with perfectly legitimate-looking queries is a harder problem to solve at the edge, and is most easily mitigated for DNS server operators by the approach "ensure great headroom". Joe
participants (16)
-
Alain Hebert -
Eric Adler -
Frank Bulk -
Jason Faraone -
Jay Ashworth -
Jean-Francois Mezei -
Jerry Dent -
Jimmy Hess -
Joe Abley -
John Kristoff -
Mark Andrews -
Mikael Abrahamsson -
nanog＠namor.ca -
Patrick W. Gilmore -
Valdis.Kletnieks＠vt.edu -
Warren Bailey