Snap, announcing a few of our /21s and a /23. Seems they did something similar a few year ago: http://www.bgpmon.net/hijack-by-as4761-indosat-a-quick-report/ I can't make any contact with Indosat (website non responsive / email queuing). This is what I have back from Aware Corp. AS18356 (first AS in the path): I can confirm that we are seeing your prefixes as advertised by AS4761, via one of our upstreams CAT AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH) We (Aware Corporation - AS18356) operate a BGPMon PeerMon node which is probably why you are seeing this alert from our AS. It is likely that your highjacked prefixes are being advertised to all of CAT's customers. I suggest contacting AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID) directly for resolution as there is little we can do as a stub AS. Regards, Lee. -----Original Message----- From: Vlade Ristevski [mailto:vristevs@ramapo.edu] Sent: 02 April 2014 20:05 To: nanog@nanog.org Subject: Re: BGPMON Alert Questions I just got the same alert for one of my prefixes one minute ago. On 4/2/2014 2:59 PM, Frank Bulk wrote:

I received a similar notification about one of our prefixes also a few minutes ago. I couldn't find a looking glass for AS4761 or AS4651. But I also couldn't hit the websites for either AS, either.

Frank

-----Original Message----- From: Joseph Jenkins [mailto:joe@breathe-underwater.com] Sent: Wednesday, April 02, 2014 1:52 PM To: nanog@nanog.org Subject: BGPMON Alert Questions

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure they are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24:

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

-- Vlad

I emailed hostmaster@indosat.com a little over an hour ago, and no response as yet. Anyone having luck making contact with Indosat themselves? On Wed, Apr 2, 2014 at 2:33 PM, Andrew (Andy) Ashley <andrew.a@aware.co.th>wrote:

Hi All,

I am a network admin for Aware Corporation AS18356 (Thailand), as mentioned in the alert. We operate a BGPMon PeerMon node on our network, which peers with the BGPMon service as a collector.

It is likely that AS4761 (INDOSAT) has somehow managed to hijack these prefixes and CAT (Communications Authority of Thailand AS4651) is not filtering them, hence they are announced to us and are triggering these BGPMon alerts.

I have had several mails to our NOC about this already and have responded directly to those. I suggest contacting Indosat directly to get this resolved. AS18356 is a stub AS, so we are not actually advertising these learned hijacked prefixes to anyone but BGPMon for data collection purposes.

Thanks.

Regards,

Andrew Ashley

Office: +27 21 673 6841 E-mail: andrew.a@aware.co.th Web: www.aware.co.th

On 2014/04/02, 21:05, "Vlade Ristevski" <vristevs@ramapo.edu> wrote:

...

I just got the same alert for one of my prefixes one minute ago.

On 4/2/2014 2:59 PM, Frank Bulk wrote:

...

I received a similar notification about one of our prefixes also a few minutes ago. I couldn't find a looking glass for AS4761 or AS4651. But I also couldn't hit the websites for either AS, either.

Frank

-----Original Message----- From: Joseph Jenkins [mailto:joe@breathe-underwater.com] Sent: Wednesday, April 02, 2014 1:52 PM To: nanog@nanog.org Subject: BGPMON Alert Questions

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure they are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24:

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

-- Vlad

I got a bounce from Indosat saying: "Dear Senders, Thank you for your email, started March,1st 2012 email address for correspondence with Indosat IP Support & All Support INP will be change and not active with detail information as follows : 1. Correspondence and complain handling for Indosat Corporate customers (INP, IDIA and INIX services) please kindly address to : corporatesolution@indosat.com (Service Desk MIDI Indosat Corporate Solution) 2. Correspondence and coordination for upstream and peering purpose please kindly address to : SNOCIPSurv@indosat.com (SNOC IP Surveillance) Thank you for your kind cooperation and understanding. Indosat IP Support" Perhaps the ³SNOC IP Surveillance² address is better? For CAT Thailand, the contact details I have are: NOC call center CAT Telecom Tel: 66 2 104 2382 FAX: 66 2 104 2281 e-mail: cusserv@cattelecom.com As someone mentioned, English may be an issue, especially at this time of the morning over there. Regards, Andrew Ashley Office: +27 21 673 6841 E-mail: andrew.a@aware.co.th Web: www.aware.co.th From: Aris Lambrianidis <effulgence@gmail.com> Date: Wednesday 02 April 2014 at 22:40 To: Andrew Ashley <andrew.a@aware.co.th> Cc: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: BGPMON Alert Questions Contacted ip.tac@indosat.com about this, I urge others to do the same. --Aris On Wed, Apr 2, 2014 at 9:33 PM, Andrew (Andy) Ashley <andrew.a@aware.co.th> wrote:

Hi All,

I am a network admin for Aware Corporation AS18356 (Thailand), as mentioned in the alert. We operate a BGPMon PeerMon node on our network, which peers with the BGPMon service as a collector.

It is likely that AS4761 (INDOSAT) has somehow managed to hijack these prefixes and CAT (Communications Authority of Thailand AS4651) is not filtering them, hence they are announced to us and are triggering these BGPMon alerts.

I have had several mails to our NOC about this already and have responded directly to those. I suggest contacting Indosat directly to get this resolved. AS18356 is a stub AS, so we are not actually advertising these learned hijacked prefixes to anyone but BGPMon for data collection purposes.

Thanks.

Regards,

Andrew Ashley

Office: +27 21 673 6841 <tel:%2B27%2021%20673%206841> E-mail: andrew.a@aware.co.th Web: www.aware.co.th <http://www.aware.co.th>

On 2014/04/02, 21:05, "Vlade Ristevski" <vristevs@ramapo.edu> wrote:

...

...

I just got the same alert for one of my prefixes one minute ago.

On 4/2/2014 2:59 PM, Frank Bulk wrote:

...

...

I received a similar notification about one of our prefixes also a few minutes ago. I couldn't find a looking glass for AS4761 or AS4651. But I also couldn't hit the websites for either AS, either.

Frank

-----Original Message----- From: Joseph Jenkins [mailto:joe@breathe-underwater.com] Sent: Wednesday, April 02, 2014 1:52 PM To: nanog@nanog.org Subject: BGPMON Alert Questions

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure they are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24 <http://8.37.93.0/24> :

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24 <http://8.37.93.0/24>

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

-- Vlad

Thanks, also emailed support@ noc@. Didn't receive any bounce emails.. eric@zerofail.com AS40191 On Apr 2, 2014 5:06 PM, Aris Lambrianidis <effulgence@gmail.com> wrote: Contacted ip.tac@indosat.com about this, I urge others to do the same. --Aris On Wed, Apr 2, 2014 at 9:33 PM, Andrew (Andy) Ashley <andrew.a@aware.co.th>wrote:

Hi All,

I am a network admin for Aware Corporation AS18356 (Thailand), as mentioned in the alert. We operate a BGPMon PeerMon node on our network, which peers with the BGPMon service as a collector.

It is likely that AS4761 (INDOSAT) has somehow managed to hijack these prefixes and CAT (Communications Authority of Thailand AS4651) is not filtering them, hence they are announced to us and are triggering these BGPMon alerts.

I have had several mails to our NOC about this already and have responded directly to those. I suggest contacting Indosat directly to get this resolved. AS18356 is a stub AS, so we are not actually advertising these learned hijacked prefixes to anyone but BGPMon for data collection purposes.

Thanks.

Regards,

Andrew Ashley

Office: +27 21 673 6841 E-mail: andrew.a@aware.co.th Web: www.aware.co.th<http://www.aware.co.th>

On 2014/04/02, 21:05, "Vlade Ristevski" <vristevs@ramapo.edu> wrote:

...

I just got the same alert for one of my prefixes one minute ago.

On 4/2/2014 2:59 PM, Frank Bulk wrote:

...

I received a similar notification about one of our prefixes also a few minutes ago. I couldn't find a looking glass for AS4761 or AS4651. But I also couldn't hit the websites for either AS, either.

Frank

-----Original Message----- From: Joseph Jenkins [mailto:joe@breathe-underwater.com] Sent: Wednesday, April 02, 2014 1:52 PM To: nanog@nanog.org Subject: BGPMON Alert Questions

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure they are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24:

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

-- Vlad

Tried the recipients mailbox is full, but it looks like all of the bgpmon alerts have cleared. On Wed, Apr 2, 2014 at 1:40 PM, Aris Lambrianidis <effulgence@gmail.com>wrote:

Contacted ip.tac@indosat.com about this, I urge others to do the same.

--Aris

On Wed, Apr 2, 2014 at 9:33 PM, Andrew (Andy) Ashley <andrew.a@aware.co.th>wrote:

...

Hi All,

I am a network admin for Aware Corporation AS18356 (Thailand), as mentioned in the alert. We operate a BGPMon PeerMon node on our network, which peers with the BGPMon service as a collector.

It is likely that AS4761 (INDOSAT) has somehow managed to hijack these prefixes and CAT (Communications Authority of Thailand AS4651) is not filtering them, hence they are announced to us and are triggering these BGPMon alerts.

I have had several mails to our NOC about this already and have responded directly to those. I suggest contacting Indosat directly to get this resolved. AS18356 is a stub AS, so we are not actually advertising these learned hijacked prefixes to anyone but BGPMon for data collection purposes.

Thanks.

Regards,

Andrew Ashley

Office: +27 21 673 6841 E-mail: andrew.a@aware.co.th Web: www.aware.co.th

On 2014/04/02, 21:05, "Vlade Ristevski" <vristevs@ramapo.edu> wrote:

...

I just got the same alert for one of my prefixes one minute ago.

On 4/2/2014 2:59 PM, Frank Bulk wrote:

...

I received a similar notification about one of our prefixes also a few minutes ago. I couldn't find a looking glass for AS4761 or AS4651. But I also couldn't hit the websites for either AS, either.

Frank

-----Original Message----- From: Joseph Jenkins [mailto:joe@breathe-underwater.com] Sent: Wednesday, April 02, 2014 1:52 PM To: nanog@nanog.org Subject: BGPMON Alert Questions

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure they are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24:

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

-- Vlad

On Thu, 3 Apr 2014, Adrian Minta wrote:

Already too late :(

*Delivery has failed to these recipients or groups:*

indriana.triyunianingtyas@indosat.com <mailto:indriana.triyunianingtyas@indosat.com> The recipient's mailbox is full and can't accept messages now. Please try resending this message later, or contact the recipient directly.

As long as that's not the only person behind the "ip.tac@indosat.com" mail alias, all hope is not lost. Still, I imagine their NOC is getting crushed with reports right now. jms

On 02.04.2014 23:40, Aris Lambrianidis wrote:

...

Contacted ip.tac@indosat.com about this, I urge others to do the same.

--Aris

On Wed, Apr 2, 2014 at 9:33 PM, Andrew (Andy) Ashley <andrew.a@aware.co.th>wrote:

...

Hi All,

I am a network admin for Aware Corporation AS18356 (Thailand), as mentioned in the alert. We operate a BGPMon PeerMon node on our network, which peers with the BGPMon service as a collector.

It is likely that AS4761 (INDOSAT) has somehow managed to hijack these prefixes and CAT (Communications Authority of Thailand AS4651) is not filtering them, hence they are announced to us and are triggering these BGPMon alerts.

I have had several mails to our NOC about this already and have responded directly to those. I suggest contacting Indosat directly to get this resolved. AS18356 is a stub AS, so we are not actually advertising these learned hijacked prefixes to anyone but BGPMon for data collection purposes.

-- Best regards, Adrian Minta

Another 5 of ours just got hit. Anyone have any ideas on what will be done about it? On Wed, Apr 2, 2014 at 1:18 PM, Frank Bulk <frnkblk@iname.com> wrote:

bgpmon has tweeted that "We're currently observing a large hijack event. Indosat AS4761 originating many prefixes not assigned to them."

Let's hope that AS4651 can quickly apply filters.

Frank

-----Original Message----- From: David Hubbard [mailto:dhubbard@dino.hostasaurus.com] Sent: Wednesday, April 02, 2014 2:03 PM To: Joseph Jenkins; nanog@nanog.org Subject: RE: BGPMON Alert Questions

If you contact bgpmon support you may be able to get some more in-depth information. I've contacted them before with alerts like those and they were able to give me specific date, time, ASN and interface information about the peering points that received the announcements; that might help make you present to the suspect party more likely to be acted upon.

-----Original Message----- From: Joseph Jenkins [mailto:joe@breathe-underwater.com] Sent: Wednesday, April 02, 2014 2:52 PM To: nanog@nanog.org Subject: BGPMON Alert Questions

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure they are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24:

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

-- eSited LLC (701) 390-9638

Saw this as well on my blocks. Is this malicious or did someone redistribute all of bgp with bad upstream filtering? On Wed, Apr 2, 2014 at 3:16 PM, James Laszko <jamesl@mythostech.com> wrote:

I have someone from cat.net.th on the phone and he doesn't speak a lot of English and I don't speak any Thai..... He knew what indosat was and their AS number. He further stated he got my email (never told him who I was), but he said he would be replying ASAP. We only had one /24 announced by indosat.

James Laszko Mythos Technology Inc

Sent from my iPad

...

On Apr 2, 2014, at 1:08 PM, "Bryan Tong" <contact@nullivex.com> wrote:

Another 5 of ours just got hit.

Anyone have any ideas on what will be done about it?

...

On Wed, Apr 2, 2014 at 1:18 PM, Frank Bulk <frnkblk@iname.com> wrote:

bgpmon has tweeted that "We're currently observing a large hijack event. Indosat AS4761 originating many prefixes not assigned to them."

Let's hope that AS4651 can quickly apply filters.

Frank

-----Original Message----- From: David Hubbard [mailto:dhubbard@dino.hostasaurus.com] Sent: Wednesday, April 02, 2014 2:03 PM To: Joseph Jenkins; nanog@nanog.org Subject: RE: BGPMON Alert Questions

If you contact bgpmon support you may be able to get some more in-depth information. I've contacted them before with alerts like those and they were able to give me specific date, time, ASN and interface information about the peering points that received the announcements; that might help make you present to the suspect party more likely to be acted upon.

-----Original Message----- From: Joseph Jenkins [mailto:joe@breathe-underwater.com] Sent: Wednesday, April 02, 2014 2:52 PM To: nanog@nanog.org Subject: BGPMON Alert Questions

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure they are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24:

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

-- eSited LLC (701) 390-9638

Same here. AS path is 18356 38794 4651 4761. Did anybody had any contact with AS 4761? Regards, Peter

Op 2 apr. 2014 om 22:57 heeft Curtis Doty <Curtis@GreenKey.net> het volgende geschreven:

...

On Wed, Apr 2, 2014 at 1:24 PM, Blake Dunlap <ikiris@gmail.com> wrote:

Is this malicious or did someone redistribute all of bgp with bad upstream filtering?

They perfectly re-advertized all mine. Loos like a huge mistake. And still ongoing.

Although this was nice to see:

==================================================================== RPKI Validation Failed (Code: 9) ==================================================================== Your prefix: 199.47.80.0/21: Prefix Description: NET-199-47-80-0-1 Update time: 2014-04-02 20:29 (UTC) Detected by #peers: 1 Detected prefix: 199.47.80.0/21 Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID) Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH) ASpath: 18356 38794 4651 4761 RPKI Status: ROA validation failed: Invalid Origin ASN, expected 46851

Albeit ineffective.

../C

where did you get that number ? aut-num: AS4761 as-name: INDOSAT-INP-AP descr: INDOSAT Internet Network Provider descr: Internet Network Access Point in INDONESIA country: ID admin-c: IH151-AP tech-c: DA205-AP mnt-by: MAINT-ID-INDOSAT-INP changed: hostmaster@indosat.com 20081006 source: APNIC person: Dewi Amalia nic-hdl: DA205-AP e-mail: dewi.amalia@indosat.com address: PT INDOSAT address: JL. Medan Merdeka Barat 21 address: Jakarta Pusat phone: +62-21-30444066 fax-no: +62-21-30001073 country: ID changed: dewi.amalia@indosat.com 20080117 mnt-by: MAINT-ID-INDOSAT-INP source: APNIC person: INDOSAT INP Hostmaster nic-hdl: IH151-AP e-mail: hostmaster@indosat.com address: PT Indosat address: Jl. Medan Merdeka Barat 21 address: Jakarta Pusat phone: +62-21-30444066 fax-no: +62-21-30001073 country: ID changed: hostmaster@indosat.com 20120104 mnt-by: MAINT-ID-INDOSAT-INP source: APNIC Bob Evans CTO

I called into +66 2104-2374

James Laszko Mythos Technology Inc

Sent from my iPad

...

On Apr 2, 2014, at 1:08 PM, "Bryan Tong" <contact@nullivex.com> wrote:

Another 5 of ours just got hit.

Anyone have any ideas on what will be done about it?

...

On Wed, Apr 2, 2014 at 1:18 PM, Frank Bulk <frnkblk@iname.com> wrote:

bgpmon has tweeted that "We're currently observing a large hijack event. Indosat AS4761 originating many prefixes not assigned to them."

Let's hope that AS4651 can quickly apply filters.

Frank

-----Original Message----- From: David Hubbard [mailto:dhubbard@dino.hostasaurus.com] Sent: Wednesday, April 02, 2014 2:03 PM To: Joseph Jenkins; nanog@nanog.org Subject: RE: BGPMON Alert Questions

If you contact bgpmon support you may be able to get some more in-depth information. I've contacted them before with alerts like those and they were able to give me specific date, time, ASN and interface information about the peering points that received the announcements; that might help make you present to the suspect party more likely to be acted upon.

-----Original Message----- From: Joseph Jenkins [mailto:joe@breathe-underwater.com] Sent: Wednesday, April 02, 2014 2:52 PM To: nanog@nanog.org Subject: BGPMON Alert Questions

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure they are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24:

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

-- eSited LLC (701) 390-9638

Got this response from HE We are not in the as-path of the routes listed below. It seems we accepted some of them from a route server. I'm not seeing them in the table at this time. -- Rob Mosher Senior Network and Software Engineer Hurricane Electric / AS6939 On Wed, Apr 2, 2014 at 2:51 PM, Seth Mattinen <sethm@rollernet.us> wrote:

On 4/2/14, 13:31, Bob Evans wrote:

...

where did you get that number ?

I think that was a number for CAT, AS4651.

~Seth

-- eSited LLC (701) 390-9638

On Wed, 2 Apr 2014, Laszlo Hanyecz wrote:

They're just leaking every route right? Is it possible to poison the AS paths you announce with their own AS to get them to let go of your prefixes until it's fixed? Would that work, or some other trick that can be done without their cooperation?

Keep in mind that the more AS hops there are between you and Indosat, the less effective that any hackery you do in your own BGP table will be. Two things need to happen: 1. Indosat needs to clean their mess up. 2. Indosat's upstreams need to apply some BGP clue to Indosat's announcements. It's pretty clear that both parties have dropped the ball in a big way, in terms of sane BGP filtering practices. jms

On Wednesday, April 02, 2014 08:59:58 PM Justin M. Streiner wrote:

It's pretty clear that both parties have dropped the ball in a big way, in terms of sane BGP filtering practices.

It's amazing, isn't it? I have a customer of one my upstreams (Upstream A), at the moment, who are leaking my routes to another one of their upstreams (Upstream B). The problem is that Upstream B is re-announcing my leaked routes from their customer to the rest of the Internet. So both Upstream B's customer as well as Upstream B are at fault. That Upstream B is simply "accepting everything" their customer is sending to them without applying proper filters, or checking to confirm that what their customer needs to send them should come from them is absolutely and unacceptably shocking! A lot of people seem to have forgotten 2008. Mark.

On Friday, April 04, 2014 09:58:42 AM Vitkovský Adam wrote:

I wonder when (or if ever) we'll have such a discussion about data packets, i.e. finding that someone is not doing packet-filtering based on BGP updates is absolutely and unacceptably shocking!

Well, filtering in the data plane is slightly easier because a single subnet can cover all traffic coming from individual sources or going to individual destinations. In the control plane, the industry like to filter on specific prefixes agreed between customer and provider, especially when using automated tools such as RPSL. This can get hairy as configurations become large, where a single entry with "le 24" or "le 48" could have sufficed. On the other hand, if you're not automating control plane filters to some extent, it becomes messy as you get bigger. Mark.

note joels careful use of 'injected'. imiho, 'hijacked' is perjorative implying evil intent. i very much doubt that is the case here. it looks much more like an accident. could we try to be less accusatory with our language. 'injected', 'mis-originated', ... would seem to descrive the situation. and, btw, how many of those whose prefixes were mis-originated had registered those prefixes in the rpki? randy

Hi Team, Confirmation from my team talking directly to Indosat - self inflected with a bad update during a maintenance window. Nothing malicious or intentional. Barry

On 03/04/2014 13:09, ML wrote:

Did you get any details on what specifically went wrong? I don't recall any switch in my routing gear to "re-originate every prefix on the planet as my own".

Easy enough to do by e.g. redistributing your ebgp into your IGP and then back again, or by a variety of other means. It happened between 05:00 and 06:00 local time, so it's reasonable to assume that it was maintenance gone wrong. Horribly wrong. Nick

I wonder who we should be going after here? Indosat or their upstream? Probably both, since if this happened with an ISP deeper in the Internet core, chances are they don't have what our concept of an "upstream" is.

you want revenge or to prevent the effects of recurrence? one nice thing about origin validation is that anyone who validates anywhere on the internet can reject the mis-origination(s). randy

On Thu, Apr 3, 2014 at 9:15 AM, Mark Tinka <mark.tinka@seacom.mu> wrote:

On Thursday, April 03, 2014 02:51:20 PM Randy Bush wrote:

...

you want revenge or to prevent the effects of recurrence?

I'd like to consider targeted suggestions for fixes that address the specific challenges affecting "seasoned" upstreams vs. their downstream customers.

at this point it's hard to come up with a suggestion aside from: "stop being negligent" :( if after so many incidents and so many years, and seeing so many of your friends trip on the stairs and break an arm, you'd think providers would route-filter their customers just to avoid going to the hospital.

I can understand how an ISP with relatively little experience can mess this up (and glad to help here to educate wherever possible). But if an "established" provider is still struggling with this, why is that?

I'm going to guess: 1) who's going to pay for the filtering setup work? 2) we have always done it this way... why change? 3) adrenaline rush?

...

one nice thing about origin validation is that anyone who validates anywhere on the internet can reject the mis-origination(s).

+1.

Mark.

On Thu, Apr 3, 2014 at 11:05 AM, Mark Tinka <mark.tinka@seacom.mu> wrote:

On Thursday, April 03, 2014 03:55:11 PM Christopher Morrow wrote:

...

I'm going to guess: 1) who's going to pay for the filtering setup work?

Well, your customers are paying you to ensure they don't get cut off due to your negligence.

I think you mean they are paying me to carry their bits across the network... and they are paying me to do it with minimal hassle to THEM... telling me prefixes to add to their list is hassle.

You also don't want to become a "watch-out-for-that-one" peer within the community.

sure... not sure how much that matters to higher-ups? there's no such thing as bad PR, right?

But, perhaps those two ideals are not significant motivation for change :-\.

apparently they are not.

...

2) we have always done it this way... why change?

This is probably a more endemic issue of our industry, where operators find it hard to keep up with the times (there is no shortage of "-bis" or BCP documents) through actual useful implementation (BCP-38) vs. talk (SDN hype).

In the case of nailing routing filters for customers, one thought that comes to mind is if your organization is large enough, throw a warm body at the issue. There are lots of interns or folk you can hire on a temporary basis to focus on cleaning all this up, and getting the NOC trained and

there's a salient point about training time and internal systems complexity to keep in mind here as well :(

clued up on the new strategy. The new strategy is not just shiny, it could actually save you loss of customers and community respect.

agreed.

But that's just me...

it's not just you.

On Thu, Apr 3, 2014 at 11:13 AM, Christopher Morrow <morrowc.lists@gmail.com

wrote:

On Thu, Apr 3, 2014 at 11:05 AM, Mark Tinka <mark.tinka@seacom.mu> wrote:

...

On Thursday, April 03, 2014 03:55:11 PM Christopher Morrow wrote:

...

I'm going to guess: 1) who's going to pay for the filtering setup work?

Well, your customers are paying you to ensure they don't get cut off due to your negligence.

I think you mean they are paying me to carry their bits across the network... and they are paying me to do it with minimal hassle to THEM... telling me prefixes to add to their list is hassle.

I know this old saw and sales people will apply pressure to Ops if their customers balk at the extra overhead. The time is now to push back, hard, against that practice. I realize you know this, Chris but are trying to characterize the mindset.

...

The new strategy is not just shiny, it could actually save you loss of customers and community respect.

agreed.

...

But that's just me...

it's not just you

Yes, let's seize the bull by the horns. Tony

...

one nice thing about origin validation is that anyone who validates anywhere on the internet can reject the mis-origination(s). +1.

a non-op sec person who follows nanog in read-only mode pointed out in private email that this is a subtle difference from prefix filtering. in general, i can not prefix filter N hops away. randy

On Thursday, April 03, 2014 02:52:16 PM Anthony Williams wrote:

Was a specific Upstream at fault or several upstream providers? It appears they have 9 upstream links -- http://www.cidr-report.org/cgi-bin/as-report?as=4761

There probably won't be only one provider at fault. It could be all an ISP's providers are at fault, or it could be that two providers along a single AS_PATH are simultaneously at fault. It's a big weakness of our Internet, but we still need to figure out the best way to fix it, until, at least, RPKI is more widely adopted. At this stage, it appears education, and implementation of that education, is our only recourse. But how do we do this at scale? Mark.

On Thursday, April 03, 2014 02:57:31 PM Nick Hilliard wrote:

I'm currently seeing ~100 prefixes originating from 4761, and an additional 725 transited through 4761. This would not be difficult to handle with prefix lists, assuming some level of automation.

Indeed. I, for example, have an upstream that filters only on AS_PATH. Naturally, we are quite aggressive and insistent about filtering both on AS_PATH and prefix list across interconnects to our downstreams, but if things were to blow up on our side, the upstream in question would not be protected (unless, of course, they are relying on "max- prefix" as well). Mark.

It is probably a bit of a hammer at this stage, but we are in limited deployment of dropping all Invalids using RPKI.

We shall be rolling out, network-wide, in 2014, where all Invalids are dropped. At this stage, short of a mis- origination, it's mostly longer prefixes of an aggregate that are not ROA'd.

sadly, my (legacy) address space is in the arin region. and arin does not see allowing me to protect my prefixes from mis-origination as a serious goal. randy

On Tuesday, April 08, 2014 11:24:07 AM Jac Kloots wrote:

We (SURFnet, AS1103) are in the same position and I wrote an article about the evaluation we did before deciding on dropping invalids (https://blog.surfnet.nl/?p=3159)

Sounds great, Jac! In your report, you mention that you're not validating customer prefixes. Is this still the case? Mark.

On Tuesday, April 08, 2014 01:20:23 PM Jac Kloots wrote:

Yes, we don't validate those prefixes cause we filter them strict. We know from all our customers which prefixes they use so we have prefix-filters placed on all their connections.

Good point. We do both - prefix list + AS_PATH filtering as well as origin validation. At this point, you're likely to lose longer prefixes from customers if they forgot to ROA them, but the rationale is that if a customer has sufficient clue to ROA their aggregate, they can quickly ROA a de-aggregate or fix it in case they forgot. Mark.

On Thursday, April 10, 2014 09:18:34 AM Randy Bush wrote:

in our measurements, an rpki-based origin check is significantly faster than an acl of non-trivial length.

Ultimately, at some point in the future, it is not completely unreasonable to think that some operators could attempt control plane filtering based purely on RPKI-based origin and AS_PATH validation, without actually needing to configure prefix or AS_PATH lists :-). Wouldn't that be something... Mark.

On Thursday, April 10, 2014 12:30:51 PM Randy Bush wrote:

as folk start to roll out rejection of invalids, we might think about how we report problems with folk registering inadequate roas, covering their customers, covering their deaggs (maybe deaggs get what they deserve), etc. if they are not clued enough to generate prudent roas, they will not be clued enough to generate ghostbusters (and neither ripe's nor apnic's software supports gbrs today).

Agree. If you are clued enough to generate ROA's, you are clued enough to generate ROA's for the de-aggregates (or, at least, respond to the errors that indicate that). But the reverse is also true. It would be useful to use BGPmon's free RPKI validation feature, which e-mails you, incessantly, about validation failures due to un-ROA'd de-aggregates. It will also help if the CA's run by the RIR's support prefix length definitions. For the Africa region, AFRINIC currently do not, meaning every de-aggregate needs to be ROA'd. It's planned, though...

if my customer can not reach foo's customer, will foo's rir be willing to help? if foo's customer can not reach mine, how to let foo know who to call/write? do we need conventions?

This was one of the questions I've always pondered, and if you recall, was part of our panel discussion on the subject in Xi'an last year. I think it would be helpful if CA delegation was supported by RIR's, and supported well, so that customers can deal with their ISP's CA instead of having to deal with the RIR instead (particularly for situations where RIR's aren't 24/7 shops). On the monitoring side, it will be critical for ISP's to provide looking glasses that customers can use to verify the delta between what has been ROA'd and what has been announced/rejected, particularly in the case of un-ROA'd de- aggregates. Mark.

We have a registered prefix that was affected. The RPKI may have helped though; only one BGPMON peer saw the mis-originated route. Much better than being on the 10+ list. -----Original Message----- From: Randy Bush [mailto:randy@psg.com] Sent: Wednesday, April 02, 2014 7:23 PM To: North American Network Operators' Group Subject: Re: BGPMON Alert Questions note joels careful use of 'injected'. imiho, 'hijacked' is perjorative implying evil intent. i very much doubt that is the case here. it looks much more like an accident. could we try to be less accusatory with our language. 'injected', 'mis-originated', ... would seem to descrive the situation. and, btw, how many of those whose prefixes were mis-originated had registered those prefixes in the rpki? randy This message and any attachments should be treated as confidential information of Griffin Technology, Inc.

...

We've detected 415,652 prefixes being hijacked by Indosat today. Those who do not understand AS7007 are doomed to repeat it?

i very much doubt this is a 7007, where bgp was redistributed into rip, which sliced it into a jillion /24s, and then redistributed from rip back into bgp. of course the lack of filtering or origin validation is an endemic disease. randy

So we're somewhat safe until the fast food burger grills and fries cookers advance to level-3 routing? Or Daquiri blenders get their own ASNs?

that happened in the late '90s

Bad enough that "professional" folks can goof to this extent

luckily, you, valdis, and i never make mistakes :) the point it to engineer the network so we are not affected by the inevitable mistakes as chris and i were noting privately, this seems not to have damaged a lot of traffic, more than compensated for by the traffic on nanog :) randy

On 3 April 2014 04:43, Randy Bush <randy@psg.com> wrote:

i very much doubt this is a 7007, where bgp was redistributed into rip, which sliced it into a jillion /24s, and then redistributed from rip back into bgp.

​I could be wrong, but I thought AS7007 was nothing to do with RIP? http://www.merit.edu/mail.archives/nanog/1997-04/msg00444.html M​

This seems to be occurring to many, I have two of my prefixes being announced by the same AS's, and I have confirmation from several others who are seeing this as well. Chris -----Original Message----- From: Seth Mattinen [mailto:sethm@rollernet.us] Sent: Wednesday, April 02, 2014 12:03 PM To: nanog@nanog.org Subject: Re: BGPMON Alert Questions On 4/2/14, 11:51, Joseph Jenkins wrote:

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Same here for one of my /21s. Origin of AS4761 through AS4651. ~Seth

route-views4 /64.25.208.71 has seen updates that contains large amount of prefixes at time 1396464452 (04 / 02 / 14 @ 6:47:32pm UTC) with path [20225, 6939, 4761] full prefixes list: http://pastebin.com/Eu4ePgp4 is it normal for single update to contain such large amount NLRI info? On Wed, Apr 2, 2014 at 12:08 PM, Octavio Alvarez <alvarezp@alvarezp.ods.org>wrote:

...

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure

On 02/04/14 11:51, Joseph Jenkins wrote: they

...

are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24:

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

Same here. I got an alert for two prefixes. Same origin AS, same AS path for one of them: 18356 9931 4651 4761, but a different one for the other: 18356 38794 4651 4761.

yeah you're seeing the impact of a pretty broad prefix injection indosat's upstream filters seem to be working for the most part. On 4/2/14, 12:10 PM, Stephen Fulton wrote:

I'm seeing the same hijack of prefixes by multiple networks under my watch, at 18:40 UTC and 19:06 UTC.

-- Stephen

On 2014-04-02 2:51 PM, Joseph Jenkins wrote:

...

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure they are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24:

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

Same here: ==================================================================== Possible Prefix Hijack (Code: 10) ==================================================================== Your prefix: 132.206.0.0/16: Prefix Description: MCGILL-NET-132-206 Update time: 2014-04-02 20:11 (UTC) Detected by #peers: 1 Detected prefix: 132.206.0.0/16 Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID) Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH) ASpath: 18356 38794 4651 4761 Alert details: https://portal.bgpmon.net/alerts.php?details&alert_id=41664976 Mark as false alert: https://portal.bgpmon.net/fp.php?aid=41664976 ==================================================================== Possible Prefix Hijack (Code: 10) ==================================================================== Your prefix: 142.157.128.0/18: Prefix Description: McGill Update time: 2014-04-02 20:11 (UTC) Detected by #peers: 1 Detected prefix: 142.157.128.0/18 Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID) Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH) ASpath: 18356 9931 4651 4761 Alert details: https://portal.bgpmon.net/alerts.php?details&alert_id=41664977 Mark as false alert: https://portal.bgpmon.net/fp.php?aid=41664977 On Wed, Apr 2, 2014 at 3:21 PM, Felix Aronsson <felix@mrfriday.com> wrote:

Seeing the same here for a /21. This seems to have happened before with AS4761? See http://www.bgpmon.net/hijack-by-as4761-indosat-a-quick-report/from january 2011.

On Wed, Apr 2, 2014 at 8:51 PM, Joseph Jenkins <joe@breathe-underwater.com>wrote:

...

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure they are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24:

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

They are advertising one of /22 right now as well, Bret On 04/02/2014 04:21 PM, Bryan Tong wrote:

They have advertised all of ours now.

On Wed, Apr 2, 2014 at 2:16 PM, Bob Evans <bob@fiberinternetcenter.com>wrote:

...

Yes, I too have alerts for some of our prefixes from the same offending origin 4761

On Wednesday April 2nd 2014 at 19:59 UTC we detected a Origin AS Change event for your prefix (66.201.48.0/20 slash 20 bottom of nor cal) The detected prefix: 66.201.48.0/20, was announced by AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID) Alert description: Origin AS Change Detected Prefix: 66.201.48.0/20 Detected Origin AS: 4761 Expected Origin AS: 26803

Bob Evans CTO

...

So I setup BGPMON for my prefixes and got an alert about someone in Thailand announcing my prefix. Everything looks fine to me and I've checked a bunch of different Looking Glasses and everything announcing correctly.

I am assuming I should be contacting the provider about their misconfiguration and announcing my prefixes and get them to fix it. Any other recommendations?

Is there a way I can verify what they are announcing just to make sure they are still doing it?

Here is the alert for reference:

Your prefix: 8.37.93.0/24:

Update time: 2014-04-02 18:26 (UTC)

Detected by #peers: 2

Detected prefix: 8.37.93.0/24

Announced by: AS4761 (INDOSAT-INP-AP INDOSAT Internet Network Provider,ID)

Upstream AS: AS4651 (THAI-GATEWAY The Communications Authority of Thailand(CAT),TH)

ASpath: 18356 9931 4651 4761

-- Spectra Access 25 Lowell Street Manchester, NH 03042 603-296-0760 www.spectraaccess.net