Hi All, Over the past couple of days we have been seeing an exponential increase (about 200-fold) in the amount of UDP SIP Control traffic in our netflow data. The past 24 hours, for example, has shown a total of nearly 300 GB of this traffic incoming and over 400 GB outgoing -- this despite the fact that we do not host any SIP services ourselves, and currently to my knowledge, we have no hosting customers running any kind of SIP services. (Total RTP traffic for 24 hours is only in the region of 150 Kb -- so a vast inbalance between control and RTP) The local sources/destinations of the traffic are within our hosting space, but are spread across a wide range of hosts (i.e. nothing really related to a single or handful of hosts). Additionally over the past couple of days we have seen an increase of mails to our abuse desk for "brute force" attempts against a number of SIP services... possibly directly related to this traffic. Is anyone aware of a new variant or modus-operandi of botnets in circulation in the past couple of days which attempt to exploit SIP services? Has anyone else notice a significant increase in this kind of traffic? Thanks Leland
On Apr 10, 2009, at 4:45 PM, Leland E. Vandervort wrote:
UDP SIP Control traffic in our netflow data.
Have you grabbed some packets in order to ensure it's actually SIP, vs. something else on the same ports? If it really is SIP-related, this could be caused by botted hosts launching a SIP DDoS, or brute-forcing said SIP services in order to steal service for resale, DoS someone else via the service at layer-7 (i.e., call avallanche), sent VoIP spam, et. al. You may have botted hosts in your hosting space, as well as hosts being scanned as potential targets for exploitation. A quick search-engine query should reveal that this sort of thing has been going on for quite some time; I believe there were some convictions in NJ or somewhere else in the northeastern US within the last year or so. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // +852.9133.2844 mobile Our dreams are still big; it's just the future that got small. -- Jason Scott
Legally speaking, we can't "grab packets" in this sense without a specific validated complaint, court orders, and that kind of thing... So all we can do in the the absence of a specific complaint is in the context of our day to day traffic analysis from the netflow data to identify anomalies.. hence this one... (We have already taken action on a handful of known and identified cases of SIP brute-force attacks in recent days). Having said that, we have seen a vast increase in the amount of abuse complaints about SIP authentication brute force attacks in the past couple of days, which would tally with the traffic in general as being actual SIP-Control. The absence of associated RTP, however, leads me to believe that it's either scanning, exploits, or botnets, rather than legitimate SIP traffic. Based on what I've seen in the past couple of days, I am sure that it's as you mentioned, a SIP DDoS or brute-force attacks on SIP services... (circumstantial evidence that it's actually SIP related rather than something else on the same ports -- given the number of abuse complaints) I was simply wondering if this was an overall trend globally, or if it's simply a handful of bozos making life "fun" for the rest of us ;) Thanks Leland On Fri, 10 Apr 2009, Roland Dobbins wrote:
On Apr 10, 2009, at 4:45 PM, Leland E. Vandervort wrote:
UDP SIP Control traffic in our netflow data.
Have you grabbed some packets in order to ensure it's actually SIP, vs. something else on the same ports?
If it really is SIP-related, this could be caused by botted hosts launching a SIP DDoS, or brute-forcing said SIP services in order to steal service for resale, DoS someone else via the service at layer-7 (i.e., call avallanche), sent VoIP spam, et. al. You may have botted hosts in your hosting space, as well as hosts being scanned as potential targets for exploitation.
A quick search-engine query should reveal that this sort of thing has been going on for quite some time; I believe there were some convictions in NJ or somewhere else in the northeastern US within the last year or so.
----------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // +852.9133.2844 mobile
Our dreams are still big; it's just the future that got small.
-- Jason Scott
On Apr 10, 2009, at 5:32 PM, Leland E. Vandervort wrote:
legally speaking, we can't "grab packets" in this sense without a specific validated complaint, court orders, and that kind of thing...
IANAL, but I suggest you check again with your legal department - I doubt this is actually the case (your jurisdiction may vary, but in most Western nations, you can grab packets for diagnostic/ troubleshooting/forensics purposes). Obviously, follow your legal counsel's advice. That being said, I've heard various SPs in various jurisdictions around the world state that they were prohibited from capturing packets, when in fact this wasn't true at all, they'd been misinformed. So, you may wish to check in order to be sure of your position.
So all we can do in the the absence of a specific complaint
But you said you *had* specific complaints, did you not? ;> ----------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // +852.9133.2844 mobile Our dreams are still big; it's just the future that got small. -- Jason Scott
On Fri, 10 Apr 2009, Roland Dobbins wrote:
IANAL, but I suggest you check again with your legal department - I doubt this is actually the case (your jurisdiction may vary, but in most Western nations, you can grab packets for diagnostic/ troubleshooting/forensics purposes).
Already did check... we can't grab packets except in response to judicial order or specific abuse case with a valid ID of the end-user, or of course for general technical diagnostics -- if for diagnostics, we cannot use such collected data in the context of only a suspicion of abuse at all as it would constitute an infringement on the individual's privacy. So in short, we can do it REACTIVELY in response to a complaint.. but if we do it PROACTIVELY, then it cannot be used and is of "educational" value only (with caveats surrounding confidentiality, non-disclosure, and destruction,, etc.)
So all we can do in the the absence of a specific complaint
But you said you *had* specific complaints, did you not?
yes.. *specific* and action was taken on those *specific* cases... (didn't actually have to grab traffic though...) L.
to answer your question, as opposed to telling you how to run your business, yes. we are seeing a low level, distributed source, sip probing across a wide swath of target space. it goes back a long time. randy
On Fri, 10 Apr 2009 10:20:35 +0000 (GMT) "Leland E. Vandervort" <leland@taranta.discpro.org> wrote:
On Fri, 10 Apr 2009, Roland Dobbins wrote:
IANAL, but I suggest you check again with your legal department - I doubt this is actually the case (your jurisdiction may vary, but in most Western nations, you can grab packets for diagnostic/ troubleshooting/forensics purposes).
Already did check... we can't grab packets except in response to judicial order or specific abuse case with a valid ID of the end-user, or of course for general technical diagnostics -- if for diagnostics, we cannot use such collected data in the context of only a suspicion of abuse at all as it would constitute an infringement on the individual's privacy. So in short, we can do it REACTIVELY in response to a complaint.. but if we do it PROACTIVELY, then it cannot be used and is of "educational" value only (with caveats surrounding confidentiality, non-disclosure, and destruction,, etc.)
You can if it the volume is interfering with your own service, I believe (though IANAL, either) -- see this text from http://www4.law.cornell.edu/uscode/18/2511.html It shall not be unlawful under this chapter for an operator of a switchboard, or an officer, employee, or agent of a provider of wire or electronic communication service, whose facilities are used in the transmission of a wire or electronic communication, to intercept, disclose, or use that communication in the normal course of his employment while engaged in any activity which is a necessary incident to the rendition of his service or to the protection of the rights or property of the provider of that service, except that a provider of wire communication service to the public shall not utilize service observing or random monitoring except for mechanical or service quality control checks. Note carefully that the second part applies to a "provider of wire communication service", which is a phone company, not an ISP -- ISPs are providers of "electronic communication service". (Just to make life fun -- if you're a VoIP *provider*, you probably fall under both sections, but if you're just carrying VoIP traffic I don't think you are). --Steve Bellovin, http://www.cs.columbia.edu/~smb
The timing of your email as well as a couple of seemingly unrelated things that I have heard about make me think this might be related to some large toll fraud scheme. Today I heard from someone who says Verizon is telling them they see about 700 calls per hour to Cuba originating from their PRI. Obviously some type of toll fraud. Got me thinking about this persons phone system and how there has always been the issue of toll fraud where someone calls in and knows how to get an outbound call routed through a poorly setup PBX. However the rate of 700 calls per hour and one PRI just don't make sense or add up in a situation like the old toll fraud method mentioned earlier since I believe that's more of a manual attack. That's when I recalled this post of yours. Made me wonder if there was some way to exploit SIP to associate with a VoIP PBX or gateway or something that was tied to PRI's and thus route your calls over someones phone system. Sure enough found some discussions and posts regarding toll fraud to Cuba (and others) in relation to SIP. For instance, Cisco's CallManager Express device which is a router as well as voip pbx is often tied to PSTN or PRI's and by default allows H323 TCP/1720 and SIP UDP/5060 ports open by default. It may seem obvious to others but new to me that these scans are related to someone or some group looking to find devices with these ports open in an effort to attach to them through SIP and hopefully exploit if attached to PRI's or PSTN for toll fraud. I really do learn something new everyday, some smart deviant people out there. On Fri, Apr 10, 2009 at 3:45 AM, Leland E. Vandervort <leland@taranta.discpro.org> wrote:
Hi All,
Over the past couple of days we have been seeing an exponential increase (about 200-fold) in the amount of UDP SIP Control traffic in our netflow data. The past 24 hours, for example, has shown a total of nearly 300 GB of this traffic incoming and over 400 GB outgoing -- this despite the fact that we do not host any SIP services ourselves, and currently to my knowledge, we have no hosting customers running any kind of SIP services. (Total RTP traffic for 24 hours is only in the region of 150 Kb -- so a vast inbalance between control and RTP)
The local sources/destinations of the traffic are within our hosting space, but are spread across a wide range of hosts (i.e. nothing really related to a single or handful of hosts).
Additionally over the past couple of days we have seen an increase of mails to our abuse desk for "brute force" attempts against a number of SIP services... possibly directly related to this traffic.
Is anyone aware of a new variant or modus-operandi of botnets in circulation in the past couple of days which attempt to exploit SIP services? Has anyone else notice a significant increase in this kind of traffic?
Thanks
Leland
Managed to get to the bottom of it, and it was indeed a SIP User-Agent brute-force attempt. Interestingly, though, that your mail mentions specifically verizon... the majority of the remote addresses during this brute-force attempt were also behind verizon... coincidence? Hmm.. Regards, Leland On Wed, 15 Apr 2009, Dane wrote:
The timing of your email as well as a couple of seemingly unrelated things that I have heard about make me think this might be related to some large toll fraud scheme.
Today I heard from someone who says Verizon is telling them they see about 700 calls per hour to Cuba originating from their PRI.
Obviously some type of toll fraud. Got me thinking about this persons phone system and how there has always been the issue of toll fraud where someone calls in and knows how to get an outbound call routed through a poorly setup PBX.
However the rate of 700 calls per hour and one PRI just don't make sense or add up in a situation like the old toll fraud method mentioned earlier since I believe that's more of a manual attack.
That's when I recalled this post of yours. Made me wonder if there was some way to exploit SIP to associate with a VoIP PBX or gateway or something that was tied to PRI's and thus route your calls over someones phone system.
Sure enough found some discussions and posts regarding toll fraud to Cuba (and others) in relation to SIP.
For instance, Cisco's CallManager Express device which is a router as well as voip pbx is often tied to PSTN or PRI's and by default allows H323 TCP/1720 and SIP UDP/5060 ports open by default.
It may seem obvious to others but new to me that these scans are related to someone or some group looking to find devices with these ports open in an effort to attach to them through SIP and hopefully exploit if attached to PRI's or PSTN for toll fraud.
I really do learn something new everyday, some smart deviant people out there.
On Fri, Apr 10, 2009 at 3:45 AM, Leland E. Vandervort <leland@taranta.discpro.org> wrote:
Hi All,
Over the past couple of days we have been seeing an exponential increase (about 200-fold) in the amount of UDP SIP Control traffic in our netflow data. �The past 24 hours, for example, has shown a total of nearly 300 GB of this traffic incoming and over 400 GB outgoing -- this despite the fact that we do not host any SIP services ourselves, and currently to my knowledge, we have no hosting customers running any kind of SIP services. �(Total RTP traffic for 24 hours is only in the region of 150 Kb -- so a vast inbalance between control and RTP)
The local sources/destinations of the traffic are within our hosting space, but are spread across a wide range of hosts (i.e. nothing really related to a single or handful of hosts).
Additionally over the past couple of days we have seen an increase of mails to our abuse desk for "brute force" attempts against a number of SIP services... possibly directly related to this traffic.
Is anyone aware of a new variant or modus-operandi of botnets in circulation in the past couple of days which attempt to exploit SIP services? �Has anyone else notice a significant increase in this kind of traffic?
Thanks
Leland
The timing of your email as well as a couple of seemingly unrelated things that I have heard about make me think this might be related to some large toll fraud scheme.
Today I heard from someone who says Verizon is telling them they see about 700 calls per hour to Cuba originating from their PRI.
Obviously some type of toll fraud. Got me thinking about this persons phone system and how there has always been the issue of toll fraud where someone calls in and knows how to get an outbound call routed through a poorly setup PBX.
However the rate of 700 calls per hour and one PRI just don't make sense or add up in a situation like the old toll fraud method mentioned earlier since I believe that's more of a manual attack.
That's when I recalled this post of yours. Made me wonder if there was some way to exploit SIP to associate with a VoIP PBX or gateway or something that was tied to PRI's and thus route your calls over someones phone system.
Sure enough found some discussions and posts regarding toll fraud to Cuba (and others) in relation to SIP.
For instance, Cisco's CallManager Express device which is a router as well as voip pbx is often tied to PSTN or PRI's and by default allows H323 TCP/1720 and SIP UDP/5060 ports open by default.
It may seem obvious to others but new to me that these scans are related to someone or some group looking to find devices with these ports open in an effort to attach to them through SIP and hopefully exploit if attached to PRI's or PSTN for toll fraud.
I really do learn something new everyday, some smart deviant people out
ACL's at the perimeter and/or on the gateways might help Thanks, Mike Goldman -----Original Message----- From: Leland E. Vandervort [mailto:leland@taranta.discpro.org] Sent: Wednesday, April 15, 2009 11:39 AM To: Dane Cc: nanog@nanog.org Subject: Re: SIP - perhaps botnet? anyone else seeing this? Managed to get to the bottom of it, and it was indeed a SIP User-Agent brute-force attempt. Interestingly, though, that your mail mentions specifically verizon... the majority of the remote addresses during this brute-force attempt were also behind verizon... coincidence? Hmm.. Regards, Leland On Wed, 15 Apr 2009, Dane wrote: there.
On Fri, Apr 10, 2009 at 3:45 AM, Leland E. Vandervort <leland@taranta.discpro.org> wrote:
Hi All,
Over the past couple of days we have been seeing an exponential increase (about 200-fold) in the amount of UDP SIP Control traffic in our netflow data. The past
24
hours, for example, has shown a total of nearly 300 GB of this traffic incoming and over 400 GB outgoing -- this despite the fact that we do not host any SIP services ourselves, and currently to my knowledge, we have no hosting customers running any kind of SIP services. (Total RTP traffic for 24 hours is only in the region of 150 Kb -- so a vast inbalance between control and RTP)
The local sources/destinations of the traffic are within our hosting space, but are spread across a wide range of hosts (i.e. nothing really related to a single or handful of hosts).
Additionally over the past couple of days we have seen an increase of mails to our abuse desk for "brute force" attempts against a number of SIP services... possibly directly related to this traffic.
Is anyone aware of a new variant or modus-operandi of botnets in circulation in the past couple of days which attempt to exploit SIP services? Has anyone else notice a significant increase in this kind of traffic?
Thanks
Leland
Leland E. Vandervort wrote:
Managed to get to the bottom of it, and it was indeed a SIP User-Agent brute-force attempt. Interestingly, though, that your mail mentions specifically verizon... the majority of the remote addresses during this brute-force attempt were also behind verizon... coincidence?
Hmm..
There are at least two projects I'm aware of and some tools released/getting released working on war-dialing over SIP. One tool to take a look at and see if it fits the bill is WarVOX from Metasploit's HD Moore. http://www.warvox.org/index.html Gadi.
On Wed, Apr 15, 2009 at 11:35:43AM -0500, Dane wrote:
Today I heard from someone who says Verizon is telling them they see about 700 calls per hour to Cuba originating from their PRI. Obviously some type of toll fraud.
In the same way that it's possible to configure a mail relay as a device that forwards mail between unintended parties, it is possible to configure a SIP proxy as a device that causes calls to be forwarded between unintended parties too. Likewise, in the same way that spammers scan network ranges for these misconfigured mail gateways, thieves look for unsecured SIP gateways to relay calls through. The SIP traffic mentioned at the start of this thread doesn't follow the pattern of this constant background noise. Kind regards, Andy
participants (8)
-
Andy Davidson
-
Dane
-
Gadi Evron
-
Leland E. Vandervort
-
Mike Goldman
-
Randy Bush
-
Roland Dobbins
-
Steven M. Bellovin