As others have said, Google's abuse systems are smart enough to understand NAT and proxies, and won't block on request volume alone. When we automatically apply a block, we'll generally offer a captcha to give innocent users a workaround and limit the annoyance until the abuse stops and the block can expire This failed at our site. Our entire IPv4 and IPv6 addresse blocks received captcha after captcha after captcha, forever and ever. There was a link on the page to get more information, but all that got was another captcha. Normally I am 100% behind Google in everything, but sadly, this has now fallen to 99.8%. derek On Wed, May 21, 2014 at 10:42 PM, Damian Menscher <damian@google.com> wrote:
On Tue, May 20, 2014 at 7:21 AM, Pui Edylie <email@edylie.net> wrote:
May I know what is the best approach so that Google would not ban our Natted IP from time to time as it suspect it as a bot.
As others have said, Google's abuse systems are smart enough to understand NAT and proxies, and won't block on request volume alone. When we automatically apply a block, we'll generally offer a captcha to give innocent users a workaround and limit the annoyance until the abuse stops and the block can expire. While we do everything we can to limit the collateral damage, if your organization has an infected user spewing abuse, you need to take responsibility for your network.
IPv6 is the best long-term solution, as this will allow Google's abuse systems to distinguish between your users and block only those violating the ToS. Please give each user a distinct /64 (this seems obvious, but I've seen someone put all their users in the same /96).
If you can't deploy IPv6 yet, some other suggestions: - Put your users behind a proxy that adds the X-Forwarded-For header with the user's internal IP. Google's abuse systems use that header to limit blocking when possible. - Review your machines for signs of infection -- many blocks are triggered by botnets that are sending abuse. Another common cause is a browser extension that automatically sends requests. Finally, don't set up monitoring to test whether you're being blocked -- those automated monitoring requests are also a violation of the ToS and only increase the chance of being blocked. - If you have a proxy, test it to ensure it's not an "open" proxy. Open proxies are frequently abused, and will get blocked as a result. - Partitioning users across different IPs can help contain the collateral damage when one user's machine goes rogue. If you load-balance all users across all your IPs then it will likely just result in the entire pool being blocked.
Is there any official channel from Google which we could work with them for
resolution?
There's no official channel for working to resolve a blocking issue. Years of experience proves the abuse systems are very accurate (and constantly being improved) -- false positives are extremely rare. Despite this certainty, due to privacy concerns no evidence can be shared back to the ISP to point to the source of abuse. Since nothing can be shared except for times abuse was seen (which is rarely helpful due to lack of logging by the ISP), the response is generally just the suggestions listed above. The blocks will expire on their own once the abuse has been stopped.
Damian -- Damian Menscher :: Security Reliability Engineer :: Google
-- Copyright 2014 Derek Andrew (excluding quotations) +1 306 966 4808 Information and Communications Technology University of Saskatchewan Peterson 120; 54 Innovation Boulevard Saskatoon,Saskatchewan,Canada. S7N 2V3 Timezone GMT-6 Typed but not read.