Google Webtools reports a problem with our HOMEPAGE "/". That page is not redirecting anywhere. They also report problems with some 48 other primary sites, none of which redirect to the offending couchtarts. matthew black information technology services california state university, long beach -----Original Message----- From: Jeremy Hanmer [mailto:jeremy.hanmer@dreamhost.com] Sent: Tuesday, June 26, 2012 9:58 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google? It's not DNS. If you're sure there's no htaccess files in place, check your content (even that stored in a database) for anything that might be altering data based on referrer. This simple test shows what I mean: Airy:~ user$ curl -e 'http://google.com' csulb.edu <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php">here</a>.</p> </body></html> Running curl without the -e argument gives the proper site contents. On Jun 26, 2012, at 9:24 PM, Matthew Black <Matthew.Black@csulb.edu> wrote:
Running Apache on three Solaris webservers behind a load balancer. No MS Windows!
Not sure how malicious software could get between our load balancer and Unix servers. Thanks for the tip!
matthew black information technology services california state university, long beach
From: Landon Stewart [mailto:lstewart@superb.net] Sent: Tuesday, June 26, 2012 9:07 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
Is it possible that some malicious software is listening and injecting a redirect on the wire? We've seen this before with a Windows machine being infected. On 26 June 2012 20:53, Matthew Black <Matthew.Black@csulb.edu<mailto:Matthew.Black@csulb.edu>> wrote: Google Safe Browsing and Firefox have marked our website as containing malware. They claim our home page returns no results, but redirects users to another compromised website couchtarts.com<http://couchtarts.com>.
We have thoroughly examined our root .htaccess and httpd.conf files and are not redirecting to the problem target site. No recent changes either.
We ran some NSLOOKUPs against various public DNS servers and intermittently get results that are NOT our servers.
We believe the DNS servers used by Google's crawler have been poisoned.
Can anyone shed some light on this?
matthew black information technology services california state university, long beach www.csulb.edu<http://www.csulb.edu><http://www.csulb.edu>
-- Landon Stewart <LStewart@Superb.Net<mailto:LStewart@Superb.Net>> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199 Web hosting and more "Ahead of the Rest": http://www.superbhosting.net<http://www.superbhosting.net/>
On 06/26/2012 11:05 PM, Matthew Black wrote:
Google Webtools reports a problem with our HOMEPAGE "/". That page is not redirecting anywhere. They also report problems with some 48 other primary sites, none of which redirect to the offending couchtarts.
Except it is redirecting as shown by Jeremy: [guppy@mrlaptop ~]$ curl -e 'http://google.com' csulb.edu <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php">here</a>.</p> </body></html> That looks like a redirect to me. Jeff
Invoking the referrer on your site recommends a redirect to couchtarts. I agree with Jeremy and Jeff check your htaccess files, conf files and anything that calls RewriteCond or Rewrite On Wed, Jun 27, 2012 at 12:05 AM, Matthew Black <Matthew.Black@csulb.edu>wrote:
Google Webtools reports a problem with our HOMEPAGE "/". That page is not redirecting anywhere. They also report problems with some 48 other primary sites, none of which redirect to the offending couchtarts.
matthew black information technology services california state university, long beach
-----Original Message----- From: Jeremy Hanmer [mailto:jeremy.hanmer@dreamhost.com] Sent: Tuesday, June 26, 2012 9:58 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
It's not DNS. If you're sure there's no htaccess files in place, check your content (even that stored in a database) for anything that might be altering data based on referrer. This simple test shows what I mean:
Airy:~ user$ curl -e 'http://google.com' csulb.edu <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php ">here</a>.</p> </body></html>
Running curl without the -e argument gives the proper site contents.
On Jun 26, 2012, at 9:24 PM, Matthew Black <Matthew.Black@csulb.edu> wrote:
Running Apache on three Solaris webservers behind a load balancer. No MS Windows!
Not sure how malicious software could get between our load balancer and Unix servers. Thanks for the tip!
matthew black information technology services california state university, long beach
From: Landon Stewart [mailto:lstewart@superb.net] Sent: Tuesday, June 26, 2012 9:07 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
Is it possible that some malicious software is listening and injecting a redirect on the wire? We've seen this before with a Windows machine being infected. On 26 June 2012 20:53, Matthew Black <Matthew.Black@csulb.edu<mailto: Matthew.Black@csulb.edu>> wrote: Google Safe Browsing and Firefox have marked our website as containing malware. They claim our home page returns no results, but redirects users to another compromised website couchtarts.com<http://couchtarts.com>.
We have thoroughly examined our root .htaccess and httpd.conf files and are not redirecting to the problem target site. No recent changes either.
We ran some NSLOOKUPs against various public DNS servers and intermittently get results that are NOT our servers.
We believe the DNS servers used by Google's crawler have been poisoned.
Can anyone shed some light on this?
matthew black information technology services california state university, long beach www.csulb.edu<http://www.csulb.edu><http://www.csulb.edu>
-- Landon Stewart <LStewart@Superb.Net<mailto:LStewart@Superb.Net>> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199 Web hosting and more "Ahead of the Rest": http://www.superbhosting.net<http://www.superbhosting.net/>
for example, from the commandline with telnet: morrowc@teensy:~$ telnet www.csulb.edu 80 Trying 134.139.1.60... Connected to gaggle.its.csulb.edu. Escape character is '^]'. GET / HTTP/1.0 Host: www.csulb.edu Referer: http://www.google.com/ HTTP/1.1 301 Moved Permanently Date: Wed, 27 Jun 2012 05:04:04 GMT Server: Apache/2.0.63 Location: http://www.couchtarts.com/media.php Content-Length: 243 Connection: close Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php">here</a>.</p> </body></html> Connection closed by foreign host. oops :( fail. On Wed, Jun 27, 2012 at 1:13 AM, Ishmael Rufus <sakamura@gmail.com> wrote:
Invoking the referrer on your site recommends a redirect to couchtarts. I agree with Jeremy and Jeff check your htaccess files, conf files and anything that calls RewriteCond or Rewrite
On Wed, Jun 27, 2012 at 12:05 AM, Matthew Black <Matthew.Black@csulb.edu>wrote:
Google Webtools reports a problem with our HOMEPAGE "/". That page is not redirecting anywhere. They also report problems with some 48 other primary sites, none of which redirect to the offending couchtarts.
matthew black information technology services california state university, long beach
-----Original Message----- From: Jeremy Hanmer [mailto:jeremy.hanmer@dreamhost.com] Sent: Tuesday, June 26, 2012 9:58 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
It's not DNS. If you're sure there's no htaccess files in place, check your content (even that stored in a database) for anything that might be altering data based on referrer. This simple test shows what I mean:
Airy:~ user$ curl -e 'http://google.com' csulb.edu <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php ">here</a>.</p> </body></html>
Running curl without the -e argument gives the proper site contents.
On Jun 26, 2012, at 9:24 PM, Matthew Black <Matthew.Black@csulb.edu> wrote:
Running Apache on three Solaris webservers behind a load balancer. No MS Windows!
Not sure how malicious software could get between our load balancer and Unix servers. Thanks for the tip!
matthew black information technology services california state university, long beach
From: Landon Stewart [mailto:lstewart@superb.net] Sent: Tuesday, June 26, 2012 9:07 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
Is it possible that some malicious software is listening and injecting a redirect on the wire? We've seen this before with a Windows machine being infected. On 26 June 2012 20:53, Matthew Black <Matthew.Black@csulb.edu<mailto: Matthew.Black@csulb.edu>> wrote: Google Safe Browsing and Firefox have marked our website as containing malware. They claim our home page returns no results, but redirects users to another compromised website couchtarts.com<http://couchtarts.com>.
We have thoroughly examined our root .htaccess and httpd.conf files and are not redirecting to the problem target site. No recent changes either.
We ran some NSLOOKUPs against various public DNS servers and intermittently get results that are NOT our servers.
We believe the DNS servers used by Google's crawler have been poisoned.
Can anyone shed some light on this?
matthew black information technology services california state university, long beach www.csulb.edu<http://www.csulb.edu><http://www.csulb.edu>
-- Landon Stewart <LStewart@Superb.Net<mailto:LStewart@Superb.Net>> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199 Web hosting and more "Ahead of the Rest": http://www.superbhosting.net<http://www.superbhosting.net/>
Thank you for that helpful instruction! curl doesn't work because our webserver is firewalled against outbound traffic. The telnet to port 80 showed me the problem. I also didn't understand when output was placed at the end of the command line, instead of starting on the next line...that looked like something I was supposed to type. matthew black information technology services california state university, long beac -----Original Message----- From: christopher.morrow@gmail.com [mailto:christopher.morrow@gmail.com] On Behalf Of Christopher Morrow Sent: Tuesday, June 26, 2012 10:17 PM To: Ishmael Rufus Cc: Matthew Black; nanog@nanog.org; Jeremy Hanmer Subject: Re: DNS poisoning at Google? for example, from the commandline with telnet: morrowc@teensy:~$ telnet www.csulb.edu 80 Trying 134.139.1.60... Connected to gaggle.its.csulb.edu. Escape character is '^]'. GET / HTTP/1.0 Host: www.csulb.edu Referer: http://www.google.com/ HTTP/1.1 301 Moved Permanently Date: Wed, 27 Jun 2012 05:04:04 GMT Server: Apache/2.0.63 Location: http://www.couchtarts.com/media.php Content-Length: 243 Connection: close Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php">here</a>.</p> </body></html> Connection closed by foreign host. oops :( fail. On Wed, Jun 27, 2012 at 1:13 AM, Ishmael Rufus <sakamura@gmail.com> wrote:
Invoking the referrer on your site recommends a redirect to couchtarts. I agree with Jeremy and Jeff check your htaccess files, conf files and anything that calls RewriteCond or Rewrite
On Wed, Jun 27, 2012 at 12:05 AM, Matthew Black <Matthew.Black@csulb.edu>wrote:
Google Webtools reports a problem with our HOMEPAGE "/". That page is not redirecting anywhere. They also report problems with some 48 other primary sites, none of which redirect to the offending couchtarts.
matthew black information technology services california state university, long beach
-----Original Message----- From: Jeremy Hanmer [mailto:jeremy.hanmer@dreamhost.com] Sent: Tuesday, June 26, 2012 9:58 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
It's not DNS. If you're sure there's no htaccess files in place, check your content (even that stored in a database) for anything that might be altering data based on referrer. This simple test shows what I mean:
Airy:~ user$ curl -e 'http://google.com' csulb.edu <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php ">here</a>.</p> </body></html>
Running curl without the -e argument gives the proper site contents.
On Jun 26, 2012, at 9:24 PM, Matthew Black <Matthew.Black@csulb.edu> wrote:
Running Apache on three Solaris webservers behind a load balancer. No MS Windows!
Not sure how malicious software could get between our load balancer and Unix servers. Thanks for the tip!
matthew black information technology services california state university, long beach
From: Landon Stewart [mailto:lstewart@superb.net] Sent: Tuesday, June 26, 2012 9:07 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
Is it possible that some malicious software is listening and injecting a redirect on the wire? We've seen this before with a Windows machine being infected. On 26 June 2012 20:53, Matthew Black <Matthew.Black@csulb.edu<mailto: Matthew.Black@csulb.edu>> wrote: Google Safe Browsing and Firefox have marked our website as containing malware. They claim our home page returns no results, but redirects users to another compromised website couchtarts.com<http://couchtarts.com>.
We have thoroughly examined our root .htaccess and httpd.conf files and are not redirecting to the problem target site. No recent changes either.
We ran some NSLOOKUPs against various public DNS servers and intermittently get results that are NOT our servers.
We believe the DNS servers used by Google's crawler have been poisoned.
Can anyone shed some light on this?
matthew black information technology services california state university, long beach www.csulb.edu<http://www.csulb.edu><http://www.csulb.edu>
-- Landon Stewart <LStewart@Superb.Net<mailto:LStewart@Superb.Net>> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199 Web hosting and more "Ahead of the Rest": http://www.superbhosting.net<http://www.superbhosting.net/>
On Wed, Jun 27, 2012 at 1:26 AM, Matthew Black <Matthew.Black@csulb.edu> wrote:
Thank you for that helpful instruction!
curl doesn't work because our webserver is firewalled against outbound traffic. The telnet to port 80 showed me the problem. I also didn't understand when output was placed at the end of the command line, instead of starting on the next line...that looked like something I was supposed to type.
sorry... often when I end up testing something like this I cut/paste from a buffer, so: telnet bloop 80 <paste> <return/return/return> read-output... In the case of your server: GET / HTTP/1.0 Host: www.csulb.edu Referer: http://www.google.com/ <empty-line!!> all gets pasted once the 'telnet www.csulb.edu 80' connects... the output is the stuff that includes the 'redirect to couchtarts'. -chris
matthew black information technology services california state university, long beac
-----Original Message----- From: christopher.morrow@gmail.com [mailto:christopher.morrow@gmail.com] On Behalf Of Christopher Morrow Sent: Tuesday, June 26, 2012 10:17 PM To: Ishmael Rufus Cc: Matthew Black; nanog@nanog.org; Jeremy Hanmer Subject: Re: DNS poisoning at Google?
for example, from the commandline with telnet:
morrowc@teensy:~$ telnet www.csulb.edu 80 Trying 134.139.1.60... Connected to gaggle.its.csulb.edu. Escape character is '^]'. GET / HTTP/1.0 Host: www.csulb.edu Referer: http://www.google.com/
HTTP/1.1 301 Moved Permanently Date: Wed, 27 Jun 2012 05:04:04 GMT Server: Apache/2.0.63 Location: http://www.couchtarts.com/media.php Content-Length: 243 Connection: close Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php">here</a>.</p> </body></html> Connection closed by foreign host.
oops :( fail.
On Wed, Jun 27, 2012 at 1:13 AM, Ishmael Rufus <sakamura@gmail.com> wrote:
Invoking the referrer on your site recommends a redirect to couchtarts. I agree with Jeremy and Jeff check your htaccess files, conf files and anything that calls RewriteCond or Rewrite
On Wed, Jun 27, 2012 at 12:05 AM, Matthew Black <Matthew.Black@csulb.edu>wrote:
Google Webtools reports a problem with our HOMEPAGE "/". That page is not redirecting anywhere. They also report problems with some 48 other primary sites, none of which redirect to the offending couchtarts.
matthew black information technology services california state university, long beach
-----Original Message----- From: Jeremy Hanmer [mailto:jeremy.hanmer@dreamhost.com] Sent: Tuesday, June 26, 2012 9:58 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
It's not DNS. If you're sure there's no htaccess files in place, check your content (even that stored in a database) for anything that might be altering data based on referrer. This simple test shows what I mean:
Airy:~ user$ curl -e 'http://google.com' csulb.edu <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php ">here</a>.</p> </body></html>
Running curl without the -e argument gives the proper site contents.
On Jun 26, 2012, at 9:24 PM, Matthew Black <Matthew.Black@csulb.edu> wrote:
Running Apache on three Solaris webservers behind a load balancer. No MS Windows!
Not sure how malicious software could get between our load balancer and Unix servers. Thanks for the tip!
matthew black information technology services california state university, long beach
From: Landon Stewart [mailto:lstewart@superb.net] Sent: Tuesday, June 26, 2012 9:07 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
Is it possible that some malicious software is listening and injecting a redirect on the wire? We've seen this before with a Windows machine being infected. On 26 June 2012 20:53, Matthew Black <Matthew.Black@csulb.edu<mailto: Matthew.Black@csulb.edu>> wrote: Google Safe Browsing and Firefox have marked our website as containing malware. They claim our home page returns no results, but redirects users to another compromised website couchtarts.com<http://couchtarts.com>.
We have thoroughly examined our root .htaccess and httpd.conf files and are not redirecting to the problem target site. No recent changes either.
We ran some NSLOOKUPs against various public DNS servers and intermittently get results that are NOT our servers.
We believe the DNS servers used by Google's crawler have been poisoned.
Can anyone shed some light on this?
matthew black information technology services california state university, long beach www.csulb.edu<http://www.csulb.edu><http://www.csulb.edu>
-- Landon Stewart <LStewart@Superb.Net<mailto:LStewart@Superb.Net>> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199 Web hosting and more "Ahead of the Rest": http://www.superbhosting.net<http://www.superbhosting.net/>
There is definitely a 301 redirect. $ curl -I --referer http://www.google.com/ http://www.csulb.edu/ HTTP/1.1 301 Moved Permanently Date: Wed, 27 Jun 2012 05:36:31 GMT Server: Apache/2.0.63 Location: http://www.couchtarts.com/media.php Connection: close Content-Type: text/html; charset=iso-8859-1 On 26 June 2012 22:05, Matthew Black <Matthew.Black@csulb.edu> wrote:
Google Webtools reports a problem with our HOMEPAGE "/". That page is not redirecting anywhere. They also report problems with some 48 other primary sites, none of which redirect to the offending couchtarts.
matthew black information technology services california state university, long beach
-----Original Message----- From: Jeremy Hanmer [mailto:jeremy.hanmer@dreamhost.com] Sent: Tuesday, June 26, 2012 9:58 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
It's not DNS. If you're sure there's no htaccess files in place, check your content (even that stored in a database) for anything that might be altering data based on referrer. This simple test shows what I mean:
Airy:~ user$ curl -e 'http://google.com' csulb.edu <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php ">here</a>.</p> </body></html>
Running curl without the -e argument gives the proper site contents.
On Jun 26, 2012, at 9:24 PM, Matthew Black <Matthew.Black@csulb.edu> wrote:
Running Apache on three Solaris webservers behind a load balancer. No MS Windows!
Not sure how malicious software could get between our load balancer and Unix servers. Thanks for the tip!
matthew black information technology services california state university, long beach
From: Landon Stewart [mailto:lstewart@superb.net] Sent: Tuesday, June 26, 2012 9:07 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
Is it possible that some malicious software is listening and injecting a redirect on the wire? We've seen this before with a Windows machine being infected. On 26 June 2012 20:53, Matthew Black <Matthew.Black@csulb.edu<mailto: Matthew.Black@csulb.edu>> wrote: Google Safe Browsing and Firefox have marked our website as containing malware. They claim our home page returns no results, but redirects users to another compromised website couchtarts.com<http://couchtarts.com>.
We have thoroughly examined our root .htaccess and httpd.conf files and are not redirecting to the problem target site. No recent changes either.
We ran some NSLOOKUPs against various public DNS servers and intermittently get results that are NOT our servers.
We believe the DNS servers used by Google's crawler have been poisoned.
Can anyone shed some light on this?
matthew black information technology services california state university, long beach www.csulb.edu<http://www.csulb.edu><http://www.csulb.edu>
-- Landon Stewart <LStewart@Superb.Net<mailto:LStewart@Superb.Net>> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199 Web hosting and more "Ahead of the Rest": http://www.superbhosting.net<http://www.superbhosting.net/>
-- Landon Stewart <LStewart@Superb.Net> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199 Web hosting and more "Ahead of the Rest": http://www.superbhosting.net
Thanks again to everyone who helped. I didn't know what to enter with curl, because Outlook clobbered the line breaks in Jeremy's original message. Also, curl failed on our primary webserver because of firewall and load balancer magic settings. The Telnet method worked better! Our team is now scouring for that hidden redirect to couchtarts. matthew black information technology services california state university, long beach From: Landon Stewart [mailto:lstewart@superb.net] Sent: Tuesday, June 26, 2012 10:37 PM To: Matthew Black Cc: Jeremy Hanmer; nanog@nanog.org Subject: Re: DNS poisoning at Google? There is definitely a 301 redirect. $ curl -I --referer http://www.google.com/ http://www.csulb.edu/ HTTP/1.1 301 Moved Permanently Date: Wed, 27 Jun 2012 05:36:31 GMT Server: Apache/2.0.63 Location: http://www.couchtarts.com/media.php Connection: close Content-Type: text/html; charset=iso-8859-1 On 26 June 2012 22:05, Matthew Black <Matthew.Black@csulb.edu<mailto:Matthew.Black@csulb.edu>> wrote: Google Webtools reports a problem with our HOMEPAGE "/". That page is not redirecting anywhere. They also report problems with some 48 other primary sites, none of which redirect to the offending couchtarts. matthew black information technology services california state university, long beach -----Original Message----- From: Jeremy Hanmer [mailto:jeremy.hanmer@dreamhost.com<mailto:jeremy.hanmer@dreamhost.com>] Sent: Tuesday, June 26, 2012 9:58 PM To: Matthew Black Cc: nanog@nanog.org<mailto:nanog@nanog.org> Subject: Re: DNS poisoning at Google? It's not DNS. If you're sure there's no htaccess files in place, check your content (even that stored in a database) for anything that might be altering data based on referrer. This simple test shows what I mean: Airy:~ user$ curl -e 'http://google.com' csulb.edu<http://csulb.edu> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php">here</a>.</p> </body></html> Running curl without the -e argument gives the proper site contents. On Jun 26, 2012, at 9:24 PM, Matthew Black <Matthew.Black@csulb.edu<mailto:Matthew.Black@csulb.edu>> wrote:
Running Apache on three Solaris webservers behind a load balancer. No MS Windows!
Not sure how malicious software could get between our load balancer and Unix servers. Thanks for the tip!
matthew black information technology services california state university, long beach
From: Landon Stewart [mailto:lstewart@superb.net<mailto:lstewart@superb.net>] Sent: Tuesday, June 26, 2012 9:07 PM To: Matthew Black Cc: nanog@nanog.org<mailto:nanog@nanog.org> Subject: Re: DNS poisoning at Google?
Is it possible that some malicious software is listening and injecting a redirect on the wire? We've seen this before with a Windows machine being infected. On 26 June 2012 20:53, Matthew Black <Matthew.Black@csulb.edu<mailto:Matthew.Black@csulb.edu><mailto:Matthew.Black@csulb.edu<mailto:Matthew.Black@csulb.edu>>> wrote: Google Safe Browsing and Firefox have marked our website as containing malware. They claim our home page returns no results, but redirects users to another compromised website couchtarts.com<http://couchtarts.com><http://couchtarts.com>.
We have thoroughly examined our root .htaccess and httpd.conf files and are not redirecting to the problem target site. No recent changes either.
We ran some NSLOOKUPs against various public DNS servers and intermittently get results that are NOT our servers.
We believe the DNS servers used by Google's crawler have been poisoned.
Can anyone shed some light on this?
matthew black information technology services california state university, long beach www.csulb.edu<http://www.csulb.edu><http://www.csulb.edu><http://www.csulb.edu>
-- Landon Stewart <LStewart@Superb.Net<mailto:LStewart@Superb.Net<mailto:LStewart@Superb.Net>>> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199<tel:888-354-6128%20x%204199> Web hosting and more "Ahead of the Rest": http://www.superbhosting.net<http://www.superbhosting.net/>
-- Landon Stewart <LStewart@Superb.Net<mailto:LStewart@Superb.Net>> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199 Web hosting and more "Ahead of the Rest": http://www.superbhosting.net<http://www.superbhosting.net/>
Matt, what happens you get on a subnet that can access the webservers directly and bypass the load balancer. Try curl then and see if its something w/ the webserver or load balancer. -Grant On Wed, Jun 27, 2012 at 12:40 AM, Matthew Black <Matthew.Black@csulb.edu>wrote:
Thanks again to everyone who helped. I didn't know what to enter with curl, because Outlook clobbered the line breaks in Jeremy's original message.
Also, curl failed on our primary webserver because of firewall and load balancer magic settings. The Telnet method worked better!
Our team is now scouring for that hidden redirect to couchtarts.
matthew black information technology services california state university, long beach
From: Landon Stewart [mailto:lstewart@superb.net] Sent: Tuesday, June 26, 2012 10:37 PM To: Matthew Black Cc: Jeremy Hanmer; nanog@nanog.org Subject: Re: DNS poisoning at Google?
There is definitely a 301 redirect.
$ curl -I --referer http://www.google.com/ http://www.csulb.edu/ HTTP/1.1 301 Moved Permanently Date: Wed, 27 Jun 2012 05:36:31 GMT Server: Apache/2.0.63 Location: http://www.couchtarts.com/media.php Connection: close Content-Type: text/html; charset=iso-8859-1
On 26 June 2012 22:05, Matthew Black <Matthew.Black@csulb.edu<mailto: Matthew.Black@csulb.edu>> wrote: Google Webtools reports a problem with our HOMEPAGE "/". That page is not redirecting anywhere. They also report problems with some 48 other primary sites, none of which redirect to the offending couchtarts.
matthew black information technology services california state university, long beach
-----Original Message----- From: Jeremy Hanmer [mailto:jeremy.hanmer@dreamhost.com<mailto: jeremy.hanmer@dreamhost.com>] Sent: Tuesday, June 26, 2012 9:58 PM To: Matthew Black Cc: nanog@nanog.org<mailto:nanog@nanog.org> Subject: Re: DNS poisoning at Google? It's not DNS. If you're sure there's no htaccess files in place, check your content (even that stored in a database) for anything that might be altering data based on referrer. This simple test shows what I mean:
Airy:~ user$ curl -e 'http://google.com' csulb.edu<http://csulb.edu> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php ">here</a>.</p> </body></html>
Running curl without the -e argument gives the proper site contents. On Jun 26, 2012, at 9:24 PM, Matthew Black <Matthew.Black@csulb.edu <mailto:Matthew.Black@csulb.edu>> wrote:
Running Apache on three Solaris webservers behind a load balancer. No MS Windows!
Not sure how malicious software could get between our load balancer and Unix servers. Thanks for the tip!
matthew black information technology services california state university, long beach
From: Landon Stewart [mailto:lstewart@superb.net<mailto: lstewart@superb.net>] Sent: Tuesday, June 26, 2012 9:07 PM To: Matthew Black Cc: nanog@nanog.org<mailto:nanog@nanog.org> Subject: Re: DNS poisoning at Google?
Is it possible that some malicious software is listening and injecting a redirect on the wire? We've seen this before with a Windows machine being infected. On 26 June 2012 20:53, Matthew Black <Matthew.Black@csulb.edu<mailto: Matthew.Black@csulb.edu><mailto:Matthew.Black@csulb.edu<mailto: Matthew.Black@csulb.edu>>> wrote: Google Safe Browsing and Firefox have marked our website as containing malware. They claim our home page returns no results, but redirects users to another compromised website couchtarts.com<http://couchtarts.com><; http://couchtarts.com>.
We have thoroughly examined our root .htaccess and httpd.conf files and are not redirecting to the problem target site. No recent changes either.
We ran some NSLOOKUPs against various public DNS servers and intermittently get results that are NOT our servers.
We believe the DNS servers used by Google's crawler have been poisoned.
Can anyone shed some light on this?
matthew black information technology services california state university, long beach www.csulb.edu<http://www.csulb.edu><http://www.csulb.edu><; http://www.csulb.edu>
-- Landon Stewart <LStewart@Superb.Net<mailto:LStewart@Superb.Net<mailto: LStewart@Superb.Net>>> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199<tel:888-354-6128%20x%204199> Web hosting and more "Ahead of the Rest": http://www.superbhosting.net<http://www.superbhosting.net/>
-- Landon Stewart <LStewart@Superb.Net<mailto:LStewart@Superb.Net>> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199 Web hosting and more "Ahead of the Rest": http://www.superbhosting.net<; http://www.superbhosting.net/>
On Tue, Jun 26, 2012 at 10:36:55PM -0700, Landon Stewart wrote:
There is definitely a 301 redirect.
$ curl -I --referer http://www.google.com/ http://www.csulb.edu/ HTTP/1.1 301 Moved Permanently Date: Wed, 27 Jun 2012 05:36:31 GMT Server: Apache/2.0.63 Location: http://www.couchtarts.com/media.php Connection: close Content-Type: text/html; charset=iso-8859-1
And if you visit http://www.couchtarts.com/media.php using the correct broser you end up back at http://google.com ...
On 26 June 2012 22:05, Matthew Black <Matthew.Black@csulb.edu> wrote:
Google Webtools reports a problem with our HOMEPAGE "/". That page is not redirecting anywhere. They also report problems with some 48 other primary sites, none of which redirect to the offending couchtarts.
matthew black information technology services california state university, long beach
-----Original Message----- From: Jeremy Hanmer [mailto:jeremy.hanmer@dreamhost.com] Sent: Tuesday, June 26, 2012 9:58 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
It's not DNS. If you're sure there's no htaccess files in place, check your content (even that stored in a database) for anything that might be altering data based on referrer. This simple test shows what I mean:
Airy:~ user$ curl -e 'http://google.com' csulb.edu <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://www.couchtarts.com/media.php ">here</a>.</p> </body></html>
Running curl without the -e argument gives the proper site contents.
On Jun 26, 2012, at 9:24 PM, Matthew Black <Matthew.Black@csulb.edu> wrote:
Running Apache on three Solaris webservers behind a load balancer. No MS Windows!
Not sure how malicious software could get between our load balancer and Unix servers. Thanks for the tip!
matthew black information technology services california state university, long beach
From: Landon Stewart [mailto:lstewart@superb.net] Sent: Tuesday, June 26, 2012 9:07 PM To: Matthew Black Cc: nanog@nanog.org Subject: Re: DNS poisoning at Google?
Is it possible that some malicious software is listening and injecting a redirect on the wire? We've seen this before with a Windows machine being infected. On 26 June 2012 20:53, Matthew Black <Matthew.Black@csulb.edu<mailto: Matthew.Black@csulb.edu>> wrote: Google Safe Browsing and Firefox have marked our website as containing malware. They claim our home page returns no results, but redirects users to another compromised website couchtarts.com<http://couchtarts.com>.
We have thoroughly examined our root .htaccess and httpd.conf files and are not redirecting to the problem target site. No recent changes either.
We ran some NSLOOKUPs against various public DNS servers and intermittently get results that are NOT our servers.
We believe the DNS servers used by Google's crawler have been poisoned.
Can anyone shed some light on this?
matthew black information technology services california state university, long beach www.csulb.edu<http://www.csulb.edu><http://www.csulb.edu>
-- Landon Stewart <LStewart@Superb.Net<mailto:LStewart@Superb.Net>> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199 Web hosting and more "Ahead of the Rest": http://www.superbhosting.net<http://www.superbhosting.net/>
-- Landon Stewart <LStewart@Superb.Net> Sr. Administrator Systems Engineering Superb Internet Corp - 888-354-6128 x 4199 Web hosting and more "Ahead of the Rest": http://www.superbhosting.net
-- - (2^(N-1))
participants (7)
-
Christopher Morrow -
Grant Ridder -
Ishmael Rufus -
Jason Hellenthal -
Jeff Fisher -
Landon Stewart -
Matthew Black