Search Engines

Above is a little picture I wanted to show you.

For those among you who are used to looking at website traffic graphics you see that something strange has happened here…

You see, the picture above is from one of my websites and shows clearly what the impact was of the following story.

I made a mistake, a simple one but with large consequences as you can see.
Because of this mistake traffic for this site almost went done to zero.
But if it had gone down to zero it would have been clear what happed right from the start, but it did not, I “just” lost traffic from the mayor three search engines.

What happened?

In simple terms: the site was hacked via  a remote inclusion attack from several forum like websites, one of them was successful.
There was just one file affected but one of the most important ones, the .htaccess file.
There were four lines of code injected at the bottom to the file:

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} (Googlebot|Slurp|msnbot)
RewriteRule ^ http://example.com/ [R=301,L]three - code used

The code in line three redirects those search engine robots too the site in line four (which I changed, the real site got enough traffic from me).

How was this possible? Simple, because I forgot to reset the security on .htaccess after doing some previous test.
The file was still set to 707 = also public write access, and yes its security is set back to the right access now.

How to find out your site is hacked

As I said before if the traffic had gone to zero you most probably would have checked your site and see a defacing of the site itself of some kind. The site would not work the way you would expect it to.

In this case the only problem I could see was that the traffic dropped hard in the statistics.
Upon looking further into the details I checked the ranking of previous Google search terms the site was ranking for.

I found that all the top 10 ranking pages ware gone! All my efforts to rank for those keywords were lost.

So I logged into Google webmaster to check if there was something horribly wrong with the site and the site had received some kind of penalty.

And yes there were errors with the URLs in the sitemap.xml, errors about unreachable URLs.
So I checked the files on the site that where modified just before the drop, and there was the .htaccess file with the above mentioned code.

Getting back into Google

After cleaning up the mess and setting the security right I realized the impact of these few lines.

All three search engine robots had gotten notice that the pages where permanently moved to another domain! which is not true, but they don't know that.

Checks on site: reference in Google, Yahoo and MSN showed that the domain was still in Yahoo and MSN Indexes, but had no entry in the Google Index anymore.

So it was time to send a reconsideration request to Google telling them what happened, what had been done to resolve the problem and what steps where taken too prevent it from happening again.

After that, you can only wait and wait…, don't expect an email form Google, just an automatically generated message in your Webmaster control center that they will look into the matter.

After a few days traffic started to build again from Google and Yahoo as well as MSN (Live.com) so you can conclude that Google really looks and acts upon your requests.

Hard Lessons learned

Now for the fun part of this episode, the lessons learned
First lesson: make sure to secure your cms installation and critical files.

Some tips:
– change passwords on a regular basis (use keepass if you have a lot off passwords to remember)
– secure your admin directory or files with .htaccess passwords, most of the hosting companies have this option in their cpanel
– update your system as soon as a new version comes out.

Check your stats once a day, or a least every few days for strange things happening, since this could have taken me a long time to get back if it wasn't for those early warning signs.

As a personal note: I need to get more incoming links from non-search engines to diversify my traffic sources :-)

What I realized after my traffic was back to its old level was that Google is really fast in acting on 301 redirects and changes in their index.
They update their index with a higher frequency than Yahoo and Live, and they really act fast on those re-inclusion request as well.

A hack like this before your start your SEO work for organic ranking would also mean that all your efforts will have no effect… so if a cusomers is really having trouble to get his or.her site into the search engines, check the .htaccess file!

If you have your own horror stories like this one, please share them in the comments and let us learn from those stories as well.