robots.txt

You are currently browsing articles tagged robots.txt.

If you read this…

April 7, 2008 in Journal | No comments

Then you have passed the robots.txt file :) I have disabled the site from all crawlers. Using robots.txt.