Community Discussion

How would you prevent this?

December 1, 2011
angie828 Author Level X3

Write the reason you're deleting this FAQ

Is there a way to prevent a spider from attaching itself to a URL that you do not want on the search engines?
4

Comments 4

December 1, 2011
robertman11
Use your robots.txt file or use .htaccess to actually blog the spider (you can block by IP or by user-client).

Enter a reason for deleting this comment

December 2, 2011
Light
Thanks for the information guys. How would you prevent this?

Enter a reason for deleting this comment

Please login or sign up to leave a commentJoin