Baldwinit.com Website Design

Website Design & Management

  • Home
  • About
  • Archives
  • Tags

Subscribe via Email

Allow Search robots

Posted by admin
March 20, 2008

This does the opposite of above, allowing ONLY these web robots access. Other than rogue robots, configuring your robots.txt file correctly will keep most robots where you want them.

SetEnvIfNoCase User-Agent .*google.* search_robot
SetEnvIfNoCase User-Agent .*yahoo.* search_robot
SetEnvIfNoCase User-Agent .*bot.* search_robot
SetEnvIfNoCase User-Agent .*ask.* search_robot

Order Deny,Allow
Deny from All
Allow from env=search_robot
htaccess

If you enjoyed this post, please consider to leave a comment or subscribe to the feed and get future articles delivered to your feed reader.

Comments

No comments yet.

Leave a comment

(required)

(required)


Search

Archives

  • May 2008
  • April 2008
  • March 2008

Categories

  • Articles
  • Css
  • Free Templates
  • htaccess
  • Html Codes
  • Icons
  • Javascript Codes
  • Software
  • Wordpress Plugins
  • Wordpress Themes

  • Recent Posts

    • Facebook Wordpress Theme
    • Free Icons download
    • Web Safe Color Chart
    • How can you select a quality domain name?
    • On mouse over alert box

  • Blogroll

    • Cheap Holidays To Egypt
    • Icmeler Hotels
    • Online Music Reviews
    • Sandalye
    • Southern Arizona Women’s Computer Bank
    • Turkey Holidays

  • Archives

    • May 2008
    • April 2008
    • March 2008

  • Recent Posts

    • Facebook Wordpress Theme
    • Free Icons download
    • Web Safe Color Chart
    • How can you select a quality domain name?
    • On mouse over alert box

Powered by WP | WP Remix Copyright 2007. Baldwinit.com Website Design. All rights reserved

  • Home
  • About
  • Archives
  • Tags