Google Community
Latest Forums Rules Resources
Custom Search

Go Back   Google Community > Webmaster Forums > Search Engine Optimization Forum

GoogleCommunity Sponsor
Cirtex Hosting
Use coupon "forum" for 50% Off!

Reply
 
LinkBack Thread Tools Display Modes
Old 03-23-2007, 02:32 AM   #1 (permalink)
Senior Googler
 
Hit man's Avatar
 
Join Date: Mar 2007
Location: Bangalore
Posts: 124
Thanks: 0
Thanked 0 Times in 0 Posts
Hit man is on a distinguished road
Send a message via MSN to Hit man Send a message via Yahoo to Hit man Send a message via Skype™ to Hit man
Red face How to avoid search engines crawling

Hi friends,

I have 2 sites , 1 is developing site and another is developed site. I need to transfer my information from developing site to developed site so within this period i don't like any search engines(google,yahoo.msn,aol........) crawl my developing site.

Kindly tell me the way to avoid or stop search engines crawling on my site

Answer would be appreciated greatly.......

Last edited by Hit man; 03-23-2007 at 02:34 AM. Reason: modification
Hit man is offline   Reply With Quote
 
Sponsored Links
Old 03-23-2007, 02:46 AM   #2 (permalink)
Elite Googler
 
alex-party's Avatar
 
Join Date: Oct 2006
Location: Ireland
Posts: 1,085
Thanks: 0
Thanked 2 Times in 2 Posts
alex-party is an unknown quantity at this point
Add this line to your page header:
<meta name="robots" content="noindex,nofollow">

It tells search robots NOT to index your page and NOT to follow links on your page.
You can also create robots.txt file with links that SHOULD NOT BE crawled and upload it to your web root directory.

Last edited by alex-party; 03-23-2007 at 03:10 AM.
alex-party is offline   Reply With Quote
Old 03-23-2007, 04:07 AM   #3 (permalink)
Googler
 
Jaguar15's Avatar
 
Join Date: Feb 2007
Posts: 80
Thanks: 0
Thanked 1 Time in 1 Post
Jaguar15 is on a distinguished road
  • If you don't want search engines to index any file of your Web site, use the following on your robots.txt:
    User-agent: *
    Disallow: /
Jaguar15 is offline   Reply With Quote
Old 04-04-2007, 10:33 PM   #4 (permalink)
Googler
 
Join Date: Mar 2007
Posts: 60
Thanks: 0
Thanked 0 Times in 0 Posts
Candy is on a distinguished road
To avoid undesirable search listings, webmasters can instruct spiders not to ... addressing technical issues that may keep search engines from crawling
Candy is offline   Reply With Quote
Old 04-09-2007, 11:36 AM   #5 (permalink)
Junior Googler
 
manager2006's Avatar
 
Join Date: Oct 2006
Posts: 34
Thanks: 0
Thanked 0 Times in 0 Posts
manager2006
Quote:
Originally Posted by Jaguar15 View Post
  • If you don't want search engines to index any file of your Web site, use the following on your robots.txt:
    User-agent: *
    Disallow: /
Yes , it is the better way to use robots.txt.
manager2006 is offline   Reply With Quote
Old 04-09-2007, 03:24 PM   #6 (permalink)
Googler
 
teeny's Avatar
 
Join Date: Jan 2006
Posts: 82
Thanks: 0
Thanked 0 Times in 0 Posts
teeny
Try this link for more info:

http://www.searchtools.com/robots/robots-txt.html

Also, you might not want to block all search engines and you can just block certain search engines.

Good luck

Teeny
teeny is offline   Reply With Quote
Old 04-10-2007, 04:34 AM   #7 (permalink)
Noogle
 
Join Date: Apr 2007
Posts: 11
Thanks: 0
Thanked 0 Times in 0 Posts
sharon is on a distinguished road
It is simple.
To prevent all robots from indexing a page on your site, you'd place the following meta tag into the <HEAD> section of your page:
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
sharon is offline   Reply With Quote
Old 04-10-2007, 11:47 AM   #8 (permalink)
tms
Senior Googler
 
Join Date: Mar 2007
Posts: 126
Thanks: 0
Thanked 1 Time in 1 Post
tms is on a distinguished road
robot.txt

sharon's way is the simpliest way to do it. I tried and worked for me
tms is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Is Search Engine Crawling Considered as hit ? mayavi Web Browsers and Statistics 2 10-07-2006 03:47 AM
How do I avoid snap back to search box? mevans42 All About Google 2 07-22-2006 01:58 PM
What do Search Engines look for? shahabr Search Engine Optimization Forum 1 04-03-2006 11:36 PM
how much triffic are you getting from search engines? gurufocus All About Google 0 06-10-2005 10:00 AM
Top 10 search engines for the U.K. geekerati General Discussion 7 03-18-2005 09:54 AM


All times are GMT -8. The time now is 10:48 AM.


Powered by vBulletin®
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
LinkBacks Enabled by vBSEO 3.1.0
© 2004–2007 Google Community