diff options
| author | Dave Lawrence <dkl@mozilla.com> | 2011-11-11 19:51:35 +0100 |
|---|---|---|
| committer | Dave Lawrence <dkl@mozilla.com> | 2011-11-11 19:51:35 +0100 |
| commit | 9cadb22315af5999004c901a8b499d95bed76743 (patch) | |
| tree | 85956ca37f52c581414cc030846f87c0cad5e16d | |
| parent | 7183c32daf75d650c5345e98dea466d64c7ac3f2 (diff) | |
| download | bugzilla-9cadb22315af5999004c901a8b499d95bed76743.tar.gz bugzilla-9cadb22315af5999004c901a8b499d95bed76743.tar.xz | |
Bug 700876 - robots.txt - discourage search engines from indexing duplicate information
r=glob
| -rw-r--r-- | robots.txt | 7 |
1 files changed, 4 insertions, 3 deletions
diff --git a/robots.txt b/robots.txt index 6759c00b1..129fe60a7 100644 --- a/robots.txt +++ b/robots.txt @@ -2,11 +2,12 @@ User-agent: Browsershots Disallow: User-agent: * -Disallow: /*.cgi -Disallow: /*show_bug.cgi*ctype=* -Allow: / +Disallow: / Allow: /*index.cgi Allow: /*page.cgi Allow: /*show_bug.cgi Allow: /*describecomponents.cgi +Disallow: /*show_bug.cgi*ctype=* +Disallow: /*show_bug.cgi*format=multiple* +Disallow: /*page.cgi*id=voting* Sitemap: http://bugzilla.mozilla.org/page.cgi?id=sitemap/sitemap.xml |