diff options
| author | Stan Hu <stanhu@gmail.com> | 2015-10-08 10:37:23 -0700 |
|---|---|---|
| committer | Stan Hu <stanhu@gmail.com> | 2015-10-08 10:37:23 -0700 |
| commit | 9440589197238d82d1121f24af6dbf4d2d557389 (patch) | |
| tree | 180f7d71653118e174e0d0ae2501242b8041e228 | |
| parent | 1115caf8e3a6632f9135c4ee965f0adde6098a78 (diff) | |
| parent | abf18abb52dbdb409d481854bde24ed880c76f56 (diff) | |
| download | gitlab-ce-9440589197238d82d1121f24af6dbf4d2d557389.tar.gz | |
Merge pull request #9693 from bbodenmiller/patch-1
allow crawling of commit page but not patch/diffs
| -rw-r--r-- | public/robots.txt | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/public/robots.txt b/public/robots.txt index 528f421083e..4f616c7f4c1 100644 --- a/public/robots.txt +++ b/public/robots.txt @@ -38,7 +38,8 @@ Disallow: /*/*/edit Disallow: /*/*/raw Disallow: /*/*/blame Disallow: /*/*/commits/*/* -Disallow: /*/*/commit +Disallow: /*/*/commit/*.patch +Disallow: /*/*/commit/*.diff Disallow: /*/*/compare Disallow: /*/*/branches/new Disallow: /*/*/tags/new |