summaryrefslogtreecommitdiff
path: root/public/robots.txt
Commit message (Collapse)AuthorAgeFilesLines
* update robots.txt disallowMatt Harrison2016-09-231-1/+1
| | | | | Allows projects in groups starting with "s" while still disallowing the snippets short urls.
* Disallow search engines from indexing uploads from a GitLab project.Connor Shea2016-05-161-0/+1
| | | | This can sometimes include sensitive information from private projects and confidential issues. It shouldn't be indexed. Resolves #15551.
* allow crawling of commit page but not patch/diffsBen Bodenmiller2015-10-041-1/+2
| | | Commit page has valuable information that search engines should be allowed to crawl however the .patch and .diff pages have no new information that is not on commit page
* disallow irrelevant pages by default in robotsBen Bodenmiller2015-08-171-1/+62
| | | | | | | Update default robots.txt rules to disallow irrelevant pages that search engines should not care about. This will still allow important pages like the files, commit details, merge requests, issues, comments, etc. to be crawled.
* init commitgitlabhq2011-10-091-0/+5
View Lorry Controller Status