revi-archive/phorge-phorge
1
0
Fork 0
mirror of https://we.phorge.it/source/phorge.git synced 2024-11-26 08:42:41 +01:00
Code Issues Releases Wiki Activity

Disallow /source/ in robots.txt

Browse source 
Summary: Ref T4245. We disallow `/diffusion/` in robots.txt already because indexers tend to get lost blaming every line of every file throughout history, but didn't update the list for the `/source/` alias. Update it.

Test Plan: Visited `/robots.txt` locally.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T4245

Differential Revision: https://secure.phabricator.com/D17637
This commit is contained in:
epriestley 2017-04-06 15:51:17 -07:00
parent 3d816e94df
commit 9856802ba2
1 changed files with 1 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
src/applications/system/controller/PhabricatorRobotsController.php
View file

@ -20,6 +20,7 @@ final class PhabricatorRobotsController extends PhabricatorController {
$out[] = 'User-Agent: *'; $out[] = 'User-Agent: *';
$out[] = 'Disallow: /diffusion/'; $out[] = 'Disallow: /diffusion/';
$out[] = 'Disallow: /source/';
// Add a small crawl delay (number of seconds between requests) for spiders // Add a small crawl delay (number of seconds between requests) for spiders
// which respect it. The intent here is to prevent spiders from affecting // which respect it. The intent here is to prevent spiders from affecting